Fw: [RFC] PI aware condvars

[Darren Hart]

Darren Hart wrote:
> Ulrich Drepper wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 02/22/2010 10:25 AM, Dinakar Guniguntala wrote:
>>  Did you get a chance to take a look at the PI aware condvar patch.
>>> More specifically, I would like your opinion on points 1 and 2
>>> listed below.
>>
>> I don't have time now.  The first look I took already put me off.  It
>> adds lots of cost to the important code paths.And this for something
>> which is hardly ever used.  I'll have more to say when I had a closer 
>> look.
> 


Just checking in. Ulrich, do the results below ease your concerns about 
negative impact to the important code paths? If not, are there other 
tests you would like me to run and report the results of?

Thanks,

Darren Hart

> Hi Ulrich,
> 
> First off, I wanted to share the testcase we use to demonstrate the hang 
> possible without PI aware Condvars as well as test the patch itself. It 
> uses dlsym to determine if the new np API exists and uses it if it can:
> 
> http://dvhart.com/darren/linux/pthread_cond_hang.c
> 
> Now, regarding the added costs. I felt the changes to the common case 
> (when PI is not involved) were pretty minimal. Certainly any significant 
> regression there would be unacceptable. To try and measure the impact to 
> the common case by this patch, I prepared the testcase here:
> 
> http://dvhart.com/darren/linux/condvar_perf.c
> 
> It performs N iterations of cond_wait/cond_signal and reports how long 
> it took and how many cycles/second it achieved. I found I could get the 
> highest cycles/sec by running as SCHED_FIFO and by not bothering with 
> the mutex lock in the signaling thread, which I understand isn't good 
> practice, but is still compliant and seemed appropriate in this case.
> 
> I then built three versions of glibc 2.11.1 from git:
> 
> 1)        git: unmodified git sources
> 2)     c_only: pthread_cond*.S files deleted
> 3) pi_condvar: same as c_only with the pi_condvar patches applied
> 
> Comparing #3 against #2 allows us to eliminate any gains #1 would have 
> solely from the hand written asm. 3 will eventually contain hand written 
> asm, but until the non-posix API is agreed upon, it doesn't make sense 
> to expend the effort of writing the asm code in my opinion.
> 
> I then ran 10 runs of 10M iterations each at SCHED_FIFO 1 priority on 
> each of the three glibcs, the results (following) suggest no significant 
> change in the non PI condvar performance, sitting right at ~270k (avg) 
> cycles/sec for each glibc.
> 
> build-x86_64-2.11.1-git
> Cycles/Second: 279831.187500
> Cycles/Second: 261911.421875
> Cycles/Second: 277664.125000
> Cycles/Second: 284847.718750
> Cycles/Second: 285067.281250
> Cycles/Second: 267918.718750
> Cycles/Second: 284785.656250
> Cycles/Second: 277402.843750
> Cycles/Second: 202379.703125
> Cycles/Second: 266421.718750
> 
> Min:  202379.703125  us
> Max:  285067.28125  us
> Avg:  268823.0375  us
> 
> 
> build-x86_64-2.11.1-c_only
> Cycles/Second: 277931.781250
> Cycles/Second: 275614.093750
> Cycles/Second: 271194.125000
> Cycles/Second: 280155.093750
> Cycles/Second: 284708.156250
> Cycles/Second: 190936.031250
> Cycles/Second: 264253.468750
> Cycles/Second: 281354.281250
> Cycles/Second: 290366.218750
> Cycles/Second: 279962.000000
> 
> Min:  190936.03125  us
> Max:  290366.21875  us
> Avg:  269647.525  us
> 
> 
> build-x86_64-2.11.1-pi_condvar
> Cycles/Second: 263975.937500
> Cycles/Second: 279577.281250
> Cycles/Second: 276504.531250
> Cycles/Second: 266163.562500
> Cycles/Second: 262115.796875
> Cycles/Second: 279219.406250
> Cycles/Second: 265263.812500
> Cycles/Second: 262226.468750
> Cycles/Second: 284592.687500
> Cycles/Second: 278975.875000
> 
> Min:  262115.796875  us
> Max:  284592.6875  us
> Avg:  271861.535938  us
> 
> 
> This is only the cond_signal case, and doesn't account for 
> cond_timedwait or cond_broadcast, but I wouldn't expect those to 
> experience any additional impact from this patch. Are there scenarios 
> you can think of that are likely to suffer greater impact that are not 
> covered by this rather simple test case?
> 
> 
> Thanks,
> 


-- 
Darren Hart
IBM Linux Technology Center
Real-Time Linux Team