pthread_mutex_unlock takes 15x longer than pthread_mutex_lock

[Akash Baid]

Hi,

I have a packet processing application which takes and releases a
pthread_mutex_lock for every packet (around 200k times a second). When
profiling the performance through Intel VTune, I noticed that the unlock
operation takes about 370ns compared to only 25ns for the lock operation.
The numbers are averaged over millions of operations and are independent of
the size of the critical section.

I suspected lock contention to be playing a role here and thus have ensured
that no other thread accesses this lock but the numbers still hover around
the same mark. Any ideas as to whether this is expected behavior and if not
what might be going on? My understanding was that the unlock operation
under no contention should be free of system calls and extremely fast. In
fact single stepping through the instructions suggest that there aren't any
system calls but maybe the LOCK assembly instruction inside the low-level
lock lll_unlock routine is taking long in the unlock case for some reason?

The application is using libpthread dynamic shared library, glibc
version 2.13 on a x86_64 GNU/Linux VM on an Intel Xeon E5 2.5Ghz processor
machine.