[3.5.3] Performance issue with System V semaphore ( semop() )

[Corinna Vinschen]

Hi Abinash,

On Sep  4 09:18, Abinash Mohanty (amohanty) via Cygwin-developers wrote:
> Hi,
> 
> Problem Statement
> Our application uses System V semaphore extensively. We are facing performance issue with it.
> 
> Problem Description
> We have created a sample code (attached with name shm_sem.cpp) which uses System V shared memory and semaphore. This code takes semaphore lock, updates the shared memory and unlock semaphore. It is done in a loop for 50k times. It takes around 21 seconds to complete. We have compiled and executed the same code on Linux which completes instantly. Following is the outcome of run from Cygwin and REHL 7.9 Linux VM:
> 
> Cygwin:
> uname -a
> CYGWIN_NT-10.0-17763 Win19Build01 3.5.3-1.x86_64 2024-04-03 17:25 UTC x86_64 Cygwin
> 
> Compiled with : g++ -o shm_sem shm_sem.cpp
> 
> shm_sem.exe
> 
> 2024-09-04 01:23:44.505687 => Starting...
> 2024-09-04 01:24:05.500330 => Finished...
> 2024-09-04 01:24:05.500989 => Shared memory and semaphore cleaned up.
> 
> RHEL Linux:
> uname -a
> Linux Linux6BuildDev03 3.10.0-1160.59.1.el7.x86_64 #1 SMP Wed Feb 16 12:17:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
> 
> Compiled with : g++ -o shm_sem shm_sem.cpp
> 
> ./shm_sem
> 
> 2024-09-04 08:22:16.302955 => Starting...
> 2024-09-04 08:22:16.385529 => Finished...
> 2024-09-04 08:22:16.385631 => Shared memory and semaphore cleaned up.

Yeah, this is way faster than what we can do.  Please note that the
code handling SysV IPC is basically FreeBSD kernel code with a couple
of changes, plus a named pipe communication strategy between a Cygwin
process and cygserver.

I patched up cygserver to add timestamps to its debug output and
removed unnecessary logging of mutex usage in the BSD code.

I found that a single semop operation plus logging takes about 350 - 450
usecs.  Including the named pipe communication overhead, semop takes
2000 - 4000 usecs.

The problem is that, looking into the code, I don't see why the
communication is so slow.  Basically it consists of Windows pipe calls
and a queue giving tasks to worker threads which, most of the time,
already have been started at cygserver start.

> Is this expected behavior or there are any configuration or any
> changes you can suggest which might improve its performance will be
> very helpful.

So I can't help you out of the box, unfortunately.  Maybe we can shave a
few usecs off the code in some places, like using srwlocks instead of
critical sections, but mostly these are a problem with multiple clients,
and your example is strictly serial.

Given this is the cygwin-developers mailing list, maybe some otherr
developer has a good idea and the time to hack on the cygserver code?


Corinna