SCHED_FIFO and pthread_cond_broadcast

[Adhemerval Zanella]

On 16/01/2020 19:59, Tadeus Prastowo wrote:
> Hello,
> 
> AFAIK, POSIX specifies that pthread_cond_broadcast should wake up
> waiting threads in their scheduling orders.  So, in the attached
> program, main is expected to always return 1 because:
> 
> 1. When main performs pthread_cond_broadcast(&b) in line 52, the
> condition variable b already has threads task_1 and task_2 waiting.
> 
> 2. Both task_1 and task_2 are scheduled using SCHED_FIFO such that
> task_2 has a priority higher than task_1.
> 
> 3. Since pthread_cond_broadcast wakes up waiting threads in their
> scheduling orders, task_2 runs first and sets the value to be returned
> to 2 and then task_1 runs and sets the value to be returned to 1.
> 
> 4. The main function returns the last value written, which is 1.
> 
> However, I observe that the main function can return the value 2,
> indicating that task_1 can run before task_2 despite POSIX
> specification by the following steps:
> $ gcc -O2 -o z z.c -pthread
> $ sudo chown root z
> $ sudo chmod u+s z
> $ for ((i = 0; i < 1000000; ++i)); do ./z; if [ $? -eq 2 ]; then echo
> It is two at $i; break; fi; done
> 
> When I follow the steps in my Ubuntu 16.04.6 (the output of uname -a
> is: Linux Eus 4.15.0-75-generic #85~16.04.1-Ubuntu SMP Wed Jan 15
> 12:30:12 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux), which uses Glibc
> 2.23 (obtained by executing /lib/x86_64-linux-gnu/libc-2.23.so -v) and
> GCC 9.2.1, I get an output like the following, indicating that task_1
> can run before task_2:
> 
> It is two at 22718
> 
> Could someone help me figure out why pthread_cond_broadcast does not
> respect the POSIX specification, please?
> 
> Thank you.
> 

It is a known issues and it is being tracked at BZ#11588 [1]. Currently
condition variable does not have PI awareness (such as userland wait
queues) and issues default futex wait/wake calls that do not take in 
consideration the waiters priority.

There were an interesting discussion about how to make condition variables
PI enabled [2], and currently there is no easy way to fulfil both POSIX
requirements and PI expectation with current kernel futex primitives.

Last Linux Plumbers Darren Hart presented an alternative conditional
variable implementation [3] that enables PI awareness, but it has its
own drawbacks (it has different semantic than POSIX conditional variable
at API level and afaik it is subject the very issue the new glibc
conditional variable implementation solved, discussed at [4]).

If your usage does not require all the POSIX specification about partial
ordering regarding signaling, you might consider use librtpi (your example
seems to show the expected output adapating to use it).

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=11588
[2] https://www.youtube.com/watch?v=D1hGc8qJCcQ&list=UL-cj2JDu_Ac4&index=1208
[3] https://github.com/dvhart/librtpi
[4] https://www.austingroupbugs.net/view.php?id=609