[PATCH] linux: fix accuracy of get_nprocs and get_nprocs_conf [BZ #28865]

[Adhemerval Zanella]

On 07/02/2022 08:51, Dmitry V. Levin wrote:
> Hi,
> 
> On Mon, Feb 07, 2022 at 08:25:11AM -0300, Adhemerval Zanella via Libc-alpha wrote:
>> On 05/02/2022 18:24, Dmitry V. Levin wrote:
>>> get_nprocs() and get_nprocs_conf() use various methods to obtain an
>>> accurate number of processors.  Re-introduce __get_nprocs_sched() as
>>> a source of information, and fix the order in which these methods are
>>> used to return the most accurate information.  The primary source of
>>> information used in both functions remains unchanged.
>>>
>>> This also changes __get_nprocs_sched() error return value from 2 to 0,
>>> but all its users are already prepared to handle that.
>>>
>>> Old behavior:
>>>   get_nprocs:
>>>     /sys/devices/system/cpu/online -> /proc/stat -> 2
>>>   get_nprocs_conf:
>>>     /sys/devices/system/cpu/ -> /proc/stat -> 2
>>>
>>> New behavior:
>>>   get_nprocs:
>>>     /sys/devices/system/cpu/online -> sched_getaffinity -> /proc/stat -> 2
>>>   get_nprocs_conf:
>>>     /sys/devices/system/cpu/ -> /proc/stat -> sched_getaffinity -> 2
>>>
>>> Fixes: 342298278e ("linux: Revert the use of sched_getaffinity on get_nproc")
>>> Closes: BZ #28865
>>
>> I think we are circling back on this, on BZ#27645 [1] we changed get_nprocs
>> to use sched_getaffinity and then we have to revert it with BZ#28310 [2] because
>> it introduced regression on some monitoring tools [3].
>>
>> In fact from BZ#27645 and BZ#28624 [4] discussion I think we can't reliable use 
>> sched_getaffinity because since some container environment returns a synthetic
>> mask that might break some programs.  Also, sched_getaffinity returns a 
>> 'per-process' mask instead of system-wide as we discussed in previous threads.
>> It should be ok to get adjusting internal tuning (as for malloc).
>>
>> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=27645
>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=28310
>> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=27645#c5
>> [4] https://sourceware.org/bugzilla/show_bug.cgi?id=28624
> 
> Is there any realistic case when 2 is a more accurate estimation for the
> number of processors than sched_getaffinity?  I suppose there are no such
> cases.  Also, /sys is consulted first anyway.

I am not sure, but my impression is on some environments sched_getaffinity
returns a synthetic value that might not represent the correct system
supported CPUs.  At least, it was my impression in the bug reports, where
it does break some programs.

> 
> I wish I saw commit 342298278e earlier to raise objections before it was
> committed.
> 
> Please note that BZ #28865 is a real regression we had to patch, this
> means glibc must behave properly in that environment without any
> additional tuning.
> 
> I suggest to install this fix and see what could be done later
> in an unlikely case anything else breaks.

I think in this case we should use sched_affinity as the last fallback
on get_nprocs as well we, so we first use either sysfs or procfs and
only then fallback to sched_getaffinity.