Re: [PATCH] glibc: Remove CPU set size checking from affinity functions [BZ #19143]

[Michael Kerrisk <mtk dot manpages at gmail dot com>]

Hi Florian,

On Tue, Nov 24, 2015 at 5:39 PM, Florian Weimer <fweimer@redhat.com> wrote:
> On 11/04/2015 09:17 PM, Florian Weimer wrote:
>> On 10/23/2015 04:00 PM, Florian Weimer wrote:
>>> 2015-10-23  Florian Weimer  <fweimer@redhat.com>
>>>
>>>      [BZ #19143]
>>>      [BZ #19164]
>>>      * nptl/check-cpuset.h: Remove.
>>>      * nptl/pthread_attr_setaffinity.c (__pthread_attr_setaffinity_new):
>>>      Remove CPU set size check.
>>>      * nptl/pthread_setattr_default_np.c (pthread_setattr_default_np):
>>>      Likewise.
>>>      * sysdeps/unix/sysv/linux/check-cpuset.h: Remove.
>>>      * sysdeps/unix/sysv/linux/pthread_setaffinity.c
>>>      (__kernel_cpumask_size, __determine_cpumask_size): Remove.
>>>      (__pthread_setaffinity_new): Remove CPU set size check.
>>>      * sysdeps/unix/sysv/linux/sched_setaffinity.c
>>>      (__kernel_cpumask_size): Remove.
>>>      (__sched_setaffinity_new): Remove CPU set size check.
>>>      * manual/threads.texi (Default Thread Attributes): Remove stale
>>>      reference to check_cpuset_attr, determine_cpumask_size in comment.
>>>      * sysdeps/unix/sysv/linux/Makefile [$(subdir) == posix] (tests):
>>>      Remove tst-getcpu.  Add tst-affinity, tst-affinity-pid.
>>>      [$(subdir) == nptl] (tests): Add tst-thread-affinity-pthread,
>>>      tst-thread-affinity-pthread2, tst-thread-affinity-sched.
>>>      * sysdeps/unix/sysv/linux/tst-affinity.c: New file.
>>>      * sysdeps/unix/sysv/linux/tst-affinity-pid.c: New file.
>>>      * sysdeps/unix/sysv/linux/tst-skeleton-affinity.c: New skeleton test file.
>>>      * sysdeps/unix/sysv/linux/tst-thread-affinity-sched.c: New file.
>>>      * sysdeps/unix/sysv/linux/tst-thread-affinity-pthread.c: New file.
>>>      * sysdeps/unix/sysv/linux/tst-thread-affinity-pthread2.c: New file.
>>>      * sysdeps/unix/sysv/linux/tst-thread-skeleton-affinity.c: New
>>>      skeleton test file.
>>>      * sysdeps/unix/sysv/linux/tst-getcpu.c: Remove.  Superseded by
>>>      tst-affinity-pid.
>>
>> Ping?
>>
>> Unfortunately, I didn't get *any* feedback from the kernel people I
>> Cc:ed.  There is very little traffic on the cgroups is list in general,
>> though.
>
> I have committed this.  I believe all glibc-specific issues have been
> addressed, and the technical contents was no longer in dispute.

With this change, I wonder if some pieces in the sched_setaffinity(2)
man page (http://man7.org/linux/man-pages/man2/sched_setaffinity.2.html)
may be in order. Below, I've quoted some relevant pieces from the man
page. Anything there that needs tweaking?

   ERRORS
       EINVAL The affinity bit mask mask contains no processors that  are
              currently  physically  on  the  system and permitted to the
              thread according to any restrictions that may be imposed by
              the "cpuset" mechanism described in cpuset(7).

       EINVAL (sched_getaffinity()   and,   in   kernels   before  2.6.9,
              sched_setaffinity()) cpusetsize is smaller than the size of
              the affinity mask used by the kernel.
   [...]
   NOTES
   [...]
     Handling systems with large CPU affinity masks
       The  underlying  system  calls  (which  represent CPU masks as bit
       masks of type unsigned long *) impose no restriction on  the  size
       of  the  CPU mask.  However, the cpu_set_t data type used by glibc
       has a fixed size of 128 bytes, meaning that the maximum CPU number
       that  can be represented is 1023.  If the kernel CPU affinity mask
       is larger than 1024, then calls of the form:

           sched_getaffinity(pid, sizeof(cpu_set_t), &mask);

       will fail with the error EINVAL, the error produced by the  underâ
       lying  system  call  for the case where the mask size specified in
       cpusetsize is smaller than the size of the affinity mask  used  by
       the  kernel.   (Depending  on  the system CPU topology, the kernel
       affinity mask can be  substantially  larger  than  the  number  of
       active CPUs in the system.)

       When  working on systems with large kernel CPU affinity masks, one
       must dynamically allocate the mask argument.  Currently, the  only
       way  to  do  this  is by probing for the size of the required mask
       using sched_getaffinity() calls with increasing mask sizes  (until
       the call does not fail with the error EINVAL).

Cheers,

Michael