This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Implement sched_[gs]etaffinity()


On Fri, 12 Apr 2019, Corinna Vinschen wrote:
On Apr 11 13:52, Mark Geisert wrote:
On Thu, 11 Apr 2019, Corinna Vinschen wrote:
On Apr 11 10:25, Corinna Vinschen wrote:
[...]
Your patch is nicely done, but what about machines with more than 64
CPUs?  Your patch only uses the standard API for up to 64 CPUs, so a
process can never use more than 64 CPUs or use CPUs from different CPU
groups.  There was also the case of this weird machine Achim Gratz once
worked on, which had less than 64 CPUs but *still* used multiple CPU
groups under Windows, for some reason.

Any chance you could update your patch to support this functionality?
For some info, see MSDN:

https://docs.microsoft.com/en-us/windows/desktop/ProcThread/processor-groups

Also, there's already some code in fhandler_proc.cc, function
format_proc_cpuinfo to handle CPU groups.  You can use the
wincap.has_processor_groups() method to check if the system
supports CPU groups.

Btw., Glibc's cpu_set_t supports up to 1024 CPUs.  See
https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/bits/cpu-set.h
This may be ok for the foreseable future, I guess.

Hi Corinna,
I will look into CPU group support; thanks for the pointers.  I also need to
fix the assumption I made about which flavor of pid would be handed to the
functions.. they will be Cygwin pids but need conversion to Windows pids
internally.

Yeah, right, I missed to notice that.  I'll add a few notes inline
over @ cygwin-patches.

I've updated my code locally to account for your notes on cygwin-patches; thanks! I've also spent some time researching Windows affinities vs Linux affinities and have come to some conclusions. I'm airing these for review before I start coding in earnest. I appreciate all comments from anybody interested.

(1) On Linux, one deals with processor affinities using a huge mask that allows to choose from all processors on the system. On Windows, one deals with processor affinities for only the current processor group, max 64 processors in a group. This implies conversion between the two "views" when getting or setting processor affinities on Cygwin.

(2) On Linux, sched_get/setaffinity() take a pid_t argument, but it can be either a process id or a thread id. If one selects a process id, the action affects just the main thread of that process. On Windows, selecting the process id affects all threads of that process.

(3) For completeness, Linux's pthread_get/setaffinity_np() should probably be supplied by the proposed code too.

(4) I was looking at Cygwin's fhandler_proc.cc, function format_proc_cpuinfo(). There's a call to __get_cpus_per_group() which is implemented in miscfuncs.cc. I haven't seen in the MSDN docs whether each processor group is guaranteed to have the same number of processors. I might even expect variations on a NUMA system. Anybody know if one can depend on the group membership of the first processor group to apply to all groups?

(5) On Windows, a process starts out in a particular processor group. One can then change thread affinities in such a way that some threads run in a different processor group than other threads of the same process. The process becomes a "multi-group" process. This has implications for the conversions discussed in (1).

(6) On Linux, processor affinity is inherited across fork() and execve(). I'll need to ensure Cygwin's implementation of those calls handle affinity the same way.

So this is looking like a more substantial project :-).
Thanks for reading,

..mark


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]