[PATCH] Count number of logical processors sharing L2 cache

[Carlos O'Donell]

On 05/31/2016 10:57 AM, H.J. Lu wrote:
> On Tue, May 31, 2016 at 7:01 AM, Carlos O'Donell <carlos@redhat.com> wrote:
>> On 05/24/2016 05:35 PM, H.J. Lu wrote:
>>> CAT dedicates part of L3 cache to certain processor or process so
>>> that L3 cache is always available to them.  Glibc tries not to take all
>>> L3 cache in memcpy/memset so thar L3 cache is available for other
>>> operations within the same process as well as to other processor/process.
>>> CAT and glibc work at different angels.  There is no direct conflict between
>>> CAT and glibc.  At the moment,  I am not sure if CAT-aware glibc will
>>> improve performance.
>>
>> What do you mean by "no direct conflict?"
>>
>> If glibc tuns its own algorithms to use 1/4 of L3, but CAT has only
>> allocated 1/5 of L3 to that process, then glibc's algoirthms, whose
>> intent was to use a small amount of L3 are now using *more* L3 than
>> the entire process has and that could impact performance?
> 
> Cache sizes are only used for instructions selections in
> string/memory functions, which don't use cache directly.
> Glibc has NO control whatsoever on how much cache
> string/memory functions use.

Even if glibc doesn't use cache directly, it's making algorithm choices
based on those sizes and tunings.

For example:

sysdeps/x86/cacheinfo.c

765   /* The large memcpy micro benchmark in glibc shows that 6 times of
766      shared cache size is the approximate value above which non-temporal
767      store becomes faster.  */
768   __x86_shared_non_temporal_threshold = __x86_shared_cache_size * 6;

If, for example, CAT changed the shared L3 cache size to be smaller, would
it invalidate the above static tuning?

If it does, then we should change the comments in several places to mention
CAT.

>> Did I miss something?
>>
>> There could be opportunities for incorrect tunings if glibc is not
>> CAT-aware?
>>
> 
> Since CAT can change at any time during the life of a process,
> checking CAT inside glibc isn't very practical.
 
Most reasonable administrators would use CAT to limit one particular
workload and leave the CAT setting until the workload changed.
Therefore it is entirely reasonable IMO to check CAT settings at
process startup and adjust accordingly e.g. CAT-aware glibc.
Checking at every use is too expensive and does not mirror what
an administrator would be doing with CAT settings?

-- 
Cheers,
Carlos.