Re: [PATCH] Count number of logical processors sharing L2 cache

["H.J. Lu" <hjl dot tools at gmail dot com>]

On Tue, May 24, 2016 at 10:49 AM, Carlos O'Donell <carlos@redhat.com> wrote:
> On 05/24/2016 11:02 AM, H.J. Lu wrote:
>> CAT applies to a specific thread/process. Cache sizes in glibc are applied
>> to string/memory functions for all threads/processes.  They both try to avoid
>> over-using shared cache by a single thread/process.  But they work at
>> different levels and have different behaviors.  Glibc also uses the cache size
>> to decide when to use non-temporal store to avoid cache pollution and speed
>> up writing a large amount of data..
>
> Don't you mean that CAT applies to a core (and all of its logical cores)?
>
> Might it be the case that a thread or process could be migrated by the linux kernel
> between various cores configured with different CAT values and the glibc heuristics
> could be poorly tuned for some of those cores?
>
> As I see it the values computed by init_cacheinfo() are only average heuristics for
> the core.
>
> I agree that Florian has a point, that these values may become less useful in the
> presence of the dynamically changing L3<->core partitioning enabled by CAT.
>
> It is silly though to think that you would allow a thread or process to migrate
> away from the CAT-tuned core. The design of CAT is such that you want to isolate
> the tuned application to one ore more cores and use CAT to control the L3 allocation
> for those cores.

I checked with our kernel CAT implementer.  CAT supports both
processor and process.

> In the case where you have a process pinned to a core, and CAT is used to limit the
> L3 of that core, do the glibc heuristics computed in init_cacheinfo() match the
> reality of the L3<->core allocation? Or would a lower L3 CAT-tuned value mean that
> glibc would be mis-tuned for that core?

CAT dedicates part of L3 cache to certain processor or process so
that L3 cache is always available to them.  Glibc tries not to take all
L3 cache in memcpy/memset so thar L3 cache is available for other
operations within the same process as well as to other processor/process.
CAT and glibc work at different angels.  There is no direct conflict between
CAT and glibc.  At the moment,  I am not sure if CAT-aware glibc will
improve performance.


-- 
H.J.