Re: 转发: malloc: Optimize the number of arenas for better application performance

[Siddhesh Poyarekar]

On 29/06/2022 08:07, Qingqing Li via Libc-alpha wrote:
>>> On 28 Jun 2022, at 15:56, DJ Delorie <dj@redhat.com> wrote:
>>>
>>> Yang Yanchao <yangyanchao6@huawei.com> writes:
>>>> However, my machine is 96 cores and I have 91 cores bound.
>>>
>>> One benchmark on one uncommon configuration is not sufficient reason to
>>> change a core tunable.  What about other platforms?  Other benchmarks?
>>> Other percentages of cores scheduled?
>>>
>>> I would reject this patch based solely on the lack of data backing up
>>> your claims.
>>>
>>>> -              int n = __get_nprocs_sched ();
>>>> +              int n = __get_nprocs ();
>>>
>>> I've heard complaints about how our code leads to hundreds of arenas on
>>> processes scheduled on only two CPUs.  I think using the number of
>>> *schedulable* cores makes more sense than using the number of *unusable*
>>> cores.
>>>
>>> I think this change warrants more research.

Agreed, but the Huawei system use case appears to suggest that at least 
that system prefers the superset, which is a data point against this 
change.  The original change[1] doesn't appear to provide any extensive 
research (as Adhemerval admitted below too), so that seems sufficient 
grounds to revert a change that has regressed performance.

Of course, if you or anyone else has data points that strongly suggest 
*better* overall performance with __get_nprocs_sched (I am aware of 
complaints about hundreds of arenas and address space usage and on many 
occasions in the past I've been able to successfully resolve them by 
showing that their code is nearly 20-30% faster because of that) then 
maybe there's scope to explore the possibility of having 
architecture-specific defaults.

>> I think this patch make sense mainly because we changed to use the
>> schedulable cores without much though either.  Maybe we can revert
>> to previous semantic and investigate that using the schedulable
>> number makes more sense.
>>
> Agreed, the variable narenas_limit just Initialize once, if there has a scenario that we dynamic adjust the cpu affnity,
> __get_nprocs_sched is not a good choice. my opinion first use  __get_nprocs as a static valule(the old default behavior),
> and user use glibc.TUNABLE to adjust arana number.
> Also, we can do more research and test to optimize the default arena number.

I'm not strongly opposed to reverting (see above for more nuance) but 
would like us to use this opportunity to at least track a project to 
improve the arenas heuristic.  Could you or Adhemerval please add a bug 
report on sourceware to study the performance impact of arena counts and 
their relationship to core counts on different architectures/systems?

Thanks,
Siddhesh

[1] 
https://patchwork.sourceware.org/project/glibc/patch/20210907122259.79800-2-adhemerval.zanella@linaro.org/