[PATCH] rtld: cache cpuid results on the stack for intel

[Florian Weimer]

* Florian Weimer:

> * Fabian Rast:
>
>> On Mon Feb 2, 2026 at 10:26 AM CET, Florian Weimer wrote:
>>> * Fabian Rast:
>>>
>>>> Does anyone have experience with slow cpuid that could explain whats going =
>>>> on?
>>>
>>> In the slow case, are you running on bare metal, or is it virtualized?
>>
>> This is on bare metal. An Intel(R) Xeon(R) Gold 6430.
>>
>> My supervisor has access to more hardware - here are the results for
>> different processors:
>>
>> model name      : Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
>> ./selfexec_patch 1000 ran 1.03 ± 0.01 times faster than ./selfexec_master 1000
>>
>> model name      : Intel(R) Core(TM) i9-7900X CPU @ 3.30GHz
>> ./selfexec_patch 1000 ran 1.03 ± 0.02 times faster than ./selfexec_master 1000
>>
>> model name      : Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
>> ./selfexec_patch 1000 ran 1.02 ± 0.01 times faster than ./selfexec_master 1000
>>
>> Note that these binaries are statically linked, which amplifies the
>> effect (less loader overhead...). For comparison, here is the Intel(R)
>> Xeon(R) Gold 6430 again:
>> ./selfexec_patch 1000 ran 1.23 ± 0.04 times faster than ./selfexec_master 1000
>>
>> It looks like there is something special going on with the 6430, or I
>> am missing something... It would be great if someone that has access to
>> a xeon gold 6430 could reproduce this.
>
> I've been struggling with a simple reproducer for this.  I can reproduce
> the performance improvement from your caching patch even with dynamic
> linking (although it less pronounced).  However, I have failed to build
> something standalone using __cpuid_count calls that shows the same
> performance behavior.
>
> In perf, I can see that a significant part of the execution time in your
> reproducer is attributed to CPUID, and >20% of all cycles (userspace and
> kernel) end up in the CPUID/machine detection code in ld.so.  On its
> own, this could be a sampling artifact because CPUID is a serializing
> instruction.  However, your caching change shows that there are
> performance improvements to be had.  And applying Amdahl's law in
> reverse suggests that these CPUID instructions must take significant
> time for real.
>
> My best guess at this point is that the CPUID instructions negatively
> interact with the loops and conditional branching in the ld.so code.
>
> I want to understand this better.  The same CPUID instructions execute
> much faster in other contexts even on the Xeon 6430 CPU, so maybe we can
> replicate these other conditions in ld.so and eliminate even more
> overhead.
>
> Ideas very welcome.

I believe the attached program shows the behavior.  It is based on a
CPUID execution trace from ld.so (instrumentation with _dl_printf),
but it does not reproduce the loop/if structure of ld.so.

The entire sequence executes roughly 4.5 times slower on Sapphire Rapids
than on Ice Lake.  This is despite the slight frequency increase between
Xeon 6330 and Xeon 6430.  On Xeon 6330, the number of CPUID instructions
in a similar trace is actually much larger (71 instructions instead of
39 instructions), but the performance benefit from the caching patch is
greatly reduced.  Despite the additional CPUID instructions executed, in
the perf profile, intel_check_word.constprop.0 is well behind
_dl_relocate_object_no_relro on Xeon 6330.

Sunil, is this sufficient for you to investigate further?

Thanks,
Florian
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: loop.c
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20260416/ee01ead3/attachment.c>