Bug 30428 - New AMD cache size computation logic does not work for some CPUs, hypervisors
Summary: New AMD cache size computation logic does not work for some CPUs, hypervisors
Status: NEW
Alias: None
Product: glibc
Classification: Unclassified
Component: dynamic-link (show other bugs)
Version: 2.38
: P1 critical
Target Milestone: 2.39
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-05-08 16:42 UTC by Florian Weimer
Modified: 2023-09-14 20:48 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fw: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Weimer 2023-05-08 16:42:13 UTC
This commit:

commit 103a469dc7755fd9e8ccf362f3dd4c55dc761908
Author: Sajan Karumanchi <sajan.karumanchi@amd.com>
Date:   Wed Jan 18 18:29:04 2023 +0100

    x86: Cache computation for AMD architecture.
    
    All AMD architectures cache details will be computed based on
    __cpuid__ `0x8000_001D` and the reference to __cpuid__ `0x8000_0006` will be
    zeroed out for future architectures.
    
    Reviewed-by: Premachandra Mallappa <premachandra.mallappa@amd.com>

changed cache size computation on the AMD architecture.

However, the new way of doing things is not supported by all AMD CPUs. This CPU:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 16
model           : 6
model name      : AMD Turion(tm) II Neo N40L Dual-Core Processor
stepping        : 3
microcode       : 0x10000c8
cpu MHz         : 800.000
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate vmmcall npt lbrv svm_lock nrip_save
bugs            : tlb_mmatch apic_c1e fxsave_leak sysret_ss_attrs null_seg amd_e400 spectre_v1 spectre_v2
bogomips        : 2995.32
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

reports all zeros for its caches after this change (build from commit cea74a4a24c36202309e8254f1f938e2166488f3, which includes commit mentioned above):

$ ./ld.so --list-diagnostics | grep -E 'level|threshold'
x86.cpu_features.non_temporal_threshold=0x4040
x86.cpu_features.rep_movsb_threshold=0x800
x86.cpu_features.rep_movsb_stop_threshold=0x0
x86.cpu_features.rep_stosb_threshold=0x800
x86.cpu_features.level1_icache_size=0x0
x86.cpu_features.level1_icache_linesize=0x0
x86.cpu_features.level1_dcache_size=0x0
x86.cpu_features.level1_dcache_assoc=0x0
x86.cpu_features.level1_dcache_linesize=0x0
x86.cpu_features.level2_cache_size=0x0
x86.cpu_features.level2_cache_assoc=0x0
x86.cpu_features.level2_cache_linesize=0x0
x86.cpu_features.level3_cache_size=0x0
x86.cpu_features.level3_cache_assoc=0x0
x86.cpu_features.level3_cache_linesize=0x0
x86.cpu_features.level4_cache_size=0xffffffffffffffff

A build from the 2.36 branch (commit b7008a92f505632f32b313d1033d6d15c99a0b31) yields this instead:

$ ./ld.so --list-diagnostics | grep -E 'level|threshold'
x86.cpu_features.non_temporal_threshold=0xc0000
x86.cpu_features.rep_movsb_threshold=0x800
x86.cpu_features.rep_movsb_stop_threshold=0x100000
x86.cpu_features.rep_stosb_threshold=0x800
x86.cpu_features.level1_icache_size=0x10000
x86.cpu_features.level1_icache_linesize=0x40
x86.cpu_features.level1_dcache_size=0x10000
x86.cpu_features.level1_dcache_assoc=0x2
x86.cpu_features.level1_dcache_linesize=0x40
x86.cpu_features.level2_cache_size=0x100000
x86.cpu_features.level2_cache_assoc=0x10
x86.cpu_features.level2_cache_linesize=0x40
x86.cpu_features.level3_cache_size=0x0
x86.cpu_features.level3_cache_assoc=0x0
x86.cpu_features.level3_cache_linesize=0x0
x86.cpu_features.level4_cache_size=0xffffffffffffffff

So it's a regression.

The CPU is probably old enough that we don't use temporal stores, so there is probably not going to be a crash in glibc. But lack of accurate cache sizes probably still causes performance regressions elsewhere (although no one is going to use CPUs that old for their performance, admittedly).

Some hypervisors also fail to pass through these CPUID values even if they identify the CPU as an AMD model: https://bugzilla.redhat.com/show_bug.cgi?id=2196271

Addressing hypervisor compatibility might be the important part here.
Comment 1 Florian Weimer 2023-05-08 20:29:47 UTC
Just to clarify: this regression affects sysconf (_SC_LEVEL2_CACHE_SIZE) and similar configuration values, so it impacts more than just glibc-internal tuning decisions.
Comment 2 Florian Weimer 2023-07-04 17:34:05 UTC
Initial patch posted (still alters results compared to what we had before):

[PATCH] x86: Fix for cache computation on AMD legacy cpus.
<https://sourceware.org/pipermail/libc-alpha/2023-June/148763.html>
Comment 3 Martin Jambor 2023-09-14 20:33:37 UTC
Has this been addressed(*)?  If so, could the respective commit ID's be posted here for reference, please?


(*) https://sourceware.org/pipermail/libc-alpha/2023-June/148812.html would suggest that it was but perhaps not entirely?
Comment 4 Florian Weimer 2023-09-14 20:48:36 UTC
Fixed for 2.39 via:

commit dcad5c8578130dec7f35fd5b0885304b59f9f543
Author: Sajan Karumanchi <sajan.karumanchi@amd.com>
Date:   Tue Aug 1 15:20:55 2023 +0000

    x86: Fix for cache computation on AMD legacy cpus.
    
    Some legacy AMD CPUs and hypervisors have the _cpuid_ '0x8000_001D'
    set to Zero, thus resulting in zeroed-out computed cache values.
    This patch reintroduces the old way of cache computation as a
    fail-safe option to handle these exceptions.
    Fixed 'level4_cache_size' value through handle_amd().
    
    Reviewed-by: Premachandra Mallappa <premachandra.mallappa@amd.com>
    Tested-by: Florian Weimer <fweimer@redhat.com>

It seems that we are still missing the backport to 2.37.