[PATCH 2/2] x86: Enable ERMS memset for AMD Zen5/Zen6 servers
Adhemerval Zanella Netto
adhemerval.zanella@linaro.org
Mon Nov 16 12:49:20 GMT 2026
On 13/11/26 09:43, Sajan Karumanchi wrote:
> Following the ERMS enablement for memcpy/memmove on AMD Zen5+ servers,
> microcode improvements also make REP STOSB outperform vectorized memset
> for buffers 32KB and larger.
>
> Extend the Zen5+ server ERMS tuning to memset by lowering the default
> rep_stosb_threshold from SIZE_MAX to 32KB when Prefer_Server_ERMS is set.
> In addition, enable Avoid_Non_Temporal_Memset on these server models so
> large memsets remain on ERMS, which edges out non-temporal stores.
>
> Performance improvements:
> - Range: [32KB, SIZE_MAX]
> - Improvement: 9% to 43%
The non-temporal memset was added for very large buffers to avoid evicting
the shared L3 and thus a single-threaded throughput benchmark does not show
the real cost.
Before we drop NT stores on server parts, please show either a multi-threaded
result (for example, a few threads running memsets larger than L3 next to a
cache-sensitive workload), or some AMD documentation that REP STOSB already
uses a streaming/no-allocate protocol at these sizes.
Also add the bench-memset-large and bench-memset-walk numbers, with a
misaligned destination.
>
> Co-authored-by: Sundari Swathy Meena H <sundari.swathymeenah@amd.com>
> Reviewed-by: Ganesh Gopalasubramanian <Ganesh.Gopalasubramanian@amd.com>
> ---
> sysdeps/x86/cpu-features.c | 9 +++++++--
> sysdeps/x86/dl-cacheinfo.h | 13 ++++++++++---
> 2 files changed, 17 insertions(+), 5 deletions(-)
>
> diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
> index bdd8ddb9ab..ac191ae7a5 100644
> --- a/sysdeps/x86/cpu-features.c
> +++ b/sysdeps/x86/cpu-features.c
> @@ -1069,8 +1069,13 @@ disable_tsx:
> || (model >= 0x90 && model <= 0x9B)
> || (model >= 0xA0 && model <= 0xAB)
> || (model >= 0xC0 && model <= 0xCF))
> - cpu_features->preferred[index_arch_Prefer_Server_ERMS]
> - |= bit_arch_Prefer_Server_ERMS;
> + {
> + cpu_features->preferred[index_arch_Prefer_Server_ERMS]
> + |= bit_arch_Prefer_Server_ERMS;
> + /* Avoid non-temporal stores for memset on these models. */
> + cpu_features->preferred[index_arch_Avoid_Non_Temporal_Memset]
> + |= bit_arch_Avoid_Non_Temporal_Memset;
> + }
> }
>
The rationale is that REP STOSB beats NT stores, so this should only
apply when ERMS is usable; and some hypervisors do not expose ERMS.
Under qemu-user with -cpu EPYC-Genoa,family=26,model=2,-erms,-fsrm, both
thresholds are 0xffffffffffffffff; with model=0x44 they stay at 0x1800000.
Please guard it with CPU_FEATURE_USABLE_P (cpu_features, ERMS).
> if (CPU_FEATURE_USABLE_P (cpu_features, AVX))
> diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h
> index 3fd7c77bc1..8a470421b5 100644
> --- a/sysdeps/x86/dl-cacheinfo.h
> +++ b/sysdeps/x86/dl-cacheinfo.h
> @@ -1325,9 +1325,16 @@ dl_init_cacheinfo (struct cpu_features *cpu_features)
> long int, NULL);
> if (cpu_features->basic.kind == arch_kind_amd
> && !TUNABLE_IS_INITIALIZED (x86_rep_stosb_threshold))
> - /* For AMD Zen3+ architecture, the performance of the vectorized loop is
> - slightly better than ERMS. */
> - rep_stosb_threshold = SIZE_MAX;
> + {
> + /* Setting erms:rep-stosb lower bound threshold to 32KB
> + for AMD Zen5+ server-class models. */
> + if (CPU_FEATURES_ARCH_P (cpu_features, Prefer_Server_ERMS))
> + rep_stosb_threshold = 32 * 1024;
> + else
> + /* For AMD Zen3+ architecture, the performance of the vectorized
> + loop is slightly better than ERMS. */
> + rep_stosb_threshold = SIZE_MAX;
> + }
>
As in 1/2, the else comment now covers Zen3, Zen4 and the family 0x1A models
outside the list.
The new lines also use spaces instead of tabs.
> /*
> For memset, the non-temporal implementation is only accessed through the
More information about the Libc-alpha
mailing list