[PATCH 00/20] alpha: tune the memory and string routines, and add IFUNC dispatch

Wed Aug 12 01:19:37 GMT 2026

Alpha's memory and string routines have had little attention since the
21264 shipped, and glibc can only pick between them at build time. Both
halves of that are addressed here: the first sixteen patches tune the
existing assembly and add what was missing, and the last four let one
build choose an implementation at run time.

Both generations were measured, on an AlphaServer ES47 (EV7, 21364,
1.3GHz) and on a DS15 (EV68CB, 1GHz), and both gain; the 21264 side is
not merely left alone. On the ES47, against what is in glibc today:

  memcpy    -11% to -21% cold, and -53% to -60% hot from 1KB upwards
  memset    -16% to -33% cold, -46% hot at 256KB
  memcmp    2.24x to 2.40x, new assembly against the generic C
  memchr    1.17x to 1.19x
  strcmp    6-9% on co-aligned compares (bench-strcmp)

On the DS15 (2MB off-chip Bcache):

  memset    3.0x on a cold 256KB call, issuing the wh64 write hint four
            cache lines ahead of the stores instead of one
  memcpy    6-8% cold from 4KB upwards, wh64 two lines ahead
  memcpy    19-39% fewer cycles hot from 512B upwards, by loading all
            eight quads of a block before storing any of them
  memcpy    1.27x to 1.93x hot for lengths that are not whole quadwords,
            which the old byte-at-a-time tail dominated
  memcpy    1.11x to 1.79x hot from 64 to 160 bytes, below the unrolled
            loop
  strspn    4.3x at 64B against the old scalar loop

The prefetching was mis-tuned on both parts, in different ways. The
21264 had the right hint at the wrong depth: memset issued wh64 a single
line ahead, which is below the architectural minimum the CWG asks for,
and was not merely useless but slower than issuing no hint at all --
453k cycles against 406k for that cold 256KB call, and 151k once the
hint runs four lines ahead. Mbox replay traps drop by 83% for it, from
69885 per call to 11467, while Bcache misses do not move, so no extra
data is being fetched; the loop was trapping and replaying its own store
stream at close to one replay per instruction retired. The 21364, by
contrast, wants a different hint entirely, which is the wh64 result
below.

Most of the win comes from the prefetch hints rather than the loops. The
existing code issues wh64 two cache lines ahead, which is far too close
to cover the 21364's memory latency, and wh64 turns out to be the wrong
hint on this part regardless of distance: it is a net loss both hot and
cold, where a plain read-with-modify-intent prefetch (ReadBlkMod, lds to
$f31) six lines out wins. Re-measured across a 90-configuration sweep of
hint type, distance and block size, memset comes out at 0.87 of the
current code hot with modify-intent, against 1.13 for wh64 and 1.74 for
its evict-next variant; no distance or block size rescues the write
hints. The EV7-specific hints are guarded on implver, so 21264 behavior
is unchanged where wh64 was measured to be the better hint there.

The wh64 result is genuinely surprising and we cannot explain it. wh64
exists precisely for this pattern: it claims a cache line for writing
without first reading it, which is exactly what a memset or the
destination of a memcpy wants, and on the 21264 it does behave that way
once it is issued deep enough -- 3.0x on the cold memset above. On the
21364 it loses to a prefetch that does fetch the line, which should be
strictly more work. Mbox replay traps were the lead, since they
concentrate on the unrolled stores and the wh64 itself, but they do not
settle it either: modify-intent prefetch records more replays than wh64
and is still faster. So the tuning here follows the cycle counts rather
than a model of the memory system, and if someone knows what the 21364
is actually doing differently, we would like to hear it.

The hint type, prefetch distance and loop block size in these patches
were all chosen by sweeping them on the hardware rather than by
inspection, and the tooling that does it is public:

  https://github.com/alphalinux-org/alpha-mem-bench

It builds one parametrized kernel per routine, self-times every
configuration with rpcc, and ranks them per CPU for hot and cold working
sets; it also reads the 21364 performance counters, which is how the
wh64 result above was tracked down to Mbox replay traps on the store
stream. The same kernels feed the correctness verifiers, so what is
measured and what is checked are the same code.

New assembly: memcmp, memmove, mempcpy (sharing the ev6 memcpy bulk
loop), and a vectorized single-character fast path for strspn.

The IFUNC patches then remove the build-time restriction. Today a distro
that builds for base alpha never gets any of this, while an
--host=alphaev67 build cannot run on earlier hardware; there is no
configuration that does both. The last four patches add init-arch.h
wrappers for implver and amask, an ifunc-impl-list for the string tests,
a direct-named implementation for the dynamic linker (which cannot
dispatch through an IFUNC before it has relocated itself), and the
dispatchers themselves: memset, memcpy, memcmp, memmove and mempcpy
dispatch on implver, strlen and the other EV67 routines on the CIX bit
in amask. One patch to abilist.awk is a prerequisite, so that
STO_ALPHA_STD_GPLOAD symbols compare as functions.

There is no ABI change; the dispatchers keep the existing symbols,
aliases and internal __GI_ names.

Note on toolchain support: IFUNC on alpha needs STT_GNU_IFUNC support in
the assembler and linker, and __attribute__((ifunc)) in the compiler.
configure already probes for the binutils side, so a toolchain without
it simply builds with multi-arch disabled and gets the build-time
selection as before. The two toolchain pieces are posted:

  binutils, STT_GNU_IFUNC and R_ALPHA_IRELATIVE:
  [PATCH 0/4] alpha: add IFUNC support
  https://sourceware.org/pipermail/binutils/2026-August/150713.html

  GCC, __attribute__((ifunc)) for alpha*-*-linux*:
  [PATCH v2] alpha: enable gnu-indirect-function support for alpha-linux
  https://gcc.gnu.org/pipermail/gcc-patches/2026-August/726971.html

Testing. Full make check on the ES47 under Linux 7.2.0-rc1 with GCC
16.2.0 and binutils 2.47, run twice: once with the multiarch dispatchers
active and once with the CPU assembly selected directly. Both give
identical results, with no regressions, so routing every call through an
IFUNC resolver changes nothing observable.

The new assembly was also checked outside the test suite, each routine
against the generic C over every size from 0 to 300 at 24 alignments
with a difference planted at each position in turn, on hardware and
under qemu-alpha.

Matt Turner (20):
  alpha: use prefetch-with-modify-intent in memset on EV7
  alpha: issue the ev6 memset wh64 four lines ahead
  alpha: use prefetch-with-modify-intent in memcpy on EV7
  alpha: issue the ev6 memcpy wh64 two lines ahead
  alpha: document ev6 memcpy design constraints
  alpha: finalize the ev6 memcpy wh64 address late to avoid Mbox replays
  alpha: deepen memchr prefetch distance for EV7
  alpha: fold null and difference tests in strcmp aligned loop
  alpha: add EV6/EV7 assembly memcmp
  alpha: add EV6/EV7 assembly memmove
  alpha: add assembly mempcpy sharing the ev6 memcpy
  alpha: finish the ev6 memcpy and memmove tails without a byte loop
  alpha: copy four quads a trip below the ev6 unrolled loop
  alpha: store four quads a trip below the ev6 wh64 loop
  alpha: load all 8 quads before storing in the ev6 memcpy unrolled loop
  alpha: add vectorized single-char fast path for strspn
  alpha: treat STO_ALPHA_STD_GPLOAD as a function in abilist.awk
  alpha: add IFUNC support to the dynamic linker
  alpha: add multiarch infrastructure for IFUNC dispatch
  alpha: add multiarch IFUNC dispatchers

 elf/elf.h                                     |   1 +
 scripts/abilist.awk                           |   4 +-
 sysdeps/alpha/alphaev6/memcmp.S               | 226 +++++++++
 sysdeps/alpha/alphaev6/memcpy.S               | 290 +++++++++--
 sysdeps/alpha/alphaev6/memmove.S              | 457 ++++++++++++++++++
 sysdeps/alpha/alphaev6/mempcpy.S              |  27 ++
 sysdeps/alpha/alphaev6/memset.S               | 126 ++++-
 sysdeps/alpha/alphaev6/multiarch/Implies      |   1 +
 sysdeps/alpha/alphaev67/multiarch/Implies     |   1 +
 sysdeps/alpha/dl-irel.h                       |  51 ++
 sysdeps/alpha/dl-machine.h                    |  29 ++
 sysdeps/alpha/memchr.c                        |  15 +-
 sysdeps/alpha/multiarch/Makefile              |  28 ++
 .../alpha/multiarch/dl-symbol-redir-ifunc.h   |  37 ++
 sysdeps/alpha/multiarch/ifunc-impl-list.c     |  80 +++
 sysdeps/alpha/multiarch/init-arch.h           |  69 +++
 sysdeps/alpha/multiarch/memcmp.c              |  60 +++
 sysdeps/alpha/multiarch/memcmp_ev6.S          |  41 ++
 sysdeps/alpha/multiarch/memcmp_generic.c      |  35 ++
 sysdeps/alpha/multiarch/memcpy.c              |  49 ++
 sysdeps/alpha/multiarch/memcpy_ev6.S          |  32 ++
 sysdeps/alpha/multiarch/memcpy_generic.c      |  26 +
 sysdeps/alpha/multiarch/memmove.c             |  49 ++
 sysdeps/alpha/multiarch/memmove_ev6.S         |  32 ++
 sysdeps/alpha/multiarch/memmove_generic.c     |  26 +
 sysdeps/alpha/multiarch/mempcpy.c             |  61 +++
 sysdeps/alpha/multiarch/mempcpy_ev6.S         |  39 ++
 sysdeps/alpha/multiarch/mempcpy_generic.c     |  39 ++
 sysdeps/alpha/multiarch/memset.c              |  49 ++
 sysdeps/alpha/multiarch/memset_ev5.S          |  33 ++
 sysdeps/alpha/multiarch/memset_ev6.S          |  32 ++
 sysdeps/alpha/multiarch/rawmemchr.c           |  55 +++
 sysdeps/alpha/multiarch/rawmemchr_ev5.S       |  36 ++
 sysdeps/alpha/multiarch/rawmemchr_ev67.S      |  35 ++
 sysdeps/alpha/multiarch/stpcpy.c              |  62 +++
 sysdeps/alpha/multiarch/stpcpy_ev5.S          |  39 ++
 sysdeps/alpha/multiarch/stpcpy_ev67.S         |  38 ++
 sysdeps/alpha/multiarch/stpncpy.c             |  55 +++
 sysdeps/alpha/multiarch/stpncpy_ev5.S         |  36 ++
 sysdeps/alpha/multiarch/stpncpy_ev67.S        |  35 ++
 sysdeps/alpha/multiarch/strcat.c              |  49 ++
 sysdeps/alpha/multiarch/strcat_ev5.S          |  33 ++
 sysdeps/alpha/multiarch/strcat_ev67.S         |  32 ++
 sysdeps/alpha/multiarch/strchr.c              |  50 ++
 sysdeps/alpha/multiarch/strchr_ev5.S          |  36 ++
 sysdeps/alpha/multiarch/strchr_ev67.S         |  35 ++
 sysdeps/alpha/multiarch/strlen.c              |  49 ++
 sysdeps/alpha/multiarch/strlen_ev5.S          |  33 ++
 sysdeps/alpha/multiarch/strlen_ev67.S         |  32 ++
 sysdeps/alpha/multiarch/strncat.c             |  49 ++
 sysdeps/alpha/multiarch/strncat_ev5.S         |  30 ++
 sysdeps/alpha/multiarch/strncat_ev67.S        |  29 ++
 sysdeps/alpha/strcmp.S                        |  24 +-
 sysdeps/alpha/strspn.c                        | 125 +++++
 54 files changed, 2976 insertions(+), 66 deletions(-)
 create mode 100644 sysdeps/alpha/alphaev6/memcmp.S
 create mode 100644 sysdeps/alpha/alphaev6/memmove.S
 create mode 100644 sysdeps/alpha/alphaev6/mempcpy.S
 create mode 100644 sysdeps/alpha/alphaev6/multiarch/Implies
 create mode 100644 sysdeps/alpha/alphaev67/multiarch/Implies
 create mode 100644 sysdeps/alpha/dl-irel.h
 create mode 100644 sysdeps/alpha/multiarch/Makefile
 create mode 100644 sysdeps/alpha/multiarch/dl-symbol-redir-ifunc.h
 create mode 100644 sysdeps/alpha/multiarch/ifunc-impl-list.c
 create mode 100644 sysdeps/alpha/multiarch/init-arch.h
 create mode 100644 sysdeps/alpha/multiarch/memcmp.c
 create mode 100644 sysdeps/alpha/multiarch/memcmp_ev6.S
 create mode 100644 sysdeps/alpha/multiarch/memcmp_generic.c
 create mode 100644 sysdeps/alpha/multiarch/memcpy.c
 create mode 100644 sysdeps/alpha/multiarch/memcpy_ev6.S
 create mode 100644 sysdeps/alpha/multiarch/memcpy_generic.c
 create mode 100644 sysdeps/alpha/multiarch/memmove.c
 create mode 100644 sysdeps/alpha/multiarch/memmove_ev6.S
 create mode 100644 sysdeps/alpha/multiarch/memmove_generic.c
 create mode 100644 sysdeps/alpha/multiarch/mempcpy.c
 create mode 100644 sysdeps/alpha/multiarch/mempcpy_ev6.S
 create mode 100644 sysdeps/alpha/multiarch/mempcpy_generic.c
 create mode 100644 sysdeps/alpha/multiarch/memset.c
 create mode 100644 sysdeps/alpha/multiarch/memset_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/memset_ev6.S
 create mode 100644 sysdeps/alpha/multiarch/rawmemchr.c
 create mode 100644 sysdeps/alpha/multiarch/rawmemchr_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/rawmemchr_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/stpcpy.c
 create mode 100644 sysdeps/alpha/multiarch/stpcpy_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/stpcpy_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/stpncpy.c
 create mode 100644 sysdeps/alpha/multiarch/stpncpy_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/stpncpy_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/strcat.c
 create mode 100644 sysdeps/alpha/multiarch/strcat_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/strcat_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/strchr.c
 create mode 100644 sysdeps/alpha/multiarch/strchr_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/strchr_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/strlen.c
 create mode 100644 sysdeps/alpha/multiarch/strlen_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/strlen_ev67.S
 create mode 100644 sysdeps/alpha/multiarch/strncat.c
 create mode 100644 sysdeps/alpha/multiarch/strncat_ev5.S
 create mode 100644 sysdeps/alpha/multiarch/strncat_ev67.S
 create mode 100644 sysdeps/alpha/strspn.c

base-commit: 9d7d6480880064cb5219aa267894ffabd0c2b20c
prerequisite-patch-id: b94cf83b54f2b172600c605a948d3ca89ca8fa28
prerequisite-patch-id: 9025a95668f5182383b6458d5e4360f7704691ab
prerequisite-patch-id: a1599078cf5182e0ad7a1b2c78fb6b5f62945722
prerequisite-patch-id: 54811c59a3a3904f2e9069035ef565a6f212abbf
-- 
2.54.0