[PATCH 09/20] alpha: add EV6/EV7 assembly memcmp
Matt Turner
mattst88@gmail.com
Wed Aug 12 01:19:46 GMT 2026
Alpha had no assembly memcmp and fell back to the generic C implementation.
Add one for the alphaev6 directory (shared by the 21264 and 21364).
When both operands share alignment, byte-align to a quadword and then compare
64 bytes per iteration, OR-accumulating the per-quad xors; a non-zero
accumulator means the difference is somewhere in the block, which a byte scan
from the block base pinpoints to return the correct sign. Operands of
differing alignment use a byte compare.
The loop issues a plain read prefetch (LDL to R31) on both streams eight cache
lines ahead. On the 21364 (EV7), whose on-chip memory controller has a long
memory latency, this hides the Bcache misses that otherwise stall the compare;
on the 21264 it is at worst a dropped hint, so no IMPLVER dispatch is needed.
A read prefetch -- rather than the modify-intent LDS/LDT used by memset/memcpy
-- is deliberate: memcmp never writes its operands, so requesting the lines for
ownership would only add coherence traffic on a multiprocessor.
The 1 to 7 bytes left after the loops are compared with a single masked
quadword rather than a byte at a time, which would cost about eight issue
slots per byte and dominate any length that is not a multiple of eight. Both
operands are 0mod8 by then, since the head aligned them and everything since
has advanced by whole quadwords, and an aligned quadword cannot cross a page
boundary, so a whole one can be loaded at either pointer even though only some
of it belongs to the compare; masking the difference discards the rest.
Nothing is stored, so reading past the end of the operands is not observable.
A mismatch still falls into the byte loop, which is what determines which byte
differed and in which direction.
Measured on an AlphaServer ES47 (EV7, 1.3GHz), cold cyc/call vs the generic C:
size generic asm
1024 3617 1502 2.35x
4096 14527 6423 2.24x
65536 231362 96424 2.40x
262144 916913 386234 2.37x
bcmp is provided as a weak alias, as in the generic implementation.
Verified on an EV68CB against the C memcmp for every length from 0 to 300 at
24 alignments, both for equal operands and with a difference planted at every
position in turn, comparing the sign of the result.
__memcmpeq is aliased to memcmp here too. string/memcmp.c provides that alias
for the C implementation and string/memcmpeq.c is deliberately empty, so an
assembly memcmp that replaces the C one has to define it, as sparc64 and s390
do. Without it an --host=alphaev6 or later build links no __memcmpeq at all,
which drops a symbol the ABI lists and fails to link string/inl-tester.
---
sysdeps/alpha/alphaev6/memcmp.S | 226 ++++++++++++++++++++++++++++++++
1 file changed, 226 insertions(+)
create mode 100644 sysdeps/alpha/alphaev6/memcmp.S
diff --git ./sysdeps/alpha/alphaev6/memcmp.S ./sysdeps/alpha/alphaev6/memcmp.S
new file mode 100644
index 0000000000..0a3d2e11e6
--- /dev/null
+++ ./sysdeps/alpha/alphaev6/memcmp.S
@@ -0,0 +1,226 @@
+/* Copyright (C) 2026 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <https://www.gnu.org/licenses/>. */
+
+/*
+ * EV6/EV7 memcmp. When both operands share alignment, byte-align to a
+ * quadword and compare 64 bytes per iteration, OR-accumulating the per-quad
+ * xors; a non-zero accumulator means a difference lies in the block, which a
+ * byte scan from the block base then pinpoints to return the correct sign.
+ * Operands of differing alignment fall back to a byte compare.
+ *
+ * The loop issues a plain read prefetch (LDL to R31) on both streams several
+ * cache lines ahead. On the 21364 (EV7), whose on-chip memory controller has
+ * a long latency, this hides the Bcache misses that otherwise stall the
+ * compare; on the 21264 it is at worst a dropped hint. A read prefetch (not
+ * the modify-intent LDS/LDT) is deliberate: memcmp never writes its operands,
+ * so requesting lines for ownership would only add coherence traffic on SMP.
+ *
+ * Temp usage:
+ * $0 - result
+ * $1,$2 - scratch
+ * $3,$4 - loaded data
+ * $5,$6 - xor / accumulator
+ */
+
+#include <sysdep.h>
+
+ .arch ev6
+ .set noreorder
+ .set noat
+
+/* Read prefetch distance, in bytes (8 cache lines ahead). The CWG asks for
+ 8/n cache blocks for n streams against the 21264's 8-entry MAF, so four for
+ the two streams here; measured on an EV68CB the curve is flat from four
+ blocks out to eight, and only a distance below two costs anything.
+
+ These must stay plain read prefetches (LDL to R31). Asking for the lines
+ with modify intent instead (LDS to F31) measures 2x slower on an EV68CB --
+ 756446 cycles against 368989 for a cold 256KB compare -- because ReadBlkMod
+ provokes about twice the Mbox replay traps on a stream that is only ever
+ read. Bcache misses are unchanged, so it is not extra memory traffic.
+ ReadBlkModEN (LDT to F31) measures identical to the plain prefetch, since
+ the evict-next variants are 21364-only and degrade here. */
+#define PF (8 * 64)
+
+ENTRY(memcmp)
+ .prologue 0
+
+ mov $31, $0 # E : default result 0 (equal)
+ beq $18, $ret # U : n == 0 -> equal
+ xor $16, $17, $1 # E : do the operands share alignment?
+ and $1, 7, $1 # E : ... mod 8
+
+ bne $1, $bytewise # U : different alignment -> byte compare
+ and $16, 7, $1 # E : already 0mod8?
+ beq $1, $aligned # U : yes
+ nop # E :
+
+ /* Same misalignment: byte-compare up to a 0mod8 boundary. */
+$head:
+ ldbu $3, 0($16) # L : a byte
+ ldbu $4, 0($17) # L : b byte
+ lda $16, 1($16) # E : a++
+ lda $17, 1($17) # E : b++
+
+ subq $3, $4, $0 # E : difference (unsigned bytes)
+ bne $0, $ret # U : differ -> return sign
+ subq $18, 1, $18 # E : n--
+ beq $18, $reteq # U : consumed all -> equal
+
+ and $16, 7, $1 # E : aligned now?
+ bne $1, $head # U : keep going
+
+$aligned:
+ cmple $18, 63, $1 # E : at least one 64B block?
+ bne $1, $quadtail # U : no
+ nop # E :
+ nop # E :
+
+ .align 4
+$loop:
+ ldl $31, PF($16) # L : read prefetch a, PF bytes ahead
+ ldl $31, PF($17) # L : read prefetch b
+ ldq $3, 0($16) # L : a[0..7]
+ ldq $4, 0($17) # L : b[0..7]
+
+ xor $3, $4, $5 # E : seed the accumulator
+ ldq $3, 8($16) # L :
+ ldq $4, 8($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 16($16) # L :
+ ldq $4, 16($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 24($16) # L :
+ ldq $4, 24($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 32($16) # L :
+ ldq $4, 32($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 40($16) # L :
+ ldq $4, 40($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 48($16) # L :
+ ldq $4, 48($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 56($16) # L :
+ ldq $4, 56($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E : full 64B xor summary
+ bne $5, $bytewise # U : a difference is in this block
+ subq $18, 64, $18 # E : n -= 64
+ lda $16, 64($16) # E : a += 64
+
+ lda $17, 64($17) # E : b += 64
+ cmple $18, 63, $1 # E : another full block?
+ beq $1, $loop # U :
+ nop # E :
+
+ /* 0..63 bytes left, still 0mod8: compare remaining quads. */
+$quadtail:
+ subq $18, 8, $2 # E : (n - 8)
+ blt $2, $tailquad # U : < 8 left -> masked quad
+
+$qt:
+ ldq $3, 0($16) # L :
+ ldq $4, 0($17) # L :
+ xor $3, $4, $5 # E :
+ bne $5, $bytewise # U : differ in this quad (n unchanged)
+
+ lda $16, 8($16) # E : a += 8
+ lda $17, 8($17) # E : b += 8
+ mov $2, $18 # E : commit n -= 8
+ subq $18, 8, $2 # E : (n - 8)
+
+ bge $2, $qt # U : another quad
+ nop # E :
+ nop # E :
+ nop # E :
+
+ /*
+ * 1..7 bytes left, both operands 0mod8 -- the head aligned them and
+ * everything since has advanced by whole quadwords. An aligned
+ * quadword cannot cross a page boundary, so loading a whole one at
+ * either pointer is safe even though only $18 bytes belong to the
+ * compare; masking the difference down to those bytes discards
+ * whatever followed. Nothing is stored, so reading past the end of
+ * the operands is not observable.
+ *
+ * This is only the tail. A mismatch still falls into the byte loop
+ * below, which is what works out which byte differed and in which
+ * direction, and which the block and quad loops above also branch to.
+ */
+$tailquad:
+ beq $18, $reteq # U : nothing left -> equal
+ nop # E :
+ nop # E :
+ nop # E :
+
+ ldq $3, 0($16) # L :
+ ldq $4, 0($17) # L :
+ xor $3, $4, $5 # E :
+ mskql $5, $18, $5 # U : only the bytes being compared
+
+ beq $5, $reteq # U : all equal
+ nop # E :
+ nop # E :
+ nop # E :
+
+ /* Tail bytes, and the unaligned / locate path. On entry from a block
+ or quad mismatch, the first differing byte lies within the bytes
+ scanned here, so the loop returns its sign before running off. */
+$bytewise:
+ beq $18, $reteq # U : nothing left -> equal
+ nop # E :
+
+$bl:
+ ldbu $3, 0($16) # L :
+ ldbu $4, 0($17) # L :
+ lda $16, 1($16) # E :
+ lda $17, 1($17) # E :
+
+ subq $3, $4, $0 # E : unsigned byte difference
+ bne $0, $ret # U : differ -> sign
+ subq $18, 1, $18 # E : n--
+ bne $18, $bl # U :
+
+$reteq:
+ mov $31, $0 # E : equal
+$ret:
+ ret $31, ($26), 1 # L0 :
+
+END(memcmp)
+libc_hidden_builtin_def (memcmp)
+#ifdef weak_alias
+# undef bcmp
+weak_alias (memcmp, bcmp)
+#endif
+#undef __memcmpeq
+strong_alias (memcmp, __memcmpeq)
+libc_hidden_def (__memcmpeq)
--
2.54.0
More information about the Libc-alpha
mailing list