[PATCH 08/20] alpha: fold null and difference tests in strcmp aligned loop

[Matt Turner]

The co-aligned main loop tested for a difference (xor; bne) and for the
terminating null (cmpbge; beq) with two separate branches per quadword.
On EV6/EV7 the loop is issue-bound and right at the one-branch-per-cycle
limit, so the second branch is a bottleneck.

Combine the two into a single syndrome -- "or" of the difference bits and
the null mask -- so one branch closes the loop and the difference test for
the first word moves out into the head.  The loop still loads the next
word only after the current one tests clean (a zero syndrome implies no
null, so the string continues and the next word is mapped), so it never
reads past the terminating null onto an unmapped page.

bench-strcmp on EV7 (21364): co-aligned compares ~6-9% faster across
lengths that stay in L1, neutral once memory-bound; the mutually
misaligned path is unchanged.  Passes string/test-strcmp.
---
 sysdeps/alpha/strcmp.S | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git ./sysdeps/alpha/strcmp.S ./sysdeps/alpha/strcmp.S
index c7f092d1b7..6454b7aab0 100644
--- ./sysdeps/alpha/strcmp.S
+++ ./sysdeps/alpha/strcmp.S
@@ -53,22 +53,32 @@ $aligned:
 	ornot	t0, t3, t0	# .. e1 :
 	cmpbge	zero, t1, t7	# e0    : bits set iff null found
 	bne	t7, $eos	# e1 (zdb)
+	xor	t0, t1, t2	# e0    : difference in the first word?
+	bne	t2, $wordcmp	# .. e1 (zdb)
+
+	/* Aligned compare main loop.  Fold the difference test (xor) and the
+	   null test (cmpbge) into a single syndrome with "or", so one branch
+	   per quadword closes the loop instead of two.  Each load is gated by
+	   the previous word being clean -- a zero syndrome means no null, so
+	   the string continues and the next word is mapped -- hence we never
+	   read past the terminating null onto an unmapped page.
 
-	/* Aligned compare main loop.
 	   On entry to this basic block:
 	   t0 == an s1 word.
-	   t1 == an s2 word not containing a null.  */
+	   t1 == an s2 word, equal to t0 and not containing a null.  */
 
 $a_loop:
-	xor	t0, t1, t2	# e0	:
-	bne	t2, $wordcmp	# .. e1 (zdb)
 	ldq_u	t1, 8(a1)	# e0    :
 	ldq_u	t0, 8(a0)	# .. e1 :
 	addq	a1, 8, a1	# e0    :
 	addq	a0, 8, a0	# .. e1 :
-	cmpbge	zero, t1, t7	# e0    :
-	beq	t7, $a_loop	# .. e1 (zdb)
-	br	$eos		# e1    :
+	xor	t0, t1, t2	# e0    : bytes that differ
+	cmpbge	zero, t1, t7	# .. e1 : bits set iff null found
+	or	t2, t7, t8	# e0    : syndrome = difference | null
+	beq	t8, $a_loop	# .. e1 (zdb) : clean word, keep going
+
+	bne	t7, $eos	# e0    : null present (handles diff-before-null)
+	br	$wordcmp	# .. e1 : a pure difference, no null
 
 	/* The two strings are not co-aligned.  Align s1 and cope.  */
 
-- 
2.54.0