[PATCH] PowerPC: stpcpy optimization for PPC64/POWER7
Adhemerval Zanella
azanella@linux.vnet.ibm.com
Thu Sep 19 16:23:00 GMT 2013
On 18-09-2013 17:40, Richard Henderson wrote:
> Hmm. That's register clobbering there. Gcc 4.7.2 generated
>
> 10000654: 7d 08 00 74 cntlzd r8,r8
> 10000658: 79 08 e8 c2 rldicl r8,r8,61,3
> 1000065c: 38 e8 ff f9 addi r7,r8,-7
> 10000660: 7c ca 38 2a ldx r6,r10,r7
> 10000664: 7c de 39 2a stdx r6,r30,r7
>
> Ah, wrong constraints on my asm, that just so happened to work here. Change
> all "=r" to "=&r" so that ralt et al does not overlap rsrc.
>
>
>
> r~
Thanks for the review and I have checked your suggestion with the modification
below on top of my patch. We still need the load/compare/store sequence to
avoid unaligned access to first doubleword.
diff --git a/sysdeps/powerpc/powerpc64/power7/stpcpy.S b/sysdeps/powerpc/powerpc64/power7/stpcpy.S
index 65ff6a0..116e8ee 100644
--- a/sysdeps/powerpc/powerpc64/power7/stpcpy.S
+++ b/sysdeps/powerpc/powerpc64/power7/stpcpy.S
@@ -41,14 +41,18 @@ EALIGN (__stpcpy, 4, 0)
li rMASK, 0
addi rRTN, rRTN, -8
ld rWORD, 0(rSRC)
- b L(g2)
+ cmpb rTMP, rWORD, rMASK
+ cmpdi rTMP, 0
+ beq L(g0)
+ mr rALT, rWORD
+ b L(g1)
.align 4
L(g0): ldu rALT, 8(rSRC)
stdu rWORD, 8(rRTN)
cmpb rTMP, rALT, rMASK
cmpdi rTMP, 0
- bne L(g1)
+ bne L(test)
ldu rWORD, 8(rSRC)
stdu rALT, 8(rRTN)
L(g2): cmpb rTMP, rWORD, rMASK
@@ -56,6 +60,16 @@ L(g2): cmpb rTMP, rWORD, rMASK
beq L(g0)
mr rALT, rWORD
+L(test):
+ addi rRTN, rRTN, 8
+ cntlzd rMASK, rTMP /* Extract bit offset of null byte. */
+ srdi rMASK, rMASK, 3 /* Convert bit offset to byte offset. */
+ addi rALT, rMASK, -7 /* Include the previous 7 bytes + nul. */
+ ldx rTMP, rSRC, rALT /* Perform one last unaligned copy. */
+ stdx rTMP, rRTN, rALT
+ add rRTN, rRTN, rMASK /* Adjust the return value. */
+ blr
+
And the results in the attached file (I used the stpcpy benchtest). As you can see
my initial patch still shows slight better latency.
-------------- next part --------------
stpcpy simple_stpcpy
Length 0, alignments in bytes 0/ 0: 5.6875 4.53125
Length 0, alignments in bytes 0/ 0: 5.15625 3.67188
Length 0, alignments in bytes 0/ 0: 4.57812 3.64062
Length 0, alignments in bytes 0/ 0: 4.21875 3.59375
Length 1, alignments in bytes 0/ 0: 4.59375 4.4375
Length 1, alignments in bytes 0/ 0: 4.5 4.1875
Length 1, alignments in bytes 0/ 1: 3.90625 5.15625
Length 1, alignments in bytes 1/ 0: 4.23438 4.375
Length 2, alignments in bytes 0/ 0: 5.42188 4.65625
Length 2, alignments in bytes 0/ 0: 5.04688 5.0625
Length 2, alignments in bytes 0/ 2: 6.25 5.25
Length 2, alignments in bytes 2/ 0: 4.32812 4.59375
Length 3, alignments in bytes 0/ 0: 6.5 5.60938
Length 3, alignments in bytes 0/ 0: 5.46875 4.79688
Length 3, alignments in bytes 0/ 3: 6.25 6.10938
Length 3, alignments in bytes 3/ 0: 5.21875 5.39062
Length 4, alignments in bytes 0/ 0: 6.25 5.96875
Length 4, alignments in bytes 0/ 0: 6.09375 5.67188
Length 4, alignments in bytes 0/ 4: 7.73438 7.25
Length 4, alignments in bytes 4/ 0: 4.1875 5.96875
Length 5, alignments in bytes 0/ 0: 6.29688 6.34375
Length 5, alignments in bytes 0/ 0: 6.25 6.54688
Length 5, alignments in bytes 0/ 5: 6.78125 6.25
Length 5, alignments in bytes 5/ 0: 5.25 6.75
Length 6, alignments in bytes 0/ 0: 7 7.96875
Length 6, alignments in bytes 0/ 0: 6.82812 7.42188
Length 6, alignments in bytes 0/ 6: 6.3125 7.125
Length 6, alignments in bytes 6/ 0: 5.6875 7.0625
Length 7, alignments in bytes 0/ 0: 6.54688 8.23438
Length 7, alignments in bytes 0/ 0: 6.54688 8.46875
Length 7, alignments in bytes 0/ 7: 5.5 7.96875
Length 7, alignments in bytes 7/ 0: 5.70312 8.32812
Length 8, alignments in bytes 0/ 0: 7.75 9.32812
Length 8, alignments in bytes 0/ 0: 7.5 9.23438
Length 8, alignments in bytes 0/ 0: 7.59375 9.09375
Length 8, alignments in bytes 0/ 0: 7.57812 9.07812
Length 9, alignments in bytes 0/ 0: 7.5 11.9844
Length 9, alignments in bytes 0/ 0: 7.5 12
Length 9, alignments in bytes 0/ 1: 6.14062 37.2188
Length 9, alignments in bytes 1/ 0: 6.1875 11.9844
Length 10, alignments in bytes 0/ 0: 7.60938 12.7969
Length 10, alignments in bytes 0/ 0: 7.60938 12.75
Length 10, alignments in bytes 0/ 2: 31.5 23.2969
Length 10, alignments in bytes 2/ 0: 6.57812 12.7812
Length 11, alignments in bytes 0/ 0: 7.53125 16.1094
Length 11, alignments in bytes 0/ 0: 7.5 16.0469
Length 11, alignments in bytes 0/ 3: 21.4688 21
Length 11, alignments in bytes 3/ 0: 6.28125 13.4688
Length 12, alignments in bytes 0/ 0: 7.57812 14.3594
Length 12, alignments in bytes 0/ 0: 7.6875 14.2969
Length 12, alignments in bytes 0/ 4: 8 16.5
Length 12, alignments in bytes 4/ 0: 5.875 14.3594
Length 13, alignments in bytes 0/ 0: 7.5 15.4062
Length 13, alignments in bytes 0/ 0: 7.60938 15.3281
Length 13, alignments in bytes 0/ 5: 14.6406 15.5
Length 13, alignments in bytes 5/ 0: 7.125 15.0469
Length 14, alignments in bytes 0/ 0: 7.67188 16.25
Length 14, alignments in bytes 0/ 0: 7.5 16.1406
Length 14, alignments in bytes 0/ 6: 12.1875 15.8281
Length 14, alignments in bytes 6/ 0: 7.76562 15.9375
Length 15, alignments in bytes 0/ 0: 5.17188 16.9375
Length 15, alignments in bytes 0/ 0: 5.21875 16.9062
Length 15, alignments in bytes 0/ 7: 11.4531 16.9375
Length 15, alignments in bytes 7/ 0: 7.92188 16.5
Length 16, alignments in bytes 0/ 0: 8.09375 17.9219
Length 16, alignments in bytes 7/ 2: 8.57812 17.4844
Length 32, alignments in bytes 0/ 0: 8.125 30.5
Length 32, alignments in bytes 6/ 4: 14.2344 30.3906
Length 64, alignments in bytes 0/ 0: 8.46875 56.2812
Length 64, alignments in bytes 5/ 6: 31.3906 232.406
Length 128, alignments in bytes 0/ 0: 21.9375 107.391
Length 128, alignments in bytes 4/ 0: 20.9688 104.328
Length 256, alignments in bytes 0/ 0: 32.4219 209.75
Length 256, alignments in bytes 3/ 2: 105.078 215.203
Length 512, alignments in bytes 0/ 0: 47.1406 414.859
Length 512, alignments in bytes 2/ 4: 2060.41 1012.72
Length 1024, alignments in bytes 0/ 0: 77.1094 824.594
Length 1024, alignments in bytes 1/ 6: 888 880.484
Length 16, alignments in bytes 1/ 2: 7.8125 62.2812
Length 16, alignments in bytes 2/ 1: 7.54688 17.4375
Length 16, alignments in bytes 1/ 1: 7.5 23.5625
Length 16, alignments in bytes 1/ 1: 7.57812 19.5
Length 32, alignments in bytes 2/ 4: 120.016 64.7344
Length 32, alignments in bytes 4/ 2: 13.7188 30.2812
Length 32, alignments in bytes 2/ 2: 13.75 30.4375
Length 32, alignments in bytes 2/ 2: 13.7344 30.5
Length 64, alignments in bytes 3/ 6: 124.75 100.906
Length 64, alignments in bytes 6/ 3: 31.25 54.5156
Length 64, alignments in bytes 3/ 3: 30.9688 56.875
Length 64, alignments in bytes 3/ 3: 30.9844 57
Length 128, alignments in bytes 4/ 0: 21.0156 104.125
Length 128, alignments in bytes 0/ 4: 27.2656 140.453
Length 128, alignments in bytes 4/ 4: 20.8594 107.922
Length 128, alignments in bytes 4/ 4: 21.0312 108.078
Length 256, alignments in bytes 5/ 2: 105.422 212.078
Length 256, alignments in bytes 2/ 5: 448.484 372.25
Length 256, alignments in bytes 5/ 5: 105.5 210.406
Length 256, alignments in bytes 5/ 5: 105.438 210.484
Length 512, alignments in bytes 6/ 4: 205.25 411.797
Length 512, alignments in bytes 4/ 6: 2063.81 977.328
Length 512, alignments in bytes 6/ 6: 204.5 415.312
Length 512, alignments in bytes 6/ 6: 204.578 415.562
Length 1024, alignments in bytes 7/ 6: 402.547 824.406
Length 1024, alignments in bytes 6/ 7: 402.766 3532.66
Length 1024, alignments in bytes 7/ 7: 402.844 824.75
Length 1024, alignments in bytes 7/ 7: 402.844 824.641
More information about the Libc-alpha
mailing list