[PATCH] AArch64: Improve A64FX memcpy

[Wilco Dijkstra]

Hi Naohiro,

> What I meant by noise is any disturbance from other tasks. I believe
> that numactl command such as "numactl --membind=7 --physcpubind=48
> make bench" can minimize that.

I managed to reduce the variations a bit further by increasing the iteration
count 8x in addition to doing a few iterations before the timing starts.
You can run benchmarks separately as well, I used 
"time taskset -c N build/benchtests/bench-memset > out".

> I see. Patch V2 improves slightly than V1 from 1KB to 8KB, but
> degraded from 8KB to 64KB.

The 64KB case might need some retuning - previously it would enter
the L1_prefetch loop, do just one iteration before jumping to unroll32...
That doesn't make any sense. Either L1_SIZE may need to be larger or
the size where it falls back to the non-prefetch loop has to be different.

> But the master is still better from 1KB to 64KB.
> I'm just thinking if it is possible to merge only the code for less
> than 1024B and more than 64KB or not.
> Is it possible?

I benchmarked various options, here are the bench-memset results:

memset	v2     unroll32	newlast
all	 4.72%	 4.07%	 4.16%
0-1024	28.80%	23.48%	27.91%
1k-4k	-2.65%	-4.35%	-3.68%
4k-8k	-2.77%	-2.32%	-2.71%

These are overall speedup (from sum of reported times) for the
v2 patch, the v2 patch using the original unroll32, unroll8 and last
code, and finally the same but using the v2 code for the last 512
bytes.

Overall the new code gives a huge speedup for the smaller sizes,
while the larger cases become slightly slower. The old unroll32 code
makes the 1KB-4KB range worse, and while it is slightly faster than v2
in the 4KB-8KB range, it remains >2% slower than the original.

It's not clear why the larger cases become slower - a small overhead
should quickly become unnoticeable with larger sizes. However here
the slowdown seems constant, so it may be caused by something else.

> Theoretically DC ZVA should make data fill operation from memory to L2
> cache omit when store instruction is called.
> Therefor non zero data store should become slower without DC ZVA.
> Do you have any idea why Patch V2 is faster in the range more than 8MB
> of memset-large without calling DC ZVA?

I have no idea, it's possible a different ZVA distance might work out better.
However having lots of different loop variations creates more opportunities
for performance anomalies...

Cheers,
Wilco