[PATCH] riscv: Skip vector code for zero length in RVV mem/str routines

Jin Ma jinma@linux.alibaba.com
Fri Nov 20 09:43:54 GMT 2026


Hi Anton,

> memcpy, memset, memcmp, memchr, memccpy, stpncpy and strncpy enter
> their vector loop without checking the length, so a zero length call
> executes vector loads and/or stores with VL=0. While these are no-ops
> architecturally, they are expensive on some implementations including
> Tenstorrent Ascalon.
>

Thanks for the patch. I strongly agree: an architectural no-op does not
mean zero execution cost.

> Return early when the length is zero, as memmove and strncmp already
> do. This resolves the performance issue on Ascalon, and a pass of the
> glibc bench tests shows no regressions.
>

> Signed-off-by: Anton Blanchard <antonb@tenstorrent.com>
> ---
>  sysdeps/riscv/rvv/memccpy.S | 2 ++
>  sysdeps/riscv/rvv/memchr.S  | 2 ++
>  sysdeps/riscv/rvv/memcmp.S  | 2 ++
>  sysdeps/riscv/rvv/memcpy.S  | 2 ++
>  sysdeps/riscv/rvv/memset.S  | 2 ++
>  sysdeps/riscv/rvv/stpncpy.S | 2 ++
>  6 files changed, 12 insertions(+)
>
> diff --git a/sysdeps/riscv/rvv/memccpy.S b/sysdeps/riscv/rvv/memccpy.S
> index 5854d463c2..0bea8400cc 100644
> --- a/sysdeps/riscv/rvv/memccpy.S
> +++ b/sysdeps/riscv/rvv/memccpy.S
> @@ -39,6 +39,7 @@
>  ENTRY (MEMCCPY)
>  .option push
>  .option arch, +v
> +    beqz num, L(not_found)
>      mv dst_ptr, dst
>  L(loop):
>      vsetvli zero, num, e8, ELEM_LMUL_SETTING, ta, ma
> @@ -54,6 +55,7 @@ L(loop):
>      add src, src, ivl
>      add dst_ptr, dst_ptr, ivl
>      bnez num, L(loop)
> +L(not_found):
>      mv a0, zero
>      ret
>  L(found):
> diff --git a/sysdeps/riscv/rvv/memchr.S b/sysdeps/riscv/rvv/memchr.S
> index 609b2f94c1..04fe348c7d 100644
> --- a/sysdeps/riscv/rvv/memchr.S
> +++ b/sysdeps/riscv/rvv/memchr.S
> @@ -39,6 +39,7 @@
>  ENTRY (MEMCHR)
>  .option push
>  .option arch, +v
> +    beqz num, L(not_found)
>  L(loop):
>      vsetvli zero, num, e8, ELEM_LMUL_SETTING, ta, ma
>
> @@ -56,6 +57,7 @@ L(loop):
>
>      bnez num, L(loop)
>
> +L(not_found):
>      li result, 0
>      ret
>  L(found):
> diff --git a/sysdeps/riscv/rvv/memcmp.S b/sysdeps/riscv/rvv/memcmp.S
> index 004bf538f0..20bfb4ef37 100644
> --- a/sysdeps/riscv/rvv/memcmp.S
> +++ b/sysdeps/riscv/rvv/memcmp.S
> @@ -42,6 +42,7 @@
>  ENTRY (MEMCMP)
>  .option push
>  .option arch, +v
> +    beqz num, L(equal)
>  L(loop):
>      vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma
>
> @@ -59,6 +60,7 @@ L(loop):
>
>      bnez num, L(loop)
>
> +L(equal):
>      li result, 0
>      ret
>  L(found):
> diff --git a/sysdeps/riscv/rvv/memcpy.S b/sysdeps/riscv/rvv/memcpy.S
> index b4d600c59d..dad7cb4a6c 100644
> --- a/sysdeps/riscv/rvv/memcpy.S
> +++ b/sysdeps/riscv/rvv/memcpy.S
> @@ -36,6 +36,7 @@
>  ENTRY (MEMCPY)
>  .option push
>  .option arch, +v
> +    beqz num, L(done)
>      mv dst_ptr, dst
>  L(loop):
>      vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma
> @@ -48,6 +49,7 @@ L(loop):
>
>      bnez num, L(loop)
>
> +L(done):
>      ret
>  .option pop
>  END (MEMCPY)
> diff --git a/sysdeps/riscv/rvv/memset.S b/sysdeps/riscv/rvv/memset.S
> index fef507b090..30994bbebc 100644
> --- a/sysdeps/riscv/rvv/memset.S
> +++ b/sysdeps/riscv/rvv/memset.S
> @@ -36,6 +36,7 @@
>  ENTRY (MEMSET)
>  .option push
>  .option arch, +v
> +    beqz num, L(done)
>      mv dst_ptr, dst
>
>      vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma
> @@ -47,6 +48,7 @@ L(loop):
>      vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma
>      bnez num, L(loop)
>
> +L(done):
>      ret
>  .option pop
>  END (MEMSET)

The memset loop still executes a vsetvli after num reaches zero,
setting VL to zero.

With VL=0, most vector instructions need no per-element execution.
Some high-performance processors may predict whether VL is zero
before its actual value is known and speculatively process subsequent
vector instructions accordingly. On processors using this mechanism,
frequent transitions between zero and nonzero VL could lead to
misspeculation with substantial performance penalties.

Could we move vsetvli to the loop head to avoid setting VL=0 on exit?
For example, after the entry check for zero length:

    ptr = dst;
    vl = setvl(num);
    vdata = broadcast(value, vl);
    do {
        vl = setvl(num);
        vector_store(ptr, vdata, vl);
        ptr += vl;
        num -= vl;
    } while (num != 0);

Does Ascalon also incur this kind of overhead, making this layout worth
trying?

BR,
Jin

> diff --git a/sysdeps/riscv/rvv/stpncpy.S b/sysdeps/riscv/rvv/stpncpy.S
> index 71bb46b04e..b038149728 100644
> --- a/sysdeps/riscv/rvv/stpncpy.S
> +++ b/sysdeps/riscv/rvv/stpncpy.S
> @@ -55,6 +55,7 @@ weak_alias (__stpncpy, stpncpy)
>  ENTRY (STPNCPY)
>  .option push
>  .option arch, +v
> +    beqz length, L(done)
>      mv dst_ptr, dst
>      /* Copy src to dst_ptr.  */
>  L(stpcpy_loop):
> @@ -73,6 +74,7 @@ L(stpcpy_loop):
>  #ifndef USE_AS_STRNCPY
>      mv dst, dst_ptr
>  #endif
> +L(done):
>      ret
>
>      /* Fill the tail zero.  */



More information about the Libc-alpha mailing list