答复: [PATCH] x86: Enable non-temporal memset without ERMS
Feifei Wang
wangfeifei@hygon.cn
Tue Jul 9 02:57:27 GMT 2024
Thanks for your kindly reply.
> -----邮件原件-----
> 发件人: Noah Goldstein <goldstein.w.n@gmail.com>
> 发送时间: 2024年7月8日 13:29
> 收件人: Feifei Wang <wangfeifei@hygon.cn>
> 抄送: libc-alpha@sourceware.org; Jing Li <lijing@hygon.cn>
> 主题: Re: [PATCH] x86: Enable non-temporal memset without ERMS
>
> On Fri, Jul 5, 2024 at 3:53 PM Feifei Wang <wangfeifei@hygon.cn> wrote:
> >
> > Currently, large memset perf can be improved with non-temporal stores[1].
> > But it just supports the case in which ERMS feature is enabled.
> >
> > For some architectures, ERMS is disabled in cpu feature as default For
> > example, AMD zen3-milan[2], Or ERMS performance is worse than
> > vectorized loop, and it is disabled.
> > For example, HygonGenuine arch.
> >
> > Thus, to let these arch achieve perf improvement from non-temporal
> > memset, and enable non-temporal memset without ERMS, a 'non-temporal
> > memset branch jump' is added in 'more_2x_vec'case.
> >
> > Test Results:
> > thread: 1
> > memset store value: 0
> > function: memset_avx2_unaligned
> >
> > hygon1 arch
> > x86_memset_non_temporal_threshold = 8MB
> > size new performance / old performance
> > 128 byte(2x -4x vec case) 1
> > 256 byte(4x - 8x vec case) 1
> > 512 byte( > 8x loop case) 1
> > 1MB 0.994
> > 4MB 0.996
> > 8MB 0.670
> > 16MB 0.343
> > 32MB 0.355
> >
> > hygon2 arch
> > x86_memset_non_temporal_threshold = 8MB
> > size new performance / old performance
> > 128 byte(2x -4x vec case) 1
> > 256 byte(4x - 8x vec case) 0.653
> > 512 byte( > 8x loop case) 0.713
> > 1MB 1
> > 4MB 0.887
> > 8MB 1.312
> > 16MB 0.822
> > 32MB 0.830
> >
> > hygon3 arch
> > x86_memset_non_temporal_threshold = 8MB
> > size new performance / old performance
> > 128 byte(2x -4x vec case) 1
> > 256 byte(4x - 8x vec case) 1
> > 512 byte( > 8x loop case) 1
> > 1MB 1
> > 4MB 0.990
> > 8MB 0.737
> > 16MB 0.390
> > 32MB 0.401
> >
> > For hygon arch with , no performance degradation on '2x - 8x branch
> > case' when extra branch jump added. And with this patch, non-temporal
> > stores can improve performance by 20% - 65%.
> >
> > amd-zen2-rome arch
> > x86_memset_non_temporal_threshold = 12MB
> > size new performance / old performance
> > 128 byte(2x -4x vec case) 0.986
> > 256 byte(4x - 8x vec case) 1.057
> > 512 byte( > 8x loop case) 1.005
> > 1MB 0.999
> > 4MB 0.998
> > 8MB 1.014
> > 16MB 1.156
> > 32MB 0.634
> >
> > For amd-zen2-rome, no performance degradation on '2x - 8x branch case'
> > when extra branch jump added. When size >=
> > x86_memset_non_temporal_threshold, for 'size = 16MB' case, performance
> > degradation is due to that amd-zen2 non_temporal perf is worse than
> > temporal[3]. And in 32MB case, it can achieve performance improvement by
> 37%.
> >
> > For intel arch, intel arch ERMS feature is enabled as default, so this
> > patch has no effect for it.
> >
> > Note:
> > Chengdu Haiguang IC Design Co., Ltd (Hygon) aims at providing high
> > performance x86 processor for China server market.
> >
> > Ref:
> > [1]
> > https://sourceware.org/git/?p=glibc.git;a=commit;h=5bf0ab80573d66e4ae5
> > d94b094659094336da90f [2]
> > https://lists.gnu.org/archive/html/qemu-devel/2022-01/msg06747.html
> > [3]
> >
> https://docs.google.com/spreadsheets/d/1opzukzvum4n6-RUVHTGddV6RjAEil4
> > P2uMjjQGLbLcU/edit?gid=120165109#gid=120165109
> >
> > Jira: HES-210
> > Signed-off-by: Feifei Wang <wangfeifei@hygon.cn>
> > Reviewed-by: Jing Li <lijing@hygon.cn>
> > Change-Id: I5f9b032c848f4e9d2e54a08138af6b9ae71e1156
> > ---
> > sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S | 10 +++++++++-
> > 1 file changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S
> > b/sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S
> > index 88bf08e..063dd51 100644
> > --- a/sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S
> > +++ b/sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S
> > @@ -229,9 +229,17 @@ L(stosb_more_2x_vec):
> > cmp __x86_rep_stosb_threshold(%rip), %RDX_LP
> > ja L(stosb_local)
> > #endif
> > - /* Fallthrough goes to L(loop_4x_vec). Tests for memset (2x, 4x]
> > + /* If rdx is less than __x86_memset_non_temporal_threshold,
> > + fallthrough goes to L(loop_4x_vec). Tests for memset (2x,
> > + 4x]
> > and (4x, 8x] jump to target. */
> > L(more_2x_vec):
> > +#if defined USE_MULTIARCH && IS_IN (libc)
> > + /* Check non-temporal store threshold.
> > + This is for the hardware which can not support ERMS feature or
> > + ERMS fearure is disabled. */
> > + cmp
> __x86_memset_non_temporal_threshold(%rip), %RDX_LP
> > + jae L(nt_memset)
> > +#endif
> > /* Store next 2x vec regardless. */
> > VMOVU %VMM(0), (%rdi)
> > VMOVU %VMM(0), (VEC_SIZE * 1)(%rdi)
> > --
> > 2.7.4
> >
>
> I am opposed to this patch getting in as is. I strongly dislike the essentially
> redundant checks for `__x86_rep_stosb_threshold` and
> `__x86_memset_non_temporal_threshold` that will occur on most processors.
> Further, in this case, it messes up the alignment of the loop.
>
> What I would like to do instead for processors which prefer non-temporal but
> not erms, is to have them set `__x86_rep_stosb_threshold ==
> __x86_memset_non_temporal_threshold` and prefer the `erms` version.
>
> Can you implement it like that instead?
[Feifei] This is also a good choice. But I have a question for this:
If my understand is right, you means that If __x86_rep_stosb_threshold ==
__x86_memset_non_temporal_threshold, we can set "Prefer_ERMS" when
init_cpu_features for servers without ERMS.
However, if like this, I think we ignore the small size case. Though we can use non temporal now,
But for "size < __x86_rep_stosb_threshold" case, it is forced to use erms whose performance is very bad
For example, hygon and amd zen2.
Thus we add a nt jump for temporal branch, and for this, we can support that :
1. small size can use temporal for good performance rather than erms.
2.big size can use non temporal to achieve better performance.
Best Regards
Feifei
More information about the Libc-alpha
mailing list