[PATCH v7] malloc: Optimize small memory clearing for calloc

[Guo, Wangyang]

On 12/4/2024 8:16 AM, H.J. Lu wrote:

> On Wed, Dec 4, 2024 at 7:32 AM Paul Eggert<eggert@cs.ucla.edu> wrote:
>> On 2024-12-02 20:42, H.J. Lu wrote:
>>> +  /* Use repeated stores with 1 branch, instead of up to 3.  */
>>> +  *(d + 0) = 0;
>>> +  *(d + 1) = 0;
>>> +  *(d + 2) = 0;
>>> +  *(d + nclears - 2) = 0;
>>> +  *(d + nclears - 2 + 1) = 0;
>>> +  if (nclears > 5)
>>> +    {
>>> +      *(d + 3) = 0;
>>> +      *(d + 3 + 1) = 0;
>>> +      *(d + nclears - 4) = 0;
>>> +      *(d + nclears - 4 + 1) = 0;
>>> +    }
>> The following alternative code has no branches, when I compile with GCC
>> 14 -O2 on x86-64. Would that be a win?
>>
>>     /* Use repeated stores with no branches.  */
>>     int big = 3 * ((nclears + 1) >> 3);
>>     d[0] = 0;
>>     d[1] = 0;
>>     d[2] = 0;
>>     d[big] = 0;
>>     d[big + 1] = 0;
>>     d[nclears - big - 1] = 0;
>>     d[nclears - big] = 0;
>>     d[nclears - 2] = 0;
>>     d[nclears - 2 + 1] = 0;
>>
> Wangyang,  can you try this against the current master branch?
>
> Thanks.

I try to apply the alternative code with attached patch, but meet 
coredump error when running bench-calloc-thread:

Fatal glibc error: malloc.c:2601 (sysmalloc): assertion failed: (old_top 
== initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= 
MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize 
- 1)) == 0)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241204/2a1a7da9/attachment.htm>
-------------- next part --------------
diff --git a/sysdeps/generic/calloc-clear-memory.h b/sysdeps/generic/calloc-clear-memory.h
index 1f9d70d267..8099657913 100644
--- a/sysdeps/generic/calloc-clear-memory.h
+++ b/sysdeps/generic/calloc-clear-memory.h
@@ -31,19 +31,17 @@ clear_memory (INTERNAL_SIZE_T *d, unsigned long clearsize)
   if (nclears < 3)
     __builtin_unreachable ();
 
-  /* Use repeated stores with 1 branch, instead of up to 3.  */
-  *(d + 0) = 0;
-  *(d + 1) = 0;
-  *(d + 2) = 0;
-  *(d + nclears - 2) = 0;
-  *(d + nclears - 2 + 1) = 0;
-  if (nclears > 5)
-    {
-      *(d + 3) = 0;
-      *(d + 3 + 1) = 0;
-      *(d + nclears - 4) = 0;
-      *(d + nclears - 4 + 1) = 0;
-    }
+   /* Use repeated stores with no branches.  */
+   int big = 3 * ((nclears + 1) >> 3);
+   d[0] = 0;
+   d[1] = 0;
+   d[2] = 0;
+   d[big] = 0;
+   d[big + 1] = 0;
+   d[nclears - big - 1] = 0;
+   d[nclears - big] = 0;
+   d[nclears - 2] = 0;
+   d[nclears - 2 + 1] = 0;
 
   return d;
 }