Re: [libc/string] State of PAGE_COPY_FWD / PAGE_COPY_THRESHOLD

[Andrew Pinski <pinskia at gmail dot com>]

On Wed, Nov 9, 2016 at 11:52 PM, Maxim Kuvyrkov
<maxim.kuvyrkov@linaro.org> wrote:
>> On Nov 10, 2016, at 11:48 AM, Andrew Pinski <pinskia@gmail.com> wrote:
>>
>> On Wed, Nov 9, 2016 at 11:39 PM, Maxim Kuvyrkov
>> <maxim.kuvyrkov@linaro.org> wrote:
>>>> On Nov 1, 2016, at 5:59 PM, Adhemerval Zanella <adhemerval.zanella@linaro.org> wrote:
>>>>
>>> ...
>>>> $ cat sysdeps/x86/pagecopy.h
>>>>
>>>> #define PAGE_SIZE           4096
>>>> #define PAGE_COPY_THRESHOLD PAGE_SIZE
>>>>
>>>> #define PAGE_COPY_FWD(dstp, srcp, nbytes_left, nbytes)  /* Implement it */
>>>>
>>>> It should work on any other architecture as well.  Now the question
>>>> is whether this actually does make sense for Linux.  Hurd/mach provided
>>>> a syscall (?) to actually copy the pages (vm_copy) which seems to apply
>>>> some tricks to avoid full copy pages. By 'linux zero page sharing' are
>>>> you referring to KSM (Kernel Samepage Merging)?
>>>>
>>>> If so, on a system without a provided kernel interface to work directed
>>>> with underlying memory mapping (such as for mach), mem{cpy,set} will
>>>> actually need to touch the pages and it will be up to kernel page fault
>>>> mechanism to actually handle it (by identifying common pages and adjusting
>>>> vma mapping accordingly). And AFAIK this are only enabled on KSM if you
>>>> actually madavise the page explicit. So I am not grasping the need to
>>>> actually implement page copying on Linux.
>>>
>>> Linux kernel has a reserved page filled with zeroes, so it there /were/ a syscall to tell kernel to map N consecutive pages starting at address PTR to that zero page, we could use that in GLIBC for really big memset(0).
>>>
>>> A quick investigation shows that there is no such syscall provided by the Linux kernel.  Doesn't mean we can't ask for / implement one.
>>
>> And then there would be a COW interrupt on the first write.  Not a
>> good idea.  Since most likely you are writing zeros to a big page for
>> security reasons before filling it again with other data.
>
> I'm looking at this as a possible performance optimization for a well-known benchmark.

Please don't do it unless you benchmark real workloads.  Doing this
for a benchmark is not a good.  Please use something like WRF, mysql,
hadoop, spark or any other real workload that does lots of
memset/memcpy.  Please don't do this just for a well-known broken
benchmark.  Seriously this is just a broken benchmark anyways.

>
>>   That mean
>> each page would need to be copied which is normally slower than
>> zeroing in the first place.
>
> It may be like you say, or it may be a significant performance improvement.  I want to see numbers before deciding on how useful this may be.

Copying is always slower than doing setting zero.  There are
instructions on most major arch (including AARCH64) for zeroing a
cache line.  Copying means loading one cache line to L1 and then doing
stores.  Yes you can mark the cache line as not going to be used later
but that still means going to the cache.

Thanks,
Andrew

>
>>
>> COW is only useful when most of the pages will not be written to; it
>> is not useful when doing memcpy or memset.  Mainly because you don't
>> need to take the overhead of taking an interrupt twice (a system call
>> is still an interrupt).
>
>
> --
> Maxim Kuvyrkov
> www.linaro.org
>