[patch] Avoid the use of mmap as buffer for stdio streams

[Arjan van de Ven]

Hi,

currently glibc uses mmap() to allocate the (anonymous) buffers that 
back a stdio stream. This is non-optimal in several ways:
* mmap and munmap are very expensive operations in a threaded/multi
   cpu envinvironment: the kernel needs to take "big" locks and also
   needs to do a cross-cpu TLB invalidate on every munmap.
* even though the standard buffer is 1Kb, mmap() causes this to take
   4Kb of memory and TLB space; this is wasting 300% !
* there is no recycling of the memory going on on frequent
   fopen/fclose cycles, each cycle leads to the expensive system calls
   without any sharing; sharing is good for avoiding work,
   tlb pressure, cpu cache utilization etc.
* unlike malloc()/free(), mmap() and munmap() have very little sanity
   checking and bugs in the implementation or use get hidden most of
   the time but can lead to sporadic and hard to debug corruption or
   other weird effects [*]

This is caused by ALLOC_BUF and FREE_BUF to be hardcoded to 
mmap/munmap in libio/libioP.h in case the operating system has mmap; 
in the same file an malloc/free based implementation is available as 
well for the non-mmap case. I suspect this came from a time where mmap 
was thought to be much faster than malloc.... which isn't the case on 
nowadays glibc/Linux.

The attached patch just removes the mmap based implementation, and 
causes glibc to always use the malloc/free based implementation.

This has the advantage of solving all the downsides of mmap/munmap, 
but has 2 potential disadvantages of it's own:
1) it's now no longer a good idea to use fopen and friends inside the 
malloc() implementation (that doesn't seem to happen anyway)

[*] 2) stricter error checking in free() compared to malloc will 
uncover bugs. For example setvbuf() is known to cause a "junk" pointer 
to be passed to FREE_BUF(), with this patch this will cause a program 
abort, rather than the existing "call munmap() on a junk pointer and 
watch the kernel give -EINVAL" behavior.
( https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=217064 )

I feel that these potential disadvantages outweigh the advantages and 
so I'd like to ask for the patch to be applied (possibly after the 
setvbuf bug is fixed first)

Greetings,
    Arjan van de Ven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glibc-mmap.patch
Type: text/x-patch
Size: 1339 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20061124/74908aff/attachment.bin>