Question about madvise(DONTNEED) in glibc malloc

[Ondřej Bílka]

On Sun, Apr 14, 2013 at 10:13:23PM -0700, KOSAKI Motohiro wrote:
> >>>       MADV_DONTNEED
> >>>              Do  not  expect  access  in the near future.  (For the time
> >>>              being, the application is finished with the given range, so
> >>>              the  kernel can free resources associated with it.)  Subse-
> >>>              quent accesses of pages in this  range  will  succeed,  but
> >>>              will  result  either  in  re-loading of the memory contents
> >>>              from the underlying mapped file (see mmap(2)) or zero-fill-
> >>>              on-demand pages for mappings without an underlying file.
> >>
> >> Current implemantation cleary does as document. It's not a bug and we don't plan to
> >> change never.
> > 
> > I didn't imply that it's any kind of bug in the kernel either.
> > 
> > Why can't we extend the TRIM_THRESHOLD to non-main arenas?  We
> > sacrifice the extra address space we could have given back to the
> > kernel by not getting rid of a heap when we could, but that's a much
> > better compromise IMO.
> 
> Yeah, I haven't consider it. but it can, I think.
> 
> 
> > I agree with you when you say that the pages we just gave back with
> > MADV_DONTNEED are the ones most likely to be needed when we decide to
> > extend the heap next and in that context, it is a suboptimal use of
> > MADV_DONTNEED.  However, the idea here is to release resources
> > periodically to avoid monopolizing it.  Maybe we could always only
> > trim half of TRIM_THRESHOLD and keep the other half as a cache.  Given
> > the semantics of dynamic thresholds, we would effectively be keeping a
> > cache for the next expected allocation size - the TRIM_THRESHOLD is
> > dynamically set to twice of MMAP_THRESHOLD on freeing of a mapped
> > location.
> 
> That's definitely better than current. but I'm incline to solve the issue
> completely by introducing new kernel feature.
> 

What having at kernel keep list of pages that are currently unused. On memory
shortage kernel reclaims them. When minor page fault occurs we do not
need to zero page but remap one of unused.