Re: offline elfutils processing committed

[Martin Hunt <hunt at redhat dot com>]

On Fri, 2006-11-03 at 18:43 -0500, Frank Ch. Eigler wrote:

> 
> Yup, now done, at least an easy way (emitting an inline query every
> time the relocatable $target variable is accessed).
> 
> It however exposes two problems with the newest symbol table code.
> First, the new _stp_module_lookup function holds a spinlock on its
> data structure, which could conceivably block.  

Is there some reason why the $target variable must be looked up each
time it is accessed?  Sounds slow.

I put the spinlock in at the last minute because I was worried that
someone might be tempted to use it to try to resolve addresses during a
kprobe. I don't think it can deadlock.


> The second problem is memory allocation.  Like ordinary associative
> arrays, the new symbol table also appears to be allocated using what
> comes down to kmalloc.  Unfortunately, with large number of entries of
> either kind, this allocation can fail, or trigger OOM handling.
> 
> The runtime needs to learn to either rely more (perhaps exclusively)
> on data segment allocation (.data or .bss arrays), or needs to make
> sure that it fails in a clean way if kernel free memory happens to be
> short.  This is essential.  (To see why, try running a script with
> -DMAXMAPENTRIES=999999.)

Moving the allocations into .data or .bss sections could still trigger
OOM situations.  You just simplify things some be having one large
allocation. I made this argument a year ago and was told to rewrite to
allocate memory in small chunks with kmalloc.  Now we are back to large
chunks with vmalloc?  I'm fine with that, but let's be clear we agree on
why we are doing this.

Regarding OOM killer, shouldn't we just set oom_adj to something like
+15?  That would put us at the top of the victims list so avoid the
situation where loading a systemtap module puts the system in a low
memory state which causes the oom killer to start ramdomly killing off
processes.

Martin