precompiled probing scenarios

[David Smith]

Frank Ch. Eigler wrote:
> Hi -
> 
> On Fri, Oct 06, 2006 at 03:33:19PM -0500, David Smith wrote:
>> [...]
>> Hmm.  Are we hashing the input script?  If so, how does this work with 
>> probe wildcards?  For example, let's say I probe "kernel.function("*")". 
>> We compile and cache this module.  I then plug in a bunch of 
>> additional hardware, which causes several extra modules to be loaded.  I 
>> then run stap again with the exact same input script.  [...]
> 
> kernel.function("*") should match exactly what was there before.
> Probes on module("*").FOO would be redefined to mean something like
> "all modules that we know at translation time that *might* exist, that
> also happen to be *loaded* at run time.  This aspect of wildcard
> expansion would thus take place at run time rather than translate
> time.  It just so happens that the same module might probe a greater
> or lesser number of modules on an actual system.  With that proviso,
> a script-source-level hash still seems to work.

Here's a patch that implements a cache for systemtap.  It works somewhat 
like ccache (which is unsurprising since I based it on ccache), but 
simpler.  (Note that if you want to apply the patch and compile it, 
you'll need to regenerate Makefile.in.)

Here's an outline of what happens.

By default, the cache is stored under ~/.stap_cache.  This can be 
overridden using the new 'SYSTEMTAP_CACHE' environment variable.

At the end of pass 2, a hash is generated (in find_hash(), which is in 
hash.cxx:70).  The hash is computed using the following data:
- kernel version and arch
- any user-specified macros
- runtime path
- gcc's path, size, and mtime
- stap's version and compile date
- pass 2 script output
The hash is then used to compute a hash pathname.

Still at the end of pass 2, we now check to see if we can find a cache 
hit (get_from_cache(), cache.cxx:42).  If the hash pathname exists, we 
copy it from the cache to the temporary directory.  We then skip to pass 
5 to run the module.

If a cache hit wasn't found, we proceed normally.  At the end of pass 4, 
we save the compiled module and its C file in the cache (add_to_cache(), 
cache.cxx:17).

By default, cache support is on.  To turn it off, you can either set 
SYSTEMTAP_CACHE to a non-directory ("SYSTEMTAP_CACHE=/dev/null stap ..") 
or use the '-m' option to change the module name (we must turn off cache 
support because we have to name the module 'stap_{hash}').  We could add 
a command line option to disable the cache if needed.

Note that currently several tests in the testsuite fail after a first 
run to seed the cache because they don't expect to see the skip from 
pass 2 to pass 5.

Stuff left to do:

- Testing, testing, testing
- Correct module handling (which Frank outlined above)
- Set a maximum cache size and expire old modules.  Is this needed?
- Documentation.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cache.patch
Type: text/x-patch
Size: 29719 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/systemtap/attachments/20061019/a0dfb8fe/attachment.bin>