Currently all command-line parameters to scripts are substituted at parse-time. This prevents us from using the cached code when the parameter value changes. In many cases we could easily do a "lazy" assignment that doesn't happen until runtime. Consider the following simple script that watches for creation of a particular process name: probe process.exec_complete { if (execname() == @1) printf("start %d\n", pid()) } The "@1" parameter might change between invocations, but it should be possible to the same cached version of the script. But since the parser substitutes the value in right away, the hash for the script is different for each parameter. One suggestion is to synthesize global variables for the command-line arguments, and initialize them at runtime through module parameters. These can be read-only globals, since parameters aren't valid lvalues, so no locks are needed. Some analysis will be needed to determine when this optimization is possible, as some cases need to be resolved at translation time -- e.g., module(@1).function(@2), kernel.statement($3), timer.sec($4). (Actually, some like timers might still be able to get delayed values, but we can consider that a future enhancement.)
There is good reason for expanding some arguments at parse time - this is necessary for example for parameters of probe points. We already support a run-time parametrization method to some extent: initializing global variables by name, which is implemented via module parameters. Do you still believe a third method (positional arguments at run time) is necessary? See also bug #2700.
(In reply to comment #1) > There is good reason for expanding some arguments at parse time - this is > necessary for example for parameters of probe points. Right -- that's why I said "Some analysis may be needed". For parameters that are used in probe points we should expand them right away. For parameters that are only used as literals in the probe bodies, we can delay assignment until runtime. > We already support a run-time parametrization method to some extent: > initializing global variables by name, which is implemented via module > parameters. Do you still believe a third method (positional arguments > at run time) is necessary? Well, there's no way to pass in module parameters through stap. To do my simple example in that way, you have to do something like this: matchname.stp: global name probe process.exec_complete { if (execname() == name) printf("start %d\n", pid()) } $ stap matchname.stp -p4 -k Keeping temporary directory "/tmp/stap5F1ijy" $ mv /tmp/stap5F1ijy/*.ko matchname.ko $ sudo staprun matchname.ko name=ls $ sudo staprun matchname.ko name=bash $ sudo staprun matchname.ko name=foobar Thus, you have to manually cache the ko and invoke staprun directly. What I'm proposing is not a new method, just to let the above process happen behind the scenes, so I can just do this: matchname.stp: probe process.exec_complete { if (execname() == @1) printf("start %d\n", pid()) } $ stap matchname.stp ls $ stap matchname.stp bash $ stap matchname.stp foobar With a delayed assignment through module parameters, it will used a cached ko. This happens transparently, so the same script will still work on older staps that don't do this.
(In reply to comment #2) > Right -- that's why I said "Some analysis may be needed". For parameters that > are used in probe points we should expand them right away. For parameters that > are only used as literals in the probe bodies, we can delay assignment until > runtime. Would this not be confusing to the user? Would you signal an error if a @1/$1 was used in a compile-time context (probe points) once, then the user tried to rerun the script with different parameters? Or just slow it down (by disabling the cache)? What if in the future, the translator does more optimizations with probe handlers (constant propagation of $1/@1 variables), which would make those similarly uncacheable? > > We already support a run-time parametrization method to some extent: > > initializing global variables by name, [...] > > Well, there's no way to pass in module parameters through stap. That is just a SMOP and should be added there for passing through to staprun. What you propose is probably implementable, but it may be too complicated to do and to explain.
(In reply to comment #3) > Would this not be confusing to the user? Would you signal an error if a > @1/$1 was used in a compile-time context (probe points) once, then the user > tried to rerun the script with different parameters? Or just slow it down > (by disabling the cache)? This should be transparent to the user -- much like any other optimization that a compiler does. Most internals of compilation & optimization are confusing to general users, but that doesn't mean we don't do it. "Slow it down" I think is the wrong view -- rather I think we are speeding it up by taking extra advantage of the cache when possible, and in other cases it works simply as before. Consider this script: probe timer.ms($1) { if (execname() == @2) printf("%d\n", pid()) } With these arguments on different runs: 1. 500 foo 2. 500 bar 3. 100 foo 4. 100 bar Currently, this will produce four different cached versions of the script -- slow, yes? With this proposal, 1 & 2 would share the same cache, and 3 & 4. The user doesn't need to do anything differently to get this new advantage. I've been thinking about how this might be implemented, and it seems like we can get away with simply expanding variables where necessary, and synthesizing globals for the others. Thus the pass-2 output for inputs 1 & 2 would be something like: global $str_arg2:string probe timer.ms(500) { if (execname() == $str_arg2) printf("%d\n", pid()) } I'm using '$' here just to indicate that this is a special global -- it's really read-only, so the translator doesn't need to emit locks. Since the caching hash depends on the pass-2 output, the cache will only depend on the first argument. > What if in the future, the translator does more optimizations with probe > handlers (constant propagation of $1/@1 variables), which would make those > similarly uncacheable? Constant propagation in particular can turn into copy propagation. But you're right, others like constant folding are in conflict with this delayed assignment. That's an optimization tradeoff decision we'll have to make -- whether the gains in startup time outweigh the gains in probe handler speed. My opinion is that scripts with arguments are meant to be called often with different arguments, and thus mitigating the startup cost would be highly beneficial. > > Well, there's no way to pass in module parameters through stap. > > That is just a SMOP and should be added there for passing through to staprun. Yes, we should provide that option, and that would allow people to manually recreate the benefits I'm proposing. Add in another SMOP to define read-only globals in the language, and you've got it all. But then, most optimizations *could* be done manually. What I propose is a simple way for users to get this benefit automatically. > What you propose is probably implementable, but it may be too complicated > to do and to explain. Some aspects of our implementation might be tricky, but the only impact to the user is that they might notice some of their scripts starting faster.
With the addition of "stap -G var=value", this issue is moot.