Bug 11207 - Support preprocessor macros
Summary: Support preprocessor macros
Status: RESOLVED FIXED
Alias: None
Product: systemtap
Classification: Unclassified
Component: translator (show other bugs)
Version: unspecified
: P2 enhancement
Target Milestone: ---
Assignee: Serguei Makarov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-21 22:07 UTC by Josh Stone
Modified: 2012-08-16 15:34 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments
First version of proposal for just using m4 (809 bytes, text/plain)
2012-06-01 18:04 UTC, Serguei Makarov
Details
First version of custom syntax proposal (1.02 KB, text/plain)
2012-06-01 18:05 UTC, Serguei Makarov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Josh Stone 2010-01-21 22:07:43 UTC
It would be nice to be able to #define things in our scripts, both for brevity
and additional flexibility.  There are many ways we could go about this.

One way is to incorporate this into our existing preprocessing style.  It could
look something like:

  %( define LIBFOO = "/path/to/libfoo.so" %)
  %( define FOOFUNC(f) = process.library(LIBFOO).function(f) %)

  probe FOOFUNC("*") { ... }

Another option is to just run our scripts through an existing preprocessor.  CPP
is not an option because we don't want to expand embedded-C code blocks.  It
should work to use m4 though, and maybe there are others.

One advantage of rolling our own is that it should be easier to maintain
column/line info for error reporting.
Comment 1 Frank Ch. Eigler 2010-11-11 22:41:08 UTC
Another instance of this sort of need is to represent the 
casting/arithmetic magic used in the linux kernel list.h
macros, like 

 void
 inode_to_filename(struct inode *inode)
 {
  struct dentry *dentry = list_first_entry(inode->i_dentry,
        struct dentry, d_alias);
  printk("%.*s\n", dentry->d_name.len, dentry->d_name.name);
 }

where list_first_entry hides quite a bit of stuff that in
our language would expand to @casts etc.  With some more
const-propagation, some of it could be done with an ordinary
script function, but not everything.
Comment 2 Serguei Makarov 2012-06-01 18:03:47 UTC
Currently mulling over the two directions (m4, or a custom preprocessor). Still in progress, but by now it's worthwhile to post the current 'best' ideas as attachments.

The next step is probably to consider the more natural invocation syntax mentioned at the bottom of the custom proposal and to try to see what happens if I just start using m4 in a more adhoc (and potentially ugly) manner, rather than trying to design-specify-build some comprehensive define_probe_template macro.
Comment 3 Serguei Makarov 2012-06-01 18:04:18 UTC
Created attachment 6432 [details]
First version of proposal for just using m4
Comment 4 Serguei Makarov 2012-06-01 18:05:37 UTC
Created attachment 6433 [details]
First version of custom syntax proposal
Comment 5 Josh Stone 2012-06-01 22:55:10 UTC
I feel like both proposals are a bit too focused on your specific need, to simplify writing repetitive tapsets.  As far as language features go, plain old user scripts should be just as capable as tapset files.

For instance, a user could have a lot of @casts to type foo.  They should be able to define some FOO(ptr) as @cast(ptr, "struct foo", "/path/to/app:<sys/foo.h"), so they can simplify their accesses as FOO(p)->bar, FOO(p)->baz, etc.

(In reply to comment #3)
> First version of proposal for just using m4

So that's my primary objection with this approach - that it's *only* available for tapsets during systemtap's own build.  Also, even if postponed to runtime, using m4 (or any external preprocessor) won't let us track errors through macro levels.


(In reply to comment #4)
> First version of custom syntax proposal

I like the %foo shorthand, and parameterized %foo(p1, p2) as well, except for the ambiguity with the modulus operator.  Maybe we can just accept that though.  For example, an expression like "x%(y-1)" already fails for a similar reason, where the "%(" is misinterpreted as starting the preprocessor.  If we accept that, then I actually prefer this over the %( foo p1 p2 %) form.

I think you need to have a real delimiter between the parameters though, if only a comma, or else parameters will be limited to a single token, which is much less flexible.  We can deal with what's our comma vs. a comma within the parameter in the same way CPP does, which I think is just with parentheses.

Similarly, I don't care for the "\n multi-line-param_n" form, but I think it's also essentially solved by using CPP-style commas and parentheses.  Let me know if you think otherwise.

The docstring case is interesting, as usually those are in comments that our parser completely throws away, so your tapset-generating use case is adding a requirement I hadn't though of.  Since we generate tapset docs during the systemtap build, would these be bootstrapped through the stap executable?  Then there's cross compilation to worry about, so it may have to be a separate mini parser compiled for the host machine.
Comment 6 Serguei Makarov 2012-06-12 15:40:58 UTC
Please refer to thread "(PR11207) Macroprocessor discussion" on the systemtap mailing list for further progress on the proposal.
Comment 7 Serguei Makarov 2012-08-07 19:58:40 UTC
Committed and documented a basic macros facility in time for 2.0pre for rawhide.
Comment 8 Serguei Makarov 2012-08-16 15:34:13 UTC
The macro facility seems solid enough that any problems cropping up might be classified as new PRs. Hence, closing this one.