relayfs changes needed

[Martin Hunt <hunt at redhat dot com>]

We need some relayfs-related changes in the transport and stpd. I know
there has been some discussion on this, but I'm not clear on what needs
to be done. 

Currently, we print to buffers which are then copied to the relayfs
buffers. At the start of every print buffer is a sequence number. stpd
reads the relayfs buffers, and simply dumps them to files. Later the
files are optionally merged by reading the sequence numbers.
Unfortunately, the data format is simply 4-bytes sequence numbers
followed by null-terminated data. This does not work with binary data.

So changes are needed. But before we decide what to change, can we
review how and why relayfs is used?  On typical desktop machines, there
seems to be no reason to use relayfs. On larger systems, especially NUMA
systems, I expect that relayfs will have better performance. For most
scripts this won't matter, but in the case of more complex kernel
tracing packages built upon systemtap, the improved performance and
per-cpu files will both be useful. And in the cases, I expect the output
will be carefully formatted binary.

So if this is the case, I'm wondering if maybe the right thing to do is
to remove all sequence numbers, formatting, and merging in the transport
layer for relayfs and simply let it run full-speed dumping data into
per-cpu files. This gives the highest performance and moves the
responsibility for any merging or other postprocessing to the
application layer, where it can be done most efficiently. So some script
(or program) will invoke systemtap, and on exit, take the binary data
files and process them. Systemtap will not timestamp, sequence, or
otherwise process the data. 

Removing some of the complexity from the transport will greatly simplify
it, allowing us to speed it up further.

We probably need a way to specify the base names for the per-cpu files.
Maybe changing the meaning of "stap -o" for relayfs.