There exist uses for procfs("file").write and especially .read, for which it is desirable for stap scripts to produce much more than MAXSTRINGLEN bytes of output. The probe point syntax could be extended to specify a .maxsize(NNN) option for the probes. Then, extra syntax of some sort would be needed to write/read the $value variable. Perhaps something like this for the output side: probe procfs("file").read.maxbuf(8192) { $value .= sprint("hello\n") $value .= sprint("more and more test\n") } (i.e., we'd have to special-case/optimize the concatenate-assign operator.) I don't have a simple idea about new syntax to parse extra-long incoming $value strings. Maybe a special substr that takes string-references instead of values.?
This is an invaluable feature for getting various statistics (e.g. @hist_log) on demand. Maybe initially it would be easier to introduce new function (procfs_print?) appending provided text to output buffer? Increasing default buffer size can be over the top, but for users who really need it (instead of writing .maxbuf in almost all procfs probes), we can also introduce some define (PROCFS_BUFSIZE?) that could be altered via -D.
Some additional info: One complication in producing bigger procfs input/output is that the maximum buffer size the kernel will input/output in one call to the procfs read_proc()/write_proc() is PAGE_SIZE bytes long. (This is why read_proc() gets an 'offset' parameter.) Some kernel procfs file handling code now uses the 'seq' file interface to help simplify large procfs file handling. Systemtap may need to consider this.
(In reply to comment #0) > There exist uses for procfs("file").write and especially .read, > for which it is desirable for stap scripts to produce much more than > MAXSTRINGLEN bytes of output. I can see the value for .read to produce more than MAXSTRINGLEN bytes of output, but I'm unsure of the value of .write to be able to read more than MAXSTRINGLEN bytes of input. Assuming we could read more than MAXSTRINGLEN bytes of input, what could we then do with that data? We can't assign (all of it) to another string variable. We could print it (which might fail if the the number of bytes is greater than STP_BUFFER_SIZE). I'm unsure of anything else we could do with it.
I've done some testing on bigger procfs input/output with the current code, trying to see what happens when MAXSTRINGLEN is changed. For the .read tests, I output a string MAXSTRINGLEN bytes long. For the .write tests, I fed a test file with > 8k worth of data into the procfs file. Here's what I found. MAXSTRINGLEN = 512 (the default) .read = OK .write = bug - probe gets called multiple times, in MAXSTRINGLEN chunks, but the last chunk is a mix of old and new data MAXSTRINGLEN = 4096 .read = bug, only seeing ~3000 chars? .write = bug, probe gets called multiple times in MAXSTRINGLEN chunks, but the last chunk is a mix of old and new data MAXSTRINGLEN = 8192 .read = bug, only seeing 1st 4k of data .write = bug, probe gets called multiple times in MAXSTRINGLEN chunks, but the last chunk is a mix of old and new data
Bug #11223 was added to cover the procfs write problem mentioned in comment 4.
(In reply to comment #4) > I've done some testing on bigger procfs input/output with the current code, > trying to see what happens when MAXSTRINGLEN is changed. For the .read tests, I > output a string MAXSTRINGLEN bytes long. For the .write tests, I fed a test file > with > 8k worth of data into the procfs file. > > Here's what I found... With the fixes to bug #11223 and bug #11078, all the problems mentioned in comment #4 are fixed. Input/output of MAXSTRINGLEN sizes of 512, 4096, and 8192 work correctly.
To build up bigger procfs output, I had assumed we'd use the string concatenation operator '.='. Assume the following script. ==== global large_string = "abcdefg" probe procfs("large_string").read { $value = large_string $value .= large_string } ==== Unfortunately, this doesn't work: # stap -v ../procfs_read3.stp Pass 1: parsed user script and 65 library script(s) using 83492virt/20220res/2260shr kb, in 120usr/40sys/164real ms. semantic error: Operator-assign expressions on target variables not implemented: operator '.=' at ../procfs_read3.stp:4:14 while resolving probe point procfs("large_string").read source: $value .= large_string ^ Pass 2: analyzed script: 0 probe(s), 1 function(s), 0 embed(s), 1 global(s) using 83888virt/20864res/2488shr kb, in 0usr/0sys/6real ms. Pass 2: analysis failed. Try again with another '--vp 01' option. From looking at var_expanding_visitor::visit_assignment() in tapsets.cxx, this is explicitly denied, with the following comment: // FIXME: for the time being, we only support plan $foo = bar, // not += or any other op= variant. This is fixable, but a bit // ugly. It might be time to fix this problem.
(In reply to comment #7) > To build up bigger procfs output, I had assumed we'd use the string > concatenation operator '.='. > [...] > From looking at var_expanding_visitor::visit_assignment() in tapsets.cxx, this > is explicitly denied, with the following comment: > > // FIXME: for the time being, we only support plan $foo = bar, > // not += or any other op= variant. This is fixable, but a bit > // ugly. > > It might be time to fix this problem. Well, the most general way I would fix that is to transform into an explicit read-modify-write, which doesn't help if these procfs strings are oversized. But in this specific case, I think we can just specialize visit_assignment() in procfs_var_expanding_visitor, mapping ".=" to a new _procfs_value_append() instead of _procfs_value_set().
(In reply to comment #8) > But in this specific case, I think we can just specialize visit_assignment() in > procfs_var_expanding_visitor, mapping ".=" to a new _procfs_value_append() > instead of _procfs_value_set(). Exactly right.
I've implemented the '.=' operator for procfs read probes in commit a50de93. Being able to increase the buffer size is still to come.
Fixed in commit 3897525. By default, procfs read probe buffers are STP_PROCFS_BUFSIZE bytes in size. (STP_PROCFS_BUFSIZE defaults to MAXSTRINGLEN.) Using the new '.maxsize(NNN)' parameter, STP_PROCFS_BUFSIZE can be overridden on a per-file basis.