Using systemtap to validate manual code inspection.

[Carlos O'Donell]

On 05/07/2013 05:14 AM, Siddhesh Poyarekar wrote:
> On Mon, May 06, 2013 at 03:41:10AM -0400, Carlos O'Donell wrote:
>> I recently had to do some manual code inspection
>> for a bug which has no reproducer (patch forthcoming).
>>
>> The manual inspection found real bugs, but I wanted
>> to get more coverage for the failure scenarios.
>>
>> I wrote up my experience using systemtap to inject
>> failures into glibc:
>> http://sourceware.org/glibc/wiki/Testing/WhiteBox
>>
>> Looking forward to white box and unit tests ;-)
> 
> This is really neat, but the problem here is not limited to systemtap
> needing root privileges.  The more subtle problem is the dependency of
> these tests on line numbers within code.  The tests will have to be
> continually updated as the code gets updated and that may blow out of
> proportion as the number of such tests per source file grows.
> 
> It would be nicer if it were possible to add annotations within source
> that systemtap picks up and set probes at those points.  I'm not
> talking about probe points, since they're additional code within the
> library and not just source level annotations.  Something like this:
> 
> some_func (int *in)
> {
>   /** SYSPROBE:probe=simulate_crash;file=simulate_crash.stp */
>   glob = *in;
> }
> 
> The problem with doing this is that it could quickly clutter up code
> with white box testing annotations.

The implementation details of adding white box testing
to glibc is certainly a complex and difficult topic that
will require some designing :-)

Line numbers are certainly a pain to update.

We really want an automated way to generate the .stp
file from an annotation and have that become a whitebox
test associated with a particular test.

e.g.

* Annotate source.
* Auto-generate .stp files from source.
* Run test-stpN with .stp file active.
* Observe PASS/FAIL.

Cheers,
Carlos.