Tests that require 'modprobe <module>'

[Mike Mason]

Stone, Joshua I wrote:
> Mike Mason wrote:
>>  I understand the desire to reduce the root requirement for users,
>> but for complete testing that may be unrealistic.
> 
> It's not unrealistic if you make it a goal.  Grep'ing for sudo in the
> testsuite right now, I see only two ways that sudo is used:

I agree with the goal, but it shouldn't be a higher priority than making 
sure every tapset is tested.

> 
> systemtap.printf/*b.exp -- calls 'sudo rm' to cleanup temp files.  This
> shouldn't be necessary, since the files are owned by the user running
> the test.
> 
> systemtap.context/context.exp -- builds and loads a test kernel module.
> Clever use of embedded-C in a stap-script could generate and load the
> test modules without needing an explicit insmod.  There's still an issue
> with resolving symbols for this test to work, but bug #2151 would help.
> 
>> If the code isn't in the kernel and isn't built as a module, the
>> tests for those probe points should fail.  I know, the test didn't
>> actually run and fail, we just failed to test.  To me, that's still
>> a failure.
> 
> But it's a failure of a different sort.  That's what skipped &
> unsupported result codes are for.

I agree, it's not the same type of failure.  I'm just concerned we don't 
track skipped and unsupported results and make sure they're tested at some 
point.

> 
>> As for whether the module's in use, in some cases that doesn't matter.
>> For example, you can still load the nfs, nfsd and sunrpc modules and
>> run build tests against them even though they're not being used.  This
>> may not be true for all modules.
> 
> I suppose I can accept that, as long as you can restore the system when
> you're done.  IIRC, some modules may trigger loading other modules
> though, so cleanup may be difficult.

Restoring the system is a good goal, but again I don't think it should be a 
higher priority than testing everything.

> 
>>>  Can you instead write your test to gracefully skip if the module
>>> isn't present?  (It shouldn't record a failure...)
>>  I don't think untested tapsets should be silently ignored.  If we
>> don't figure out how to test *everything*, even if it's hard, how
>> will we know when some change in the kernel breaks a tapset?  When a
>> systemtap user stumbles on the problem?  IMHO, that's unacceptable.
> 
> I don't mean to call it a 'pass' -- again, that's what skipped &
> unsupported are for.  It's important to distinguish between "I can't
> test that on this system" and "something is really broken".

The bottom line for me is how do we make sure every tapset can at least be 
built, including those that require that modules be loaded?  I realize 
triggering the probe points is impossible in some environments, but 
resolving the probe points and making sure the resulting code builds is 
doable.  I'm open to suggestions.

Mike

> 
> Josh