what if I wanted to write some tests...

[Vara Prasad]

Looking at this experience of Brad i am thinking (a loud) may be it is 
time for us to do regular integration testing .

A simple system using the test bed to do a periodic build on UP/SMP 
x86/x86_64 platforms for now,  from the CVS and then probably run few 
basic tests to see if they pass. Once these two steps are passed, create 
a tar ball of the extracted code and have an automatic link to that tar 
ball on the website.

This brings up following issues that we have to address

1) How often we check out the code and do the build?
My guess, considering we are not doing way too many changes at this time 
probably once a week may be on Tuesday so that before Thursday we will 
have issues observed to discuss on the call if need be. Of course if  
someone checks a major change it would be nice to initiate this process 
on demand as well.

2) Where is the test bed?
I thought Will is looking into this using the RH test grid (Will correct 
me if i am wrong). If this is correct is that the right test bed to do 
this kind of build and sanity test? If this is not which other place we 
can use? The other one that comes to my mind is the test suit Rusty 
mentioned that he is planning to put together. Is there a test bed in 
Intel that can pull the code and push the tar ball after successful 
tests? Will there be any firewall issues in accessing and pushing the 
tar ball to the website/CVS?

3) How about other architectures mainly PPC 64 and IA 64?

Regarding PPC 64 i am thinking Ananth has access to PPC 64 machines that 
he can access without firewall issues, so that might work . I don't know 
if there are any IA64 machines available within the RH firewall if not 
how Intel can can help here.

4) Do we have a sanity test suit for us to declare a good build?

For now i am thinking the examples in the CVS could be a good start. If 
we have the kprobes tests Rusty/Anil was putting together that would be 
good too.

Any other issues i am missing?

Please let me know your thoughts on this proposal.

Thanks,
Vara Prasad

Chen, Brad wrote:

>>Of course - you can check out the translator and/or runtime sources.
>>The former has various self-tests ("make check"); the latter has
>>various kprobes-flavoured test programs.
>>    
>>
>
>Indeed; I had tried to work from runtime/tests but 
>get a bunch of syntax errors. I was hoping somebody
>could confirm that this was supposed to work, and
>take a guess at what I might have wrong. I just did 
>a "recreate_links" from ../user; doesn't help. See
>test output below.
>
>It seems I am able to ./build in runtime/probes,
>so perhaps I'll work there for a while and try 
>runtime/tests later.
>
>I don't want to complain but, in trying to run the
>where_func probe:
>- name of .ko has changed
>- nothing to indicate you should use the ./stp script
>  in the directory
>- ./stp lsmod assumes /sbin in path
>and I still haven't gotten it running yet. So if anybody
>has hints let me know, and please let me know if you own
>this directory and wouldn't mind me me updating things a 
>bit when I get them figured out.
>
>Brad
>
>========================================================================
>=
>
>tclsh all.tcl
>Running all SystemTap tests
>Tests running in interp:  /usr/bin/tclsh
>Tests located in:  /home/bchen/proj/systemtap/cvs/src/runtime/tests
>Tests running in:  /home/bchen/proj/systemtap/cvs/src/runtime/tests
>Temporary files stored in
>/home/bchen/proj/systemtap/cvs/src/runtime/tests
>Test files run in separate interpreters
>Running tests that match:  *
>Skipping test files that match:  l.*.test
>Only running test files that match:  *.test
>Tests began at Mon Jun 20 01:34:08 PM PDT 2005
>
>Tests ended at Mon Jun 20 01:34:08 PM PDT 2005
>all.tcl:        Total   0       Passed  0       Skipped 0       Failed
>0
>Sourced 0 Test Files.
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>maps test began at Mon Jun 20 01:34:08 PM PDT 2005
>
>Tests running in interp:  /usr/bin/tclsh
>Tests located in:  /home/bchen/proj/systemtap/cvs/src/runtime/tests/maps
>Tests running in:  /home/bchen/proj/systemtap/cvs/src/runtime/tests
>Temporary files stored in
>/home/bchen/proj/systemtap/cvs/src/runtime/tests
>Test files run in separate interpreters
>Running tests that match:  *
>Skipping test files that match:  l.*.test
>Only running test files that match:  *.test
>Tests began at Mon Jun 20 01:34:08 PM PDT 2005
>map.test
>
>==== ii Test of int64 keys and int64 values FAILED
>==== Contents of test case:
>
>        exec ./test
>
>---- Test setup failed:
>In file included from ../../user/runtime.h:36,
>                 from ii.c:1:
>../../user/emul.h:29: error: redefinition of '__alloc_percpu'
>/lib/modules/2.6.12-rc6-mm1/build/include/linux/percpu.h:44: error:
>previous def
>inition of '__alloc_percpu' was here
>../../user/emul.h: In function `__alloc_percpu':
>../../user/emul.h:31: error: dereferencing pointer to incomplete type
>../../user/emul.h:37: error: dereferencing pointer to incomplete type
>../../user/emul.h:39: error: dereferencing pointer to incomplete type
>../../user/emul.h:42: error: dereferencing pointer to incomplete type
>../../user/emul.h:42: error: dereferencing pointer to incomplete type
>../../user/emul.h:42: error: dereferencing pointer to incomplete type
>../../user/emul.h:42: error: dereferencing pointer to incomplete type
>../../user/emul.h: At top level:
>../../user/emul.h:51: error: redefinition of 'free_percpu'
>/lib/modules/2.6.12-rc6-mm1/build/include/linux/percpu.h:51: error:
>previous def
>inition of 'free_percpu' was here
>../../user/emul.h: In function `free_percpu':
>../../user/emul.h:56: error: dereferencing pointer to incomplete type
>==== ii FAILED
>
>
>==== is Test of int64 keys and string values FAILED
>==== Contents of test case:
>
>-----Original Message-----
>From: fche@redhat.com [mailto:fche@redhat.com] 
>Sent: Monday, June 20, 2005 12:46 PM
>To: Chen, Brad
>Cc: systemtap@sources.redhat.com
>Subject: Re: what if I wanted to write some tests...
>
>
>Hi -
>
><brad.chen@intel.com> writes:
>
>  
>
>>Is there a way to check out the latest systemtap CVS tree, build it,
>>and figure out where to start writing tests? [...]
>>    
>>
>
>Of course - you can check out the translator and/or runtime sources.
>The former has various self-tests ("make check"); the latter has
>various kprobes-flavoured test programs.
>
>
>- FChE
>
>  
>