Re: request for comments (long)

["Raif S. Naffah" <raif at fl dot net dot au>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

hello Tom,

On Wednesday 11 December 2002 09:30, Tom Tromey wrote:
> Raif S. Naffah <raif@fl.net.au>)> writes:
>>
>> i'm trying to add Mauve regression tests to the GNU Crypto
>> project.  currently regression tests are done with JUnit
>> (framework and implementation).
>
> Don't let me discourage you, but in the past there's been a fair
> amount of discussion on this list along the lines of "let's convert
> Mauve to use JUnit" and "Mauve's test framework is a pain".
>
> Given that, I'm curious to know why you'd want to convert...

we already have worked out --thanks to Olivier-- how to build, and use, 
an .so from JUnit 3.7 and JUnit 3.8.1  which we currently use.  the 
problems with this are:

* building a JUnit .so is not straightforward --because of the lack of 
support for awt and swing in gcj.
* because of the state of GCJ, we end up debugging the compiler rather 
than testing our implementations.

another reason is, which would be moot, if we go with separate trees, is 
that to run the tests, one has to go and download/install YAP (yet 
another package).

also, because some developers asked for it; and it seems to be _the_ 
choice par excellence for testing GCJ-related work --e.g. GNU 
Classpath, Kissme.

finally because it's simpler to learn and use, considering how little 
(feature-wise) we need/use from JUnit. Mauve, in constrast, is few 
classes, 1 README and off you go.  i believe there's a place for such a 
light-weight testing framework.


>> II. the 'choose' script seems to consider the value passed to
>> KEYS; e.g. foo as a Tag to look for in the source files, _in
>> addition_ to any other Tag that may be used in the '// Tags: '
>> line.  in other Raif>  words, if in my 'mauve-gnu-crypto' file i have
>> the following lines:
>>
>>    BAR
>>    !gnu.crypto.cipher.BaseCipherTestCase
>>
>> and in gnu/testlet/gnu/crypto/cipher i have a file, say
>> TestOfSquare, that has the following line:
>>
>>    // Tags: gnu-crypto
>>
>> it will get chosen by 'choose'.
>
> In this case you're running with `KEYS=gnu-crypto'?

correct.

>...  That behavior seems a bit unintentional to me.

noted.


>> III. when i tried to use the exclusion mechanism --prefix the
>> name of a class with ! after chopping off the gnu.testlet
>> prefix-- the class, when it contains the '// Tags: xxx' line is
>> always chosen.
>
> I don't follow :-(.

if line (was) #64 in the 'choose' script:

    !java.* | java.* | !javax.* | javax.*)

is not amended to include the root directory to add to the exiting 
'java' and 'javax' ones (see the diff in previous mail), then any 
exclusion directive in the tags file --e.g.

   !gnu.blah

has no effect; and the gnu.blah class/file is included anyway.  
sometimes you dont want to include files that are for example abstract 
or base classes, and do not have a test() method, but live in the same 
folder hierarchy as the test classes.


>> IV. the mechanics of configuring Mauve, and GNU Crypto, are
>> complicated...
>> another approach is to import the Mauve base classes and
>> scripts into GNU Crypto, and handle the lot with the same
>> toolchain.  the problem with this approach is that every time
>> an enhancement is made to Mauve, the same has to be adopted for
>> its twin in GNU Crypto...
>> is there a better way that we can use today?
>
> Maybe not :-(.  We're definitely open to suggestions and patches
> improving the situation.
>
> For libgcj we just have our test harness run Mauve's own configure
> and build.  However, as you point out this is a pain since the test
> code is a different project which must somehow be pointed to.

understand.  i'm after something as simple as that too :-)  putting the 
gnu.crypto.xxx test classes under the gnu.testlet folder, amending the 
'choose' script, and Makefile.am to use a $(project_so) seem to do the 
job.  i'll wait and see if others suggest different alternatives/ideas 
before deciding which way to jump.

thanks + cheers;
rsn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Que du magnifique

iD8DBQE99wT/+e1AKnsTRiERA0egAKCXypgpgpjGHW0F6owwjfbGacCzvQCghiOS
ZkdbDkjctR+Yl90RM9AD9pk=
=S6CK
-----END PGP SIGNATURE-----