Re: Bytecode test suite

[Patrick Doyle <doylep at eecg dot toronto dot edu>]

On 6 Aug 2001, Tom Tromey wrote:

> Patrick> I'm thinking of starting a new project on SourceForge for
> Patrick> this.
> 
> If you like, we could just add a new module to the Mauve repository.
> Then it would be one-stop shopping...

That might be a better idea.  It may be the case that new JVMs will want
to run a bytecode test before they are ready to run a comprehensive JVM
test.  Is it easy to run a single Mauve module?

Currently, the test is two files: a Java program which produces a bunch of
text output on stdout, and the correct output.  I run diff to check for
correctness.

How does this compare to the format of a Mauve module?  Would Mauve prefer
the test to be entirely self-contained, and return a yes/no?  If so, I
could modify the program to read in the expected output and do the
comparison as it goes, though that would raise the entry level for JVMs
that would be able to run the test.

> Do you mean just raw bytecodes?  Because it is also worthwhile to have
> tests which are written in Java, then compiled and run against the
> JVM.  For instance you can test things like proper operation of
> various JVM-level things (what does "throw null" do, etc) without
> having to write a bunch of bytecode by hand. 

Actually the whole test is currently a single Java program.  However, I
expect that eventually someone will want to test things that are more
convenient to test from hand-constructed class files.  (Or perhaps there
are bytecode assemblers that would make the job easier.)

> There are a few, but not many, tests like this in the libgcj tree.  
> Of course we would contribute those.

Great.  Would it be kosher to read the Mozilla tests to get ideas from
them too?  (Even better to just use them as-is, though I suspect there
might be sticky licensing issues with that.)

> Another thing I'd be interested in seeing is a test suite for bytecode
> verifiers.  The idea would be to collect unusual pass/fail cases to
> stress test a verifier.
>
> It might be useful to group all the JVM-level tests into a single
> place.  I don't know.

Ok, well I personally don't have an interest in verifiers, but it may
indeed make sense to include all the tests together.

> I can contribute adminstratively -- set you up with an account on
> sources.redhat.com, show you how to update the web pages (if you want
> to), etc.

Ok, I'll email you about this.

> I don't have time to write tests right now :-(

No problem.  If anyone reading has an in-depth understanding of the
intricacies of a particular bytecode instruction and wants to write a test
program for it, speak up!

--
Patrick Doyle
doylep@eecg.toronto.edu