This is the mail archive of the mauve-discuss@sourceware.cygnus.com mailing list for the Mauve project. See the Mauve home page for more information.
>>>>> "Godmar" == Godmar Back <gback@cs.utah.edu> writes: README> You'll notice that each test has a public constructor. Godmar> That wouldn't allow me to test JNI compatibility, plus it Godmar> would allow me to run any of the tests that deal the Godmar> complexities of the threading and initialization parts. (For Godmar> instance: what happens if you get an Godmar> ExceptionInInitializerError in the class whose main() method Godmar> is invoked, etc.) I'm afraid I don't understand either of these objections. What is it about this approach that makes it impossible to test JNI compatibility? For the second point: are you concerned about testing for exceptions thrown in `main' in particular? Or do you want to be able to test constructors that cause ExceptionInInitializerError to be thrown? You can certainly test the latter; you just need two classes. Can you give me an actual example of a test you can't write in this framework? I mean, based on one of the above things? I know there are tests you can't write -- for instance, you can't test that a running daemon thread won't prevent the test program from exiting. Also, you can't test finalization on exit. I guess that's too bad. I personally don't like it, but there it is anyway. Some of the untestable things really approach being tests of the runtime and not the library per se. I don't know if these really fall within mauve's remit. Or maybe you're asking about what happens if there is a bug in the test case's constructor. All the existing constructors are empty. That seems like a wise course to take in general. So I'd say this isn't an important consideration. Godmar> I would prefer something that would simply diff the expected Godmar> against the actual output, and voila. I'm with you there, but most everybody else was against this. Anyway, the plan is to add some code to the harness to let you do this in a certain way -- you'll be able to embed the expected output in the test case, and the harness will do the comparison. This isn't written yet. Godmar> It would also be nice if it didn't require anything but Godmar> /bin/sh to run. I believe we currently require GNU make, sh, a Java runtime, and a Java compiler. It would probably be possible to get rid of the GNU make dependency. I'm not particularly motivated to do it. Godmar> On the other hand, I would like it if I could use it to run a Godmar> group of tests or single tests quickly. You can do this. Feed the class list you care about into the test harness' stdin. Note also that SimpleTestHarness is just a simple, generic test harness. You can write a more complicated one if you have different needs. We're going to write another harness in-house so we can do DejaGNU-based embedded system testing (though we might use the simple harness for ordinary testing) Godmar> That is not to say that test using Class.forName etc. would be Godmar> useless: in fact, they would probably test durability if the Godmar> engine isn't shut down between tests. Right now we run all the tests in a single harness invocation. However, this isn't really fixed. There is some info in README about an idea to change this (grouping). If you want, you could run a separate invocation of the harness for each test. Note that being able to run big batches of tests all at once is nice for us gcj developers. That's because we have to relink the test harness when we compile. If we put a main() into every test, then we'd at least double the time it takes to run the test suite. Tom