Re: Build Breakage

[Mark Wielaard <mark at klomp dot org>]

On Wed, 2006-06-28 at 17:34 +0800, Wu Zhou wrote:
> > Yes, I agree, and there is.  It's really simple:  frysk always builds; frysk's
> > testsuite always passes.
> 
> I totally agree to the first one: frysk has to always build.  But in my
> opinion, the testcases exist to find bug.  To expose new problem is the
> value of the testcase, right? So I hold the point that we can let these
> failing tests enabled.  We would like to see all frysk's testsuite pass.
> But if there are anything failing, why not just let them open.  This is a
> sign that there are some problem for us to resolve.  A 100 percent pass
> rate might obscure our eyes.  What do you think?

We do have XFAIL and <<BROKEN>>PASS. These are not really failures, but
indicate that either it is a known bug if it is a XFAIL, or that the
testcase is written badly (or cannot currently be written correctly), if
it is a <<BROKEN>>PASS. These two make sure that make check "passes".
That way we have it both ways. Make check passes and we have an overview
of things that still need to be fixed.

> The second concern I have is on the execution of the presribed policy. I 
> am thinking that if we can test at least on two platforms before we 
> check in the code, that will be great.

Yes indeed. But not everybody has access to multiple platforms. It would
probably be good if people could setup some kind of autobuilder for the
platforms they care for. Then we would have an overview of what works on
what platform over time.

Mozilla has a nice tool for this Tinderbox. But I don't know how hard it
is to setup. http://www.mozilla.org/projects/tinderbox/
There is also http://buildbot.sourceforge.net/ which seems popular with
the gnome hackers.

Cheers,

Mark