Re: Build Breakage

[woodzltc at linux dot ibm dot com]

On Thu, 6 Jul 2006, Mark Wielaard wrote:

> On Wed, 2006-06-28 at 17:34 +0800, Wu Zhou wrote:
> > > Yes, I agree, and there is.  It's really simple:  frysk always  
> builds; frysk's
> > > testsuite always passes.
> >
> > I totally agree to the first one: frysk has to always build.  But in my
> > opinion, the testcases exist to find bug.  To expose new problem is the
> > value of the testcase, right? So I hold the point that we can let these
> > failing tests enabled.  We would like to see all frysk's testsuite pass.
> > But if there are anything failing, why not just let them open.  This is a
> > sign that there are some problem for us to resolve.  A 100 percent pass
> > rate might obscure our eyes.  What do you think?
>
> We do have XFAIL and <<BROKEN>>PASS. These are not really failures, but
> indicate that either it is a known bug if it is a XFAIL, or that the
> testcase is written badly (or cannot currently be written correctly), if
> it is a <<BROKEN>>PASS. These two make sure that make check "passes".
> That way we have it both ways. Make check passes and we have an overview
> of things that still need to be fixed.

XFAIL and <<BROKEN>>PASS is good for tracking problems.  If it is very
important for "make check" to pass, this policy is ok with me.

> > The second concern I have is on the execution of the presribed policy. I
> > am thinking that if we can test at least on two platforms before we
> > check in the code, that will be great.
>
> Yes indeed. But not everybody has access to multiple platforms. It would
> probably be good if people could setup some kind of autobuilder for the
> platforms they care for. Then we would have an overview of what works on
> what platform over time.

I had some script to do similar job.  But on PPC64, I need to resolve
quite some 64-bit libraries dependence issues to make it build ok.  This
make it somewhat hard to build completely automatically on ppc64.  I am
also interested in x86 and x86-64 platforms, though ppc64 is what
interested most. :-)  now I can run these test automatically on x86.
But I only have remote access to x86-64 machine,  and it can't be
dedicated to frysk usage.  Anyway, I can put the test results on x86
first.  Will do a while later.

> Mozilla has a nice tool for this Tinderbox. But I don't know how hard it
> is to setup. http://www.mozilla.org/projects/tinderbox/
> There is also http://buildbot.sourceforge.net/ which seems popular with
> the gnome hackers.

I can have some looks into these two tools to see if they are proper for
our usage. If any of you have any experience in them, feel free to help
me.  :-)

Regards
- Wu Zhou