Basic requirements for supporting OS and machine ports.

[Joseph Myers]

On Fri, 21 Apr 2017, Florian Weimer wrote:

> * Joseph Myers:
> 
> > Well, we need a glibc-specific script to collect whatever information we 
> > think is relevant from the results of a test run for sending to 
> > libc-testresults (for use for both manual and buildbot builds).
> 
> Do you agree that the build bots are too difficult to use right now?
> 
> I found it very hard to interpret the results, but my difficulties
> baffle others.

I think the mysterious "make[1]: *** [tests] Error 1" at the end of a 
large proportion of test runs, with no previous errors reported, is pretty 
fatal to simple use of the bots to see what tests are failing.  Someone 
running a bot showing that issue needs to debug and fix it.

Even with that fixed, it would be a lot better if the bot showed failures 
only when there were regressions - that is, if any issues causing known 
test failures were fixed, or the tests in question made more robust if 
they aren't actually showing glibc bugs, or XFAILed if applicable.

-- 
Joseph S. Myers
joseph@codesourcery.com