RFC: libc-testresults mailing list

[Joseph Myers]

On Wed, 2 Nov 2016, Carlos O'Donell wrote:

> Do we expect that libc-testresults is a superset of those builds we have
> connected to our buildbot CI infrastructure?
> 
> Do we expect buildbot test results to go to libc-testresults for archiving?

If we add a script to mail out results then I think it would be 
appropriate to make our buildbot scripts use it (having made sure that 
existing buildbot instances have working outgoing mail and documented that 
as a requirement for buildbot instances).  In that case, yes, it would be 
a superset of the buildbot builds (note Andrew was asking about where to 
send results from a system not hooked up to our buildbot).

We can consider questions such as how much goes in test results mails from 
such a script, e.g. do we include the full tests.sum (possibly gzipped and 
attached) or just results that aren't PASS, what standard information 
about the test environment goes in mails, whether to attach (gzipped) .out 
files of failing tests if they aren't too many / too big, how mails should 
declare that they are the standard output of the script to allow for 
automatic parsing.

It would of course be expected that if someone's build system starts 
flooding the list with multiple mails from identical builds (keeping 
rebuilding the same versions continuously, or mail system problems 
resulting in duplicate mails) then those mails would be blocked until the 
problem is sorted out.

-- 
Joseph S. Myers
joseph@codesourcery.com