Re: Status of build bots?

[Tulio Magno Quites Machado Filho <tuliom at linux dot ibm dot com>]

Stefan Liebler <stli@linux.ibm.com> writes:

> On 8/20/19 11:08 PM, Carlos O'Donell wrote:
>> Just look at Sergio's gdb buildbot and how nice it is to see logs snippets etc.
>> https://gdb-buildbot.osci.io/#/
>> 
> I've just had a look to the gdb buildbot.
> You can see that the builds were green (all fine) or red (fails).
> But compared to glibc buildbot, it's much easier and faster to see which 
> tests have failed (see regressions in the build summary).
>
> We could also distinguish somehow between red (fails) and red* (fails, 
> but further fails appeared or are now passing compared to last build)

Notice the GDB buildbot has a mechanism to distinguish between new failures
(regressions) and existent failures, e.g.
https://gdb-buildbot.osci.io/#/builders/7/builds/542

This is important because it avoids hiding a second regression while we're
fixing the first one.

> Perhaps we could also dump the out files of all failing tests.
> At least sometimes this would help.

I think the following steps would be helpful:

    # Send a script to capture the output of testcases from the
    # master to the slave.
    self.factory.addStep(
    	FileDownload(
    		mastersrc='glibc-report.sh',
    		workerdest='glibc-report.sh'
    	)
    )
    #
    # Collect the output from the tests that failed.
    self.factory.addStep(
    	ShellCommand(
    		name="report",
    		command=[
    			"bash",
    			"./glibc-report.sh"
    		],
    		workdir="vpath",
    		warnOnFailure=True,
    	)
    )

Where glibc-report.sh is:

    #/bin/bash
    
    files=$(find . -name '*.test-result');
    for f in ${files}; do
    	if ! grep -qE "PASS|XFAIL|UNSUPPORTED" "${f}"; then
    		out=$(echo ${f} | sed -e 's/\.test-result/.out/');
    		echo "---=== ${out} ===---";
    		cat "${out}"
    		echo
    		cat "${f}"
    		echo
    	fi;
    done

-- 
Tulio Magno