Re: [PATCH buildbot] keep build directory after failure

[Szabolcs Nagy <Szabolcs dot Nagy at arm dot com>]

On 02/11/18 11:33, Szabolcs Nagy wrote:
> aarch64 buildbot failed with
> 
> FAIL: malloc/tst-malloc-tcache-leak
>
> 4 times so far even though there were no related commits,
> the failure happens about 1 out of 15 builds and only if
> the build was done in an unclean directory, but i could
> not manually reproduce it yet.

this is most likely caused by a cgroup task limit.

is there a known safe task limit setting for make check -j1 ?

the current setting is
$ cat /sys/fs/cgroup/pids/user.slice/user-1002.slice/pids.max
12288

it also caused other failures

env GCONV_PATH=.... /home/szabolcs/tx1-ubuntu-aarch64/glibc-aarch64-linux/build/build/nptl/tst-eintr1  >
/home/szabolcs/tx1-ubuntu-aarch64/glibc-aarch64-linux/build/build/nptl/tst-eintr1.out; \
../scripts/evaluate-test.sh nptl/tst-eintr1 $? false false >
/home/szabolcs/tx1-ubuntu-aarch64/glibc-aarch64-linux/build/build/nptl/tst-eintr1.test-result
/bin/sh: 2: Cannot fork
make[2]: *** [/home/szabolcs/tx1-ubuntu-aarch64/glibc-aarch64-linux/build/build/nptl/tst-eintr1.out] Error 2


> 
> the buildbot first builds glibc in the previous build
> directory, if that fails then that immediately gets
> deleted and glibc is rebuilt starting from an empty
> build directory, so it's impossible to tell what went
> wrong in case of such non-reproducible failures.
> 
> i suggest keeping the last failed build directory for
> analysis.
> 
> the build directory rename is not optimal since that
> breaks absolute paths pointing to it (e.g. paths in
> testrun.sh), but it allows looking at test outputs
> and failed binaries.
>