[PATCH] Makefile: add allow-list for failures

[Florian Weimer]

* Maxim Kuvyrkov:

> Where is allowed-failures.txt is supposed to be stored?  Is it going
> to be located in the glibc git repo or locally?

We wouldn't check it into the upstream glibc source tree and keep using
XFAIL as before (which already supports architecture-specific
overrides).

> Also, did you consider using validate_failures.py script from GCC [1]?
> It is already compatible with glibc's .sum files, and provides good
> support for expected FAILs and flaky tests.
>
> [1] https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=contrib/testsuite-management

I think the glibc situation is different because “make check“ in GCC
guards against compilation failures in tests.  With glibc, you can't
tell from the make exit status today whether the test suite has run to
completion with some failures, or stopped halfway through with some
compiler or linker error.

Downstream, we plan to treat failing “make check” as a build error.
Presently, we search for the expected output to detect an incomplete
build:

  if ! grep -Eq '^\s+=== Summary of results ===$' rpmbuild.check.log ; then
    echo "FAIL: test suite build of target: $(basename "$(pwd)")" >& 2
    exit 1
  fi

With Martin's check, we can drop that, and still have an override
mechanism if the build is no longer clean (maybe due to infrastructure
changes not under our direct control).

Does this make sense to you?  I think Martin's approach complements the
GCC script to some extent.

Thanks,
Florian