Re: Status of build bots?

[Carlos O'Donell <carlos at redhat dot com>]

On 8/20/19 6:05 AM, Florian Weimer wrote:
> * Carlos O'Donell:
> 
>> We really need functioning build bots for all major targets:
>>
>> * x86_64 / i686
>> * aarch64 / arm
>> * s390x / s390
>> * ppc32 / ppc64 / ppc64le
>>
>> It would be great if we got s390x backup and running.
> 
> You should qualify whether this is a community perspective or a Red Hat
> perspective.

Let me clarify.

Firstly I don't want this list to imply any bias about which machines
are important or not important, they are simply a reflection of how easy
it is to get hardware and setup systems that can be used publicly for the
given hardware.

>From a Red Hat perspective I care about:

* x86_64 / i686
* aarch64
* s390x
* ppc64le

>From an upstream perspective I care about:

* x86_64 / i686 / x32
* aarch64 / armv7hf
* s390x / s390
* ppc32 / ppc64 / ppc64le
* mips (several variants)
* riscv (several variants)
* c-sky
* nios II

I have a wishlist which would be:

* alpha
* sparc
* ia64
* hppa
* m68k
* microblaze
* sh

That is *my* list, and it may not reflect the desires and priorities of
others in the community, and you should feel free to put your efforts
where you want them to go.

> I have to admit that I have not been able to make any sense whatsoever
> of the buildbot output.  Is this really something from which regular
> glibc contributors derive value?  If not, why are we doing it?  Joseph's
> build-only tester is much more useful to me, even though it provides so
> little diagnostic output.

We need to do 3 things today to make our situation better:

(a) Commit to ownership of the various build bots.
    - Machine maintainer contact for help with machine issues.
    - Admin contact to help with system issues.

(b) Commit to fixing machine-specific test failures.
    - Machine maintainers get on board to cleanup machine failures as xfails.

(c) Commit to fixing generic test failures.
    - Community starts committing xfails for tests that fail frequently.

With that we get a clean set of builds.

We also need to upgrade to newer buildbot master so we can get
newer UI with cleaner interfaces.

Just look at Sergio's gdb buildbot and how nice it is to see logs snippets etc.
https://gdb-buildbot.osci.io/#/

> I also find the choice of architectures peculiar.  With the potential
> exception of arm (which variant?), these are exactly those architectures
> which are (comparatively) easy to get access to.  The regular
> contributors either have them in-house, or can access Debian
> porterboxes, the GCC compile farm, IBM's community resources (never used
> those, admittedly), or the public Fedora machines.

Some boxes are much harder to setup, let alone give "developer access" for the
box on a public network.

We should walk before we run, and setting up boxes for all the "easy" machines
is something we should succeed at before asking the machine maintainers to
help with the "harder" machines.

To answer your earlier question about value.

We want to see the results of execution failures given a particular commit
and make sure we aren't regressing. We should be doing this for all architectures
which are easy to setup and run. And you should get an immediate notification via
email if you broke something, and you should have 2 points of contact for fixing
the issue (the admin and machine maintainer).

Immediately after that you want to have "developer access" to the environment to
duplicate the failure and fix the issue (something we haven't discussed here).
Getting "developer access" is a key part of the equation here if machine maintainers
want help maintaining their port. It isn't required if you have a responsive machine
maintainer that steps in quickly to fix any issues.

-- 
Cheers,
Carlos.