Re: [BuildBot] News and announcements

[Doug Evans <dje at google dot com>]

On Wed, Jun 17, 2015 at 1:09 PM, Sergio Durigan Junior
<sergiodj@sergiodj.net> wrote:
> Hi there,
>
> It has been some time since I set our BuildBot instance at:
>
>   <http://gdb-build.sergiodj.net>
>
> And I figured I could send a message announcing some news here.
>
> New buildslaves
> ===============
>
> First of all, I would like to thank IBM again for the old POWER7
> buildslaves by new, more capable POWER8 VM's.  We now have one PPC64BE
> and one PPC64LE buildslaves, both running Fedora 22.  Special thanks to
> Tulio Magno Quites Machado Filho and to Edjunior Machado, both from IBM
> Brazil, for all the help with the new VM's.  Much appreciated!
>
> Now, we have the following buildslaves in our BuildBot:
>
>   - One Fedora 21 x86_64 machine (owned by me; more expected to come in
>     the next months)
>
>   - One Fedora 22 PPC64BE machine (owned by Edjunior Machado)
>
>   - One Fedora 22 PPC64LE machine (owned by Edjunior Machado)
>
>   - One Fedora 21 s390x machine (owned by Mark Wielaard)
>
>   - One Debian Wheezy x86_64 machine (owned by Mark Wielaard)
>
>   - One Debian Wheezy i686 machine (owned by Mark Wielaard)
>
> (There are also 4 FreeBSD buildslaves, but they are currently inactive.)
>
> As I said, I expect to receive two more x86_64 machines in the next
> months; I intend to install Fedora in at least one of them, because the
> current Fedora x86_64 machine (owned by me) is struggling to keep the
> pace with the other buildslaves.
>
> I also tried to dedicate some time to put a buildslave on an AIX machine
> (in the GCC Farm).  This was a request by David Edelsohn.  However, I
> did not manage to compile GDB there, and after spending some time trying
> to figure out why, I had to switch context to another task.  If you are
> able to help with this, I would really appreciate.

I have successfully run the gdb testsuite on the aix machine on the gcc farm,
so it is doable. I can (try to) help (if I can find the time).

>
> Needless to say, we also welcome donations of new machines for our
> BuildBot!  For example, we don't have an ARM buildslave yet, which would
> be very nice.  So if you're interested in helping us, you can either
> take a look at this wiki page:
>
>   <https://sourceware.org/gdb/wiki/BuildBot>
>
> Or contact me directly.  The more machines we have, the more we can test
> GDB :-).
>
> New notifications
> =================
>
> If you commit a piece of code that breaks GDB in any of the current
> builders, you will now receive a direct e-mail from BuildBot.  This was
> an old feature I had in mind, and Mark Wielaard put some pressure to
> implement it :-).  The way it is implemented today, you will receive a
> message for every builder that is not able to compile your code.  This
> means that you can end up receiving lots of e-mails (currently, we have
> 24 builders).  While I agree that this may be annoying, well, breaking
> GDB is also annoying, so consider this a punishment!  Also, it is not
> uncommon to see commits that break only specific builders, so I think it
> is important to trigger a new notification every time something bad
> happens.
>
> I have also enabled a new notification for when a buildslave goes
> offline.  I decided to do that after having problems with buildslaves
> that were unstable and required constant monitoring.  Now, BuildBot
> automatically sends an e-mail to the buildslave's owner, notifying him.
>
> New builders
> ============
>
> Pedro asked for a new builder to test just the compilation of the new
> --enable-build-with-cxx flag.  It is now created on:
>
>   <http://gdb-build.sergiodj.net/builders/Fedora-x86_64-cxx-build-m64>
>
> This builder just compiles GDB, i.e., it does not run the testsuite like
> the others.
>
> Doug asked me if it was possible to create a new builder to test things
> with --target_board=stabs.  Unfortunately, we don't have resources to
> create this builder on the x86_64 machines; I will try to create the
> builder on the PPC64 buildslaves and see how they behave.

Let me take the opportunity repeat some of my feature requests. :-)

Instead of running a build per commit, just pick up the current tree
on each iteration of the builder (and early exit if nothing has changed).
And if there's a problem, bisect.
This should scale better when there are more frequent commits.
Commits usually don't break the build so picking up the tree
where it is should be just fine (has been for me with my
nightly build of trunk for years).

If a test fails, run it several times.
This should help catch hard fails vs flaky fails.

IWBN if one build handled multiple test configs.

As for new test configs I'd like to see:
gcc vs llvm
gdb-generated index vs gold-generated index vs no index
dwarf2 vs dwarf4 (with type units)
fission vs fission-with-dwp vs no-fission
stabs
gdbserver vs no gdbserver
Each of these would use the same gdb, it's just
the test run that's different.

>
> Problematic testcases
> =====================
>
> Efforts on reducing the number of problematic testcases are being done,
> mostly by Pedro.  Thanks, Pedro!  Hopefully, sometime in the future we
> will reach a state when all the notifications of regressions will
> actually be trustworthy.

It's difficult to prevent flaky tests from being added in the future,
which is why I'd like to see the build-bot help find them.

>
> And that's it.  Just a remind, all the reports generated by the BuildBot
> are publicly accessible on the gdb-testers mailing list:
>
>   <https://sourceware.org/ml/gdb-testers/>
>
> This is now a very high-volume mailing list, so be aware if you decide
> to subscribe to it!
>
> Other than that, I am looking forward to improving the BuildBot in any
> way that the project needs (taking into consideration the amount of
> spare time I have, of course).  My next steps are:
>
>   - Try to fit the builder requested by Doug using the current resources

My internal nightly test runs cover 16 different test configs
(alluded to above).
They take a few hours to run with a 6 core machine.

>
>   - Update the existing x86_64 machine to Fedora 22
>
>   - Get the new x86_64 machines and set them up for our BuildBot
>
>   - Get the AIX buildslave up and running
>
>   - Continue trying to convince others to donate buildslaves to us :-)

Working on it ...

> Any help is appreciated, of course.
>
> Cheers,

Thanks for setting this up!