This is the mail archive of the gdb@sourceware.org mailing list for the GDB project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[ANNOUNCEMENT] GDB BuildBot


Hi all,

It took a bit more time than I was planning, but finally I am back to
announce our BuildBot.  I really expect that it will help us improve our
codebase and catch regressions/failures sooner (in fact, it already
helped me to catch at least 2 regressions).

The initial idea was to deploy our BuildBot master directly on
sourceware, but unfortunately this was not possible.  So, after trying
to find other places, I decided to use my own personal server.  However,
my main goal is to turn this project into something official for the GDB
project, and I do hope that using my own server does not give a false
sense of "this is Sergio's pet-project".

Having said that, the current address for our BuildBot is:

  <http://gdb-build.sergiodj.net/>

You will see many pages there, each one displaying the builds in a
particular way.  The one that I find most useful is the Waterfall view:

  <http://gdb-build.sergiodj.net/waterfall>

You can already see the 8 build configurations (a.k.a. "builders") that
I created to test Fedora in a variety of ways; it is obviously possible
to create as many builders as we want, and I intend to expand this list
very soon (I'm trying to obtain access to a PPC64 Fedora machine).

Each build is composed by a number of steps.  There are preparation
steps (i.e., cleaning up the build directory, updating the git
repository), compilation steps (i.e., configure && make && make check,
each one with customizable parameters), a step to compare the test
results (kindly contributed by Tom Tromey), and another two steps to
save the test results in a nice way (details later).

The comparison of the test results is done by using a baseline, which is
the best testsuite run done so far (think of it as the best gdb.sum we
had so far).  For each testsuite run, we get its results and compare
against this baseline; if something that was PASSing turns to FAIL, it
is reported as a regression.  If a test that was FAILing turns to PASS,
it is added in the baseline.  If no regressions are found, then nothing
is done; otherwise, an e-mail is sent to <gdb-testers@sourceware.org>
with a report of what failed.  Therefore, if you are not subscribed to
this list, I strongly recommend you to do so!

Still on the comparison topic, another thing worth mentioning is the
presence of non-deterministic testcases in our testsuite.  To deal with
this, a special file called "xfail" can be created for each builder,
which will contain lines to be ignored in the gdb.sum file (i.e., test
failures to be ignored).  For now, this file needs to be updated by
hand, but I am already thinking about a way to semi-automatize the
process.  If you want me to update the xfail file for a certain builder,
please send me an e-mail (and feel free to include the GDB mailing list
in the Cc).

After struggling *a lot* trying to find a nice way to store the
testsuite results, I came up with a solution using git as a DB-like
backend.  You can see the test results here:

  <http://gdb-build.sergiodj.net/cgit>

You will notice that, for each builder, there is a git repository.
Inside it, I store the gdb.{sum,log} files.  When a new commit has been
tested by BuildBot, it stored the files in the repository, and commits
the result.  Additionally, it also creates a tag whose name is composed
by a timestamp and the git SHA1 that has been tested from the GDB
repository.  For example, suppose that I pushed a new patch to the GDB
repository, and its SHA1 is 0a1b2c3d.  Then, after BuildBot has tested
my commit on the builder Fedora-x86_64, it will store the resulting
gdb.{sum.log} files on the repository named Fedora-x86_64, commit them,
and create a tag named 20150114-182610-0a1b2c3d.

The cgit interface's main purpose, for now, is just to allow everybody
to have access to the full testsuite logs that were generated when
certain commit was tested.  It is also not possible (yet) to clone the
repositories containing the logs; maybe I will enable it in the future.

The configuration files for our BuildBot are currently located at:

  <http://git.sergiodj.net/?p=gdb-buildbot.git;a=summary>

I intend to submit them soon upstream, and put them under the contrib/
directory.  The BuildBot also offers an IRC bot that can stay in the
channel and send messages if some regression is found, but I haven't
enabled this because it can be a little verbose for some people.
However, if the majority wants, I can easily enable it.

I am maintaining a wiki page describing the details of our BuildBot
here:

  <https://sourceware.org/gdb/wiki/BuildBot>

And I am looking for new buildslaves to add to our setup!  If you are
interested, take a look at the wiki page above and do not hesitate to
contact me.  I personally believe that the following setups would help a
lot:

  - ARM
  - PPC
  - Some Windows/MinGW machine

I am also open for suggestions/improvements, of course.  I still think
our configuration needs a lot of polishing, and I already have some
ideas that I would like to implement (testing other branches, enabling
the "Try mode", which will allow people to send their patches to be
tested, etc.), but IMNSHO it is good to be used "in production" already.
I hope it helps!

Before I forget: special thanks to Tom Tromey (for starting to work on a
previous GDB BuildBot in 2011), and Jan Kratochvil for valuable feedback
during the configuration of the current BuildBot.

Cheers,

-- 
Sergio
GPG key ID: 0x65FC5E36
Please send encrypted e-mail if possible
http://sergiodj.net/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]