Differences between revisions 11 and 12
Revision 11 as of 2015-02-04 21:19:31
Size: 11015
Comment:
Revision 12 as of 2015-02-12 18:23:11
Size: 11122
Editor: PedroAlves
Comment: mention gdb-testers
Deletions are marked like this. Additions are marked like this.
Line 17: Line 17:

Regression reports are sent to the [[https://sourceware.org/ml/gdb-testers/|gdb-testers]] mailing list.

GDB BuildBot

This wiki page describes how the GDB BuildBot works. Read it if you are interested in adding a new buildslave to the system, if you want to know who is responsible for each existing buildslave, or if you are just curious about how the BuildBot is configured.

1. TL;DR

The web interface for the GDB BuildBot is http://gdb-build.sergiodj.net/. Specifically, you may be interested in going directly to the Waterfall page, which displays in a nice manner the builds.

The URL for connecting your buildslave to the GDB BuildBot is gdb-build.sergiodj.net:16123. You must contact us first in order to obtain permission to do that (and a password); send a message to <gdb@sourceware.org> and put <sergiodj@redhat.com> in the Cc.

The interface for viewing the results of the builds is http://gdb-build.sergiodj.net/cgit .

Regression reports are sent to the gdb-testers mailing list.

2. New terms

In this page, we will use a few specific terms to describe things, so it is good to understand what they mean.

  • BuildBot: This is the name of the project we use to test GDB (see more info here).

  • buildmaster: BuildBot has a master node, which is responsible for deciding which tasks each slave will perform, collecting the results of each task, and providing the interface for viewing the results.

  • buildslave': This is the slave node of BuildBot. Slaves are "dumb", in the sense that they only execute things when master tells them to, and they usually do not have any knowledge of the entire task being performed. A typical configuration for BuildBot is to have one buildmaster, and one or more buildslaves connected to it. This is what GDB does.

  • build step: A step is what the name says: a step in the build process. For example, checking out the source code of the project that will be tested is a step.

  • build factory: A factory is like a recipe of how to perform a build. It can contain instructions for how to prepare the build directory, how to checkout the sources, how to compile them, how to test them, and how to compare the test results. A build factory is composed by one or more build steps.

  • builder: A builder is an instance of a factory. It is the build process itself. For example, one possible builder for GDB is Fedora-x86_64-native-gdbserver-m64, which is responsible for building and testing a 64-bit GDB on a x86_64 machine, and testing the native-gdbserver there.

3. How to use the cgit interface to view the logs

The interface for looking at the log files generated by the testsuite (i.e., gdb.sum and gdb.log files) is http://gdb-build.sergiodj.net/cgit. When you enter this page, you will many git repositories whose names are exactly the same names of the existing builders. Inside each repository, you will notice that there is only the master branch, and lots of tags. Each tag is named using the following convention:

TIMESTAMP-COMMIT

Where TIMESTAMP is composed by YEARMONTHDAY-HOURMINUTESECOND, and COMMIT is the SHA1 hash representing the GDB git repository's commit that has been tested. Therefore, if you pushed a commit to the GDB repository (say, 0abcd), and you want to take a look at the log files generated by testing your patch against the builder Fedora-x86_64-native-gdbserver-m64, you just have to go the repository named Fedora-x86_64-native-gdbserver-m64/.git and search (there is a search box in the top-right corner) for 0abcd.

4. How to add your buildslave

If you are interested in adding a new buildslave (and possibly a new builder) to the GDB BuildBot, thank you! There are some steps you need to perform in order to make sure everything is OK.

4.1. Buildslave specs

Your machine needs to:

  • Be able to properly compile and test GDB. Make sure all the necessary dependencies are installed in your system.

  • Be able to run Python, and to run the buildbot-slave == 0.8.10 package (previous version might work, but we have not tested them).

  • Have a decent amount of disk space to perform the git clone, the build and the test.

It is also recommended that your machine is constantly updated (e.g., on Fedora you might consider running a cronjob to yum update every day), because GDB's test results are impacted by updates on GCC, glibc, binutils, etc.

4.2. Buildslave owner responsibilities

You will be the responsible for the buildslave, which means that we may contact you if there is anything you should do (i.e., update the buildbot-slave package, restart your buildslave, etc.). Be sure to fill the contact fields on your buildslave configuration file (see below).

4.3. Contact us first

Please, contact us in order to discuss further details, and to let us know about you. The best way to contact the GDB community is to write to <gdb@sourceware.org>. Please, make sure you also include Sergio Durigan Junior in the Cc (e-mail: <sergiodj@redhat.com>), because he is the current maintainer of the GDB BuildBot.

4.4. Adding your buildslave to the configuration

You will need to patch the following files and add your buildslave configuration there:

  • config.json
  • master.cfg

On config.json, you will have to add your buildslave information on the slaves section. You should provide:

  • The name of your buildslave. Please pick a descriptive name. Do not use whitespaces nor special characters.

  • The arch, i.e., the architecture of your buildslave. Possible values are x86_64, ppc64, s390x, etc.

  • The jobs property, which says how many parallel jobs can be triggered by make when compiling GDB. This does not affect make check, because it is always done in non-parallel.

This is an example of how a new entry should look like:

"slaves" : [ { "name" : "fedora-x86-64-1", "arch" : "x86_64", "jobs" : "4" },
             ...
             { "name" : "debian-x86-64-1", "arch" : "x86_64", "jobs" : "2" }
           ],

Then, you will have to edit the builders section. In this section we describe what kind of build we are interested in, and what are the buildslaves that will perform it. If you are adding a buildslave with a new architecture, then you will most probably want to create a new builder for it. However, you can also be interested in adding more buildslaves to an existing builder. In both cases, you will have to edit the following properties:

  • The name of the builder. Please pick a descriptive name; for example, if the builder will build and test native-gdbserver 32-bit on Fedora, with a 64-bit GDB, your builder should be called Fedora-x86_64-native-gdbserver-m32 (actually, this is the name of an existing builder). Do not use whitespaces or special characters.

  • The type of the builder. This is one of the most important fields. It will be used as the prefix for the RunTestGDB* set of classes, in order to determine how to build and test GDB. Using the same example as above, the type would be NativeGDBServer_c64t32. The c64t32 part means that GDB will be compiled as a 64-bit binary, but the tests will be run in 32-bit mode (i.e., RUNTESTFLAGS='--target_board=native-gdbserver/-m32').

  • The builddir, which is the name of the directory that will be created on the buildmaster to store information about this build. Please use the name of the builder as the builddir, but with lowercase chars.

  • The slavenames property, which is a list of buildslaves that can perform this build. It is extremely important to include only buildslaves capable of building for the specified architecture. This is what a new addittion should look like:

"builders" : [ { "name" : "Fedora-x86_64-m64", "type" : "Plain_c64t64",
                 "builddir" : "fedora-x86-64",
                 "slavenames" : [ "fedora-x86-64-1" ] },
               ...
               { "name" : "Debian-x86_64-m64", "type" : "Plain_c64t64",
                 "builddir" : "debian-x86-64",
                 "slavenames" : [ "debian-x86-64-1" ] },

At last, if you created a new builder, you will also want to include its name in the builderNames property of the schedulers section.

After you have done this, please send a patch to the <gdb-patches@sourceware.org> mailing list. You will be provided a password used by the buildslave to connect to the buildmaster.

4.5. Buildslave configuration

The first thing you will need to do is to install the buildbot-slave, version 0.8.10, package on your system. To do that, run (as root):

pip install buildbot-slave==0.8.10

If this doesn't work, you will have to find the specific version on BuildBot's website. If you are running Windows, take a look at BuildBot's instructions.

You will need to install the dependencies to compile and test GDB. For Fedora systems, a good list of dependencies would be:

texinfo dejagnu gcc-c++ gcc-gfortran gcc-gnat fpc flex bison systemtap-sdt-devel expat-devel libselinux-devel ncurses-devel python-devel readline-devel rpm-devel xz-devel zlib-devel emacs

/!\ Because of the make TAGS step, you will also have to install emacs.

/!\ If your machine will also test 32-bit packages, remember to install the 32-bit version of the above packages as well.

It is also recommended that you create a separate user to run the buildslave. This user should not need/have root nor SSH access; it only needs to be able to build and test GDB.

For example, let's assume this new user is called buildbot. Log into its account in the system, and create a new buildslave. To do that, you are going to need three things:

  • The name of the buildslave (should be the same name you put in the JSON configuration file)
  • The address of the buildmaster, which is gdb-build.sergiodj.net:16123

  • The password that was provided to you

Then, you can issue the command:

% buildslave create-slave DIR URL:PORT NAME PASSWORD

Where DIR is the directory that will be used by the buildslave to store local files (it can be the same as NAME), URL:PORT is the address of the buildmaster, and NAME is the name of the buildslave. The PASSWORD needs to be obtained from the BuildBot admin.

After that, you can start the buildslave:

% buildslave start NAME

Make sure it is started OK. It is also recommended that you create some special cronjob to restart this on every reboot.

None: BuildBot (last edited 2015-02-12 18:23:11 by PedroAlves)

All content (C) 2008 Free Software Foundation. For terms of use, redistribution, and modification, please see the WikiLicense page.