Add build-many-glibcs.py support for building more GCC libraries [committed]

[Joseph Myers <joseph at codesourcery dot com>]

Every so often we get libsanitizer or libgo builds breaking with new
glibc because of some change in the glibc headers.

glibc's build-many-glibcs.py deliberately disables libsanitizer and
GCC languages other than C and C++ because the point is to test glibc
and find glibc problems (including problems shown up by new compiler
warnings in new GCC), not to test libsanitizer or libgo; if the
compiler build fails because of libsanitizer or libgo failing to
build, that could hide the existence of new problems in glibc.
However, it seems reasonable to have a non-default mode where
build-many-glibcs.py does build those additional pieces, which this
patch adds.

Note that I do not intend to run a build-many-glibcs.py bot with this
new option.  If people concerned with libsanitizer, libgo or other
potentially affected GCC libraries wish to find out about such
problems more quickly, they may wish to run such a bot or bots (and to
monitor the results and fix issues found - obviously there will be
some overlap with issues found by my bots not using that option).  I can 
provide instructions on setting up a bot if desired.

Note also that building a non-native Ada compiler requires a
sufficiently recent native (or build-x-host, in general) Ada compiler
to be used, possibly more or less the same version as being built.
That needs to be in the PATH when build-many-glibcs.py --full-gcc is
run; the script does not deal with setting up such a compiler (or any
of the other host tools needed for building GCC and glibc, beyond the
GMP / MPFR / MPC libraries), but perhaps it should, to avoid the need
to keep updating such a compiler manually when running a bot.

Tested by running build-many-glibcs.py with the new option, with
mainline GCC.  There are build failures for various configurations,
which may be of interest to Go / Ada people even if you're not
interested in running such a bot:

* mips64 / mips64el (all configuration): ICE building libstdc++, as
  seen without using the new option
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87156>.

* aarch64_be: error building libgo (little-endian aarch64 works fine):

version.go:67:13: error: expected ';' or ')' or newline
67 |   BigEndian =
   |             ^
version.go:67:3: error: reference to undefined name 'BigEndian'
67 |   BigEndian =
   |   ^

* arm (all configurations): error building libgo:

/scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:29:5: error: reference to undefined name 'randomTrap'
29 |  if randomTrap == 0 {
   |     ^
/scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:38:34: error: reference to undefined name 'randomTrap'
38 |  r1, _, errno := syscall.Syscall(randomTrap,
   |                                  ^

What's happening there is, I think, that the arm*b*-*-* case in
libgo/configure.ac is wrongly matching arm-glibc-linux-gnueabi with
the 'b' in the vendor part, and then something else is failing to
handle GOARCH=armbe.  Given that you can have configurations with
multilibs of both endiannesses, endianness should always be detected
by configure.ac, for all architectures, using a compile test of
whether __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__, not based on textual
matches to the host (= target at top-level) triplet.

* armeb (all configurations): error building libada (for some reason
  the Arm libada configuration seems to do different things for EH for
  big-endian, which makes no sense to me and doesn't actually work):

a-exexpr.adb:87:06: "System.Exceptions.Machine" is not a predefined library unit
a-exexpr.adb:87:06: "Ada.Exceptions (body)" depends on "Ada.Exceptions.Exception_Propagation (body)"
a-exexpr.adb:87:06: "Ada.Exceptions.Exception_Propagation (body)" depends on "System.Exceptions.Machine (spec)"

* hppa: error building libgo (same error as for aarch64_be).

* ia64: ICE building libgo.  I've filed
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87281> for this.

* m68k: ICE in the Go front end building libgo
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84948>.

* microblaze, microblazeel, nios2, sh3, sh3eb: build failure in libada
  for lack of a libada port to those systems (I'm not sure sh3 would
  actually need anything different from sh4):

a-cbdlli.ads:38:14: violation of restriction "No_Finalization" at system.ads:47

* i686-gnu: build failure in libada, might be fixed by the patch
  attached to <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81103>
  (not tested):

terminals.c:1115:13: fatal error: termio.h: No such file or directory

Committed.

2018-09-11  Joseph Myers  <joseph@codesourcery.com>

	* scripts/build-many-glibcs.py (Context.__init__): Add full_gcc
	argument.
	(Config.build_gcc): Use --disable-libsanitizer for first GCC
	build, but not for second build if --full-gcc.  Use
	--enable-languages=all for second build if --full-gcc.
	(get_parser): Add --full-gcc option.
	(main): Update call to Context.

diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py
index a7c2b67bce..61f98651c6 100755
--- a/scripts/build-many-glibcs.py
+++ b/scripts/build-many-glibcs.py
@@ -91,13 +91,14 @@ class Context(object):
     """The global state associated with builds in a given directory."""
 
     def __init__(self, topdir, parallelism, keep, replace_sources, strip,
-                 action):
+                 full_gcc, action):
         """Initialize the context."""
         self.topdir = topdir
         self.parallelism = parallelism
         self.keep = keep
         self.replace_sources = replace_sources
         self.strip = strip
+        self.full_gcc = full_gcc
         self.srcdir = os.path.join(topdir, 'src')
         self.versions_json = os.path.join(self.srcdir, 'versions.json')
         self.build_state_json = os.path.join(topdir, 'build-state.json')
@@ -1122,6 +1123,8 @@ class Context(object):
         """Run a copy of this script with given options."""
         cmd = [sys.executable, sys.argv[0], '--keep=none',
                '-j%d' % self.parallelism]
+        if self.full_gcc:
+            cmd.append('--full-gcc')
         cmd.extend(opts)
         cmd.extend([self.topdir, action])
         sys.stdout.flush()
@@ -1328,15 +1331,13 @@ class Config(object):
 
     def build_gcc(self, cmdlist, bootstrap):
         """Build GCC."""
-        # libsanitizer commonly breaks because of glibc header
-        # changes, or on unusual targets.  libssp is of little
-        # relevance with glibc's own stack checking support.
-        # libcilkrts does not support GNU/Hurd (and has been removed
-        # in GCC 8, so --disable-libcilkrts can be removed once glibc
-        # no longer supports building with older GCC versions).
+        # libssp is of little relevance with glibc's own stack
+        # checking support.  libcilkrts does not support GNU/Hurd (and
+        # has been removed in GCC 8, so --disable-libcilkrts can be
+        # removed once glibc no longer supports building with older
+        # GCC versions).
         cfg_opts = list(self.gcc_cfg)
-        cfg_opts += ['--disable-libsanitizer', '--disable-libssp',
-                     '--disable-libcilkrts']
+        cfg_opts += ['--disable-libssp', '--disable-libcilkrts']
         host_libs = self.ctx.host_libraries_installdir
         cfg_opts += ['--with-gmp=%s' % host_libs,
                      '--with-mpfr=%s' % host_libs,
@@ -1359,14 +1360,20 @@ class Config(object):
                          '--disable-libitm',
                          '--disable-libmpx',
                          '--disable-libquadmath',
+                         '--disable-libsanitizer',
                          '--without-headers', '--with-newlib',
                          '--with-glibc-version=%s' % self.ctx.glibc_version
                          ]
             cfg_opts += self.first_gcc_cfg
         else:
             tool_build = 'gcc'
-            cfg_opts += ['--enable-languages=c,c++', '--enable-shared',
-                         '--enable-threads']
+            # libsanitizer commonly breaks because of glibc header
+            # changes, or on unusual targets.
+            if not self.ctx.full_gcc:
+                cfg_opts += ['--disable-libsanitizer']
+            langs = 'all' if self.ctx.full_gcc else 'c,c++'
+            cfg_opts += ['--enable-languages=%s' % langs,
+                         '--enable-shared', '--enable-threads']
         self.build_cross_tool(cmdlist, 'gcc', tool_build, cfg_opts)
 
 
@@ -1658,6 +1665,8 @@ def get_parser():
                         'with the wrong version of a component')
     parser.add_argument('--strip', action='store_true',
                         help='Strip installed glibc libraries')
+    parser.add_argument('--full-gcc', action='store_true',
+                        help='Build GCC with all languages and libsanitizer')
     parser.add_argument('topdir',
                         help='Toplevel working directory')
     parser.add_argument('action',
@@ -1676,7 +1685,7 @@ def main(argv):
     opts = parser.parse_args(argv)
     topdir = os.path.abspath(opts.topdir)
     ctx = Context(topdir, opts.parallelism, opts.keep, opts.replace_sources,
-                  opts.strip, opts.action)
+                  opts.strip, opts.full_gcc, opts.action)
     ctx.run_builds(opts.action, opts.configs)
 
 

-- 
Joseph S. Myers
joseph@codesourcery.com