Help needed with maintainer-mode

[Richard Earnshaw (lists)]

On 29/02/2024 10:22, Christophe Lyon via Gcc wrote:
> Hi!
> 
> Sorry for cross-posting, but I'm not sure the rules/guidelines are the
> same in gcc vs binutils/gdb.
> 
> TL;DR: are there some guidelines about how to use/enable maintainer-mode?
> 
> In the context of the Linaro CI, I've been looking at enabling
> maintainer-mode at configure time in our configurations where we test
> patches before they are committed (aka "precommit CI", which relies on
> patchwork).
> 
> Indeed, auto-generated files are not part of patch submissions, and
> when a patch implies regenerating some files before building, we
> currently report wrong failures because we don't perform such updates.
> 
> I hoped improving this would be as simple as adding
> --enable-maintainer-mode when configuring, after making sure
> autoconf-2.69 and automake-1.15.1 were in the PATH (using our host's
> libtool and gettext seems OK).
> 
> However, doing so triggered several problems, which look like race
> conditions in the build system (we build at -j160):
> - random build errors in binutils / gdb with messages like "No rule to
> make target 'po/BLD-POTFILES.in". I managed to reproduce something
> similar manually once, I noticed an empty Makefile; the build logs are
> of course difficult to read, so I couldn't figure out yet what could
> cause this.
> 
> - random build failures in gcc in fixincludes. I think this is a race
> condition because fixincludes is updated concurrently both from
> /fixincludes and $buillddir/fixincludes. Probably fixable in gcc
> Makefiles.
> 
> - I've seen other errors when building gcc like
> configure.ac:25: error: possibly undefined macro: AM_ENABLE_MULTILIB
> from libquadmath. I haven't investigated this yet.
> 
> I've read binutils' README-maintainer-mode, which contains a warning
> about distclean, but we don't use this: we start our builds from a
> scratch directory.
> 
> So... I'm wondering if there are some "official" guidelines about how
> to regenerate files, and/or use maintainer-mode?  Maybe I missed a
> "magic" make target (eg 'make autoreconf-all') that should be executed
> after configure and before 'make all'?
> 
> I've noticed that sourceware's buildbot has a small script
> "autoregen.py" which does not use the project's build system, but
> rather calls aclocal/autoheader/automake/autoconf in an ad-hoc way.
> Should we replicate that?
> 
> Thanks,
> 
> Christophe

There are other potential gotchas as well, such as the manual copying of the generated tm.texi back into the source repo due to relicensing.  Perhaps we haven't encountered that one because patches generally contain that duplicated output.

If we want a CI to work reliably, then perhaps we should reconsider our policy of stripping out regenerated code.  We have a number of developer practices, such as replying to an existing patch with an updated version that the CI can't handle easily (especially if the patch is part of a series), so there may be space for a discussion on how to work smarter.

My calendar says we have a toolchain office hours meeting today, perhaps this would be worth bringing up.

R.