Question about autoreconf to regenerate configuration files

[R. Diez]

>>> [...]
>>> there is only one set of top-level files.  they get manually synced from
>>> gcc to the other projects by devs who notice & need the fixes.
>>
>> OK, I gather that copying those files across from GCC is the standard practice.
>> But why do we need to keep doing that in Newlib? According to the (outdated)
>> cross toolchain build instructions, when combining the tree the GCC versions should prevail.
> 
> i don't understand what you're asking.  the top-level provides files that
> newlib uses.  they need to be kept up-to-date.  see AC_CONFIG_AUX_DIR use
> throughout the tree.

So the Newlib build system is a mix of something specific to Newlib, and some bits shared with GCC. But I would still like to understand what files are shared between Newlib, GCC, GDB, Binutils, etc.

I gathered that all AC_CONFIG_AUX_DIR around Newlib point to the top-level directory, which is not a good idea. Auxiliary files should be placed in a subdirectory in order to reduce clutter in the top-level project directory. This also prevents the configuration script from looking for helper scripts outside the project, in ../ and ../../ , possibly finding older versions or incompatible tools with the same name.

But I understand that we are sharing at least the multilib logic with GCC, probably config/multi.m4 , and those shared files perhaps expect AC_CONFIG_AUX_DIR to point to the top-level directory. So we shouldn't change it in Newlib.

Talking about config/multi.m4: this file is almost identical to the one in gcc-10.3.0, so it has fallen only a little behind. But other files like config/override.m4 have changed more. Maybe it is time to upgrade the copies in Newlib then, if they are truly shared and used?

Does Newlib have to include copies of those files anyway? If you are building multilibs, is that not specific to GCC? Wouldn't you need the GCC sources at hand anyway?

Does Newlib have to have files like include/gdb/sim-arm.h ? I thought the simulators lived with GDB, so is that not part of GDB? If you are building the simulators, do you not need the GDB sources anyway?

Does it make sense to have a MAINTAINERS file in Newlib talking about directories like binutils/ which do not exist? Shouldn't we have a different one called MAINTAINERS.Newlib? And the same with ChangeLog.

Why do we need to have our own copy of configure.ac at top-level? If you will be building in a "combined tree", then the other configure.ac copy should override ours, be it GCC, GDB or Binutils. If we are building just Newlib for another compiler or outside a "combined tree", would we be using that same configure.ac at top level? Or do you have to configure newlib/ and libgloss/ separately anyway?

Is it worth sharing the Autotools files with GCC at all? On the one hand, we can reuse GCC's multilib implementation, but on the other hand, I can imagine that dealing with integration problems as GCC etc. evolve is problematic. How do other libc libraries like Musl cope? Do they share the same Autoconf logic with GCC, or do they implement their own multilib logic?


>> It is not like anybody would take Newlib without GCC and use those top-level files (the
>> 'configure' script) to build say just Newlib and Binutils, is it?
> 
> why not ?  i have a gdb/binutils dir that i link just newlib & libgloss
> into.  the GNU simulator is part of those trees, and it pulls ABI info
> out of newlib & libgloss.  being able to iterate on those parts without
> rebuilding gcc is helpful.

Newlib alone is not going to suffice, because it is a "leaf" component. You will always need something else, like GCC or the GDB simulators. GDB brings its own configure.ac, config/ , etc. Should these not override the ones in Newlib when building in this smaller "combined tree"? Or does that overriding rule not apply here?


The main problem is that you need too much undocumented information in order to be able to maintain Newlib, to create a new release, or simply to contribute or to troubleshoot it. Or maybe I haven't found yet the place where all this is documented.

I would have expected to see documentation like this:

---------8<---------8<---------8<---------

README-Newlib.txt

In order to create a Newlib release:

1) Sync the build files shared with GCC.

The Newlib sources are designed to be used inside a "combined tree" with GCC and/or GDB etc. in order to build several toolchain components at the same time.

The files to synchronise from GCC (the master copy) are:

xxx
config/
yyy/
zzz

2) Regenerate the Autoconf files, and check the new versions in.

The exact Autools versions that should be used are listed in file README-maintainer-mode .

Use these commands (or better, provide a script):
( cd newlib && time autoreconf --force --verbose --warnings=all )
...

3) Test building in a "combined tree" with recent GCC and GDB simulator versions.

4) The maintainer (or the user?) can rebuild the Newlib documentation like this:

5) ...

---------8<---------8<---------8<---------


>> If we want to strictly keep in sync with GCC, shouldn't we check the exact Automake version too?
> 
> we pin the min version via AM_INIT_AUTOMAKE to 1.15.1.  you're right we don't
> currently pin the maximum, it's more through everyone agreeing to use that
> version.  if gcc et al had a macro to pin the automake version, we'd prob
> leverage it too.
> 
> pinning the versions in general is meant to keep generated noise/conflicts
> down between devs working on these projects, and to make expectations more
> obvious to users.
> [...]
> putting "we used xxx versions" into README

The implementation is inconsistent. We pin the exact Autoconf version in order to minimise the noise/conflicts when regenerating those files, and state that a README has proven to be insufficient, but we do not pin the exact Automake version. Why not then?

Regards,
   rdiez