This is the mail archive of the
mailing list for the glibc project.
Re: Use gen-libm-test.py to generate ulps table for manual
[answering to a bounced mail, I'm not on libc-alpha@, so keep me CCed]
I'm randomly using one mail of this thread as it covers most topics, but I
use it for making general points, not only specifically answering to your
On Tue, 14 Aug 2018, Carlos O'Donell wrote:
> > This is a capability that deteriorates very quickly if you do not
> > continuously exercise it.
> Agreed. How does adding python prevent you from continuously exercising
> the bootstrap? Do you argue that it makes the whole build cycle too
It's technically not (very) difficult to include python into the bootstrap
set. But that's not the most important part. All SUSE distros since
forever use this scheme of building the distro, a bootstrap set (iterating
builds until stability), and once the bootstrap set is finished building
all dependend packages in a topological order (breaking cycles in that
other non-bootstrap set, if they exist, at defined points).
That implies the following thing: every change in any package of the
bootstrap set implies a whole-distro rebuild (any change in any of the
other packages implies only a rebuild on the depending packages). So
enlarging the bootstrap set makes full distro rebuild more common, and
given that this takes about two to four days (depending on architecture)
it's something you don't want to do lightly. So changes to packages in
the bootstrap set are done more seldom than for other packages.
> > We can't even do periodic (say weekly) mass rebuilds in Fedora, and we
> > know that there are uses for that, such as generating accurate markup
> > (be it annobin or CET) based on the latest toolchain fixes.
> Either we can't do it technically (not true, we can).
I bet you theoretically can, in reality you don't. Such bootstrap
capability detoriates within weeks if not done regularly (we for instance
see this with old enterprise products where this is not regularly done;
once you try this you go "oh my god, what is this, what happened here?").
> I don't know why Andreas wants the core bootstrap small.
It's not Andreas, it's all (open)SUSE distros. It's the way we do this
since ages and it's not small work (occasionally) to keep the bootstrap
set small. Which is why we try very hard to avoid extending it, including
having these very discussions.
> I don't understand Andreas' motivation.
I can bring up the original philosophy of how this came to be in the SUSE
distros: whenever there's a change in any package, all other packages
depending on it (per buildrequires, and transitively so) should be rebuilt
(I won't got into the reasoning for this policy here). That requires a
topological sort, and as such can't handle cycles. So we allowed exactly
one cycle of packages, the bootstrap cycle. That one builds until a fixed
point (or well, in reality all packages a max of three times). But all
other packages are build in topological order. Obviously it's a good idea
to let this cycle be small, so that at least the rest of packages can be
built according to the golden rule. So, that's where we're coming from.
As you can see in Andreas' list the bootstrap cycle is small (and with
minor additions over time contains still the same things since 20 years;
though I think we could remove some more things from it with a little
Over time we of course got more cycles in BuildRequires than this (and the
reason is often programmers not taking these things into account, e.g.
adding a dependency on java from within gettext; just an example, not
necessarily what happened), but still only the bootstrap cycle is iterated
and implies a full distro rebuild.
> Using python in upstream glibc makes it easier to maintain, and the only
> downside is the bootstrap issue.
(sidenote: if I were a glibc maintainer I'd disagree; I know perl better
than python. Alas, I'm not :) Though one point I'll make is that
everybody sufficiently intelligent can learn any reasonable language; and
hence the like or dislike for specific languages should never in itself be
a reason to choose one or the other. Rather all other reasons to prefer
one over the other should trump such personal preferences. Such
considerations include: preexisting use of language $X, wideness of
availability for language $X interpreters/compilers and so on)
Another point I made over at gcc@ is that for base tools like gcc and
glibc the list of dependencies should be as minimal as possible, even at
the expense of a little more work. In my world the former is of much
higher importance than a little convenience for authors of helper scripts.
To my liking even depending on perl should be avoided (surely a little
text processing can just as well be done with awk).
> The solution *today* is to not build the subsystems in glibc that need
> python (manual, benchtests) during bootstrap.
To note, for all of the above I'm not talking about use of anything for
building optional stuff. We do the above, and if python is to be used for
building the manual we'll split that off as well from the main glibc
build. All my points above only apply to dependencies for
building essential part, which for glibc would be, ... well, libc ;-)
> If more things require python then we may need a minimal python for the
> bootstrap. Investing in that is a good idea only if you value what we
> have just discussed as a bootstrap.
We split python already into a base and a full python (which requires
tk and hence all of X). It might be that we "only" need to add openssl to
the bootstrap cycle in addition to python3-base. It would be good if we
can avoid all of this though.
> Other more useful bootstraps are IMO something like "Build all C++
> programs with the latest C++ compiler changes, but do it in a side tag"
> which would allow the Fedora toolchain team to test C++ changes. These
> kinds of CI-esque workflows don't need a minimal python or a minimal
> bootstrap since the C++ applications you are building will likely
> already require much of the distro to be present.
Actually at SUSE we do something similar to this as well. All compiler
changes are going into a staging project where many packages are rebuilt.
A new compiler version (and new glibc and similar things) goes through
larger stagings. That is indeed nice, but orthogonal to the bootstrap
> Agreed. This still doesn't convince me that you have a compelling enough
> use case for minimal bootstraps that upstream glibc must not adopt
> python as a bootstrap requirement (which it isn't yet).
What I'm always wondering in these discussion about switching languages
for helper scripts (of all!) are compelling reasons to switch _to_ new
infrastructures. Shouldn't the onus of finding such reasons be on those
suggesting a change? (I realize that one stated reason is "easier
maintenance", I merely disagree that even if this is true (which is at
least questionable) that it'd be a very compelling reason).
> To convince myself that the real solution is a python3-minimal solution
> I started with the python3 in Fedora Rawhide, and with maybe 5 minutes
> of spec file hacking I had a Python3 which built (in a clean rawhide
> mock chroot) *without* all the dependencies enabled.
As without any dependencies no C compiler would be installed I assume the
mock chroot contains a multitude of things by default. So is your
experiment for creating a -minimal variant valid?
> Is there a technical problem with needing python3-minimal? It seems like
> such a package would solve all sorts of early bootstrap issues with
> needing python?
As we currently don't have any such problems including it wouldn't solve
anything. If you chose to create such problem, then yes, including a
python3-base will be the technical solution to it.