Re: Accelerating Y2038 glibc fixes

[Rich Felker <dalias at libc dot org>]

On Wed, Jul 17, 2019 at 04:15:25PM +0200, Arnd Bergmann wrote:
> On Tue, Jul 16, 2019 at 4:52 PM Wolfgang Denk <wd@denx.de> wrote:
> > In message <874l3mjgi6.fsf@oldenburg2.str.redhat.com> you wrote:
> >
> > > Since I'm opposed to this entire project, I have largely refrained from
> > > reviews, except for things that looked obviously wrong to me (e.g.,
> > > things that definitely break compatibility with older kernels or
> > > existing ABIs), but even for those cases, my feedback probably wasn't
> > > very helpful.
> >
> > I can understand your point of view, but indeed this is not helpful
> > here.  glibc is a central resource for the whole FOSS system, and we
> > are not pushing these changes for a fancy, but because a solution
> > is needed for a large number of affected projects and products.
> >
> > We need to find a constructive way to proceed with this matter.  It
> > will not go away if we try to ignore it.
> 
> My impression is that we have two mostly separate groups
> of users for 32-bit user space:
> 
> a) Those that absolutely require 100% ABI compatibility to run
>     legacy code but will eventually migrate to 64-bit user space,
>     almost certainly within the coming ten years before we really
>     need to worry (too much) about things like key expiration dates
>     beyond 2038.
> 
> b) Those that already need support for 64-bit time_t because
>     they are deploying new 32-bit binaries that are expected to run
>     beyond 2038, while not caring at all about compatibility
>     with existing binaries that are already known to be broken
>     for this purpose.
> 
> If we just work to satisfy both these groups but ignore the intersection
> (those that have existing 32-bit binaries and want those to run
> after 2038?), things could become noticeably easier.

Well it replaces one sticky problem with two separate, individually
easier problems. But I'm not sure it reduces the overall amount of
work needed; it might increase it.

The above roughly corresponds to what my original idea for musl and
the ".2 ABI" was, and I'm now fairly skeptical of it. Binary distros
are probably not going to want to expend extra effort to satisfy the
needs of both groups, so they'll either EOL the 32-bit ports they have
now (effectively leaving group (a) screwed, not because glibc dropped
them, but because their distro did) to produce something that will
continue to be relevant into the future, or they'll stick with the old
ABI and EOL 32-bit entirely in the near-ish future.

The segment of group (b) that's building everything themselves
probably doesn't care in the latter case, but then do they even care
about using glibc (vs something else) anyway?

>From this analysis it seems like, if you attack the problems
separately, "no action" is the right choice for distros, but that also
leads to EOL for distro support of all existing 32-bit targets.

As a result, I'm leaning strongly towards a conclusion that a solution
that doesn't split the two cases is better, and it's probably what
we'll do for musl.

> The price for that is a mid-2030s EOL time on any existing 32-bit

That's really wishful thinking. I suspect we'll hit critical failures
much sooner due to need to represent dates in the future.

> glibc ports, any binaries linked to them, and any distros built upon
> that, and replacing them with something else that is binary
> incompatible but can coexist within e.g. a container.

I'm not sure how "coexist within a container" is helpful; that might
as well just be a separate system. Multi-arch type approaches might
work a little bit better, but in order for these to be useful to end
users, there need to be automated ways of setting them up. That's a
big burden for distros to spend on archs that many of them would
rather just EOL/drop...

> There are several options what such a replacement might look
> like, some that come to mind would include:
> 
> - A patchset against glibc that replaces all the time32 interfaces
>   with time64 ones and that is maintained outside of the glibc
>   code base by whoever is interested in this working.

If you're trying to use incompatible old and new ABI alongside each
other, I would think you only want to do that as long as you still
have old stuff around. But if you're using patches with ad-hoc ABI
that have bypassed the process of upstream review to ensure that
they're making good decisions compatible with the long term, it's
likely that it's going to keep breaking ABI, and you're going to need
to do multiple painful replacements or keep N ABIs alongside each
other rather than 2 (possibly using something like nix to make it
practical).

So I don't think "use out-of-tree patches for time64 support" is a
good approach.

> - A set of glibc architecture ports that can coexist with the
>   existing ports but have new target triples and/or soname.
>   These could be implemented and merged one architecture
>   at a time and share most of the code with the existing
>   ports.

This is basically the same proposal as the musl ".2 ABI", and it's one
of the least-bad, but as explained above I no longer really favor it.

> - Leave glibc unchanged and require users/distros to migrate to
>   musl or something else if they need the time64 support.

As much as I'd like this, I don't think distros that have stability
contracts with their users for how things work can replace the libc
with one which has significantly different behaviors in application-
and user-facing ways. It's a viable solution for users to run new
32-bit stuff alongside a 32-bit glibc target that's being phased-out
and nearing EOL, but if this is the only option I think it would just
lead to distros dropping the targets.

> Any of the above approaches (including the current plan of
> supporting time32 and time64 in a single glibc binary) can of
> coexist, but the more incompatible approaches get implemented,
> the more fragmentation of ABIs we get.

I'm moderately in favor of supporting both in a single binary. As
noted elsewhere, this fundamentally does not break ABI between libc
consumers (main apps or 3rd party libs) and libc, only between pairs
of libc consumers that have used the affected types as part of their
public interfaces.

To answer Wolfgang Denk's original question, in my opinion, the single
most valuable thing for accelerating the time64 process that third
parties could contribute is detailed analysis of the interfaces that
will break. This (1) will give us an idea of the extent of breakage
that could happen in the worst case, and (2) can be translated
directly into a graph of "requires/conflicts-with" relationships
between library packages that a distro can use to ensure consistency
when updating packages.

Are others willing to start a discussion of exactly what form this
analysis should take, how results should be reported, and how to use
them to make such a transition go smoothly?

> Note that we have never had such a forced transition on x86, but
> there are several examples on other architectures:
> - arm oabi/gnueabi/gnueabihf
> - ppc64/ppc64le
> - mips o32/n32/64
> The migration is not much fun for users to say the least, but it's
> quite possible that the alternative is even worse.

I wouldn't characterize all of those as transitions, more as different
options. As far as I know glibc still supports them all except ARM
OABI, which was dropped long before there was serious interest in
end-user-facing Linux on ARM.

Rich