Re: Accelerating Y2038 glibc fixes

[Lukasz Majewski <lukma at denx dot de>]

Hi Rich,

Thanks for sharing your view.

> On Wed, Jul 17, 2019 at 04:15:25PM +0200, Arnd Bergmann wrote:
> > On Tue, Jul 16, 2019 at 4:52 PM Wolfgang Denk <wd@denx.de> wrote:  
> > > In message <874l3mjgi6.fsf@oldenburg2.str.redhat.com> you wrote:
> > >  
> > > > Since I'm opposed to this entire project, I have largely
> > > > refrained from reviews, except for things that looked obviously
> > > > wrong to me (e.g., things that definitely break compatibility
> > > > with older kernels or existing ABIs), but even for those cases,
> > > > my feedback probably wasn't very helpful.  
> > >
> > > I can understand your point of view, but indeed this is not
> > > helpful here.  glibc is a central resource for the whole FOSS
> > > system, and we are not pushing these changes for a fancy, but
> > > because a solution is needed for a large number of affected
> > > projects and products.
> > >
> > > We need to find a constructive way to proceed with this matter.
> > > It will not go away if we try to ignore it.  
> > 
> > My impression is that we have two mostly separate groups
> > of users for 32-bit user space:
> > 
> > a) Those that absolutely require 100% ABI compatibility to run
> >     legacy code but will eventually migrate to 64-bit user space,
> >     almost certainly within the coming ten years before we really
> >     need to worry (too much) about things like key expiration dates
> >     beyond 2038.
> > 
> > b) Those that already need support for 64-bit time_t because
> >     they are deploying new 32-bit binaries that are expected to run
> >     beyond 2038, while not caring at all about compatibility
> >     with existing binaries that are already known to be broken
> >     for this purpose.
> > 
> > If we just work to satisfy both these groups but ignore the
> > intersection (those that have existing 32-bit binaries and want
> > those to run after 2038?), things could become noticeably easier.  
> 
> Well it replaces one sticky problem with two separate, individually
> easier problems. But I'm not sure it reduces the overall amount of
> work needed; it might increase it.
> 
> The above roughly corresponds to what my original idea for musl and
> the ".2 ABI" was, and I'm now fairly skeptical of it. Binary distros
> are probably not going to want to expend extra effort to satisfy the
> needs of both groups, so they'll either EOL the 32-bit ports they have
> now (effectively leaving group (a) screwed, not because glibc dropped
> them, but because their distro did) to produce something that will
> continue to be relevant into the future, or they'll stick with the old
> ABI and EOL 32-bit entirely in the near-ish future.
> 
> The segment of group (b) that's building everything themselves
> probably doesn't care in the latter case, but then do they even care
> about using glibc (vs something else) anyway?

If I may add my 2$ - I think they care. Otherwise there wouldn't be the
effort to adapt "upstream first" policy with Y2038 support.

> 
> From this analysis it seems like, if you attack the problems
> separately, "no action" is the right choice for distros, but that also
> leads to EOL for distro support of all existing 32-bit targets.

But as it was mentioned in the other mail. There are _real_ users of 32
bit targets, so distros may re-consider dropping them.

> 
> As a result, I'm leaning strongly towards a conclusion that a solution
> that doesn't split the two cases is better, and it's probably what
> we'll do for musl.

The approach would be very similar to what was done with
_FILE_OFFSET_BITS==64. The end user would need to add -D_TIME_BITS==64
when compiling programs against glibc on 32 bit architecture.

> 
> > The price for that is a mid-2030s EOL time on any existing 32-bit  
> 
> That's really wishful thinking. I suspect we'll hit critical failures
> much sooner due to need to represent dates in the future.
> 
> > glibc ports, any binaries linked to them, and any distros built upon
> > that, and replacing them with something else that is binary
> > incompatible but can coexist within e.g. a container.  
> 
> I'm not sure how "coexist within a container" is helpful; that might
> as well just be a separate system.
> 1 Multi-arch type approaches might
> work a little bit better, but in order for these to be useful to end
> users, there need to be automated ways of setting them up. That's a
> big burden for distros to spend on archs that many of them would
> rather just EOL/drop...
> 
> > There are several options what such a replacement might look
> > like, some that come to mind would include:
> > 
> > - A patchset against glibc that replaces all the time32 interfaces
> >   with time64 ones and that is maintained outside of the glibc
> >   code base by whoever is interested in this working.  
> 
> If you're trying to use incompatible old and new ABI alongside each
> other, I would think you only want to do that as long as you still
> have old stuff around. But if you're using patches with ad-hoc ABI
> that have bypassed the process of upstream review to ensure that
> they're making good decisions compatible with the long term, it's
> likely that it's going to keep breaking ABI, and you're going to need
> to do multiple painful replacements or keep N ABIs alongside each
> other rather than 2 (possibly using something like nix to make it
> practical).
> 
> So I don't think "use out-of-tree patches for time64 support" is a
> good approach.

I do agree here - the "out-of-tree" is not a good approach. Basically
one would need to rebase those "hacks" on top of each glibc release.

If you are wondering how such set of "out-of-tree" patches ("hacks" is
a better word I think) look like - please check following link [1].

They provide support for Y2038 on 32 bit arch (and even passes some
tests) [2].

> 
> > - A set of glibc architecture ports that can coexist with the
> >   existing ports but have new target triples and/or soname.
> >   These could be implemented and merged one architecture
> >   at a time and share most of the code with the existing
> >   ports.  
> 
> This is basically the same proposal as the musl ".2 ABI", and it's one
> of the least-bad, but as explained above I no longer really favor it.
> 
> > - Leave glibc unchanged and require users/distros to migrate to
> >   musl or something else if they need the time64 support.  
> 
> As much as I'd like this, I don't think distros that have stability
> contracts with their users for how things work can replace the libc
> with one which has significantly different behaviors in application-
> and user-facing ways.

+1

> It's a viable solution for users to run new
> 32-bit stuff alongside a 32-bit glibc target that's being phased-out
> and nearing EOL, but if this is the only option I think it would just
> lead to distros dropping the targets.
> 
> > Any of the above approaches (including the current plan of
> > supporting time32 and time64 in a single glibc binary) can of
> > coexist, but the more incompatible approaches get implemented,
> > the more fragmentation of ABIs we get.  
> 
> I'm moderately in favor of supporting both in a single binary. As
> noted elsewhere, this fundamentally does not break ABI between libc
> consumers (main apps or 3rd party libs) and libc, only between pairs
> of libc consumers that have used the affected types as part of their
> public interfaces.

As I've written above. For the end user it shall be enough to add
-D_TIME_BITS==64 during compilation.

> 
> To answer Wolfgang Denk's original question, in my opinion, the single
> most valuable thing for accelerating the time64 process that third
> parties could contribute is detailed analysis of the interfaces that
> will break.

Such analysis is already done and available on glib'c wiki page [3].

> This (1) will give us an idea of the extent of breakage
> that could happen in the worst case, and (2) can be translated
> directly into a graph of "requires/conflicts-with" relationships
> between library packages that a distro can use to ensure consistency
> when updating packages.

As stated above - please visit [3].

> 
> Are others willing to start a discussion of exactly what form this
> analysis should take, how results should be reported, and how to use
> them to make such a transition go smoothly?

There is even the OE/Yocto's meta-y2038 layer (thud YPRR) [4] , which
allows building glibc for testing with:

1. Kernel 5.1+ (with 64 bit time syscalls support on 32 bit archs). 

2. Kernel < 5.1 (to check if it works with older syscalls)

3. A set of tests for syscalls mentioned in [3][2] (with and without
Y2038 support in glibc)

4. Several different architectures for testing (qemu):
	- ARM 32 bit - vexpress-a9
	- i386 / x86_64
	- x32 - the "special" one
	- All the OE/Yocto supported qemu* targets (mips, ppc)


5. One can run the glibc xtest with mentioned above qemu running system
with test-wrapper='/opt/Y2038/glibc/src/scripts/cross-test-ssh.sh
(detailed instruction in [4]).

6. It is also possible to run plain glibc tests for archs mentioned in
point 4 above.

> 
> > Note that we have never had such a forced transition on x86, but
> > there are several examples on other architectures:
> > - arm oabi/gnueabi/gnueabihf
> > - ppc64/ppc64le
> > - mips o32/n32/64
> > The migration is not much fun for users to say the least, but it's
> > quite possible that the alternative is even worse.  
> 
> I wouldn't characterize all of those as transitions, more as different
> options. As far as I know glibc still supports them all except ARM
> OABI, which was dropped long before there was serious interest in
> end-user-facing Linux on ARM.
> 
> Rich


Note:

[1] -
https://github.com/lmajewski/y2038_glibc/commits/Y2038-2.29-glibc-11-03-2019

[2] - https://github.com/lmajewski/y2038-tests

[3] -
https://sourceware.org/glibc/wiki/Y2038ProofnessDesign?highlight=%28y2038%29

[4] - https://github.com/lmajewski/meta-y2038/tree/master


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

Attachment: pgp9Fuq9aWSIA.pgp
Description: OpenPGP digital signature