Re: [PATCH 1/1] Y2038: add function __difftime64

[Albert ARIBAUD <albert dot aribaud at 3adev dot fr>]

Hello all,

On Wed, 27 Jun 2018 13:03:42 +0200, Albert ARIBAUD
<albert.aribaud@3adev.fr> wrote :

> Hi Paul,
> 
> On Mon, 25 Jun 2018 16:56:14 -0700, Paul Eggert <eggert@cs.ucla.edu>
> wrote :
> 
> > On 06/25/2018 03:32 PM, Albert ARIBAUD wrote:  
> > > I replaced the original 32-bit difftime with a wrapper
> > > around the 64-bit time; and in both cases, there were two functions, on
> > > for each time size.    
> > 
> > Yes, and all this is in the patch you posted in 
> > <https://www.sourceware.org/ml/libc-alpha/2018-06/msg00605.html>. What 
> > I'm not understanding is how the functions it defines (__difftime64 and 
> > __difftime on 32-bit hosts, and just __difftime on 64-bit hosts) 
> > interoperate the with user-defined macro (is it still _TIME_BITS? I 
> > can't recall) that specifies whether the user wants time_t is 64 or 32 
> > bits, and how they interoperate with the macro that Joseph Myers 
> > suggested was needed by the system to say whether a 32-bit time_t 
> > variant is supported.
> > 
> > In short, I'm still missing the bigger picture here.  
> 
> OK, so let me try to sum things up (I'll update the design document
> accordingly).
> 
> Right now, on 32-bit architectures, application source code which uses
> glibc and manipulates time can only go as far as Y2038, because the
> public API declares only 32-bit types (time_t and related structures)
> and functions (difftime etc). Any time beyond Y2038 is undefined
> behavior  -- I've documented behaviors ranging from just time rollover
> (back to Dec. 1901 IIRC) to full system freeze.
> 
> My goal is:
> 
> - to make glibc provide, when _TIME_BITS is is defined equal to 64, the
>   same set of type and function names but with 64-bit time support (and
>   with _TIME_BITS undefined or defined equal to 32, glibc would stick
>   with the old types and functions);
> 
> - to ensure that, at least for now, the existing ABI remain unchanged,
>   i.e. application modules resulting from compiling against current
>   glibc should link against a 64-bit-time-capable glibc and run
>   without errors.
> 
> In other words, source code referring to difftime would result in
> object code referring either to the existing __difftime symbol if
> _TIME_BITS is not equal to 64, or to some new __difftime64 symbol if
> _TIME_BITS is equal to 64; and the symbol __difftime in glibc should
> always represent the existing 32-bit implementation so that existing,
> already compiled, client code would execute the difftime they expect
> to.
> 
> How to achieve this:
> 
> - since a single glibc header install must support compiling both
>   32-bit-time and new 64-bit-time object code, glibc must provide two
>   sets of public API symbol definitions, one for 32-bit-time, exposed
>   if _TIME_BITS is defined equal to 32 or undefined, and one for 64-bit
>   time, exposed if _TIME_BITS is defined equal to 64.
> 
> - since a single glibc library must support both existing 32-bit-time
>   and new 64-bit-time object code, glibc must therefore provide two
>   sets of implementation symbols at the same time: the current set for
>   object client code which was compiled for 32-bit time, and a new set
>   for object client code which was compiled for64-bit time.
> 
> Refining the how:
> 
> - we cannot introduce the public API and _TIME_BITS support in small
>   batches, because if you switch time_t to 64-bit, that switches all
>   functions using time_t to 64-bit too, and all types using time_t as
>   well. Basically, _TIME_BITS and the switch of the API to 64-bit time
>   has to be a single, atomic, patch.
> 
> - but we can introduce the new 64-bit symbols, including the
>   publically visible ones, even though we won't expose them as they
>   will be in the end. These new symbols do not need to be all introduced
>   in a single patch; this can be done progressively.
> 
> For instance, in the end, we will have to map the public API's difftime
> symbol either to the existing 32-bit __difftime or to a new 64-bit
> __difftime64, depending on _TIME_BITS. So we will /have/ to provide a
> public symbol __difftime64. Only in the final _TIME_BITS patch will
> this symbol be aliased with 'difftime' -- for client code which compiles
> with _TIME_BITS=64.
> 
> Further refining the how:
> 
> - We can create 64-bit implementations from existing 32-bit ones, with
>   some hand-optimizing based on time_t assumptions which we know
>   always / never apply to __time64_t (such as possibly being integer or
>   being floating point) or size considerations (such as some precision
>   loss risks which might not apply to __time64_t).
> 
> - We can/must rewrite some internals of glibc to handle __time64_t
>   rather than time_t.
> 
> - Once there is a 64-bit implementation for a given functionality in
>   32-bit architectures, then the 32-bit implementation can be turned
>   into a wrapper around the 64-bit one (if the 32-bit implementation
>   is even needed; for 64-bit architectures and x32, it does not need
>   to be defined at all).
> 
>   This is where we need another macro, __TIME_SIZE. While _TIME_BITS
>   tells us, at client code compile time, whether client code wants a
>   64-bit time API or not, __TIMESIZE tells us, at glibc compile time,
>   whether the time_t for the considered architecture is 64-bit or not.
>   Architectures with 64-bit time_t do not need 32-bit wrappers, and can
>   use time_t for their __time64_t; others need the 32-bit wrappers, and
>   need separate time_t and __time64_t.
> 
> Final picture:
> 
> - the set of patches to support Y2038 will consist in new types and new
>   functions (either visible from the public or purely internal to
>   glibc) or internal glibc function conversions to 64 bit; these
>   patches will have some of the implementation dependent on __TIMESIZE.
>   Then in the end a single patch will expose the 'old' or 'new' set
>   depending on _TIME_BITS. 
> 
> Is this clear enough? Don't hesitate to ask for further clarifications.
> 
> > > Since you want the changes in gnulib first, then I suspect I should
> > > provide branches above gnulib as well as above glibc? If so, what
> > > would you recommend as a good source on setting up a build and test
> > > setup for gnulib, similar to build-many-glibcs.py is for glibc?  
> > 
> > Yes, it would be helpful to have this in Gnulib too. The basic idea is 
> > that Gnulib and glibc sources should be as close to each other as 
> > possible; preferably identical. You can build and test a Gnulib module 
> > by running './gnulib-tool --test modulename'; run './gnulib-tool --help' 
> > for more.  
> 
> Since gnulib is on Savannah, not Sourceware, I assume I will need to be
> given some level of write access to the Savannah gnulib repository in
> order to provide branches there too, similar to what is done in glibc.
> Who should I ask this?

Cross-posting this to both the glibc and gnulib lists for now; if anyone
thinks this thread should stay on only gnulib, feel free.

I have had a look at gnulib in the meantime, and I would like to know
if the following assumptions are correct:

- gnulib does not contain any module which provides the time_t type, but
  some of gnulib's modules assume there is such a type, and it might be
  wider than 32 bits.

- gnulib contains a year2038 module which is only intended to check
  whether time_t is limited to Y2038 or not.

- gnulib does not provide difftime either, so ATM a difftime patch
  would only make sense in glibc, not gnulib.

- gnulib makes no assumption on how it is used in object form; notably,
  it makes no assumption that it will be compiled into a shared object
  form which will provide the same functionalities for both 64-bit and
  32-bit callers.

Are all these correct, and if not, could someone develop in what
respect they aren't?

Cordialement,
Albert ARIBAUD
3ADEV