• Y2038ProofnessDesign

Clear message

DRAFT: Y2038 Proofness Design

---

History

Revision 24 was reviewed on the libc-alpha mailing list as 1st draft: https://www.sourceware.org/ml/libc-alpha/2015-10/msg00893.html

Revision 55 was reviewed on the libc-alpha mailing list as 2nd draft: https://www.sourceware.org/ml/libc-alpha/2016-01/msg00832.html

Revision 63 was reviewed on the libc-alpha mailing list as 3rd draft: https://www.sourceware.org/ml/libc-alpha/2016-06/msg00243.html

Revision 83 was reviewed on the libc-alpha mailing list as 4th draft: https://www.sourceware.org/ml/libc-alpha/2016-06/msg00824.html

Scope

The intent of this page is to serve as a central point for describing the Y2038 'proofness' design.

Y2038 'proofness' means that application calls to glibc-provided function should never return wrong results when UTC times outside -2³¹..2³¹-1 seconds from the Unix Epoch are involved.

This document is only about Y2038 (and Y1901) and is not about any other time boundaries such as Y2106 (unsigned 32-bit Epoch-based times) or Y9999 (four-digit years).

For example, asctime_r in practice might possibly not be Y9999-proof (for instance, it might overrun a too-small output buffer). However, this document is not about Y9999; it is about Y2038, for which asctime_r is safe since it does not handle Epoch-based 32-bit signed values.

Useful Definitions

• 32-bit time denotes a representation of time as a 32-bit signed quantity of seconds relative to the Unix Epoch of 1970-01-01 00:00:00 UTC.

• 64-bit time denotes a representation of time as a 64-bit signed quantity of seconds relative to the Unix Epoch of 1970-01-01 00:00:00 UTC.

• Y2038 denotes the problem where a 32-bit time value rolls over beyond +2147483647 into negative values.

  • This will happen to the system clock time at 2038-01-19 03:14:07 UTC (possibly throwing it back to 1901-12-13 20:45:52 UTC).

  • This will also happen if, on 2038-01-16 12:00:00 UTC, the command at now + 5 days is executed.

  • etc.

• Y2038-incompatible means 'unable to hold (for types) or handle (for APIs) values representing post-Y2038 dates.

• Y2038-compatible means 'able to hold (for types) or handle (for APIs) values representing post-Y2038 dates.

Goals

• Provide Y2038-compatible 64-bit time support to 32-bit.
• Keep providing legacy Y2038-incompatible 32-bit time support to 32-bit.

Constraints

There are a number of constraints which dictate the direction of the design. They are either definite or debatable. Debatable design preclusions should be finalized before this design document leaves DRAFT.

Definite

• 64-bit time would be supported as a replacement of 32-bit time; user code (including libraries) would require either 32-bit or 64-bit time support, with 32-bit remaining the default for now.

• Existing 32-bit API *symbols* would remain unmodified so that existing binary code would link properly. New 32-bit symbols would be named after their corresponding 32-bit symbol plus "64",

  • e.g. the 64-bit implementation of clock_gettime would be named clock_gettime64.

• To ensure the same source code would compile properly for either 32- or 64-bit time, when defaulting to 64-bit time support, the 32-bit API and type names would be #define'ed to their 64-bit counterparts,

  • e.g. #define clock_gettime clock_gettime64.

• Any newly introduced function MUST be declared in the implementation, not user, namespace (see https://sourceware.org/bugzilla/show_bug.cgi?id=14106).

• Any newly introduced struct MUST have a name (see https://sourceware.org/bugzilla/show_bug.cgi?id=15766).

Debatable

• 64-bit time should only be supported if 64-bit file offsets are. This sounds like something totally unrelated, but the underlying idea is to not multiply the possible combinations.
• fixing 32-bit time support is not a goal; 32-bit time functions which returns wrong results now will keep returning wrong results after introduction of 64-bit time support.

• Make sure no change affects 64-bit platforms where time_t is already 64-bit (see http://www.sourceware.org/ml/libc-alpha/2015-08/msg00038.html).

• No security fixes to existing 32-bit time code in this feature development:

  • It can be argued that overflowing 32-bit time could be used as an attack vector against 32-bit applications. Such applications running on 64-bit kernels could be improved to detect overflow conditions. For example the stat64 kernel syscall could be enhanced to return EOVERFLOW when time conversion overflows (Bug 1419736: 32-bit stat returns wrong st_mtime if file timestamp does not fit in 32 bits)

  • Fixes need not be all-or-nothing and each API could be taken on a case-by-case basis and fixed as best as possible given the interfaces. The previous example of stat64 fixes in the linux kernel can be used to fix 32-bit stat when called while running on a 64-bit kernel. Running 32-bit on 64-bit kernel is quickly becoming the most common form of running 32-bit applications (ignoring IoT applications which will need 64-bit time).
  • However, security fixes in general should not be part of a feature development, but rather should be fast-tracked outside of any long-term project.

  • But of course, any security flaw uncovered during implementation of Y2038 support will be raised at once.

Analysis

This section details the problems with Y2038 and glibc.

Y2038-incompatible types and APIs

glibc provides many APIs which involve time. Among these:

• some are already Y2038-compatible, and do not need to be modified in order to handle dates beyond Y2038. An example of such an already Y2038-compatible API is char *asctime(const struct tm *tm).

• some others are Y2038-incompatible: they need to be modified in order to handle dates beyond Y2038. An example of such a Y2038-incompatible API is time_t mktime(struct tm *tm).

• some others yet do not need modifications but their internal implementation many need it; see below.

This sections describes the Y2038-incompatible glibc APIs, which need changes in order to become Y2038-compatible.

Y2038-incompatible types

These are types whose values include a date and which cannot properly denote dates past Y2038.

The essential example is time_t, which is a 32-bit signed integer count of seconds since the Epoch; its maximum value is Y2038.

By extension, any struct containing at least one time_t is not Y2038-compatible.

Y2038-incompatible APIs

APIs which do not involve any Y2038-incompatible type are Y2038-compatible, whereas APIs which involve at least one Y2038-incompatible type are Y2038-incompatible.

This applies to APIs which involve a pointer to a Y2038-incompatible type.

Below is the list of Y2038-incompatible APIs, based on the list of Y2038-incompatible types.

Y2038-incompatible socket timestamping

Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are passing time data:

SO_TIMESTAMP, SO_TIMESTAMPNS and SO_TIMESTAMPING enable socket timestamping, which allows socket messages to be augmented with timestamps through cmsgs. More precisely:

• SO_TIMESTAMP uses a struct timeval as its cmsg data;

• SO_TIMESTAMPNS uses a struct timespec as its cmsg data;

• SO_TIMESTAMPING uses an integer bitfield which is not sensitive to time bit-size.

Getting Y2038-compatible timestamping affects both the kernel and glibc:

The kernel must provide Y2038-compatible timestamps. This would be done indirectly through 64-bit versions of struct timeval and struct timespec.

However, to prevent incompatibilities with existing 32-bit-time code, the existing socket options would remain in place and new options SO_TIMESTAMP64, SO_TIMESTAMPNS64 and SO_TIMESTAMPING64 would be adde which would use Y2038-compatible timestamps.

glibc would use either the old Y2038-incompatible set of socket options or the new Y2038-compatible set depending on __USE_TIME_BITS64, and provide the correct 'struct timeval' and 'struct timespec' so that application source code works for both 32 and 64 bits time size.

SO_RCVTIMEO and SO_SNTTIMEO are socket options for controlling socket time-outs. They are passed along with a pointer to a struct timeval another argument specifying the struct's length.

This length could be used to determine whether a Y2038-incompatible and Y2038-compatible struct timeval was passed, but there is no way to distinguish a struct timeval by size between a true 64-bit kernel and a 32-bit kernel with 64-bit time support. To avoid risks, these options whould be handled the same way as SO_TIMESTAMP*, using new numbers for the flags.

NOTE: at the moment, GLIBC defines absolutely no SO_TIMESTAMP* macros. To introduce time-size-insensitive versions discussed above, these would have to be introduced in bits/sockets.h.

Y2038-incompatible IOCTLs

Some Linux IOCTLs may be Y2038-incompatible, or use types defined by glibc that do not match the kernel internal types. Known important cases are:

• An ioctl command number is defined using the _IOR/_IOW/_IORW macros by the kernel with a structure whose size changes based on glibc's time_t.

  • The kernel can handle these transparently by implementing handlers for both command numbers with the correct structure format. glibc can keep a single implementation for both 32-bit and 64-bit time cases.

• The ABI changes based on the glibc time_t size, but the command number does not change.

  • In such cases, a new command number should be defined when __USE_TIME_BITS64 is set, and the situation becomes similar to the previous case.

• An ioctl command passes time information in a structure that is not based on time_t but another integer type that does not get changed.

  • [AA - which IOCTLs would that be?]

    The kernel header files will provide both a new structure layout and command number when __USE_TIME_BITS64 is set.

GLIBC itself uses some ioctls (which can be found by running git grep 'ioctl *(' -- "*.c" | sed -e 's/^.*ioctl *([^,]*, *\([^,)]*\)[,)].*$/\1/' on the GLIBC source repository, minus some false positives). None seem to be time-sensitive.

Y2038-compatible types and APIs

This section lists type and APIs which are time-related but are Y2038-compatible

Internal changes

Some functions use time_t internally, and may thus be Y2038-incompatible and require changes in order to become Y2038-compatible. The following table lists these functions:

Implementation

General principles

In order to avoid duplicating APIs for 32-bit and 64-bit time, glibc will provide either one but not both for a given application; the application code will have to choose between 32-bit or 64-bit time support, and the same set of symbols (e.g. time_t or clock_gettime) will be provided in both cases.

The following is proposed:

• User code defines _TIME_BITS=64 to get 64-bit time support instead of the legacy 32-bit time.

• If glibc sees _TIME_BITS=64, then it defines __USE_TIME_BITS64 to indicate that time support is 64-bit rather than 32-bit.

This allows user code to check, by verifying whether __USE_TIME_BITS64 is defined once glibc headers are #included, that they are using a glibc which actually supports 64-bit time (or claims to).

• On 64-bit systems, only 64-bit time is supported, __USE_TIME_BITS64 is defined regardless of _TIME_BITS, and the same symbols are used as they were in 32-bit time.

• On 32-bit systems, if __USE_TIME_BITS64 is defined, time support is provided for 64-bit time; otherwise, it is provided for 32-bit time.

For instance, providing 64-bit time support for the time() function would result in the following:

• on a 64-bit system, glibc would provide an already Y2038-compatible time_t and a Y2038-compatible time_t time (time_t *result)

• on a 32-bit system with 32-bit time support glibc would continue providing the existing Y2038-incompatible 32-bit time_t and time_t time (time_t *result)

• on a 32-bit system with 64-bit time support glibc would provide a Y2038-compatible 64-bit time_t and a Y2038-compatible time_t time (time_t *result) BUT time() and stime() might still be unable to handle time_t values past Y2038.

Y2038-compatible types

This sections list examples of Y2038-compatible types with their rationale

Y2038-compatible time_t

time_t is a 32-bit signed Epoch-related count of seconds, and is Y2038-incompatible because Y2038 is the maximum positive value of time_t.

Y2038-compatible APIs will need a type which can hold Epoch-related counts of seconds beyond Y2038; the most sensible choice for this type is 64-bit signed integer (denoted time64_t from now on), because:

• 64-bit Epoch-based time in seconds is already what 64-bit systems use, and

• all valid time_t values are also valid for time64_t and have the same semantics.

Note that Posix currently does not mandate any specific time representation for handling the Y2038 problem, and while it considers a 64-bit time_t approach, it also considers just mandating that a future date be representable, leaving implementaters free to choose the details of said representation.

In order for applications to be written (as) independently of time size (as possible), there needs to be three 'time types' in GLIBC:

• the newly introduced 64-bit time, denoted time64_t, for use in 64-bit-time syscalls as well as 64-bit-time derived types;

• the existing 32-bit time, currently time_t, renamed to time32_t, for use in existing 32-bit-time syscalls as well as 32-bit-time derived types;

• an application-compile-time-defined type denoted time_t, equal to either time32_t or time64_t depending on which default time bit-size the application code requests.

Y2038-compatible struct timespec

struct timespec contains two fields, a time_t called tv_sec and a long called tv_nsec

The time_t should be replaced by a time64_t in order for post-Y2038 dates to be represented.

The long type of tv_nsec is mandated by Posix. Also, even for 64-bit time, the number of microseconds within a second does not need more than 32 bits of representation, and could remain unchanged.

On the other hand, on 64-bit systems, both fields are 64 bits (which makes sense as a long is 64 bit on these systems), and it would make sense to use the same physical layout for 32-bit systems handling 64-bit time.

Therefore a possible implementation would be

struct timespec {
  time64_t tv_sec;
  long     tv_nsec;
  long     padding;
};

for little-endian systems, and

struct timespec64 {
  time64_t tv_sec;
  long     tv_nsec;
  long     padding;
};

for big-endian systems.

Similar to time_t, the existing struct timespec could be renamed struct timespec32, and struct timespec would be defined at application compile time as either struct timespec32 or struct timespec64 depending on _TIME_BITS==64.

Y2038-compatible struct timeval

struct timeval contains two fields, a time_t called tv_sec and a suseconds_t called tv_usec

The time_t should be replaced by a time64_t in order for post-Y2038 dates to be represented.

The suseconds_t type of tv_usec is mandated by Posix to be able to hold numbers in the range [-1, 1000000], but the actual size is left to implementors.

As there is no mandated size for suseconds_t, and as on 64-bit systems it is 64 bits, it would make sense to use 64 bits on 32-bit systems handling 64-bit time.

Therefore a possible implementation would be

struct timeval64 {
  time64_t tv_sec;
  int64_t  tv_nsec;
};

for both little- and big-endian systems.

However, for the sake of maximum performance, tv_nsec could be defined as a signed 32-bits field with a signed 32-bit padding, similar to struct timespec, in order to allow operations involving tv_nsec to use 32-bit arithmetic rather than the slower 64-bit arithmetic; but then, glibc would have to sign-extend tv_nsec into its padding in every struct timeval received from application code.

Similar to time_t and struct timespec, the existing struct timeval could be renamed struct timeval32, and struct timeval would be defined at application compile time as either struct timeval32 or struct timeval64 depending on _TIME_BITS==64.

Y2038-compatible APIs

This sections list examples of Y2038-compatible APIs with their rationale

difftime()

double difftime(time_t time1, time_t time0) takes two date/tyime values and returns their difference.

From an API standpoint, it is Y2038-incompatible because :

• dates beyond Y2038 cannot be passed to it through time1 or time0;

• if time1 and time0 could hold 64-bit time, then their difference might not be able to fit in a double

There is therefore a need for a 64-bit counterpart to difftime. Argument types are obvious:

double difftime64(time64_t time1, time64_t time0)

However, the double result type might not hold enough mantissa bits for a 64-bit difference, which could cause imprecision of the result. In order to keep the function as precise as possible, its result would have to be extended:

long double difftime64(time64_t time1, time64_t time0)

But then, application code using this API would have to be written differently for 32-bit and 64-bit time cases.

Here, we will assume that difftime is used to compute time differences which will never be bigger than what a double can hold in its mantissa (which, for a 64-bit GCC, is empirically about 15 decimal digits, which allows encoding about 999 999 999 999 999 seconds or roughly 31 million years).

We will thus choose this definition:

double difftime64(time64_t time1, time64_t time0)

As with types, the existing difftime would be renamed difftime32 and a difftime symbol would be declared at application compile time to be either difftime32 or difftime64 depending on the time bit-size default chosen at application compile-time.

Thus, since time_t would also be either time32_t or time64_t depending on _TIME_BITS==64, the whole difftime(time_t,time_t) would actually be difftime32(time32_t,time32_t) when _TIME_BITS is not equal to 64, and difftime64(time64_t,time64_t) when _TIME_BITS==64.

clock_gettime()

As far as the API is concerned, (clock_gettime(clockid_t, struct timespec *) would be duplicated the same way as difftime, with a 32-bit-callable clock_gettime32(clockid_t,struct timespec32 *) and a 64-bit-callable clock_gettime64(clockid_t,struct timespec64 *), plus a clock_gettime symbol defined as either clock_gettime32 or clock_gettime64.

Also, as GLIBC should work on 32-bit-time as well as 64-bit-time kernels, both clock_gettime32 and clock_gettime64 would have to determine whether the kernel is Y2038-compatible and thus provides 64-bit clock_gettime/clocksettime syscalls, and try these first; only if they failed with ENOSYS would GLIBC try the 32-bit syscalls.

The implementations would also have to handle the cases where:

• A 32-bit client wants to get the time but the kernel is Y2038-compatible and the current date is post-Y2038 and thus cannot fit in a time32_t: in this case, clock_gettime should return -1 and set errno to EOVERFLOW.

• A 64-bit client wants to set the time to a date post-Y2038 but the kernel is not Y2038-compatible: in this case, clock_settime should return -1 and set errno to EOVERFLOW.

Issues

Build-time kernel/glibc incompatibilities

There is an open problem whereby incompatible kernel and glibc headers might be used at build time, e.g. new, 64-bit-time-enabled, kernel headers and old, 32-bit-time-only-capable, glibc headers.

In this case, there will be a mismatch between what the glibc uses and what the kernel expects; for instance, the kernel will expect a 64-bit time_t while glibc will use a 32-bit time_t.

The kernel cannot detect this at all, as kernel headers should not rely on glibc defines.

Older (pre-Y2038-support) glibcs cannot detect this either.

Such a detection will need to rest on an application-level mechanism.

Run-time kernel/glibc incompatibilities

Whatever time_t size the client code asks, the glibc code will rely on the running kernel which might provide only 32-bit time, or only 64-bit time, or both (providing neither is impossible).

Therefore, glibc Y2038-compatible functions must try and use 64-bit syscalls, which may fail with ENOSYS, in which case they should fall back to 32-bit syscalls, which may also fail with ENOSYS (but then the 64-bit syscall would have succeeded).

In either cases, 64-bit time 'rvalues' may not fit in 32-bit time 'lvalues'. For instance, calling the 64-bit-time gettimeofday() on behalf of a 32-bit-time application will work until Y2038, at which point glibc will have to return EOVERFLOW. Concersively, a 64-bit application can call settimeofday() on a 32-bit-time kernel until Y2038, at which point glibc will return EOVERFLOW.

utmp types and APIs

The issue of utmp is distinct from the issue of time_t size, because utmp is essentially unrelated to the kernel, and essentially concerned with the structure of the utmp, wtmp and btmp files.

In order to support several glibcs running on the same file system, the utmp/wtmp/btmp file format must be defined independently from the architecture word size and endianness.

On the other hand, the utmp API must conform to POSIX, which means the argument sizes and endianness will vary from architecture to architecture.

In addition, there must be a way to transition from a system's current format to the future single format for the utmp, wtmp and btmp files.

The proposed solution would implement the following changes:

• Introduce a Y2038-compatible structure for 'new' utmp file records, with field ut_tv.tv_sec defined as a signed 64-bit int, ideally with an explicit endianness.
• Make the API utmp[x] struct's ut_tv field a struct timeval as expected by POSIX.
• Whenever necessary, translate between struct utmp[x] values and utmp file records; when translating a 64-bit tv_sec into a 32-bit tv_sec, if the value would overflow, then set errno to EOVERFLOW and return a failure value.
• At run time, check for file utmp.trans. If it exists, then use file utmp to read and write the utmp state in 'old' format, and keep a 'new' format copy in utmp.trans. If utmp.trans does not exist, then use utmp to read and write the utmp state in the 'new' format. Also, if a 'new' GLIBC detects that changed were made in the utmp file but were not also made in the new utmp.trans file because the changes were performed by an 'old' GLIBX, then the 'new' GLIBC should copy the changes over from the utmp file to the utmp.trans file on behalf of the 'old' GLIBC.
• System upgrade from 'old' to 'new' format would be as follows: once all glibcs on the system are 'new' ones, all able to handle the 'new' format, and all system applications are ready to handle the new utmp format, then the utmp.trans file should be renamed into the utmp file (thus deleting the old utmp content).

References

• Y2038 information: http://kernelnewbies.org/y2038

• Y2038 Linux patch series (WIP): https://lists.linaro.org/pipermail/y2038/2015-May/000233.html