DRAFT: Y2038 Proofness Design
Contents
History
Revision 24 was reviewed on the libc-alpha mailing list as 1st draft: https://www.sourceware.org/ml/libc-alpha/2015-10/msg00893.html
Revision 55 was reviewed on the libc-alpha mailing list as 2nd draft: https://www.sourceware.org/ml/libc-alpha/2016-01/msg00832.html
Revision 63 was reviewed on the libc-alpha mailing list as 3rd draft: https://www.sourceware.org/ml/libc-alpha/2016-06/msg00243.html
Scope
The intent of this page is to serve as a central point for describing the Y2038 proofness design.
Y2038 proofness means that application calls to glibc-provided function should never return wrong results when UTC times outside -231..231-1 seconds from the Unix Epoch are involved.
This document is only about Y2038 (and Y1901) and is not about any other time boundaries such as Y2106 (unsigned 32-bit Epoch-based times) or Y9999 (four-digit years).
For example, asctime_r in practice might possibly not be Y9999-safe (for instance, it might overrun a too-small output buffer). However, this document is not about Y9999; it is about Y2038, for which asctime_r is safe since it does not handle Epoch-based 32-bit signed values.
Useful Definitions
32-bit time denotes a representation of time as a 32-bit signed quantity of seconds relative to the Unix Epoch of 1970-01-01 00:00:00 UTC;
64-bit time denotes a representation of time as a 64-bit signed quantity of seconds relative to the Unix Epoch of 1970-01-01 00:00:00 UTC;
Y2038 denotes the problem where a 32-bit time value rolls over beyond +2147483647 into negative values.
- This will happen to the system clock time at 2038-01-19 03:14:07 UTC (possibly throwing it back to 1901-12-13 20:45:52 UTC);
This will also happen if, on 2038-01-16 12:00:00 UTC, the command at now + 5 days is executed.
- etc.
Goals
- Provide Y2038-safe 64-bit time support to 32-bit.
- Keep providing legacy Y2038-unsafe 32-bit time support to 32-bit.
Constraints
There are a number of constraints which dictate the direction of the design. They are either definite or debatable. Debatable design preclusions should be finalized before this design document leaves DRAFT.
Definite
64-bit time would be supported as a replacement of 32-bit time; user code (including libraries) would require either 32-bit or 64-bit time support, with 32-bit remaining the default for now.
Any newly introduced function MUST be declared in the implementation, not user, namespace (see https://sourceware.org/bugzilla/show_bug.cgi?id=14106).
Any newly introduced struct MUST have a name (see https://sourceware.org/bugzilla/show_bug.cgi?id=15766).
Debatable
- 64-bit time should only be supported if 64-bit file offsets are.
This sounds like something totally unrelated, but the underlying idea is to not multiply the possible combinations.
- 32-bit time support will not be modified; 32-bit time functions which returns wrong results now will keep returning wrong results after introduction of 64-bit time support.
IOW, this design does not aim at trying to fix existing 32-bit time support at all; any fix to the Y2038 problem will be within new 64-bit time support.
Make sure no change affects 64-bit platforms where time_t is already 64-bit (see http://www.sourceware.org/ml/libc-alpha/2015-08/msg00038.html).
API changes
This sections describes the API changes required for supporting Y2038 in GLIBC.
Y2038-unsafe types and APIs
This section lists all Y2038-unsafe GLIBC types and APIs, which might fail to handle calendar times beyond Y2038; whether they actually fails or succeeds will be determined by analyzing the implementation.
Y2038-unsafe types
Y2038-unsafe types |
Types containing an Expoch-based value |
time_t |
Types using struct time_t |
struct lastlog |
struct msqid_ds |
struct semid_ds |
struct timeb |
struct timespec |
struct timeval |
struct utimbuf |
Types using struct timespec |
struct itimerspec |
struct stat |
Types using struct timeval |
struct clnt_ops |
struct elf_prstatus |
struct itimerval |
struct ntptimeval |
struct rusage |
struct timex |
struct utmp |
struct utmpx |
Y2038-unsafe APIs
Y2038-unsafe APIs |
APIs using time_t |
char * ctime (const time_t *) |
char * ctime_r (const time_t *, char *) |
double difftime (time_t, time_t) |
struct tm * gmtime (const time_t *) |
struct tm * gmtime_r (const time_t *, struct tm *resultp) |
struct tm * localtime (const time_t *) |
struct tm * localtime_r (const time_t *, struct tm *) |
time_t mktime (struct tm *brokentime) |
int stime (const time_t *) |
time_t timegm (struct tm *brokentime) |
time_t timelocal (struct tm *brokentime) |
time_t time (time_t *result) |
APIs using struct lastlog |
(none) |
APIs using struct msqid_ds |
int msgctl(int, int, struct msqid_ds *) |
APIs using struct semid_ds |
(none) |
APIs using struct timeb |
int ftime(struct timeb *) |
APIs using struct timespec |
int aio_suspend (const struct aiocb *const *, int, const struct timespec *restrict) |
int clock_getres (clockid_t, struct timespec *) |
int clock_gettime (clockid_t, struct timespec *) |
int clock_nanosleep (clockid_t, int, const struct timespec *, struct timespec *) |
int clock_settime (clockid_t, const struct timespec *) |
int futimens (int, const struct timespec *) |
ssize_t mq_timedreceive (mqd_t, char *restrict, int, unsigned int *restrict, const struct timespec *restrict) |
int mq_timedsend (mqd_t, const char *, int, unsigned int, const struct timespec *) |
int nanosleep (const struct timespec *, struct timespec *) |
int pselect (int, fd_set *restrict, fd_set *restrict, fd_set *restrict, const struct timespec *restrict, const __sigset_t *restrict) |
int pthread_cond_timedwait (pthread_cond_t *restrict, pthread_mutex_t *restrict, const struct timespec *restrict) |
int pthread_mutex_timedlock (pthread_mutex_t *restrict, const struct timespec *restrict) |
int pthread_rwlock_timedrdlock (pthread_rwlock_t *restrict, const struct timespec *restrict) |
int pthread_rwlock_timedwrlock (pthread_rwlock_t *restrict, const struct timespec *restrict) |
int sched_rr_get_interval (__pid_t, struct timespec *) |
int sem_timedwait (sem_t *restrict, const struct timespec *restrict) |
int sigtimedwait (const sigset_t *restrict, siginfo_t *restrict, const struct timespec *restrict) |
int timespec_get (struct timespec *, int) |
int utimensat (int, const char *, const struct timespec *, int) |
APIs using struct timeval |
int adjtime (const struct timeval *, struct timeval *) |
enum clnt_stat pmap_rmtcall (struct sockaddr_in *, const u_long, const u_long, const u_long, xdrproc_t, caddr_t, xdrproc_t, caddr_t, struct timeval, u_long *) |
CLIENT * clntudp_bufcreate (struct sockaddr_in *, u_long, u_long, struct timeval, int *, u_int, u_int) |
CLIENT * clntudp_create (struct sockaddr_in *, u_long, u_long, struct timeval, int *) |
int futimes (int, const struct timeval *) |
int gettimeofday (struct timeval *restrict, __timezone_ptr_t) |
int lutimes (const char *, const struct timeval *) |
int select (int, fd_set *restrict, fd_set *restrict, fd_set *restrict, struct timeval *restrict) |
int settimeofday (const struct timeval *, const struct timezone *) |
int utimes (const char *, const struct timeval *) |
APIs using struct utimbuf |
int utime(const char *, const struct utimbuf *) |
APIs using struct itimerspec |
int timerfd_gettime(int, struct itimerspec *) |
int timerfd_settime(int, int, const struct itimerspec *, struct itimerspec *) |
int timer_gettime(timer_t, struct itimerspec *) |
int timer_settime(timer_t, int, const struct itimerspec *restrict, struct itimerspec *restrict) |
APIs using struct stat |
int fstatat(int, const char *restrict, struct stat *restrict, int) |
int fstat(int, struct stat *) |
int __fxstatat(int, int, const char *, struct stat *, int) |
int __fxstat(int, int, struct stat *) |
int lstat(const char *restrict, struct stat *restrict) |
int __lxstat(int, const char *, struct stat *) |
int stat(const char *restrict, struct stat *restrict) |
int __xstat(int, const char *, struct stat *) |
APIs using struct clnt_ops |
(none) |
APIs using struct elf_prstatus |
(none) |
APIs using struct itimerval |
struct itimerval |
int setitimer (int which, const struct itimerval *new, struct itimerval *old) |
int getitimer (int which, struct itimerval *old) |
APIs using struct ntptimeval |
int ntp_gettime(struct ntptimeval *) |
APIs using struct rusage |
int getrusage(__rusage_who_t, struct rusage *) |
__pid_t wait3(int *, int, struct rusage *) |
__pid_t wait4(__pid_t, int *, int, struct rusage *) |
APIs using struct timex |
int adjtimex (struct timex *timex) |
int ntp_adjtime (struct timex *tptr) |
APIs using struct utmp |
int getutent_r(struct utmp *, struct utmp **) |
struct utmp * getutent() |
int getutid_r(const struct utmp *, struct utmp *, struct utmp **) |
struct utmp * getutid(const struct utmp *) |
int getutline_r(const struct utmp *, struct utmp *, struct utmp **) |
struct utmp * getutline(const struct utmp *) |
void login(const struct utmp *) |
struct utmp * pututline(const struct utmp *) |
void updwtmp(const char *, const struct utmp *) |
APIs using struct utmpx |
struct utmpx * getutxent() |
struct utmpx * getutxid(const struct utmpx *) |
struct utmpx * getutxline(const struct utmpx *) |
struct utmpx * pututxline(const struct utmpx *) |
Y2038-safe types and APIs
This section lists type and APIs which are time-related but are Y2038-safe.
Time-related but Y2038-safe types |
struct rpc_timeval |
struct tm |
Time-related but Y2038-safe APIs |
APIs using struct rpc_timeval |
int rtime(struct sockaddr_in *, struct rpc_timeval *, struct rpc_timeval *) |
APIs using struct tm |
char * asctime (const struct tm *brokentime) |
char * asctime_r (const struct tm *brokentime, char *buffer) |
size_t strftime (char *s, size_t size, const char *template, const struct tm *brokentime) |
size_t wcsftime (wchar_t *s, size_t size, const wchar_t *template, const struct tm *brokentime) |
char * strptime (const char *s, const char *fmt, struct tm *tp) |
struct tm * getdate (const char *string) |
int getdate_r (const char *string, struct tm *tp) |
Others |
unsigned int alarm (unsigned int seconds) -- TBC depending on implementation |
unsigned int sleep (unsigned int seconds) -- TBC depending on implementation |
clock_t clock(void) |
struct tms { clock_t tms_utime; clock_t tms_stime; clock_t tms_cutime; clock_t tms_cstime; } |
clock_t times(struct tms *buffer) |
struct timezone { int tz_minuteswest; int tz_dsttime; } |
rlim_t |
|| int sched_rr_get_interval (pid_t pid, struct timespec *interval)
IOCTLs and Y2038
Some Linux IOCTLs may be Y2038-unsafe. However, at the GLIBC level, none of the IOCTLs argument types described are time_t or derived from it.
ABI changes
Time-related ABI changes
On 64-bits systems, time_t is already 64 bit wide Therefore, 64-bit applications should be able to directly use the (already 64-bit) time ABI, and a 64-bit GLIBC needs only provide 64-bit time support (applications might have internal bugs in their handling of time_t values, but this is not a GLIBC concern; as long as applications can exchange 64-bit time_t values with GLIBC and GLIBC processes them correctly, the existing "64-bit-time_t ABI" is safe)
On 32-bit systems, the size of time_t will be different for 32-bit and 64-bit time support. Consequently, structures containing time_t will have different sizes and field offsets. Therefore, the 64-bit-time ABI will be incompatible with the 32-bit-time ABI; and in order to remain backward-compatible with existing application binaries, a 32-bit GLIBC must keep providing the 32-bit time ABI unchanged.
Other ABI changes
(to be completed)
Internal changes
Some functions use time_t internally, may be Y2038-unsafe, and may require changes in order to become Y2038-safe. These are:
Y2038-unsafe internal functions |
addgetnetgrentX(struct database_dyn *, int, int *, const char *, uid_t, struct hashentry *, struct datahead *, struct dataset **) |
addgrbyX(struct database_dyn *, int, request_header *, union keytype, const char *, uid_t, struct hashentry *, struct datahead *) |
addhstaiX(struct database_dyn *, int, request_header *, void *, uid_t, struct hashentry *const, struct datahead *) |
addhstbyX(struct database_dyn *, int, request_header *, void *, uid_t, struct hashentry *, struct datahead *) |
addinitgroupsX(struct database_dyn *, int, request_header *, void *, uid_t, struct hashentry *const, struct datahead *) |
addinnetgrX(struct database_dyn *, int, int *, char *, uid_t, struct hashentry *, struct datahead *) |
addpwbyX(struct database_dyn *, int, request_header *, union keytype, const char *, uid_t, struct hashentry *, struct datahead *) |
addservbyX(struct database_dyn *, int, request_header *, char *, uid_t, struct hashentry *, struct datahead *) |
bigtime_test(int) |
cache_addgr(struct database_dyn *, int, request_header *, const void *, struct group *, uid_t, struct hashentry *const, struct datahead *, int) |
cache_addhst(struct database_dyn *, int, request_header *, const void *, struct hostent *, uid_t, struct hashentry *const, struct datahead *, int, int32_t) |
cache_addpw(struct database_dyn *, int, request_header *, const void *, struct passwd *, uid_t, struct hashentry *const, struct datahead *, int) |
cache_addserv(struct database_dyn *, int, request_header *, const void *, struct servent *, uid_t, struct hashentry *const, struct datahead *, int) |
ctime_r(const time_t *, char *) |
dbg_log(const char *, ...) |
__difftime(time_t, time_t) |
do_notfound(struct database_dyn *, int, int *, const char *, struct dataset **, ssize_t *, time_t *, char **) |
do_prepare(int, char **) |
do_test() |
evConsTime(struct timespec *, time_t, long) |
first_shoot(const_nis_name, directory_obj *) |
ftime(struct timeb *) |
__getdate_r(const char *, struct tm *) |
__get_nprocs() |
getopt(int, char *const *, const char *) |
__gettimeofday(struct timeval *, struct timezone *) |
__gmtime_r(const time_t *, struct tm *) |
guess_time_tm(long_int, long_int, int, int, int, const time_t *, const struct tm *) |
help_filter(int, const char *, void *) |
hunt(timezone_t, char *, time_t, time_t) |
libc_freeres_ptr(time_t *) |
__localtime_r(const time_t *, struct tm *) |
main() |
main(int, char **) |
main_loop_poll() |
mktime_internal(struct tm *, struct tm *(*)(const time_t *, struct tm *), time_t *) |
mktime_test1(time_t) |
mktime_test(time_t) |
my_gmtime_r(time_t *, struct tm *) |
my_localtime_r(const time_t *, struct tm *) |
nscd_run_prune(void *) |
__offtime(const time_t *, long, struct tm *) |
quantize_timeval(struct timeval *) |
ranged_convert(struct tm *(*)(const time_t *, struct tm *), time_t *, struct tm *) |
restart() |
restart_p(time_t) |
show(timezone_t, char *, time_t, int) |
__sleep(unsigned int) |
spring_forward_gap() |
start_threads() |
__strptime_internal(const char *, const char *, struct tm *, void *) |
subtract(time_t, time_t) |
tformat() |
thread_burn_any_cpu(void *) |
time_t_add_ok(time_t, time_t) |
time_t_avg(time_t, time_t) |
time_t_int_add_ok(time_t, int) |
__tzfile_compute(time_t, int, long *, int *, struct tm *) |
__tzfile_read(const char *, int, char **) |
utime(const char *, const struct utimbuf *) |
verify_persistent_db(void *, struct database_pers_head *, int) |
xcalloc(int, int) |
ydhms_diff(long_int, long_int, int, int, int, int, int, int, int, int) |
yeartot(intmax_t) |
zdump_localtime_rz(timezone_t, time_t *, struct tm *) |
Implementation
In order to avoid duplicating APIs for 32-bit and 64-bit time, GLIBC will provide either one but not both for a given application; the application code will have to choose between 32-bit or 64-bit time support, and the same set of symbols (e.g. time_t or clock_gettime) will be provided in both cases.
The following is proposed:
User code defines _TIME_BITS=64 to get 64-bit time support instead of the legacy 32-bit time.
If glibc sees _TIME_BITS=64, then it defines __USE_TIME_BITS64 to indicate that time support is 64-bit rather than 32-bit.
This allows user code to check, by verifying whether __USE_TIME_BITS64 is defined once glibc headers are #included, that they are using a GLIBC which actually supports 64-bit time (or claims to).
On 64-bit systems, only 64-bit time is supported, __USE_TIME_BITS64 is defined regardless of _TIME_BITS, and the same symbols are used as they were in 32-bit time.
On 32-bit systems, if __USE_TIME_BITS64 is defined, time support is provided for 64-bit time; otherwise, it is provided for 32-bit time.
For instance, providing 64-bit time support for the time() function would result in the following:
on a 64-bit system, GLIBC would provide an already Y2038-safe time_t and a Y2038-safe time_t time (time_t *result)
on a 32-bit system with 32-bit time support GLIBC would continue providing the existing Y2038-unsafe 32-bit time_t and time_t time (time_t *result)
on a 32-bit system with 64-bit time support GLIBC would provide a Y2038-safe 64-bit time_t and a Y2038-safe time_t time (time_t *result)
There is an open problem whereby incompatible kernel and GLIBC headers might be used, e.g. new, 64-bit-time, kernel headers and old, 32-bit-time, GLIBC headers. In this case, there will be a mismatch between what the GLIBC uses and what the kernel expects; for instance, the kernel will expect a 64-bit time_t while GLIBC will use a 32-bit time_t.
Support for non-Y2038-safe kernels
When running above 32-bit kernels without 64-bit time support, GLIBC will emulate a 64-bit time application API using the 32-bit kernel time API, converting 64-bit GLIBC types to and from 32-bit kernel types. Out-of-range conversions will be notified to application as appropriate using EOVERFLOW.
This will allow 64-bit-time applications to run on older kernels until (around) 2038 or until these kernels are retired, whichever happens first.
Notes
* tv_usec: to be addressed. May need GLIBC to convert structs.
* tv_nsec size: to be addressed, needs to be long as per Posix / C11.
* The implementation needs further thinking about, as application code defining _TIME_BITS=64 and gets built against
new kernel headers and old GLIBC headers, then GLIBC will use 32-bit time_t and kernel will expect 64-bit time_t, and there is no way to ensure detection of this case.
References
* Y2038 information: http://kernelnewbies.org/y2038
* Y2038 Linux patch series (WIP): https://lists.linaro.org/pipermail/y2038/2015-May/000233.html