nptl: Return EINVAL for pthread_mutex_clocklock/PI with CLOCK_MONOTONIC [BZ #26801]
Linux futex FUTEX_LOCK_PI operation only supports CLOCK_REALTIME,
so pthread_mutex_clocklock operation with priority aware mutexes
may fail depending of the input timeout.
Also, it is not possible to convert a CLOCK_MONOTONIC to a
CLOCK_REALTIME due the possible wall clock time change which might
invalid the requested timeout.
nptl: Replace __futex_clocklock_wait64 with __futex_abstimed_wait64
For non null timeouts, the __futex_clocklock_wait64 creates an a
relative timeout by subtracting the current time from the input
argument. The same behavior can be obtained with FUTEX_WAIT_BITSET
without the need to calculate the relative timeout. Besides consolidate
the code it also avoid the possible relative timeout issues [1].
The __futex_abstimed_wait64 needs also to return EINVAL syscall
errors.
It can be replaced with a __futex_abstimed_wait_cancelable64 call,
with the advantage that there is no need to further clock adjustments
to create a absolute timeout. It allows to remove the now ununsed
futex_timed_wait_cancel64 internal function.
The __futex_abstimed_wait usage was remove with 3102e28bd11 and the
__futex_abstimed_wait_cancelable by 323592fdc92 and b8d3e8fbaac.
The futex_lock_pi can be replaced by a futex_lock_pi64.
Matheus Castanho [Mon, 19 Oct 2020 14:06:56 +0000 (11:06 -0300)]
powerpc: Make PT_THREAD_POINTER available to assembly code
PT_THREAD_POINTER is currenty defined inside a #ifndef __ASSEMBLER__ block, but
its usage should not be limited to C code, as it can be useful when accessing
the TLS from assembly code as well.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Stefan Liebler [Mon, 23 Nov 2020 09:51:24 +0000 (10:51 +0100)]
Use libnss_files.so for tests posix/bug-ga2 and resolv/tst-leaks2 [BZ #26821]
The tests posix/bug-ga2-mem and resolv/mtrace-tst-leaks2 are failing on
fedora 33 as mtrace reports memory leaks.
The /etc/nsswitch.conf differs between
Fedora 32: hosts: files dns myhostname
Fedora 33: hosts: files resolve [!UNAVAIL=return] myhostname dns
Therefore /lib64/libnss_resolve.so.2 (from systemd) and the dependencies
libgcc_s.so.1 and libpthread.so.0 are loaded.
Usually all malloc'ed resources from getaddrinfo / gethostbyname are freed
and the libraries are dlclose'd in nss/nsswitch.c:libc_freeres_fn (free_mem).
Unfortunately, /lib64/libnss_resolve.so.2 is marked with DF_1_NODELETE.
As this library is not unmapped, you'll see "Memory not freed".
Therefore those tests are now only relying on libnss_files.so by making
them test-container tests and providing the required configuration files.
By moving the tests to tests-container, those are now running with
"make check". Therefore the mtrace part of the tests are also moved
from "make xcheck" to "make check".
bug-ga2.c is now using test-driver.c in order to support WAIT_FOR_DEBUGGER
environment variable.
Samuel Thibault [Mon, 23 Nov 2020 00:31:41 +0000 (00:31 +0000)]
hurd report-wait: Fix stpcpy usage
We shall not overflow the size of the description parameter. This makes
describe_number and describe_port behave like strpcpy (except for not filling
all the end of buffer with zeroes) and _S_msg_report_wait use series of
stpncpy-like call. If we were to overflow, we can now detect it and
return ENOMEM.
Samuel Thibault [Sun, 22 Nov 2020 23:15:36 +0000 (23:15 +0000)]
hurd: Fix strcpy calls
strcpy cannot be used with overlapping buffer, we have to use memmove
instead. strcpy also cannot be safely used when the destination buffer
is smaller that the source, we need to use strncpy to truncate the
source if needed.
Samuel Thibault [Fri, 13 Nov 2020 19:37:41 +0000 (19:37 +0000)]
Add {,sysdep-}ld-library-path make variable
On GNU/Hurd we not only need $(common-objpfx) in LD_LIBRARY_PATH when loading
dynamic objects, but also $(common-objpfx)/mach and $(common-objpfx)/hurd. This
adds an ld-library-path variable to be used as LD_LIBRARY_PATH basis in
Makefiles, and a sysdep-ld-library-path variable for sysdeps to add some
more paths, here mach/ and hurd/.
Florian Weimer [Mon, 16 Nov 2020 18:33:30 +0000 (19:33 +0100)]
nptl: Move stack list variables into _rtld_global
Now __thread_gscope_wait (the function behind THREAD_GSCOPE_WAIT,
formerly __wait_lookup_done) can be implemented directly in ld.so,
eliminating the unprotected GL (dl_wait_lookup_done) function
pointer.
Lukasz Majewski [Thu, 12 Nov 2020 11:24:04 +0000 (12:24 +0100)]
nanosleep: Pass NULL when rem == NULL on ports with __TIMESIZE != 64
On ports with __TIMESIZE != 64 the remaining time argument always receives
pointer to struct __timespec64 instance. This is the different behavior
when compared to 64 bit versions of clock_nanosleep and nanosleep
functions, which receive NULL.
To avoid any potential issues, we also pass NULL when *rem pointer is
NULL.
Reported-by: Andreas Schwab <schwab@suse.de> Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Lukasz Majewski [Wed, 11 Nov 2020 23:24:48 +0000 (00:24 +0100)]
y2038: Convert thrd_sleep to support 64 bit time
The thrd_sleep function has been converted to support 64 bit time.
It was also necessary to provide Linux specific copy of it to avoid
problems on i686-gnu (i.e. HURD) port, which is not providing
clock_nanosleep() supporting 64 bit time.
The thrd_sleep is a wrapper on POSIX threads to provide C11 standard
threads interface. It directly calls __clock_nanosleep64().
Lukasz Majewski [Wed, 11 Nov 2020 23:09:03 +0000 (00:09 +0100)]
y2038: Convert mtx_timedlock to support 64 bit time
The mtx_timedlock function has been converted to support 64 bit time.
It was also necessary to provide Linux specific copy of it to avoid
problems on i686-gnu (i.e. HURD) port, which is not providing
pthread_mutex_timedlock() supporting 64 bit time.
The mtx_timedlock is a wrapper on POSIX threads to provide C11 standard
threads interface. It directly calls __pthread_mutex_timedlock64().
Lukasz Majewski [Thu, 29 Oct 2020 13:19:25 +0000 (14:19 +0100)]
y2038: Convert cnd_timedwait to support 64 bit time
The cnd_timedwait function has been converted to support 64 bit time.
It was also necessary to provide Linux specific copy of it to avoid
problems on i686-gnu (i.e. HURD) port, which is not providing
pthread_cond_timedwait() supporting 64 bit time.
Moreover, a linux specific copy of thrd_priv.h header file has been
added as well.
The cnd_timedwait is a wrapper on POSIX threads to provide C11 standard
threads interface. It directly calls __pthread_cond_timedwait64().
Samuel Thibault [Fri, 13 Nov 2020 15:56:51 +0000 (15:56 +0000)]
hurd: break relocation loop between libc.so and lib{mach,hurd}user.so
See
https://sourceware.org/pipermail/libc-alpha/2020-November/119575.html
lib{mach,hurd}user.so gets relocated before libc.so, but its references
to strpcpy and memcpy would need an ifunc decision, which e.g. on
x86 relies on cpu_features, but libc.so's _rtld_global_ro is not
relocated yet.
We can however just make lib{mach,hurd}user.so only call non-ifunc
functions, which can be relocated before libc.so is relocated.
Florian Weimer [Fri, 13 Nov 2020 14:19:41 +0000 (15:19 +0100)]
nptl: Eliminate <smp.h> and __is_smp
Most systems are SMP, so optimizing for the UP case is no longer
approriate. A dynamic check based on the kernel identification
has been only implemented for i386 anyway.
To disable adaptive mutexes on sh, define DEFAULT_ADAPTIVE_COUNT
as zero for this architecture.
Samuel Thibault [Fri, 13 Nov 2020 10:23:08 +0000 (10:23 +0000)]
hurd: Make sure signals get started
Now that _hurd_libc_proc_init is idempotent, we can always call it,
independently of the __libc_multiple_libcs test which may not match
whether signals should be started or not.
The builtin has been available in gcc since 4.7.0 and in clang since
2.6. This fixes stpncpy fortification with clang since it does a
better job of plugging in __stpncpy_chk in the right place than the
header hackery.
This has been tested by building and running all tests with gcc 10.2.1
and also with clang tip as of a few days ago (just the tests in debug/
since running all tests don't work with clang at the moment) to make
sure that both compilers pass the stpncpy tests.
Samuel Thibault [Wed, 11 Nov 2020 12:52:35 +0000 (12:52 +0000)]
hurd: Notify the proc server later during initialization
Notifying the proc server is an involved task, and unleashes various signal
handling etc. so we have to do this after e.g. ifunc relocations are
completed.
Florian Weimer [Wed, 11 Nov 2020 10:59:11 +0000 (11:59 +0100)]
struct _Unwind_Exception alignment should not depend on compiler flags
__attribute__((__aligned__)) selects an alignment that depends on
the micro-architecture selected by GCC flags. Enabling vector
extensions may increase the allignment. This is a problem when
building glibc as a collection of ELF multilibs with different
GCC flags because ld.so and libc.so/libpthread.so/&c may end up
with a different layout of struct pthread because of the
changing offset of its struct _Unwind_Exception field.
Samuel Thibault [Tue, 10 Nov 2020 23:35:19 +0000 (23:35 +0000)]
hurd: keep only required PLTs in ld.so
We need NO_RTLD_HIDDEN because of the need for PLT calls in ld.so.
See Roland's comment in
https://sourceware.org/bugzilla/show_bug.cgi?id=15605
"in the Hurd it's crucial that calls like __mmap be the libc ones
instead of the rtld-local ones after the bootstrap phase, when the
dynamic linker is being used for dlopen and the like."
We used to just avoid all hidden use in the rtld ; this commit switches to
keeping only those that should use PLT calls, i.e. essentially those defined in
sysdeps/mach/hurd/dl-sysdep.c:
Samuel Thibault [Tue, 10 Nov 2020 23:50:04 +0000 (23:50 +0000)]
hurd: Add missing startup calls
DL_SYSDEP_INIT and DL_PLATFORM_INIT were not getting called, leading to
missing x86 platform tuning, now mandatory with 0f09154c6400
("x86: Initialize CPU info via IFUNC relocation [BZ 26203]")
Zong Li [Thu, 29 Oct 2020 17:33:38 +0000 (01:33 +0800)]
riscv: Get cache information through sysconf
Add support to query cache information on RISC-V through sysconf()
function. The cache information had been added in AUX vector of RISC-V
architecture in Linux kernel v.5.10-rc1. Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>
linux: Allow adjtime with NULL argument [BZ #26833]
The adjtime interface allows return the amount of time remaining
from any previous adjustment that has not yet been completed by
passing a NULL as first argument. This was introduced with y2038
support 0308077e3a.
Samuel Thibault [Tue, 26 May 2020 20:20:10 +0000 (22:20 +0200)]
Rearrange bsd_getpt vs bsd_openpt and implement posix_openpt on BSD
* sysdeps/unix/bsd/getpt.c (__getpt): Add oflag parameter, pass
it to the _open call and rename to...
(__bsd_openpt): ... new function.
(__getpt): Reimplement on top of __bsd_openpt.
(__posix_openpt): Replace stub with implementation on top of __bsd_openpt.
(posix_openpt): Remove stub warning.
Non-gcc compilers (clang and possibly other compilers that do not
masquerade as gcc 5.0 or later) are unable to use
__warn_memset_zero_len since the symbol is no longer available on
glibc built with gcc 5.0 or later. While it was likely an oversight
that caused this omission, the fact that it wasn't noticed until
recently (when clang closed the gap on _FORTIFY_SUPPORT) that the
symbol was missing.
Given that both gcc and clang are capable of doing this check in the
compiler, drop all remaining signs of __warn_memset_zero_len from
glibc so that no more objects are built with this symbol in future.
Arjun Shankar [Wed, 4 Nov 2020 11:19:38 +0000 (12:19 +0100)]
iconv: Accept redundant shift sequences in IBM1364 [BZ #26224]
The IBM1364, IBM1371, IBM1388, IBM1390 and IBM1399 character sets
share converter logic (iconvdata/ibm1364.c) which would reject
redundant shift sequences when processing input in these character
sets. This led to a hang in the iconv program (CVE-2020-27618).
This commit adjusts the converter to ignore redundant shift sequences
and adds test cases for iconv_prog hangs that would be triggered upon
their rejection. This brings the implementation in line with other
converters that also ignore redundant shift sequences (e.g. IBM930
etc., fixed in commit 692de4b3960d).
Lukasz Majewski [Thu, 22 Oct 2020 11:09:00 +0000 (13:09 +0200)]
msg: Remove redundant #include <sys/msg.h> header
The #include <sys/msg.h> is redundant as we do not use message specific
types for issuing syscalls to handle msg and shm. Only msgctl requires
this header.
Szabolcs Nagy [Mon, 26 Oct 2020 15:48:01 +0000 (15:48 +0000)]
aarch64: Add variant PCS lazy binding test [BZ #26798]
This test fails without bug 26798 fixed because some integer registers
likely get clobbered by lazy binding and variant PCS only allows x16
and x17 to be clobbered at call time.
The test requires binutils 2.32.1 or newer for handling variant PCS
symbols. SVE registers are not covered by this test, to avoid the
complexity of handling multiple compile- and runtime feature support
cases.
The variant PCS support was ineffective because in the common case
linkmap->l_mach.plt == 0 but then the symbol table flags were ignored
and normal lazy binding was used instead of resolving the relocs early.
(This was a misunderstanding about how GOT[1] is setup by the linker.)
In practice this mainly affects SVE calls when the vector length is
more than 128 bits, then the top bits of the argument registers get
clobbered during lazy binding.
Joseph Myers [Fri, 30 Oct 2020 22:25:42 +0000 (22:25 +0000)]
Avoid -Wstringop-overflow warning in pthread_cleanup_push macros
GCC 11 introduces a -Wstringop-overflow warning for calls to functions
with an array argument passed as a pointer to memory not large enough
for that array. This includes the __sigsetjmp calls from
pthread_cleanup_push macros, because those use a structure in
__pthread_unwind_buf_t, which has a common initial subsequence with
jmp_buf but does not include the saved signal mask; this is OK in this
case because the second argument to __sigsetjmp is 0 so the signal
mask is not accessed.
To avoid this warning, use a function alias __sigsetjmp_cancel with
first argument an array of exactly the type used in the calls to the
function, if using GCC 11 or later. With older compilers, continue to
use __sigsetjmp with a cast, to avoid any issues with compilers
predating the returns_twice attribute not applying the same special
handling to __sigsetjmp_cancel as to __sigsetjmp.
Tested with build-many-glibcs.py for arm-linux-gnueabi that this fixes
the testsuite build failures.
Joseph Myers [Fri, 30 Oct 2020 21:40:25 +0000 (21:40 +0000)]
Disable spurious -Warray-bounds for ypclnt.c (bug 26687)
Included among the GCC 11 warnings listed in bug 26687, but not fixed
when that bug was marked as FIXED, are -Warray-bounds warnings in
nis/ypclnt.c. These are all for different calls to the same piece of
code, which already has a comment explaining that the element accessed
is in a common prefix of the various structures. On the basis of that
comment, this patch treats the warning as a false positive and
disables it for that code.
Tested with build-many-glibcs.py for arm-linux-gnueabi, where,
together with my previous two patches, this allows the build of glibc
to complete with GCC 11 (further build failures appear in the
testsuite).
Joseph Myers [Fri, 30 Oct 2020 21:39:12 +0000 (21:39 +0000)]
Do not use array parameter to new_composite_name (bug 26726)
Among the warnings causing a glibc build with GCC 11 to fail is one
for a call new_composite_name in setlocale.c. The newnames argument
is declared as an array with __LC_LAST elements, but when the category
argument is not LC_ALL, it actually only has one element. Since the
number of elements depends on the first argument to the function, it
seems clearer to declare the argument as a pointer.
Tested with build-many-glibcs.py for arm-linux-gnueabi, where this
allows the build to get further.
Joseph Myers [Fri, 30 Oct 2020 21:38:31 +0000 (21:38 +0000)]
Disable spurious -Wstringop-overflow for setjmp/longjmp (bug 26647)
Building glibc with GCC 11 fails with (among other warnings) spurious
-Wstringop-overflow warnings from calls to setjmp and longjmp with a
pointer to a pthread_unwind_buf that is smaller than jmp_buf. As
discussed in bug 26647, the warning in libc-start.c is a false
positive, because setjmp and longjmp do not access anything (the
signal mask) beyond the common prefix of the two structures, so this
patch disables the warning for that call to setjmp, as well as for two
calls in NPTL code that produce the same warning and look like false
positives for the same reason.
Tested with build-many-glibcs.py for arm-linux-gnueabi, where this
allows the build to get further.
the reproduction is like that:
setp1: modify related Makefile.
vim ../glibc/malloc/Makefile
CPPFLAGS-malloc.o += -DMALLOC_DEBUG=2
step2: ../configure --prefix=/usr
make -j32
this will cause the compile error:
/home/liqingqing/glibc_upstream/buildglibc/malloc/malloc.o
In file included from malloc.c:1899:0:
arena.c: In function 'dump_heap':
arena.c:422:58: error: 'struct malloc_chunk' has no member named 'size'
fprintf (stderr, "chunk %p size %10lx", p, (long) p->size);
^~
arena.c:428:17: error: 'struct malloc_chunk' has no member named 'size'
else if (p->size == (0 | PREV_INUSE))
Érico Rolim [Tue, 27 Oct 2020 14:17:41 +0000 (11:17 -0300)]
Make elf.h header self contained.
The elf/elf.h header is shared, verbatim, by the elfutils project.
However, elfutils can be used on systems with libcs other than glibc,
making the presence of __BEGIN_DECLS, __END_DECLS and <features.h> in
the file something that downstream distros may have to add patches for.
Furthermore, this file doesn't declare anything with language linkage,
so `extern "C" {}` blocks aren't necessary; it also doesn't have any
conditional definitions based on feature test macros, making inclusion
of features.h unnecessary.
Sajan Karumanchi [Wed, 28 Oct 2020 07:35:33 +0000 (13:05 +0530)]
x86: Optimizing memcpy for AMD Zen architecture.
Modifying the shareable cache '__x86_shared_cache_size', which is a
factor in computing the non-temporal threshold parameter
'__x86_shared_non_temporal_threshold' to optimize memcpy for AMD Zen
architectures.
In the existing implementation, the shareable cache is computed as 'L3
per thread, L2 per core'. Recomputing this shareable cache as 'L3 per
CCX(Core-Complex)' has brought in performance gains.
As per the large bench variant results, this patch also addresses the
regression problem on AMD Zen architectures.
Joseph Myers [Mon, 26 Oct 2020 16:40:50 +0000 (16:40 +0000)]
Update kernel version to 5.9 in tst-mman-consts.py.
This patch updates the kernel version in the test tst-mman-consts.py
to 5.9. (There are no new MAP_* constants covered by this test in 5.9
that need any other header changes.)
Stefan Liebler [Mon, 26 Oct 2020 09:51:28 +0000 (10:51 +0100)]
Remove timing related checks of time/tst-cpuclock1
Starting with the commit 04deeaa9ea74b0679dfc9d9155a37b6425f19a9f
"Fix time/tst-cpuclock1 intermitent failures" (2020-07-11),
this test fails quite often on s390x/s390 with one/multiple of those:
"before - after" / "nanosleep time" / "dead - after" ourside reasonable range.
On a zVM/kvm guest the CPUs are shared between multiple guests.
And even on the lpar (kvm host) the CPUs are usually shared between multiple lpars.
The defined CPUs for a lpar/zVM-system could also have lower weights compared
to other lpars which let the steal time further grow.
Usually I build (-j$(nproc)) and test (PARALLELMFLAGS="-j$(nproc)") glibc multiple
times, e.g. with different GCCs, on various lpars or zVM guests at the same time.
During this time, I've run the test for 13500 times and obvserved the following fails:
~600x "before - after"
~60x "nanosleep time"
~70x "dead - after"
I've also observed a lot of "before - after" fails on a intel kvm-guest while
building/testing glibc on it.
The test itself forks a child process which chew_cpu (user- and kernel-space).
The parent process sleeps with nanosleep(0.5s) and measures the child_clock time:
diff = after - before
With much workload on the machine, the child won't make much progess
and it can fall much beyond the minimum limit.
Afterwards the parent process sleeps with clock_nanosleep (child_clock, 0.1s):
diff = afterns - after
The test currently also allows 0.9 * 0.1s which would be an error.
Depending on the workload, the maximum limit can exceed the 1.2 * 0.1s.
For "dead - after", the parent process kills the child process and waits long
enough to let the child finish dying. Then it gets the time of the child:
diff = dead - after
Note that diff also contains the time for the previous clock_nanosleep.
Thus you'll often see both fails at the same time.
After discussion on the mailing list, we've decided to keep the functional
checks for the clock* functions and remove the timing related checks as those
are prone to false positives.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Joseph Myers [Fri, 23 Oct 2020 16:31:11 +0000 (16:31 +0000)]
Update syscall lists for Linux 5.9.
Linux 5.9 has one new syscall, close_range. Update syscall-names.list
and regenerate the arch-syscall.h headers with build-many-glibcs.py
update-syscalls.