Frederic Berat [Mon, 12 Jun 2023 15:18:21 +0000 (17:18 +0200)]
tests: replace fgets by xfgets
With fortification enabled, fgets calls return result needs to be checked,
has it gets the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Frederic Berat [Mon, 12 Jun 2023 15:18:20 +0000 (17:18 +0200)]
tests: replace fread by xfread
With fortification enabled, fread calls return result needs to be checked,
has it gets the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The tst-mallocfork2 and tst-mallocfork3 create large number of
subprocesss, around 11k for former and 20k for latter, to check
for malloc async-signal-safeness on both fork and _Fork. However
they do not really exercise allocation patterns different than
other tests fro malloc itself, and the spawned process just exit
without any extra computation.
The tst-malloc-tcache-leak is similar, but creates 100k threads
and already checks the resulting with mallinfo.
These tests are also very sensitive to system load (since they
estresss heavy the kernel resource allocation), and adding them
on THP tunable and mcheck tests increase the pressure even more.
For THP the fork tests do not add any more coverage than other
tests. The mcheck is also not enable for tst-malloc-tcache-leak.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
There is no fork detection on current arc4random implementation, so
use lower subprocess on fork tests. The tests now run on 0.1s
instead of 8s on a Ryzen9 5900X.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
tst-getdate: Improve testcase flexibility and add test.
The getdate testcases all expect successful results. Add support for
negative testcases and testcases where a full date and time are not
supplied by skipping the tm checks in the test. Add a testcase that
would catch a use-after-free that was recently found.
Noah Goldstein [Wed, 7 Jun 2023 18:18:03 +0000 (13:18 -0500)]
x86: Make the divisor in setting `non_temporal_threshold` cpu specific
Different systems prefer a different divisors.
From benchmarks[1] so far the following divisors have been found:
ICX : 2
SKX : 2
BWD : 8
For Intel, we are generalizing that BWD and older prefers 8 as a
divisor, and SKL and newer prefers 2. This number can be further tuned
as benchmarks are run.
[1]: https://github.com/goldsteinn/memcpy-nt-benchmarks Reviewed-by: DJ Delorie <dj@redhat.com>
Noah Goldstein [Wed, 7 Jun 2023 18:18:02 +0000 (13:18 -0500)]
x86: Refactor Intel `init_cpu_features`
This patch should have no affect on existing functionality.
The current code, which has a single switch for model detection and
setting prefered features, is difficult to follow/extend. The cases
use magic numbers and many microarchitectures are missing. This makes
it difficult to reason about what is implemented so far and/or
how/where to add support for new features.
This patch splits the model detection and preference setting stages so
that CPU preferences can be set based on a complete list of available
microarchitectures, rather than based on model magic numbers. Reviewed-by: DJ Delorie <dj@redhat.com>
Noah Goldstein [Wed, 7 Jun 2023 18:18:01 +0000 (13:18 -0500)]
x86: Increase `non_temporal_threshold` to roughly `sizeof_L3 / 4`
Current `non_temporal_threshold` set to roughly '3/4 * sizeof_L3 /
ncores_per_socket'. This patch updates that value to roughly
'sizeof_L3 / 4`
The original value (specifically dividing the `ncores_per_socket`) was
done to limit the amount of other threads' data a `memcpy`/`memset`
could evict.
Dividing by 'ncores_per_socket', however leads to exceedingly low
non-temporal thresholds and leads to using non-temporal stores in
cases where REP MOVSB is multiple times faster.
Furthermore, non-temporal stores are written directly to main memory
so using it at a size much smaller than L3 can place soon to be
accessed data much further away than it otherwise could be. As well,
modern machines are able to detect streaming patterns (especially if
REP MOVSB is used) and provide LRU hints to the memory subsystem. This
in affect caps the total amount of eviction at 1/cache_associativity,
far below meaningfully thrashing the entire cache.
As best I can tell, the benchmarks that lead this small threshold
where done comparing non-temporal stores versus standard cacheable
stores. A better comparison (linked below) is to be REP MOVSB which,
on the measure systems, is nearly 2x faster than non-temporal stores
at the low-end of the previous threshold, and within 10% for over
100MB copies (well past even the current threshold). In cases with a
low number of threads competing for bandwidth, REP MOVSB is ~2x faster
up to `sizeof_L3`.
The divisor of `4` is a somewhat arbitrary value. From benchmarks it
seems Skylake and Icelake both prefer a divisor of `2`, but older CPUs
such as Broadwell prefer something closer to `8`. This patch is meant
to be followed up by another one to make the divisor cpu-specific, but
in the meantime (and for easier backporting), this patch settles on
`4` as a middle-ground.
Benchmarks comparing non-temporal stores, REP MOVSB, and cacheable
stores where done using:
https://github.com/goldsteinn/memcpy-nt-benchmarks
Sheets results (also available in pdf on the github):
https://docs.google.com/spreadsheets/d/e/2PACX-1vS183r0rW_jRX6tG_E90m9qVuFiMbRIJvi5VAE8yYOvEOIEEc3aSNuEsrFbuXw5c3nGboxMmrupZD7K/pubhtml Reviewed-by: DJ Delorie <dj@redhat.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Martin Coufal [Fri, 9 Jun 2023 11:58:04 +0000 (13:58 +0200)]
Remove unused DATEMSK file for tst-getdate
tst-getdate used to rely on an in-tree datemsk file that was
subsequently replaced by a file created during test execution. This
commit removes the unused file and corresponding env-var and uses a more
appropriate name for the temp file. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Arjun Shankar [Tue, 6 Jun 2023 17:20:31 +0000 (19:20 +0200)]
time: Fix use-after-free in getdate
getdate would free the buffer pointed to by the result of its call to
strptime, then reference the same buffer later on -- leading to a
use-after-free. This commit fixes that.
Reported-by: Martin Coufal <mcoufal@redhat.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Frédéric Bérat [Fri, 2 Jun 2023 15:28:12 +0000 (17:28 +0200)]
Move {read,write}_all functions to a dedicated header
Since these functions are used in both catgets/gencat.c and
malloc/memusage{,stat}.c, it make sense to move them into a dedicated
header where they can be inlined.
linux: Fail as unsupported if personality call is filtered
Container management default seccomp filter [1] only accepts
personality(2) with PER_LINUX, (0x0), UNAME26 (0x20000),
PER_LINUX32 (0x8), UNAME26 | PER_LINUX32, and 0xffffffff (to query
current personality)
Although the documentation only state it is blocked to prevent
'enabling BSD emulation' (PER_BSD, not implemented by Linux), checking
on repository log the real reason is to block ASLR disable flag
(ADDR_NO_RANDOMIZE) and other poorly support emulations.
So handle EPERM and fail as UNSUPPORTED if we can really check for
BZ#19408.
Joseph Myers [Mon, 5 Jun 2023 14:35:25 +0000 (14:35 +0000)]
Remove MAP_VARIABLE from hppa bits/mman.h
As suggested in
<https://sourceware.org/pipermail/libc-alpha/2023-February/145890.html>,
remove the MAP_VARIABLE define from the hppa bits/mman.h, for
consistency with Linux 6.2 which removed the define there.
Tested with build-many-glibcs.py for hppa-linux-gnu.
Since the area of the user's stack we use for the registers dump (and
otherwise as __sigreturn2's stack) can and does overlap the sigcontext,
we have to be very careful about the order of loads and stores that we
do. In particular we have to load sc_reply_port before we start
clobbering the sigcontext.
Carlos O'Donell [Thu, 18 May 2023 20:18:22 +0000 (16:18 -0400)]
Add lint-makefiles Makefile linting test.
We add a 'make check' test that lints all Makefiles in the source
directory of the glibc build. This linting test ensures that the
lines in all Makefiles will be sorted correctly as developers
creates patches. It is added to 'make check' because it is
light-weight and supports the existing developer workflow
epoll_pwait2(2)'s second argument should be nonnull. We're going to add
__nonnull to the prototype, so let's fix the test accordingly. We can
use a dummy variable to avoid passing NULL.
Frédéric Bérat [Thu, 1 Jun 2023 14:27:47 +0000 (16:27 +0200)]
tests: fix warn unused results
With fortification enabled, few function calls return result need to be
checked, has they get the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
H.J. Lu [Thu, 1 Jun 2023 15:53:35 +0000 (08:53 -0700)]
x86-64: Use YMM registers in memcmpeq-evex.S
Since the assembly source file with -evex suffix should use YMM registers,
not ZMM registers, include x86-evex256-vecs.h by default to use YMM
registers in memcmpeq-evex.S Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
support: Don't fail on fchown when spawning sgid processes
In some cases (e.g. when podman creates user containers), the only other
group assigned to the executing user is nobody and fchown fails with it
because the group is not mapped. Do not fail the test in this case,
instead exit as unsupported.
io: Fix F_GETLK, F_SETLK, and F_SETLKW for powerpc64
Different than other 64 bit architectures, powerpc64 defines the
LFS POSIX lock constants with values similar to 32 ABI, which
are meant to be used with fcntl64 syscall. Since powerpc64 kABI
does not have fcntl, the constants are adjusted with the
FCNTL_ADJUST_CMD macro.
The 4d0fe291aed3a476a changed the logic of generic constants
LFS value are equal to the default values; which is now wrong
for powerpc64.
Fix the value by explicit define the previous glibc constants
(powerpc64 does not need to use the 32 kABI value, but it simplifies
the FCNTL_ADJUST_CMD which should be kept as compatibility).
Checked on powerpc64-linux-gnu and powerpc-linux-gnu.
io: Fix record locking contants on 32 bit arch with 64 bit default time_t (BZ#30477)
For architecture with default 64 bit time_t support, the kernel
does not provide LFS and non-LFS values for F_GETLK, F_GETLK, and
F_GETLK (the default value used for 64 bit architecture are used).
This is might be considered an ABI break, but the currenct exported
values is bogus anyway.
The POSIX lockf is not affected since it is aliased to lockf64,
which already uses the LFS values.
Checked on i686-linux-gnu and the new tests on a riscv32.
Florian Weimer [Tue, 30 May 2023 11:25:50 +0000 (13:25 +0200)]
elf: Make more functions available for binding during dlclose (bug 30425)
Previously, after destructors for a DSO have been invoked, ld.so refused
to bind against that DSO in all cases. Relax this restriction somewhat
if the referencing object is itself a DSO that is being unloaded. This
assumes that the symbol reference is not going to be stored anywhere.
The situation in the test case can arise fairly easily with C++ and
objects that are built with different optimization levels and therefore
define different functions with vague linkage.
caiyinyu [Thu, 25 May 2023 09:01:11 +0000 (17:01 +0800)]
LoongArch: Fix inconsistency in SHMLBA macro values between glibc and kernel
The LoongArch glibc was using the value of the SHMLBA macro from common code,
which is __getpagesize() (16k), but this was inconsistent with the value of
the SHMLBA macro in the kernel, which is SZ_64K (64k). This caused several
shmat-related tests in LTP (Linux Test Project) to fail. This commit fixes
the issue by ensuring that the glibc's SHMLBA macro value matches the value
used in the kernel like other architectures.
Noah Goldstein [Tue, 9 May 2023 03:10:20 +0000 (22:10 -0500)]
x86: Use 64MB as nt-store threshold if no cacheinfo [BZ #30429]
If `non_temporal_threshold` is below `minimum_non_temporal_threshold`,
it almost certainly means we failed to read the systems cache info.
In this case, rather than defaulting the minimum correct value, we
should default to a value that gets at least reasonable
performance. 64MB is chosen conservatively to be at the very high
end. This should never cause non-temporal stores when, if we had read
cache info, we wouldn't have otherwise. Reviewed-by: Florian Weimer <fweimer@redhat.com>
Joseph Myers [Fri, 26 May 2023 15:04:51 +0000 (15:04 +0000)]
Add MFD_NOEXEC_SEAL, MFD_EXEC from Linux 6.3 to bits/mman-shared.h
Linux 6.3 adds new constants MFD_NOEXEC_SEAL and MFD_EXEC. Add these
to bits/mman-shared.h (conditional on MFD_NOEXEC_SEAL not already
being defined, similar to the existing conditional on the older MFD_*
macros).
Joseph Myers [Fri, 26 May 2023 15:03:31 +0000 (15:03 +0000)]
Add AT_RSEQ_* from Linux 6.3 to elf.h
Linux 6.3 adds constants AT_RSEQ_FEATURE_SIZE and AT_RSEQ_ALIGN; add
them to glibc's elf.h. (Recall that, although elf.h is a
system-independent header, so far we've put AT_* constants there even
if Linux-specific, as discussed in bug 15794. So rather than making
any attempt to fix that issue, the new constants are just added there
alongside the existing ones.)
Roy Eldar [Thu, 25 May 2023 14:41:58 +0000 (17:41 +0300)]
elf: fix handling of negative numbers in dl-printf
_dl_debug_vdprintf is a bare-bones printf implementation; currently
printing a signed integer (using "%d" format specifier) behaves
incorrectly when the number is negative, as it just prints the
corresponding unsigned integer, preceeded by a minus sign.
For example, _dl_printf("%d", -1) would print '-4294967295'.
Signed-off-by: Roy Eldar <royeldar0@gmail.com> Reviewed-by: Florian Weimer <fweimer@redhat.com>
elf: Add test for locating libraries in root dir (bug 30435)
When dlopen is being called, efforts have been made to improve
future lookup performance. This includes marking a search path
as non-existent using `stat`. However, if the root directory
is given as a search path, there exists a bug which erroneously
marks it as non-existing.
The bug is reproduced under the following sequence:
1. dlopen is called to open a shared library, with at least:
1) a dependency 'A.so' not directly under the '/' directory
(e.g. /lib/A.so), and
2) another dependency 'B.so' resides in '/'.
2. for this bug to reproduce, 'A.so' should be searched *before* 'B.so'.
3. it first tries to find 'A.so' in /, (e.g. /A.so):
- this will (obviously) fail,
- since it's the first time we have seen the '/' directory,
its 'status' is 'unknown'.
4. `buf[buflen - namelen - 1] = '\0'` is executed:
- it intends to remove the leaf and its final slash,
- because of the speciality of '/', its buflen == namelen + 1,
- it erroneously clears the entire buffer.
6. it then calls 'stat' with the empty buffer:
- which will result in an error.
7. so it marks '/' as 'nonexisting', future lookups will not consider
this path.
8. while /B.so *does* exist, failure to look it up in the '/'
directory leads to a 'cannot open shared object file' error.
This patch fixes the bug by preventing 'buflen', an index to put '\0',
from being set to 0, so that the root '/' is always kept.
Relative search paths are always considered as 'existing' so this
wont be affected.
Fix special case for C2x strtol binary constant handling (BZ# 30371)
When the base is 0 or 2 and the first two characters are '0' and 'b',
but the rest are no binary digits. In this case this is no error,
and strtol must return 0 and ENDPTR points to the 'x' or 'b'.
Samuel Thibault [Mon, 22 May 2023 23:44:43 +0000 (01:44 +0200)]
mach: Fix installing mach_i386.h
We do not want mach_i386.h to get installed into machine/, but into
i386/ or x86_64/ depending where mach_i386.defs was found, i.e.
according to 32/64 bitness.
Stefan Liebler [Fri, 12 May 2023 10:44:49 +0000 (12:44 +0200)]
S390: Use compile-only instead of also link-tests in configure.
Some of the s390-specific configure checks are using compile and
link configure tests. Now use only compile tests as the link
tests fails when e.g. bootstrapping a cross-toolchain due to
missing crt-files/libc.so. This is achieved by using
AC_COMPILE_IFELSE in configure.ac file.
This is observable e.g. when using buildroot which builds glibc
only once or the build-many-glibcs.py script. Note that the latter
one is building glibc twice in the compilers-step (configure-checks
fails) and in the glibcs-step (configure-checks succeed).
Note, that the s390 specific configure tests for static PIE have to
link an executable to test binutils support. Thus we can't fix
those tests.
Sergey Bugaev [Sat, 20 May 2023 11:55:29 +0000 (14:55 +0300)]
hurd: Use __hurd_fail () instead of assigning errno
The __hurd_fail () inline function is the dedicated, idiomatic way of
reporting errors in the Hurd part of glibc. Not only is it more concise
than '{ errno = err; return -1; }', it is since commit 6639cc10029e24e06b34e169712b21c31b8cf213
"hurd: Mark error functions as __COLD" marked with the cold attribute,
telling the compiler that this codepath is unlikely to be executed.
In one case, use __hurd_dfail () over the plain __hurd_fail ().
Samuel Thibault [Fri, 19 May 2023 18:44:01 +0000 (20:44 +0200)]
hurd 64bit: Re-introduce gai_suspend symbol
4d3f846b88d3 ("hurd: Fix __TIMESIZE on x86_64") incidentaly dropped it
because it fixed hurd 64bit into setting __TIMESIZE to 64, and that case
was not having gai_suspend defined yet.
Carlos O'Donell [Wed, 17 May 2023 12:27:59 +0000 (08:27 -0400)]
nptl: Reformat Makefile.
Reflow all long lines adding comment terminators.
Rename files that cause inconsistent ordering.
Sort all reflowed text using scripts/sort-makefile-lines.py.
No code generation changes observed in binary artifacts.
No regressions on x86_64 and i686.