Szabolcs Nagy [Fri, 12 Mar 2021 09:46:15 +0000 (09:46 +0000)]
malloc: Use different tag after mremap
The comment explained why different tag is used after mremap, but
for that correctly tagged pointer should be passed to tag_new_usable.
Use chunk2mem to get the tag.
Szabolcs Nagy [Mon, 8 Mar 2021 12:59:05 +0000 (12:59 +0000)]
malloc: Use memsize instead of CHUNK_AVAILABLE_SIZE
This is a pure refactoring change that does not affect behaviour.
The CHUNK_AVAILABLE_SIZE name was unclear, the memsize name tries to
follow the existing convention of mem denoting the allocation that is
handed out to the user, while chunk is its internally used container.
The user owned memory for a given chunk starts at chunk2mem(p) and
the size is memsize(p). It is not valid to use on dumped heap chunks.
Moved the definition next to other chunk and mem related macros.
Szabolcs Nagy [Tue, 9 Feb 2021 17:59:11 +0000 (17:59 +0000)]
aarch64: Optimize __libc_mtag_tag_zero_region
This is a target hook for memory tagging, the original was a naive
implementation. Uses the same algorithm as __libc_mtag_tag_region,
but with instructions that also zero the memory. This was not
benchmarked on real cpu, but expected to be faster than the naive
implementation.
Szabolcs Nagy [Tue, 9 Feb 2021 17:56:02 +0000 (17:56 +0000)]
aarch64: Optimize __libc_mtag_tag_region
This is a target hook for memory tagging, the original was a naive
implementation. The optimized version relies on "dc gva" to tag 64
bytes at a time for large allocations and optimizes small cases without
adding too many branches. This was not benchmarked on real cpu, but
expected to be faster than the naive implementation.
Szabolcs Nagy [Thu, 4 Feb 2021 10:04:07 +0000 (10:04 +0000)]
aarch64: inline __libc_mtag_address_get_tag
This is a common operation when heap tagging is enabled, so inline the
instruction instead of using an extern call.
The .inst directive is used instead of the name of the instruction (or
acle intrinsics) because malloc.c is not compiled for armv8.5-a+memtag
architecture, runtime cpu support detection is used.
Prototypes are removed from the comments as they were not always
correct.
Szabolcs Nagy [Wed, 17 Feb 2021 10:15:18 +0000 (10:15 +0000)]
malloc: Use mtag_enabled instead of USE_MTAG
Use the runtime check where possible: it should not cause slow down in
the !USE_MTAG case since then mtag_enabled is constant false, but it
allows compiling the tagging logic so it's less likely to break or
diverge when developers only test the !USE_MTAG case.
Szabolcs Nagy [Mon, 8 Feb 2021 12:39:01 +0000 (12:39 +0000)]
malloc: Use branches instead of mtag_granule_mask
The branches may be better optimized since mtag_enabled is widely used.
Granule size larger than a chunk header is not supported since then we
cannot have both the chunk header and user area granule aligned. To
fix that for targets with large granule, the chunk layout has to change.
So code that attempted to handle the granule mask generally was changed.
This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size.
Szabolcs Nagy [Tue, 16 Feb 2021 17:02:44 +0000 (17:02 +0000)]
malloc: Change calloc when tagging is disabled
When glibc is built with memory tagging support (USE_MTAG) but it is not
enabled at runtime (mtag_enabled) then unconditional memset was used
even though that can be often avoided.
This is for performance when tagging is supported but not enabled.
The extra check should have no overhead: tag_new_zero_region already
had a runtime check which the compiler can now optimize away.
Szabolcs Nagy [Fri, 29 Jan 2021 17:07:28 +0000 (17:07 +0000)]
malloc: Only support zeroing and not arbitrary memset with mtag
The memset api is suboptimal and does not provide much benefit. Memory
tagging only needs a zeroing memset (and only for memory that's sized
and aligned to multiples of the tag granule), so change the internal
api and the target hooks accordingly. This is to simplify the
implementation of the target hook.
Szabolcs Nagy [Wed, 27 Jan 2021 15:45:43 +0000 (15:45 +0000)]
malloc: Use global flag instead of function pointer dispatch for mtag
A flag check can be faster than function pointers because of how
branch prediction and speculation works and it can also remove a layer
of indirection when there is a mismatch between the malloc internal
tag_* api and __libc_mtag_* target hooks.
Memory tagging wrapper functions are moved to malloc.c from arena.c and
the logic now checks mmap_enabled. The definition of tag_new_usable is
moved after chunk related definitions.
This refactoring also allows using mtag_enabled checks instead of
USE_MTAG ifdefs when memory tagging support only changes code logic
when memory tagging is enabled at runtime. Note: an "if (false)" code
block is optimized away even at -O0 by gcc.
Szabolcs Nagy [Tue, 16 Feb 2021 14:12:25 +0000 (14:12 +0000)]
malloc: Refactor TAG_ macros to avoid indirection
This does not change behaviour, just removes one layer of indirection
in the internal memory tagging logic.
Use tag_ and mtag_ prefixes instead of __tag_ and __mtag_ since these
are all symbols with internal linkage, private to malloc.c, so there
is no user namespace pollution issue.
Szabolcs Nagy [Wed, 17 Feb 2021 10:39:37 +0000 (10:39 +0000)]
malloc: Ensure the generic mtag hooks are not used
Use inline functions instead of macros, because macros can cause unused
variable warnings and type conversion issues. We assume these functions
may appear in the code but only in dead code paths (hidden by a runtime
check), so it's important that they can compile with correct types, but
if they are actually used that should be an error.
Currently the hooks are only used when USE_MTAG is true which only
happens on aarch64 and then the aarch64 specific code is used not this
generic header. However followup refactoring will allow the hooks to
be used with !USE_MTAG.
Note: the const qualifier in the comment was wrong: changing tags is a
write operation.
Szabolcs Nagy [Thu, 4 Feb 2021 11:52:14 +0000 (11:52 +0000)]
malloc: Avoid taggig mmaped memory on free
Either the memory belongs to the dumped area, in which case we don't
want to tag (the dumped area has the same tag as malloc internal data
so tagging is unnecessary, but chunks there may not have the right
alignment for the tag granule), or the memory will be unmapped
immediately (and thus tagging is not useful).
Szabolcs Nagy [Thu, 4 Feb 2021 11:38:23 +0000 (11:38 +0000)]
malloc: Simplify __mtag_tag_new_usable
The chunk cannot be a dumped one here. The only non-obvious cases
are free and realloc which may be called on a dumped area chunk,
but in both cases it can be verified that tagging is already
avoided for dumped area chunks.
Szabolcs Nagy [Tue, 2 Feb 2021 15:02:09 +0000 (15:02 +0000)]
Remove PR_TAGGED_ADDR_ENABLE from sys/prctl.h
The value of PR_TAGGED_ADDR_ENABLE was incorrect in the installed
headers and the prctl command macros were missing that are needed
for it to be useful (PR_SET_TAGGED_ADDR_CTRL). Linux headers have
the definitions since 5.4 so it's widely available, we don't need
to repeat these definitions. The remaining definitions are from
Linux 5.10.
To build glibc with --enable-memory-tagging, Linux 5.4 headers and
binutils 2.33.1 or newer is needed.
Szabolcs Nagy [Thu, 11 Mar 2021 14:09:56 +0000 (14:09 +0000)]
malloc: Fix a potential realloc issue with memory tagging
At an _int_free call site in realloc the wrong size was used for tag
clearing: the chunk header of the next chunk was also cleared which
in practice may work, but logically wrong.
The tag clearing is moved before the memcpy to save a tag computation,
this avoids a chunk2mem. Another chunk2mem is removed because newmem
does not have to be recomputed. Whitespaces got fixed too.
Szabolcs Nagy [Thu, 25 Feb 2021 14:49:58 +0000 (14:49 +0000)]
malloc: Fix a realloc crash with heap tagging [BZ 27468]
_int_free must be called with a chunk that has its tag reset. This was
missing in a rare case that could crash when heap tagging is enabled:
when in a multi-threaded process the current arena runs out of memory
during realloc, but another arena still has space to finish the realloc
then _int_free was called without clearing the user allocation tags.
The generic implementation basically handle the system agnostic logic
(filtering out the invalid signals) while the __libc_sigaction is
the function with implements the system and architecture bits.
The instructions xsxexpdp and xsxexpqp introduced on POWER9 extract
the exponent from a double-precision and quad-precision floating-point
respectively, thus they can be used to improve ilogb, ilogbf and ilogbf128.
Build get-cpuid-feature-leaf.c without stack-protector [BZ #27555]
__x86_get_cpuid_feature_leaf is called during early startup, before
the stack check guard is initialized and is hence not safe to build
with stack-protector.
Additionally, IFUNC resolvers for static tst-ifunc-isa tests get
called too early for stack protector to be useful, so fix them to
disable stack protector for the resolver functions.
This fixes all failures seen with --enable-stack-protector=all
configuration.
Nick Alcock (who committed the stack protector code) marked the IFUNC
resolvers with inhibit_stack_protector when he done the original work and
suggested doing so again @ BZ #25680. This patch adds
inhibit_stack_protector to ifuncmain9.
After patch is applied, --enable-stack-protector=all does not fail the
above tests.
Florian Weimer [Mon, 15 Mar 2021 09:33:43 +0000 (10:33 +0100)]
elf: ld.so --help calls _dl_init_paths without a main map [BZ #27577]
In this case, use the link map of the dynamic loader itself as
a replacement. This is more than just a hack: if we ever support
DT_RUNPATH/DT_RPATH for the dynamic loader, reporting it for
ld.so --help (without further command line arguments) would be the
right thing to do.
x86: Move x86 processor cache info to cpu_features
missed _SC_LEVEL1_ICACHE_LINESIZE.
1. Add level1_icache_linesize to struct cpu_features.
2. Initialize level1_icache_linesize by calling handle_intel,
handle_zhaoxin and handle_amd with _SC_LEVEL1_ICACHE_LINESIZE.
3. Return level1_icache_linesize for _SC_LEVEL1_ICACHE_LINESIZE.
Carlos O'Donell [Fri, 12 Mar 2021 15:44:47 +0000 (16:44 +0100)]
elf: Always set l in _dl_init_paths (bug 23462)
After d1d5471579eb0426671bf94f2d71e61dfb204c30 ("Remove dead
DL_DST_REQ_STATIC code.") we always setup the link map l to make the
static and shared cases the same. The bug is that in elf/dl-load.c
(_dl_init_paths) we conditionally set l only in the #ifdef SHARED
case, but unconditionally use it later. The simple solution is to
remove the #ifdef SHARED conditional, because it's no longer needed,
and unconditionally setup l for both the static and shared cases. A
regression test is added to run a static binary with
LD_LIBRARY_PATH='$ORIGIN' which crashes before the fix and runs after
the fix.
Stefan Liebler [Fri, 12 Mar 2021 13:31:49 +0000 (14:31 +0100)]
s390x: Regenerate ULPs.
Updates needed after recent commit: db3f7bb5586392d9809fc6397c7184983aed6008
"math: Remove slow paths from asin and acos [BZ #15267]"
Compre to the required ulps update for x86_64.
Both htl and nptl uses a different data structure to implement atfork
handlers. The nptl one was refactored by 27761a1042d to use a dynarray
which simplifies the code.
This patch moves the nptl one to be the generic implementation and
replace Hurd linked one. Different than previous NPTL, Hurd also uses
a global lock, so performance should be similar.
Checked on x86_64-linux-gnu, i686-linux-gnu, and with a build for
i686-gnu.
io: Return UNSUPPORTED if filesystem do not support 64 bit timestamps
Some Linux filesystems might not fully support 64 bit timestamps [1],
which make some Linux specific tests to fail when they check for the
functionality.
This patch adds a new libsupport function, support_path_support_time64,
that returns whether the target file supports or not 64 bit timestamps.
The support is checked by issuing a utimensat and verifying both the
last access and last modification time against a statx call.
The tests that might fail are also adjusted to check the file support
as well:
DJ Delorie [Thu, 11 Mar 2021 17:50:02 +0000 (12:50 -0500)]
test-container: Always copy test-specific support files [BZ #27537]
There's a small chance that a fresh checkout will result in some of
the test-specific container files will have the same timestamp and
size, which breaks the rsync logic in test-container, resulting in
tests running with the wrong support files.
This patch changes the rsync logic to always copy the test-specific
files, which normally would always be copied anyway. The rsync logic
for the testroot itself is unchanged.
Wilco Dijkstra [Wed, 10 Mar 2021 12:39:56 +0000 (12:39 +0000)]
math: Remove slow paths from asin and acos [BZ #15267]
This patch series removes all remaining slow paths and related code.
First asin/acos, tan, atan, atan2 implementations are updated, and the final
patch removes the unused mpa files, headers and probes. Passes buildmanyglibc.
Remove slow paths from asin/acos. Add ULP annotations based on previous slow
path checks (which are approximate). Update AArch64 and x86_64 libm-test-ulps.
Reviewed-By: Paul Zimmermann <Paul.Zimmermann@inria.fr>
io: Return EBAFD for negative file descriptor on fstat (BZ #27559)
Now that fstat is implemented on top fstatat we need to handle negative
inputs. The implementation now rejects AT_FDCWD, which would otherwise
be accepted by the kernel.
Checked on x86_64-linux-gnu and on i686-linux-gnu.
Stefan Liebler [Thu, 11 Mar 2021 09:45:15 +0000 (10:45 +0100)]
Allow gdb version 10 in pretty printer tests.
With gdb 10, the pretty printer tests are UNSUPPORTED::
The gdb version string (gdb -v) is incorrectly formatted.
This is observable in:
nptl/test-cond-printers, nptl/test-condattr-printers,
nptl/test-mutex-printers, nptl/test-mutexattr-printers,
nptl/test-rwlock-printers, nptl/test-rwlockattr-printers
After updating the regexp and building with debug-info,
all those tests are passing.
Joseph Myers [Wed, 10 Mar 2021 14:26:57 +0000 (14:26 +0000)]
Update kernel version to 5.11 in tst-mman-consts.py.
This patch updates the kernel version in the test tst-mman-consts.py
to 5.11. (There are no new MAP_* constants covered by this test in
5.11 that need any other header changes.)
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
Linux: misc/tst-ofdlocks-compat can be a regular test
Now that compat_symbol_reference works in non-internal tests.
Also do not build and run the test at all on architectures which
do not have the pre-2.28 symbol version of fcntl.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
Linux: dirent/tst-readdir64-compat can be a regular test
compat_symbol_reference works in non-internal tests now. Also
avoid building the test for unsupported configurations at all.
I verified by building with build-many-glibcs.py that GLIBC_2.1.3
works as the predecessor of GLIBC_2.2. (Symbol versions in
the early days are complex.)
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
resolv: tst-p_secstodate can be a regular test
Now that compat_symbol_reference works for non-internal tests, too.
Also do not build and run the tests on architectures which lack the
__p_secstodate compatibility symbol.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
posix: tst-spawn4-compat can be a regular test
compat_symbol_reference now works for non-internal tests, too.
Also stop building and running the tests on those architectures
that lack the test symbol versions.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
posix: tst-glob_lstat_compat no longer needs to be an internal test
compat_symbol_reference is now available for regular tests as well.
Also avoid building and running the tests in case the pre-2.27
symbol version of glob is not available. This avoids a spurious
UNSUPPORTED result.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
math: test-matherr and test-matherr-2 can be regular tests
compat_symbol_reference is now available without tests-internal.
Do not build the test at all on glibc versions that lack the symbols,
to avoid spurious UNSUPPORTED results.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
malloc: Turn tst-mallocstate into a non-internal test
compat_symbol_reference no longer needs tests-internal. Do not build
the test at all for newer targets, so that no spurious UNSUPPORTED
result is generated. Use compat_symbol_reference for
__malloc_initialize_hook as well, eliminating the need for -rdynamic.
Florian Weimer [Tue, 9 Mar 2021 20:07:24 +0000 (21:07 +0100)]
<shlib-compat.h>: Support compat_symbol_reference for _ISOMAC
This is helpful for testing compat symbols in cases where _ISOMAC
is activated implicitly due to -DMODULE_NAME=testsuite and cannot
be disabled easily.
__nss_database_lookup2's extra arguments were left unused in the
nsswitch reloading patch set; this broke compat (default config
ignored) and shadow files (secondary name ignored) which relies on
these fallbacks.
This patch adds in the previous behavior by correcting the
initialization of the database list to reflect the fallbacks. This
means that the nss_database_lookup2 interface no longer needs to be
passed the fallback info, so API and callers were adjusted.
Since all callers needed to be edited anyway, the calls were changed
from __nss_database_lookup2 to the faster __nss_database_get. This
was an intended optimization which was deferred during the initial
lookup changes to avoid touching so many files.
The test case verifies that compat targets work (passwd) and that the
default configuration works (group). Tested on x86-64.
Lukasz Majewski [Wed, 20 Jan 2021 11:10:42 +0000 (12:10 +0100)]
tst: Add test for clock_adjtime
This code privides test to check if time on target machine is properly
adjusted.
The time is altered only when cross-test-ssh.sh is executed with
--allow-time-setting flag.
As the delta added to CLOCK_REALTIME is only 1 sec the original time is
not restored and further tests are executed with this bias.
Lukasz Majewski [Mon, 18 Jan 2021 15:53:26 +0000 (16:53 +0100)]
support: Provide xclock_settime test helper function
The xclock_settime is a wrapper function on the clock_settime syscall
to be used in the test code.
It checks if the GLIBC_TEST_ALLOW_TIME_SETTING env variable is defined
in the environment in which test is executed. If it is not - the test
ends as unsupported. Otherwise, the clock-settime is executed and return
value is assessed.
Lukasz Majewski [Fri, 15 Jan 2021 15:40:39 +0000 (16:40 +0100)]
tst: Extend cross-test-ssh.sh to specify if target date can be altered
This code adds new flag - '--allow-time-setting' to cross-test-ssh.sh
script to indicate if it is allowed to alter the date on the system
on which tests are executed. This change is supposed to be used with
test systems, which use virtual machines for testing.
The GLIBC_TEST_ALLOW_TIME_SETTING env variable is exported to the
remote environment on which the eligible test is run and brings no
functional change when it is not.
H.J. Lu [Tue, 2 Feb 2021 21:45:58 +0000 (13:45 -0800)]
x86: Set minimum x86-64 level marker [BZ #27318]
Since the full ISA set used in an ELF binary is unknown to compiler,
an x86-64 ISA level marker indicates the minimum, not maximum, ISA set
required to run such an ELF binary. We never guarantee a library with
an x86-64 ISA level v3 marker doesn't contain other ISAs beyond x86-64
ISA level v3, like AVX VNNI. We check the x86-64 ISA level marker for
the minimum ISA set. Since -march=sandybridge enables only some ISAs
in x86-64 ISA level v3, we should set the needed ISA marker to v2.
Otherwise, libc is compiled with -march=sandybridge will fail to run on
Sandy Bridge:
$ ./elf/ld.so ./libc.so
./libc.so: (p) CPU ISA level is lower than required: needed: 7; got: 3
Set the minimum, instead of maximum, x86-64 ISA level marker should have
no impact on the glibc-hwcaps directory assignment logic in ldconfig nor
ld.so.
Florian Weimer [Fri, 5 Mar 2021 11:02:20 +0000 (12:02 +0100)]
posix: glob, glob64 should not be declared __THROW [BZ #27522]
These functions invoke callbacks with GLOB_ALTDIRFUNC, so they
are not leaf functions (as implied by _THROW). Use __THROWNL
and __REDIRECT_NTHNL to express this.
Jakub Jelinek [Thu, 4 Mar 2021 14:15:33 +0000 (15:15 +0100)]
[PATCH] pthread_once hangs when init routine throws an exception [BZ #18435]
This is another attempt at making pthread_once handle throwing exceptions
from the init routine callback. As the new testcases show, just switching
to the cleanup attribute based cleanup does fix the tst-once5 test, but
breaks the new tst-oncey3 test. That is because when throwing exceptions,
only the unwind info registered cleanups (i.e. C++ destructors or cleanup
attribute), when cancelling threads and there has been unwind info from the
cancellation point up to whatever needs cleanup both unwind info registered
cleanups and THREAD_SETMEM (self, cleanup, ...) registered cleanups are
invoked, but once we hit some frame with no unwind info, only the
THREAD_SETMEM (self, cleanup, ...) registered cleanups are invoked.
So, to stay fully backwards compatible (allow init routines without
unwind info which encounter cancellation points) and handle exception throwing
we actually need to register the pthread_once cleanups in both unwind info
and in the THREAD_SETMEM (self, cleanup, ...) way.
If an exception is thrown, only the former will happen and we in that case
need to also unregister the THREAD_SETMEM (self, cleanup, ...) registered
handler, because otherwise after catching the exception the user code could
call deeper into the stack some cancellation point, get cancelled and then
a stale cleanup handler would clobber stack and probably crash.
If a thread calling init routine is cancelled and unwind info ends before
the pthread_once frame, it will be cleaned up through self->cleanup as
before. And if unwind info is present, unwind_stop first calls the
self->cleanup registered handler for the frame, then it will call the
unwind info registered handler but that will already see __do_it == 0
and do nothing.