Sergey Bugaev [Sun, 12 Feb 2023 11:10:34 +0000 (14:10 +0300)]
mach, hurd: Cast through uintptr_t
When casting between a pointer and an integer of a different size, GCC
emits a warning (which is escalated to a build failure by -Werror).
Indeed, if what you start with is a pointer, which you then cast to a
shorter integer and then back again, you're going to cut off some bits
of the pointer.
But if you start with an integer (such as mach_port_t), then cast it to
a longer pointer (void *), and then back to a shorter integer, you are
fine. To keep GCC happy, cast through an intermediary uintptr_t, which
is always the same size as a pointer.
Sergey Bugaev [Sun, 12 Feb 2023 11:10:33 +0000 (14:10 +0300)]
hurd: Use mach_msg_type_number_t where appropriate
It has been decided that on x86_64, mach_msg_type_number_t stays 32-bit.
Therefore, it's not possible to use mach_msg_type_number_t
interchangeably with size_t, in particular this breaks when a pointer to
a variable is passed to a MIG routine.
Sergey Bugaev [Sun, 12 Feb 2023 11:10:32 +0000 (14:10 +0300)]
hurd: Refactor readlinkat()
Make the code flow more linear using early returns where possible. This
makes it so much easier to reason about what runs on error / successful
code paths.
Paul Pluzhnikov [Fri, 10 Feb 2023 16:14:30 +0000 (16:14 +0000)]
Use __builtin_FILE instead of __FILE__ in assert in C++.
Likewise use __builtin_LINE instead of __LINE__.
When building C++, inline functions are required to have the exact same
sequence of tokens in every translation unit. But __FILE__ token, when
used in a header file, does not necessarily expand to the exact same
string literal, and that may cause compilation failure when C++ modules
are being used. (It would also cause unpredictable output on assertion
failure at runtime, but this rarely matters in practice.)
Samuel Thibault [Thu, 9 Feb 2023 18:58:43 +0000 (19:58 +0100)]
hurd: Fix unwinding over INTR_MSG_TRAP
We used to use .cfi_adjust_cfa_offset around %esp manipulation
asm instructions to fix unwinding, but when building glibc with
-fno-omit-frame-pointer this is bogus since in that case %ebp is the CFA and
does not move.
Instead, let's force -fno-omit-frame-pointer when building intr-msg.c so
that %ebp can always be used and no .cfi_adjust_cfa_offset is needed.
extern int clone3 (struct clone_args *__cl_args, size_t __size,
int (*__func) (void *__arg), void *__arg);
The powerpc64 ABI requires an initial stackframe so the child can
store/restore the TOC. It is create prior calling clone3 by
adjusting the stack size (since kernel will compute the stack as
stack plus size).
Checked on powerpc64-linux-gnu (power8, kernel 6.0) and
powerpc64le-linux-gnu (power9, kernel 4.18). Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com>
Florian Weimer [Wed, 8 Feb 2023 17:11:04 +0000 (18:11 +0100)]
elf: Smoke-test ldconfig -p against system /etc/ld.so.cache
The test is sufficient to detect the ldconfig bug fixed in
commit 9fe6f6363886aae6b2b210cae3ed1f5921299083 ("elf: Fix 64 time_t
support for installed statically binaries").
quxm [Wed, 25 Jan 2023 09:47:33 +0000 (17:47 +0800)]
C-SKY: Strip hard float abi from hard float feature.
The hard float abi and hard float are different,
Hard float abi: Use float register to pass float type arguments.
Hard float: Enable the hard float ISA feature.
So the with_fp_cond cannot represent these two features. When
-mfloat-abi=softfp, the float abi is soft and hard float is enabled.
So add 'with_hard_float_abi' in preconfigure and define 'CSKY_HARD_FLOAT_ABI'
if float abi is hard, and use 'CSKY_HARD_FLOAT_ABI' to determine
dynamic linker because it is what determines compatibility.
And with_fp_cond is still needed to tell glibc whether to enable
hard floating feature.
In addition, use AC_TRY_COMMAND to test gcc to ensure compatibility
between different versions of gcc. The original way has a problem
that __CSKY_HARD_FLOAT_FPU_SF__ means the target only has single
hard float-points ISA, so it's not defined in CPUs like ck810f.
Stefan Liebler [Thu, 2 Feb 2023 13:57:50 +0000 (14:57 +0100)]
S390: Influence hwcaps/stfle via GLIBC_TUNABLES.
This patch enables the option to influence hwcaps and stfle bits used
by the s390 specific ifunc-resolvers. The currently x86-specific
tunable glibc.cpu.hwcaps is also used on s390x to achieve the task. In
addition the user can also set a CPU arch-level like z13 instead of
single HWCAP and STFLE features.
Note that the tunable only handles the features which are really used
in the IFUNC-resolvers. All others are ignored as the values are only
used inside glibc. Thus we can influence:
- HWCAP_S390_VXRS (z13)
- HWCAP_S390_VXRS_EXT (z14)
- HWCAP_S390_VXRS_EXT2 (z15)
- STFLE_MIE3 (z15)
The influenced hwcap/stfle-bits are stored in the s390-specific
cpu_features struct which also contains reserved fields for future
usage.
The ifunc-resolvers and users of stfle bits are adjusted to use the
information from cpu_features struct.
On 31bit, the ELF_MACHINE_IRELATIVE macro is now also defined.
Otherwise the new ifunc-resolvers segfaults as they depend on
the not yet processed_rtld_global_ro@GLIBC_PRIVATE relocation.
It uses the bitmanip extension to optimize index_fist and index_last
with clz/ctz (using generic implementation that routes to compiler
builtin) and orc.b to check null bytes.
Checked the string test on riscv64 user mode.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
hppa: Add string-fza.h, string-fzc.h, and string-fzi.h
Use UXOR,SBZ to test for a zero byte within a word. While we can
get semi-decent code out of asm-goto, we would do slightly better
with a compiler builtin.
For index_zero et al, sequential testing of bytes is less expensive than
any tricks that involve a count-leading-zeros insn that we don't have.
GCC's combine pass cannot merge (x >> c | y << (32 - c)) into a
double-word shift unless (1) the subtract is in the same basic block
and (2) the result of the subtract is used exactly once. Neither
condition is true for any use of MERGE.
By forcing the use of a double-word shift, we not only reduce
contention on SAR, but also allow the setting of SAR to be hoisted
outside of a loop.
string: Improve generic strrchr with memrchr and strlen
Now that both strlen and memrchr have word vectorized implementation,
it should be faster to implement strrchr based on memrchr over the
string length instead of calling strchr on a loop.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu,
and powerpc64-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
New algorithm read the lastaligned address and mask off the unwanted
bytes. The loop now read word-aligned address and check using the
has_eq macro.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu,
and powerpc64-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
New algorithm read the first aligned address and mask off the
unwanted bytes (this strategy is similar to arch-specific
implementations used on powerpc, sparc, and sh).
The loop now read word-aligned address and check using the has_eq
macro.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu,
and powerpc64-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Now that stpcpy is vectorized based on op_t, it should be better to
call it instead of strlen plus memcpy.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu,
and powerpc-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
- Align the destination on word boundary using byte operations.
- If source is also word aligned, read a word per time, check for
null (using has_zero from string-fzb.h), and write the remaining
bytes.
- If source is not word aligned, loop by aligning the source, and
merging the result of two reads. Similar to aligned case,
check for null with has_zero, and write the remaining bytes if
null is found.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu,
and powerpc-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
- Align the first input to word boundary using byte operations.
- If second input is also word aligned, read a word per time, check
for null (using has_zero), and check final words using byte
operation.
- If second input is not word aligned, loop by aligning the source,
and merge the result of two reads. Similar to aligned case, check
for null with has_zero, and check final words using byte operation.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu,
and powerpc-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
- Align the first input to word boundary using byte operations.
- If second input is also word aligned, read a word per time, check for
null (using has_zero), and check final words using byte operation.
- If second input is not word aligned, loop by aligning the source, and
merging the result of two reads. Similar to aligned case, check for
null with has_zero, and check final words using byte operation.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu,
and powerpc-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu,
and powerpc64-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
New algorithm read the first aligned address and mask off the unwanted
bytes (this strategy is similar to arch-specific implementations used
on powerpc, sparc, and sh).
The loop now read word-aligned address and check using the has_zero_eq
function.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu,
and powerpc-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
New algorithm read the first aligned address and mask off the
unwanted bytes (this strategy is similar to arch-specific
implementations used on powerpc, sparc, and sh).
The loop now read word-aligned address and check using the has_zero
macro.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu,
and powercp64-linux-gnu by removing the arch-specific assembly
implementation and disabling multi-arch (it covers both LE and BE
for 64 and 32 bits).
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Add string vectorized find and detection functions
This patch adds generic string find and detection meant to be used in
generic vectorized string implementation. The idea is to decompose the
basic string operation so each architecture can reimplement if it
provides any specialized hardware instruction.
The 'string-misc.h' provides miscellaneous functions:
- extractbyte: extracts the byte from an specific index.
- repeat_bytes: setup an word by replicate the argument on each byte.
The 'string-fza.h' provides zero byte detection functions:
- find_zero_low, find_zero_all, find_eq_low, find_eq_all,
find_zero_eq_low, find_zero_eq_all, and find_zero_ne_all
The 'string-fzb.h' provides boolean zero byte detection functions:
- has_zero: determine if any byte within a word is zero.
- has_eq: determine byte equality between two words.
- has_zero_eq: determine if any byte within a word is zero along with
byte equality between two words.
The 'string-fzi.h' provides positions for string-fza.h results:
- index_first: return index of first zero byte within a word.
- index_last: return index of first byte different between two words.
The 'string-fzc.h' provides a combined version of fza and fzi:
- index_first_zero_eq: return index of first zero byte within a word or
first byte different between two words.
- index_first_zero_ne: return index of first zero byte within a word or
first byte equal between two words.
- index_last_zero: return index of last zero byte within a word.
- index_last_eq: return index of last byte different between two words.
The 'string-shift.h' provides a way to mask off parts of a work based on
some alignmnet (to handle unaligned arguments):
- shift_find, shift_find_last.
Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Wilco Dijkstra [Fri, 3 Feb 2023 12:01:33 +0000 (12:01 +0000)]
Replace rawmemchr (s, '\0') with strchr
Almost all uses of rawmemchr find the end of a string. Since most targets use
a generic implementation, replacing it with strchr is better since that is
optimized by compilers into strlen (s) + s. Also fix the generic rawmemchr
implementation to use a cast to unsigned char in the if statement.
Wilco Dijkstra [Wed, 1 Feb 2023 18:45:19 +0000 (18:45 +0000)]
AArch64: Improve SVE memcpy and memmove
Improve SVE memcpy by copying 2 vectors if the size is small enough.
This improves performance of random memcpy by ~9% on Neoverse V1, and
33-64 byte copies are ~16% faster.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Carlos O'Donell [Thu, 19 Jan 2023 11:50:20 +0000 (12:50 +0100)]
Account for grouping in printf width (bug 30068)
This is a partial fix for mishandling of grouping when formatting
integers. It properly computes the width in the presence of grouping
characters when the width is larger than the number of significant
digits. The precision related issue is documented in bug 23432.
If using -D_FORITFY_SOURCE=3 (in my case, I've patched GCC to add
=3 instead of =2 (we've done =2 for years in Gentoo)), building
glibc tests will fail on testmb like:
```
<command-line>: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>: note: this is the location of the previous definition
cc1: all warnings being treated as errors
make[2]: *** [../o-iterator.mk:9: /var/tmp/portage/sys-libs/glibc-2.36/work/build-x86-x86_64-pc-linux-gnu-nptl/stdlib/testmb.o] Error 1
make[2]: *** Waiting for unfinished jobs....
```
It's just because we're always setting -D_FORTIFY_SOURCE=2
rather than unsetting it first. If F_S is already 2, it's harmless,
but if it's another value (say, 1, or 3), the compiler will bawk.
(I'm not aware of a reason this couldn't be tested with =3,
but the toolchain support is limited for that (too new), and we want
to run the tests everywhere possible.)
Signed-off-by: Sam James <sam@gentoo.org> Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Flavio Cruz [Thu, 2 Feb 2023 07:42:18 +0000 (02:42 -0500)]
Remove sysdeps/mach/i386/machine-sp.h
This file is not used today since we end up using
sysdeps/i386/htl/machine-sp.h. Getting the stack pointer does not need
to be hurd specific and can go into sysdeps/<arch>.
Message-Id: <Y9tpWs2WOgE/Duiq@jupiter.tail36e24.ts.net>
Define the __glibc_fortify and other macros only when __FORTIFY_LEVEL >
0. This has the effect of not defining these macros on older C90
compilers that do not have support for variable length argument lists.
Also trim off the trailing backslashes from the definition of
__glibc_fortify and __glibc_fortify_n macros.
Sergey Bugaev [Mon, 30 Jan 2023 12:52:16 +0000 (15:52 +0300)]
hurd: Implement SHM_ANON
This adds a special SHM_ANON value that can be passed into shm_open ()
in place of a name. When called in this way, shm_open () will create a
new anonymous shared memory file. The file will be created in the same
way that other shared memory files are created (i.e., under /dev/shm/),
except that it is not given a name and therefore cannot be reached from
the file system, nor by other calls to shm_open (). This is accomplished
by utilizing O_TMPFILE.
This is intended to be compatible with FreeBSD's API of the same name.
Sergey Bugaev [Mon, 30 Jan 2023 12:52:15 +0000 (15:52 +0300)]
hurd: Implement O_TMPFILE
This is a flag that causes open () to create a new, unnamed file in the
same filesystem as the given directory. The file descriptor can be
simply used in the creating process as a temporary file, or shared with
children processes via fork (), or sent over a Unix socket. The file can
be left anonymous, in which case it will be deleted from the backing
file system once all copies of the file descriptor are closed, or given
a permanent name with a linkat () call, such as the following:
int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700);
/* Do something with the file... */
linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH);
In between creating the file and linking it to the file system, it is
possible to set the file content, mode, ownership, author, and other
attributes, so that the file visibly appears in the file system (perhaps
replacing another file) atomically, with all of its attributes already
set up.
The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to
user programs. Previously, dir_mkfile was used by glibc internally, in
particular for implementing tmpfile (), but not exposed to user programs
through a Unix-level API.
O_TMPFILE was initially introduced by Linux. This implementation is
intended to be compatible with the Linux implementation, except that the
O_EXCL flag is not given the special meaning when used together with
O_TMPFILE, unlike on Linux.
Sergey Bugaev [Mon, 30 Jan 2023 12:52:14 +0000 (15:52 +0300)]
hurd: Consolidate file_name_lookup implementation
Instead of __file_name_lookup_at delegating to __file_name_lookup
in simple cases, make __file_name_lookup_at deal with both cases, and
have __file_name_lookup simply wrap __file_name_lookup_at.
Add an optimization to avoid calling clone3 when glibc detects that
there is no kernel support. It also adds __ASSUME_CLONE3, which allows
skipping this optimization and issuing the clone3 syscall directly.
It does not handle the the small window between 5.3 and 5.5 for
posix_spawn (CLONE_CLEAR_SIGHAND was added in 5.5).
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
linux: Add clone3 CLONE_CLEAR_SIGHAND optimization to posix_spawn
The clone3 flag resets all signal handlers of the child not set to
SIG_IGN to SIG_DFL. It allows to skip most of the sigaction calls
to setup child signal handling, where previously a posix_spawn
had to issue 2 times NSIG sigaction calls (one to obtain the current
disposition and another to set either SIG_DFL or SIG_IGN).
With POSIX_SPAWN_SETSIGDEF the child will setup the signal for the case
where the disposition is SIG_IGN.
The code must handle the fallback where clone3 is not available. This is
done by splitting __clone_internal_fallback from __clone_internal.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
All internal callers of __clone3 should provide an already aligned
stack. Removing the stack alignment in __clone3 is a net gain: it
simplifies the internal function contract (mask/unmask signals) along
with the arch-specific code.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Different than kernel, clone3 returns EINVAL for NULL struct
clone_args or function pointer. This is similar to clone
interface that return EINVAL for NULL function argument.
It also clean up the Linux clone3.h interface, since it not
currently exported.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Noah Goldstein [Tue, 31 Jan 2023 23:46:56 +0000 (17:46 -0600)]
x86: Fix strncat-avx2.S reading past length [BZ #30065]
Occurs when `src` has no null-term.
Two cases:
1) Zero-length check is doing:
```
test %rdx, %rdx
jl L(zero_len)
```
which doesn't actually check zero (was at some point `decq` and the
flag never got updated).
The fix is just make the flag `jle` i.e:
```
test %rdx, %rdx
jle L(zero_len)
```
2) Length check in page-cross case checking if we should continue is
doing:
```
cmpq %r8, %rdx
jb L(page_cross_small)
```
which means we will continue searching for null-term if length ends at
the end of a page and there was no null-term in `src`.
The fix is to make the flag:
```
cmpq %r8, %rdx
jbe L(page_cross_small)
```
Florian Weimer [Tue, 31 Jan 2023 21:22:02 +0000 (22:22 +0100)]
libio: Update number of written bytes in dprintf implementation
The __printf_buffer_flush_dprintf function needs to record that
the buffer has been written before reusing it. Without this
accounting, dprintf always returns zero.
Paul Pluzhnikov [Wed, 25 Jan 2023 00:39:31 +0000 (00:39 +0000)]
Document '%F' format specifier
The '%F' format specifier was implemented in commit 6c46718f9f0 on
2000-08-23, but remains undocumented in the manual.
https://stackoverflow.com/questions/75157669/format-specifier-f-missing-from-glibcs-documentation
Sajan Karumanchi [Wed, 18 Jan 2023 17:29:04 +0000 (18:29 +0100)]
x86: Cache computation for AMD architecture.
All AMD architectures cache details will be computed based on
__cpuid__ `0x8000_001D` and the reference to __cpuid__ `0x8000_0006` will be
zeroed out for future architectures.
Joseph Myers [Tue, 17 Jan 2023 20:41:04 +0000 (20:41 +0000)]
Add IPPROTO_L2TP from Linux 6.1 to netinet/in.h
Linux 6.1 adds a define IPPROTO_L2TP to its include/uapi/linux/in.h
(not strictly a new constant, since it's moved from
include/uapi/linux/l2tp.h). Add this constant to glibc's
netinet/in.h.
Wilco Dijkstra [Wed, 11 Jan 2023 13:53:19 +0000 (13:53 +0000)]
AArch64: Improve strrchr
Use shrn for narrowing the mask which simplifies code and speeds up small
strings. Unroll the first search loop to improve performance on large
strings.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Wilco Dijkstra [Wed, 11 Jan 2023 13:53:05 +0000 (13:53 +0000)]
AArch64: Optimize strnlen
Optimize strnlen using the shrn instruction and improve the main loop.
Small strings are around 10% faster, large strings are 40% faster on
modern CPUs.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>