Joseph Myers [Fri, 20 Sep 2024 23:24:45 +0000 (23:24 +0000)]
Add tests of more strtod special cases
There is very little test coverage of inputs to strtod-family
functions that don't contain anything that can be parsed as a number
(one test of ".y" in tst-strtod2), and none that I can see of skipping
initial whitespace. Add some tests of these things to tst-strtod2.
Joseph Myers [Fri, 20 Sep 2024 23:24:02 +0000 (23:24 +0000)]
Add more tests of strtod end pointer
Although there are some tests in tst-strtod2 and tst-strtod3 for the
end pointer provided by strtod when it doesn't parse the whole string,
they aren't very thorough. Add tests of more such cases to
tst-strtod2.
Joseph Myers [Fri, 20 Sep 2024 23:23:13 +0000 (23:23 +0000)]
Make tst-strtod2 and tst-strtod5 type-generic
Some of the strtod tests use type-generic machinery in tst-strtod.h to
test the strto* functions for all floating types, while others only
test double even when the tests are in fact meaningful for all
floating types.
Convert tst-strtod2 and tst-strtod5 to use the type-generic machinery
so they test all floating types. I haven't tried to convert them to
use newer test interfaces in other ways, just made the changes
necessary to use the type-generic machinery.
Implement run-built-tests=no for make xcheck, always build xtests
Previously, the second occurrence of the xtests target
expected all xtests to run (as the result of specifying
$(xtests)), but these tests have not been run due to
the the first xtests target is set up for run-built-tests=no:
it only runs tests in $(xtests-special). Consequently,
xtests are reported as UNSUPPORTED with “make xcheck
run-built-tests=no”. The xtests were not built, either.
After this change always, xtests are built regardless
of the $(run-built-tests) variable (except for xtests listed
in $(tests-unsupported)). To fix the UNSUPPORTED issue,
introduce xtests-expected and use that manage test
expectations in the second xtests target.
Add new testcase elf/tst-startup-errno.c which tests that errno is set
to 0 at first ELF constructor execution and at the start of the
program's main function.
Add new file libio/tst-fclose-unopened2.c that tests whether fclose on an
unopened file returns EOF.
This test differs from tst-fclose-unopened.c by ensuring the file's buffer
is allocated prior to double-fclose. A comment in tst-fclose-unopened.c
now clarifies that it is testing a file with an unallocated buffer.
Calling fclose on unopened files normally causes a use-after-free bug,
however the standard streams are an exception since they are not
deallocated by fclose.
iconv: Support in-place conversions (bug 10460, bug 32033)
Check if any of the input files overlaps with the output file, and use
a temporary file in this case, so that the input is no clobbered
before it is read. This fixes bug 10460. It allows to use iconv
more easily as a functional replacement for GNU recode.
The updated output buffer management truncates the output file
if there is no input, fixing bug 32033.
In several converters, a __GCONV_ILLEGAL_INPUT result gets overwritten
with __GCONV_FULL_OUTPUT. As a result, iconv (the function) returns
E2BIG instead of EILSEQ. The iconv program does not see the original
EILSEQ failure, does not recognize the invalid input, and may
incorrectly exit successfully.
To address this, a new __flags bit is used to indicate a sticky input
error state. All __GCONV_ILLEGAL_INPUT results are replaced with a
function call that sets this new __GCONV_ENCOUNTERED_ILLEGAL_INPUT and
returns __GCONV_ILLEGAL_INPUT. The iconv program checks for
__GCONV_ENCOUNTERED_ILLEGAL_INPUT and overrides the exit status.
The converter changes introducing __gconv_mark_illegal_input are
mostly mechanical, except for the res variable initialization in
iconvdata/iso-2022-jp.c: this error gets overwritten with __GCONV_OK
and other results in the following code. If res ==
__GCONV_ILLEGAL_INPUT afterwards, STANDARD_TO_LOOP_ERR_HANDLER below
will handle it.
The __gconv_mark_illegal_input changes do not alter the errno value
set by the iconv function. This is simpler to implement than
reviewing each __GCONV_FULL_OUTPUT result and adjust it not to
override a previous __GCONV_ILLEGAL_INPUT result. Doing it that way
would also change some E2BIG errors in to EILSEQ errors, so it had to
be done conditionally (under a flag set by the iconv program only), to
avoid confusing buffer management in other applications.
iconv: Do not use mmap in iconv (the program) (bug 17703)
On current systems, very large files are needed before
mmap becomes beneficial. Simplify the implementation.
This exposed that inptr was not initialized correctly in
process_fd. Handling multiple input files resulted in
EFAULT in read because a null pointer was passed. This
could be observed previously if an input file was not
mappable and was reported as bug 17703.
Joe Ramsay [Thu, 19 Sep 2024 16:34:02 +0000 (17:34 +0100)]
AArch64: Add vector logp1 alias for log1p
This enables vectorisation of C23 logp1, which is an alias for log1p.
There are no new tests or ulp entries because the new symbols are simply
aliases.
hurd: Avoid file_check_access () RPC for access (F_OK)
A common use case of access () / faccessat () is checking for file
existence, not any specific access permissions. In that case, we can
avoid doing the file_check_access () RPC; whether the given path had
been successfully resolved to a file is all we need to know to answer.
This is prompted by GLib switching to use faccessat (F_OK) to implement
g_file_query_exists () for local files.
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/4272
tst: Extend cross-test-ssh.sh to support passing glibc tunables
This patch adds new flag --glibctunables to the cross-test-ssh.sh script
to pass Glibc tunables to the system on which tests are executed.
The value to pass can be also provided via the GLIBC_TUNABLES environment
variable.
This works similar to the TIMEOUTFACTOR variable.
Sometimes it is useful to cross test glibc with some non-default tunable,
and a global environment variable is the easiest way to inject some
tunable value into most tests. With this patch using cross-test-ssh.sh
script becomes very similar to running a test natively on the local host
when using non-default tunable is important. Reviewed-by: Arjun Shankar <arjun@redhat.com>
Linux: Add the sched_setattr and sched_getattr functions
And struct sched_attr.
In sysdeps/unix/sysv/linux/bits/sched.h, the hack that defines
sched_param around the inclusion of <linux/sched/types.h> is quite
ugly, but the definition of struct sched_param has already been
dropped by the kernel, so there is nothing else we can do and maintain
compatibility of <sched.h> with a wide range of kernel header
versions. (An alternative would involve introducing a separate header
for this functionality, but this seems unnecessary.)
The existing sched_* functions that change scheduler parameters
are already incompatible with PTHREAD_PRIO_PROTECT mutexes, so
there is no harm in adding more functionality in this area.
The documentation mostly defers to the Linux manual pages.
The reading loops did not check for read failures. Addresses
a static analysis report.
Manually tested by compiling a program with the GCC's
-finstrument-functions option, running it with
“LD_PRELOAD=debug/libpcprofile.so PCPROFILE_OUTPUT=output-file”,
and reviewing the output of “debug/pcprofiledump output-file”.
Improve small memsets by avoiding branches and use overlapping stores.
Use DC ZVA for copies over 128 bytes. Remove unnecessary code for ZVA sizes
other than 64 and 128. Performance of random memset benchmark improves by 24%
on Neoverse N1.
Joe Ramsay [Mon, 9 Sep 2024 12:00:01 +0000 (13:00 +0100)]
aarch64: Avoid redundant MOVs in AdvSIMD F32 logs
Since the last operation is destructive, the first argument to the FMA
also has to be the first argument to the special-case in order to
avoid unnecessary MOVs. Reorder arguments and adjust special-case
bounds to facilitate this.
Joseph Myers [Fri, 6 Sep 2024 20:38:23 +0000 (20:38 +0000)]
Document limitations on streams passed to freopen
As recently discussed, document that freopen does not work with
streams opened with functions such as popen, fmemopen, open_memstream
or fopencookie. I've filed
<https://austingroupbugs.net/view.php?id=1855> to clarify this issue
in POSIX.
stdlib: Do not use GLIBC_PRIVATE ABI for errno in libc_nonshared.a
Using TLS directly introduces a GLIBC_PRIVATE ABI dependency
into libc_nonshared.a, and thus indirectly into applications.
Adding the !defined LIBC_NONSHARED condition deactivates direct
TLS access, and libc_nonshared.a code switches to using
__errno_location, like application code.
Currently, this has no effect because there is no code in
libc_nonshared.a that accesses errno.
Sync tzselect, zdump, zic to TZDB 2024b. This patch incorporates the
following TZDB source code changes:
6903dde3 Release 2024b 812aff32 Improve historical transitions in Mexico 1921-1997 52662566 Adjust to mailing list software change 7748036b Mention Internet RFC 9557 339e81d1 Mention Levine’s proposal to replace leap seconds b4e6ad2d No leap second on 2024-12-31 7eb5bf88 Asia/Choibalsan is now an alias for Asia/Ulaanbaatar 43450cbf Improve historical data for Portugal and former possessions. 13d7348b Typo and validation fixes. 3c39cde8 Fix typo for “removed” in a comment 03fd9e45 More documentation updates for POSIX.1-2024 eb3bcceb POSIX.1-2014 is now published 913b0410 tzselect: support POSIX.1-2024 offset range b5318b55 Document POSIX.1-2024 better 837609b7 Fix typo when making .txt man pages d56ae6ee SUPPORT_C89 now defaults to 1, not 0 b1fe113d Port ! to Solaris make 8f1fd321 Avoid crash in Solaris 10 /usr/xpg4/bin/make e0fcfdd6 Use ‘export VAR=VAL’ syntax eba43166 Avoid an awk invocation via $'...' 36479a80 Avoid some subshells in tzselect 7f6cf054 * tzselect.ksh: Assume POSIX.2 awk. a1cf1daf * tzselect.ksh: Assume POSIX.2 $PWD. a9b8e536 Assume POSIX.2 command substitution eaa4ef16 Avoid subshells when possible 9dac9eb7 Prefer $PWD to $(pwd) in Makefile fada6a4c Prefer $(CMD) to `CMD` in Makefile 3e871b9a Assume POSIX.2 and eschew ‘expr’ c5d67805 difftime isn’t pure either 5857c056 * CONTRIBUTING: Document build assumptions. 6822cc82 ‘make check’ no longer depends on curl+Internet cc6eb255 Document GCC bug 114833 and workaround bcbc86bf Scale back on function attribute use c0789e46 C23 [[reproducible]] and [[unsequenced]] fixups bbd88154 More updates to GCC_DEBUG_FLAGS for GCC 14 1a35b7c8 Spelling fixes f71085f2 POSIX.1-2024 removes asctime_r, ctime_r 70856f8e Adjust to refactored location of ctime, ctime_r aacd151d Update GCC_DEBUG_FLAGS for GCC 14 967dcf3b Sub-second history for Maputo and Zurich 782d0826 Make EET, MET and WET links a0b09c02 Mark CET, CST6CDT etc. as obsolescent db7fb40d Document SMPTE timecodes and rolling leaps 97232e18 Don’t be so sure about leap seconds going away 5b6a74fb Update some URLs a75a6251 * zic.8: Tweak for consistency. 1e75b31f Document what %s means before any rule applies 00c96cbb Conform to RFC 8536 section 3.2 for default type 3e944959 Document problems with stripped-down TZif readers 20fc91cf Shanks is likely wrong about Maputo switch to CAT d99589b6 * zic.8: Add missing tab character. 94e6b3b0 Switch to %z in main dataform 2cd57b93 Treat W-Eur like Port when reguarding ad6f6d94 Check that main.zi agrees with sources a43b030f .gitignore: Add .pdf, .ps, .s. Remove obsolete ‘yearistype’. 253ca020 * theory.html: ‘CLT’ → ‘LTC’ (per Michael H Deckers) a3dee8c8 * NEWS: ‘how’ → ‘now’ (thanks to Paul Goyette). ea6341c5 * theory.html: Mention NASA and CLT (per Arthur David Olson). 0dcebe37 America/Scoresbysund matches America/Nuuk from now on b1e07fb0 Update Vzic link (thanks to Allen Winter) a4b05030 Fix wday/mday typo in previous patch 732a4803 Document how to detect mktime failure reliably a64067e9 ziguard.awk: generalize for proposed Portugal patch 59c861fd Line up zdump examples 66c106c9 tzfile.5: srcfix e5553001 Fix .RS/.RE problem in tzfile.5 d647eb01 Add Doctorow book 59d4a1ba Asia/Almaty matches Asia/Tashkent from now on d4d3c3ba * asia: Update Philippine URLs (thanks to Guy Harris). 9fc11a27 Port unlikely overflow check to C23 b52a2969 Fix 2023d NEWS typo e48c5b53 Cite "The NTP Leap Second File" b1dc2122 Update Israel tz-link 6cf4e912 Extrapolate less from the 2022 CGPM resolution.
It fixes glibc build with gcc master [1].
Checked on x86_64-linux-gnu and on i686-linux-gnu.
[1] https://sourceware.org/pipermail/libc-alpha/2024-September/159571.html Reviewed-by: Paul Eggert <eggert@cs.ucla.edu>
Joseph Myers [Thu, 5 Sep 2024 20:08:10 +0000 (20:08 +0000)]
Fix freopen handling of ,ccs= (bug 23675)
As reported in bug 23675 and shown up in the recently added tests of
different cases of freopen (relevant part of the test currently
conditioned under #if 0 to avoid a failure resulting from this bug),
freopen wrongly forces the stream to unoriented even when a mode with
,ccs= is specified, though such a mode is supposed to result in a
wide-oriented stream. Move the clearing of _mode to before the actual
reopening occurs, so that the main fopen implementation can leave a
wide-oriented stream in the ,ccs= case.
powerpc64le: Build new strtod tests with long double ABI flags (bug 32145)
This fixes several test failures:
=====FAIL: stdlib/tst-strtod1i.out=====
Locale tests
all OK
Locale tests
all OK
Locale tests
strtold("1,5") returns -6,38643e+367 and not 1,5
strtold("1.5") returns 1,5 and not 1
strtold("1.500") returns 1 and not 1500
strtold("36.893.488.147.419.103.232") returns 1500 and not 3,68935e+19
Locale tests
all OK
Aaron Merey [Thu, 29 Aug 2024 16:02:25 +0000 (12:02 -0400)]
Test fclose on an unopened file.
Add new file libio/tst-fclosed-unopened.c that tests whether fclose on
an unopened file returns EOF.
Calling fclose on unopened files normally causes a use-after-free bug,
however the standard streams are an exception since they are not
deallocated by fclose.
fclose returning EOF for unopened files is not part of the external
contract but there are dependancies on this behaviour. For example,
gnulib's close_stdout in lib/closeout.c.
Joseph Myers [Thu, 5 Sep 2024 11:16:59 +0000 (11:16 +0000)]
Fix memory leak on freopen error return (bug 32140)
As reported in bug 32140, freopen leaks the FILE object when it
returns NULL: there is no valid use of the FILE * pointer (including
passing to freopen again or to fclose) after such an error return, so
the underlying object should be freed. Add code to free it.
Note 1: while I think it's clear from the relevant standards that the
object should be freed and the FILE * can't be used after the call in
this case (the stream is closed, which ends the lifetime of the FILE),
it's entirely possible that some existing code does in fact try to use
the existing FILE * in some way and could be broken by this change.
(Though the most common case for freopen may be stdin / stdout /
stderr, which _IO_deallocate_file explicitly checks for and does not
deallocate.)
Note 2: the deallocation is only done in the _IO_IS_FILEBUF case.
Other kinds of streams bypass all the freopen logic handling closing
the file, meaning a call to _IO_deallocate_file would neither be safe
(the FILE might still be linked into the list of all open FILEs) nor
sufficient (other internal memory allocations associated with the file
would not have been freed). I think the validity of freopen for any
other kind of stream will need clarifying with the Austin Group, but
if it is valid in any such case (where "valid" means "not undefined
behavior so required to close the stream" rather than "required to
successfully associate the stream with the new file in cases where
fopen would work"), more significant changes would be needed to ensure
the stream gets fully closed.
Joseph Myers [Thu, 5 Sep 2024 11:15:29 +0000 (11:15 +0000)]
Clear flags2 flags set from mode in freopen (bug 32134)
As reported in bug 32134, freopen does not clear the flags set in
fp->_flags2 by the "e", "m" or "c" mode characters. Clear these so
that they can be set or not as appropriate from the mode string passed
to freopen. The relevant test for "e" in tst-freopen2-main.c is
enabled accordingly; "c" is expected to be covered in a separately
written test (and while tst-freopen2-main.c does include transitions
to and from "m", that's not really a semantic flag intended to result
in behaving in an observably different way).
Florian Weimer [Fri, 30 Aug 2024 19:51:46 +0000 (21:51 +0200)]
Bundle <linux/fuse.h> userspace header from Linux 6.10
And include the required licensing information. The only
change is a removed trailing empty line in
LICENSES/exceptions/Linux-syscall-note.
Bundling <linux/fuse.h> is the recommended way to deal with
the evolution of the FUSE userspace interface because
structs change sizes over time. The kernel maintains
compatibility, but source-level compatibility on recompilation
may require additional code that is aware of older struct sizes.
Signed-off-by: Florian Weimer <fweimer@redhat.com> Reviewed-by: DJ Delorie <dj@redhat.com>
Joseph Myers [Wed, 4 Sep 2024 16:32:21 +0000 (16:32 +0000)]
Add more thorough tests of freopen
freopen is rather minimally tested in libio/tst-freopen and
libio/test-freopen. Add some more thorough tests, covering different
cases for change of mode in particular. The tests are run for both
freopen and freopen64 (given that those functions have two separate
copies of much of the code, so any bug fix directly in the freopen
code would probably need applying in both places).
Note that there are two parts of the tests disabled because of bugs
discovered through running the tests, with bug numbers given in
comments. I expect to address those separately. The tests also don't
cover changes to cancellation ("c" in mode); I think that will better
be handled through a separate test. Also to handle separately:
testing on stdin / stdout / stderr; documenting lack of support for
streams opened with popen / fmemopen / open_memstream / fopencookie;
maybe also a chroot test without /proc; maybe also more thorough tests
for large file handling on 32-bit systems (freopen64).
Joseph Myers [Wed, 4 Sep 2024 13:21:23 +0000 (13:21 +0000)]
Do not set errno for overflowing NaN payload in strtod/nan (bug 32045)
As reported in bug 32045, it's incorrect for strtod/nan functions to
set errno based on overflowing payload (strtod should only set errno
for overflow / underflow of its actual result, and potentially if
nothing in the string can be parsed as a number at all; nan should be
a pure function that never sets it). Save and restore errno around
the internal strtoull call and add associated test coverage.
Joseph Myers [Wed, 4 Sep 2024 13:20:18 +0000 (13:20 +0000)]
Improve NaN payload testing
There are two separate sets of tests of NaN payloads in glibc:
* libm-test-{get,set}payload* verify that getpayload, setpayload,
setpayloadsig and __builtin_nan functions are consistent in their
payload handling.
* test-nan-payload verifies that strtod-family functions and the
not-built-in nan functions are consistent in their payload handling.
Nothing, however, connects the two sets of functions (i.e., verifies
that strtod / nan are consistent with getpayload / setpayload /
__builtin_nan).
Improve test-nan-payload to check actual payload value with getpayload
rather than just verifying that the strtod and nan functions produce
the same NaN. Also check that the NaNs produced aren't signaling and
extend the tests to cover _FloatN / _FloatNx.
Florian Weimer [Fri, 30 Aug 2024 18:37:18 +0000 (20:37 +0200)]
io: Add error tests for fchmod
On Linux most descriptors that do not correspond to file system
entities (such as anonymous pipes and sockets) have file permissions
that can be changed. While it is possible to create a custom file
system that returns (say) EINVAL for an fchmod attempt, testing this
does not appear to be useful.
powerpc64: Fix syscall_cancel build for powerpc64le-linux-gnu [BZ #32125]
In __syscall_cancel_arch, there's a tail call to __syscall_do_cancel.
On P10, since the caller uses the TOC and the callee is using
PC-relative addressing, there's only a branch instruction with no NOPs
to restore the TOC, which causes the build error. The fix involves adding
the NOTOC directive to the branch instruction, informing the linker
not to generate a TOC stub, thus resolving the issue.
Joseph Myers [Tue, 27 Aug 2024 20:41:54 +0000 (20:41 +0000)]
Make __strtod_internal tests type-generic
Some of the strtod tests use type-generic machinery in tst-strtod.h to
test the strto* functions for all floating types, while others only
test double even when the tests are in fact meaningful for all
floating types.
Convert the tests of the internal __strtod_internal interface to cover
all floating types. I haven't tried to convert them to use newer test
interfaces in other ways, just made the changes necessary to use the
type-generic machinery. As an internal interface, there are no
aliases for different types with the same ABI (however,
__strtold_internal is defined even if long double has the same ABI as
double), so macros used by the type-generic testing code are redefined
as needed to avoid expecting such aliases to be present.
Joseph Myers [Tue, 27 Aug 2024 12:41:02 +0000 (12:41 +0000)]
Fix strtod subnormal rounding (bug 30220)
As reported in bug 30220, the implementation of strtod-family
functions has a bug in the following case: the input string would,
with infinite exponent range, take one more bit to represent than is
available in the normal precision of the return type; the value
represented is in the subnormal range; and there are no nonzero bits
in the value, below those that can be represented in subnormal
precision, other than the least significant bit and possibly the
0.5ulp bit. In this case, round_and_return ends up discarding the
least significant bit.
Fix by saving that bit to merge into more_bits (it can't be merged in
at the time it's computed, because more_bits mustn't include this bit
in the case of after-rounding tininess detection checking if the
result is still subnormal when rounded to normal precision, so merging
this bit into more_bits needs to take place after that check).
Joseph Myers [Tue, 27 Aug 2024 12:38:01 +0000 (12:38 +0000)]
More thoroughly test underflow / errno in tst-strtod-round
Add tests of underflow in tst-strtod-round, and thus also test for
errno being unchanged when there is neither overflow nor underflow.
The errno setting before the function call to test for being unchanged
is adjusted to set errno to 12345 instead of 0, so that any bugs where
strtod sets errno to 0 would be detected.
This doesn't add any new test inputs for tst-strtod-round, and in
particular doesn't cover the edge cases of underflow the way
tst-strtod-underflow does (none of the existing test inputs for
tst-strtod-round actually exercise cases that have underflow with
before-rounding tininess detection but not with after-rounding
tininess detection), but at least it provides some coverage (as per
the recent discussions) that ordinary non-overflowing non-underflowing
inputs to these functions do not set errno.
Florian Weimer [Wed, 7 Aug 2024 12:57:41 +0000 (14:57 +0200)]
manual: Add Descriptor-Relative Access section
Reference this new section from the O_PATH documentation.
And document the functions openat, openat64, fstatat, fstatat64.
(The safety assessment for fstatat was already obsolete because
current glibc assumes kernel support for the underlying system
call.)
Feifei Wang [Mon, 19 Aug 2024 06:57:54 +0000 (14:57 +0800)]
x86: Add cache information support for Hygon processors
Add hygon branch in dl_init_cacheinfo function to initialize
cache size variables for hygon processors. In the meanwhile,
add handle_hygon() function to get cache information.
Signed-off-by: Feifei Wang <wangfeifei@hygon.cn> Reviewed-by: Jing Li <lijing@hygon.cn> Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Feifei Wang [Mon, 19 Aug 2024 06:57:53 +0000 (14:57 +0800)]
x86: Add new architecture type for Hygon processors
Add a new architecture type arch_kind_hygon to spilt Hygon branch
from AMD. This is to facilitate the Hygon processors to make settings
that are suitable for its own characteristics.
Signed-off-by: Feifei Wang <wangfeifei@hygon.cn> Reviewed-by: Jing Li <lijing@hygon.cn> Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Samuel Thibault [Sun, 25 Aug 2024 01:35:29 +0000 (03:35 +0200)]
mach: Fix bogus negative return
One can be very unlucky to call time_now first just before a second switch,
and mach_msg sleep just a bit more enough for the second time_now call to
count one second too many (or even more if scheduling is really unlucky).
So we have to protect against returning a bogus negative value in such case.
Mahesh Bodapati [Fri, 23 Aug 2024 21:48:32 +0000 (16:48 -0500)]
powerpc64: Optimize strcpy and stpcpy for Power9/10
This patch modifies the current Power9 implementation of strcpy and
stpcpy to optimize it for Power9 and Power10.
No new Power10 instructions are used, so the original Power9 strcpy
is modified instead of creating a new implementation for Power10.
The changes also affect stpcpy, which uses the same implementation
with some additional code before returning.
Improvements compared to the old Power9 version:
Use simple comparisons for the first ~512 bytes:
The main loop is good for long strings, but comparing 16B each time is
better for shorter strings. After aligning the address to 16 bytes, we
unroll the loop four times, checking 128 bytes each time. There may be
some overlap with the main loop for unaligned strings, but it is better
for shorter strings.
Loop with 64 bytes for longer bytes:
Use 4 consecutive lxv/stxv instructions.
Showed an average improvement of 13%.
Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com> Reviewed-by: Peter Bergner <bergner@linux.ibm.com>
nptl: Fix Race conditions in pthread cancellation [BZ#12683]
The current racy approach is to enable asynchronous cancellation
before making the syscall and restore the previous cancellation
type once the syscall returns, and check if cancellation has happen
during the cancellation entrypoint.
As described in BZ#12683, this approach shows 2 problems:
1. Cancellation can act after the syscall has returned from the
kernel, but before userspace saves the return value. It might
result in a resource leak if the syscall allocated a resource or a
side effect (partial read/write), and there is no way to program
handle it with cancellation handlers.
2. If a signal is handled while the thread is blocked at a cancellable
syscall, the entire signal handler runs with asynchronous
cancellation enabled. This can lead to issues if the signal
handler call functions which are async-signal-safe but not
async-cancel-safe.
For the cancellation to work correctly, there are 5 points at which the
cancellation signal could arrive:
[ ... )[ ... )[ syscall ]( ...
1 2 3 4 5
1. Before initial testcancel, e.g. [*... testcancel)
2. Between testcancel and syscall start, e.g. [testcancel...syscall start)
3. While syscall is blocked and no side effects have yet taken
place, e.g. [ syscall ]
4. Same as 3 but with side-effects having occurred (e.g. a partial
read or write).
5. After syscall end e.g. (syscall end...*]
And libc wants to act on cancellation in cases 1, 2, and 3 but not
in cases 4 or 5. For the 4 and 5 cases, the cancellation will eventually
happen in the next cancellable entrypoint without any further external
event.
The proposed solution for each case is:
1. Do a conditional branch based on whether the thread has received
a cancellation request;
2. It can be caught by the signal handler determining that the saved
program counter (from the ucontext_t) is in some address range
beginning just before the "testcancel" and ending with the
syscall instruction.
3. SIGCANCEL can be caught by the signal handler and determine that
the saved program counter (from the ucontext_t) is in the address
range beginning just before "testcancel" and ending with the first
uninterruptable (via a signal) syscall instruction that enters the
kernel.
4. In this case, except for certain syscalls that ALWAYS fail with
EINTR even for non-interrupting signals, the kernel will reset
the program counter to point at the syscall instruction during
signal handling, so that the syscall is restarted when the signal
handler returns. So, from the signal handler's standpoint, this
looks the same as case 2, and thus it's taken care of.
5. For syscalls with side-effects, the kernel cannot restart the
syscall; when it's interrupted by a signal, the kernel must cause
the syscall to return with whatever partial result is obtained
(e.g. partial read or write).
6. The saved program counter points just after the syscall
instruction, so the signal handler won't act on cancellation.
This is similar to 4. since the program counter is past the syscall
instruction.
So The proposed fixes are:
1. Remove the enable_asynccancel/disable_asynccancel function usage in
cancellable syscall definition and instead make them call a common
symbol that will check if cancellation is enabled (__syscall_cancel
at nptl/cancellation.c), call the arch-specific cancellable
entry-point (__syscall_cancel_arch), and cancel the thread when
required.
2. Provide an arch-specific generic system call wrapper function
that contains global markers. These markers will be used in
SIGCANCEL signal handler to check if the interruption has been
called in a valid syscall and if the syscalls has side-effects.
A reference implementation sysdeps/unix/sysv/linux/syscall_cancel.c
is provided. However, the markers may not be set on correct
expected places depending on how INTERNAL_SYSCALL_NCS is
implemented by the architecture. It is expected that all
architectures add an arch-specific implementation.
3. Rewrite SIGCANCEL asynchronous handler to check for both canceling
type and if current IP from signal handler falls between the global
markers and act accordingly.
4. Adjust libc code to replace LIBC_CANCEL_ASYNC/LIBC_CANCEL_RESET to
use the appropriate cancelable syscalls.
5. Adjust 'lowlevellock-futex.h' arch-specific implementations to
provide cancelable futex calls.
Some architectures require specific support on syscall handling:
* On i386 the syscall cancel bridge needs to use the old int80
instruction because the optimized vDSO symbol the resulting PC value
for an interrupted syscall points to an address outside the expected
markers in __syscall_cancel_arch. It has been discussed in LKML [1]
on how kernel could help userland to accomplish it, but afaik
discussion has stalled.
Also, sysenter should not be used directly by libc since its calling
convention is set by the kernel depending of the underlying x86 chip
(check kernel commit 30bfa7b3488bfb1bb75c9f50a5fcac1832970c60).
* mips o32 is the only kABI that requires 7 argument syscall, and to
avoid add a requirement on all architectures to support it, mips
support is added with extra internal defines.
Checked on aarch64-linux-gnu, arm-linux-gnueabihf, powerpc-linux-gnu,
powerpc64-linux-gnu, powerpc64le-linux-gnu, i686-linux-gnu, and
x86_64-linux-gnu.
[1] https://lkml.org/lkml/2016/3/8/1105 Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Joseph Myers [Thu, 22 Aug 2024 11:25:14 +0000 (11:25 +0000)]
Test mkdirat use of mode argument
The test io/tst-mkdirat doesn't verify the permissions on the created
directory (thus, doesn't verify at all anything about how mkdirat uses
the mode argument). Add checks of this to the existing test.
Joseph Myers [Wed, 21 Aug 2024 19:58:14 +0000 (19:58 +0000)]
Add more tests of getline
There is very little test coverage for getline (only a minimal
stdio-common/tstgetln.c which doesn't verify anything about the
results of the getline calls). Add some more thorough tests
(generally using fopencookie for convenience in testing various cases
for what the input and possible errors / EOF in the file read might
look like).
Note the following regarding testing of error cases:
* Nothing is said in the specifications about what if anything might
be written into the buffer, and whether it might be reallocated, in
error cases. The expectation of the tests (required to avoid memory
leaks on error) is that at least on error cases, the invariant that
lineptr points to at least n bytes is maintained.
* The optional EOVERFLOW error case specified in POSIX, "The number of
bytes to be written into the buffer, including the delimiter
character (if encountered), would exceed {SSIZE_MAX}.", doesn't seem
practically testable, as any case reading so many characters (half
the address space) would also be liable to run into allocation
failure along (ENOMEM) the way.
* If a read error occurs part way through reading an input line, it
seems unclear whether a partial line should be returned by getline
(avoid input getting lost), which is what glibc does at least in the
fopencookie case used in this test, or whether getline should return
-1 (error) (so avoiding the program misbehaving by processing a
truncated line as if it were complete). (There was a short,
inconclusive discussion about this on the Austin Group list on 9-10
November 2014.)
* The POSIX specification of getline inherits errors from fgetc. I
didn't try to cover fgetc errors systematically, just one example of
such an error.
Samuel Thibault [Tue, 20 Aug 2024 14:22:07 +0000 (16:22 +0200)]
Rules: Also build memcheck tests even when not running them
This will avoid in the future cases like a57cbbd85379 ("malloc: Link
threading tests with $(shared-thread-library") missing the memcheck
cases added in 251843e16fce ("malloc: Link threading tests with
$(shared-thread-library)")
x86: Unifies 'strnlen-evex' and 'strnlen-evex512' implementations.
This commit uses a common implementation 'strnlen-evex-base.S' for both
'strnlen-evex' and 'strnlen-evex512'
This patch serves both to reduce the number of implementations, and it also does some small optimizations that benefit strnlen-evex and strnlen-evex512.
All tests pass on x86.
Benchmarks were taken on SKX.
https://www.intel.com/content/www/us/en/products/sku/123613/intel-core-i97900x-xseries-processor-13-75m-cache-up-to-4-30-ghz/specifications.html
Geometric mean for strnlen-evex over all benchmarks (N=10) was (new/old) 0.881
Geometric mean for strnlen-evex512 over all benchmarks (N=10) was (new/old) 0.953
Florian Weimer [Mon, 19 Aug 2024 13:48:03 +0000 (15:48 +0200)]
string: strerror, strsignal cannot use buffer after dlmopen (bug 32026)
Secondary namespaces have a different malloc. Allocating the
buffer in one namespace and freeing it another results in
heap corruption. Fix this by using a static string (potentially
translated) in secondary namespaces. It would also be possible
to use the malloc from the initial namespace to manage the
buffer, but these functions would still not be safe to use in
auditors etc. because a call to strerror could still free a
buffer while it is used by the application. Another approach
could use proper initial-exec TLS, duplicated in secondary
namespaces, but that would need a callback interface for freeing
libc resources in namespaces on thread exit, which does not exist
today.
Florian Weimer [Fri, 16 Aug 2024 14:05:20 +0000 (16:05 +0200)]
support: Use macros for *stat wrappers
Macros will automatically use the correct types, without
having to fiddle with internal glibc macros. It's also
impossible to get the types wrong due to aliasing because
support_check_stat_fd and support_check_stat_path do not
depend on the struct stat* types.
Florian Weimer [Fri, 16 Aug 2024 14:05:20 +0000 (16:05 +0200)]
io: Use struct statx and xstatx in tests
This avoids the need to define struct_statx to an appropriate
struct stat type variant because struct statx does not change
based on time/file offset flags.
nptl: Fix extraneous testing run by tst-rseq-nptl in the test driver
Fix an issue with commit 8f4632deb354 ("Linux: rseq registration tests")
and prevent testing from being run in the process of the test driver
itself rather than just the test child where one has been forked. The
problem here is the unguarded use of a destructor to call a part of the
testing. The destructor function, 'do_rseq_destructor_test' is called
implicitly at program completion, however because it is associated with
the executable itself rather than an individual process, it is called
both in the test child *and* in the test driver itself.
Prevent this from happening by providing a guard variable that only
enables test invocation from 'do_rseq_destructor_test' in the process
that has first run 'do_test'. Consequently extra testing is invoked
from 'do_rseq_destructor_test' only once and in the correct process,
regardless of the use or the lack of of the '--direct' option. Where
called in the controlling test driver process that has neved called
'do_test' the destructor function silently returns right away without
taking any further actions, letting the test driver fail gracefully
where applicable.
This arrangement prevents 'tst-rseq-nptl' from ever causing testing to
hang forever and never complete, such as currently happening with the
'mips-linux-gnu' (o32 ABI) target.
Carlos O'Donell [Thu, 15 Aug 2024 12:12:35 +0000 (08:12 -0400)]
Report error if setaffinity wrapper fails (Bug 32040)
Previously if the setaffinity wrapper failed the rest of the subtest
would not execute and the current subtest would be reported as passing.
Now if the setaffinity wrapper fails the subtest is correctly reported
as faling. Tested manually by changing the conditions of the affinity
call including setting size to zero, or checking the wrong condition.
ungetc: Fix backup buffer leak on program exit [BZ #27821]
If a file descriptor is left unclosed and is cleaned up by _IO_cleanup
on exit, its backup buffer remains unfreed, registering as a leak in
valgrind. This is not strictly an issue since (1) the program should
ideally be closing the stream once it's not in use and (2) the program
is about to exit anyway, so keeping the backup buffer around a wee bit
longer isn't a real problem. Free it anyway to keep valgrind happy
when the streams in question are the standard ones, i.e. stdout, stdin
or stderr.
Also, the _IO_have_backup macro checks for _IO_save_base,
which is a roundabout way to check for a backup buffer instead of
directly looking for _IO_backup_base. The roundabout check breaks when
the main get area has not been used and user pushes a char into the
backup buffer with ungetc. Fix this to use the _IO_backup_base
directly.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
ungetc: Fix uninitialized read when putting into unused streams [BZ #27821]
When ungetc is called on an unused stream, the backup buffer is
allocated without the main get area being present. This results in
every subsequent ungetc (as the stream remains in the backup area)
checking uninitialized memory in the backup buffer when trying to put a
character back into the stream.
Avoid comparing the input character with buffer contents when in backup
to avoid this uninitialized read. The uninitialized read is harmless in
this context since the location is promptly overwritten with the input
character, thus fulfilling ungetc functionality.
Also adjust wording in the manual to drop the paragraph that says glibc
cannot do multiple ungetc back to back since with this change, ungetc
can actually do this.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Noah Goldstein [Wed, 14 Aug 2024 06:37:31 +0000 (14:37 +0800)]
x86: Add `Avoid_STOSB` tunable to allow NT memset without ERMS
The goal of this flag is to allow targets which don't prefer/have ERMS
to still access the non-temporal memset implementation.
There are 4 cases for tuning memset:
1) `Avoid_STOSB && Avoid_Non_Temporal_Memset`
- Memset with temporal stores
2) `Avoid_STOSB && !Avoid_Non_Temporal_Memset`
- Memset with temporal/non-temporal stores. Non-temporal path
goes through `rep stosb` path. We accomplish this by setting
`x86_rep_stosb_threshold` to
`x86_memset_non_temporal_threshold`.
3) `!Avoid_STOSB && Avoid_Non_Temporal_Memset`
- Memset with temporal stores/`rep stosb`
3) `!Avoid_STOSB && !Avoid_Non_Temporal_Memset`
- Memset with temporal stores/`rep stosb`/non-temporal stores. Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Noah Goldstein [Wed, 14 Aug 2024 06:37:30 +0000 (14:37 +0800)]
x86: Use `Avoid_Non_Temporal_Memset` to control non-temporal path
This is just a refactor and there should be no behavioral change from
this commit.
The goal is to make `Avoid_Non_Temporal_Memset` a more universal knob
for controlling whether we use non-temporal memset rather than having
extra logic based on vendor. Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Noah Goldstein [Tue, 13 Aug 2024 15:29:14 +0000 (23:29 +0800)]
x86: Fix bug in strchrnul-evex512 [BZ #32078]
Issue was we were expecting not matches with CHAR before the start of
the string in the page cross case.
The check code in the page cross case:
```
and $0xffffffffffffffc0,%rax
vmovdqa64 (%rax),%zmm17
vpcmpneqb %zmm17,%zmm16,%k1
vptestmb %zmm17,%zmm17,%k0{%k1}
kmovq %k0,%rax
inc %rax
shr %cl,%rax
je L(continue)
```
expects that all characters that neither match null nor CHAR will be
1s in `rax` prior to the `inc`. Then the `inc` will overflow all of
the 1s where no relevant match was found.
This is incorrect in the page-cross case, as the
`vmovdqa64 (%rax),%zmm17` loads from before the start of the input
string.
If there are matches with CHAR before the start of the string, `rax`
won't properly overflow.
The arithmetic shift will clear any matches prior to the start of the
string while maintaining the signbit so the 1s can properly overflow
to zero in the case of no matches. Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Joseph Myers [Wed, 14 Aug 2024 17:15:46 +0000 (17:15 +0000)]
Test errno setting on strtod overflow in tst-strtod-round
We have no tests that errno is set to ERANGE on overflow of
strtod-family functions (we do have some tests for underflow, in
tst-strtod-underflow). Add such tests to tst-strtod-round.
git://sourceware.org / glibc.git / log