This is the mail archive of the
glibc-cvs@sourceware.org
mailing list for the glibc project.
[glibc] Deleted branch azanella/posix_spawn-optimizations
- From: Adhemerval Zanella <azanella at sourceware dot org>
- To: glibc-cvs at sourceware dot org
- Date: 31 Jul 2019 20:32:22 -0000
- Subject: [glibc] Deleted branch azanella/posix_spawn-optimizations
The branch 'azanella/posix_spawn-optimizations' was deleted.
It previously pointed to:
1ebd459... posix: Use posix_spawn for wordexp
Diff:
!!! WARNING: THE FOLLOWING COMMITS ARE NO LONGER ACCESSIBLE (LOST):
-------------------------------------------------------------------
1ebd459... posix: Use posix_spawn for wordexp
d91b617... linux: posix_spawn signal handling optimization
2ad2560... posix: Optimize Linux posix_spawn
432ce57... posix: Add posix_spawn_file_actions_closefrom
339200e... mips: Do not malloc on getdents64 fallback
0841f12... Linux: Add internal direntries interface
commit 1ebd45910cfbb4b3f8d902e7d973810a7ea34bd7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Thu Apr 25 17:54:03 2019 +0000
posix: Use posix_spawn for wordexp
This patch replaces the fork+exec by posix_spawn on wordexp, which
allows a better scability on Linux and simplifies the thread
cancellation handling.
The only change which can not be implemented with posix_spawn the
/dev/null check to certify it is indeed the expected device. I am
not sure how effetive this check really is since /dev/null
tampering means something very wrong with the system and this
is the least of the issues. My view is the tests is really out of
the place and the hardening provided is minimum.
If the idea is still to provide such check, I think a possibilty
would be to open /dev/null, check it, add a dup2 file action, and
close the file descriptor.
Checked on powerpc64le-linux-gnu and x86_64-linux-gnu.
* include/spawn.h (__posix_spawn_file_actions_addopen): New
prototype.
* posix/spawn_faction_addopen.c (posix_spawn_file_actions_addopen):
Add internal alias.
* posix/wordexp.c (create_environment, free_environment): New
functions.
(exec_comm_child, exec_comm): Use posix_spawn instead of fork+exec.
* posix/wordexp-test.c: Use libsupport and remove atfork usage.
commit d91b61793810089850ac365cbe99ed3bd7ea32a1
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Wed May 8 15:13:18 2019 -0300
linux: posix_spawn signal handling optimization
The child helper process on Linux posix_spawn child must ensure that no signal
handler are enabled, so the signal disposition must be either SIG_DFL or
SIG_IGN. However, it requires a sigprocmask to obtain the current signal mask
and at least _NSIG sigaction calls to reset the signal handlers for each
posix_spawn call.
This patch optimizes it by tracking on sigaction when a signal action
different than SIG_DFL or SIG_IGN is used on a per-process variable. It
allows to replace the sigprocmask on posix_spawn with an iteration over the
only non default set signals. It allows to call sigaction only for the
signals that has been actually changed by the process. It assumes the usual
usage as either not use signal handlers at all or use a limited set at
program start, so only new actions are tracked.
Checked on x86_64 and i686.
* include/atomic.h (atomic_fetch_or_seq_cst, atomic_fetch_or_seq_cst):
New macros.
* posix/Makefile (tests): Add tst-spawn6.
* posix/tst-spawn6.c: New file.
* sysdeps/generic/sigsetops.h (__sigorset_atomic): New macro.
* sysdeps/unix/sysv/linux/internal-signals.h (__get_sighandler_set):
New prototype.
* sysdeps/unix/sysv/linux/sigaction.c (__get_sighandler_set): New
function.
(__libc_sigaction): Set the internal handler_set for a new action.
* sysdeps/unix/sysv/linux/sigsetops.h (__sigorset_atomic,
__sigaddset_atomic): New macros.
* sysdeps/unix/sysv/linux/spawni.c (spawni_child): Replace
__sigprocmask with __get_sighandler_set.
commit 2ad25603b31282a83a3dd69af6a8aeca14967382
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Mon May 6 14:33:20 2019 -0300
posix: Optimize Linux posix_spawn
The current internal posix_spawn symbol for Linux (__spawni) requires
to allocate a dynamic stack based on input arguments to handle the
SPAWN_XFLAGS_USE_PATH internal flag, which re-issue the input binary
as a shell script if execve call return ENOEXEC (to execute shell
scripts with an initial shebang).
This is done only for compatibility mode and the generic case does not
require the extra calculation plus the potential large mmap/munmap
call. For default case, a pre-defined buffer is sufficed to use on the
clone call instead.
This patch optimizes Linux spawni by allocating a dynamic stack only
for compatibility symbol (SPAWN_XFLAGS_USE_PATH). For generic case,
a stack allocated buffer is used instead along with a guard page,
similar to what NPTL uses for thread stacks.
Checked x86_64-linux-gnu and i686-linux-gnu.
* sysdeps/unix/sysv/linux/spawni.c (posix_spawn_args): Remove
argc member.
(maybe_script_execute): Remove function.
(execve_compat, __spawni_clone, __spawnix_compat): New function.
(__spawni_child): Remove maybe_script_execute call.
(__spawnix): Remove magic stack slack constant with stack_slack
identifier.
(__spawni): Only allocates a variable stack when
SPAWN_XFLAGS_TRY_SHELL is used.
* posix/stackmap.h: New file.
* sysdeps/ia64/nptl/pthreaddef.h (NEED_SEPARATE_REGISTER_STACK): Move
to ...
* sysdeps/ia64/stackinfo.h: ... here.
commit 432ce57dab1808e753e5569ed256d063c4814618
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Tue Jul 16 09:49:00 2019 -0300
posix: Add posix_spawn_file_actions_closefrom
This patch adds a way to close a range of file descriptors on posix_spawn
as a new file action. The API is similar to the one provided by Solaris
11 [1], where the file action causes the all open file descriptors greater
than or equal to input on to be closed when the new process is spawned.
There are some discussions on BZ#10353 [2], although the bug itself asks
for a generic solution (similar to the closeall provided by some BSD).
The posix_spawn is safe to be implemented by interacting over /proc/self/fd,
the Linux spawni.c does not use CLONE_FILES, so the helper process has its
own file descriptor table and any failure (in /proc operation) aborts the
process creation and returns an error to the caller.
I am aware that this file action might be redundant to the current approach
of POSIX in promoting O_CLOEXEC in more interfaces. However O_CLOEXEC is still
not the default and for some specific usages, the caller needs to close all
possible file descriptors to avoid them leaking. Some examples are CPython
(discussed in BZ#10353) and OpenJDK jspawnhelper [3] (where OpenJDK spawns a
helper process to exactly closes all file descriptors). Most likely any
environment which calls functions that might open file descriptor under the
hood and aim to use posix_spawn might face the same requirement.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu, and
aarch64-linux-gnu.
* posix/Makefile (routines): Add spawn_faction_addclosefrom.
(tests): Add tst-spawn5.
(tst-spawn5-ARGS): New rule.
* posix/Versions [GLIBC_2.30] (libc): Add
posix_spawn_file_actions_addclosefrom_np.
* posix/spawn.h (posix_spawn_file_actions_addclosefrom_np): New
prototype.
* posix/spawn_faction_addclosefrom.c: New file
* posix/spawn_faction_destroy.c (__posix_spawn_file_actions_destroy):
Handle spawn_do_closefrom.
* posix/spawn_int.h (__spawn_action): Add closefrom_action and
spawn_do_closefrom.
* posix/spawn_int_abi.h: New file.
* sysdeps/unix/sysv/linux/spawn_int_abi.h: Likewise.
* posix/tst-spawn5.c: Likewise.
* sysdeps/mach/hurd/spawni.c (__spawni, __spawni_child): Handle
spawn_do_closefrom.
* sysdeps/posix/spawni.c (__spawni_child): Likewise.
* sysdeps/unix/sysv/linux/spawni.c (__spawni_child, __spawnix):
Likewise.
(spawn_closefrom): New function.
* sysdeps/mach/hurd/i386/libc.abilist (2.30): Add
posix_spawn_file_actions_addclosefrom_np.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/csky/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
commit 339200e22e165b4d18bb9ddd42d6020536175542
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Wed Jul 24 13:48:12 2019 -0300
mips: Do not malloc on getdents64 fallback
This patch changes how the fallback getdents64 implementation calls
non-LFS getdents by replacing the scratch_buffer with static buffer
plus a loop on getdents calls. This avoids the potential malloc
call on scratch_buffer_set_array_size for large input buffer size
at the cost of more getdents syscalls.
It also adds a small optimization for older kernels, where the first
ENOSYS failure for getdents64 disable subsequent calls.
Check the dirent tests on a mips64-linux-gnu with getdents64 code
disable.
* sysdeps/unix/sysv/linux/mips/mips64/getdents64.c (__getdents64):
Add small optimization for older kernel to avoid issuing
__NR_getdents64 on each call and replace scratch_buffer usage with
a static allocated buffer.
commit 0841f12653e9af21543805a4c3944f5e17da219a
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Mon Jul 15 17:40:27 2019 -0300
Linux: Add internal direntries interface
Add new internal interfaces direntries_init, direntries_read, and
direntries_next meant to access directories entries similar to dirent
functions. Using these interfaces together with getdents64 has the
following benefits:
* The combination is async-signal-safe, since the buffer is allocated
externally to the call.
* There is no buffer bloat due to auto-sizing the DIR * buffer based on
preferred I/O sizes, as advertised by the file system.
* The combination does not suffer from the telldir problem (which
returns long on 32-bit architectures, which is incompatible with
struct dirent64 in the kernel). (Only seeking to the beginning is
supported.)
* Exposing the block read size means that the application knows when it
has to rewind after deleting enumerated files, to avoid skipping
entries or returning entries twice.
* sysdeps/unix/sysv/linux/Makefile (sysdep_routines): Add direntries.
* sysdeps/unix/sysv/linux/direntries.c: New file.
* sysdeps/unix/sysv/linux/direntries.h. Likewise.