Takashi Yano [Mon, 27 Jun 2022 01:03:04 +0000 (10:03 +0900)]
Cygwin: poll: Fix a bug on inquiring same fd with different events.
- poll() has a bug that it returns event which is not inquired if
events are inquired in multiple pollfd entries on the same fd at
the same time. This patch fixes the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251732.html
Stefan Eßer [Thu, 30 Dec 2021 11:20:32 +0000 (12:20 +0100)]
Make CPU_SET macros compliant with other implementations
The introduction of <sched.h> improved compatibility with some 3rd
party software, but caused the configure scripts of some ports to
assume that they were run in a GLIBC compatible environment.
Parts of sched.h were made conditional on -D_WITH_CPU_SET_T being
added to ports, but there still were compatibility issues due to
invalid assumptions made in autoconfigure scripts.
The differences between the FreeBSD version of macros like CPU_AND,
CPU_OR, etc. and the GLIBC versions was in the number of arguments:
FreeBSD used a 2-address scheme (one source argument is also used as
the destination of the operation), while GLIBC uses a 3-adderess
scheme (2 source operands and a separately passed destination).
The GLIBC scheme provides a super-set of the functionality of the
FreeBSD macros, since it does not prevent passing the same variable
as source and destination arguments. In code that wanted to preserve
both source arguments, the FreeBSD macros required a temporary copy of
one of the source arguments.
This patch set allows to unconditionally provide functions and macros
expected by 3rd party software written for GLIBC based systems, but
breaks builds of externally maintained sources that use any of the
following macros: CPU_AND, CPU_ANDNOT, CPU_OR, CPU_XOR.
One contributed driver (contrib/ofed/libmlx5) has been patched to
support both the old and the new CPU_OR signatures. If this commit
is merged to -STABLE, the version test will have to be extended to
cover more ranges.
Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do
no longer require that option.
The FreeBSD version has been bumped to 1400046 to reflect this
incompatible change.
Stefan Eßer [Wed, 8 Dec 2021 07:47:42 +0000 (08:47 +0100)]
sys/_bitset.h: Fix fall-out from commit 5e04571cf3c
The changes to the bitset macros allowed sched.h to be included
into userland programs without name space pollution due to BIT_*
and BITSET_* macros.
The definition of a "struct bitset" had been overlooked. This name
space pollution caused the build of port print/miktex to fail.
This commit makes the definition of struct bitset depend on the
same condition as the visibility of the BIT_* and BITSET_* macros,
i.e. needs _KERNEL or _WANT_FREEBSD_BITSET to be defined before
including sys/_bitset.h.
It has been tested with "make universe" since a prior attempt to
fix the issue broke the PowerPC64 kernel build.
This commit shall be MFCed together with commit 5e04571cf3c.
Stefan Eßer [Tue, 7 Dec 2021 19:29:26 +0000 (20:29 +0100)]
sys/_bitset.h: Fix fall-out from commit 5e04571cf3c
There is a reference to malloc() in #define __BITSET_ALLOC. Even
though this macro is only defined but not used, it causes the lang/gcc
ports to fail. The gcc ports "poison" a number of functions including
malloc() and prevent their use (including in macro definitions).
This commit moved the declaration of __BITSET_ALLOC into the
conditional block that depends on _KERNEL or _WANT_FREEBSD_BITSET
being defined.
There is no use of __BITSET_ALLOC in the FreeBSD sources, and userland
programs that want to use BITSEC_ALLOC will define _WANT_FREEBSD_BITSET
anyway.
This patch has been tested by building lang/gcc11 and a successful
make buildworld.
This commit shall be MFCed together with commit 5e04571cf3c.
Stefan Eßer [Sun, 5 Dec 2021 21:27:33 +0000 (22:27 +0100)]
sys/bitset.h: reduce visibility of BIT_* macros
Add two underscore characters "__" to names of BIT_* and BITSET_*
macros to move them to the implementation name space and to prevent
a name space pollution due to BIT_* macros in 3rd party programs with
conflicting parameter signatures.
These prefixed macro names are used in kernel header files to define
macros in e.g. sched.h, sys/cpuset.h and sys/domainset.h.
If C programs are built with either -D_KERNEL (automatically passed
when building a kernel or kernel modules) or -D_WANT_FREENBSD_BITSET
(or this macros is defined in the source code before including the
bitset macros), then all macros are made visible with their previous
names, too. E.g., both __BIT_SET() and BIT_SET() are visible with
either of _KERNEL or _WANT_FREEBSD_BITSET defined.
The main reason for this change is that some 3rd party sources
including sched.h have been found to contain conflicting BIT_*
macros.
As a work-around, parts of shed.h have been made conditional and
depend on _WITH_CPU_SET_T being set when sched.h is included.
Ports that expect the full functionality provided by sched.h need
to be built with -D_WITH_CPU_SET_T. But this leads to conflicts if
BIT_* macros are defined in that program, too.
This patch set makes all of sched.h visible again without this
parameter being passed and without any name space pollution due
to BIT_* macros becoming visible when sched.h is included.
This patch set will be backported to the STABLE branches, but ports
will need to use -D_WITH_CPU_SET_T as long as there are supported
releases that do not contain these patches.
Mark Johnston [Tue, 21 Sep 2021 15:32:23 +0000 (11:32 -0400)]
bitset(9): Introduce BIT_FOREACH_ISSET and BIT_FOREACH_ISCLR
These allow one to non-destructively iterate over the set or clear bits
in a bitset. The motivation is that we have several code fragments
which iterate over a CPU set like this:
This is slow since CPU_FFS begins the search at the beginning of the
bitset each time. On amd64 and arm64, CPU sets have size 256, so there
are four limbs in the bitset and we do a lot of unnecessary scanning.
A second problem is that this is destructive, so code which needs to
preserve the original set has to make a copy. In particular, we have
quite a few functions which take a cpuset_t parameter by value, meaning
that each call has to copy the 32 byte cpuset_t.
The new macros address both problems.
Reviewed by: cem, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32028
Patrick Kelsey [Mon, 26 Apr 2021 04:25:59 +0000 (00:25 -0400)]
iflib: Improve mapping of TX/RX queues to CPUs
iflib now supports mapping each (TX,RX) queue pair to the same CPU
(default), to separate CPUs, or to a pair of physical and logical CPUs
that share the same L2 cache. The mapping mechanism supports unequal
numbers of TX and RX queues, with the excess queues always being
mapped to consecutive physical CPUs. When the platform cannot
distinguish between physical and logical CPUs, all are treated as
physical CPUs. See the comment on get_cpuid_for_queue() for the
entire matrix.
The following device-specific tunables influence the mapping process:
dev.<device>.<unit>.iflib.core_offset (existing)
dev.<device>.<unit>.iflib.separate_txrx (existing)
dev.<device>.<unit>.iflib.use_logical_cores (new)
The following new, read-only sysctls provide visibility of the mapping
results:
dev.<device>.<unit>.iflib.{t,r}xq<n>.cpu
When an iflib driver allocates TX softirqs without providing reference
RX IRQs, iflib now binds those TX softirqs to CPUs using the above
mapping mechanism (that is, treats them as if they were TX IRQs).
Previously, such bindings were left up to the grouptaskqueue code and
thus fell outside of the iflib CPU mapping strategy.
D Scott Phillips [Mon, 21 Sep 2020 22:19:12 +0000 (22:19 +0000)]
bitset: expand bit index type to `long`
An upcoming patch to use the bitset macros for tracking vm page
dump information could conceivably need more than INT_MAX bits.
Expand the bit type to long so that the extra range is available
on 64-bit platforms where it would most likely be needed.
CPUSET_COUNT and DOMAINSET_COUNT are also modified to remain of
type `int`.
Ryan Libby [Fri, 13 Dec 2019 09:32:16 +0000 (09:32 +0000)]
bitset: rename confusing macro NAND to ANDNOT
s/BIT_NAND/BIT_ANDNOT/, and for CPU and DOMAINSET too. The actual
implementation is "and not" (or "but not"), i.e. A but not B.
Fortunately this does appear to be what all existing callers want.
Don't supply a NAND (not (A and B)) operation at this time.
Discussed with: jeff
Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22791
Ryan Libby [Tue, 3 Dec 2019 17:43:57 +0000 (17:43 +0000)]
bitset: avoid pessimized code when bitset size is not constant
We have a couple optimizations for when the bitset is known to be just
one word. But with dynamically sized bitsets, it was actually more work
to determine the size than just to do the necessary computation. Now,
only use the optimization when the size is known to be constant.
Takashi Yano [Sun, 19 Jun 2022 03:53:16 +0000 (12:53 +0900)]
Cygwin: console: Handle setting very long window title correctly.
- Previously, the console code could not handle escape sequence
setting window title longer than 256 byte correctly. This patch
fixes the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251662.html
- Currently, ENABLE_VIRTUAL_TERMINAL_PROCESSING flag is disabled
unconditionally when exiting from cygwin. This causes ANSI escape
sequence disabled in Windows Terminal where it is enables by
default. This patch retains that flag if it is originally enabled.
Ken Brown [Thu, 9 Jun 2022 22:42:03 +0000 (18:42 -0400)]
Cygwin: restore '#ifdef __x86_64__' for CPU-specific code
Commit e1ce752a1d, "Cygwin: remove miscellaneous 32-bit code", removed
most occurrences of '#ifdef __x86_64__'. Restore those occurrences
that guarded code specific to the AMD64 processor, and #error out if
the processor is different. This will make it easier to find
AMD64-specific code if we ever want to add support for a different
64-bit processor (e.g., ARM64).
Sebastian Huber [Tue, 17 May 2022 16:28:52 +0000 (18:28 +0200)]
Use global stdio streams for all configurations
The _REENT_GLOBAL_STDIO_STREAMS was introduced by commit 668a4c8722090fffd10869dbb15b879651c1370d in 2017. Since then it was enabled by
default for RTEMS. Recently, the option was enabled for Cygwin which
previously used an alternative implementation to use global stdio streams.
In Newlib, the stdio streams are defined to thread-specific pointers
_reent::_stdin, _reent::_stdout and _reent::_stderr. If the option is disabled
(the default for most systems), then these pointers are initialized to
thread-specific FILE objects which use file descriptors 0, 1, and 2,
respectively. There are at least three problems with this:
(1) The thread-specific FILE objects are closed by _reclaim_reent(). This
leads to problems with language run-time libraries that provide wrappers to
the C/POSIX stdio streams (for example C++ and Ada), since they use the
thread-specific FILE objects of the initialization thread. In case the
initialization thread is deleted, then they use freed memory.
(2) Since thread-specific FILE objects are used with a common output device via
file descriptors 0, 1 and 2, the locking at FILE object level cannot ensure
atomicity of the output, e.g. a call to printf().
This patch enables the _REENT_GLOBAL_STDIO_STREAMS behaviour for all Newlib
configurations and removes the option. This removes a couple of #ifdef blocks.
Sebastian Huber [Wed, 18 May 2022 11:26:13 +0000 (13:26 +0200)]
Fix __fp_lock_all() and __fp_unlock_all()
For _REENT_GLOBAL_STDIO_STREAMS, lock/unlock all FILE objects. In the
repository, this function is only used by Cygwin during process forks. Since
Cygwin enabled _REENT_GLOBAL_STDIO_STREAMS recently, without this fix no FILE
object at all was locked.
Mark Geisert [Thu, 9 Jun 2022 04:47:31 +0000 (21:47 -0700)]
Cygwin: Have gmondump support ssp-generated gmon.out
Cygwin tool ssp generates gmon.out files with different address
resolution than other tools do. Two address bytes per bucket rather than
the usual four address bytes. Gprof can deal with the difference but
gmondump can't because the latter's gmon.out header validation fails.
- Remove the offending portion of the header validation code.
- Make sure all code can handle differing address resolutions.
- Display address resolution in verbose data dumps.
- Change "rawarc" to "struct rawarc" in certain sizeof expressions to
avoid buffer overrun faults.
- When "-v" (verbose) is specified, note when there is missing bucket
data or rawarc data.
Sebastian Huber [Tue, 7 Jun 2022 05:55:02 +0000 (07:55 +0200)]
Fix __sglue inititialization
Do not initialize __sglue with the FILE objects of _GLOBAL_REENT to avoid a
double use in the !_REENT_SMALL and !_REENT_GLOBAL_STDIO_STREAMS configurations
which didn't use a thread-specific reentrancy structure.
Ken Brown [Tue, 7 Jun 2022 17:46:31 +0000 (13:46 -0400)]
Cygwin: restore two instances of __stdcall
In the previous commit, __stdcall was removed from _dll_crt0 in
winsup.h and dcrt0.cc but not in lib/cygwin_crt0.c. For consistency,
restore the first two occurrences of __stdcall. We could instead
remove it from the declaration in lib/cygwin_crt0.c, but this might
appear to affect binary compatibility, even though it really doesn't.
Ken Brown [Mon, 6 Jun 2022 16:00:45 +0000 (12:00 -0400)]
Cygwin: remove most occurrences of __stdcall and __cdecl
These have no effect on x86_64. Retain a few occurrences of __cdecl
in files imported from other sources.
Also retain all occurrences of WINAPI, even though the latter is
simply a macro that expands to __stdcall. Most of these occurrences
are associated with Windows API functions, and removing them might
make the code confusing instead of simpler.
Jon Turney [Fri, 30 Apr 2021 13:33:21 +0000 (14:33 +0100)]
Cygwin: Drop use of loadlib.h in regtool
Link directly with RegDeleteKeyExW(), available since Vista.
(It's unclear the LoadLibrary wrapper was ever doing anything useful
here, as (i) DLL lookup in PATH was avoided as advapi32 is already
loaded into the process, and (ii) advapi32 is a 'known DLL' which is
only ever loaded from system directory)
[For most of these, the corresponding 64-bit entry points are obtained
by exporting aliases. For example, acl is an alias for acl32, and
truncate is an alias for truncate64.]
Remove the following structs and all code using them (which is 32-bit
only): __stat32, __group16, __flock32, __aclent16_t.
Remove the typedefs of __blkcnt32_t __dev16_t, __ino32_t, which are
used only in code that has been removed.
Put the typedefs of __uid16_t and __gid16_t in one header, instead of
one header if __INSIDE_CYGWIN__ is defined and a different header
otherwise.
Takashi Yano [Fri, 27 May 2022 12:54:36 +0000 (21:54 +0900)]
Cygwin: cygheap: Fix the issue of cygwin1.dll in the root directory.
- After the commit 6d898f43, cygwin fails to start if cygwin1.dll
is placed in the root directory. This patch fixes the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-May/251548.html
Ken Brown [Sun, 22 May 2022 15:43:44 +0000 (11:43 -0400)]
Cygwin: fix mknod (64-bit only)
The current definition of mknod in syscalls.cc has a third argument of
type __dev16_t instead of dev_t. Fix this on 64-bit Cygwin by making
the existing mknod 32-bit only and then exporting mknod as an alias
for mknod32. (No fix is needed on 32-bit because mknod is redirected
to mknod32 via NEW_FUNCTIONS in Makefile.am.)
Sebastian Huber [Tue, 17 May 2022 14:39:34 +0000 (16:39 +0200)]
Fix __sFILE::_lock initialization
The __sFILE::_lock member is present if __SINGLE_THREAD__ is not defined. In
this case, it is initialized in __sfp(). It is a bug to do it sometimes also
in std().
Corinna Vinschen [Thu, 19 May 2022 08:46:33 +0000 (10:46 +0200)]
Cygwin: make sure exec'ed process exists early in process list
killpg(pgid, 0) (or kill_pgrp(pgid, si_signo=0), in signal.cc)
fails (returns -1) even when there is a process in the process
group pgid, if the process is in the middle of spawnve(), see
When exec'ing a process the assumption is that the exec'ed process creates its
own symlink (in pinfo::thisproc() in pinfo.cc). If the exec'ing process
calls NtClose on it's own winpid symlink, but the exec'ed process didn't
progress enough into initialization, there's a slim chance that neither
the exec'ing process, nor the exec'ed process has a winpid symlink
attached.
Always create the winpid symlink in spawn.cc, even for exec'ed Cygwin
processes. Make sure to dup the handle into the new process, and stop
creating the winpid symlink in exec'ed processes.
Takashi Yano [Wed, 18 May 2022 17:51:46 +0000 (02:51 +0900)]
Cygwin: Use two pass parse for tlsoffsets generation.
- The commit "Cygwin: fix new sigfe.o generation in optimized case"
fixed the wrong tlsoffsets generation by adding -O0 to compile
options. Current gentls_offsets expects entry of "start_offset"
is the first entry in the assembler code. However, without -O0,
entry of "start_offset" goes to the last entry for some reason.
Currently, -O0 can prevents assembler code from reversing the
order of the entries, however, there is no guarantee that it will
retain the order of the entries in the future.
This patch makes gentls_offsets parse the assembler code in the
two pass to omit -O0 option dependency.
Sebastian Huber [Fri, 13 May 2022 11:44:13 +0000 (13:44 +0200)]
Use global atexit data for all configurations
For the exit processing only members of _GLOBAL_REENT were used by default. If
the _REENT_GLOBAL_ATEXIT option was enabled, then the data structures were
provided through dedicated global objects. Make this option the default.
Remove the option. Rename struct _reent members _atexit and _atexit0 to
_reserved_6 and _reserved_7, respectively. Provide them only if
_REENT_BACKWARD_BINARY_COMPAT is defined.
Sebastian Huber [Fri, 13 May 2022 12:56:24 +0000 (14:56 +0200)]
Optional struct _reent::_new::_unused
Rename struct _reent::_new::_unused members _nextf and _nmalloc to _reserved_3
and _reserved_4, respectively. Rename struct _reent::_new member _unused to
_reserved_5. Provide them only if _REENT_BACKWARD_BINARY_COMPAT is defined.
Sebastian Huber [Fri, 13 May 2022 11:55:06 +0000 (13:55 +0200)]
Add --enable-newlib-reent-binary-compat
Add the --enable-newlib-reent-binary-compat configure option. This option is
disabled by default. If enabled, then unused members in struct _reent are
preserved to maintain the structure layout.
Takashi Yano [Sat, 14 May 2022 11:39:42 +0000 (20:39 +0900)]
Cygwin: pty: Fix script command crash on console in Win7.
- Previously, the command "cmd /c script -c cmd" in console of Win7
crashes. This seems to be due to a bug (?) of AttachConsole().
This patch adds workaround for this issue.
Currently, pty reattaches to the console of the process which is
predetermined by ConsoleProcessList() after temporarily attaching
to another console. After that, the console output handle opened
with the name "CONOUT$" may not be accessible in Win7.
This seems to happen when the attached process does not have the
same handle even if the console attached is the same. With this
patch, cygwin-console-helper which is started when pty master is
opened in console, is utilized to be a target process to which
pty reattaches if the OS is Win7.
Corinna Vinschen [Fri, 13 May 2022 19:22:44 +0000 (21:22 +0200)]
Cygwin: fix new sigfe.o generation in optimized case
Commit 0597c84b9bdb ("Cygwin: revamp TLS offsets computation")
introduced a really weird problem when building Cygwin with
optimization.
First of all, the tlsoffsets file is broken with -O2. This
can easily be fixed by running the compiler with -O0 when called
from the gentls_offsets script.
But it gets worse:
When creating sigfe.o with optimization, the generated machine code
uses incorrect offsets: For some reason the assembler codes using
_cygtls.stackptr as offset value are assembled into machine code
using _cygtls.pstackptr as offsets.
And as if that isn't already absurd enough, renaming _cygtls.pstackptr
to, say, _cygtls.blurb, fixes the assembled machine code expressions;
they use the value of _cygtls.stackptr again.
So I changed gentls_offsets and gendef to use _cygtls.foo_p rather
than _cygtls.pfoo and that fixes the assembled code in the optimized
case.
No, I can't explain that. There's no system in that behaviour.
It looks absolutely crazy.
- convert gentls_offsets to a shell script, only running the target
compiler and gawk.
- Simplify cygtls.h. The new gentls_offsets script only requires two
lines with the "public:" keyword as markers. The comments are not
used anymore, the output is a preprocesses file without comments.
Align Makefile rules accordingly.
- Rather than generating perl variables and C #defines, just generate
.ecu statements and .include the TLS offsets file right from the
generated assembler file sigfe.s. It's the only place we really
need (some of) the offsets.
- Drop the target-specific name of the TLS offsets file and generate
it on the fly in the build dir. Fix configure and Makefile rules
accordingly.
Corinna Vinschen [Thu, 27 Jan 2022 11:41:55 +0000 (12:41 +0100)]
Cygwin: configure: Define default valus for target specific variables
Define default values for DLL_ENTRY, DIN_FILE, and TLSOFFSETS_H
and drop them from the x86_64-specific branch. Keep the mechanism
intact to allow other target CPUs if there ever will be.
Corinna Vinschen [Thu, 27 Jan 2022 11:39:57 +0000 (12:39 +0100)]
Cygwin: config.h: stop including auto-generated tlsoffsets.h file
This was a hack to begin with. Clean this mess up:
- Move definition of CYGTLS_PADSIZE to cygwin/config.h and drop
local cygtls_padsize.h
- Rename CYGTLS_PADSIZE to __CYGTLS_PADSIZE__ to keep namespace
clean. Redefine as macro, rather than as const.
- Move struct _reent first in struct _cygtls to allow using
__CYGTLS_PADSIZE__ as offset in __getreent().
Matt Joyce [Mon, 2 May 2022 06:58:25 +0000 (08:58 +0200)]
Add global __sglue object for all configurations
Added a new global __sglue object for all configurations.
Decouples the global file object list from the _GLOBAL_REENT
structure by using this new object instead of the __sglue member
of _GLOBAL_REENT in __sfp() and _fwalk_sglue().
Sebastian Huber [Fri, 1 Apr 2022 11:32:32 +0000 (13:32 +0200)]
stdio: Replace _fwalk_reent() with _fwalk_sglue()
Replaced _fwalk_reent() with _fwalk_sglue(). The change adds an
extra __sglue object as a parameter, which will allow the passing
of a global __sglue object separate from the __sglue member of
struct _reent. The global __sglue object will be added in a
follow-on patch.
Matt Joyce [Tue, 3 May 2022 04:29:36 +0000 (06:29 +0200)]
Add CLEANUP_FILE define
Define the configuration-dependent constant CLEANUP_FILE for use in
cleanup_stdio(). This will reduce duplicate code during the addition
of a dedicated stdio atexit handler in a follow-on patch.
Matt Joyce [Tue, 3 May 2022 04:17:15 +0000 (06:17 +0200)]
Move __sglue initializations to __sfp()
Moved last remaining __sglue initializations from __sinit() to
__sfp(). The move better encapsulates access to __sglue and
facilitates its decoupling from struct _reent in a follow-on patch.