Ken Brown [Wed, 3 Aug 2022 20:45:23 +0000 (16:45 -0400)]
Cygwin: syscalls.cc: remove ".dll" from blessed_executable_suffixes
This reverts commit d9e9c7b5a7. The latter added ".dll" to the
blessed_executable_suffixes array because on 32-bit Windows, the
GetBinaryType function would report that a 64-bit DLL is an
executable, contrary to the documentation of that function.
That anomaly does not exist on 64-bit Windows, so we can remove ".dll"
from the list. Reverting the commit does, however, change the
behavior of the rename(2) syscall in the following unlikely situation:
Suppose we have an executable foo.exe and we make the call
rename ("foo", "bar.dll");
Previously, foo.exe would be renamed to bar.dll. So bar.dll would
then be an executable without the .exe extension. The new behavior is
that foo.exe will be renamed to bar.dll.exe. [Exception: If there
already existed an executable (not a DLL!) with the name bar.dll, then
.exe will not be appended.]
Cygwin: syscalls.cc: drop masking macros for standard IO functions
The actual reason for these wrappers are lost in time, there's no
hint even in the pre-2000 ChangeLog files. Apparently they were
masking the prototypes or, alternatively, macros from newlib to
clash with the definitions in syscalls.cc.
They are not needed anymore, so just drop them.
This uncovered that the buffer pointer to pwrite is erronously
non-const. Fix this on the way out.
To avoid linear searches for error codes, autogenerate errmap as
simple array of errno values indexed by Windows error codes.
Restrict to the first 9000 Windows error codes, we don't care for
most of them anyway.
Define errmap in its own file errmap.h to clean up errno.cc.
Cygwin: drop requirement to build newlib's stdio64
Given that 64 bit Cygwin defines all file access types (off_t,
fpos_t, and derived types) as 64 bit anyway, there's no reason
left to rely on the stdio64 part of newlib. Use base functions
and base types.
Takashi Yano [Mon, 1 Aug 2022 00:02:23 +0000 (09:02 +0900)]
Cygwin: path: Make some symlinks to /cygdrive/* work.
- Previously, some symbolic links to /cygdrive/* (e.g. /cygdrive/C,
/cygdrive/./c, /cygdrive//c, etc.) did not work. This patch fixes
the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-July/251994.html
- Previously, LoadLibraryA() is hooked for ConEmu cygwin connector.
With this patch, GetProcAddress() for "RequestTermConnector" is
hooked instead which is more essential for ConEmu cygwin connector.
Cygwin: console: Avoid accessing NULL pointer via cygheap->ctty.
- Recent commit "Cygwin: console: Add missing input_mutex guard."
has a problem that causes NULL pointer access if cygheap->ctty
is NULL. This patch fixes the issue.
Jon Turney [Thu, 19 May 2022 16:27:39 +0000 (17:27 +0100)]
Cygwin: Set threadnames with SetThreadDescription()
gdb master recently learnt how to use GetThreadDescription() [1], so set
threadnames using SetThreadDescription() [available since Windows 10
1607] as well.
This is superior to using a special exception to indicate the thread
name to the debugger, because the thread name isn't missed if you don't
have a debugger attached at the time it's set.
It's not clear what the encoding of a thread name string is, we assume
UTF8 for the moment.
For the moment, continue to use the old method as well, for the benefit
of older gdb versions etc.
Cygwin: rename __cygwin_environ and drop env redirection via cur_environ()
Back in early Cygwin development a function based access to the
environment was exported, the internal environ in Cygwin was called
__cygwin_environ and cur_environ() was used to access the environment
indirectly . The history of that necessity is not documented,
but kept in i686 for backward compatibility.
The x86_64 port eventually used __cygwin_environ directly and exported
it as DATA under the usual name environ.
We don't need the i686 workaround anymore, so just rename
__cygwin_environ to environ, drop the cur_environ() macro and
simply export environ under its own name.
- rename "is64bit" to "is_x86_64"
- Always set sym_prefix to empty string and add a FIXME
- speclib: rename uscore to sym_prefix and fix bug in string handling
Cygwin: console: Add workaround for ConEmu cygwin connector.
- ConEmu cygwin connector conflicts with cons_master_thread since
it does not use read() to read console input. With this patch,
cons_master_thread is disabled in ConEmu cygwin connector.
Matt Joyce [Mon, 16 May 2022 09:51:54 +0000 (11:51 +0200)]
Add --enable-newlib-reent-thread-local option
By default, Newlib uses a huge object of type struct _reent to store
thread-specific data. This object is returned by __getreent() if the
__DYNAMIC_REENT__ Newlib configuration option is defined.
The reentrancy structure contains for example errno and the standard input,
output, and error file streams. This means that if an application only uses
errno it has a dependency on the file stream support even if it does not use
it. This is an issue for lower end targets and applications which need to
qualify the software according to safety standards (for example ECSS-E-ST-40C,
ECSS-Q-ST-80C, IEC 61508, ISO 26262, DO-178, DO-330, DO-333).
If the new _REENT_THREAD_LOCAL configuration option is enabled, then struct
_reent is replaced by dedicated thread-local objects for each struct _reent
member. The thread-local objects are defined in translation units which use
the corresponding object.
Matt Joyce [Mon, 23 May 2022 09:29:57 +0000 (11:29 +0200)]
Add _REENT_IS_NULL()
In a follow up patch, struct _reent is optionally replaced by dedicated
thread-local objects. In this case,_REENT is optionally defined to NULL. Add
the _REENT_IS_NULL() macro to disable this check on demand.
Matt Joyce [Fri, 4 Feb 2022 10:47:18 +0000 (11:47 +0100)]
Add _REENT_SIG_FUNC(ptr)
Add a _REENT_SIG_FUNC() macro to encapsulate access to the
_sig_func member of struct reent. This will help to replace the
struct member with a thread-local storage object in a follow up
patch.
Matt Joyce [Thu, 3 Feb 2022 11:24:26 +0000 (12:24 +0100)]
Add _REENT_CVTBUF(ptr)
Add a _REENT_CVTBUF() macro to encapsulate access to the _cvtbuf
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow up patch.
Matt Joyce [Thu, 3 Feb 2022 10:19:14 +0000 (11:19 +0100)]
Add _REENT_CVTLEN(ptr)
Add a _REENT_CVTLEN() macro to encapsulate access to the _cvtlen
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow-up patch.
Matt Joyce [Thu, 3 Feb 2022 09:18:53 +0000 (10:18 +0100)]
Add _REENT_CLEANUP(ptr)
Add a _REENT_CLEANUP() macro to encapsulate access to the
__cleanup member of struct reent. This will help to replace the
struct member with a thread-local storage object in a follow up
patch.
Matt Joyce [Wed, 2 Feb 2022 11:47:30 +0000 (12:47 +0100)]
Add _REENT_LOCALE(ptr)
Add a _REENT_LOCALE() macro to encapsulate access to the _locale
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow up patch.
Matt Joyce [Wed, 2 Feb 2022 08:49:00 +0000 (09:49 +0100)]
Add _REENT_INC(ptr)
Add a _REENT_INC() macro to encapsulate access to the _inc member
of struct reent. This will help to replace the struct member with
a thread-local storage object in a follow up patch.
Matt Joyce [Tue, 1 Feb 2022 12:05:53 +0000 (13:05 +0100)]
Add _REENT_STDERR(ptr)
Add a _REENT_STDERR() macro to encapsulate access to the _stderr
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow up patch.
Matt Joyce [Tue, 1 Feb 2022 11:37:46 +0000 (12:37 +0100)]
Add _REENT_STDOUT(ptr)
Add a _REENT_STDOUT() macro to encapsulate access to the _stdout
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow up patch.
Matt Joyce [Fri, 28 Jan 2022 09:58:36 +0000 (10:58 +0100)]
Add _REENT_STDIN(ptr)
Add a _REENT_STDIN() macro to encapsulate access to the _stdin
member of struct reent. This will help to replace the struct
member with a thread-local storage object in a follow up patch.
Matt Joyce [Tue, 18 Jan 2022 09:13:04 +0000 (10:13 +0100)]
Add _REENT_ERRNO(ptr)
Add a _REENT_ERRNO() macro to encapsulate the access to the
_errno member of struct reent. This will help to replace the
structure member with a thread-local storage object in a follow
up patch.
Replace uses of __errno_r() with _REENT_ERRNO(). Keep __errno_r() macro for
potential users outside of Newlib.
Matt Joyce [Mon, 16 May 2022 08:54:31 +0000 (10:54 +0200)]
Define _REENT_EMERGENCY(ptr) only once
Use this macro to access the _emergency member of struct _reent. This macro
will help to replace the _emergency member of struct _reent with a thread-local
storage object in a follow up patch.
Cygwin: clipboard: Add workaround for setting clipboard failure.
- OpenClipboard() just after CloseClipboard() sometimes fails. Due
to this, /dev/clipboard sometimes fails to set CF_UNICODETEXT
data. This patch add a workaround for this issue.
Gleb Smirnoff [Fri, 24 Jun 2022 16:09:11 +0000 (09:09 -0700)]
libc/syslog: fully deprecate and don't try to open "/dev/log"
The "/dev/log" socket existed in pre-FreeBSD times. Later it was
substituted to a compatibility symlink. The symlink creation was
deprecated in FreeBSD 10.2 and 9-STABLE.
Provide sticky ARP flag for network interface which marks it as the
"sticky" one similarly to what we have for bridges. Once interface is
marked sticky, any address resolved using the ARP will be saved as a
static one in the ARP table. Such functionality may be used to prevent
ARP spoofing or to decrease latencies in Ethernet networks.
The drawbacks include potential limitations in usage of ARP-based
load-balancers and high-availability solutions such as carp(4).
The implemented option is disabled by default, therefore should not
impact the default behaviour of the networking stack.
Alan Somers [Thu, 5 May 2022 21:35:23 +0000 (15:35 -0600)]
Correctly measure system load averages > 1024
The old fixed-point arithmetic used for calculating load averages had an
overflow at 1024. So on systems with extremely high load, the observed
load average would actually fall back to 0 and shoot up again, creating
a kind of sawtooth graph.
Fix this by using 64-bit math internally, while still reporting the load
average to userspace as a 32-bit number.
tcp: LRO code to deal with all 12 TCP header flags
TCP per RFC793 has 4 reserved flag bits for future use. One
of those bits may be used for Accurate ECN.
This patch is to include these bits in the LRO code to ease
the extensibility if/when these bits are used.
Mike Karels [Wed, 27 Oct 2021 03:01:09 +0000 (22:01 -0500)]
kernel: deprecate Internet Class A/B/C
Hide historical Class A/B/C macros unless IN_HISTORICAL_NETS is defined;
define it for user level. Define IN_MULTICAST separately from IN_CLASSD,
and use it in pf instead of IN_CLASSD. Stop using class for setting
default masks when not specified; instead, define new default mask
(24 bits). Warn when an Internet address is set without a mask.
Peter Lei [Tue, 26 Oct 2021 03:08:54 +0000 (20:08 -0700)]
tcp: socket option to get stack alias name
TCP stack sysctl nodes are currently inserted using the stack
name alias. Allow the user to get the current stack's alias to
allow for programatic sysctl access.
Randall Stewart [Fri, 22 Oct 2021 11:10:28 +0000 (07:10 -0400)]
tcp: Add hystart-plus to cc_newreno and rack.
TCP Hystart draft version -03:
https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-hystartplusplus
Is a new version of hystart that allows one to carefully exit slow start if the RTT
spikes too much. The newer version has a slower-slow-start so to speak that then
kicks in for five round trips. To see if you exited too early, if not into congestion avoidance.
This commit will add that feature to our newreno CC and add the needed bits in rack to
be able to enable it.
Reviewed by: tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D32373
Randall Stewart [Tue, 17 Aug 2021 20:29:22 +0000 (16:29 -0400)]
tcp: Add support for DSACK based reordering window to rack.
The rack stack, with respect to the rack bits in it, was originally built based
on an early I-D of rack. In fact at that time the TLP bits were in a separate
I-D. The dynamic reordering window based on DSACK events was not present
in rack at that time. It is now part of the RFC and we need to update our stack
to include these features. However we want to have a way to control the feature
so that we can, if the admin decides, make it stay the same way system wide as
well as via socket option. The new sysctl and socket option has the following
meaning for setting:
00 (0) - Keep the old way, i.e. reordering window is 1 and do not use DSACK bytes to add to reorder window
01 (1) - Change the Reordering window to 1/4 of an RTT but do not use DSACK bytes to add to reorder window
10 (2) - Keep the reordering window as 1, but do use SACK bytes to add additional 1/4 RTT delay to the reorder window
11 (3) - reordering window is 1/4 of an RTT and add additional DSACK bytes to increase the reordering window (RFC behavior)
The default currently in the sysctl is 3 so we get standards based behavior.
Reviewed by: tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D31506
Andrew Gallatin [Thu, 5 Aug 2021 21:16:30 +0000 (17:16 -0400)]
tsleep: Add a PNOLOCK flag
Add a PNOLOCK flag so that, in the race circumstance where
wakeup races are externally mitigated, tsleep() can be
called with a sleep time of 0 without triggering an
an assertion.
Roy Marples [Wed, 28 Jul 2021 15:46:59 +0000 (08:46 -0700)]
socket: Implement SO_RERROR
SO_RERROR indicates that receive buffer overflows should be handled as
errors. Historically receive buffer overflows have been ignored and
programs could not tell if they missed messages or messages had been
truncated because of overflows. Since programs historically do not
expect to get receive overflow errors, this behavior is not the
default.
This is really really important for programs that use route(4) to keep
in sync with the system. If we loose a message then we need to reload
the full system state, otherwise the behaviour from that point is
undefined and can lead to chasing bogus bug reports.
Kristof Provost [Thu, 20 May 2021 09:54:41 +0000 (11:54 +0200)]
pf: syncookie support
Import OpenBSD's syncookie support for pf. This feature help pf resist
TCP SYN floods by only creating states once the remote host completes
the TCP handshake rather than when the initial SYN packet is received.
This is accomplished by using the initial sequence numbers to encode a
cookie (hence the name) in the SYN+ACK response and verifying this on
receipt of the client ACK.
Randall Stewart [Wed, 26 May 2021 10:43:30 +0000 (06:43 -0400)]
tcp: Add a socket option to rack
so we can test various changes to the slop value in timers.
Timer_slop, in TCP, has been 200ms for a long time. This value dates back
a long time when delayed ack timers were longer and links were slower. A
200ms timer slop allows 1 MSS to be sent over a 60kbps link. Its possible that
lowering this value to something more in line with todays delayed ack values (40ms)
might improve TCP. This bit of code makes it so rack can, via a socket option,
adjust the timer slop.
Randall Stewart [Thu, 6 May 2021 15:22:26 +0000 (11:22 -0400)]
This brings into sync FreeBSD with the netflix
versions of rack and bbr. This fixes several breakages (panics) since the
tcp_lro code was committed that have been reported. Quite a few new features
are now in rack (prefecting of DGP -- Dynamic Goodput Pacing among the
largest). There is also support for ack-war prevention. Documents comming soon
on rack..
Michael Tuexen [Sun, 18 Apr 2021 14:08:08 +0000 (16:08 +0200)]
tcp: add support for TCP over UDP
Adding support for TCP over UDP allows communication with
TCP stacks which can be implemented in userspace without
requiring special priviledges or specific support by the OS.
This is joint work with rrs.
Bjoern A. Zeeb [Wed, 10 Mar 2021 22:17:07 +0000 (22:17 +0000)]
termios: add more speeds
A lot of small arm64 gadgets are using 1500000 as console speed.
While cu can perfectly deal with this some 3rd party software, e.g.,
comms/conserver-con add speeds based on B<n> being defined.
Having it defined here simplifies enhancing other software.
Obtained-from: NetBSD sys/sys/termios.h 1.36
MFC-after: 2 weeks
Reviewed-by: philip (,okayed by imp)
Differential Revision: https://reviews.freebsd.org/D29209
should be handled as errors. Historically receive buffer overflows have been
ignored and programs could not tell if they missed messages or messages had
been truncated because of overflows. Since programs historically do not expect
to get receive overflow errors, this behavior is not the default.
This is really really important for programs that use route(4) to keep in sync
with the system. If we loose a message then we need to reload the full system
state, otherwise the behaviour from that point is undefined and can lead
to chasing bogus bug reports.
Alex Richardson [Wed, 3 Feb 2021 15:27:17 +0000 (15:27 +0000)]
Expose clang's alignment builtins and use them for roundup2/rounddown2
This makes roundup2/rounddown2 type- and const-preserving and allows
using it on pointer types without casting to uintptr_t first. Not
performing pointer-to-integer conversions also helps the compiler's
optimization passes and can therefore result in better code generation.
When using it with integer values there should be no change other than
the compiler checking that the alignment value is a valid power-of-two.
I originally implemented these builtins for CHERI a few years ago and
they have been very useful for CheriBSD. However, they are also useful
for non-CHERI code so I was able to upstream them for Clang 10.0.
Rationale from the clang documentation:
Clang provides builtins to support checking and adjusting alignment
of pointers and integers. These builtins can be used to avoid relying
on implementation-defined behavior of arithmetic on integers derived
from pointers. Additionally, these builtins retain type information
and, unlike bitwise arithmetic, they can perform semantic checking on
the alignment value.
There is also a feature request for GCC, so GCC may also support it in
the future: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98641
Gleb Smirnoff [Thu, 28 Jan 2021 21:08:48 +0000 (13:08 -0800)]
Catch up with 6edfd179c86: mechanically rename IFCAP_NOMAP to IFCAP_MEXTPG.
Originally IFCAP_NOMAP meant that the mbuf has external storage pointer
that points to unmapped address. Then, this was extended to array of
such pointers. Then, such mbufs were augmented with header/trailer.
Basically, extended mbufs are extended, and set of features is subject
to change. The new name should be generic enough to avoid further
renaming.
Add tcgetwinsize(3) and tcsetwinsize(3) to termios
These functions get/set tty winsize respectively, and are trivial wrappers
around corresponding termio ioctls.
The functions are expected to be a part of POSIX.1 issue 8:
https://www.austingroupbugs.net/view.php?id=1151#c3856.
They are currently available in NetBSD and in musl libc.
Andrew Gallatin [Sat, 19 Dec 2020 22:04:46 +0000 (22:04 +0000)]
Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain
In order to efficiently serve web traffic on a NUMA
machine, one must avoid as many NUMA domain crossings as
possible. With SO_REUSEPORT_LB, a number of workers can share a
listen socket. However, even if a worker sets affinity to a core
or set of cores on a NUMA domain, it will receive connections
associated with all NUMA domains in the system. This will lead to
cross-domain traffic when the server writes to the socket or
calls sendfile(), and memory is allocated on the server's local
NUMA node, but transmitted on the NUMA node associated with the
TCP connection. Similarly, when the server reads from the socket,
he will likely be reading memory allocated on the NUMA domain
associated with the TCP connection.
This change provides a new socket ioctl, TCP_REUSPORT_LB_NUMA. A
server can now tell the kernel to filter traffic so that only
incoming connections associated with the desired NUMA domain are
given to the server. (Of course, in the case where there are no
servers sharing the listen socket on some domain, then as a
fallback, traffic will be hashed as normal to all servers sharing
the listen socket regardless of domain). This allows a server to
deal only with traffic that is local to its NUMA domain, and
avoids cross-domain traffic in most cases.
This patch, and a corresponding small patch to nginx to use
TCP_REUSPORT_LB_NUMA allows us to serve 190Gb/s of kTLS encrypted
https media content from dual-socket Xeons with only 13% (as
measured by pcm.x) cross domain traffic on the memory controller.
Brooks Davis [Fri, 11 Dec 2020 01:00:07 +0000 (01:00 +0000)]
style(9): Correct whitespace in struct definitions
struct ifconf and struct ifreq use the odd style "struct<tab>foo".
struct ifdrv seems to have tried to follow this but was committed with
spaces in place of most tabs resulting in "struct<space><space>ifdrv".
Conrad Meyer [Tue, 17 Nov 2020 20:01:21 +0000 (20:01 +0000)]
unix(4): Enhance LOCAL_CREDS_PERSISTENT ABI
As this ABI is still fresh (r367287), let's correct some mistakes now:
- Version the structure to allow for future changes
- Include sender's pid in control message structure
- Use a distinct control message type from the cmsgcred / sockcred mess
Conrad Meyer [Tue, 3 Nov 2020 01:17:45 +0000 (01:17 +0000)]
unix(4): Add SOL_LOCAL:LOCAL_CREDS_PERSISTENT
This option is intended to be semantically identical to Linux's
SOL_SOCKET:SO_PASSCRED. For now, it is mutually exclusive with the
pre-existing sockopt SOL_LOCAL:LOCAL_CREDS.
Warner Losh [Fri, 30 Oct 2020 22:00:35 +0000 (22:00 +0000)]
Integrate 4.4BSD-Lite2 changes to IOC_* definitions
Bring in the long-overdue 4.4BSD-Lite2 rev 8.3 by cgd of
sys/ioccom.h. This uses UL suffix for the IOC_* constants so they
don't sign extend. Also bring in the handy diagram from NetBSD's
version of this file. This alters the 4.4BSD-Lite2 code slightly
in a way that's semantically the same but more compact.
This should stop the warnings from Chrome for bogus sign extension.
git://sourceware.org / newlib-cygwin.git / log