Corinna Vinschen [Mon, 14 Jan 2019 19:38:24 +0000 (20:38 +0100)]
Cygwin: wincap: split has_posix_file_info
While FileRenameInformationEx is defined starting with Windows
10 1709 per MSDN, it only starts working in W10 1809, apparently.
Users of 1803 report "Function not implemented".
Introduce wincap_10_1809 and change the version check in
wincapc::init accordingly. Split has_posix_file_info into
has_posix_unlink_semantics and has_posix_rename_semantics.
Enable the latter only starting with W10 1809.
Corinna Vinschen [Mon, 14 Jan 2019 16:19:37 +0000 (17:19 +0100)]
Cygwin: signalfd: implement non-polling select
Allow the signal thread to recognize we're called in consequence of
select on a signalfd. If the signal is part of the wait mask, don't
call any signal handler and don't remove the signal from the queue,
so a subsequent read (or sigwaitinfo/sigtimedwait) still gets the
signal. Instead, just signal the event object at
_cygtls::signalfd_select_wait for the thread running select.
The addition of signalfd_select_wait to _cygtls unearthed the alignment
problem of the context member again. To make sure this doesn't get lost,
improve the related comment in the header file so that this (hopefully)
doesn't get lost (again).
Corinna Vinschen [Sun, 13 Jan 2019 22:13:33 +0000 (23:13 +0100)]
Cygwin: signal: implement signalfd
First cut of a signalfd implementation.
Still TODO: Non-polling select.
This should mostly work as on Linux except for missing support
for some members of struct signalfd_siginfo, namely ssi_fd,
ssi_band (both SIGIO/SIGPOLL, not fully implemented) and ssi_trapno
(HW exception, required HW support).
Corinna Vinschen [Sun, 13 Jan 2019 21:43:52 +0000 (22:43 +0100)]
Cygwin: select: fix overwriting fd sets if poll returns no fd
There's a long-standing bug in select. If we have poll-only
descriptors in the fd set, select overwrites the incoming
fd sets with the polling result. If none of the fds is ready,
select has to loop again. But now the fd sets are set to all
zero and select hangs.
Fix this by utilizing the local fd sets r, w, e as storage for
the incoming fd sets and use them to initialize select_stuff.
If we have to loop, overwritung the incoming fd sets doesn't matter.
While at it, rename r, w, e to readfds_in, writefds_in, exceptfds_in.
Cygwin: try_to_bin: don't check recycler filename all the time
So far we check the recycler name all the time, and the last interation
also only managed to handle two ways to write the recycler. However,
an adventurous user might change the case of the recycler arbitrarily.
Fix this problem by keeping track of the name in a somewhat relaxed
fashion. Use camel back on drive C by default, all upper case elsewhere.
Only if the rename op fails do we fix the recycler name on the fly
when trying to create it, and it turns out it already existed.
Cygwin: try_to_bin: fix rootdir handle after reopening
If the first rename fails, we reopen the rootdir for creating a subdir.
The rootdir handle can change its value at this point, but the code
doesn't take this into account. The subsequent rename then fails with
STATUS_INVALID_HANDLE. Fix this by copying the new rootdir value to
pfri->RootDirectory.
So far we reopened the file if it was opened case sensitive to
workaround the problem that the recycler could be named in
camel back or all upper case, depending on who created it.
That's a problem for O_TMPFILE on pre-W10. As soon as the
original HANDLE gets closed, delete-on-close is converted to full
delete disposition and all useful operations on the file cease to
work (STATUS_ACCESS_DENIED or STATUS_FILE_DELETED).
To avoid that problem drop the reopen code and check for the exact
recycler filename, either $Recycle.Bin or $RECYCLE.BIN, if the file
has been opened case sensitive.
Cygwin: introduce fhandler_process_fd and add stat(2) handling
move special fd symlink code into own fhandler_process_fd class
to simplify further additions to /proc/PID/fd/DESCRIPTOR symlink
handling.
Add a method to handle stat(2) on such a proc fd symlink by handle.
This allows correct reply from stat(2) if the target file has been
deleted. This eventually fixes `awk -f /dev/fd/3 3<<eof'.
Cygwin: path_conv: add PATH_RESOLVE_PROCFD path_types flag
path_conv now sets the PATH_RESOLVE_PROCFD flag in path_flags if
the PC_SYM_NOFOLLOW_PROCFD pathconv_arg flag has been set on input
*and* the file is actually a proc fd symlink.
Add matching path_conv::follow_fd_symlink method for checking and
use it in open(2).
Cygwin: path_conv: decouple path_types from mount types
- Remove another unfortunate amalgamation: Mount flags (MOUNT_xxx)
are converted to path_types (PATH_xxx) and mixed with non-mount
path_types flags in the same storage, leading to a tangled,
pell-mell usage of mount flags and path flags in path_conv and
symlink_info.
- There's also the case of PC_NONULLEMPTY. It's used in exactly
one place with a path_conv constructor only used in this single
place, just to override the automatic PC_NULLEMPTY addition
when calling the other path_conv constructors. Crazily,
PC_NONULLEMPTY is a define, no path_types flag, despite its
name.
- It doesn't help that the binary flag exists as mount and path
flag, while the text flag only exists as path flag. This leads
to mount code using path flags to set text/binary. Very confusing
is the fact that a text mount/path flag is not actually required;
the mount code sets the text flag on non binary mounts anyway, so
there are only two states. However, to puzzle people a bit more,
path_conv::binary wrongly implies there's a third, non-binary/non-text
state.
Clean up this mess:
- Store path flags separately from mount flags in path_conv and
symlink_info classes and change all checks and testing inline
methods accordingly.
- Make PC_NONULLEMPTY a simple path_types flag and drop the
redundant path_check constructor.
- Clean up the definition of pathconv_arg, path_types, and mount flags.
Use _BIT expression, newly define in cygwin/bits.h.
Cygwin: path_conv: decouple pathconv_flags from path_types
There's an unfortunate amalgamation of caller-provided pathconv_arg
flags with path_types flags which in turn are mostly mount flags.
This leads to a confusion of flag values in sylink_info::pflags and,
in turn, in path_conv::path_flags.
This patch decouples pathconv_flags from the other flags by making
sure that a pathconv_flag is never copied into a variable used for
path_types flags. Also, remove PATH_NO_ACCESS_CHECK since it's
not necessary.
Cygwin: Move O_TMPFILE to bin and allow linkat by handle
Along the same lines as the previous patch: By reopening an
O_TMPFILE by handle, we can now move the file to the bin at
open time and thus free'ing up the parent dir and *still*
open the file as /proc/PID/fd/DESCRIPTOR by linkat(2).
Cygwin: implement /proc/PID/fd/DESCRIPTOR reopening by handle
Allows expressions along the lines of `cat /proc/self/fd/0 <<EOF'.
The problem here is that the temporary file used for the here script
has already been deleted by the shell. Opening by filename, as
implemented so far, doesn't work because the file has been moved
to the bin.
Allow reopening files by handle the same way from another process
as long as we have sufficient permissions on the foreign process.
Cygwin: pinfo: add method to send a serialized path_conv and HANDLE
To allow reopening a file open in another process by HANDLE, introduce
a matching file_pathconv method, taking a file descriptor as parameter.
The result is a serialized path_conv and a HANDLE value. The HANDLE is
valid in the foreign process and MUST be duplicated into the target
process before usage.
Cygwin: fhandler_base::open: allow to reopen file from handle
So far io_handle is NULL when calling fhandler_base::open to
open or create a file. Add a check for io_handle to allow
priming the fhandler with a HANDLE value so we can reopen a
file from a HANDLE on file systems supporting it. This allows
to open already deleted files for further action. This will
be used by open("/proc/PID/fd/DESCRIPTOR") scenarios.
There's no POSIX semantics allowing to link a file with a link
count of 0 and making it available in the file system again.
In fact, the Linux linkat extension AT_EMPTY_PATH explicitely
disallows to link a file descriptor to a file with a link count
of 0, except for O_TMPFILE without O_EXCL.
Wilco Dijkstra [Mon, 31 Dec 2018 18:01:52 +0000 (18:01 +0000)]
Improve performance of memmem
This patch significantly improves performance of memmem using a novel
modified Horspool algorithm. Needles up to size 256 use a bad-character
table indexed by hashed pairs of characters to quickly skip past mismatches.
Long needles use a self-adapting filtering step to avoid comparing the whole
needle repeatedly.
By limiting the needle length to 256, the shift table only requires 8 bits
per entry, lowering preprocessing overhead and minimizing cache effects.
This limit also implies worst-case performance is linear.
Small needles up to size 2 use a dedicated linear search. Very long needles
use the Two-Way algorithm (to avoid increasing stack size inlining is now disabled).
The performance gain is 6.6 times on English text on AArch64 using random
needles with average size 8 (this is even faster than the recently improved strstr
algorithm, so I'll update that in the near future).
The size-optimized memmem has also been rewritten from scratch to get a
2.7x performance gain.
Tested against GLIBC testsuite and randomized tests.
Corinna Vinschen [Wed, 26 Dec 2018 20:51:13 +0000 (21:51 +0100)]
Cygwin: open(2): Change comment in disabled O_TMPFILE POSIX unlink code
- Turns out, the definition of POSIX unlink semantics is half-hearted
so far: It's not possible to link an open file HANDLE if it has
been deleted with POSIX semantics, nor is it possible to remove
the delete disposition. This breaks linkat on an O_TMPFILE.
Corinna Vinschen [Sun, 23 Dec 2018 20:36:42 +0000 (21:36 +0100)]
Cygwin: utilize FILE_DISPOSITION_POSIX_SEMANTICS
- short-circuit most code in unlink_nt since it's not necessary
anymore if FILE_DISPOSITION_POSIX_SEMANTICS is supported.
- Immediately remove O_TMPFILE from filesystem after creation.
Disable code for now because we have to implement /proc/self/fd
opening by handle first, lest linkat fails.
Various new file info class members adding important POSIX semantics
have been added with W10 1709. We may want to utilize them, so add
a matching wincaps.
Rearrange checking the W10 build number to prefer the latest builds
over the older builds. Rename wincap_10 to wincap_10_1507 for
enhanced clarity.
Corinna Vinschen [Sat, 22 Dec 2018 23:17:34 +0000 (00:17 +0100)]
Cygwin: ntdll.h: Update _FILE_INFORMATION_CLASS
- Add missing members added in later OS versions
- Rearrange accompanying FILE_foo_INFORMATION structs
ordered by info class
- Add promising FILE_foo_INFORMATION structs of later
Windows 10 releases plus accompanying enums
- Drop "Checked on 64 bit" comments since that's self-evident
these days
Corinna Vinschen [Wed, 19 Dec 2018 20:10:37 +0000 (21:10 +0100)]
Cygwin: fix heap allocation on WOW64 and /3GB enabled 32 bit machines
The check for the TEB being allocated beyond the first 2GB area is not
valid anymore. At least on W10 WOW64, the TEB is allocated in the
lower 2GB even in large-address aware executables. Use VirtualQuery
instead. It fails for invalid addresses so that's a simple enough test.
Szabolcs Nagy [Mon, 10 Dec 2018 14:40:01 +0000 (14:40 +0000)]
Fix powf overflow handling in non-nearest rounding mode
The threshold value at which powf overflows depends on the rounding mode
and the current check did not take this into account. So when the result
was rounded away from zero it could become infinity without setting
errno to ERANGE.
Example: pow(0x1.7ac7cp+5, 23) is 0x1.fffffep+127 + 0.1633ulp
If the result goes above 0x1.fffffep+127 + 0.5ulp then errno is set,
which is fine in nearest rounding mode, but
powf(0x1.7ac7cp+5, 23) is inf in upward rounding mode
powf(-0x1.7ac7cp+5, 23) is -inf in downward rounding mode
and the previous implementation did not set errno in these cases.
The fix tries to avoid affecting the common code path or calling a
function that may introduce a stack frame, so float arithmetics is used
to check the rounding mode and the threshold is selected accordingly.
imp [Tue, 20 Nov 2018 07:11:23 +0000 (07:11 +0000)]
Ensure that all values of ns, us and ms work
for {n,u,m}stosbt
Integer overflows and wrong constants limited the accuracy of these
functions and created situatiosn where sbttoXs(Xstosbt(Y)) != Y. This
was especailly true in the ns case where we had millions of values
that were wrong.
Instead, used fixed constants because there's no way to say ceil(X)
for integer math. Document what these crazy constants are.
Also, use a shift one fewer left to avoid integer overflow causing
incorrect results, and adjust the equasion accordingly. Document this.
Allow times >= 1s to be well defined for these conversion functions
(at least the Xstosbt). There's too many users in the tree that they
work for >= 1s.
This fixes a failure on boot to program firmware on the mlx4
NIC. There was a msleep(1000) in the code. Prior to my recent rounding
changes, msleep(1000) worked, but msleep(1001) did not because the old
code rounded to just below 2^64 and the new code rounds to just above
it (overflowing, causing the msleep(1000) to really sleep 1ms).
A test program to test all cases will be committed shortly. The test
exaustively tries every value (thanks to bde for the test).
imp [Thu, 15 Nov 2018 16:02:13 +0000 (16:02 +0000)]
When converting ns,us,ms to sbt, return the ceil()
of the result rather than the floor(). Returning the floor means that
sbttoX(Xtosbt(y)) != y for almost all values of y. In practice, this
results in a difference of at most 1 in the lsb of the sbintime_t. This
difference is meaningless for all current users of these functions, but
is important for the newly introduced sysctl conversion routines which
implicitly rely on the transformation being idempotent.
pfg [Mon, 20 Nov 2017 19:43:44 +0000 (19:43 +0000)]
sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
ian [Sat, 29 Jul 2017 17:00:23 +0000 (17:00 +0000)]
Add inline functions to convert between sbintime_t
and decimal time units. Use them in some existing code that is
vulnerable to roundoff errors.
The existing constant SBT_1NS is a honeypot, luring unsuspecting folks into
writing code such as long_timeout_ns*SBT_1NS to generate the argument for a
sleep call. The actual value of 1ns in sbt units is ~4.3, leading to a
large roundoff error giving a shorter sleep than expected when multiplying
by the trucated value of 4 in SBT_1NS. (The evil honeypot aspect becomes
clear after you waste a whole day figuring out why your sleeps return early.)
imp [Tue, 28 Feb 2017 23:42:47 +0000 (23:42 +0000)]
Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
While reformatting the script, backticks `` were replaced with
brackets $(). This in turn invalidated the \\( ... \\) expressions in the
sed script because backslash resolution in $() works differently from
backslash resolution in ``. Only a single backslash is valid now.
While at it, fix up the uname(2) date representation when building a
snapshot.
Corinna Vinschen [Fri, 30 Nov 2018 21:39:57 +0000 (22:39 +0100)]
Cygwin: clocks: fix a hang on pre-Windows 10 machines
when calling clocks too early in DLL init, the vtables are not correctly
set up for some reason. Calls to init() from now() fail because the init
pointer in the vtable is NULL.
Real life example is mintty which runs into a minor problem at startup,
triggering a system_printf call. Strace is another problem, it's called
the first time prior to any class initialization.
Workaround is to make sure that no virtual methods are called in an
early stage. Make init() non-virtual and convert resolution() to a
virtual method instead. Add a special non-virtual
clk_monotonic_t::strace_usecs.
While at it:
- Inline internal-only methods.
- Drop the `inited' member. Convert period/ticks_per_sec toa union.
Initialize period/ticks_per_sec via InterlockeExchange64.
- Fix GetTickCount64 usage. No, it's not returning ticks but
milliseconds since boot (unbiased).
Corinna Vinschen [Tue, 27 Nov 2018 12:47:02 +0000 (13:47 +0100)]
Cygwin: implement extensible clock interface
- Drop hires_[nm]s clocks, rename hires.h to clock.h.
- Implement clk_t class as an extensible clock class in new file clock.cc.
- Introduce get_clock(clock_id) returning a pointer to the clk_t instance
for clock_id. Provide the following methods along the lines of the former
hires classes:
Corinna Vinschen [Tue, 27 Nov 2018 12:59:37 +0000 (13:59 +0100)]
Cygwin: timers: clock_setres: make no-op
clock_setres is a questionable function only existing on QNX.
Disable the function, just return success for CLOCK_REALTIME
to maintain backward compatibility.
Corinna Vinschen [Mon, 26 Nov 2018 16:47:53 +0000 (17:47 +0100)]
Cygwin: timers: use spinlock to prime hires_ns thread-safe
The current method to make hires_ns priming thread-safe isn't
thread-safe. Rather than hoping that running the thread in
TIME_CRITICAL priority is doing the right thing, use a spinlock.
A previous commit introduced the ability to use the semi-hosting
SYS_EXIT_EXTENDED operation to libgloss, this commit adds the same
ability to the sys/arm/ backend so that building newlib only will
provide the same capabilities.
Jim Wilson [Mon, 19 Nov 2018 23:27:05 +0000 (15:27 -0800)]
Add libgloss install-pdf and install-html targets.
The toplevel makefile used by binutils/gcc/newlib/etc has install-pdf and
install-html targets, but they fail because libgloss doesn't support them.
Tested with an arm-eabi combined tree build and install, and verifying that
the install-pdf and install-html targets now work, and that the pdf and html
doc files are now in the install tree.
libgloss/
* Makefile.in (install-html, install-pdf): New.
* doc/Makefile.in (htmldir, pdfdir): New.
(porting.ps): Delete white space on blank line.
(install-pdf, install-html): New.
Enable return code with semi-hosting SYS_EXIT_EXTENDED
The _exit function currently passes -1 as a "sig" to the _kill function as an
invalid signal number so that _kill can distinguish between an abort and a
standard exit.
For boards using the SYS_EXIT_EXTENDED semi-hosting operation to return a
status code, this means that the "status" paramter to _exit is ignored and the
return code is always -1.
https://developer.arm.com/docs/100863/latest/semihosting-operations/sys_exit_extended-0x20
This patch puts shared code between _kill and _exit into a new function
_kill_shared that takes the semi-hosting "reason" to use (if semi-hosting is
available) as an argument.
For semi-hosting _kill_shared provides that "reason".
Without the "sig" argument being used to distinguish between a normal and
abnormal exit, the _exit function can provide the return code to be used if the
SYS_EXIT_EXTENDED operation is available.
Wilco Dijkstra [Thu, 8 Nov 2018 15:50:42 +0000 (15:50 +0000)]
Fix issue with dst bias in memset
This patch fixes an issue in the previous memset loop change. If the
zva size is >= 256 and there are more than 64 bytes left in the
tail, we could enter the loop and thus need to rebias dst by 32 as
well.
Since no known CPUs use this size this can't be tested natively, so I've
tested it on a simulator initialized with a large zva size.