Sebastian Huber [Tue, 5 Oct 2021 13:53:47 +0000 (15:53 +0200)]
sys/tree.h: Red child with black sibling rotations
Add specialized rotations RB_RED_ROTATE_LEFT() and RB_RED_ROTATE_RIGHT() which
may be used if we rotate a red child which has a black sibling. Such a red
node must have at least two child nodes so that the following red-black tree
invariant is fulfilled:
Every path from a given node to any of its descendant NULL nodes goes through
the same number of black nodes.
Sebastian Huber [Tue, 5 Oct 2021 13:43:34 +0000 (15:43 +0200)]
sys/tree.h: Add parent rotations
Add specialized rotations RB_PARENT_ROTATE_LEFT() and RB_PARENT_ROTATE_RIGHT()
which may be used if the parent node exists and the direction of the child is
known. The specialized rotations are derived from RB_ROTATE_LEFT() and
RB_ROTATE_RIGHT() where the RB_SWAP_CHILD() was replaced by a simple
assignment.
So, the RB_ISRED() contains an implicit check for NULL. In
RB_GENERATE_REMOVE_COLOR() the "elm" pointer cannot be NULL in the while
condition. Use RB_COLOR(elm) == RB_BLACK instead.
Cygwin: pipe: Use ProcessHandleInformation if available.
- The commit b531d6b0 introduced temporary_query_hdl() which uses
SystemHandleInformation. With this patch, ProcessHandleInformation
rather than SystemHandleInformation is used if it is available.
This request is faster, however, is only available since Windows 8,
therefore, SystemHandleInformation is used for Windows Vista and 7
as before.
- The commit f79a4611 introduced query_hdl, which is the read pipe
handle kept in the write pipe instance in order to determine if
the pipe is ready to write in select(). This implementation has
a potential risk that the write side fails to detect the closure
of the read side if more than one writer exists and one of them
is a non-cygwin process.
With this patch, the strategy of commit f79a4611 is used only if
the process is running as a service. For a normal process,
instead of keeping query_hdl in the write pipe instance, it is
retrieved temporarily when select() is called. Actually, we
want to use tenporary query_hdl for all processes, however, it
does not work for service processes due to OpenProcess()
failure.
Ken Brown [Sat, 18 Sep 2021 13:10:23 +0000 (09:10 -0400)]
Cygwin: make callers of open_setup check for failure
open_setup is called by dtable::init_std_file_from_handle and
fhandler_base::open_with_arch. In both cases, failure of open_setup
is now a fatal error.
Currently this can only happen in the following situation: A Cygwin
process is started by a non-Cygwin process, one of the standard IO
handles is a pipe handle, and Cygwin is unable to create a required
mutex (read_mtx or hdl_cnt_mtx).
Cygwin: pipe, fifo: Move query_hdl and hdl_cnt_mtx to fhandler_pipe.
- query_hdl and hdl_cnt_mtx are moved from fhandler_pipe_fifo to
fhandler_pipe. Then reader_closed() is changed to virtual and
overridden in fhandler_pipe.
- This patch fixes the race issue in the handle counting to detect
closure of read pipe, which is introduced by commit f79a4611.
A mutex hdl_cnt_mtx is introduced for this issue.
Cygwin: close_all_files: Do not duplicate stderr for write pipe.
- Currently, the stderr handle is duplicated in close_all_files().
This interferes the handle counting for detecting closure of read
pipe, which is introduced by commit f79a4611. This patch stops
duplicating stderr handle if it is write pipe.
Mike Frysinger [Wed, 15 Sep 2021 04:27:13 +0000 (00:27 -0400)]
libgloss: add missing aclocal.m4 files
These subdirs were missing aclocal.m4 files pulling in macros from
../acinclude.m4 which caused some macros to not be expanded. For
example, autoconf complains:
configure.ac:25: error: possibly undefined macro: LIB_AC_PROG_CC
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
These were generated with aclocal-1.9 as that seems to be what was
used in these dirs previously, and with whatever version of autoconf
the specific subdir was using. This should minimize diffs.
Mike Frysinger [Wed, 15 Sep 2021 04:20:58 +0000 (00:20 -0400)]
libgloss: epiphany: rename symbol prefix cache var
Autoconf emits a warning for this:
configure.ac:75: warning: AC_CACHE_VAL(libc_symbol_prefix, ...): suspicious cache-id, must contain _cv_ to be cached
Rename the variable to match the naming in libnosys/ subdir.
When running autoconf-2.69 in here, we get:
configure.ac:57: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2503: _AC_PREPROC_IFELSE is expanded from...
../../lib/autoconf/general.m4:2518: AC_PREPROC_IFELSE is expanded from...
configure.ac:57: the top level
configure.ac:61: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2503: _AC_PREPROC_IFELSE is expanded from...
../../lib/autoconf/general.m4:2518: AC_PREPROC_IFELSE is expanded from...
configure.ac:61: the top level
Cygwin: pipe: Do not call PeekNamedPipe() if it is not necessary.
- In pipe_data_available() in select.cc, PeekNamedPipe() call is
not needed if WriteQuotaAvailable is non-zero because we already
know the write pipe has a space. Therefore, with this patch,
PeekNamedPipe() is called only when WriteQuotaAvailable is zero.
This makes select() on pipe faster a bit.
Cygwin: pipe, fifo: Release select_sem semaphore as much as needed.
- Currently, raw_read(), raw_write() and close() release select_sem
unconditionally even if no waiter for select_sem exists. With this
patch, only the minimum number of semaphores required is released.
Cygwin: pipe: Use read pipe handle for select() on write pipe.
- Usually WriteQuotaAvailable retrieved by NtQueryInformationFile()
on the write side reflects the space available in the inbound buffer
on the read side. However, if a pipe read is currently pending,
WriteQuotaAvailable on the write side is decremented by the number
of bytes the read side is requesting. So it's possible (even likely)
that WriteQuotaAvailable is 0, even if the inbound buffer on the
read side is not full. This can lead to a deadlock situation:
The reader is waiting for data, but select on the writer side
assumes that no space is available in the read side inbound buffer.
Currently, to avoid this stuation, read() does not request larger
block than pipe size - 1. However, this mechanism does not take
effect if the reader side is non-cygwin app.
The only reliable information is available on the read side, so
fetch info from the read side via the pipe-specific query handle
(query_hdl) introduced.
If the query_hdl (read handle) is kept in write side, writer can
not detect closure of read pipe. Therefore, raw_write() counts
write handle and query_hdl. If they are equal, only the pairs of
write handle and query_hdl are alive. In this case, raw_write()
returns EPIPE and raises SIGPIPE.
- Nonblocking pipes (PIPE_NOWAIT) are not well handled by non-Cygwin
tools, so convert pipe handles to PIPE_WAIT handles when spawning
a non-Cygwin process.
Cygwin: pipes: do not duplicate sec_none{_nih} locally when creating objects
We already fetched the correct SECURITY_ATTRIBUTES at the start of
fhandler_pipe::create, so using another SECURITY_ATTRIBUTES object for
the mutex and semaphore objects doesn't make much sense.
Cygwin: fix inheritence of select_sem on write side of pipe
select_sem gets created on the read side with inheritence settings
depending on the O_CLOEXEC flag. Then it gets duplicated to the write
side with unconditional inheritence. Fix that.
Ken Brown [Tue, 7 Sep 2021 21:40:21 +0000 (17:40 -0400)]
Cygwin: set buffer size for pipes created by non-Cygwin processes
Rename fhandler_pipe_and_fifo::max_atomic_write to pipe_buf_size.
This reflect its actual meaning better. The fhandler_pipe_and_fifo
constructor initializes it to DEFAULT_PIPEBUFSIZE (== 64K), which is
the buffer size for the windows pipes created by fhandler_pipe and
fhandler_fifo. But if we inherit a stdio pipe handle from a
non-Cygwin process, the buffer size could be different.
To remedy this, add a method fhandler_pipe::set_pipe_buf_size that
queries the OS for the pipe buffer size, and use it in
dtable::init_std_file_from_handle.
Cygwin: pipes: always terminate async IO in blocking mode
In blocking mode, the underlying IO must always be terminated,
one way or the other, to make sure the application knows the exact
state after returning from the IO function. Therefore, always call
CancelIo in blocking mode.
Cygwin: pipes: cancel async IO if thread cancellation is in progress
Just cancelling a thread doesn't cancel async IO started by this thread.
Fix this by returning from cygwait and calling CancelIo before canceling
self.
Cygwin: pipe: Stop counting reader and read all available data.
- By guarding read with read_mtx, no more than one ReadFile can
be called simultaneously. So couting read handles is no longer
necessary.
- Make raw_read code as similar as possible to raw_write code.
Ken Brown [Fri, 3 Sep 2021 23:41:52 +0000 (19:41 -0400)]
Cygwin: new class fhandler_pipe_fifo
This is a parent of fhandler_pipe and fhandler_fifo for code that is
common between the two classes. Currently it just contains
max_atomic_write and raw_write(). The latter is identical to what
used to be fhandler_pipe::raw_write().
FILE_PIPE_LOCAL_INFORMATION::WriteQuotaAvailable is unreliable.
Usually WriteQuotaAvailable on the write side reflects the space
available in the inbound buffer on the read side. However, if a
pipe read is currently pending, WriteQuotaAvailable on the write side
is decremented by the number of bytes the read side is requesting.
So it's possible (even likely) that WriteQuotaAvailable is 0, even
if the inbound buffer on the read side is not full. This can lead to
a deadlock situation: The reader is waiting for data, but select
on the writer side assumes that no space is available in the read
side inbound buffer.
This patch implements a workaround by never trying to read more than
half the buffer size blocking if the read buffer is empty. This first
cut tries to take the number of open readers into account by reducing
the amount of requested bytes accordingly.
Cygwin: pipes: fix POSIX requirement for non-blocking pipe writes
POSIX requires atomicity for non-blocking writes <= PIPE_BUF bytes
and writing of at least 1 byte if any buffer space is left.
Windows NtWriteFile returns STATUS_SUCCESS and "0 bytes written"
if the write doesn't match buffer space. Fix this discrepancy.
The buffer pointer is incremented by "chunk", which is what we
typically try to write, but this isn't what actually got written.
Increment the buffer pointer by what we actually wrote, as returned
by NtWriteFile.
Cygwin: move get_obj_handle_count() to miscfuncs.cc
get_obj_handle_count() is used in flock only so far, but pipe
handling might have a usage, too, soon. Given that this function
might be generally useful and isn't restricted to flock usage,
move it to miscfuncs.cc and make it non-static. Add a prototype
in miscfuncs.h.
Corinna Vinschen [Tue, 31 Aug 2021 08:50:23 +0000 (10:50 +0200)]
Cygwin: _pipe: add a comment
I wasted valuable minutes of my life just to find out why we export
this weird version of pipe. In the pre-2000 era the idea was Cygwin
could be used as drop-in replacement for msvcrt.dll, apparently.
Ken Brown [Sat, 25 May 2019 14:27:38 +0000 (10:27 -0400)]
Cygwin: fhandler_pipe: fix permission problem
The read handles of pipes created by CreateNamedPipe don't have
FILE_WRITE_ATTRIBUTES access unless the pipe is created with
PIPE_ACCESS_DUPLEX. This causes set_pipe_non_blocking to fail on such
handles. To fix this, add a helper function nt_create, which uses
NtCreateNamedPipeFile instead of CreateNamedPipe and gives us more
flexibility in setting access rights.
Use this helper function in fhandler_pipe::create (fhandler_pipe *[2],
unsigned, int), which is the version of fhandler_pipe::create used by
the pipe and pipe2 system calls.
For convenience, also add a static member function
fhandler_pipe::npfs_handle similar to those used by fhandler_fifo and
fhandler_socket_unix.
Ken Brown [Sat, 25 May 2019 17:05:49 +0000 (13:05 -0400)]
Cygwin: fhandler_pipe: control blocking mode of the Windows pipe
Add methods 'set_pipe_non_blocking' and 'fcntl' to keep the blocking
mode of the Windows pipe in sync with that of the fhandler_pipe
object. This applies to pipes created with the 'pipe' and 'pipe2'
system calls.
Ken Brown [Thu, 26 Aug 2021 20:57:21 +0000 (16:57 -0400)]
Cygwin: fhandler_pipe: derive from fhandler_base
Previously fhandler_pipe was derived from fhandler_base_overlapped,
which we are going to remove in a future commit. Make minimal changes
so that the build still succeeds.
Cygwin: fix initializing MEM_EXTENDED_PARAMETER, take 2
In https://cygwin.com/pipermail/cygwin/2021-September/249361.html
Brian pointed out that initializing the structure would be more
future proof, should the developers at Microsoft ever decide to
split the Reserved field and use some bits of the struct for
other purposes.
MEM_EXTENDED_PARAMETER consists of a 64 bit bitfield which contains
the Type and the Reserved members. The former usage of designated
initializer lists initialized Type, but not Reserved. Since that's
not possible anymore due to a g++ 11.2 bug, Cygwin initializes the
MEM_EXTENDED_PARAMETER structs explicitely. This results in a
random value in Reserved, which at least VirtualAlloc2 chokes on
(ERROR_INVALID_PARAMETER).
libm/machine/i386/f_ldexp.S:30: Warning: no instruction mnemonic suffix given and no register operands; using default for `fild'
libm/machine/i386/f_ldexpf.S:30: Warning: no instruction mnemonic suffix given and no register operands; using default for `fild'
Joel Sherrill [Fri, 27 Aug 2021 13:39:31 +0000 (08:39 -0500)]
libc/include/sys/signal.h: Change __STDINT_EXP() to __SIZEOF_INT__
__STDINT_EXP() is provided by newlib but not by stdint-gcc.h. stdint-gcc.h
is used when the GCC argument -ffreestanding is used and this results in this
file not compiling.
Corinna Vinschen [Fri, 27 Aug 2021 12:26:42 +0000 (14:26 +0200)]
Cygwin: loader script: add DWARF 5 sections
Modern gcc's generate additional DWARF 5 debug sections, which were
still missing in our Cygwin loader script. With ld from binutils 2.37,
this results in diagnostic output when linking the Cygwin DLL...
ld: cygwin0.dll:/4: section below image base
ld: cygwin0.dll:/20: section below image base
ld: cygwin0.dll:/36: section below image base
...and the section addresses given to these sections (.debug_loclists,
.debug_rnglists, debug_line_str) will be wrong.
Fix this by adding the missing DWARF 5 sections to our linker script
template cygwin.sc.in. Add a comment in terms of the deprecated
DWARF 4 section .debug_types.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de> Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Corinna Vinschen [Thu, 26 Aug 2021 20:21:21 +0000 (22:21 +0200)]
Cygwin: Allow using CFLAGS also for C++ files
This was used before switching to automake to allow easy tweaking
of optimization and debugging settings from the command line during
testing. Reenable.
Roger Sayle [Tue, 24 Aug 2021 16:24:19 +0000 (17:24 +0100)]
nvptx: Emulate clock and other machine stubs.
This patch to the libc/machine/nvptx port of newlib implements an
approximation of "clock" and provides some additional stub routines.
These changes not only reduce the number of (link) failures in the GCC
testsuite when targeting nvptx-none, but also allow the NIST scimark4
benchmark to compile and run without modification.
newlib already contains support for backends to provide their own
clock implementations via -DCLOCK_PROVIDED. That functionality is
used here to return an approximate elapsed time based on the NVidia
GPU's clock64 cycle counter. Although not great, this is better than
the current behaviour of link error from the unresolved symbol
_times_r.
The other part of the patch is to add a small number of stub functions
to nvptx's misc.c. Adding isatty, for example, resolves linking
problems in libc from the dependency in __smakebuf_r, and the sync
stub, for example, fixes the failure with GCC's
testsuite/gfortran.dg/ISO_Fortran_binding_14.f90 [which simply tests
that gfortran can call a/any C function].
newlib/
configure.host: Add -DCLOCK_PROVIDED to newlib_cflags on nvptx*.
newlib/libc/machine/nvptx
Makefile.am: Add clock.c to lib_a_SOURCES.
clock.c: New source file to implement/approximate clock().
misc.c: Add stubs for fstat, isatty, open, sync and unlink.
Aleksand Malikov [Mon, 23 Aug 2021 14:27:48 +0000 (17:27 +0300)]
fix race condition in List_insert
Revert mx parameter and mutex lock while operating the list.
Mutex was removed with 94d24160 informing that:
'Use InterlockedCompareExchangePointer to ensure race safeness
without using a mutex.'
But it does not.
Calling pthread_mutex_init and pthread_mutex_destroy from two or
more threads occasionally leads to hang in pthread_mutex_destroy.
To not change the behaviour of other cases where List_insert was called,
List_insert_nolock is added.
Corinna Vinschen [Mon, 23 Aug 2021 07:56:22 +0000 (09:56 +0200)]
stdlib: conditionalize locale usage
_strtod_l as well as the gethex function both fetch the decimal point
from the current LC_NUMERIC locale info. This pulls in _C_numeric_locale
unconditionally even on targets not supporting locales at all.
Another problem is that strtod.c and gdtoa-gethex.c are ELIX 1, while
locale information in general isn't. This leads to potential build
breakage on bare metal targets.
Fix this by setting the decimal point to "." on all targets not
defining __HAVE_LOCALE_INFO__.
While at it, const'ify the entire local decimal point info in the
affected functions.
As outlined in the previous patch, the non-atomicity of iterating
over a directory in the NT namespace via NtQueryDirectoryObject
one entry each, results in potential duplication of directory entries.
Fix this for fhandler_procsys::readdir as well by fetching the entire
dir inside fhandler_procsys::opendir, storing it in a buffer, and just
return buffer content from fhandler_procsys::readdir.
Fixes: 43f65cdd7dae ("fhandler_procsys.cc: New file.") Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Corinna Vinschen [Thu, 19 Aug 2021 14:42:23 +0000 (16:42 +0200)]
Cygwin: fix all usages of NtQueryDirectoryObject
Due to reports on the Cygwin mailing list[1][2], it was uncovered
that a NtOpenDirectoryObject/NtQueryDirectoryObject/NtClose sequence
with NtQueryDirectoryObject iterating over the directory entries,
one entry per invocation, is not running atomically. If new entries
are inserted into the queried directory, other entries may be moved
around and then accidentally show up twice while iterating.
Change (almost) all NtQueryDirectoryObject invocations so that it gets
a really big buffer (64K) and ideally fetches all entries at once.
This appears to work atomically.
"Almost" all, because fhandler_procsys::readdir can't be easily changed.
Fixes: e9c8cb31930bd ("(format_proc_partitions): Revamp loop over existing harddisks by scanning the NT native \Device object directory and looking for Harddisk entries.") Fixes: a998dd7055766 ("Implement advisory file locking.") Fixes: 3b7cd74bfdf56 ("(winpids::enum_processes): Fetch Cygwin processes from listing of shared cygwin object dir in the native NT namespace.") Fixes: 0d6f2b0117aa7 ("syscalls.cc (sync_worker): Rewrite using native NT functions.") Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Corinna Vinschen [Thu, 19 Aug 2021 14:08:34 +0000 (16:08 +0200)]
Cygwin: return better inode numbers for block device entries in /proc/sys
Commit 3434d35a64736f0b77a12f61784c2caa33ac44cf fixed a problem when
accessing block devices via their /proc/sys/Device entries. This
changed the way stat info is generated for these devices, resulting
in identical inode numbers for all block devices under /proc/sys/Device.
This patch fixes that by faking a device number for these devices, just as
before.
Keith Packard [Tue, 17 Aug 2021 18:51:52 +0000 (11:51 -0700)]
svfwscanf: Simplify _sungetwc_r to eliminate apparent buffer overflow
svfwscanf replaces getwc and ungetwc_r. The comments in the code talk
about avoiding file operations, but they also need to bypass the
mbtowc calls as svfwscanf operates on wchar_t, not multibyte data,
which is a more important reason here; they would not work correctly
otherwise.
The ungetwc replacement has code which uses the 3 byte FILE _ubuf
field, but if wchar_t is 32-bits, this field is not large enough to
hold even one wchar_t value. Building in this mode generates warnings
about array overflow:
In file included from ../../newlib/libc/stdio/svfiwscanf.c:35:
../../newlib/libc/stdio/vfwscanf.c: In function '_sungetwc_r.isra':
../../newlib/libc/stdio/vfwscanf.c:316:12: warning: array subscript 4294967295 is above array bounds of 'unsigned char[3]' [-Warray-bounds]
316 | fp->_p = &fp->_ubuf[sizeof (fp->_ubuf) - sizeof (wchar_t)];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../newlib/libc/stdio/stdio.h:46,
from ../../newlib/libc/stdio/vfwscanf.c:82,
from ../../newlib/libc/stdio/svfiwscanf.c:35:
../../newlib/libc/include/sys/reent.h:216:17: note: while referencing '_ubuf'
216 | unsigned char _ubuf[3]; /* guarantee an ungetc() buffer */
| ^~~~~
However, the vfwscanf code *never* ungets data before the start of the
scanning operation, and *always* ungets data which matches the input
at that point, so the code always hits the block which backs up over
the input data and never hits the block which uses the _ubuf field.
In addition, the svfwscanf code will always start with the unget
buffer empty, so the ungetwc replacement never needs to support an
unget buffer at all.
Simplify the code by removing support for everything other than
backing up over the input data, leaving the check to make sure it
doesn't get underflowed in case the vfscanf code has a bug in it.
Matt Joyce [Fri, 6 Aug 2021 20:04:42 +0000 (22:04 +0200)]
libc: Added prototypes for new POSIX APIs
Added function prototypes to newlib/libc/include/pthread.h
for the following Issue 8 Standard APIs:
pthread_cond_clockwait()
pthread_mutex_clocklock()
pthread_rwlock_clockrdlock()
pthread_rwlock_clockwrlock()
The current implementation does not reliably initialize t0 once.
Additionally the initialization requires two calls to _gettimeofday().
Let's sacrifice a byte to keep the initialization status
and reduce the maximum number of calls to _gettimeofday().
This has caused issues in an application that invokes clock().
The problematic situation is as follows:
1) The program calls clock() which calls _times().
2) _gettimeofday(&t0, 0) puts 0 in t0.tv_usec (because less than 1 us has
elapsed since the beginning of time).
3) _gettimeofday(&t, 0) puts 1 in t.tv_usec (since now more than 1 us has
elapsed since the beginning of time).
4) That call to clock() returns 1 (the value from step 3 minus the value in
step 2).
5) The program does a second call to clock().
6) The code above still sees 0 in t0 so it tries to update t0 again and
_gettimeofday(&t0, 0) puts 1 in t0.tv_usec.
7) The _gettimeofday(&t, 0) puts 1 in t.tv_usec (since less than 1us has
elapsed since step 3).
8) clock() returns 0 (step 7 minus step 6) and indicates that time is
moving backwards.
Signed-off-by: Christoph Muellner <cmuellner@gcc.gnu.org>
git://sourceware.org / newlib-cygwin.git / log