Cygwin: Allow accessing 48 bit address space in Windows 8.1 or later
64 bit Windows started out with a 44 bit address space due to a
restriction of the AMD64 CPUs at the time. Starting with Windows
8.1, these CPUs are not supported anymore and Windows switched to
the full 48 bit address space supported by AMD64.
Cygwin didn't follow suit yet so mmaps are still restricted to
the lower 44 bit address space. Fix that by using a system-specific
upper address for mmap allocations, 44 bit up to Windows 8, 48 bit
starting with Windows 8.1.
While at it, move the heap by another 8 Gigs to leave some space
for a potential extension of DLL address space, and restrict the
mmap lower address so the heap can grow to 32 Gigs before colliding
with mmaps.
Do not remember the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like cleaning up the process table and suppressing SIGCHILD delivery
with multiple threads ("waitproc") involved. Compared to that, the
potential slowdown due to an extra yield () call should be negligible.
Pause the child process after performing fork fixups even if there were
no dynamically loaded dlls with extra data/bss transfers to wait for.
This allows the parent process to cancel the current fork call even if
the child process was successfully initialized already.
This is a preparation for when the parent does remember the child no
earlier than after successful child initialization.
Cygwin: dll_list: no recursive use of nt_max_path_buf
Querying the ntlength and existence of the /var/run/cygfork directory in
the very first Cygwin process should not use nt_max_path_buf, as that
one is used by dll_list::alloc already.
NtQueryVirtualMemory for MemorySectionName may return some old path even
if the process was just started, for when some directory in between was
renamed - maybe because the NT file cache is hot for the old path still.
This was seen during gcc bootstrap, returning a MemorySectionName of
".../gcc/xgcc.exe" even if started as ".../prev-gcc/xgcc.exe", where the
directory rename from "gcc" to "prev-gcc" was done the moment before.
As we stat the module's real file right after loading now, there is no
point in using NtQueryVirtualMemory with MemorySectionName any more, and
we can use what GetModuleFileName returned instead.
Fix vfwscanf(3) assignment suppression flag handling bug
newlib's vfwscanf(3) (or specifically, __SVFWSCANF_R()) fails to correctly set
the assignment-suppressing character (`*') flag[1] which, when present in the
formatting string, results in undefined behaviour comprising retrieving and
dereferencing a pointer that was not supplied by the caller as such or at all.
When compared to the vfscanf(3) implementation, this would appear to be over
the missing goto match_failure statement preceded by the flags test seen below.
Hence, this patch (re)introduces it.
Jinke Fan [Fri, 17 May 2019 03:29:11 +0000 (11:29 +0800)]
Add support for Hygon Dhyana processor
-Add vendor identification
-Support in get_cpu_cache
Background:
Chengdu Haiguang IC Design Co., Ltd (Hygon) is a Joint Venture
between AMD and Haiguang Information Technology Co.,Ltd., aims at
providing high performance x86 processor for China server market.
Its first generation processor codename is Dhyana, which
originates from AMD technology and shares most of the
architecture with AMD's family 17h, but with different CPU Vendor
ID("HygonGenuine")/Family series number(Family 18h).
Sebastian Huber [Thu, 16 May 2019 12:40:34 +0000 (14:40 +0200)]
Fix <sys/_types.h> issues with <stddef.h>
A commit from 2016 tried to address this GCC provided <stddef.h> issue
#if (defined (__FreeBSD__) && (__FreeBSD__ >= 5)) \
|| defined(__DragonFly__) \
|| defined(__FreeBSD_kernel__)
/* __size_t is a typedef on FreeBSD 5, must not trash it. */
#elif defined (__VMS__)
/* __size_t is also a typedef on VMS. */
#else
#define __size_t
#endif
with an include of <stddef.h> before <sys/_types.h> in <sys/types.h>.
Is is not robust enough. Do the include of <stddef.h> in <sys/_types.h>
directly and request only the necessary types.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Sebastian Huber [Thu, 16 May 2019 12:40:32 +0000 (14:40 +0200)]
Avoid cyclic header dependencies
RTEMS uses a considerable part of FreeBSD kernel and user space sources.
These sources are compiled with a __FreeBSD__ define. On 2018-06-26
Gerald Pfeifer changed the GCC provided <stddef.h> so that it includes
<sys/_types.h> if __FreeBSD__ is defined. The Newlib <sys/_types.h>
included <sys/lock.h> which includes <sys/cdefs.h> on RTEMS which
includes <stddef.h>. To get rid of this cyclic dependency move the
optional _flock_t definition to <sys/reent.h>.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Jim Wilson [Thu, 23 May 2019 00:41:25 +0000 (17:41 -0700)]
RISC-V: Add _LITE_EXIT in crt0.S.
This patch adds _LITE_EXIT in crt0.S to enable "lite exit" technique in
RISC-V. The changes have been tested in riscv/riscv-gnu-toolchain by
riscv-dejagnu with riscv-sim.exp/riscv-sim-nano.exp.
Jim Wilson [Thu, 23 May 2019 00:36:57 +0000 (17:36 -0700)]
RISC-V: Add size optimized memcpy, memmove, memset and strcmp.
This patch adds implementations of memcpy, memmove, memset and strcmp
optimized for size. The changes have been tested in
riscv/riscv-gnu-toolchain by riscv-dejagnu with
riscv-sim.exp/riscv-sim-nano.exp.
Make .data section placement coincide with _fdata symbol
The _fdata symbol in MIPS linker scripts is aligned to a 16-byte
boundary. The ALIGN function does not implicitly update current
location counter. If sections positioned after the assignment
do not have the same natural alignment as the ALIGN function then
the start of the section group will not coincide with the value
of the symbol.
Given the linker command sequence:
symbol = ALIGN (NN);
(.section*)
where the idiom implies a desire to mark the beginning of .section
with symbol, there must be an assignment to the location counter
between the assignment to symbol and the .section pattern.
Ken Brown [Thu, 9 May 2019 16:35:58 +0000 (12:35 -0400)]
Cygwin: FIFO: code simplifications.
There's no longer a need to consider the connect_evt after fork/exec.
After stopping the listen_client thread, all client handlers should be
in the fc_connected or fc_invalid states, so their connect_evt members
won't be used again.
Also remove code in fhandler_fifo::dup that just repeats things
already done in stop_listen_client.
Ken Brown [Thu, 9 May 2019 16:28:21 +0000 (12:28 -0400)]
Cygwin: FIFO: improve the check for the listen_client thread
Add a method fhandler_fifo::check_listen_client_thread that checks
whether the thread is running. Use it in raw_read instead of just
testing the handle listen_client_thr.
Ken Brown [Thu, 9 May 2019 15:55:30 +0000 (11:55 -0400)]
Cygwin: FIFO: don't leave a pending listen request
On exit from the listen_client thread, make sure there's no pending
FSCTL_PIPE_LISTEN request. Otherwise we might get a client connection
after restarting the thread, and we won't have a handle for
communicating with that client.
Remove the retry loop in the case of STATUS_PIPE_LISTENING; that case
shouldn't occur.
Remove the now-unused fc_connecting value from
fifo_client_connect_state.
Ken Brown [Thu, 9 May 2019 15:23:44 +0000 (11:23 -0400)]
Cygwin: FIFO: re-implement duplexers
When opening a duplexer, open a client connection to the first client
handler. Previously we gave the duplexer a bogus write handle, which
was just a duplicate of the first client handler's handle. This meant
that we had a pipe server with no clients connected, and all I/O
attempts failed with STATUS_PIPE_LISTENING.
Extend the last fcntl change to duplexers.
Remove a now unused fifo_client_handler constructor, as well as the
long unusued method fifo_client_handler::connect.
Don't create the pipe in duplex mode; the server handle will only be
used for reading.
NtQueryVirtualMemory for MemorySectionName does not reliable return the
changed dll file name when another process does move the file around,
and we may end up creating forkable hardlinks to wrong dll files.
So query the file id when loading the dll rather than before fork.
Cygwin: dll_list: stat_real_file_once as dll method
Make stat_real_file_once a method of struct dll, to be more flexible on
where to use. Also, debug print memory section name queried for a dll.
This is a preparation to query the file id when loading the dll.
The compiler driver positions the linker script at the end of the linker
command-line, after crtend.o. As a result, any INPUT objects and archive
GROUPs introduced by the linker script are placed after crtend.o and the
end-of-frame marker provided by crtend.o ends up in between .eh_frames
instead of being at the end.
This has always been a problem, but a binutils update to clean-up
redundant NULL markers in .eh_frame exposes it as a execution failure in
exception-handling tests. This patch re-orders .eh_frames in all
MIPS linker scripts so that the one from crtend.o is always placed last.
Ken Brown [Thu, 25 Apr 2019 22:21:11 +0000 (18:21 -0400)]
Cygwin: FIFO: synchronize the blocking mode of a writer's pipe
The blocking mode of the Windows pipe underlying a writer is set to
match that of the writer itself when the latter is opened. Define
fhandler_fifo::fcntl to keep the pipe and the writer in sync if the
blocking mode is changed via fcntl.
Ken Brown [Sat, 20 Apr 2019 15:41:12 +0000 (11:41 -0400)]
Cygwin: FIFO: stop the listen_client thread before fork/exec
Add methods need_fixup_before, init_fixup_before, and
fixup_before_fork_exec to accomplish this. Stopping the thread makes
sure that the client handler lists of the parent and child remain in
sync while the forking/execing is in progress.
Ken Brown [Sat, 20 Apr 2019 15:49:35 +0000 (11:49 -0400)]
Cygwin: FIFO: duplicate the i/o handle when opening a duplexer
Don't use the same i/o handle for the first client handler as is used
for the fhandler itself; this can lead to a later attempt to close the
same handle twice. Instead use a duplicate.
Ken Brown [Thu, 18 Apr 2019 15:39:52 +0000 (15:39 +0000)]
Cygwin: FIFO: avoid hang after exec
Define fhandler:fifo::fixup_after_exec, which sets listen_client_thr
and lct_termination_evt to NULL. This forces the listen_client thread
to restart on the first attempt to read after an exec. Previously the
exec'd process could hang in fhandler_fifo::raw_read.
MSP430: Make the inclusion of run_*_array fns depend on defined assembler symbols
Many of the MSP430 crt functions (e.g. to initialize bss) are linked
"dynamically", based on symbols defined in the program.
The GNU assembler defines the symbols corresponding to the crt
functions by examining the section names in the input file.
If GCC has been configured with --enable-initfini-array, then
.init_array and .fini_array will hold pointers to global
constructors/destructors. These sections can also hold functions that
need to be executed for other purposes.
The attached patch puts the __crt0_run_{preinit,init,fini}_array and
__crt0_run_array functions in their own object files, so they will
only be linked when needed.
Successfully regtested the DejaGNU GCC testsuite using the binutils and
newlib changes together with GCC trunk configured with
--enable-initfini-array.
Ken Brown [Sun, 14 Apr 2019 19:16:04 +0000 (19:16 +0000)]
Cygwin: FIFO: fix and simplify listen_client_thread
Remove fifo_client_handler::connect and move its code into
listen_client_thread. That way we can check the return status when a
client handler's connect_evt is signaled. Previously we incorrectly
assumed there was a successful connection.
Also simplify listen_client_thread in the following ways:
- Replace fhandler_fifo::disconnect_and_reconnect by a new
delete_client_handler method. Now we just delete invalid client
handlers rather than trying to re-use them.
- Try to maintain a client handler list that consists of connected
client handlers and exactly one that is listening for a connection.
This allows us to call WaitForMultipleObjects with only two wait
objects.
- Remove 'dummy_evt' from the fifo_client_handler struct; it is no
longer needed.
- On exit from listen_client_thread, delete the "extra" (listening)
client handler. Otherwise there could be a connection that doesn't
get recorded in the client handler list. This could happen when a
file descriptor is being duplicated.
Ken Brown [Sun, 14 Apr 2019 19:16:02 +0000 (19:16 +0000)]
Cygwin: FIFO: fix clone
After copyto is called, make the new fhandler's pipe_name point to the
new fhandler's pipe_name_buf, which is a *copy* of the old fhandler's
pipe_name_buf. Previously, get_pipe_name would return the wrong
result after a clone/dup, causing create_pipe_instance and open_pipe
to fail.
Also, stop the listen_client thread when cloning. Otherwise the
thread can keep accepting connections that the cloned fhandler won't
know about.
Do this via a new method fhandler_fifo::stop_listen_client, extracted
from fhandler_fifo::close.
Ken Brown [Sun, 14 Apr 2019 19:16:01 +0000 (19:16 +0000)]
Cygwin: FIFO: fix the use of the read_ready event
Make read_ready a manual reset event. It should always be set shortly
after startup of the listen_client thread and remain set until the
thread terminates. (We don't want writers to connect without being
recorded in the client handler list.)
Remove the unnecessary code that checks for read_ready when a reader
is opening.
Ken Brown [Sun, 14 Apr 2019 19:15:57 +0000 (19:15 +0000)]
Cygwin: FIFO: remember the type of the fhandler
Add data members 'reader', 'writer', and 'duplexer' to the
fhandler_fifo class. Set them in fhandler_fifo::open. ('duplexer'
replaces the previous '_duplexer'.)
Ken Brown [Sun, 14 Apr 2019 19:15:56 +0000 (19:15 +0000)]
Cygwin: FIFO: rename client[] to fc_handler[]
The word "client" suggests something that holds a handle to the client
side of the pipe (in Windows terminology). But our
fifo_client_handlers hold a handle the server side of the pipe, and
they *connect* to clients.
Implement reduced code size "tiny" printf and puts
"tiny" printf is derived from _vfprintf_r in libc/stdio/nano-vfprintf.c.
"tiny" puts has been implemented so that it just calls write, without
any other processing.
Support for buffering, reentrancy and streams has been removed from
these functions to achieve reduced code size.
This reduced code size implementation of printf and puts can be enabled
in an application by passing "--wrap printf" and "--wrap puts" to the
GNU linker. This will replace references to "printf" and "puts" in user
code with "__wrap_printf" and "__wrap_puts" respectively.
If there is no implementation of these __wrap* functions in user code,
these "tiny" printf and puts implementations will be linked into the
final executable.
The wrapping mechanism is supposed to be invisible to the user:
- A GCC wrapper option such as "-mtiny-printf" will be added to alias
these wrap commands.
- If the user is unaware of the "tiny" implementation, and chooses to
implement their own __wrap_printf and __wrap_puts, their own
implementation will be automatically chosen over the "tiny" printf and
puts from the library.
Newlib must be configured with --enable-newlib-nano-formatted-io for
the "tiny" printf and puts functions to be built into the library.
Code size reduction examples:
printf("Hello World\n")
baseline - msp430-elf-gcc gcc-8_3_0-release
text data bss
5638 214 26
"tiny" puts enabled
text data bss
714 90 20
printf("Hello %d\n", a)
baseline - msp430-elf-gcc gcc-8_3_0-release
text data bss
10916 614 28
Cygwin: use win pid+threadid for forkables dirname
Rather than newest last write time of all dlls loaded, use the forking
process' windows pid and windows thread id as directory name to create
the forkable hardlinks into. While this may create hardlinks more
often, it does avoid conflicts between dlls not having the newest last
write time.
Commit 69f4c4029183fb26d2fcae00790881620c1978a3 improved most
macro checks to be ARMv8-M baseline proof, but missed a few
occurrences which otherwise fail to build when using a CPU setting
such as cortex-m0 or cortex-m23. This patch brings the same
changes as the ones that were committed to libgloss at that time.
newlib:
* libc/sys/arm/crt0.S: Use THUMB1_ONLY rather than
__ARM_ARCH_6M__.
Erik M. Bray [Wed, 10 Apr 2019 15:05:22 +0000 (17:05 +0200)]
Improve error handling in /proc/[pid]/ virtual files.
* Changes error handling to allow /proc/[pid]/ virtual files to be
empty in some cases (in this case the file's formatter should return
-1 upon error, not 0).
* Better error handling of /proc/[pid]/stat for zombie processes:
previously trying to open this file on zombie processes resulted
in an EINVAL being returned by open(). Now the file can be read,
and fields that can no longer be read are just zeroed.
* Similarly for /proc/[pid]/statm for zombie processes.
* Similarly for /proc/[pid]/maps for zombie processes (in this case the
file can be read but is zero-length, which is consistent with observed
behavior on Linux.
Cygwin: console: fix key input for native console application
- After 24 bit color support patch, arrow keys and function keys
do not work properly in native console applications if they
are started in cygwin console. This patch fixes this issue.
Cygwin: signals: restore sigmask from context given to signal handler
In case SA_SIGINFO flag is given, the signal handler may change
the context and the application is supposed to pick up from the
changed context. So far we don't do that, so the context given
to the signal handler is basically read-only, unless the signal
handler calls setcontext or swapcontext.
For a start, restore the thread's signal mask from the uc_sigmask
value of the context given to the signal handler.
If that's feasible for Cygwin, we restore the entire context from
the context changed by the signal handler in a followup patch.
The location of the handler at offset 0x20 from the start of memory,
immediately after the 32-byte reset vector, matches the expectations
of real hardware (e.g., a 3c120 board).
Cygwin: winpids: Fix getting process multiple times, take 2
commit d1be0a59d48222d8ea6261ee3e59de2bc3d149e4,
"Cygwin: winpids: Fix getting process multiple times"
fixed duplicate processes in ps -W output, but it fixed
the symptom, not the cause. It also didn't fix the problem
that the `ps' process itself may show up twice in its own
output.
This patch fixes it. The spawn worker only deleted the
"winpid.PID" symlink of the current process if the child is
a non-Cygwin process, under the assumption that the exec'ing
process exits anyway. However, the Window in which both
winpid.PID symlinks point to the same cygpid.PID area is just
too long. The spawn worker now also deletes its own winpid.PID
symlink if the exec'ed process is a Cygwin process.
Takashi Yano [Sun, 31 Mar 2019 15:47:46 +0000 (00:47 +0900)]
Cygwin: console: support 24 bit color
- Add 24 bit color support using xterm compatibility mode in
Windows 10 1703 or later.
- Add fake 24 bit color support for legacy console, which uses
the nearest color from 16 system colors.
In dll_crt0_0, both threadinterface->Init and sigproc_init allocate
windows object handles using unpredictable memory regions, which may
collide with dynamically loaded dlls when they were relocated.
Ken Brown [Wed, 27 Mar 2019 21:29:21 +0000 (21:29 +0000)]
Cygwin: FIFO: implement clear_readahead
Make fhandler_base::clear_readahead virtual, and implement
fhandler_fifo::clear_readahead. This is called by
dtable::fixup_after_exec; it clears the readahead in each client.
Cygwin: fix: seteuid32() must return EPERM if privileges are not held.
Starting w/ the intro of S4U, seteuid32() calls lsaprivkeyauth(), then
s4uauth(). s4uauth calls LsaRegisterLogonProcess().
LsaRegisterLogonProcess fails w/ STATUS_PORT_CONNECTION_REFUSED, if the
proper privileges are not held.
Because of RtlNtStatusToDosError(), this status would be mapped to
ERROR_ACCESS_DENIED, which in turn would map to EACCES. Therefore it is
useless to add this status to errmap[] (errno.cc), as s4auauth() should
return EPERM as errno here (i.e. if process is not privileged).
Hence the kludge.
Before the intro of S4U, seteuid32() called lsaprivkeyauth(), then
lsaauth(), then create_token(). Before the intro of Vista, the latter
would have called NtCreateToken().
NtCreateToken() would have failed w/ STATUS_PRIVILEGE_NOT_HELD for a
process w/o the proper privileges. In that case, calling seteuid32()
would have returned EPERM (as required).
Since the intro of Vista, and if the process had been started from an
UNelevated shell, create_token() does NOT reach NtCreateToken()!
As create_token() failed to properly set errno in that case, calling
seteuid32() would return errno as set by lsaauth(), i.e. EACCES, not
in agreement w/ Posix (a bug which was present for years).
(lsaauth() called LsaRegisterLogonProcess() which would fail)
Ken Brown [Mon, 25 Mar 2019 23:06:10 +0000 (23:06 +0000)]
Cygwin: FIFO: add support for the duplex case
If a FIFO is opened with O_RDWR access, create the pipe with
read/write access, and make the first client have the handle of that
pipe as its I/O handle.
Adjust fhandler_fifo::raw_read to account for the result of trying to
read from that client if there's no data.
Ken Brown [Mon, 25 Mar 2019 23:06:09 +0000 (23:06 +0000)]
Cygwin: FIFO: avoid crashes when cloning a client
fhandler_fifo::clone called fhandler_base::clone on each client
fhandler. But those fhandlers are actually fhandler_fifo objects, so
when fhandler_base::clone calls copyto, it's actually
fhandler_fifo::copyto that gets called. This can lead to mysterious
crashes.
Fix this by simply calling clone (which translates to
fhandler_fifo::clone) on each client fhandler.
Ken Brown [Fri, 22 Mar 2019 19:30:41 +0000 (19:30 +0000)]
Cygwin: FIFO: update select
Add static functions peek_fifo, thread_fifo, start_thread_fifo, and
fifo_cleanup to select.cc. These are based on the corresponding pipe
functions, the main difference being that peek_fifo loops through the
connected clients to see if any of them have data available for
reading.
Add the fhandler_fifo methods select_read, select_write, and
select_except.
Add accessor methods get_nclients, get_handle, and is_connected that
are needed by peek_fifo.
Ken Brown [Fri, 22 Mar 2019 19:30:38 +0000 (19:30 +0000)]
Cygwin: FIFO: improve EOF detection
Add a hit_eof method that tries to detect whether any clients are
connected. Before concluding that there are none, it gives the
listen_client thread time to update the client data.
Ken Brown [Fri, 22 Mar 2019 19:30:37 +0000 (19:30 +0000)]
Cygwin: FIFO: allow multiple writers
Introduce a 'fifo_client_handler' structure that can be used by a
reader to communicate with a writer using an instance of the named
pipe. An fhandler_fifo opened for reading creates a thread that does
the following:
- maintains a list of fifo_client_handlers
- listens for_clients trying to connect
- creates new pipe instances as needed so that there's always at
least one available for connecting.
The pipe instances are initially created in blocking mode, but they
are set to be non-blocking after a connection is made.
fhandler_fifo::raw_read now loops through the connected clients and
reads from the first one that has data available.
New fhandler_fifo methods: add_client, listen_client,
listen_client_thread, check_listen_client_thread.
Replace the create_pipe method by create_pipe_instance, which allows
unlimited pipe instances.
New helper functions: create_event, set_pipe_non_blocking.
Ken Brown [Fri, 22 Mar 2019 19:30:36 +0000 (19:30 +0000)]
Cygwin: FIFO: stop using overlapped I/O
Make fhandler_fifo a derived class of fhandler_base instead of
fhandler_base_overlapped.
Replace the create_pipe macro, which is based on
fhandler_pipe::create, by new create_pipe and open_pipe methods.
These use NT functions instead of Win32 functions. Replace fifo_name
by get_pipe_name, which returns a pointer to a UNICODE_STRING.
Remove the fnevent macro, which would now be needed only once.
Add a raw_write method, adapted from fhandler_base::raw_write.
Corinna Vinschen [Wed, 27 Mar 2019 12:53:32 +0000 (13:53 +0100)]
Cygwin: winpids: Fix getting process multiple times
Switching to Cywin-only PIDs introduced a new problem when collecting
Cygwin processes for `ps -W': A process can show up multiple times
again, if the Cygwin procinfo has been opened for a just execing
process. The execed process then shows up twice, once as Cygwin
process, but with the wrong Windows PID of the execing process,
once as Windows-only process.
The mechanism used to exclude these stray processes didn't work with
the new Cygwin pid handling anymore. To fix this
* check if the incoming Windows PID is the same as the PID in the
procinfo. If not, we have the PID of the execing process while
procinfo was already changed,
* always check if the process has already been handled, not only
for processes we got a procinfo for,
* simplify adding pid to pidlist since pid is now always correct.
Andrew Stubbs [Mon, 25 Mar 2019 15:33:53 +0000 (15:33 +0000)]
Add missing includes.
These missing includes were causing build warnings, but also a real bug in
which the "size" parameter to "write" was being passed in 32-bit, whereas it
ought to be 64-bit. This led to intermittent bad behaviour.
git://sourceware.org / newlib-cygwin.git / log