Ken Brown [Mon, 17 May 2021 21:18:45 +0000 (17:18 -0400)]
Cygwin: AF_UNIX: keep a reference count of open descriptors
Add a data member _ndesc to the af_unix_shmem_t class, along with
methods to increment and decrement it.
Increment it during socket/fork/exec/dup, and decrement it during
close. When the last descriptor is closed, call shutdown (SHUT_RDWR)
and unlink the socket's message queue.
The shutdown call will make the socket's peer see EPIPE if it tries to
write.
Ken Brown [Sun, 9 May 2021 22:15:56 +0000 (18:15 -0400)]
Cygwin: AF_UNIX: adapt connect and accept to mqueues
- Flesh out connect_mqueue, which replaces connect_pipe.
- A new method wait_mqueue replaces wait_pipe.
- A new method wait_mqueue_thread replaces wait_pipe_thread.
- New methods send_mqueue_name and recv_peer_mqueue_name are used for
making connection requests and responding to them.
- A new method xchg_sock_info is called after a connecting socket
successfully receives its peer's mqueue name. This functionality
was previously in open_pipe.
- Remove open_pipe, listen_pipe, and create_pipe_instance, which are
no longer needed.
Ken Brown [Fri, 7 May 2021 18:19:20 +0000 (14:19 -0400)]
Cygwin: AF_UNIX: add create_mqueue to replace create_pipe
Also add new data members mqd_in and mqd_out, along with getter and
setter methods. mqd_in and mqd_out are mqueue descriptors for
input/output. create_mqueue creates an mqueue for reading and sets
mqd_in to its descriptor.
Make other provisional changes so that compilation will succeed.
Ken Brown [Wed, 5 May 2021 20:50:18 +0000 (16:50 -0400)]
Cygwin: AF_UNIX: adapt symlink and reparse point to mqueues
create_symlink, create_reparse_point, and create_file now take a char
*mqueue_name argument instead of a PUNICODE_STRING. For
create_symlink, convert the string to a UNICODE_STRING and make that
the target of the symlink. For create_reparse_point, store the string
in the reparse point as is.
Make other provisional changes so that compilation will succeed. This
includes adding a methode connect_mqueue that does nothing for now.
Ken Brown [Wed, 5 May 2021 19:45:12 +0000 (15:45 -0400)]
Cygwin: AF_UNIX: replace gen_pipe_name by gen_mqueue_name
The generated name is now an ordinary string instead of a UNICODE_STRING.
Store it in pc.path. For clarity, add methods set_mqueue_name and
get_mqueue name to do the storing and retrieving. Add a method
path_conv::get_path to help with this.
Ken Brown [Wed, 5 May 2021 18:45:13 +0000 (14:45 -0400)]
Cygwin: AF_UNIX: prepare for use of message queues
We will switch to using POSIX message queues instead of Windows pipes
for data transport. In preparation for that, update the commentary in
fhandler_socket_unix.cc.
Also move the definition of class af_unix_pkt_hdr_t to fhandler.h and
make a few minor changes.
Ken Brown [Wed, 12 May 2021 21:59:34 +0000 (17:59 -0400)]
Cygwin: AF_UNIX: add saw_shutdown_read and saw_shutdown_write
For consistency between the fhandler_socket_unix and
fhandler_socket_wsock classes, add virtual methods saw_shutdown_read
and saw_shutdown_write to fhandler_socket. Implement them for
fhandler_socket_unix. (They were already implemented for
fhandler_socket_wsock.)
Ken Brown [Thu, 22 Oct 2020 18:55:52 +0000 (14:55 -0400)]
Cygwin: AF_UNIX: reopen_shmem: fix view size
Use sizeof (af_unix_shmem_t) as the view size, as when the shared
memory was created. Previously PAGESIZE was used, causing
NtMapViewOfSection to fail with STATUS_INVALID_VIEW_SIZE.
Richard Earnshaw [Wed, 26 May 2021 14:17:11 +0000 (15:17 +0100)]
aarch64: support binary mode for opening files
Newlib for aarch64 uses libgloss for the backend. One common libgloss
implementation is the 'rdimon' implementation, which uses the Arm
Semihosting protocol. In order to support a remote host that runs on
Windows we need to know whether a file is to be opened in binary or
text mode. That means that we need to preserve this information via
O_BINARY until we know what the libgloss binding will be.
This patch simply copies the arm implementation from sys/arm/sys and
puts it in machine/aarch64/sys, because we don't have a 'sys' subtree
on aarch64.
Corinna Vinschen [Tue, 25 May 2021 20:03:36 +0000 (22:03 +0200)]
Cygwin: path_conv: fix mqueue path check
The check for a file or dir within /dev/mqueue is accidentally using
the incoming path, which could be a relative path. Make sure to
restore the absolute POSIX path in path_copy and only then test the
path.
Also, move the actual check for a valid path below /dev/mqueue into
the fhandler_mqueue class.
Corinna Vinschen [Tue, 25 May 2021 19:46:13 +0000 (21:46 +0200)]
Cygwin: cwdstuff: check if /dev exists
/dev has been handled as virtual dir in cwdstuff, thus not allowing
to start native apps from /dev as CWD, even if /dev actually exists
on disk. Unfortunately this also affects Cygwin executables started
from a debugger.
When chdir'ing to /dev, check if /dev exists on disk. If so, treat
it as any other existing path.
reuse fhandler_virtual implementation to implement read and lseek.
The output from read(2) is modelled after the output when reading
from an mq file on Linux.
Corinna Vinschen [Tue, 25 May 2021 18:15:16 +0000 (20:15 +0200)]
Cygwin: POSIX msg queues: move all mq_* functionality into fhandler_mqueue
The POSIX entry points are just wrappers now, calling into
fhandler_mqueue. While at it, eliminate mqi_flags, replace with
standard fhandler nonblocking flag.
Corinna Vinschen [Tue, 25 May 2021 14:49:04 +0000 (16:49 +0200)]
Cygwin: POSIX msg queues: implement open/mq_open entirely in fhandler
The mq_open call is just a framework now. This patch moves the
entire functionality into fhandler_mqueue. To support standard
OS calls (as on Linux), make fhandler_mqueue a derived class from
fhandler_disk_file and keep the base handle the handle to the
default stream, to allow implementing O_PATH functionlaity as well
as reading from the file and NOT reading binary message queue data.
Implement a standard fhandler_mqueue::open method, allowing, for
instance, to touch a file under /dev/mqueue and if it doesn't exist,
to create a message queue file.
FIXME: This introduces a BAD HACK into path_conv::check, which needs
reviewing.
Keep the posix path intact in the fhandler, and change get_proc_fd_name
accordingly to return only the basename plus leading slash for
/proc/<PID>/fd.
Corinna Vinschen [Tue, 25 May 2021 14:24:58 +0000 (16:24 +0200)]
Cygwin: get_nt_native_path: allow to append suffix
POSIX message queues will be moved into NTFS streams.
Extend get_nt_native_path to provide a filename suffix which is not
subject to special character transposition, to allow specifying
a colon.
Corinna Vinschen [Fri, 21 May 2021 13:21:29 +0000 (15:21 +0200)]
Cygwin: POSIX ipc: fix evaluation of naming rules
The function evaluating correctness of ipc object names was a
bit half-hearted. Fix the tests to follow more closely the
desriptions in the Linux man pages.
Corinna Vinschen [Fri, 21 May 2021 10:01:56 +0000 (12:01 +0200)]
Cygwin: POSIX msg queues: Convert mqd_t to a descriptor
So far, the mqd_t type returned a pointer to an allocated
area under the hood. The mutex and event objects attached
to the message queue were implemented as inheritable types.
As unfortunate side effect the HANDLEs to these objects
were inherited by exec'd child processes, even though all
other message queue properties are not inherted, per POSIX.
Fix this by converting an mqd_t to a descriptor, and create a
matching fhandler_mqueue object to handle various aspects of
the message queues inside the fhandler. Especially, create the
IPC objects as non-inheritable and duplicate the HANDLEs as
part of the fixup_after_fork mechanism.
Drop using mmap and create the memory map with NT functions.
This allows to control duplication of handle and mapping in the
forked child process, without the requirement to regenerate the
map in the same spot. It also allows to dup() the descriptor,
as on Linux, albeit this isn't implemented yet.
This patch is the first cut. There's a bit more to do, like
moving more functionality from the POSIX functions into the
fhandler and making sure the mqd_t type can't be used in other
descriptor-related functions willy-nilly.
Christian Franke [Thu, 20 May 2021 09:05:29 +0000 (11:05 +0200)]
Cygwin: utils: chattr: Improve option parsing, fix some messages.
Allow multiple characters also in first '-mode' argument.
Use '-H' instead of '-h' for '--help' to fix ambiguity with
hidden attribute. Fix help and usage texts and documentation.
Signed-off-by: Christian Franke <christian.franke@t-online.de>
Jeremy Drake [Wed, 19 May 2021 09:43:48 +0000 (11:43 +0200)]
Cygwin: suppress FAST_CWD warnings on ARM64
The old check was insufficient: new insider preview builds of Windows
allow running x86_64 process on ARM64. The IsWow64Process2 function
seems to be the intended way to figure this situation out.
Jon Turney [Tue, 27 Apr 2021 22:41:07 +0000 (23:41 +0100)]
Move source files used in utils/mingw/ into that subdirectory
Move all the source files used in utils/mingw/ into that subdirectory,
so the built objects are in the expected place.
(path.cc requires some more unpicking, and even then there is genuinely
some shared code, so use a trivial file which includes the real path.cc
so the object file is generated where expected)
Jon Turney [Wed, 28 Apr 2021 13:54:47 +0000 (14:54 +0100)]
Unpick cygpath TESTSUITE
Rather than having testsuite.h do various things, depending on defines,
just have it do one thing, and then explicitly redirect to test stubs in
path.cc when building test.
Cygwin: fetch Windows directory on all platforms and use throughout
Rather than fetching the system Windows directory at dll init time
only on 32 bit, fetch it on all platforms. Store as WCHAR and
UNICODE_STRING. Use where appropriate to simplify code.
Cygwin: path_conv: Rework handling native symlinks as inner path components
commit 456c3a46386f was only going half-way. It handled symlinks and
junction points as inner path components and made realpath return the
correct path, but it ignored drive letter substitution, i. e., virtual
drives created with, e. g.
subst X: C:\foo\bar
It was also too simple. Just returning an error code from
symlink_info::check puts an unnecessary onus on the symlink evaluation
loop in path_conv::check.
Rework the code to use GetFinalPathNameByHandle, and only do this after
checking the current file for being a symlink failed.
If the final path returned by GetFinalPathNameByHandle is not the same
as the incoming path, replace the incoming path with the POSIXified
final path. This also short-circuits path evaluation, because
path_conv::check doesn't have to recurse over the inner path components
multiple times if all symlinks are of a native type, while still getting
the final path as end result.
Virtual drives are now handled like symlinks. This is a necessary change
from before to make sure virtual drives are handled identically across
different access methods. An example is realpath(1) from coreutils. It
doesn't call readlink(2), but iterates over all path components using
lstat/readlink calls. Both methods should result in the same real path.
Fixes: 456c3a46386f ("path_conv: Try to handle native symlinks more sanely") Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Ola Olsson [Mon, 26 Apr 2021 08:57:02 +0000 (10:57 +0200)]
Nano-malloc: Fix for unwanted external heap fragmentation
The only reason why it is tough for us to use nano malloc
is because of the small shortcoming where nano_malloc()
splits a bigger chunk from the free list into two pieces
while handing back the second one (the tail) to the user.
This is error prone and especially bad for smaller heaps,
where nano malloc is supposed to be superior. The normal
malloc doesn't have this issue and we need to use it even
though it costs us ~2k bytes compared to nano-malloc.
The problem arise especially after giving back _every_
malloced memory to the heap and then starting to exercise
the heap again by allocating something small. This small
item might split the whole heap in two equally big parts
depending on how the heap has been exercised before.
I have uploaded the smallest possible application
(only tested on ST and Nordic devices) to show the issue
while the real customer applications are far more complicated:
https://drive.google.com/file/d/1kfSC2KOm3Os3mI7EBd-U0j63qVs8xMbt/view?usp=sharing
The application works like the following pseudo code,
where we assume a heap of 100 bytes
(I haven't taken padding and other nitty and gritty
details into account. Everything to simplify understanding):
void *ptr = malloc(52); // We get 52 bytes and we have
// 48 bytes to use.
free(ptr); // Hand back the 52 bytes to nano_malloc
// This is the magic line that shows the issue of
// nano_malloc
ptr = malloc(1); // Nano malloc will split the 52 bytes
// in the free list and hand you a pointer
// somewhere in the
// middle of the heap.
ptr2 = malloc(52); // Out of memory...
I have done a fix which hands back the first part of the
splitted chunk. Once this is fixed we obviously
have the 1 byte placed in position 0 of the heap instead
of somewhere in the middle.
However, this won't let us malloc 52 new bytes even though
we potentially have 99 bytes left to use in the heap. The
reason is that when we try to do the allocation,
nano-malloc looks into the free list and sees a 51 byte
chunk to be used.
This is not big enough so nano-malloc decides to call
sbrk for _another_ 52 bytes which is not possible since
there is only 48 bytes left to ask for.
The solution for this problem is to check if the last
item in the free list is adjacent to sbrk(0). If it is,
as it is in this case, we can just ask sbrk for the
remainder of what is needed. In this case 1 byte.
NB! I have only tested the solution on our ST device.
Brian Inglis [Fri, 30 Apr 2021 13:19:20 +0000 (07:19 -0600)]
format_proc_swaps: ensure space between fields for clarity
page/swap space name >= 40 or size/used >= 8 leaves no space between fields;
ensure a space after name and add extra tabs after size and used fields;
output appears like Linux 5.8 after changes to mm/swapfile(swap_show);
proc-swaps-space-before.log:
==> /proc/swaps <==
Filename Type Size Used Priority
/mnt/c/pagefile.sys file 11567748292920 0
/mnt/d/pagefile.sys file 12582912205960 0
proc-swaps-space-after.log:
==> /proc/swaps <==
Filename Type Size Used Priority
/mnt/c/pagefile.sys file 11567748 241024 0
/mnt/d/pagefile.sys file 12582912 182928 0
Jon Turney [Tue, 27 Oct 2020 16:29:45 +0000 (16:29 +0000)]
Cygwin: CI configuration update
Install autoconf and automake, and run winsup/autogen.sh, and don't have
it silently ignore failures.
On AppVeyor:
- use latest VM image, to reduce time spent installing updates.
- run the testsuite, but ignore the result, as some tests don't work
correctly.
- hardcode the python-lxml and python-ply packages to install, so we get
ones for the right python.
- install texlive collections now needed to build documentation.
On github:
- Use a copr for cocom, since RPMSphere's package updates don't track
fedora:latest very efficently.
Cygwin: POSIX mq: avoid double calls to ipc_mutex_unlock
_mq_send as well as _mq_receive call ipc_mutex_unlock twice in case
of success, after having introduced __try/__except blocks.
Fixes: 3f3bd10104550 ("* Throughout, use __try/__except/__endtry blocks, rather than myfault handler.") Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Cygwin: POSIX mq: only allo priority up to MQ_PRIO_MAX - 1
POSIX requires that message priorities range from 0 (low) to
sysconf(_SC_MQ_PRIO_MAX) - 1 (high). Cygwin's mq_send erroneously
allowed a message priority of sysconf(_SC_MQ_PRIO_MAX). Fix it.
Jon Turney [Wed, 28 Apr 2021 09:46:02 +0000 (11:46 +0200)]
Cygwin: specify toollibdir toolincludedir in terms of tooldir
Specify toollibdir and toolincludedir in terms of tooldir, so the
install location is correct if tooldir is the default
($(exec_prefix)/$(target_alias)), or explicitly specified on the 'make'
command line.
Linux 5.12 Frozen Wasteland added features and changes:
add AMD 0x8000000a EDX:20 v_spec_ctrl virtual speculation control support
add Intel 0x00000007 ECX:24 bus_lock_detect bus lock detect debug exception
Jon Turney [Tue, 6 Oct 2020 15:11:54 +0000 (16:11 +0100)]
Cygwin: Use automake (v5)
v2:
* Include tzmap.h in BUILT_SOURCES
* Make per-file flags appear after user-supplied CXXFLAGS, so they can
override optimization level.
* Correct .o files used to define symbols exported by libm.a
* Drop gcrt0.o mistakenly included in libgmon.a
* Add missing line continuations in GMON_FILES value
v3:
* use per-file flags for .c compilation
* override C{XX,}FLAGS, as they are set on the command line by top-level make
v4:
* Drop -Wno-error=write-strings from path_testsuite CXXFLAGS
v5:
* Update for changes in master
- Add -fno-threadsafe-statics to CXX flags
- Add hypotl.cc
- Remove fenv.cc (in favour of newlib), add fenv.c stub
- Add proc.5 manpage rules
Ken Brown [Fri, 23 Apr 2021 22:17:57 +0000 (18:17 -0400)]
Cygwin: connect: implement resetting a connected DGRAM socket
Following POSIX and Linux, allow a connected DGRAM socket's connection
to be reset (so that the socket becomes unconnected). This is done by
calling connect and specifing an address whose family is AF_UNSPEC.
Ken Brown [Fri, 23 Apr 2021 18:40:07 +0000 (14:40 -0400)]
Cygwin: connect: set connect state for DGRAM sockets
When connect is called on a DGRAM socket, the call to Winsock's
connect can immediately return successfully rather than failing with
WSAEWOULDBLOCK. Set the connect state to "connected" in this case.
Previously the connect state remained "connect_pending" after the
successful connection.
- In commit bb93c6d7, the race issue was not completely fixed. In
the pseudo console inheritance, if the destination process to
which the ownership of pseudo console switches, is found but exits
before switching, the inheritance fails. Currently, this extremely
rarely happens. This patch fixes the issue.
git://sourceware.org / newlib-cygwin.git / log