Revisiting O_SEARCH and O_EXEC

[Rich Felker <dalias at aerifal dot cx>]

I had previously posed questions to the glibc community about how we
should define O_SEARCH and O_EXEC, which glibc presently lacks. At the
time, this was mostly a "future direction" thing, since Linux
seemingly lacked the capability to provide conforming behavior for
them. However, with my proposed solution to issue #14578, I believe
it's now possible to have a fully conforming implementation of these
modes, safe for use even on old kernels, that works as long as /proc
is mounted. See my notes on:

http://sourceware.org/bugzilla/show_bug.cgi?id=14578

The basic idea is to implement O_SEARCH and O_EXEC with O_PATH.
However, one detail is wrong at open time. O_PATH|O_NOFOLLOW means
"open a reference to the inode of the symlink itself", whereas, per
POSIX, O_SEARCH|O_NOFOLLOW means "fail if the pathname names a
symlink", and likewise for O_EXEC. Thus, merely defining O_SEARCH and
O_EXEC as O_PATH would require either non-conforming behavior for
O_NOFOLLOW, or that we artifically make open fail if it got a symlink
descriptor, in which case, the very useful O_PATH|O_NOFOLLOW semantics
would be unavailable to the application. Note that the POSIX semantics
really are necessary too: the usual meaning of O_NOFOLLOW is that the
application deems following a symlink would be dangerous.

For O_SEARCH, I believe simply chording O_DIRECTORY into it would
solve the problem. The Linux documentation suggests that O_DIRECTORY
is honored with O_PATH, and even if it's not, the open implementation
in libc could use it as a hint to reject symbolic links.

For O_EXEC (or for both), another option would be to chord with
O_EXCL, whose behavior is explicitly undefined if not used with
O_CREAT. This would require the library function open to mask off
O_EXCL before passing it to the kernel, and manually reject symbolic
links after open returns.

Another option, which I like better, is defining both O_EXEC and
O_SEARCH to 3. This makes them look more like access modes, as they
should, and avoids the need to change the definition of O_ACCMODE.
However, I think this option would require agreement from the kernel
folks, since it would be big trouble if they later wanted to use this
value for something else. On the plus side, using the value of 3 would
allow the kernel folks to explicitly support O_SEARCH and O_EXEC
semantics kernel-side in the future, without help from the userspace
library functions.

Opinions? I would like to coordinate this process between glibc and
musl so we can agree on common values for these modes, and I'm hoping
the process can take place sooner rather than later.

Rich