Re: setrlimit change to prlimit change in behavior?

[Mark Wielaard <mark at klomp dot org>]

On Wed, 2017-10-18 at 16:13 +0200, Andreas Schwab wrote:
> On Okt 18 2017, Mark Wielaard <mark@klomp.org> wrote:
> 
> > The valgrind testsuite has a testcase that checks that
> >  setrlimit (RLIMIT_NOFILE, NULL)
> > returns failure and sets errno to EFAULT.
> 
> In general, you cannot count on EFAULT.  The call has undefined
> behaviour, thus unbounded effect (a crash would be ok too).

Interesting. I didn't know undefined behavior extended to the Posix
functions. So does this hold for all functions defined by Posix that
have pointer arguments that are outside the accessible address space?
It seems a bit harsh for NULL arguments and at least the setrlimit man
page sounds like it is also describing the glibc implementation, not
just the direct linux setrlimit system call.

>   If you want
> to check the effect of a syscall you have to call it directly, not
> via a libc function.

Sure, the valgrind test actually is about the underlying system calls.
We will probably change the specific test to directly call the system
calls. But the observed behavior for the glibc setrlimit is what
changed in this case. It is not only the errno value that changed, but
also that the call now succeeds, while before it explicitly failed. But
I assume your argument is that it is undefined behavior so even though
the documentation said it would fail that was just by accident?

> > The man page http://man7.org/linux/man-pages/man2/prlimit.2.html
> > doesn't really make clear what happens if both old_limit and
> > new_limit
> > are NULL. But apparently the kernel interprets that as a NOP.
> 
> Similar to sigaction, if an operand is NULL then the respective
> operation is not performed, which implies that if both are NULL there
> is nothing to do.

OK, it would be nice to get that explicitly documented. Especially if
this implies none of the access/permission checks are done.

For valgrind the question is whether or not we want to warn about this.
If passing NULL might be explicitly done then maybe the user doesn't
want to be warned about the fact that they provided a NULL pointer to
setrlimit (or two NULL pointers to prlimit).

Cheers,

Mark