Re: system and popen fail in case of big application

[Zack Weinberg <zackw at panix dot com>]

On Thu, Sep 13, 2018 at 8:11 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
> On 12/09/18 17:29, Zack Weinberg wrote:
> > I don't want to encourage people to use posix_spawn because
> > posix_spawn is a badly designed API.  It's difficult to use correctly.
> > It's *tedious* to use correctly, which means people won't bother.  It
> > can't do everything you might want to do on the child side (witness
> > the discussion of adding an extension to let it do chdir()).  Its
> > behavior in case of errors is underspecified.  And it might be
>
> the misdesigned api is vfork: it is not just difficult to use
> correctly but impossible: it breaks c language semantics
> (gcc fixes this up to some extent when it sees the actual vfork
> symbol in the code, but there is no guarantee in c that the
> compiler can see that, and even then the shared stack between
> parent and child has various issues as Adhemerval described)
[etc]

The way I see it is that that is all stuff that could be fixed.  I
looked at what Adhemerval did in s/u/s/linux/spawni.c and it appears
that generalizing it to `pid_t nvfork(void (*)(void *))` would solve
_all_ of the problems you list and even be safe to expose to
applications.  (Name provisional.)

On the other hand, in my opinion posix_spawn is an irretrievably
flawed design, because of its underspecification, because of its
license to lose error details, and above all, because it _doesn't_
allow you to perform arbitrary computation on the child side, which
means we will forever be playing catch-up with the list of "actions".
I want to see it deprecated and removed from POSIX, and that means
coming up with a genuinely better way to do what it does.  nvfork()
seems like a step in the right direction.

(I've actually had an even better plan in mind for years now,
involving a new "embryo" process state, but someone would have to pay
me to argue with Linus, or else take over that part of the project
themselves.)

> vfork can be implemented in terms of fork too, memcpy can be
> implemented using syscalls, these are QoI issues

I don't see it that way.  The kind of underspecification that both
vfork and posix_spawn suffer is the same kind of underspecification
that renders it unwise to use signal(), and continuing the analogy,
the virtue of sigaction() is that it _does_ specify all of the
behavioral details like whether or not signal delivery will cause
blocking system calls to fail and set errno to EINTR.

If we were going to add the hypothetical nvfork(), I would write into
its documentation that it is _guaranteed_ to have vfork-like behavior
(that is, it calls the supplied procedure in the memory space of the
parent process; writes through the closure parameter are visible to
the caller, as are any changes to errno; control does not return to
the caller until the procedure finishes) and it will fail and set
errno to ENOSYS if the OS doesn't make that possible.

zw