Re: system and popen fail in case of big application

[Zack Weinberg <zackw at panix dot com>]

On Tue, Sep 11, 2018 at 4:16 PM >
Adhemerval Zanella <adhemerval.zanella@linaro.org> wrote:
> On 09/09/2018 19:44, Zack Weinberg wrote:
> > On Sun, Sep 9, 2018 at 6:16 PM Sergey Melnikov
> > <melnikov.sergey.v@gmail.com> wrote:
> >> So, my question is if does it worth to rewrite existing popen/system
> >> calls with posix_spawn as StackOverflow recommends? If so, I may
> >> implement and contribute this to glibc.
> >
> > What would be worthwhile is to change popen and system so that they
> > internally use *vfork*, on systems where vfork is not the same as
> > fork. posix_spawn is just an elaborate and inconvenient wrapper around
> > vfork, and I don't think we should be using it internally or
> > encouraging its use by others.  Also, unlike vfork, it does not
> > guarantee to avoid the problem you are experiencing.
>
> We changed posix_spawn to exactly avoid call vfork due its inherent usage
> issue with signal handlers (BZ#14750), cancellation state being shared
> with the parent (BZ#10354), and the vfork limitation of possible parent
> clobber due stack spilling (which would require some specific compiler
> support to mark the call).

When I first read this, I thought you were saying you changed
posix_spawn to call plain old fork, which would have meant it would no
longer solve Sergey's problem.  After reading over BZ#14750, I see
that posix_spawn does still use the vfork primitive operation on
Linux, it just does it in a lower-level manner that allows you to
avoid the bugs you list (clone() with CLONE_VM|CLONE_VFORK and a
temporary stack, plus blocking internal signals, which application
code can't do for itself because we don't let it).

But I don't see any reason we couldn't expose your bug fixes to all
callers of vfork.  We might need to change its calling convention
(hypothetically, pid_t nvfork(void (*proc)(void *closure))) but we
could do that as a GNU extension.  That would fix #14750 and #10354
for _everyone_, and changing all existing internal subprocess creation
to use the hypothetical nvfork would be less work than changing it to
use posix_spawn, and give a cleaner result.

> I can't really see why you think not using posix_spawn, which addresses
> all this issues, would be better to either user a problematic interface
> (vfork)

I don't want to encourage people to use posix_spawn because
posix_spawn is a badly designed API.  It's difficult to use correctly.
It's *tedious* to use correctly, which means people won't bother.  It
can't do everything you might want to do on the child side (witness
the discussion of adding an extension to let it do chdir()).  Its
behavior in case of errors is underspecified.  And it might be
implemented in terms of fork, which means it doesn't guarantee to
solve Sergey's problem.

zw