Re: system and popen fail in case of big application

[Rich Felker <dalias at libc dot org>]

On Mon, Sep 17, 2018 at 06:58:56PM -0700, Adhemerval Zanella wrote:
> >> For former I see there is no easy way to provide a similar closefrom
> >> function without either being racy (as *BSD and openjdk implementations)
> >> or to remove some scalability by adding serialization (for open* syscall
> >> to avoid create new FD while closeall is closing them). A better alternative
> >> would to request for some kernel helper, but I am not convinced that
> >> current way of explicit open all file descriptors with O_CLOEXEC is not
> >> the better option. Unfortunately it does not help run external code
> >> (through shared libraries) that still call open in default mode.
> > 
> > It seems like traditional methods to create file descriptors will
> > never set the CLOEXEC flag by default, and most programmers will not
> > be aware of the problem, so those of us implementing subprocess
> > creation cannot ever rely on all the file descriptors having the flag
> > set.
> 
> But from a QoI standpoint O_CLOEXEC is still the better option. And f we 
> aim to add such extension on posix_spawn, I will expect we also push for 
> some kernel support. There are reports where trying to interacting over 
> all possible FD takes a lot of time [1].
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=837033

While closeall is an awful operation that should not be done (fix
stuff to use O_CLOEXEC), looping over the whole fd range and making a
close syscall for each probably-already-closed one is the most awful
possible way to implement it. You can make it fast by calling poll
with 1000 or more fds at a time in the pollfd set, checking revents
for POLLNVAL to determine which ones are open, and only close those.

Rich