This is the mail archive of the cygwin mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Excessive thrashing when popen()ing after a large malloc()

On Wed, Nov 15, 2006 at 04:18:10PM +0000, Eric Blake wrote:
>Eric Blake <ebb9 <at>> writes:
>>>     else
>>>       sprintf(cmd, "sh -c '%s' %d>&-", program, pdes[1])
>>>   }
>>>   pid = spawnl(_P_NOWAIT, _PATH_BSHELL, "sh", "-c", cmd, NULL);
>Why are you going through two levels of sh?  That seems like a waste to
>me; the whole idea of using spawn is to avoid a fork(), but when you
>>invoke "sh" "-c" "sh -c 'cmd'", you are right back to a fork.  True,
>>the new invocation of sh uses less memory than the 1 GB process that
>>invoked popen, so less thrashing will occur, but your whole approach
>>seems fundamentally flawed if you are trying to use spawn to avoid a
>You know, maybe a better approach to this would be to first implement
>posix_spawn() in cygwin, since that API seems like it would be possible
>to do the necessary fd manipulation in a spawned child process without
>the penalty of a full fork/exec.  Then you could teach popen how to use
>posix_spawn when it is available, which is probably a patch more likely
>to be accepted in newlib, rather than your current idea of having to go
>through an intermediary sh invocation.  But I do not have a copyright
>on file for cygwin contributions, so I am currently unable to work on
>contributing posix_spawn; you are at the mercy of someone willing and
>able to write such a patch.

Fully implementing posix_spawn is quite a bit of work although I did
tell Linus Torvalds that I'd look into implementing it in Cygwin
eventually (it would make git-on-cygwin faster).

I have a patch in my sandbox which implements a more lightweight
approach for popen using cygwin internals.


Unsubscribe info:
Problem reports:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]