Extend faq.using to discuss fork failures

[Ryan Johnson]

On 19/08/2011 10:41 AM, Christopher Faylor wrote:
> On Fri, Aug 19, 2011 at 09:43:10AM -0400, Ryan Johnson wrote:
>> Hi all,
>>
>> I propose to add an entry to cygwin's faq.using which covers fork
>> failures. Frankly, I'm surprised it wasn't there years ago... it's
>> certainly frequently-asked, and the answer is always the same. Right now
>> users have to trawl the archives to figure out what to do (or more
>> likely, just blindly spam the list and get told to rebase and/or trawl
>> the list archives).
> I appreciate that you're trying to do this.  I was actually going to
> ask someone if they wanted to write a section like this but assumed
> I wouldn't get any takers.
Well, at some point there will be significantly fewer characters typed 
to make the patch than to keep answering emails complaining about fork 
failures...

> Wrt, the spawn function, they harken from a time when Cygwin was
> confused about what API it was exporting.  They *are* deprecated.  I
> don't see any pressing need to document them.
>
> (And, yes, we know about the posix functions with _spawn in their names)
Should we nuke the corresponding text from the user guide and be done 
with it, then?

>> ..., and reports of fork failures are
>> probably the single most common thread topic in the cygwin mailing list.
> I don't think comments like this are appropriate.  If there was a
> magical time when we've fixed fork failures (and Corinna's proposed
> changes to run rebase during setup.exe should at least cut back on them)
> then this would be out-of-date.  It doesn't provide any useful information
> to the user anyway.
Fair enough.

BTW, I hope that rebasing wouldn't require every invocation of setup.exe 
to shut down all cygwin processes... I really like how right now I can 
pull new packages some configure script needs, without having to shut 
down 4-5 sessions of emacs.

>> Common error messages include:
>> - unable to remap $dll to same address as parent
>> - couldn't allocate heap
>> - died waiting for dll loading
>> - child -1 - died waiting for longjmp before initialization
>> - STATUS_ACCESS_VIOLATION
>> - resource temporarily unavailable
>>
>> The problem often (re)appears or worsens after installing up updating
>> cygwin packages (which can undo the effects of rebaseall and peflagsall,
>> see below). Applications which dynamically compile and load dlls (e.g.
>> perl, ruby, some lisps, building gcc from sources) are also especially
>> prone to fork failures for the same reason. Fork failures in general
>> also became significantly more common with the introduction of Vista and
>> Win7, whose address space layout randomization (ASLR) often causes child
>> processes to spawn with dlls, thread stacks, heaps, and other memory
>> objects allocated in different locations than the parent. While cygwin
>> compensates for as many of these relocations as possible, there always
>> remains a possibility of fork failures.
>>
>> If you find that frequent fork failures interfere with normal use of
>> cygwin, please try the following steps:
>>
>> 1. Disable or uninstall applications known to interfere with cygwin (see
>> http://cygwin.com/faq/faq.using.html#faq.using.bloda). Many of them
>> inject dlls into processes at inconsistent locations, which breaks
>> fork() semantics.
>>
>> 2. Rebase your system (see /usr/share/doc/Cygwin/rebase-3.0.1.README).
>> Every dll in the system specifies a base address -- the preferred memory
>> location it should load at -- and the Windows loader does not break ties
>> consistently when it encounters base address conflicts.
>>
>> 3. With Vista and later, use peflagsall to set the TS-aware bit on all
>> cygwin dlls (see /usr/share/doc/Cygwin/rebase-3.0.1.README, reboot
>> needed for changes to take effect). This exploits a side effect of
>> address space layout randomization which (ironically) causes dlls to
>> nearly always load at the same address.
>>
>> 4. If you have access to the source code of the offending application
>> (this applies to all cygwin packages), consider replacing calls to
>> fork() with calls to the spawn family of functions. These are a native
>> (= reliable and highly efficient) replacement for fork+exec, which is by
>> far the most common usage of fork(), and are documented at
>> http://msdn.microsoft.com/en-us/library/20y988d2%28v=VS.100%29.aspx.
> I appreciate your thoroughness but I think there are way too many words
> above.  The FAQ should be solution-oriented.  If it is important to
> discuss the details behind why fork() fails then maybe another section
> could be added.  Otherwise, I'd prefer to see something which shows the
> error messages and then, as briefly as possible, shows solutions.
> While people do ask "Why does fork fail?", the majority of the askers
> don't really care.  They are really asking "How do I make Cygwin fork
> work?"  So, I don't think that it is really FAQ-appropriate to dive
> too deep here.
I'm definitely a fan of brevity. My main motivation for all the verbage 
was so that users who read the faq aren't as shocked when they do all of 
the above and fork still fails more often than they'd like, and so 
they'd have some idea of which steps are most applicable to their situation.

Two sections might work very well: "How can I prevent fork failures?" 
and "Why does fork() still fail after I run rebaseall?" Does that sound 
good to you?

> And, again, we don't want to tell people to use non-POSIX solutions
> except as a last resort.  Telling people to rewrite their source code
> flies in the face of what Cygwin is trying to do.
>
> (And, yes, I presciently can hear the argument to the above paragraph
> coming)
You would prefer that it remain an unadvertized last resort, then?

I guess the idea is less imposing to me after porting lots of code 
between linux/gcc and solaris/suncc.

Off topic: to be honest, I'd *love* it if bash, make, and gcc used spawn 
instead of fork+exec when compiled under cygwin, though I don't know how 
I/O redirection would fit in.

Ryan