Strange errors running gcc tests on Cygwin

Daniel Santos daniel.santos@pobox.com
Thu Mar 9 22:48:00 GMT 2017


First of all, thank you for your response!

On 03/08/2017 02:21 AM, Brian Inglis wrote:
> After any Windows Update, or a lot of package installs, you may want
> look at running
> 	rebase-trigger full[rebase]
> before rebooting to remove all Cygwin and Windows processes, then
> (with no Cygwin services or processes running) run Cygwin
> setup-x86{,_64} to rebase everything and maximize memory available
> to Cygwin processes.

So does rebase organize the Cygwin dlls and reassign them all new base 
loading addresses?  So far I've only done the 64-bit tests using Cygwin. 
I haven't even gotten to 64-bit mingw or 32 bit. Also, I have *not* done 
a Windows Update since I have installed. Could this be an OS issue that 
was addressed in some patch?  I'm running Windows 7 Ultimate SP1.

> This could be critical if you are doing any builds under Cygwin 32
> and have a lot of packages and/or large exes/dlls installed.
>
> If you are running a lot of Cygwin services, cron or Scheduled Tasks,
> and/or background processes, you may want to look at running cygserver
> to cache process info and common system info (including SAM/AD).

I'm only running sshd -- no cron or "at" jobs (except whatever Windows 
installs by its self).  However, gcc's make check spawns a LOT of processes.

> Setup cygserver initially by running cygserver-config with
> elevated/admin rights, and start it at system startup by running
> cygrunsrv with elevated/admin rights.
> If you ever expect to be running more than 62 simultaneous Cygwin
> processes on a system, bump kern.srv.process_cache_size in
> /etc/cygserver.conf created by cygserver-config to one of the higher
> values recommended in the man page.
> I found this seemed to reduce process startup overhead and eliminated
> random broken pipes, hung, and failed processes due to a lot of shell
> forking.
I didn't know about cygserver, so I tried it out and set the 
kern.srv.process_cache_size to 256.  I ran a make -kj4 check and had 
broken pipes within a few short minutes.  I tried it again running make 
-k check, but on two directories at once and got broken pipes somewhere 
along the way (before they completed).  So I should probably try it with 
only a single make running at once.

Even with just the two instances of make (with one job each), there are 
many processes of bash, make, expect, etc.  That's just one of the 
challenges of a Cygwin venture -- Windows was never designed for light 
process creation (or thread creation for that matter).  A lot of the 
*nix software concepts are hinged upon that lightweight, copy-on-write 
fork, whereas Windows-based concepts seem to be centered more around 
pre-spawning and maintaining thread pools and task-specific services.

> Preallocate contiguous paging space at double RAM to reduce the chance
> that any set of Windows processes will reduce Windows free memory too
> low for an instant, and cause something to hang or fail, by giving
> Windows somewhere to page out a lot of LRU pages from inactive
> processes.

I'm not sure how to go about this.  Should I bring the VM down, mount 
the image with ntfs-g3 and dd if=/dev/zero it or is there another way?  
Also, I don't know what Windows expects to see in a pagefile.  Or maybe 
defrag. I'll look at it.

> I found that if Windows (at least up to W7) free memory ever got too
> low, especially when all processors are pegged, it seemed unable to
> dynamically allocate and use more paging space to alleviate the
> crunch.

 From watching the system monitor, I can see that I never get above 35% 
memory usage.  I've assigned 4GiB of ram to the VM and Windows never 
seems to use more than 280MiB for page cache (I guess that's due to the 
license-specific limitation) -- at least there is abundant caching 
happening at the host level (3-ish GiB).

> The situation may have improved on Windows 10, but I've already
> allocated the paging space, and uninstalled some (probably buggy)
> hogs that seemed to only ever allocate memory and never free any.
>
> YMMV

As far as the dll search path issue, I *think* I have a woorking 
patchset in gcc and DejaGnu to resolve the dll load problems.  (I 
haven't yet done the on-off switch yet, so it's basically hard-coded.  I 
found a Tcl "is_Cygwin" procedure in git-gui that I'm going to borrow 
for now.)  So while my make check is still failing with broken pipes, I 
appear to have resolved the issue with the dll search path.

It would probably help if I can figure out where the broken pipe 
actually occurs and try to get that piece to log more verbosly.  I'm 
guessing it's from expect.  This is the exact error message:

FAIL: gcc.c-torture/execute/20000822-1.c   -O2 -flto -fuse-linker-plugin 
-fno-fat-lto-objects  execution test
parent: sync byte write: broken pipe^M
make[3]: Leaving directory 
'/d/builds/head-test-moutline-x86_64-pc-cygwin/gcc'

Don't worry about the FAILed test.  The main thing is that after each 
broken pipe message, I see make "leaving" that directory, so it doesn't 
finish the rest of the tests in that directory.

Daniel

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



More information about the Cygwin mailing list