slowness issue between 20060309 and 20060313

[clayne at anodized dot com]

I'm trying to debug a sever slowdown seen between 0309 and 0313 snapshots. I noticed
this when upgrading to the latest and had to trace it back (part of the impetus for
writing the script I mentioned earlier). It appears to affect all file operations
within a shell, but not necessarily within a process itself, post-fork. E.g. tar
runs fast on an extract, but if I were to extract the same amount of files, but
through multiple calls to tar, even in a serial fashion, the slowdown is huge. It's
not even slowdown that could be explained by typical fork overhead, it's literally
200% the difference.

For my actual test, I'm using a small source directory of about 20-30 C source files
from one of my projects and:

$ make clean
$ time make all

Actual times seen:

20060309:
real    1m47.328s
user    0m17.074s
sys     0m59.145s

20060318:
real    3m20.953s
user    0m11.703s
sys     0m45.895s

20060314:
real    3m21.594s
user    0m11.713s
sys     0m45.894s

20060313:
real    3m21.594s
user    0m11.715s
sys     0m46.134s


>From 20060313 and any snapshot above that, I see the exact same behavior. From any
below *and including* 20060309 I do *not* see the behavior.



The changelog for 20060313 specifies:

ChangeLog for 20060309 to 20060313



---------- winsup/cygwin/ChangeLog ----------

2006-03-13  Christopher Faylor  <cgf@timesys.com>

        * child_info.h (child_info_fork::handle_failure): Declare new function.
        (child_info_fork::retry): New field.
        * dcrt0.cc (__api_fatal_exit_val): Define.
        (child_info_fork::handle_failure): Define new function.
        (__api_fatal): Exit using __api_fatal_exit_val value.
        * environ.cc (set_fork_retry): Set fork_retry based on CYGWIN
        environment variable.
        (parse_thing): Add "fork_retry" setting.
        * fork.cc (fork_retry): Define.
        (frok::parent): Reorganize to allow retry of failed child creation if
        child signalled that it was ok to do so.
        * heap.cc (heap_init): Signal parent via handle_failure when
        VirtualAlloc fails.
        * pinfo.h (EXITCODE_RETRY): Declare.
        * sigproc.cc (child_info::sync): Properly exit with failure condition
        if called for fork and didn't see subproc_ready.
        * spawn.cc (spawn_guts): Use windows pid as first argument.
        * winsup.h: Remove obsolete NEW_MACRO_VARARGS define.
        (__api_fatal_exit_val): Declare.
        (set_api_fatal_return): Define.
        (in_dllentry): Declare.
        * exceptions.cc (inside_kernel): Remove unneeded in_dllentry
        declaration.

2006-03-13  Christopher Faylor  <cgf@timesys.com>

        * dcrt0.cc (dll_crt0_0): Reorganize so that sigproc_init is called a
        little later.  Add a comment.
        * fork.cc (resume_child): Make void.
        (frok::parent): Only zero pi when necessary.  Explicitly zero si.  Set
        this_errno when child_copy fails.  Accommodate change to resume_child.
        * sigproc.cc (sigalloc): Move global_sigs initialization here.
        (sigproc_init): Move global_sigs.
        (sig_send): Just check for flush signals once.

        * wincap.h: Define supports_setconsolectrlhandler_null throughout.
        * wincap.cc: Ditto.

2006-03-13  Corinna Vinschen  <corinna@vinschen.de>

        * autoload.cc (LoadDLLfuncNt): New define to wrap NT native functions.
        Use for NT native functions throughout.
        * dtable.cc (handle_to_fn): Treat return value of NtQueryObject as
        NTSTATUS value.

2006-03-12  Christopher Faylor  <cgf@timesys.com>

        * cygtls.cc (_cygtls::remove): Reset initialized flag right away if we
        were previously initialized.
        * cygtls.h (_cygtls::initialized): Move nearer to the end to catch
        situation when Windows 98 mysteriously changes parts of _my_tls when
        thread is detaching.
        * gendef (__sigfe_maybe): Simplify slightly.
        * tlsoffsets.h: Regenerate.

2006-03-12  Christopher Faylor  <cgf@timesys.com>

        * cygtls.h (CYGTLS_INITIALIZED): Change to a little more unlikely value.
        (CYGTLSMAGIC): Delete.
        * dcrt0.cc (dll_crt0_0): Call sigproc_init during init startup.
        (_dll_crt0): Don't worry about sync_startup.  Just wait for sigthread here.
        * dll_init.cc (cygwin_detach_dll): Only pick up tls version of retaddr
        if we have a valid tls.
        * fork.cc (frok::child): Remove sigproc_init initialization since it
        happens much earlier now.
        * gendef: Recognize SIGFE_MAYBE.
        (fefunc): Generate calls to _sigfe_maybe, if appropriate.
        (_sigfe_maybe): New function.
        * init.cc (search_for): Always initialize search_for, even on fork.
        (calibration_thread): Delete.
        (calibration_id): Delete.
        (prime_threads): Delete.
        (munge_threadfunc): Remove calibration_thread special case.  Avoid
        calling thread function if we haven't yet hit the "search_for" thread.
        (dll_entry): Remove prime_threads call.  Only call munge_threadfunc
        when hwait_sig is active.  Ditto. for _my_tls.remove ();
        * sigproc.cc (hwait_sig): Make global.
        (sigproc_init): Don't bother with sync_startup.
        (sig_send): Treat flush as a no-op when signals are held.
        (wait_sig): Cause signals to be held after fork.



---------- winsup/doc/ChangeLog ----------

2006-03-13  Christopher Faylor  <cgf@timesys.com>

        * cygwinenv.sgml: Explain about new fork_retry CYGWIN environment
        variable setting.






And this is where I think there *might* be an issue, but it is purely assumption
based on the type of changes seen:

---

2006-03-12  Christopher Faylor  <cgf@timesys.com>

-->     * dcrt0.cc (dll_crt0_0): Call sigproc_init during init startup.
-->     (_dll_crt0): Don't worry about sync_startup.  Just wait for sigthread here.
        * dll_init.cc (cygwin_detach_dll): Only pick up tls version of retaddr
        if we have a valid tls.
-->     * fork.cc (frok::child): Remove sigproc_init initialization since it
        happens much earlier now.
        * gendef: Recognize SIGFE_MAYBE.
-->     (fefunc): Generate calls to _sigfe_maybe, if appropriate.
        (_sigfe_maybe): New function.
-->     * init.cc (search_for): Always initialize search_for, even on fork.

--


-cl

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/