This is the mail archive of the
guile@cygnus.com
mailing list for the Guile project.
Re: (seeks while reading) Re: Performance, argh
Greg Badros <gjb@cs.washington.edu> writes:
> Telford Tendys <telford@eng.uts.edu.au> writes:
>
>
> Then it will be much slower than it needs to be.
>
> That said, guile does seem to spend an inordinate amount of time in
> system calls (on PPro200, Linux 2.2.x, libc5, with guile installed in a
> directory read via NFS):
>
> % strace -c guile -c '(+ 1 2)'
> execve("/uns/bin/guile", ["guile", "-c", "(+ 1 2)"], [/* 92 vars */]) = 0
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 42.89 0.184235 2193 84 70 access
> 28.05 0.120460 2231 54 40 stat
> 17.43 0.074850 1826 41 read
> 6.63 0.028495 750 38 22 open
> 3.17 0.013606 1512 9 fstat
That's a lot of access, stat and fstat calls there, and those are
expensive as syscalls go.
This motivated me to find out where they are coming from (apologies
for the csh shell syntax, and see below for analysis):
Using guile 1.3:
[mstx@207-172-216-87 ~/scwm]$ strace guile -c '(+ 1 2)' |& egrep 'access\(|stat\('
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
stat("/usr/local/share/guile/site/init.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/init.scm.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/init.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/init.scm.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/init.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/init.scm.scm", 0xbffff7fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/ice-9/boot-9.scm", 0xbffff7e4) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/ice-9/boot-9.scm.scm", 0xbffff7e4) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/ice-9/boot-9.scm", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/boot-9.scm", R_OK) = 0
fstat(3, {st_mode=0, st_size=0, ...}) = 0
stat("/usr/local/share/guile/site/ice-9/r4rs.scm", 0xbffff6fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/ice-9/r4rs.scm.scm", 0xbffff6fc) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/ice-9/r4rs.scm", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/r4rs.scm", R_OK) = 0
fstat(4, {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/site/ice-9/", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/readline", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/readline.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/readline", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/readline.scm", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/readline.scm", {st_mode=0, st_size=0, ...}) = 0
fstat(4, {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/site/ice-9/", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/session", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/session.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/session", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/session.scm", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/session.scm", {st_mode=0, st_size=0, ...}) = 0
fstat(5, {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/site/ice-9/", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/regex", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/regex.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/regex", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/regex.scm", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/regex.scm", {st_mode=0, st_size=0, ...}) = 0
fstat(5, {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/site/ice-9/", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/debug", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/site/ice-9/debug.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/.scm", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/debug", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/share/guile/1.3/ice-9/debug.scm", F_OK) = 0
stat("/usr/local/share/guile/1.3/ice-9/debug.scm", {st_mode=0, st_size=0, ...}) = 0
fstat(4, {st_mode=0, st_size=0, ...}) = 0
stat("/usr/local/share/guile/site/ice-9/session.scm", 0xbffff5f4) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/ice-9/session.scm.scm", 0xbffff5f4) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/ice-9/session.scm", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/session.scm", R_OK) = 0
stat("/usr/local/share/guile/site/ice-9/version.scm", 0xbffff484) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/site/ice-9/version.scm.scm", 0xbffff484) = -1 ENOENT (No such file or directory)
stat("/usr/local/share/guile/1.3/ice-9/version.scm", {st_mode=0, st_size=0, ...}) = 0
access("/usr/local/share/guile/1.3/ice-9/version.scm", R_OK) = 0
fstat(4, {st_mode=0, st_size=0, ...}) = 0
Several problems are immediately apparent:
* Whenever Guile loads a file named "somefile.scm", it will also go
looking for "somefile.scm.scm". This is, IMO, semantically
incorrect, in addition to being a pessimization.
* Whenever Guile loads a file by base name only (I am guessing that's
what's happening in the other cases) then in each directory it tries
to load ".scm", then "basename", then "basename.scm". Checking for
".scm" is probably a bug. Maybe the basename loading code is a bad
thing in general. Being able to `(load "my-file")' and get
"my-file.scm" loaded is at best a miniscule convenience.
* When Guile access(2)s a directory and finds it doesn't exist, it
will go ahead and try to call access on files in it anyway. That
seems fairly bogus.
* Many files are access'd and then stat'd, or vice versa. The
duplication should be investigated and removed (if `load' is trying
to do an access() or a stat() before open()ing the file that is
bogus, it should just do the open and check for an error return).
* A lot of files are checked for (uncucessfully) in guile/site/ice-9/
instead of guile/1.5/ice-9. Perhaps it should be reconsidered
whether the `site' directory should be first in the load path, as I
can't think of any good coming of a site shadowing, e.g., debug.scm
with something different.
I will try to send patches for at least the more obvious these
problems if no one beats me to it.
- Maciej