Making Guile slower (3.4 and 6.7%). (was Re: NIMP thing)

[Mikael Djurfeldt <mdj at mdj dot nada dot kth dot se>]

[I said I wouldn't push any more for reverting the change.  I won't do
 that---you guys do as you like.  But since you persist in not
 understanding Marius' and my viewpoint I'll use some new data (3.4
 and 6.7% slowdown of the evaluator as measured on two different
 machines) as an excuse to give some counter fire.  This counter fire
 is mainly motivated by the picture you have painted in your replies
 to Marius and me, and not by the NIMP change itself.]

"Greg J. Badros" <gjb@cs.washington.edu> writes:

> Mikael Djurfeldt <mdj@mdj.nada.kth.se> writes:
> 
> > While I of course share your goals concerning Guile and Guile code
> > (and prefer coding over discussing), we obviously completely disagree
> > regarding the approach of your change.  [It is this approach which I
> > call "careless".  Made by a less careful programmer it would have led
> > to a larger mess.]
> > 
> > It's not that I'm complaining about it.  I think it's plain wrong and
> > should be taken out.  (This is the first main message of this letter.)
> 
> What's the antecedent of "it"?  The approach or the change?  I'm
> absolutely positively certain of the change.

I was referring to the approach.  My first gut reaction to your NIMP
change was that I couldn't immediately overview all consequences, and,
since I didn't know about your post on the list (should have checked
that!), I thought it was too wild.

After some thinking, I realized that changing all SCM_NIMP && SCM_FOOP
cases to SCM_FOOP was good, but didn't like the way this change was
performed and, in particular, didn't like that you had inserted an
extraneous SCM_NIMP for SCM_FOOP-only cases.  My gut feeling was that
this could slow down the evaluator in some cases.

> > The purpose of my "alarm signal" and the following discussion was to
> > get you to revert the NIMP changes and do it in a safer and more
> > controlled way.  Yes, that would have taken valuable time, but we are
> > now dealing with very high values: Guile efficiency and Guile
> > stability.
> 
> Until someone demonstrates that guile is slower somewhere, I put *no*
> stock in the efficiency argument.  I've demonstrated undeniably that for
> a prominent platform with the standard compiler there is zero difference
> in the common case where performance could've suffered.  Period.

I think you're not using straight arguments.  In your replies you
paint a picture of Marius and me having subjective worries while your
actions are based on solid software engineering.

The reality in this case is quite the opposite.  Marius and myself
worry because you are using an approach which is not solid, an
approach where you are replacing knowledge of the effects of the
change with _belief_ in the effects.  You have demonstrated that in
one type of case, on one platform, the compiler optimizer eliminates
the redundant operations.  I trust your judgment that it is common
among optimizers to have this ability but I don't know how common this
case is in Guile and I actually don't care, because it's enough to
make an unfortunate change in just one or two critical places to slow
down the evaluator (which is, BTW, highly hand-optimized).

Marius proposed a way to make this change with preserved knowledge of
the effects of the change, while you have let go of this knowledge and
just _believe_ that the changes won't influence efficiency.  In this
perspective, I find it strange that the burden of demonstration is put
on us.

In any case, I configured a Guile from 1999-12-10 and a current Guile
with configure option --disable-shared, compiled, and ran the `eval1'
benchmark on an i686 and a sparc machine.  The results are 3.4%
slowdown on the i686 and 6.7% slowdown on the sparc.  While one should
of course always be suspicious of benchmarks (can we get it repeated
elsewhere?) it is an indication that Guile has now taken a step down
the slippery slope which I talked about earlier.

/mdj

mdj:/usr/local/guile> guile-pre-nimp/libguile/guile -c '(begin (use-modules (benchmarks eval1)) (eval1 5000000))'
Benchmark: eval1 revision 1
procedure-sum        1  13.08s user  11.12s gc   0.01s sys  24.21s tot
procedure-sum        2  13.13s user  11.12s gc   0.00s sys  24.25s tot
procedure-sum        3  13.17s user  11.11s gc   0.02s sys  24.30s tot
procedure-sum        4  13.19s user  11.10s gc   0.01s sys  24.30s tot
procedure-sum        5  13.11s user  11.16s gc   0.01s sys  24.28s tot

===> 5                  13.11s user  11.16s gc   0.01s sys  24.28s tot

mdj:/usr/local/guile> guile-post-nimp/libguile/guile -c '(begin (use-modules (benchmarks eval1)) (eval1 5000000))'
Benchmark: eval1 revision 1
procedure-sum        1  14.09s user  10.93s gc   0.02s sys  25.04s tot
procedure-sum        2  14.15s user  10.95s gc   0.00s sys  25.10s tot
procedure-sum        3  14.11s user  10.97s gc   0.03s sys  25.11s tot
procedure-sum        4  14.19s user  10.96s gc   0.01s sys  25.16s tot
procedure-sum        5  14.10s user  10.96s gc   0.04s sys  25.10s tot

===> 3                  14.11s user  10.97s gc   0.03s sys  25.11s tot

*** 3.4% increase on i686-pc-linux-gnu (450 Mhz Pentium III)

sans04:~/exp> guile-pre-nimp/libguile/guile -l ../.guile -c '(begin (use-modules (benchmarks eval1)) (eval1 800000))'
Benchmark: eval1 revision 1
procedure-sum        1   9.04s user   5.03s gc   0.01s sys  14.08s tot
procedure-sum        2   9.10s user   4.95s gc   0.00s sys  14.05s tot
procedure-sum        3   9.09s user   5.00s gc   0.01s sys  14.10s tot
procedure-sum        4   9.07s user   5.04s gc   0.01s sys  14.12s tot
procedure-sum        5   9.07s user   5.03s gc   0.01s sys  14.11s tot

===> 3                   9.09s user   5.00s gc   0.01s sys  14.10s tot

sans04:~/exp> guile-post-nimp/libguile/guile -l ../.guile -c '(begin (use-modules (benchmarks eval1)) (eval1 800000))'
Benchmark: eval1 revision 1
procedure-sum        1   9.80s user   5.22s gc   0.01s sys  15.03s tot
procedure-sum        2   9.85s user   5.18s gc   0.00s sys  15.03s tot
procedure-sum        3   9.81s user   5.25s gc   0.01s sys  15.07s tot
procedure-sum        4   9.80s user   5.29s gc   0.00s sys  15.09s tot
procedure-sum        5   9.81s user   5.22s gc   0.01s sys  15.04s tot

===> 5                   9.81s user   5.22s gc   0.01s sys  15.04s tot

*** 6.7% increase on sparc-sun-solaris2.6 (Ultra 1)