Everyone is thinking about the future

[Telford Tendys <telford at eng dot uts dot edu dot au>]

Seems to be all the rage in mailing lists but these days I download
as much header as content...

> > it is a clear signal that something is wrong with
> > the Guile project and that something needs to happen.
> >
> [...] 
> >
> > Now, what's going to happen with Guile?  Is it dead?  Or merely ill?
> > Is its illness infectious?
> > 
> > I don't know.  I'm kind of emotionally attached to Guile, because it
> > is the project that was my ticket into the Free Software community.  I
> > still have the letter from the FSF pinned to my wall that I got when
> > signing over my copyrights.  So, it's hard for me to just let Guile
> > die.

You don't need a ticket into the free software community, nor a letter from
the FSF. Free software is much bigger than the FSF and looks ready to keep
growing -- I presume that was the original intention, freedom is like that.

> > I never really resonated with other Scheme implementations either.  I
> > slowly began to think that SCM was a good choice as the basis for rms'
> > goals of a universal extension language, although I didn't understand
> > it in the beginning.  Its design decisions fit well with being a
> > library in a C world, and yet it is efficient and makes no
> > functionality compromises.  So, I don't think Guile is fundamentally
> > technically flawed.

Scheme is a nice language, it is fast to write and difficult to hurt
yourself with. It is easy to use interactively. However, guile has too many
goals and with people saying that ``scheme is the ultimate language for
everything'' trying to narrow the scope of the guile project is difficult.

My basic problem with R4RS scheme is that the language lacks modularity,
it is entirely oriented around a global namespace, with no method of
narrowing the scope. Naturally SCM is built the same way. I have yet to
see a satisfactory solution for that, although things like GOOPS come
close.

> If Guile is going to be the "one" extension language,

It isn't... we should concentrate on making it work for us before
pushing it at other people. Do one job and do it well.

> it should have very clean, very simple internals.

everything should have very clear, very simple internals.
Achieving this requires a lot of thinking.

> This includes a very clean, very simple facility to extend it.

agreed completely, modularity is fundamental to an extension language.

> Guile has alot of good
> ideas, but i think it's partly hampered by it's difficult, non
> thought-through internals. 
> 
> The way it is currently, i won't be surprised if Guile will be
> completely rewritten (from scratch, probably) by next year.
> 
> - The numerical tower will be replaced.

The current tower is a bit messy but reasonable. The trouble is that
it is monolithic -- when I added matricies I found myself writing a completely
new tower that had the prefix ``matrix-'' in front of everything. The
BLAS wrappers were similar.

> - The environments will be replaced.
> - The module system will be replaced.

Is there yet a cohesive concept of what these things do and how?
I'd be happy to release my code as guile modules but I'm yet to figure
out how it should be done... Can I release some modules on shared object
form and some in scheme form?

> - The garbage collector will be replaced.

The whole storage system has to be cut from the evaluator and some well
defined API constructed to join the two. The garbage collector goes with
the storage system. Half the problems with using guile are storage related,
possibly more than half so if a range of storage and GC methods are available
(or specialised apps can write their own and stay compatible with the
evaluator) then there will exist more options for handling difficult
situations.

> - The reader will be replaced.

Writing a new reader is pretty easy, some apps will need to do this or
will need to embed chunks of scheme in a bigger file etc. Nicest would be
to have exposed APIs for some partial readers (like read a number, read a
string, etc) making it easier to bolt together what is needed at the time.

> - We'll have two kinds of heaps.
> - The evaluator maybe might be replaced (there have been long
>   discussions of replacing it by a bytecode compile-evaluator
>   pair).
> - Most of the stuff in the guile core is slowly being put out into
>   modules.

This last one is the most important. I would like to see guile
cut into independent chunks, each with an independent maintainer to
prevent monolithic vitrification:

* A storage system with
--- garbage collector
--- type flags
--- really primitive types, just strings and numbers should be enough

* A hash table system that sits on the storage system

* A port system that can read and write files but does not know
  about any data types (except ports). Must have a way of allowing
  higher level units to register types for reading and writing (difficult)

* A general purpose evaluator that contains
--- namespaces
--- symbols
--- conditionals / loops / function-definitions / procedure-calls

* A numerical tower that works with or without the evaluator
  (i.e. depends only on the storage system)

* A POSIX chunk to provide system interfaces

* Advanced string manipulation

* etc... from here bolt on what you like


The missing link in all of this is how to allow these modules to fit
together in a dynamic fashion (e.g. punt POSIX if it doesn't work on your
system, etc).

In the meantime, I'm chugging along with what is there, because it works
well enough, but I can't say I'm head over heels in love with it (and yes
I do use perl and C a lot of the time too).

	- Tel