Allocation notes

[Greg Harvey <Greg.Harvey@thezone.net>]

This is a bit of informative documentation & ranting on the allocation
functions in guile. They generally tend to be used incorrectly (mostly
because of the misleading names), so hopefully this can be a bit of a
clarification.

The builtin allocation functions in guile are scm_must_(malloc,
realloc), which are misleading in that they do more than insure that
the storage is malloc'd or error (which is what the must implies);
they also increment the amount of storage that the garbage collector
considers to be under scheme control (represented in the variable
scm_mallocated). For normal objects (those that are reaped by the gc),
there's no problem, because the gc insures that scm_mallocated is
decremented apropriately. This is the same reason that smob_free
functions are supposed to return the size of the storage that was
freed (I personally think this is a bit of grossness). It's objects
that aren't under gc control that cause problems.

For deallocating the storage, we have scm_must_free; you'd think that
this undoes all of what the other scm_must functions have done (and
this is even done in the guile source); however, this doesn't
decrement scm_mallocated, so guile still thinks that the memory is
allocated somewhere. For small runs and programs, this isn't a large
problem (maybe an extra superfluous gc here and there, but mostly no
harm done); for a larger, longer running program that does a lot of
dynamic allocation and freeing, however, scm_mallocated could
eventually overflow (especially on 32 bit machines), causing a gc for
every dynamic allocation, which will grind things to a smashing halt,
as well as cause numerous complaints about the poor, confused garbage
collector.

I think the best solution (and the one I've put in place in thing) is
to have two sets of allocation functions:

scm_obj_foo: which serves the purpose of the current scm_must
functions

scm_nonobj_foo: which will gc to attempt to get memory if necessary
and checks the result of the malloc, but doesn't fiddle with
scm_mallocated.

It also adds a scm_obj_done_free(scm_sizet size) function, which
should be used for scm_obj allocations that don't fall under gc
control. A quick fix for the current source is to add a scm_done_free
function to serve the same purpose (you could use scm_done_malloc with
a negative number, but that's gross). All it needs is:

void
scm_done_free(scm_sizet size)
{
	scm_mallocated -= size;
}

and away you go, to fix any broken code ;). I also think it'd be
better to use this method for smobs (rather than returning the amount
deallocated; for intricate smobs this is a lot more flexible).

-- 
Greg: and I didn't screw up a url this time (yay!)