Re: struct environment

[David Carlton <carlton at math dot stanford dot edu>]

On Tue, 17 Sep 2002 15:49:57 -0400, Daniel Berlin <dberlin@dberlin.org> said:

>> Personally, I'd rather tighten up the interface to symbol lookups
>> first and only then move on to optimizations like changing how
>> reading in a part of the symbol information is implemented.

> These are [not? - DBC] orthogonal issues. Reading in part of the
> symbol information is *not* an optimization (as the source stands
> now). It is a necessity controlled *by the symbol lookup*.

> If psymtabs were private to stabs, then this would not be the cast
> anymore, but as it stands now, the readin of psymtabs and conversion
> to symtabs is controlled by the symbol lookup functions.

Right.  So you juggle the layers by which global symbol lookups
happen, leaving the implementation more or less the same as it
currently is but designing the interface so that it supports future
flexibility.  For example, we could modify the current design to,
rather than first search all the symtabs and then all the psymtabs,
search each file's symtab and psymtab (and minimal symbol table)
before moving on to the next file.  (And yes, I know that we have to
get rid of the idea of searching every single file: that's a somewhat
separate issue, and the above should only be taken as a general
illustration rather than a precise suggestion.)  That's 'tighten[ing]
up the interface to symbol lookups'.  Then, eventually, you think
about ways in which to actually use that flexibility to optimize the
process.

I'm not saying that the interface design shouldn't keep in mind future
possible changes.  I'm just saying that I don't want to wait until
those changes are in place before changing the interface.

>> For struct block, I'm tracking existing design extremely closely.

> No offense, but you shouldn't do that.  The current design does
> *not* scale. Or at least, not until hash tables were introduced,
> which happened in the past 3 months.

> I've only tested scaling of hash tables up to 100 meg of debug
> info. Even then, due to the number of blocks one has to search,
> things start to get a bit slow.  There are plenty of projects these
> days with 500 or 1 gig of debug info.

Yes.  But are there plenty of projects with a single block with a
gigabyte of debug info?  I put the words 'for struct block' in there
for a reason: the current design of the global environment doesn't
scale.  But I don't think struct block has the same problems right
now.

And, even if it does, I repeat what I said above: it makes sense to me
to first redesign the interface to make it more flexible, and to leave
less-pressing optimizations to a later stage.  (For global symbols,
not searching all files is more pressing whereas not requiring the
current psytab->symtab stuff is less pressing.)

Part of my motivation behind this design is to make it very easy to
change the way you're storing data in a dictionary.  So if, 5 years
from now, we run into blocks for which the current mechanisms really
don't work, the changes that I'm doing now will make it much easier to
change the way those blocks are implemented.

>> Most of these can't grow, but jv-lang.c and mdebugread.c both have
>> code by which lists can grow.

>> When the struct block conversion is done, the only change will be
>> that sorted lists will go away (since they're only used by
>> mdebugread.c, which should be converted over to using hash tables
>> instead), and that the common code in jv-lang.c and mdebugread.c to
>> allow lists that grow will all be in one place.  No other changes:
>> extremely similar data structures, exactly the same algorithms.

> Which haven't served particularly well for the past 5 years (before
> that, maybe).

> No offense, and not your fault.  And i'm not saying they *aren't*
> adequate.  It just needs more thought than "it's currently done this
> way, thus, we should do it that way in the future".

What do you think my motivations are for introducing a special case
just to handle mdebugread.c and jv-lang.c?  That last sentence of
yours above really isn't a very good representation of my thought
process.

>> But it is, I think, a bad idea if we miss a symbol that we should
>> find, or if we return the wrong symbol instead of the right one.

> Which is what happens if you don't track the name lookup rules
> perfectly.

> The rules specify what is ill formed. If you find symbols you
> shouldn't, it's not only confusing to the user, it's the wrong
> symbol.

Maybe.  But it's what we do right now: if we don't find a symbol in a
global block, we search all the static blocks.  My first reaction was
that that's a bad idea; after thinking about it for a while, I'm much
less sure.  Certainly when I get down to more concrete design on
global symbol stuff, I'll raise this issue on the mailing list.

David Carlton
carlton@math.stanford.edu