Re: A generic reader for Guile?

[Ken Raeburn <raeburn@raeburn.org>]

Wow.  When you mentioned generic readers, I didn't think you were
going to go this far. :-)

Mikael Djurfeldt <mdj@mdj-pc.nada.kth.se> writes:
>      ...
>      (exp ((NUM)         (lambda (x) x))
>           ((exp #\+ exp) (lambda (x y) (+ x y))))
>      ...

(Aside: A mechanism for a first-cut translation from yacc to scheme
form would be helpful, to get easier access to existing yacc parsers;
just put any C actions not understood into a string and report an
error, in case the programmer hasn't replaced them with the
appropriate scheme actions yet.)

> or something like that.  (It may be interesting to ponder if there is
> some other way to specify the action than with lamdas's which could
> achieve higher performance.)

Maybe, but keep it intuitive.  The lambda or "(set! $$ (+ $1 $3))" is
probably best.

This relates to something else we were discussing in private mail:
This scanner/parser generation seems like another example of an
operation heavy-weight enough that it makes sense to use some sort of
optimizer, since you're already committed to using non-trivial time.
So if a more intuitive form can be optimized with some extra
transformations and knowledge of the context into a more efficient
form, use the intuitive form for the programmer interface.


A couple things to watch out for, from the g++ grammar:

	| maybe_attribute
		{ tree d;
		  $$ = parse_decl ($<ttype>-1, $<ttype>-2, $1, 0, &d);
		  cp_finish_decl (d, NULL_TREE, $<ttype>0, 1, 0); }

Note the use of stack elements preceding those listed in the
derivation, to provide additional context for the action.  It could be
done with extra global state variables describing a stack manually
unwound as the relevant tokens are popped from their stack, but this
usage gets the info while the parser does all the stack maintenance.

	| nested_name_specifier self_template_type '(' 
                { $$ = begin_constructor_declarator ($1, $2); }
	  parmlist ')' cv_qualifiers exception_specification_opt
		{ $$ = make_call_declarator ($<ttype>4, $5, $7, $8); }

Note the action in the middle, before some of the tokens.  The form of
the grammar input and action specification need to allow for these
possibilities, unless you want to handle only a subset of what yacc
can do.


A parser might need a set of state variables.  In fact, some state may
need to be shared between parser and scanner, such as a set of typedef
names in C.  If we want to avoid using global variables, I think
facility for this needs to go into the parser and scanner generation.


What about specifying value-containing versus valueless tokens?  No
need to push #\+ on the value stack just because it's in the grammar
as a terminal; its presence might be entirely described by the state
numbers of the parser.