Beguiling a text interpreter...

[Bruce Korb <korb@datadesign.com>]

A couple of months ago, I was sent the following note that looks to
be very helpful.  On the basis of this, I have altered my program
so that the template interpreter inverts its input from more-human-friendly
format to a more programmatic order.  (In my terminology, from output-
text-centric to control-logic-centric.)  I am now ready to s-l-o-w-l-y
try to convert autogen text functions into guile/scheme functions.
All I need to do is figure out how to construct these lists you talk
about.  It is not clear to me from the guile-ref.info or the guile-tut.info
what is the proper approach.  I understand the start up sequence
(main calls gh_enter calls inner_main).  I can see how to get Guile to
interpret a block of text.  I can see how to call scm_shell to let it
take over and do everything.  Here is what I think I need.

I have two basic inputs.  The first is my set of definitions
which simply map names to values.  The values may be either strings
or nested name-value pairs.  Multiple copies of the same name create
an implicit array.  The index values for any entry of the array may
be (optionally) specified explicitly.  The definitions do not and
will not have any other semantics.  (This has been stable in my code
for years.)

The second is the template that is to be interpreted based on the
definitions found above.  This is what makes sense to beguile.
However, because to me, a human, it is easier to comprehend what
I want the output to look like rather than to figure out how to build
control structures, the syntax is inside-out with respect to "normal"
code/text generation programs.  E.g.

   This is a text document.  The definitions for this contain
   several points.  They are:
   <<FOR point>>
       <<point_name>> is described as follows:
            <<point_text>>
   <</point>>
   END-OF-DOCUMENT.

In other words, it is the syntax between the `<<' and `>>' that is
to be beguiled, the text outside the markers is to be copied to the
output once for every pass the interpreter makes over that portion of text.
I have now reworked the code such that the template scanner restructures
the template into a tree.  Each node has a function code, an associated text
pointer and a next node.  Certain functions (FOR, CASE and IF, specifically)
are "block" functions and have member links as well.  I know how to
use "snarf" to set up and activate my various functions.  I do not know
how to make my tree into a list that Guile will then be able to interpret.

So, finally, here are my questions:

1.  Am I missing something documented in guile-ref.info?
2.  Is there any particular source that would be good to use as a pattern
    for writing what I want to write?
3.  Is this possible using the "gh_*" interface, or should I forget that
    and go anhead and plunge into the "scm_*" interface?
4.  Is there some subset of the guile-core code that would be most
    relevent for me to study?

Below is a little context of the message of 3 months ago.

Thank you again for your kind attention.

Regards,
    Bruce

P.S.  I am still willing to review preliminary documentation, especially if
it would be helpful for me with this project.  Also, if it would be helpful
for either your interest or understanding, I could send you my autogen
texi/info file(s).  Fundamentally, it is a vastly more powerful, flexible
and usable M4/C-preprocessor.

Andrew Archibald wrote:

> ; If I understand this correctly, converting the existing stack-machine
> ; (forth-like) syntax could simply be a matter of inverting the token stack as

> ; it is parsed, adding parens as we go, and letting guile evaluate that?
> ; Presumably the autogen primitives would need to be registered as guile
> ; functions somehow...
>
> That's about it.  There's no need to produce text output; you could
> construct a list (scheme's internal representation of code
> (everything, really)).  For example, suppose we wanted to translate
> the following C code:
>
> if (<COND>) <STMT1> else <STMT2>
>
> You would then execute the Scheme code:
>
> (list 'if (translate <COND>) (translate <STMT1>) (translate <STMT2>))
>
> Basically constructing Scheme's internal representation directly,
> instead of constructing text which guile would then have to parse.
> There's some more wizradry you could do; CTAX does "just-in-time"
> translation.  It has memoizing macros do the translation only the
> first time a piece of code needs to be executed.