Re: Translators again (Re: Python transformer for Guile?)

[Neil Jerram <neil at ossau dot uklinux dot net>]

Lalo Martins writes:

   On Sat, Jan 22, 2000 at 01:33:26AM +0100, Mikael Djurfeldt wrote:
   > 
   > Guile doesn't currently have a framework for language transformation.
   > Hopefully, we'll begin working on this after other high priorities,
   > such as the module system.  But it's probably still a good idea to
   > start looking into supporting Python already now.

   [...]

   Let me try once again :-) What we need, IMHO, is to ``spec'',
   as in ``settle'', a few basic things:

Here's my thinking.  It's quite bound up with the module system, and
not really ready for presentation yet.  On the other hand, the
discussion space for talking about the translator framework is, at the
moment, just too huge, and I think that a fixed point may help.

1. The result of "translation" is a stream of Scheme expressions (as
though returned by "read") that are to be evaluated in a specified
environment.

2. Translation is performed by an instance of a "<*-reader>" class.  For
each new language that we want to translate, e.g. Python, the
top-level interface for translating that language is provided by a
corresponding reader class, e.g. "<python-reader>".

3. The arguments to "load" specify

   - the reader class that should do the reading

   - the environment in which the read expressions should be evaluated

   - where to get the code from.

To illustrate this with some (completely hypothetical, untried) code...

;; Metaclass for reader classes.  All reader classes should specify
;; this (or a subclass) as their metaclass.
(define-class <reader-class> (<class>))

;; Base class for reader classes.
(define-class <reader> () #:metaclass <reader-class>)

;; Reader class example for Python.
(define-class <python-reader> (<reader>)
  (src #:init-keyword #:from)       ;; where does the code come from?
  ...slots...                       ;; for internal read state
  )

(define-method read ((reader <python-reader>))
  ;; Return next expression to be evaluated;
  ;; or EOF if at "end of file".
  )

;; Load interface.
(define-method load ((reader-class <reader-class>)
                     (env <environment>)
                     . sources)
  (load (make reader-class #:from sources) env))

(define-method load ((reader <reader>) (env <environment>))
  ;; Illustration only! - no error/stack handling etc.
  (let loop ((expr (read reader)))
    (or (eof-object? expr)
        (begin
          (eval expr env)
          (loop (read reader))))))

(define-method load ((reader-class <reader-class>) . sources)
  (load (make reader-class #:from sources) (the-environment)))

(define-method load ((reader <reader>))
  (load reader (the-environment)))

;; Default load method if reader class not specified.
(define-method load sources
  (apply load <scm-reader> sources))

3. Interpretation of the "sources" argument is theoretically delegated
to the reader classes, but in practice the classes use common utility
routines to interpret "sources" in as uniform a manner as possible,
plus language-specific tweaks.  Various forms of the sources argument
support:

- loading from a single file/module

- loading from a sequence of files/modules

- files referenced by filename, e.g. "/home/neil/guile/gore.scm"

- module references, e.g. '(ice-9 and-let*)

- loading modules that are defined by only a subset of a file,
  e.g. '(ice-9 collation english) where only ice-9/collation.EXT
  exists in the load path: a language-specific mechanism identifies
  the section of the file that corresponds to 'english

- any reasonable combination of the above.

[ 4. For what it's worth, my nascent design for the module system
builds on top of this: essentially "import" = create an environment +
load in that environment + add environment to import list for current
environment + an imported environments cache keyed by reader-class and
sources.  But I really don't want to jump the gun on the module
specification discussions, and I haven't even evaluated my ideas yet
against the complete wishlist. ]

Given this, here are my answers to some of your questions.

   1: how are translated scripts executed?

Normally they are translated and evaluated at the same time, by
"load".

      Is the ``client'' application responsible to finding out the
      language, or is it Guile (libguile)? [...]

This is a detail, IMO.  We can implement any reasonable combination of
-    client specification
-    looking at file extensions
-    simple source code analysis.

The problem here is analogous to how Emacs chooses major modes, and we
can provide a corresponding set of solutions.  This could include
something like "-*-Python-*-" in a Python comment.

      There is a third approach, which is making _the script_
      responsible for setting the language; it would have to start
      as Scheme, and then run some procedure (or evaluator
      command) to switch. IMHO this would be a PITA for translator
      writers (who would have to ignore the scheme portion
      somehow), for whoever will modify the evaluator, and for
      script writers.

Yes - PITA all round.  Very bad idea.  If we want to impose a burden
on script writers, a much easier burden is simply to use a standard
file extension.  (Which they probably already use anyway.)

      Assuming you want to write a script in a language that exists
      only as a Guile translator (CTAX, Sonya, whatever), your best
      bet is to write an interpreter and use a shebang; #!
      /usr/bin/ctax-run (and ctax-run would be a very small program, a
      REPL that has CTAX as the default language.)

Agreed.

      So perhaps our best bet is to set a standard for the first
      line (actually, the first string; you terminate it with \n
      if you want it to be a whole line); each translator module
      (or whatever for a translator ships in) supplies the
      expected first line, then the evaluator builds a table and
      checks this table to find out which translator it should
      dispatch the file to. This approach lets each language
      define the first line in a form that is a valid line
      (presumably a comment) in its language - so a PS reader can
      register ``%!PS''.

I think that setting new standards will not work.  But if you
generalize your idea to semi-intelligent analysis of the first few
lines of the file, it works fine.

   3: and what form should translators ship in. I suggest a module
      with a well-defined API (there isn't a lot to it actually,
      IIRC it's a ``magic-string'' variable or somesuch and a
      ``translate'' procedure).

Well in my suggestion it's a module containing a reader class
implementation for the target language.  The module can of course be
partly written in C.

Here's to the future of translators!

       Neil