Opinions wanted: What should (.list pmacro-name 42) do?

[Doug Evans]

Jim Blandy wrote:
> On Thu, Aug 20, 2009 at 10:30 AM, Doug Evans<dje@sebabeach.org> wrote:
>   
>> Hi.
>>
>> What should the output of
>>
>> (define-pmacro (foo a) (add a 1))
>> (.list foo 42)
>>
>> be?
>>
>> Currently the result is (<pmacro> 42)
>> where <pmacro> is the pmacro object for "foo".
>>
>> It's not ideal because it means that a pmacro object "escapes"
>> pmacro-processing and can be seen by, for example, rtl compilation.
>>
>> However, it's not that unexpected.  The user asked for a list containing two
>> objects, the pmacro and 42.
>> This is akin to saying (list + 42) in Scheme.
>>
>> We *could* have the pmacro evaluator re-examine the result and if it sees
>> (<pmacro> mumble) then re-expand it.  That's what we do for symbols today
>> (i.e. if the result of pmacro-expansion is a symbol that names a pmacro, we
>> re-evaluate it), but that has problems, e.g.
>> http://sourceware.org/ml/cgen/2009-q3/msg00052.html
>> and I'm leaning toward removing that behavior.
>>     
>
> Macros should definitely be able to expand to calls to other macros
> somehow; doesn't (.list 'foo 42) do this?

Hey Jim!  Thanks for the feedback.

I'm not quite sure what you mean, alas.  [Apologies.  I can guess, but ...]

CGEN's pmacro system doesn't (currently) have quoting a la ' (or quote), 
btw.
[The Scheme reader will accept it of course, but the pmacro system won't 
(currently) do anything sensible with (quote foo).]

> If it does, then I don't
> see much point in allowing (.list foo 42) to do so as well.  The
> source world and the value world need to interact only in
> easy-to-reason-about ways, or you end up with stuff like m4, which is
> just chaos.
>
> This reminds me of the "3-d macro" problem in Scheme, where macros can
> place values that can't actually be written in Scheme source code
> (procedure objects, say) into the tree that the compiler eventually
> sees.  I've always thought the cleanest thing was to reject such code,
> as it didn't seem useful, and made a weird breach between macro
> expansion time and run time.
>   

Note that one can pass pmacros as parameters to other pmacros and invoke 
them.  But the user is required to explicitly do this, CGEN won't 
(currently) take the result of a macro and rescan it for further pmacro 
invocations (except, currently, in the case where the result of the 
pmacro expansion is a single symbol).

In Scheme the result of (list + 1 2) is (<primitive-procedure-plus> 1 
2), not 3.

For reference sake, in Scheme,
(define-macro (foo a) `(bar ,a))
(define-macro (bar a) `(baz ,a))
(foo 3) ;; expands to (bar 3) which in turn expands to (baz 3)

In CGEN that would be
(define-pmacro (foo a) (bar a))
(define-pmacro (bar a) (baz a))

Using pmacro-trace in Guile I get:

guile> (pmacro-trace '(foo 3) (unspecified-location))
Pmacro expanding: (foo 3)
Pmacro location: standard input:10:16
    Expanding: (foo 3)
          env: ()
     location: standard input:10:16
    Expanding: (bar a)
          env: ((a . 3))
     location: standard input:3:26
       result: (baz 3)
       result: (baz 3)
Pmacro result: (baz 3)
(baz 3)

And in Scheme,
(define-macro (foo a) `(list bar ,a))
(define-macro (bar a) `(baz ,a))
(foo 3) ;; ==> (#<macro! bar> 3) ;; Note that Scheme didn't re-expand 
the result and turn it into (baz 3).

In CGEN,
(define-pmacro (foo a) (.list bar a))
(define-pmacro (bar a) (baz a))
(foo 3) ;; ==> (<pmacro bar> 3)

We don't have to follow Scheme per se, but I'm worried that being 
excessively clever here will lead to issues (like in  
http://sourceware.org/ml/cgen/2009-q3/msg00052.html).
One issue is: what if the user *wants* (.list foo 42) to be (<pmacro 
foo> 42).  S/he can always explicitly re-expand the expression to turn 
it into (add 42 1), e.g., with (.eval (.list foo 42)), but there's no 
(current) way to prevent the re-expansion if it was done automagically 
(unless, for example, we add a quoting system, which I hesitate to do in 
part because it's yet another complication to the language).