GSOC | Extending Common Lisp support

[Jamison Hope]

On Jul 1, 2012, at 4:34 AM, Per Bothner wrote:

> On 06/29/2012 10:51 PM, Charles Turner wrote:
>> I've finished ironing out the bugs I immediately noticed in test  
>> cases
>> from the hyperspec and some other literature, so I think I now have a
>> decent start for packages (most of the use cases appear to work
>> correctly).

Great!

>> I'm currently breaking the "core" functions into separate
>> packages to facilitate further testing. That raises a few questions  
>> of
>> its own. Currently I am manually adding the various symbol names to
>> the respective packages (just CL), but maybe this should be done
>> automatically. Doesn't seem straightforward when you consider that a
>> lot of the functionality is coming from Scheme.
>>
>> The special casing of lower-case symbol names is unpleasant in the
>> code (I'm upper-casing symbol names in the LispPackage code in  
>> various
>> places), so I'm thinking about ways of fixing the CL reader. My first
>> plan was to modify LispLanguage#parse to call a stub method
>> setReadCase() which does nothing for Scheme, but for CommonLisp, it
>> calls LispReader#setReadCase('U'). This of course means all the
>> symbols imported at CL boot are not found, due to their names being
>> hashed as lower case into the symbol table. It's pretty
>> straightforward to modify the defun to upper case the symbols given  
>> at
>> boot, but the imports from Scheme land seem to pose a problem. Am I
>> approaching this incorrectly?
>
> In the medium term I think it's accept to implement a dialect of  
> Common
> Lisp where the reader by default folds to lower-case (rarher than  
> upper-case),
> and where built-in and predefined named are lower-case rather than  
> upper-case.

That would be fine with me. Any sort of case folding seems like an
improvement over what we've got now.

> Or we can do it right in which case as you say the imports from  
> Scheme are
> a problem.  However using loadClass is probably not an proper  
> solution.
> If think we'll have to explicitly define each symbol that we want to
> predefined - like we do for Scheme.

That's what I was expecting, too. Charles, when you say you are  
"manually"
adding the symbol names to the packages, what do you mean? It would make
sense to me to see something for symbols/packages in CommonLisp#initLisp
akin to how stuff gets defined (implicitly) in different Environments in
Scheme#initScheme:

initScheme has some definitions, then something like

       nullEnvironment.setLocked();
       environ = r4Environment;

and then some more definitions. Perhaps initLisp could have something
like

      package = common_lisp_package; // like (in-package "COMMON-LISP")

      defProcStFld(); // now this symbol ends up in that package
      ...

>  Even if we used loadClass as
> bulk import, one could override it.  For example the statement
>
> 	sym = Symbol.make(uri == null ? "" : uri,
> 			  fdname.toString().intern());
>
> in ClassMemberLocation#defineAll could perhaps be replace by
>
>        sym = language.nameFromField(fdname, uri)
>
> where the default for nameFromField is as above.

That's an interesting idea. Would CommonLisp's nameFromField defer to
the current value of (readtable-case *readtable*)? Loading a class
doesn't directly invoke the reader at all, but it seems like it should
be consistent with it.

>> Apropos the midterm, I'm sitting on a near 4000 line diff, and hardly
>> none of it has been reviewed yet. I've talked about most of it at  
>> some
>> point, but I remember being told to show and tell early and often.
>> None of the code is bullet proof, but for the CL stuff, does that
>> matter as much? Would it be better to commit alpha quality code so
>> that everyone can see it for CL, since it's CL isn't really usable
>> anyway at the moment? It's slightly worrying as I'm not sure how to
>> break it up into smaller patches, as most of it depends on lots of
>> other things!
>
> Well, I think we need to break it up before it gets checked in.
> Not necessarily from review for the mid-term - part of the review
> might be to figure but how to break it up.  Jamison?

Yeah, that makes sense. Make a list of the changes at a logical level,
and come up with a dependency ordering. Any small fixes that are self
contained can go first, and stuff that requires those to be in place
can go next, etc.


Speaking of uncommitted changes, Per, did you get a chance to look
over my patches for gnu/kawa/util/RunTestScript.java and
gnu/commonlisp/testsuite to facilitate using RunTestScript with CL?
Also, what's your preferred way to fix Lisp2's getNamespaceOf for
syntax declarations? You had suggested something like
getFlag(Declaration.PROCEDURE|Declaration.IS_SYNTAX) but the
former flag is not public. So I'm sitting on a change that looks like

$ svn diff gnu/commonlisp/lang/Lisp2.java
Index: gnu/commonlisp/lang/Lisp2.java
===================================================================
--- gnu/commonlisp/lang/Lisp2.java	(revision 7259)
+++ gnu/commonlisp/lang/Lisp2.java	(working copy)
@@ -61,7 +61,9 @@
      // function and variable position.
      if (decl.isAlias())
        return FUNCTION_NAMESPACE+VALUE_NAMESPACE;
-    return decl.isProcedureDecl() ? FUNCTION_NAMESPACE :  
VALUE_NAMESPACE;
+    return (decl.isProcedureDecl() ||
+            decl.getFlag(Declaration.IS_SYNTAX)) ?
+      FUNCTION_NAMESPACE : VALUE_NAMESPACE;
    }

    /** Get a symbol for a given (interned) Java string. */

which works but is ugly. Methinks we need either an isSyntaxDecl()  
method,
or we need Declaration.PROCEDURE to be public, for better consistency.

-Jamie

--
Jamison Hope
The PTR Group
www.theptrgroup.com