Newsgroups: comp.lang.lisp
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!pipex!uunet!franz.com!franz!smh
From: smh@Franz.COM (Steve Haflich)
Subject: Re: keywords in Common Lisp and (MIT) Scheme
In-Reply-To: charest@underdog.jpl.nasa.gov's message of 25 Jan 1995 16:57:59 -0800
Message-ID: <SMH.95Jan28100149@vapor.Franz.COM>
Sender: news@franz.com
Nntp-Posting-Host: vapor
Organization: Franz Inc., Berkeley, CA
References: <19950125T070519Z.enag@naggum.no> <3g6s2n$2aa@underdog.jpl.nasa.gov>
Date: Sat, 28 Jan 1995 18:01:49 GMT
Lines: 58

In article <3g6s2n$2aa@underdog.jpl.nasa.gov> charest@underdog.jpl.nasa.gov (Len Charest) writes:

   From: charest@underdog.jpl.nasa.gov (Len Charest)

   In article <19950125T070519Z.enag@naggum.no>,
   Erik Naggum  <erik@naggum.no> wrote:
   >a trailing package marker (:) in a symbol, as in foo:, causes `read' to
   >signal an error, as CLtL2 (p 521) indicates it should.  however, ANSI CL
   >2.3.5 (5) says undefined patterns including the package marker are reserved
   >for implementation-dependent use.  well, that's not entirely bad if I can
   >find a way to do it.

   CLtL2, page 253 agress with the ANSI doc regarding trailing package
   markers.

   >is there a reasonable way I can convince the Lisp reader that a trailing
   >colon is not really a major disaster, but largely equivalent to a keyword?

   Unfortunately, the 'character attributes' of the colon are hard-wired
   into the Common Lisp reader.

Yes, this doesn't prevent portable code from assigning an
interpretation to any reserved reader syntax, although it's a litle
more work than one would like.

First, make sure you understand the details how the CL reader works,
and particularly that it processes extended tokens in two distinct
phases.  First a token is collect (CLtL Section 22.1.1) and then it is
parsed into a number of symbol (CLtL Section 22.1.2).

To change the way symbols are parsed, you indeed need to make
readtable changes.  Specifically, change the dispatch of every escape
or constituent character to a new reader macro that you write.  This
macro must duplicate the behavior of the token-collecting phase,
i.e. rules 4 through 10 in section 22.1.1.  Whether you cheat on
escape conventions etc. is up to you, but the algorithm is simple to
write just by following the given rules carefully.

Once an entire token has been collected, examine the collected token
string and see if its syntax conforms to the extension you are
implementing.  If so, do it.  If not, lambda-bind *READTABLE* back to
the original readtable and call READ-FROM-STRING on the token.  This
trick saves you from having to reimplement the really difficult parts
of the reader, especially all the number constructing code.

IMO it is unfortunate that the token-collection phase and the
token-parsing phases were not separated and made visible by being tied
to exported, named functions.  That would have eliminated the useless
duplication of the token-colleciton algorithm.

This strategy is awkward (aside from the implementation effort) only
in that by using multiple readtables, it prevents subsequent
modification of `the' readtable by other unrelated applications in the
same lisp image.  But this is true any time an application makes
readtable changes for any purpose.  The CL readtable mechanism is
elegant in the ability instantaneously to lambda bind the whole
readtable, but deficient in facilities for independent incremental
customizations.
