Newsgroups: comp.lang.lisp
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news4.ner.bbnplanet.net!news.ner.bbnplanet.net!news3.near.net!paperboy.wellfleet.com!news-feed-1.peachnet.edu!gatech!newsfeed.internetmci.com!in2.uu.net!netnews.jhuapl.edu!aplcenmp!hall
From: hall@aplcenmp.apl.jhu.edu (Marty Hall)
Subject: Re: Functional Languages and Caches
Message-ID: <DoxILM.zp@aplcenmp.apl.jhu.edu>
Organization: JHU/APL Research Center, Hopkins P/T CS Faculty
References: <1996Mar22.205645.12160@wavehh.hanse.de> <4iq667$nkv@csugrad.cs.vt.edu> <4ja20f$ki3@tandem.CAM.ORG>
Date: Wed, 27 Mar 1996 13:53:45 GMT
Lines: 78

In article <4ja20f$ki3@tandem.CAM.ORG> Vassili Bykov <vbykov@cam.org> writes:

>I think Jon's point was different.  It was not that caching can
>violate functional style.  The problem is that if a function does have
>state, we cannot cache the value it returns using only the function's
>argument tuple as a cache lookup key.

The problem is even worse than this. Suppose that the procedure really
is a true function (output completely and deterministically determined
by input) with no side effects. However, suppose that someone calling
the routine does destructive operations on the returned
value. Memoizing the function may now break something that worked
before. For instance, imagine that Calculate-Values is a true function
that returns a list of numbers given some inputs. Imagine that
Normalize-Values uses DELETE to throw away the highest and lowest
values.

Before memoization (Normalize-Values (Calculate-Values <Args>))
might have been perfectly safe is Calculate-Values consed a fresh list
each time. But after memoizing Calculate-Values each call would get a
shorter list. 

>Technically, such caching is trivial, and a Scheme version is given in
>SICP, I believe.

SICP gives a basic sketch, but with no particular implementation of
the cache. The obvious way to do the cache is via hashing, and
Norvig's PAIP gives such an implementation in Common Lisp.

A much more complete library to do this in Common Lisp can be found at 
<http://www.apl.jhu.edu/~hall/lisp/Memoization>. This includes a bunch
of routines for evaluating the benefits of memoization, saving tables
to disk to be reloaded in later sessions, etc. The top portion of
<http://www.apl.jhu.edu/~hall/lisp/Memoization.lisp> describes the
routines briefly. Slightly more detail is given in
<http://www.apl.jhu.edu/~hall/lisp/Memoization/Users-Guide-Draft.ps>
Links to this source code and to a paper on the subject can all be
found on my Lisp WWW page at <http://www.apl.jhu.edu/~hall/lisp.html>

Although the basic implementation is indeed straightforward, I think
that saying the implementation is "trivial" is a bit of an
overstatement. For instance, doing inexact matches against the cache
(as Michie originally proposed) without losing the O(1) behavior of
hashing is a bit of a challenge. Furthermore, limiting the size of the
caches in a useful way (e.g. based on frequency of use) without
significant performance penalties is also non-trivial. Also, how does
one notice when you are memoizing when you are not supposed to (side
effects, destructive operations)? Or how do you tell if a persistent
cache is now obsolete due to changes in subfunctions called by the
memoized routines?

None of this is to say that automatic memoization is not useful. On
the contrary, when we applied it widely on a large DARPA program we
were surprised at how widely applicable it was, how many new
applications we discovered for it, and what a large payoff we got.
But although the payoffs were quite large, you need to be aware of the
pitfalls to use if effectively.

>Of course, it is the same old time/space tradeoff.

This is certainly frequently true. But memoizing a routine that
generates a bunch of temporary storage to perform its calculations has
the beneficial effect of reducing this consing in later calls if you
get cache hits. In the DARPA program mentioned above, we provided the
automatic memoization library to a bunch of developers from 3 or 4
different companies who were working on the system. Later, we went
back and evaluated the benefits memoization had provided. The time
improvement was large (over 100 fold in the total system), but what
surprised us was that memoization reduced consing by more than 1000
fold. Of course this is only in our particular application which
needed to allocate a lot of temporary storage. And of course the same
speedups could have been obtained by other approaches. But the results
were interesting nonetheless. If a technique is convenient, lots of
people will try it. If not, it will be used a lot less frequently.

Cheers-
						- Marty
(proclaim '(inline skates))
