Newsgroups: comp.lang.scheme
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!oitnews.harvard.edu!purdue!lerc.nasa.gov!magnus.acs.ohio-state.edu!math.ohio-state.edu!howland.reston.ans.net!newsfeed.internetmci.com!news.mathworks.com!uunet!in1.uu.net!siemens!scr.siemens.com!bap
From: Barak.Pearlmutter@scr.siemens.com (Barak Pearlmutter)
Subject: Re: Is Garbage Collection faster?
In-Reply-To: Barak.Pearlmutter@scr.siemens.com's message of Fri, 13 Oct 1995 00:16:59 GMT
Message-ID: <BAP.95Oct13030733@scr.siemens.com>
Sender: news@scr.siemens.com (NeTnEwS)
Nntp-Posting-Host: gull.scr.siemens.com
Reply-To: Barak.Pearlmutter@scr.siemens.com
Organization: Siemens Corporate Research
References: <DFzHB6.654@cogsci.ed.ac.uk>> <KANDERSO.95Oct7105511@lager.bbn.com>
	<45h1g6$g37@news.parc.xerox.com> <45jhjo$cdd@camelot.ccs.neu.edu>
	<hbaker-1210951409350001@10.0.2.15> <BAP.95Oct12201659@scr.siemens.com>
Date: Fri, 13 Oct 1995 07:07:33 GMT
Lines: 113

On second though I suppose I should not have been quite so flip.

Let us suppose that it is advantageous for a garbage collector to have
an estimate of the expected lifetime of each object.  The simplest
possible assumption is that object death is a Poisson process, in
other words that lifetimes are distributed exponentially,

 Pr[lifetime = x] = p(x) = 1/T exp -x/T

where we denote by p(x) the p.d.f. of lifetimes, and where T > 0 is
the expected lifetime, since int_0^inf x p(x) = T.  Under these
conditions, an object's expected future lifetime is not a function of
its age, so generational collection would confer no benefits.

A slightly more complicated assumption would be that object lifetimes
are drawn from a mixture model of exponentials,

 p(x) = sum_i a_i p_i(x)

where the a_i = Pr[object drawn from class i]
are the priors, so sum_i a_i = 1,
and the p.d.f.'s of the class-conditional lifetimes are
p_i(x) = 1/T_i exp -x/T_i,
in which T_i > 0 is the expected lifetime of an object drawn from class i.

This gives an expected lifetime of

 E[lifetime] = sum_i Pr[object drawn from i] E[lifetime | from class i]
	     = sum_i a_i T_i

If an object whose lifetime was drawn from this distribution p(x) has
already survived for a length of time w, how much longer can we expect
it to live?

The posterior probability of having been drawn from class i can be
computed in closed form, because the conditional memorylessness of the
distribution allows us to apply Bayes rule easily.  These are

 Pr[object drawn from i | already lived for w] = a_i z_i / Z

where z_i = Pr[surviving past age w | from class i]
= int_w^inf p_i(x) dx = exp - w/T_i,
and Z = sum_i a_i z_i
is the denominator of Bayes's rule, which serves to normalize the
distribution. Again the memorylessness of exponential distributions
comes into play, allowing us to say that the expected additional
lifetime, which we denote f(w), is

 f(w) = E[time left to live | already lived for w]
      = sum_i Pr[object drawn from i | already lived for w]
	      E[time left to live | already lived for w & from class i]
      = sum_i Pr[object drawn from i | already lived for w]
	      E[lifetime | from class i]
      = (1/Z) sum_i z_i a_i T_i

This is simple to compute given a set of T_i's in hand, and it might
even be worthwhile to see what it looks like for some reasonable T_i's
drawn from fitting a sum of exponentials to an empirical distribution
of object lifetimes.  But even without knowing the particular T_i's,
there is one thing we can say.

It would be interesting to know that the longer an object has lived,
the longer its future expected lifetime is, regardless of the
particular T_i's in force.  This amounts to showing that df/dw >= 0,
so let us compute

  df/dw = (d/dw) (1/Z) sum_i z_i a_i T_i
	= - ((dZ/dw) / Z^2) sum_i z_i a_i T_i + (1/Z) sum_i a_i T_i dz_i/dw

(here compute the terms

 dz_i/dw = (d/dw) exp - w/T_i = - (1/T_i) exp - w/T_i = - z_i/T_i

 dZ/dw = sum_i a_i dz_i/dw = sum_i - z_i/T_i = - sum_i z_i/T_i

and plug them in)

  df/dw = (sum_i z_i/T_i) (sum_i z_i a_i T_i) / Z^2 - (sum_i a_i z_i) / Z
	= (sum_i z_i/T_i) f(w) / Z - 1

We wish to show that this cannot be negative, or that

 (sum_i z_i/T_i) f(w) >= Z
 (sum_i z_i/T_i) (sum_i z_i a_i T_i) >= Z^2

which expands to

 sum_ij (T_i/T_j) a_i a_j z_i z_j >= sum_ij a_i a_j z_i z_j

starting with the left hand side,

 sum_ij (T_i/T_j) a_i a_j z_i z_j
	= sum_i (a_i z_i)^2 + sum_i<j (T_i/T_j + T_j/T_i) a_i a_j z_i z_j

since (T_i/T_j + T_j/T_i) is of the form (r+1/r) with r>0,
(T_i/T_j + T_j/T_i) >= 2, so

	>= sum_i (a_i z_i)^2 + sum_i<j 2 a_i a_j z_i z_j
	= sum_ij a_i a_j z_i z_j

QED.

So we seem to have proved a

THEOREM: under an exponential lifetime mixture distribution of object
lifetimes, an object's expected remaining lifetime only grows with time.

This appears to justify generational garbage collection schemes, and
might even form the theoretical basis of an algorithm that first fits
a set of T_i's to the data available to the garbage collector by
virtue of its operation, and then allocates generations and decides
when to collect and how much space to allocate based on these
empirical estimated T_i's.
