Newsgroups: comp.lang.lisp
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!news.sprintlink.net!howland.reston.ans.net!usc!sdd.hp.com!night.primate.wisc.edu!aplcenmp!hall
From: hall@aplcenmp.apl.jhu.edu (Marty Hall)
Subject: Re: Help
Message-ID: <DFtpHG.GIH@aplcenmp.apl.jhu.edu>
Organization: JHU/APL AI Lab, Hopkins P/T CS Faculty
References: <445t0j$skl@umbc9.umbc.edu> <DFKx3r.75z@inesc.pt> <KANDERSO.95Sep29123338@lager.bbn.com>
Date: Mon, 2 Oct 1995 13:29:40 GMT
Lines: 72

In article <KANDERSO.95Sep29123338@lager.bbn.com>
kanderso@lager.bbn.com (Ken Anderson) writes:
>Becareful when doing such experiments since results can be misleading.

Ken has an award-winning (*) paper on Lisp benchmarking/profiling that
I would recommend, except that the FTP site where I got it
(wheaton.bbn.com) is no longer accessible.

(*) Cleverest-title Award: "Courage in Profiles" :-)

>Generally both your test DEPTH2  function  and the function that calls TIME
>should be compiled.  The test  function should iterated enough time to make
>the times reported meaningful.

On the other hand, this is a nontrivial matter also. If the function
you are timing is small, a LOOP can skew your results, since the LOOP
overhead is included in your timing. Furthermore, if you compile a
LOOP, the compiler is free to take out things that don't affect the
result. For instance, (loop repeat 10000 do (* Var1 (+ Var2 (sqrt Var3))))
could be taken out altogether at compile time and replaced by NIL.

What I typically use is a macro that expands into a PROGN repeating
the function call N times. Here it is:

;;; Part of http://www.apl.jhu.edu/~hall/lisp/Simple-Metering.lisp

;;;===========================================================================
;;; Time-Form: Takes a single form and optionally an integer N (default 20). It
;;; =========  runs the form N times and prints out the average time. Useful
;;;            for timing very small, fast functions when the time-step of
;;;            the builtin TIME function is too coarse.
;;;            > (Time-Form (Foo))     ; Call (Foo) 20 times + print avg time
;;;            > (Time-Form (Bar) 100) ; Call (Bar) 100 times + print avg time
;;; 1994 Marty Hall.

(defmacro Time-Form (Form &optional (Repetitions 20))
  "Runs FORM N times, printing avg execution time and returning FORM's value"
  (declare (optimize speed (safety 1) (compilation-speed 0)))
  `(let* ((Start (get-internal-run-time))
	  (Value (progn ,@(loop for I from 1 to Repetitions collecting Form)))
	  (Stop (get-internal-run-time)))
    (format t "~%Time to do ~S is ~0,5F sec."
     ',Form
     (float (/ (- Stop Start)
	       (* ,Repetitions internal-time-units-per-second))))
    Value))

Example: (pprint (macroexpand-1 '(Time-Form (Foo 1 2 3) 10)))
(LET* ((START (GET-INTERNAL-RUN-TIME))
       (VALUE
        (PROGN
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)
          (FOO 1 2 3)))
       (STOP (GET-INTERNAL-RUN-TIME)))
  (FORMAT T "~%Time to do ~S is ~0,5F sec." '(FOO 1 2 3)
          (FLOAT
           (/ (- STOP START) (* 10 INTERNAL-TIME-UNITS-PER-SECOND))))
  VALUE) 

Note that if you really care, you could subtract off the time of one
call to get-internal-run-time. But since this only happens once, not
for each N, I usually ignore it.
						- Marty
(proclaim '(inline skates))
