Newsgroups: alt.lang.design,comp.lang.c++,comp.lang.lisp
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!newshost.marcam.com!charnel.ecst.csuchico.edu!waldorf.csc.calpoly.edu!kestrel.edu!mcdonald
From: mcdonald@kestrel.edu (Jim McDonald)
Subject: Re: Comparing productivity: LisP against C++ (was Re: Reference Counting)
Message-ID: <1994Dec21.032901.23929@kestrel.edu>
Sender: mcdonald@kestrel.edu (Jim McDonald)
Organization: Kestrel Institute, Palo Alto, CA
References: <19941203T221402Z.enag@naggum.no> <BUFF.94Dec15103904@pravda.world> <D0xAIp.3Dn@rheged.dircon.co.uk> <vrotneyD11MDv.Ks7@netcom.com> <vogtD12y8D.HLL@netcom.com> <3d5alh$6j7@celebrian.otago.ac.nz>
Date: Wed, 21 Dec 1994 03:29:01 GMT
Lines: 319
Xref: glinda.oz.cs.cmu.edu comp.lang.c++:104290 comp.lang.lisp:16122


Note: My reply is rather long because I've included transcripts for three
      sessions, including one testing a program that is substantially more 
      capable than Nick requested.
      The functions and files alluded to here are available on request
      (not that I think they're worth much) to avoid making this post
      impossibly long.

In article <3d5alh$6j7@celebrian.otago.ac.nz>, nmein@bifrost.otago.ac.nz (Nick Mein) writes:
|> 
|> Richard Billington, Simon Brooke, William Vrotney, Jim McDonald and Christopher
|> Vogt have written that programming in Lisp is 2 - 10 time more productive
|> than programming in C (two of the posters professed to having little 
|> experience with C++). Surprisingly, no suggestion is made that this 
|> productivity advantage is restricted to any particular problem domain (perhaps
|> Lisp is 10x more productive for an NLP system, but only 2x more productive
|> if you are writing a Unix device driver?).
|> 
|> Anyway, I found the claims rather surprising, so here is a challenge. The
|> following toy program loads a 256x256 8 bit grayscale image from a file, 
|> inverts it, and saves it to another file. I consider myself to be an
|> intermediate-level C++ programmer. The program took me an hour (almost to
|> the minute) to design, code, debug and test (Ok, I didn't test it very much).
|> I look forward to seeing the previous posters' Lisp implementations of the
|> same. It should only take them 6 - 30 minutes to knock them together, less
|> if they are reasonably experienced in Lisp.
|> 

[text of toy C++ prgram deleted]

There are at leas a couple of problems with this challenge.

The main problem is that you have given an overly specified problem.
 
   Better would be to say something like this:

   Read an N dimensional image from a file where the byte size and 
   dimensions are given by data in the file.
    
   Perform a series of operations on that image.

   Save the result to another file.

   For simple test cases, use one inversion to see that a different
   file results, and two inversions to get a fixpoint.

The second problem is that the task you chose to measure is very low
level and doesn't really use any interesting features of a language--
it's almost nothing more than a few calls to the OS.

But, lest I be blamed for whining, I'll do three versions:
one very simple and close to your example, a second slightly
more abstract, and a third closer to the problem I've given
above.

TRANSCRIPT FOR SIMPLE VERSION 
Total elapsed time: 4 minutes, 8 seconds to write and (crudely) test.

.sa_394. date
Tue Dec 20 16:36:30 PST 1994
.sa_395. ./bin/lisp
[startup messages elided...]

> (defun invert-image (in-file out-file)
    (with-open-file (in-stream in-file :element-type '(unsigned-byte 8))
     (with-open-file (out-stream out-file :element-type '(unsigned-byte 8)
                       :direction :output)
       (dotimes (i 256)
         (dotimes (j 256)
           (write-byte (- 255 (read-byte in-stream)) out-stream))))))
INVERT-IMAGE
> (compile *)
;;; You are using the compiler in DEVELOPMENT mode (compilation-speed = 3)
;;; If you want faster code at the expense of longer compile time,
;;; you should use the production mode of the compiler, which can be obtained
;;; by evaluating (proclaim '(optimize (compilation-speed 0)))
;;; Generation of full safety checking code is enabled (safety = 3)
;;; Optimization of tail calls is disabled (speed = 2)
INVERT-IMAGE
> (with-open-file (s "/tmp/hack" :element-type '(unsigned-byte 8) :direction :output)
    (dotimes (i 256) (dotimes (j 256) (write-byte 11 s))))
NIL
> (time (invert-image "/tmp/hack" "/tmp/hack2"))
Elapsed Real Time = 4.85 seconds
Total Run Time    = 4.78 seconds
User Run Time     = 4.77 seconds
System Run Time   = 0.01 seconds
Process Page Faults    =         47
Dynamic Bytes Consed   =          0
Ephemeral Bytes Consed =      3,472
NIL
> (time (invert-image "/tmp/hack2" "/tmp/hack3"))
Elapsed Real Time = 5.03 seconds
Total Run Time    = 4.79 seconds
User Run Time     = 4.77 seconds
System Run Time   = 0.02 seconds
Process Page Faults    =         18
Dynamic Bytes Consed   =          0
Ephemeral Bytes Consed =      3,424
NIL
> (quit)
.sa_396. diff /tmp/hack /tmp/hack3
.sa_397. date
Tue Dec 20 16:40:38 PST 1994
.sa_398. 

SECOND VERSION 
I used parameterized types and sizes, 2D arrays, and added an
explicit test routine.   This time the code was written in a
file (instead of at the lisp listener as above) and compiled.
I compiled it a second time with the optimizing compiler but
the test showed very little timing improvement in this case.
Note that the test uses /bin/diff to compare the resulting files.
Total elapsed time: 9 minutes, 26 seconds.

.sa_399. date
Tue Dec 20 16:40:39 PST 1994
.sa_400. ./bin/lisp
[startup messages elided...]
[preparation of hack.lisp not shown here]
> (load (compile-file "~/hack.lisp"))
;;; You are using the compiler in DEVELOPMENT mode (compilation-speed = 3)
;;; If you want faster code at the expense of longer compile time,
;;; you should use the production mode of the compiler, which can be obtained
;;; by evaluating (proclaim '(optimize (compilation-speed 0)))
;;; Generation of full safety checking code is enabled (safety = 3)
;;; Optimization of tail calls is disabled (speed = 2)
;;; Reading source file "/usr/home/kestrel/mcdonald/hack.lisp"
;;; Writing binary file "/usr/home/kestrel/mcdonald/hack.sbin"
;;; Loading binary file "/usr/home/kestrel/mcdonald/hack.sbin"
#P"/usr/home/kestrel/mcdonald/hack.sbin"
> (prepare-test "/tmp/hack1")
NIL
> (test "/tmp/hack1" "/tmp/hack2" "/tmp/hack3")
Elapsed Real Time = 6.18 seconds
Total Run Time    = 6.11 seconds
User Run Time     = 6.05 seconds
System Run Time   = 0.06 seconds
Process Page Faults    =         19
Dynamic Bytes Consed   =    131,088
Ephemeral Bytes Consed =      5,216
Elapsed Real Time = 5.97 seconds
Total Run Time    = 5.92 seconds
User Run Time     = 5.88 seconds
System Run Time   = 0.04 seconds
Process Page Faults    =         18
Dynamic Bytes Consed   =    131,088
Ephemeral Bytes Consed =      5,168


Diff1:

(NIL NIL 0 NIL) 

Diff2:
Binary files /tmp/hack1 and /tmp/hack2 differ

(NIL NIL 1 NIL) 
NIL
> (proclaim '(optimize (speed 3) (safety 0) (compilation-speed 0)))
T
> (load (compile-file "~/hack.lisp"))
;;; You are using the compiler in PRODUCTION mode (compilation-speed = 0)
;;; If you want shorter compile time at the expense of reduced optimization,
;;; you should use the development mode of the compiler, which can be obtained
;;; by evaluating (proclaim '(optimize (compilation-speed 3)))
;;; Generation of runtime error checking code is disabled (safety = 0)
;;; Optimization of tail calls is enabled (speed = 3)
;;; Reading source file "/usr/home/kestrel/mcdonald/hack.lisp"
;;; Writing binary file "/usr/home/kestrel/mcdonald/hack.sbin"
;;; Loading binary file "/usr/home/kestrel/mcdonald/hack.sbin"
#P"/usr/home/kestrel/mcdonald/hack.sbin"
> (test "/tmp/hack1" "/tmp/hack2" "/tmp/hack3")
Elapsed Real Time = 5.40 seconds
Total Run Time    = 5.33 seconds
User Run Time     = 5.27 seconds
System Run Time   = 0.06 seconds
Process Page Faults    =         12
Dynamic Bytes Consed   =    131,088
Ephemeral Bytes Consed =      5,184
Elapsed Real Time = 5.14 seconds
Total Run Time    = 5.09 seconds
User Run Time     = 5.06 seconds
System Run Time   = 0.03 seconds
Process Page Faults    =         12
Dynamic Bytes Consed   =    131,088
Ephemeral Bytes Consed =      5,168


Diff1:

(NIL NIL 0 NIL) 

Diff2:
Binary files /tmp/hack1 and /tmp/hack2 differ

(NIL NIL 1 NIL) 
NIL
> (quit)
.sa_401. date
Tue Dec 20 16:50:05 PST 1994

THIRD VERSION (Timing is problematic since I had to deal with many
other things [like work!] interleaved with this -- estimate is 30 
minutes during an elapsed time of about 90 minutes.)

This version has a file format that lets you specify the byte size
and the dimensions of the image (e.g. 256*256, 100*100*8, etc.).
There is a reasonably general mapping function that lets you
apply a sequence of functions to the locations in one image to
pointwise produce a new image.  

The test scenario for this version is
 (1) Make an image file of size 256 * 256 * 4 with 16-bit bytes,
     initialized with 1's for all 262,144 elements.
     Note that the image dimensions and byte size are parameters 
     passed to the routines to be tested.
 (2) Read that image file.
 (3) Create a partial inversion of the image by inverting all the 
     elements in the first plane (i.e., at positions whose third 
     index is 0), but keep the 3 other planes intact.
 (4) Write the result back out to a second file.
 (5) Repeat 2,3,4 starting with the second file, to get an image 
     in a third file that is a partial inversion of the second file.
     This file should be equivalent to the first file.
 (6) Repeat 2,3,4 but for step 3 use the inversion function twice 
     before writing the fourth file, which also should be 
     equivalent to the first file.

It should also take less than a minute to prepare and start other
tests (e.g., compute a new value at x,y that averages the 9 values 
found at the locations around x,y, or compute the gradient at each
point from the old image and store it in planes 2 and 3, etc), 
plus maybe a minute to run them.  

So the challenge back to you is to provide a C++ version with 
that functionality.  Can you do it in less than an hour?

[preparation of hack3.lisp occurred bere]
.sa_411. date
Tue Dec 20 18:20:43 PST 1994
.sa_412. ./bin/lisp

lisp startup messages elided...

> (load (compile-file "hack3.lisp"))
;;; You are using the compiler in DEVELOPMENT mode (compilation-speed = 3)
;;; If you want faster code at the expense of longer compile time,
;;; you should use the production mode of the compiler, which can be obtained
;;; by evaluating (proclaim '(optimize (compilation-speed 0)))
;;; Generation of full safety checking code is enabled (safety = 3)
;;; Optimization of tail calls is disabled (speed = 2)
;;; Reading source file "hack3.lisp"
;;; Writing binary file "hack3.sbin"
;;; Loading binary file "hack3.sbin"
#P"/usr/home/kestrel/mcdonald/hack3.sbin"

> (time (prepare-test "/tmp/jjj.xxx" '(unsigned-byte 16) '(256 256 4)))
Elapsed Real Time = 18.83 seconds
Total Run Time    = 18.46 seconds
User Run Time     = 18.19 seconds
System Run Time   = 0.27 seconds
Process Page Faults    =         81
Dynamic Bytes Consed   =    524,296
Ephemeral Bytes Consed = 11,280,720
There were 21 ephemeral GCs
NIL

> (test "/tmp/jjj.xxx" "/tmp/jjj.two" "/tmp/jjj.three" "/tmp/jjj.four")

[copying jjj.xxx to jjj.two, inverting first plane] 
Elapsed Real Time = 57.87 seconds
Total Run Time    = 56.91 seconds
User Run Time     = 56.45 seconds
System Run Time   = 0.46 seconds
Process Page Faults    =         70
Dynamic Bytes Consed   =  1,048,712
Ephemeral Bytes Consed = 41,960,424
There were 80 ephemeral GCs

[copying jjj.two to jjj.three inverting first plane] 
Elapsed Real Time = 59.70 seconds
Total Run Time    = 57.48 seconds
User Run Time     = 57.11 seconds
System Run Time   = 0.37 seconds
Process Page Faults    =        117
Dynamic Bytes Consed   =  1,082,288
Ephemeral Bytes Consed = 41,960,624
There were 2 dynamic GCs
There were 81 ephemeral GCs

[copying jjj.one to jjj.four inverting inversion of first plane] 
Elapsed Real Time = 81.40 seconds (1 minute, 21.40 seconds)
Total Run Time    = 78.81 seconds (1 minute, 18.81 seconds)
User Run Time     = 77.85 seconds (1 minute, 17.85 seconds)
System Run Time   = 0.96 seconds
Process Page Faults    =         68
Dynamic Bytes Consed   =  2,104,904
Ephemeral Bytes Consed = 61,626,624
There were 2 dynamic GCs
There were 118 ephemeral GCs


Diff 1 2 -- File 2 is inversion of file 1
Binary files /tmp/jjj.xxx and /tmp/jjj.two differ

(NIL NIL 1 NIL) 

Diff 1 3 -- File 3 is inversion of file 2

(NIL NIL 0 NIL) 

Diff 1 4 -- File 4 is inversion of inversion of file 1

(NIL NIL 0 NIL) 
NIL
> (quit)
.sa_413. date
Tue Dec 20 18:27:36 PST 1994
