Newsgroups: comp.lang.scheme
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!news.sprintlink.net!siemens!princeton!news.princeton.edu!blume
From: blume@dynamic.cs.princeton.edu (Matthias Blume)
Subject: Re: factorial algorithm
In-Reply-To: rockwell@nova.umd.edu's message of 9 Nov 1994 13:20:47 -0500
Message-ID: <BLUME.94Nov9164643@dynamic.cs.princeton.edu>
Originator: news@hedgehog.Princeton.EDU
Sender: news@Princeton.EDU (USENET News System)
Nntp-Posting-Host: dynamic.cs.princeton.edu
Organization: Princeton University
References: <39l7er$q02@wsiserv.informatik.uni-tuebingen.de>
	<1994Nov8.173643.17118@njitgw.njit.edu> <39peot$4to@larry.rice.edu>
	<ROCKWELL.94Nov9132045@nova.umd.edu>
Date: Wed, 9 Nov 1994 21:46:43 GMT
Lines: 133

In article <ROCKWELL.94Nov9132045@nova.umd.edu> rockwell@nova.umd.edu (Raul Deluth Miller) writes:

   David Richter:
   . If I remember my algorithms correctly, the brute force search
   . quickly gets unduly expensive (there are _so_ many possibilities);
   . the solution presented used dynamic programming.

   What's a "brute force search"?

   The way to minimize the size of intermediate results in matrix
   multiply is get rid of the biggest dimensions first.  Another phrasing
   is: perform the multiplications with the smallest dimension's last.

I think the original problem was to minimize the total number of
single multiplications (multiply an ab matrix with a bc matrix --
you get an ac matrix each element of which requires b single
multiplications, therefore you need abc of them in total (we are
using the nave algorithm for each individual matrix multiply, not
Strassen's algorithm or such something even better)).

This problem is probably the one that's used most often in algorithms
classes as an introduction to dynamic programming.  Instead of trying
all possible ways to put parentheses into the expression (there are
C(n) such ways, where C(n) is the `Catalan' number of n -- a quantity
that grows exponetially fast), you solve the problem for short
sub-products first and remember the results in a table.  This results
in an O(n^3) algorithm, which is much faster than the O(C(n)) brute
force approach (even though O(n^3) algorithms are not particularly
fast for practical purposes).

Note that you arrive at the same solutiuon if you use a memoizing
version of the brute-force algorithm.

   This corresponds to a simple post-order tree walk.  You have a list of
   dimension pairs to be combined, you arbitrarily pick from the smallest
   -- this partitions the rest into two pieces.  Then recursively apply
   this mechanism to each of the pieces.  On the way back, you perform
   matrix multiplies.

   I think that ordering of multiplication of bignums would require the
   reverse criteria -- it seems like you'd want to perform the largest
   multiplications last.

This algorithm does not solve the original poster's problem (optimally).

   Hmm... how about the ordering of matrix multiplies with matrices of
   bignums?  Seems like this might be a packing problem?

Maybe we should first go and solve the problem at hand before going
way over the top...

---- DYNAMIC PROGRAMMING ALGORITHM FOR OPTIMAL MATRIX GROUPING ---
(define (opt v)

  ;; n matrices (numbered 1..n)
  ;; dimension of matrix i is (v[i-1] x v[i])

  ;; result is a pair of the form (cost . splitpath)
  ;; with cost being the number of multiplications required and
  ;; splitpath being a tree structure of the form:
  ;;  splitpath := #f             -- for the leafs (single matrices)
  ;;             | (k splitpath splitpath)
  ;;    where k is the ``splitpoint'' (index of the matrix to the
  ;;                                   right of the )
  ;;    and the two sub-splitpaths are the splitpaths for the left and right
  ;;    sub-products, respectively

  (let* ((n+1 (vector-length v))
	 (n (- n+1 1))
	 (2n-1 (+ n n -1))		; for index calculation
	 (a-size (/ (* n+1 n) 2))	; size for triangular matrix
	 (costs (make-vector a-size 0))
	 (paths (make-vector a-size #f)))

    ;; index computation for triangular matrix
    (define (idx i j)			; always 0 <= i < j <= n
      (+ (/ (* i (- 2n-1 i)) 2) j -1))

    (define (a-ref a i j)
      (vector-ref a (idx i j)))

    (define (a-set! a i j x)
      (vector-set! a (idx i j) x))

    ;; compute costs[i1][j1] under the assumption that all costs[i][j]
    ;; with j - i < j1 - i1 have already been calculated
    ;; register best choice in paths[i][j]

    (define (find-best i j)

      ;; cost for splitting as (i,k)(k,j), i < k, k < j
      (define (split-cost k)
	(+ (a-ref costs i k)       ; cost of left subproduct (already in table)
	   (a-ref costs k j)       ; cost of right subproduct (already ...)
	   (* (vector-ref v i)     ; cost of multiplying the two
	      (vector-ref v k)
	      (vector-ref v j))))

      ;; this is a straight-forward minimum-finding algorithm
      (let loop
	  ((k (+ i 2))
	   (best-k (+ i 1))
	   (best-cost (split-cost (+ i 1))))
	(if (>= k j)
	    (begin
	      (a-set! costs i j best-cost)
	      (a-set! paths i j
		      (list best-k
			    (a-ref paths i best-k)
			    (a-ref paths best-k j))))
	    (let ((cost (split-cost k)))
	      (if (< cost best-cost)
		  (loop (+ k 1) k cost)
		  (loop (+ k 1) best-k best-cost))))))

    ;; fill in all the stuff for a given gap size (g = j - i)
    (define (gaps g)
      (let ((max-i (- n g)))
	(do ((i 0 (+ i 1)))
	    ((> i max-i) 'done)
	  (find-best i (+ i g)))))

    ;; go through all possible gap sizes (g = 2 .. n)
    (do ((g 2 (+ g 1)))
	((> g n) (cons (a-ref costs 0 n) (a-ref paths 0 n)))
      (gaps g))))
---- END ----

This is certainly not well-tuned, but you get the idea...

Regards
--
-Matthias
