PRENTICE-HALL SCI ENCE R.G. Dromey How fo Solve it by Computer C.A.R. HOARE SERIES EDITORHOW TO SOLVE IT BY COMPUTER R. G. DROMEY Department of Computing Science The University of Wollongong ENGLEWOOD CLIFFS, NEW JERSEY LONDON NEW DELHI SINGAPUKE «= SYUNEY = GUKYU)— TURUNTU WELLINGTONLibrary of Congress Cataloging in Publication Daa DROMEY,R.G, — 1946- How 10 Soive it by Computer, Bibliography: p. Includes index 1. Mathematics—-Daia processing 2. Problem solving—Data processing 2, teetronic aigitat computers—rrogramming 1. Title. QA76.95.D76 S194 B1-19164 ISBN 0.13-433095.9 AACR? ISBN 0-13-434001-9 (phk.) British Library Cataloging in Publication Data DROMEY, R.G. How to Solve it by Computer. SU8 —QAS.58 ISBN 0-13-433995.9 ISBN 0-13.434001-9 (PBK) © 1982 by PRENTICE-HALL INTERNATIONAL, INC., London © 1982 by PRENTICE-HALL INC., Englewood Cliffs, NJ. 07632 All rights reserved. ‘No part of this publication may be reproduced, without the prior permission of the publisher TSRN Q-13-uaz99s-4 ISBN O-13-434001~95 {PBK} PRENTICE HALL INTERNATIONAL, INC, London PRENTICE-HALL OF AUSTRALIA PTY. LTD. Sydney PRENTICE-HALL CANADA. INC., Toronto PRENTICE-HALL OF INDIA PRIVATE LTD., New Delhi PRENTICE-HALL OF JAPAN, INC,, Tokyo PRENTICE-HALL OF SOUTHEAST ASIA PTE., TD., Singapore PRENTICE-HALL, INC,, Englewood Clif, New Jersey WHITEHALL BOOKS LIMITED, Wellimgion, New Zealand Printed in the United States of Amecica 10987654321This book is dedicated to George PolyaWe learn most when We have to invent PiagetCONTENTS PREFACE xili ACKNOWLEDGEMENTS xx INTRODUCTION TO COMPLITER PROBLEM-SOLVING 1 Introduction 1 2 ‘The Problem-solving Aspect 3 3 Top-down Design 7 4 Implementation of Algorithms 14 6 7 i L 1. 1 LS Progiam Ve ion 19 4.6 The Efficiency of Algorithms 29 1.7. The Analysis of Algorithms 33 Bibliography 39 FUNDAMENTAL ALGORITHMS 42 Introduction 42 Algorithm 2.1 Exchanging the Values of Two Variables 43 Algorithm 2,2. Counting 47 Algorithm 2.3 Summation of a Set of Numbers 51 Algorithm 2.4 Factorial Computation 56 Algorithm 2.5. Sine Function Computation 60 Algorithm 2.6 Generation of the Fibonacci Sequence 64 Algorithm 2.7 Reversing the Digits of an Integer 69 Algorithm 2.8 Base Conversion 74 Algorithm 2.9 Character to Number Conversion 80 Bibliography 84x CONTENTS 3 FACTORING METHODS: 85 intraduction 85 Algorithm 3.1 Finding the Square Root of a Number. 86 Algorithm 3.2. The Smallest Divisor of an Integer 92 Algorithm 3.3. The Greatest Common Divisor of Two inegers 97 Algorithm 3.4 Generating Prime Numbers 105 Algorithm 3.5 Computing the Prime Factors of an Integer 116 Algorithm 3.6 Generation of Pseudo-random Numbers 120 Algorithm 3.7. Raising a Number to a Large Power 124 Algorithm 3.8 Computing the nth Fibonacei Number 132 Bibliography 138 ARRAY TECHNIQUES 139 is 129 ithm 4.1 Aray Order Reversal 140 Algorithm 4.2 Array Counting or Histogramming 144 Algorithm 4.3 Finding the Maximum Number ina Set 147 Algorithm 4.4 Removal of Duplicates from an Ordered Alg Algorithm 4.5 joning an Array 156 Algorithm 4.6 Finding the kth Smailest Element 166 Algorithm 4.7 Longest Monotone Subsequence 174 Bibliography 180 MERGING, SORTING AND SEARCHING 181 Introduction 181 Algorithm 5.1 The Two-way Merge 182 Algorithm 5.2 Sorting by Selection 192 Algorithm 5.3 Sorting by Exchange 198 Algorithm 5.4 Sorting by Insertion 204 Algorithm 5.5 Sorting by Diminishing Increment 209 Algorithm 5.6 Sorting by Partitioning 216 Algorithm 5.7. Binary Search 227 Algorithm 5. Hash Scarching 237 Bibliography 246 TEXT PROCESSING AND PATTERN SEARCHING 248 Introduction 248 Algorithm 6.1 Text Line Length Adjustment 249 Algorithm 6.2 Left and Right Justification of Text 256 Algorithm 6.3 Keyword Searching in Text 267 Algorithm 6.4 Text Line Editing 274 Algorithm 6.5 Linear Pattern Search 282 Algorithm 6.6 Sublinear Pattern Search 293 Bibliography 303CONTENTS. xi DYNAMIC DATA STRUCTURE ALGORITHMS 304 Introduction 304 Algorithm 7.1 Stuck Operations 306 Algorithm 7.2. Queue Addition and Deletion 314 Algorithm 7.3 Linked List Search 375 Algorithm 7.4 Linked List Insertion and Deletion 331 Algorithm 7.5 Binary Tree Search 342 Algorithm 7.6 Binary Tree Insertion and Deletion 349 Bibliography 366 RECURSIVE ALGORITHMS 367 Introduction 367 Algorithm 81 Binary Tree Traversal 373 m8.2 Re st 383 Algorithm 83 Towers of anal Problem 391 Algorithm 8.4 Sample Generation 404 Algorithm 8.5 Combination Generation 414 Algorithm 8.6 Permutation Generation 422 graphy 433 INDEX = =—s_ «435,PREFACE The inspiration for this book has come from the classic work of Polya on general and mathematical problem-solving. As in mathematics, many beginners in computing science stumble not because they have difficulty with learning a programming language but rathcr because they are ill- prepared to handle the problem-solving aspects of the discipline. Unfortu- nately, the school system seems to concentrate on training people to answer questions and remember facts but not to really solve problems, in response to this situation, it was felt there was a definite need for a book written in the spirit of Polya’s work, but translated into the computing science context. Much of Polya's work is relevant in the new context but, with computing problems, because of their general requirements for itera- tive or recursive solutions. another dimension is added to the problem- solving process. If we can develop problem-solving skills and couple them with top- down design principles, we are well on the way to becoming competent at algorithm design and program implementation. Emphasis in the book has been placed on presenting strategies that we might employ to “discover” efficient, well-structured computer algorithms. Throughout, a conscious effort has been made to convey something of the flavor of either a personal dialogue or an instructor-student dialogue that might take place in the solution of a problem. This style of presentation coupled with a carefully chosen set of examples, should make the book attractive to a wide range of readers. The student embarking on a first course in Pascal should find that the material provides a lot of useful guidance in separating the tasks of learning how to develop computer aigorithms and of then impiementing thems in a programming language like Pascal. Too often. the end is confused with the xiixiv PREFACE means. A good way to work with the book for self-study is to read only as much as you need to get started on a problem. Then, when you have developed your own solution, compare it with the one given and consciously reflect on the strategies that were helpful or hindersome in tackling the problem. As Yohe! has rightly pointed out “programming [computer probiem-soiving] is an art andj as in aii art forms, each individual must develop a style which seems natural and genuine.” Instructors should also find the style of presentation very useful as lecture material as it can be used to provide the cues for a balanced amount of instructor-class dialogue. The author has found students receptive and responsive to lectures presented in this style. The home-computer hobbyist wishing to develop his or her problem- solving and programming skills without any formal lectures should find the completeness of presentation of the various algorithms a very valuable suppiement to any introductory Pascai text. itis hoped that even some of the more advanced algorithms have been rendered more accessible to beginners, than is usually the case. The material presented, although elementary for the most part, con- tains some examples and topics which should be of sufficient interest and challenge to spark, in the beginning student, the enthusiasm for computing that we so often see in students who have begun to master their subject. Readers are urged not to accept passively the algorithm solutions given. Wherever it is possible and practical, the reader should strive for his or her own simpier or betier aigoritims. As aiways, the limitations of space have meant that some topics and examples of importance have either had to be omitted or limited in scope. To some degree the discussions and supplemen- tary problems that accompany each algorithm are intended to compensate for this. The problem sets have been carefully designed to test, reinforce, and extend the reader's understanding of the strategies and concepts presented. Readers are therefore strongly urged to attempt a considerable proportion of these problems. Each chapter starts off with a discussion of relatively simple, graded examples, that are usually related in some fairly direct way. Towards the middle and end of each chapter, the examples are somewhat more involved. A good way to work with the book is to only consider the more fundamental algorithms in cach chapter at a first reading. This should allow the necessary build-up of background needed to study in detail some of the more advanced algorithms at a later stage. The chapters are approximately ordered in increasing order of concep- tual difficulty as usually experienced by students, The first chapter intro- + J.M. Yohe, “An overview of programming practices”, Comp. Surv., 6 221-46 (1974)PREFACE xv duces the core aspects of computer problem-solving and algorithm design. Ata first reading the last half of the chapter caa be skimmed through. Where possible, the ideas are reinforced with examples, The problem-solving dis- cussion tries to come to grips with the sort of things we can do when we are stuck with a problem. The important topic of program verification is pre- semed in a way that shouid de reiatively easy for the reader to grasp and apply at least to simple algorithms. Examples are used to convey something of the flavor of a probabilistic analysis of an algorithm. ‘The second chapter concentrates on developing the skill for formulating iterative solutions to problems—a skill we are likely to have had little experience with before an encounter with computers. Several problems are also considered which should allow us to come to a practical understanding of how computers represent and manipulate information. The problem of conversion from one representation to another is also addressed. The aigorithms in this chapter are aii treated as if we were faced with the task of discovering them for the first time. Since our aimis to foster skill in computer problem-solving, this method of presentation would seem to be appropriate from a pedagogical standpoint. Algorithms in the other chapters are also discussed in this style. In the third chapter, we consider a number of problems that have their origin in number theory. These algorithms require an extension of the iterative techniques developed in the first chapter. Because of their nature, most of these problems can be solved in a number of ways and can differ widciy in their computational cost. Confronting the question of efficiency adds an extra dimension to the problem-solving process. We set out to “discover” these algorithms as if for the first time. Atray-processing algorithms are considered in detail in Chapter 4. Facility with the use of arrays needs to be developed as soon as we have understood and are able to use iteration in the development of algorithms. Arrays, when coupled with iteration, provide us with a very powerful means of performing computations on collections of data that share some common attribute. Strategies for the efficient processing of array information raise some interesting problem-solving issues. The algorithms discussed are intended to address at least some of the more important of these issues. ‘Once arrays and iteration have been covered, we have the necessary tools to consider merging, sorting and searching algorithms. These topics are discussed in Chapter 5. The need to organize and subsequently efficiently search large volumes of information is central in computing science. A number of the most well-known internal sorting and searching algorithms are considered. An attempt is made to provide the settings that would allow us to “discover” these algorithms. ‘The sixth chapter on texi and suring processing represents a diversion from the earlier emphasis on “numeric” computing. The nature and organ-xvi PREFACE ization of character information raises some important new problem-solving issues, The demands for efficiency once again lead to the consideration of some interesting algorithms that are very different from those reviewed in earlier chapters. The algorithms in this chapter emphasize the need to develop the skill of recognizing the essentials of a problem without being confused or misied by extraneous det. ing’ some of these algorithms provides an interesting test-bed for us to apply some of the problem-solving skills we should have absorbed after covering the first five chapters. Chapter 7 is devoted to the fundamental algorithms for maintaining and searching the primary dynamic data structures (i.e. stacks, queues, lists, and binary trees). The issues in this chapter arise mostly at the implementation level and with the use of pointers, In the final chapter, the topic of recursion is introduced. A recursive call is presented as an extension of the conventional subprocedure cali. The simplest form of recursion, linear recursion, is only briefly treated because the view is taken that there are almost always simple iterative schemes that can be found for performing what amounts to linear recursion. Furthermore, linear recursion does little to cither convey anything of the power of recur- sion or to really deepen our understanding of the mechanism. The same cannot usually be said for problems involving either binary recursion or the more general non-linear recursion, Algorithms that belong to both these classes are discussed in detail, with special emphasis being placed on recog- nizing their underlying simijarities and deepening our undersianding of recursion mechanisms. As a concluding remark about the book, if the author were to hope that this work may achieve anything, it would be that it stimulated students and others to embark on a journey of discovering things for themselves! A SUGGESTION FOR INSTRUCTORS “...any idea or problem or body of knowledge can be presented in a form simple enough so that any particular Iearner can understand it in a recognizable form.” —Bruner. In comparison with most other human intellectual activities, computing is i its infancy despite the progress we seem to have made in such a short time. great in our rapidly changing world, we have not had time to sit back andA SUGGESTION FOR INSTRUCTORS | xvii reflect on the best way to convey the “computing concept” on a wide scale. In time, computing will mature and cyolve as a well-understood discipline with clear and well-defined methods for introducing the subject to begin- ners—but that is in the future. For the present, because we are not in this happy situation, it is most prudent to look to other mature disciplines for guidance on how we shouid introduce computing to beginners, Assuming we want to have the highest possible success rate in introducing computing to students (a situation which clearly at present does not prevail) and that we want to teach computing on the widest possible scale, then the most obvious discipline to turn to is that of learning to read and write. Educators have mastered the process of teaching people to read and write to the extent that these skills are able to be transmitted on a very wide scale. efficiently, and with a very high rate of success. Putting aside the various methods within the learning-to-tead-and-write discipline we see that there are some fundamentai principies acknowledged by aii methods that are highly relevant to teaching computing. In teaching people to read and write a very substantial time (years in fact) is devoted to reading. Only after such preparation is it considered appropriate that they should attempt to write stories or essays, or books, etc. In fact, even before people attempt to learn to read they undergo several yeurs of listening to language, speaking, and being “read to”. Clearly, in learning a language, the ultimate goal is to be able to verbalize and write fluently in that language. Similarly in computing the goal is to be able to program (or design and impiement algorithms) effectively. in earning to read and write it has long been recognized that reading is easier and that it must precede writing by a considerable margin of time to allow the assimila- tion of enough models and constructs of text to make writing possible. In teaching computing we seem to have overlooked or neglected what corresponds to the reading stage in the process of learning to read and write. To put it strongly, asking people to design and write programs early on in their computing experience islike expecting that they be able to competently write essays before they have learned to read or even to write short sen- tences—it is expecting just too much of a lot of people. It also probably explains why many otherwise very able people just don’t get started in computing. What we are therefore proposing is that in teaching computing we should draw as much as we can from the learning-to-read-and-write analogy. This means that we must be able to provide the beginning student with his or her “reading experience” in computing before embarking on the more difficult computer problem-solving tasks which require considerably more creative effort, discipline, and technical skill. Ai this puini ii is imporiant iv secugnize thai ihe “reading expeticuce” cannot be gained by sitting down with a book of programs and attempting toxviii PREFACE “read” them. The problem with this approach is that program instructions as written on a picce of paper are a static representation of a computer algorithm. As such they do not very fluently convey anything of the dynamic aspect of computer algorithms and programs. It can be argued that it is important to have a clear and in-depth understanding of the dynamic character of programs before attempting algorithm design and program implementation. What is needed isa practical and economical method of giving students their “reading experience” in computing. To gain this experience the students need to “see” programs written in a high level language executing. Traditionally, all the student sees is the output printed on the terminal or line printer and prompts by the program when it requires input from the terminal. This level of “seeing” a program execute is unsatisfactory for beginners because it conveys very little about the program’s flow of control or about the effects of individual and groups of program starements. What we need is much more expiicit demon- strations of what is happening in a program that causes it to produce the outputs that it does. A far more transparent way to do this is to place before the student on a terminal the text of the program being executed and then to trace the execution on the displayed program while at the same time dynam- ically updating on the screen the changes in program variables as they are made and related to steps in the displayed program. This dynamic method of studying algorithms can greatly assist the student in acquiring a level of understanding necessary to design and implement his or her own algorithms. ‘The faciiities and sofiware woois needed to adequately demonstrate execution of a program in this way are relatively economical to provide. All that is needed is a terminal with cursor addressing facilities and a small set of software tools (several hundsed lines of code) that can be inserted into the program being studied to make its execution “visible” on the screen. Only ‘one procedure call per statement of the program being studied is needed to operate itin a visible mode (this software is available from the author). The display software is transparent to the user. The appropriate tools allow us to see a program execute in single-step mode while monitoring the changes in variables as they are dynamically displayed on the screen. The student can cause the program to execute the next statement at each stage by simply pressing a single key (e.g. the RETURN) on the terminal. As an cxample, the screen layout for “visible” execution of a selection sorting procedure in Pascal is as illustrated in Fig. 1. The next statement to be executed at each stage is tracked by having an arrow that can be moved from statement to statement on the procedure text displayed on the screen. A more complete description of the software is given elsewhere.t + R. G. Dromey, Before Programming—On Teaching Introductory Contputing, Technical Report No. 81/6, Dept. of Computing Science, University of Wolton- gong (1981).A SUGGESTION POR INSTRUCTORS xix procedure selectionsort(a:nelements; n:integer); var i {index for sorted part}, j {index for unsorted part}, P {position of minimum}. min {minimum in unsorted part): integer: begin for i= 1 ton-t do 1 | af 12 begin ‘min += afi}; pmis de} 2 falls] 4 for j:= i+ tondo if off] < min then begin 1 lap):) 12 min = ai jf; ann aan a 7 Ch == ap] 8 | min: 4 aw: 12 4 56 67 9 23 45 Fig. 1 Screen layout far visible program execution, The cause-and-effect nature of what is displayed when executing prog- sunny in this manner (automatic singie-step mode) can accomplisa a number of things. (1) Most importantly, it provides a good understanding of the dynamic nature of algorithms—an essential prerequisite if students are to later design and implement their own algorithms. (2). Iegives a deep insight into the basic laws of composition of computer algorithms (sequence, selection, iteration, and modularity), (3) Itis a useful vehicle for helping students to learn various programming constructs. Actually observing constructs being used within different programming contexts and linking these constructs to changes in the values of variables and conditions gives the student the necessary concrete demonstrations that will enable him or her to master the use of these constructs. (4) Itconveys in a very vivid fashion the workings of a particular algorithm For exainple, the selection sort algorithm when visibly executed con- veys the difficult concept of how we can have one loop executing within another loop. It also highlights the difference between subscripted variables and subscripis.xx PREFACE (5) It also provides an ideal tool for teaching students debugging tech- niques. That is, programs with logical bugs can be implemented and the student asked to diagnose the problem after studying the program in visible execution mode. Visible program execution has the added advantage of being ideally suited for use in the lecture room, classroom, or library, Most terminals have the capability of video output which can be connected to a video recorder. Using these facilities it is very easy and cheap to monitor and record the visible program execution of a variety of programs for later use on video monitors in the classrouin or library. This gives us a leaching aid far superior to handwritten blackboard examples when lecturing about algorithms in the classroom. We have recorded and used a number of programs in this way We cai i fc tude of piogiain execution a step further by making it more active with respect to the student. At each step in the program's execution we can ask the student to supply the appropriate value of the variable or condition etc. How this works can be best illustrated by referring back to the figure. The arrow is indicating that the 7 statement (ie. p:= j)is the next one to be executed. What the software can do is move the cursor to the “‘p”’ box on the screen and indicate to the student that he must supply the appropriate value for p (in this case 2) betore the program will continue. If the user enters the wrong value the word ERROR will flash the user to again try to enter the proper p vatue. If the user gets it wrong twice the software flashes VALUE in the p-hox and then supplies the user with the correct valuc, switches from interactive to automatic single step mode, and moves to the next program statement to be executed. Using a visible program in interactive single-step mode can in a very direct way reinforce the student's comprchcnsion of how a program really works and what individual program statements accomplish, At the same time it can give the student very positive and direct feedback when he gets something wrong. ‘What we are therefore advocating is that students be given considerable exposure ta visible “program-reading” in both the modes described as a preparatory step to considering the problem-solving aspect of computer algorithm design. Exposure to visible program execution (VPE) should be accompanied by a study of the basic laws of composition of computer algorithms (sequence, selection, itcration and modularity) including how these laws of form relate to the way programs are executed. Concurrently, a study should also be made of the syntax of the programming language beingACKNOWLEDGEMENTS The inspiration for the approach taken in this book grew out of an admira- tion for George Polya’s classic works on problem-solving. During the summer and autumn of 1980, I had the pleasure of spending a number of afternoons chatting with Professor and Mrs. Polya. k and that of the pioneers of the discipline ‘The influence of Polya’s of computing science, E. stra, Floyd, C. A. R. Hoare, D. E. Knuth. andN, Wirth. is freely acknowledged. There is also a strong influence of Jeff Rohl’s work in the last chapter on recursion. Isincerely appreciate the guidance and encouragement given to me by Professor Hoare. Henry Hirschberg. Ron Decent. and my reviewers. [am also grateful to the University of Wollongong and Stanford Uni- versity for the use of their facilities. A number of people have generously given helpful comments and support during the preparation of the manuscript. I would particularly like to thank Tom Bailey, Miranda Baker, Harold Brown, Bruce Buchanan, Ann Cartwright. John Farmer. Tony Guttman, Joan Hutchinson. Leanne Koring, Donald Knuth, Rosalyn Maloney, Richard Miller, Ross Nealon, Jurg Nievergell, Richard Patis, Ian Pirie, Juris Reinfelds, Tom Richards, Michael Shepanksi, Stanley Smerin and Natesa Sridharan and my students at Wol- longong. The understanding and encouragement throughout this whole project that I have received from my wife Aziza is deeply appreciated. Finally, I wish to extend my deepest gratitude to Bronwyn James for her loyalty and untiring and able suppor in ihe preparation aud typing of the manuscript. xxiChapter 1 INTRODUCTION TO COMPUTER PROBLEM-SOLVING 1.1 INTRODUCTION Computer problem-solving can be summed up in one word—it is demand- ing! It is an intricate process requiring much thonght, careful planning logical precision, persistence, and attention to detail. At the same time it can be a challenging, exciting, and satisfying experience with considerable room for personal creativity and eypressinn Tf eamputer problem-solving is approached in this spitit then the chances of success are greatly amplified. In the discussion which follows in this introductory chapter we will attempt to lay the foundations for our study of computer problem-solving 1.1.1 Programs and algorithms The vehicle for the computer solution to a problem is a set of explicit and unambiguous instructions expressed in a programming language. This set of instructions is called a program. A program may also be thought of as an algorithm expressed in a programming language. An algorithm therefore corresponds to a solution to a problem that is independent of any program- ming language. ‘To obtain the computer solution to a problem once we have the pro- gram we usually have to supply the program with input or data. The program then takes this input and manipulates it according to its instructions and eventually produces an output which represents the computer solution to the problem. The realization of the computer output is but the last step in a very long chain of events that have led up to the computer solution to the problem.2 COMPUTER PROBLEM-SOLVING CHAP. Our goal in this work is to study in depth the process of algorithm design with particular emphasis on the problem-solving aspcets of the task. There are many definitions of an algorithm. The following definition is appropriate in computing science. An algorithm consists of a set of explicit and unam- biguous finite steps which, when carried out for a given set of initial condi- tions, produce the corresponding output and terminate in a finite time. 1.1.2. Requirements for solving problems by computer From time to time in our everyday activities, we employ algorithms to solve problems. For example, to look up someone’s telephone number in a tele- phone directory we need to employ an algorithm. Tasks such as this are usually performed automatically without any thought to the complex under- iying mechanism needed to effectively conduct the search. it therefore comes as somewhat of a surprise to us when developing computer algorithms that the solution must be specified with such logical precision and in such detail, After studying even a small sample of computer problems it soon becomes obvious that the conscious depth of understanding needed to design effective computer algorithms is far greater than we are likely to encounter in almost any other problem-solving situation. Let us reflect for a moment on the telephone directory look-up prob- lem. A telephone directory quite often contains hundreds of thousands of names and telephone numbers yet we have iittie troubie finding the desired telephone number we are seeking. We might therefore ask why do we have so little difficulty with a problem of seemingly great size? The answer is simple. We quite naturally take advantage of the order in the directory to quickly climinate large sections of the list and home in on the desired name and number. We would never contemplate looking up the telephone number of J. R. Nash by starting at page 1 and examining each name in turn until we finally come to Nash’s name and telephone number. Nor are we likely to contemplate looking up the name of the person whose number is 2987533. ‘To conduct such a search, there is no way in which we can take advantage of the order in the directory and so we are faced with the prospect of doing a number-by-number search starting at page 1. If, on the other hand, we had a list of telephone numbers and names ordered by telephone number rather than name, the task would be straightforward. What these examples serve to emphasize is the important influence of the data organization on the perfor- mance of algorithms. Only when a data structure is symbiotically linked with an algorithm can we expect high performance. Before considering these and other aspects of algorithm design we need to address the topic of problem- soiving in some detaSEC. 12 THE PROBLEM-SOLVING ASPECT 3 1.2 THE PROBLEM-SOLVING ASPECT It iy widely recognized that problen-solving is a creative process which largely defies systematization and mechanization. This may not sound very Teo bal during their schooling, acquire at least a modest set of probiem- solving skills which they may or may not be aware of. Even if one is not naturally skilled at problem-solving there are a number of steps that can be taken to raise the level of one's performance. itis not implied or intended that the suggestions in what follows are in any way a recipe for problem-solving. The plain fact of the matter is that there is no universat method. Different strategies appear to work for different people. Within this context, then, where can we begin to say anything useful computer problem-solving is about understanding. 1.2.1 Problem definition phase Success in solving any problem is only possible after we have made the effort tocome to terms with or understand the problem at hand. We cannot hope to make useful progress in solving a problem until we fully understand what itis we are trying to solve. This preliminary investigation may be thought of as the prodiem definition phase. in other words, what we must do during this phase is work out what must be done rather than how to do it. That is. we must try to extract from the problem statement (which is often quite impre- cise and maybe even ambiguous) a set of precisely defined tasks. Inexperi- enced problem-solvers too often gallop ahead with how they are going to solve the problem only to find that they are either solving the wrong problem or they ate solving just a very special case of what is actually required. In short, a lot of care should be taken in working out precisely what must be done. The development of algorithms for finding the square root (algorithm 3.1) and the greatest common divisor (algorithin 3.3) are good illustrations ‘of how important it is to carefully define the problem. Then, from the definitions, we are led in a natural way to algorithm designs for these two problems. 1.2.2. Getting started on a problem ‘There are many ways to solve most problems and also many solutions to most problems, This situation docs not make the job of problem-solving easy. When confronted with many possible lines of attack it is usually4 COMPUTER PROBLEM-SOLVING cuap.t difficult to recognize quickly which paths are likely to be fruitless and which paths may be productive. Perhaps the more common situation for people just starting to come to grips with the computer solution to problems is that they just do not have any dea where to start on the problem, even after the problem definition phase. en confronted with this situation, what can we do? A Diock often occurs at this point because people become concerned with details of the implemen- tation before they have completely understood or worked out an implementation-independent solution. The best advice here is not to be too concerned about detail. That can come later when the complexity of the problem as a whole has been brought under control. The old computer proverb! which says “the sooner you start coding your program the longer it is going to take” is usually painfully true. 1.2.3 The use of specific examples ‘A useful strategy when we are stuck is to use some props or heuristics (i.e. rules of thumb) to try to get a start with the problem. An approach that often allows us to make a start on a problem is to pick a specific example of the general problem we wish tosolve and try to work out the mechanism that will allow us to solve this particular problem (e.g. if you want to find the maximum ina set of numbers, choose a particular set of numbers and work out the mechanism for finding the maximum in this set—see for example algorithm 4.3), itis usually much easier to work out the details of a solution to a specific problem because the relationship between the mechanism and the particular problem is more clearly defined. Furthermore, a specific problem often forces us to focus on details that are not so apparent when the problem is considered abstractly. Geometrical or schematic diagrams rep- resenting certain aspects of the problem can be usefully employed in many instances (see, for example, algorithm 3.3). ‘This approach of focusing on a particular problem can often give us the foothold we need for making a start on the solution to the general problem. ‘The method should, however, not be abused. It is very easy to fall into the trap of thinking that the solution to a specific problem or a specific class-of problems is also a solution to the general problem. Sometimes this happens but we should always be very wary of making such an assumption. Ideally, the specifications for our particular problem need to be examined very carefully to try to establish whether or not the proposed algorithm can meet those requirements, If the full specifications are difficult to formulate sometimes a well-chosen set of test cases can give us a degree of i HF, Ledgard, Programming Proverbs, tiayden, Rochetie Fark, N.i., 1973.SEC. 1.2 ‘THE PROBLEM-SOLVING ASPECT 5S confidence in the generality af our solution. However, nothing less than a complete proof of correctness of our algorithm is entirely satisfactory. We will discuss this matter in more detail a little later. 1.24 Similarities amona problems We have already seen that one way to make a start on a problem is by considering a specific example. Another thing that we should always try to do is bring as much past experience as possible to bear on the current problem, In this respect it is important to see if there are any similarities between the current problem and other problems that we have solved or we have scen solved. Once we have had a little experience in computer problem-solving it is unlikely that a new problem will be completely divorced from other problems we have seen. A good habit therefore is to aiways make an effort to be aware of he sitnitarities anvung problems. The more experience one has the more/tools and techniques one can bring to bear in tackling a given problem. The contribution of experience to our ability to solve problems is not always helpful. In fact, sometimes it blocks us from discovering a desirable or better solution to a problem. A classic case of experience blocking progress was Einstein's discovery of relativity. For a considerable time before Einstein made his discovery the scientists of the day had the necessary facts that could have led them to relativity but it is almost certain that their experience blocked them from even contemplating such a proposal—Newion's theory was coreei aud dai wa all there was ie it! On this point it is therefore probably best to place only cautious reliance ‘on past experience. In trying to get a better solution to a problem, sometimes too much study of the existing solution or a similar problem forecs us down the same reasoning path (which may not be the best) and to the same dead end. In trying to get a better solution to a problem, it is usually wise, in the first instance at least, to try to independenily solve the problem. We then give ourselves a better chance of not falling into the same traps as our predeces- sors, In the final analysis, every problem must be considered an its merits. A skill that it is important to try to develop in problem-solving is the ability to view a problem from a variety of angles. One must be able to metaphorically turn a problem upside down, inside out, sideways, hack- wards, forwards and so on. Once onc has developed this skill it should be possible to get started on any problem. 4.2.5 Working backwards from the solution There are still other things we can try when we do not know where to start on a problem. We can, tor example, in some cases assume that we already have6 COMPUTER PROBLEM-SOLVING. CHAP. 4 the solution to the problem and then try to work backwards to the starting conditions, Even a guess at the solution to the problem may be enough to give us a foothold to start on the problem. (See, for example, the square root problem—algorithm 3.1). Whatever attempts that we make to get started on a problem we should write down as we go along the various steps and explorations made. This can be important in allowing us to systematize our investigations and avoid duplication of effort. Another practice that helps us develop our problem-solving skills is, once we have solved a problem, to consciously reflect back on the way we went about discovering the solution. ‘This can help us significantly. The most crucial thing of all in developing problem-solving skills is practice, Piaget summed this up very nicely with the statement that “we learn most when we have to invent.” 1.26 General problem-solving strategies There are a number of general and powerful computational strategies that arc repeatedly used in various guiscs in computing science. Often it is possible to phrase a problem in terms of one of these strategies and achieve very considerable gains in computational efficiency Probably the most widcly known and most often used of these principles is the divide-and-conquer strategy. The basic idea with divide-and-conquer is to divide the original problem into two or more subproblems which can hopefully be solved more efficiently by the same technique. Ifit is possible to proceed with this splitting into smaiier and smaiier subprobiems we will eventually reach the stage where the subproblems are small enough to be solved without further splitting. This way of breaking down the solution to a problem has found wide application in particular with sorting, selection, and searching algorithms. We will see later in Chapter 5 when we consider the binary search algorithm how applying this strategy to an ordered data set results in an algorithm that needs to make only log, rather than n compa sons to locate a given item in an ordered list n elements long. When this principle is used in sorting algorithms, the number of comparisons can be reduced from the order of n? steps to n logyn steps, a substantial gain particularly for large n. The same idea can be applied to file comparison and in many other instances to give substantial gains in computational efficiency It is not absolutely necessary for divide-and-conquer to always exactly halve the problem size. The algorithm used in Chapter 4 to find the k'" smallest element repeatedly reduces the size of the problem. Although it does not divide the problem in half al each step it has very good average performance. It is also possible to apply the divide-and-conquer strategy essentially in reverse to some problems. The resulting binary doubling strategy can give the saine suri of gains in computational efficiency. We will cousider in Chapter 3 how this compiementary technique can be used to advantage toSEC.1L3 ‘TOP-DOWN DESIGN 7 raise a number to a large power and to calculate the n" Fibonacci number. With this doubling strategy we nccd to express the next term n to be computed in terms of the current term which is usually a function of 7/2 in order to avoid the need to generate intermediate terms. Another general problem-solving strategy that we will briefly consider is that of dynamic programming. This method is used most often when we have to build up a solution to a problem via a sequence of intermediate steps. ‘The monotone subsequence problem in Chapter 4 uses a variant of the dynamic programming method. This method relies on the idea that a good solution to a large problem can sometimes be built up from good or optimal solutions to smailer problems. This type of strategy is particularly relevant for many optimization problems that one frequently encounters in opera- tions research. The techniques of greedy search, backtracking and branch- and-bound evaluations are all variations on the basic dynamic programming idea. They ail tend to guide a computation in such a way that the minimum, amount of effort is expended on exploring solutions that have been estab- lished to be suboptimal. There are still other general computational strategies that we could consider but because they are usually associated with more advanced algorithms we will not proceed further in this direction. 1.3 TOP-DOWN DESIGN The primary goal in computer problem-solving is an algorithm which is capable of being implemented as a correct and efficient computer program. In our discussion leading up to the consideration of algorithm design we have been mostly concerned with the very broad aspects of problem-solving. We now need to consider those aspects of problem-solving and algorithm design which are closer to the algorithm implementation. Once we have defined the problem to be solved and we have at least a vague idea of how to solve it, we can begin to bring to bear powerful techniques for designing algorithms. The key to being able to successfully design algorithms lies in being able to manage the inherent complexity of most problems that require computer solution. People as problem-solvers are onlly able to focus on, and comprehend at one time, a very limited span of togic or instructions, A technique for algorithm design that tries to accom- modate this human limitation is known as top-down design or stepwise refinement. Top-down design is a strategy that we can apply to take the solution of a computer problem from a vague outline to a precisely defined algorithm and program impicuentaiion. Top-down design provides us with a way of hand- ling the inherent logical complexity and detail frequently encountered in8 COMPUTER PROBLEM-SOLVING CHAP. 1 computer algorithms. It allows us to build our solutions to a problem in a stepwise fashion. In this way, specifie and complex details of the implemen- tation are encountered only at the stage when we have done sufficient groundwork on the overall structure and relationships among the various parts of the problem. 1.3.1. Breaking a problem into subproblems Before we can apply top-down design to a problem we must first do the probiem-soiving groundwork that gives us at ieast the broadest of outiines of a solution. Sometimes this might demand a lengthy and creative investiga- tion into the problem while at other times the problem description may in itself provide the necessary starting point for top-down design, The general outline may consist of a single statement or a set of statements. Top-down design suggests that we take the general statements that we have about the solution, one at a time, and break them down into a set of more precisely defined subtasks. These subtasks should more accurately describe how the final goal is to be reached. With each splitting of a task into subtasks it is essential that the way in which the subtasks need to interact with each other be precisely defined. Only in this way is it possible to preserve the overall structure of the solution to the problem. Preservation of the overall structure in the solution to a problem is important both for making the algorithm comprehensible and also for making it possible to prove the correctness of the solution. The process of repeatedly breaking a task down into subtasks and then each subtask into still smaller subtasks must continue until we eventually end up with subtasks that can be implemented as program state- ments. For most algorithms we only need to go down to two or three levels alihough obviously for large sofware projecis this is not true. The larger and more complex the problem, the more it will need to be broken down to be made tractable. A schematic breakdown of a problem is shown in Fig. 1.1. ‘The process of breaking down the solution to a problem into subtasks in the manner described results in an implementable set of subtasks that fit quite naturally into block-structured languages like Pascal and Algol. There can therefore be a smooth and natural interface between the stepwise- refined algorithm and the final program implementation—a highly desirable situation for keeping the implementation task as simple as possible. 13.2 Choice of a suitable data structure One of the most important decisions we have to make computer solutions to problems is the choice of appropriate data structures.SEC13 TOP-DOWN DESIGN 9 General outline a Input conditions Output requirements Body of algorithm | —. Subtask 1 Subtask 3 ‘Subtask 1a te [ Subiask 2a}. Fig. 1.1 Schematic breakdown of a problem into cubtasks as employed in top: down design. All programs operate on data and consequently the way the data is organ- ized can have a profound effect on every aspect of the final solution, In sticular, an inappropriate choice of data structure often leads to clumsy. incfficicnt, and difficult implementations. On the other hand, an appropriate choice usualty teads to a simple, transparent, and efficient implementation. There is no hard and fast rule that tells us at what stage in the develop- ment of an algorithm we need to make decisions about the associated data structures. With some problems the data structure may need to be consi- dered at the very outset of our problem-solving explorations before the top-down design, while in other problems it may be postponed until we are weil advanced with the details of the implementation, In other cases still, it can be refined as the algorithm is developed. The key to effectively solving many problems really comes down to making appropriate choices about the associated data structures. Data struc- tures and algorithms are usually intimately linked to one another. A small change in data organization can have a significant influence on the algorithm required to solve the probiem. it is not easy to formulate any generally applicable rules that say for this class of problem this choice of data structure10 COMPUTER PROBLEM-SOLVING chart is appropriate. Unfortunately with regard to data structures each problem must be considered on its merits, ‘The sort of things we must however be aware of in setting up data Structures are such questions as: (1) How can intermediate results be arranged to allow fast access to information that will reduec the amount of computation required at a later stage? (2) Can the data structure be easily searched? (3) Can the data structure be easily updated? (4) Does the data structure provide a way of recovering an earlier state in the computation? (3) Does the data structure involve the excessive use of storage? (6) Is it possible to impose some data structure on a problem that is not initially apparent? (7) Can the problem be formulated in terms of one of the common data structures (e.g. array, set, queue, stack, tree, graph, list)? These considerations are seemingly general but they give the flavor of the sort of things we need to be asking as we proceed with the development of an algorithm. 1.3.3. Construction of loops In moving from general statements about the implementation towards sub- tasks that can be realized as computations, almost invariably we are led toa series of iterative constructs, or loops, and structures that are conditionally executed, These structures, together with input /ouiput statements, comput- able expressions, and assignments, make up the heart of program implemen- tations, AL the time when a subtask has been refined to something that can be realized as an iterative construct, we can make the task of implementing the loop easier by being aware of the essential structure of all loops. To construct any loop we must take into account three things, the initial conditions that need to apply before the loop begins to execute, the invariant relation that must apply after each iteration of the loop, and the conditions under which the iterative process must terminate. In constructing loops people often have trouble in getting the initialSEC. 13 TOP-DOWN DESIGN 11 of times rather than one too few or one too many times. For most prob- lems there is a straightforward process that can be applied to avoid these errors. 4 ‘To establish the initial conditions for a loop, a usually effective strategy is to set the loop variables to the values that they would have to assume in order to. solve the smallest problem associated with the loop. Typically the number of iterations 1 tira must be made by a ioop are in tie range GxiSn, The smallest problem usually corresponds to the case where i either equals 0 ori equals 1. Algorithms 2.2. and 4.3 illustrate both these situations. To bring this point home let us suppose that we wish to sum a set of numbers in an array using an iterative construct. The loop variables are i the array and loop index, and s the variable for accumulating the sum of the array elements. ‘The smallest problem in this case corresponds to the sum of zero numbers. The sum of zero numbers is zero and so the initial values of i and s musi be: 0 } solution for n=0 5-0 1.3.5 Finding the iterative construct Once we have the conditions for solving the smallest problem the next step is to try to extend it to the next smallest problem (in this case when i= 1). That is we want to build on the solution to the problem for i= 0 to get the solution for i-1. ‘The solution for n=1 is: i= salt] } solution for n = 1 This solution for for i and s when 1 can be built from the solution for n and the two expressions using the values at \ generalized solution for n>0 stafi} ‘The same two steps can be used to extend the soluti when n=2 and so on, These two steps wi st n from when n=1 to in general extend the solution12 COMPUTER PROBLEM-SOLVING. CUA, 4 from the (i~ 1)" case to the i'® case (where i 1). They can therefore be used as the basis of our iterative construct for solving the problem for n>1. 7 initialization conditions 5 } for loop and solution to summing problem when n=0 while i0) and (x<10) do begin end With loops of this type it cannot be directly determined in advance how many iterations there will be before the loop will terminate. In fact there isSEC. 13 TOP-DOWN DESIGN 13 no guarantee that loops of this type will terminate at all. In these cireum- stances the responsibility for making surc that the loop will terminate rests with the algorithm designer. If the model for the computation is straightfor- ward it may be a simple matter to guarantee termination (e.g. in our example above if x is changed with each iteration in either a monotonically increasing or decreasing fashion, then eventuaiiy the conditional expression (x >0) and (x<10) will become false. There are loops of this kind where it is very difficult to prove that termination is guaranteed. Algorithm 3.1 (for comput- ing square roots) contains a loop that is typicat for this type of termination. Yet another way in which termination of a loop can be set up is by forcing the condition under which the loop will continue to iterate to become false. This approach to termination can be very useful for simplifying the test that must be made with each iteration. An example best illustrates this method of loop termination. Suppose we wish to establish that an array of 1 ciements is in strictly ascending order (i.e. af i}= i+1 Ifn was assigned the value 5 and the data set was 2,3, 5,11, 14, then the first assignment prior to the loop would result in the array configuration below: afi] af2] alr] ain+t] 243 | 5 [11 ]i4 | 14 nand The two 14s guarantee that the test a[/]alrmiddle] then lower := middle+1 else upper := middle = allower)) For the search value x and the array af 1.7} where x= 44 and n= 15 we may have: Initial ay af15] configuration |10} 12] 20} 23|27| 30] 31] 39} 421 44|45149]57}63) 70 t t t lower middle upperSEG. 14 IMPLEMENTATION OF ALGORITHMS 17 Then the associated execution table is given by Table 1.1 Table 1.1 Stepwise execution table for binary search. Iteration no. lower middle upper lower < upper afmiddle] x > almiddle] Initially = 2 1S. true = _ 1 9 8 15 true 39 true 2 9 12 12 true 49 false 3 9 10 10 true 44 false 4 10 910 false 42 true NOTE: The values of variables associated with each iteration apply after the iteration has been completed, If we get to the stage where our program is executing but producing incorrect results (e.g. it might be a sort routine that places most, but not all, elements in order) the hest idea is to first use. a dehugging trace to print ant strategic information. The next step is to follow the program through by hand in a stepwise fashion checking against the computer's debugging out- put as we go. Whenever we embark on a debugging procedure of this kind we should be careful to ensure that we follow in a straight line along the path of execution. itis usuaily a wasted effort to assume that some things work and only start a systematic stdy of the algorithm halfway through the execution path. A good rule to follow when debugging is not to assume anything. 1.45 Program testing In attempting to test whether or not a program will handle all variations of the problem it was designed to solve we must make every effort to be sure that it will cope with the limiting and unusual cases. Some of the things we might check are whether the program solves the smallest possible problem, whether it handles the case when all data values are the same, and so on. Unusual cases like these are usually the ones that can cause a program to falter. Asan example, consider the testing we would need for the binary search algorithm (i.e. algorithm 5.7). Appropriate data sets and tests are given in Table 1.2. It is often not possible or necessary to write programs that handle all input conditions that may be supplied for a given problem. Wherever poss- ible programs should be accompanied by input and output assertions as described in the section on program verification. Although it is not always practical to implement programs that can handle all possible input condi- it to gracefully and informatively respond to the user when it receives input conditions it was not designed to handle.18 COMPUTER PROBLEM-SOLVING CHAP. Table 1.2 Appropriate data sets for testing binary search algorithm Test Search value(s) Sample data @) Git) (iv) @) (vii) (viii) (ix) Will the algorithm handle the search of array of one Will it handle the case where all array values are equal? Will it handle the case where the element songht equals the first value in the array? Will it handle the ease where the value sought equals the last value in the array? Will it handle the case where. the value sought is less than the first element in the array? Will it handle the case where the value sought is greater than the last value in the aay? Will it handle the case where the value sought isat an even array location? Will it handle the case where the value sought is at an odd array location? Will it handle the case where the value sought is absent but within the range of array values? 0,152 01.2 n4t a@fiJ=1 aft}=1 afj=1 aft}=1 at}=1 afi}=1 atij=t a{t}=2 af}=2 aQ]=4 afnj=1 afnj=n atnj=n atnjen afnj=n otnl=n afn}=2n The last statement should not, however, be taken to mean that we should only design algorithms to solve specific problems, This approach should be far from our goal. Almost without exception we shoutd design algorithms to be very general so that they will handle a whole class of problems rather than just one specific case. The latter approach of writing programs for specific cases usually, in the long run, amounts to a lot of wasted effort and time. It is far better to do a thorough job in the first place and produce a program that will solve a wide range of problems. This brings us to a bad practice that beginning programmers (and some others) often adopt. The practice referred to is that of using fixed constants where variablesSECS PROGRAM VERIFICATION 19 should be used. For example. we should not use statements of the form while i<100 do ‘The 100 should always be replaced by a variable, i.e. while i en there are no restrictions on the values of the input variables the input assertion is given the logical value true. The output assertion must specify symbolically the results that the program is expected to produce for input data that satisfies the input assertion (e.g. if a program is designed to calculate the quotient q and remainder r resulting from the division of x by y then the output assertion can be written as: (em gry tAA(rQ where the logical connective “3” is read as “implies”. P is termed the assumption and Q the conclusion. The associated truth table defining impli- cation is given in Table 1.3. Fable 1.3 truth table defming implication P Q P=Q ue true true true false false false rue true false false true In order to show that these implications or propositions are logically true, it is necessary to take into account the effects of executing the program. for arbitrary inputs that satisfy the input assertion. A relatively straight- forward way tu do this is to use the technique of symbolic execution. With symbolic execution, aff input data values are replaced by symbolic values and all arithmetic operations on numbers translate into algebraic manipulation uf syinbutic capressious. Ay an exampie, consider tie following program segment labelled with input and output assertions:22 COMPUTER PROBLEM-SOLVING CHAP. 1 A readinix.y); {assert: truc} xI=x-y; x+y; yox a B {assert x~ yOAy~ x0} where x0 and y0 refer to the initial values of x and y respectively. Table 1.4 Normal and symbolic execution for exchange mechanism Step Normal execution Symbolic execution input values: x3 y= 1 input values: xa y~@ 1 a= acy x=a-8 2 Yrs ty > y=(a-f)1p=a0 3 I= you > x= ((a-B)+B)-(a-B)=8 that the values of x and y are exchanged. Symbolic execution enables us to transform the verification procedure into proving that the input assertion with symbotic values substituted for atl input variables implies the output assertion with final symbolic values substi- tuted for all variables. A proposition phrased in this way is referred to as a verification condition (VC) over the program segment from the input asser- tion (o the output assertion, To proceed with the verification procedure it is usually necessary to set on the i and output assertions. Taken to the limit this involves carrying out the verifica- tion procedure statement by statement. For practical purposes it is, however, usually sufficient to only consider verification conditions for blocks of a program as marked by straight-line segments, branching segments, and loop segments. We will adopt the convention that VC(A ~ B) refers to the veritica- tion condition over the program segment from A to B. We will now consider the verification of cach of these basic segment types in tun. 1.5.4 Verification of straight-line program segments The best way to illustrate the verification procedure is by example. Our exchange mechanism mentioned above will serve as an example of a straight-line program segment.SEC 15 PROGRAM VERIFICATION 23 The verification condition for this program segment is: VC(A~B): true > {x= yOAy = x0} Un substitution of the initial and final values of all variables, we get: VC(A~B): true > ((a~B)+B)~(a~B)) = BNa~B)+B) =a ‘The conclusion part of the verification condition can be simplified to yield 8=B and a=a which is clearly true and so the implication is true 4.55 Verification of program segmenis with branches To handle prograin segments that contain branches it is necessary to set up and prove verification conditions for each branch separately. As anexample, consider the following program segment that ensures that x is less than or cquat iv y. readIn(x,y); A (assert Py: true) if x>y then B (assert Pp: ((x<=y)Alx = x0Ay =y0)) Vix = yOAy = x00} In general, the propositions that must be proved for the basic if construct are: PAAC>P5 PAA~CoP a where C, is the branch condition. The two verification conditions needed in this case are given below, where the initial values of x and y are respectively « and 8 VC(A~()-B): true Aa>B>((o=B)A(B= aha = B))V(B = BAe =a) Since a> fis true and the second part of the conclusion (i.c. a aA =A) is true, the verification condition for the true path is true. The verification condition for the false path is: VC(A~(f}~ B): true A~(a>B)> (af) A(a= ah f = BV a = BAB = 0 Since ~(a>A)= (af) and the conclusion (a = «Af = 8) is true the verifica- tion condition for the false path is true. It follows that the labelled program segment (A~B) is true. Case statements can be treated simiiariy24 COMPUTER PROBLEM-SOLVING CHAP. 1 1.5.6 Verification of pragram segments with loops There are problems with trying to verify loop segments directly by symbolic execution because the number of iterations required is usually arbitrary. To overcome this problem, a special kind of assertion called a loop invariant 4 be employed. A loop st be employed. A leop 1d be a property (predicate) that captures the progressive computational role of the loop while at the same time remaining true before and after each loop traversal irrespective of how many times the loop is executed, Once the loop invariant is established, there are several steps that must be taken to verify the loop segment. To understand this loop verification procedure, we will use the following single loop program structure as our model. A. {input assertion P,} | straight-line program women B_ {loop invariant 15} while loop-condition do begin loop-free program segment end C {output assertion Pc} (a) The first step that must be taken is to show that the loop invariant is ¥, before the loop a verification condition VC(A~ B) for the program segment from A to B. That is, the input assertion, tagether with any changes to variables caused by the segment A —B, must imply that the loop invariant is true. ‘We can use symbolic execution to carry out this verification step. That is, we must show P,-/p (b) The second part of verifying a loop involves showing that the loop invariant is still true after the segment of program within the loop has been executed. To do this we can set up a verification condition VC(B-B). This involves showing that the loop invariant with initial values of variables set, together with the condition for loop execution Cy, implies the truth of the loop invariant with final values of variables, ive. be donc by be donc by LAC 21, IghCgrly Symbolic execution can also be used to verify this step.SEC.15 PROGRAM VERIFICATION 25 (©) Asa final step in verifying a loop segment, it is necessary to show that the invariant, together with the negation of the loop-entry condition, implies the assertion that applies on exiting from the loop. The verifica- tion condition in this case for our basic loop structure will be VC(B~C). The corresponding proposition will be: Ip\~Cy>Pe A better understanding of the application of the loop verification technique can be gained by studying a specific example. Consider the follow- ing program segment which can be used to compute factorial. A. assert Py: n>=0) i:= 0: fact := 1; B {invarlant I: fact =i!Airn} while i Ty 1 y2021=01A0=y Now 1 = Olis tue by definition and y2-030%y is trivially true and so the VOLA~ B) ii 2. vee 8) : inCty 1 B=alhasy\aB =! Since ~{aB=~! is true and so the verification condition VC(B-O) is true26 COMPUTER PROBLEM-SOLVING CHAP. 1 Since the verification conditions for all three program segments are correct, the complete program segment is said to be partially correct. What the verification method we have described gives us is only a proof of partial correctness, with the implication that if the algorithm terminates the result produced will be correct. For programs that contain loops we are therefore ieft with the separate but necessary task of proving that the program terminates in order to establish total correctness. Before considering in detail a method for proof of termination, we will consider setting up the verification conditions for a program example that employs an array. 1.5.7 Verification of program segments that employ arrays The idea of symbolic execution developed in the preceding sections can be extended ta some of the simpler examples that employ arrays although it now becomes necessary to account for the symbolic values of all array elements, As an example of verification of a program segment containing an array we will set up the verification conditions for a program that finds the position of the smallest clement in the array. ‘The program annotated with assertions may take the following form: A. assert Py: n>} B {invariant fy: (10 “The condition =0 becomes an invariant of the loop. in some instances, the invariant used for establishing partial cor- rectness is not sufficient for use in a proof of termination. This problem can be overcome by attaching additional conditions (derived from earlier assertions) to the invariant. (ii) ‘The second proposition that must be praven is to show that the loop invariant Ip, together with the condition for loop execution, Cy, implies that the value e, of the expression before execution is strictly greater than its value € after loop execution, i.e. for a loop B TCYUB~B): Iy\Cy>(6>€)M€20) The final value of © can be obtained by symbolic execution of the statements in the loop. ‘Once these two propositions have been shown to be true, we can immediately conclude that the repetition process is finite since e can only be decreased a finite number of times while remaining positive as required by TCUs). i inati vps fc similar line of reasoning. A proof of termination for the quotient/remainder program given below will now be outlined. begin A {assert P,: (x>0)My>0)} risa i= 0; B {invariant Ig: r=0Ax = y+q+r} while y0 from P, to the invariant J. We then have TCUB): (F>O)AGe= y9q+AG>O)AVSalr+y>0) Now (2 0)A(y>Da(r+y>0) and so TC1(B) can be shown to be true. Assuming that x, y, rand q respectively have the symbolic valuesa, B, y, and 5 on entering the loop, then for TC2(B~ B) we have: TC2UB-B): (y20)\ Ma = B*5+-y)A\(B>0)\(B< y+ B>(y-B)+8 where fo=ytB and — e=(y~B)+B8 which can easily be shown to be true and so the proof of termination is complete. Once both partial correctness and termination have been established, the proof is complete. To embark on detailed formal proofs of programs requires a considerable degree of mathematical maturity and experience. The iengiit sud detail of such prouis is usually considerably greater than the size of the original program text, In the discussions which follow, we will therefore only attempt to annotate programs with relevant assertions. 1.6 THE EFFICIENCY OF ALGORITHMS Efficiency considerations for algorithms are inherently tied in with the design, implementation, and analysis of algorithms. Every algorithm must use up some of a computer's resources to complete its task. The resources most relevant in relation to efficiency are central processor time (CPU time) and internal memory. Because of the high cost of computing resources it is always desirable to design algorithms that are economical in the use of CPU time and memory. This is an easy statement to make but one that is often difficult to follow either because of bad design habits, or the inherent compiexiiy of the proviem, or botit. As with most oiler aspecis uf aigoritiun design, there is no recipe for designing efficient algorithms. Despite there30 COMPUTER PROBLEM-SOLVING cuar.t being some generalities each problem has its own characteristics which demand specific responses to solve the problem efficiently. Within the framework of this last statement we will try to make a few suggestions that can sometimes be useful in designing efficient algorithms. 1.6.1. Redundant computations Most of the inefficiencies that creep into the implementation of algorithms come about because redundant computations are made or unnecessary storage is used. The effects of redundant computations are most serious when they are embedded within a loop that must be executed many times. ‘The most common mistake using loops is to repeatedly recalculate part of an expression that remains constant throughout the entire execution phase of the loop. The example below illustrates this point: x= 0: for i: 1 ten do begin x= x40.01; y= (atatatcpexex+bedex; writeln ('x =", x,'y = ‘y) end This loop does awice the number of muliptications necessary to com plete the computation. The unnecessary multiplications and additions can be removed by precomputing two other constants a3c and b2 before executing the loop: = adeaxex + b2x; writeln ("x = '.x,'y end »” ‘The savings in this instance are not all that significant but there are many other situations where they are much more significant. It is always most important to strive to eliminate redundancies in the innermost loops of computations a9 these inefficiencies cau be musi cusily.SEC. Ls TUE EFFICIENCY OF ALGORITHMS 31 1.6.2. Referencing array elements If care is not exercised redundant computations can also easily creep into array processing. Consider, for example, the two versions of an algorithm for finding the maximum and its position in an array. Version (1) 1; Pp: for i:= 2tondo if a[ia[p] then p : max := ofp} Version (2) [1]; for i:= Ziv n do if ali]>max then begin max := afi}; D end The version (2) implementation would normally be preferred because the conditional test (ie. aff]>max) which is the dominant instruction is more efficient to perform than the corresponding test in version (1). It is more efficient because to use the variable max only one memory reference instrue tion is requited, whereas to use the variable a[p] requires wo memory references and an addition operation to locate the correct value for use in the test. Also in version (2), introduction of the variable max makes it clearer whai task is iv be accomplisiied. 1.6.3 Inefficiency due to late termination Another place inefficiencies can come into an implementation is where considerably more tests are done than are required to solve the problem at hand, This type of inefficiency can be best illustrated by example. Suppose we had to linear search an alphabetically ordered list of names for some particular name. An inefficient implementation in this instance would be ‘one where all names were examined even if the point in the list was reached where it was known that the name could not occur later (e.g. suppose we were looking for the name MOORE, then, as soon as we had encountered a ai occurred alphabetically laicr than MOORE, c.g. MORRIS, there32 COMPUTER PROBLEM-SOLVING CHAP. would be no need to proceed further). The inefficient implementation could have the form: 1. while name sought <> current name and no end-of-file do (a) get next name trom list. A more efficient implementation would be: 1. while name sought > current name and not end-of-file do (a) get next name from list. 2. test if current name is equal to name sought. The same sort of inefficiency can be built into the bubblesort algorithm. {algorithm 5.3) if care isnot taken with the implementation. This can happen if the inner ioop that drives the exchange mechanism aiways goes the fuii length of the array. For example. for i:= 1ton-1 for j:= 1ton-1 if affj-a[j+ 1] then “exchange afj} with afj+ij" With this sorting mechanism, after the é iteration, the last / values in the array will be in sorted order. Thus, for any given i the inner loop should not proceed beyond n—i. The loop structure will then be: for i:= Lton-1 for j:= lton-i if af fl>alj+1] then “exchange alj] with alj+ 11” The lesson to be learned from this is that we should always try to take advantage of any order or structure in a problem 1.6.4 Early detection of desired output conditions ‘The bubblesort also provides us with an example of another related type of inefficiency involving termination. It sometimes happens, due to the nature of the input data, that the algorithm establishes the desired output condition before the general conditions for termination have been met. For example, a bubblesort might be used to sort a set of data that is already almost in sorted order. When this happens it is very likely that the algorithm will have the data in sorted order tong before the loop termination conditions are met. Itis therefore desirable to terminate the sort as soon as it is established that the daia is alicady suited. Tu dy this all we uced iv du is check wetter thereSEC.17 THE ANALYSIS OF ALGORITHMS 33 have been any exchanges in the current pass of the inner loop. If there have been no exchanges in the current pass the data must be sorted and so early termination can be applied (a check of algorithm 5.3 reveals how this is implemented for the bubblesort). In general, we must include additional steps and tests to detect the conditions for early termination. However, if they can be kept inexpensive (as in the Dubbiesort) then it is worth including them. That is. when early termination is possible. we always have to trade tests and maybe even storage to bring about the early termination. 1.85 Trading storage for efficiency gains A trade between storage and efficiency is often used to improve the perfor- mance of an algorithm. What usually happens in this type of tradeoff is that we precompute or save some intermediate resuits and avoid having to do a lot of unnecessary testing and computation later on. (See for example the longest monotone subsequence problem-—algorithm 4.7.) ‘One strategy that it sometimes used to try to speed up an algorithm is to implement it using the least number of loops. While this is usually possible. inevitably it makes programs much harder to read and debug. It is therefore usually better to stick to the rule of having one loop do one job just us we have one variable doing one job. When a more efficient solution to a problem is required it is far better to try to improve the algorithm rather than resorting to “programming tricks” that tend to obscure winat is being done. A clear implementation of a better algorithm is to be preferred to a “tricky” implementation of an algorithm that is not as good. We are now left with the task of ying to measure the efficiency of algorithms. 1.7 THE ANALYSIS OF ALGORITHMS There are usually many ways to solve any given problem. In computing. asin most efforts of human endeavor, we are generally concerned with “good” solutions to problems, This raises certain questions as to just what do we mean by a “good” solution to a problem? In algorithm design “good” has both qualitative and quantitative aspects. There are often certain esthetic and personal aspects (o this but, on a more practical level, we ate usually interested in a solution that is economical in the use of computing and human resources. Among other things, good algorithms usually possess the follow- ing quaiiities and capabilities:34 COMPUTER PROBLEM-SOLVING CHAP. ‘They are simple but powerful and general solutions They can be casily understood by others, that is, the implementation is clear and concise without being “tricky”. RP 3. They can be easily modified if necessary. 4, They are correct for clearly defined situations. 5. ‘They are abie to be understood on a number of ieveis 6. They are economical in the use of computer time, computer storage and peripherals. 7. They are documented well enough to be used by others who do not have a detailed knowledge of their inner workings. 8. They are not dependent on being run on a particular computer. 9. They are able to be used as a subprocedure for other problems. 10. The solution is pleasing and satisfying to its designer—-a product that the designer feels proud to have created. These qualitative aspects of a good algorithm are very important but it is also necessary to try to provide some quantitative measures to complete the evaluation of “goodness” of an algorithm. Quantitative measures are valu- able in that they can give us a way of directly predicting the performance of an algorithm and of comparing the relative performance of two or more algorithms that are intended to solve the same problem. This can be impor- tant because the use of an algorithm that is more efficient means a saving in computing resources which translates into a saving in time and money. 1.7.1 Computational complexity To make a quantitative measure of an algorithm’s performance it is neces- sary to set up a computational model that reftects its behavior under specified input conditions. This model must capture the essence of the computation while at the same time it must be divorced from any program- ming language. We are therefore required to characterize an algorithm's performance in terms of the size (usually 2) of the problem being solved. Obviously, more computing resources are needed to solve larger problems in the same class The important question that we must setile is, how does the cost of solving the problem vary as 7 increases? Our first response to this question might be that as increases then so does the cost in the same manner (e.g. a problem fur n= 200 takes twice as long as a problem fur 2 = 100). While this linear dependence on n is true for some simple algorithms in general the behavior with n follows some other completely different pattern. At the lower end of ihe seale we fave alguritiuns with lugariihunic (or beiics) Gcpeudenee on 7, while at the higher end of the scaie we have algorithms with an exponentialSEC.17 THE ANALYSIS OF ALGORITHMS 35, dependence on n. With increasing n the relative difference in the cast of a computation is enormous for these two extremes, Table 1.5 illustrates the comparative cost for a range of n values. Table 1.5 Computational cost as a function of problem size for a range of computational complevities logon n n logon nt ea 1 2 2 4 8 4 3.322 10, 33.22 10° 10 > 10? 6.644 10% 604.4 108 Lo® >>10% 9.966 10? 9966.0 10° 10" >>108" 13.287 104 132,877 108 308 >> 10s" What is obvious from this table is that we can soive only very small problems w at exhibits cxpom ing that a computer can do about one million operations per second, an exponential algorithm with n= 100 would take immeasurably longer than the existence of the earth to terminate. At the other extreme, for an algorithm with logarithmic dependence on n, a problem with n= 10* would require only 13 steps which amounts ta about 13 microseconds of computer time. These cxampics emphasize how important it is to have an understand- ing of the way in which algorithms behave as a function of the problem size. What can also come out of analyzing algorithms is a theoretical model of the j te hain algovit of particular problems of particular problems. In deciding how to characterize the behavior of an algorithm as a function of the size of the problem m, we must study the mechanism very carefully to decide just what constitutes the dominant mechanism. It may be the number of times a particular arithmetic (or other) expression is evalu- ated, or the number of comparisons or exchanges that must be made as n grows. For example, comparisons, exchanges, and moves characterize most sorting algorithms. The number of comparisons usually dominates so we use comparisons in our computational model for sorting algorithms. 1.7.2 The order not: A standard notation has been developed to represent functions which bound the computing time for algorithms. It is an order notation and it is usually referred to as the O-notation. An algorithm in which the dominant mechan- ism is executed en? times for ¢, a constant, and # the problem size, is said to have an order re complexity which is written as O(72). Formally, a function gin) is Olf(n)) provided there is a constant ¢ for which the relationship gind=cf{n)36 COMPUTER PROBLEM-SOLVING CHAP. t holds for atl values of n that are finite and positive. With these conventions we have a means of charactcrizing the asymptotic complexity of an algorithm and hence of determining the size of problems that it can solve using a conventional sequential computer. We can write the above relationship in the form: lim, 82. rie Fim) where c is not equal to zero As an example, suppose we have an algorithm that requires (3n?+62+3) comparisons to complete its task. According to the convention just outlined we have: ein) 3n?+6n+3 and tim hOB no Hi fotlows iii this particular aigoritiun bay an asympiviic compiexity of O(n), In using this methodology to decide upon the superiority of one algorithm over another we need to pay careful attention to the constants of proportionality. It can happen that an algorithm with a higher asymptotic complexity has a very small constant of proportionality and hence for some particular range of n it will give better performance than an algorithm with lower compleaity and a higher proportionality constant. 1.7.3. Worst and average case behavior In analyzing any given algorithm there are two measures of performance that are usually considered. These are the worst and average case behavior of the algorithm. These two measures can be applied to bath the time and space complexity of an algorithm. The worst case complexity for a given problem size n corresponds to the maximum complexity encountered among all problems of size n. Determination of the worst case complexity for many algorithms is relatively straightforward. We choose a set of input conditions that force the algorithm to make the least possible progress towards its final goal at each step. Jn many practical applications it is much more important to have a measure of the expected complexity of a given algorithm rather than the worst case behavior. The expected complexity gives a measure of the behavior of the algorithm averaged over all possible problems of size n. In comparing two algorithms to solve a given problem, we would generally opt in preterence for the algorithm that has the lower expected complexity. Unfortunately, setting up a computational model to characterize the average behavior often involves very complea ai sophisticated combinatuiial analyses.sec ut ‘THE ANALYSIS OF ALGORITHMS 37 1.7.4 Probabilistic average case analysis Asa simple example. suppose we wish to characterize the behavior of an algorithm that linearly searches an ordered list of elements for some value x, 1)2\.. n In the worst case it will be necessary for the algorithm to examine all values in the list before terminating. ‘The average case situation is somewhat different. For a probabilistic average case analysis it is generally assumed that all possible points of termination are equally fikeiy, i.e. the probability that x witi be found at position t is 1 /m, and at position 2 is 1/n, and so on. The average search cost is therefore the sum of all possible search costs each multiplied by their associated probability. For example, if n=5 we would have: average search cost = i/3 Noting that 1424+3+ + +n=n(n+1)/2 (ie. from Gauss’ formula) then in the general case we have: average search cost = ‘As a second, slightly more complicated example, let us consider the average case analysis for the binary search procedure described in algorithm 5.7. What we wish to establish in this analysis is the average number of iterations of the search loop that are required before the algorithm termi nates in a successful search. This analysis will correspond only to the first binary search implementation proposed in algorithm 5.7 which terminates as soon as the search value is found. The associated binary search tree for an array of size 15 is given in Fig. 4.2 Referring to the tree we see that 1 element can be found with 1 comparison, 2 elements with 2 comparisons, 4 elements with 3 comparisons, and so on, i. sum over all possible elements = 142+243+343+3+4+ +> In the general case, 2! elements require i+ 1 comparisons. Now assuming that all items present in the array are equally likely to be retrieved (i.e. their probability of retrieval is 1/n), then the average search cost is again just the sum over all possible search costs, each multiplied by their associated proba- bility. That is,? ven freer average search eost=2 grip} | SY ixzien ind ‘m0 + Where | {is used to indicate that [logon] is the greatest integer less than or equal to logon (it is referred to as “Floor” logy), and Sis the mathematical summation symbol,CHAP.1 38 COMPUTER PROBLEM-SOLVING / / supsueduos (1) 0 \ \ / il suosueduos 2 (21 _ uosuedwoo | 8 x Fig. 1.2 Binary decision tree for a set of 15 elementsBIBLIOGRAPIY 39 ‘This formula is exact only when a is one less than a power of two. Some calculus is needed to cvaluate the sum: Nosy] s SX ix2i We may recognize that this sum is a geometric progression and we know that in general: wind x-1 Takats o hak = We may also note that: ixz= f x), It follows that to compute our sum we can take the derivative of (x'=1)/(x~1) muttiptied by 2 and evaluated at x=2 where k= [log,n] When this is done we get: (n+)) (log nJ-1) +2 And, after substituting for the sum in our average search cost expression, we find: average search cost = @*) logun|+1 ~[Jogon) for large n. This example illustrates that even for a relatively simple example, a degree of mathematical sophistication is required to carry out the analysis. We now have some of the tools necessary to embark on a study of algorithm design. BIBLIOGRAPHY Problem-solving The classic book on problem-solving is Polya’s How to Solve (1971), which was first published in the 1940s. This book is highly recommended to all embarking on computer problem-solving. Polya’s other works on mathematical problem-solving contain a wealth of useful hints and exercises, Gardiner's Aha! Insight is also a delightful book. The other references listed also contain important ideas on problem-solving.40 COMPUTER PROBLEM-SOLVING cHaP. 1 1. Gardiner. M., Aha! Insight, Freeman, N.Y., 1978. 2. Polya, G., How to Soive It, Princeton University Press, Princeton, N.J., 1971 3. Polya, G., Mathematical Discovery: On Understanding, Learning and Teaching Problem Solving, Vols 1 and I, Wiley, N.Y., 1962, 4. Polya, G., Mathematics and Plausible Reasoning, Vols 1 and U1, Princeton University Press, Princeton, N.J., 1954. >. Watkins, K. P., Computer Problem-Solving, Wiley, Hong Kong, 1974. 6. Wickelgren, W. A., How to Solve Problems: Elemenis of a Theory of Problems and Problem-Solving, Freeman, S.F., 1974. Top-down design The references in this section provide a valuable scholarly treatment of structured programming. rats, Singers Verlag, N.Y., 1978 2. Dahl, O.., E. W. Dijkstra and C. A, R. Hoare, Swructured Progrumming, Academic Press, NLY., 1972. 3. Dijkstra, E. W., A Discipline of Programming, Prentice-Hall, Englewood Cliffs, N.J., 1976. 4, Wirth, N.. “Program development by stepwise refinement”. Comm. ACM, 14, 221-7, 1971 5. Wirth, N., Systematic Programming: An Introduction, Prentice-Hall Englewood Cliffs, N.J., 1973. Program verification The references in this section represent some of the most important and influential contributions to the discipline of computing science. The article by Kidman (1978) er introd paper by Wegbreit (1977) also provides important guidelines. The original and most influential contributions to the field of program verification were made by Floyd (1967) and Hoare (1969). 1. Elspas, B., K. N. Levitt, R. J. Waldinger and A. Waksman, “An assessment of techniques for proving program correctness”, Camp. Surv, 4, 97-147, 1972. 2. Floyd, R. W., “Assigning meanings to programs”, in Proc. Symp. in Appl. Math., Mathematical Aspects of Computer Science, ed. 3. T. Schwartz, Am, Math. Soc., Vol. 19, pp.19-32, 1967. 3. Dijkstra, E. W,, A Discipline of Programming, Prentice-Hall, Englewood Cliffs, N.J., 1976. 4, Hantler, S. L. and J. C. King, “An introduction to proving the correctness of programs”, Comp. Surv., 18, 331-53, 1976. 5. Hoare, C.'A. R.. “An axiomatic basis for computer programming”, Comm. ACM. 12, 576-80. 1969. 6 Kidman, BP “An introduotinn to program verification", Prac 8th Australian Comp. Conf., Canberra, Vol. 2, 977-902, 1978,BIBLIOGRAPHY 41 7. Linden, T. A., “A summary of progress toward proving program correctness". Fall Joint Computer Conf., Vol. 41, Part 1, 201-11, 1972 8 Manna, Z., Mathematical Theory of Computation, MeGraw-Hill, N.Y., 1974, 9. Ramamoorthy, C. V. and R. T. Yeh, Tutorial: Software Methodology, IEEE. Computer Socicty, Chicago, 1978. 10, Wegbreit, B., “Constructive methods in program verification”, IEEE Trans. Soft. kng., SE-3, 193-209, 1977. Design and analysis of algorithms The contribution of Knuth (1969, 1972) has been singularly outstanding in this field. The book by Aho et al. (1974) is also an important contribution. 1, Aho, A. V., J. E. Hoperoft and J. D. Ullman, The Design and Analysis of iputer Algorithms, Addison. y, Rea 2. Goodinan, S. E. and S. T. Hedetnicini, Introduction to the Design and Analysis of Algorithms, McGraw-Hill, N.Y., 1977 3. Guttmann, A. J., Programming and Algorithms, Heinemann, London, 1977. 4 Knuth, D. E., The Art of Computer Programming, Vol. 1: Fundamental Algorithms, Addison-Wesley, Reading, Mass., 196° 5. Knuth, D. E., “Mathematical analysis of algorithms”, Proceedings IFIP Con- gress, Ljubljana, 135-143, 1972 6. Maly, K. and A. R. Hanson, Fundamentals of Computing Science, Prentice- Hall, Englewood Cliffs, N.J., 1978.INTRODUCTION The computer has come close to rcaching the status of a universal machine because of its unique ability to perform an almost infinite range of tasks. It combines versatility with lightning speed. What comes as a surprise in this context is that only a very small number of basic instructions are used to perform all computations. At the most fundamental level a computer has instructions for storing information, for performing arithmetic, and compar- ing information. There are also instructions for cgnfrolling the way a compu- tation is performed. These very basic instructions are used to uitimately build the more powerful and complex instructions that we find as primitives in the high level programming languages like Pascal. At the programming language level the same thing happens. Once again, a few computational mechanisms are used to build much larger computational procedures. It is the latter fundamental mechanisms along with considcrations of computer information representation that we will examine in this chapter. One of the most widely used of these fundamental mechanisms is that for interchanging the values associated with two variables. In particular, this operation is widely used in many sorting algorithms. The idea of counting is also an essential part of many more elaborate computational procedures. An extension of the counting idea forms the basis of the often-used mechanism for summing a set of data. Seemingly, an unlimited number of problems can be expressed in a way that allows them to be solved by repeating a very basic mechanism over and over. For example, a set of n numbers can be summed by simply adding successive numbers from the set to an accumulating sum (see algorithm 2.3). The basic mechanism that is used in the iterative or repetitive process must cause changes in the variables and/or information considered by the mechanism. Only in this way dan progress be made towards termination of 42SEC. 2.1 EXCHANGING THE VALUES OF TWO VARIABLES 43 the iterative process which is always conditionally terminated In implementing computer solutions to problems, probably the most important skill that we need to develop is the ability to phrase problems ina way that will allow an iterative solution to be formulated. Unfortunately, for most of us, before our first encounter with computer problem-solving we have had iitie experience (at the conscious ievei, at ieast) of formuiating iterative solutions to problems. Time and again in our early programming days we are confronted by simple problems that we can solve very easily (e.g. like picking the maximum number in a set) but for which we have great difficulty in formulating an iterative solution. This happens because we can solve many problems without explicitly formulating or understanding the method used to obtain the solution. When dealing with computers, we are not granted such a luxury, Once we have grasped the idea of “thinking iteratively (and also recursively)”, we start to place ourselves in a position where we can formuiate computer soiutions to otherwise very compiex problems. Another foundation that we must build early in our computing days, is an understanding of the way in which computers represent and manipulate information. In particular. we must appreciate the differences between number and character representations of information and how to manipu Jate and convert between the various representations. Only after we have digested the essence of these fundamental concepts are we in a position to tackle the more involved aspects of computer probiem-soiving. Algorithm 2.1 EXCHANGING THE VALUES OF TWO VARIABLES Problem Given two variables, a and b, exchange the values assigned to them. Algorithm development The problem of interchanging the values associated with two variables involves a very fundamental mechanism that occurs in many sorting and data manipulation algorithms. To define the problem more clearly we will examine a specific example.44 FUNDAMENTAL ALGORITUMS CHAP. 2 Consider that the variables a and b are assigned values as outlined below. That is, Starting configuration a b 721 463 This means that memory cell or variable a contains the value 721, and memory cell or variable 6 contains the value 463. Our task is to replace the contents of a with 463, and the contents of 6 with 721. In other words we want to end up with the configuration below: Target configuration a b 463 Ti To change the value of a variable we can use the assignment operator. Because we want @ to assume the value currently belonging to , and b the value belonging to a we could perhaps make the exchange with the following assignments: a:—b; (ql) b:=a (2) where “:=" is the assignment operator. In (1) “:=" causes the value stored in memory cell 6 to be copied into memory cell a. Let us work through these two steps to make sure they have the desited effect. We started out with the configuration a b 72 463 then after execution of the assignment a:=b we have a b 463 463 The assignment (1) has changed the value of a but has left the value of b untouched. Checking with our target configuration we see that a has assumed the value 463 as required. So far so good! We must also check on b. When the assignment step (2) i.e. b:=a is made after executing step (1) we end up with: [40a] 463SEC.24 EXCHANGING THE VALUES OF TWO VARIABLES 45 In executing step (2) a is not changed while b takes on the value that currently belongs to a, The configuration that we have ended up with docs not represent the solution we are seeking. The problem arises because in making the assignment: a:=b we have lost the value that originally belonged to a (i.e. 721 has been lost). It isthis value that we want to finally assume. Our problem must therefore be stated more carefully as: new value of a new value of b : oid vaiue of ; old value of a What we have done with our present proposal is to make the assignment new value of b := new value of a Inother words when we execute step (2) we are notusing the value a that will make things work correctly---because a has already changed. To solve this exchange problem we need to find a way of not destroying “the old value of a” when we make the assignment =b A way to do this is to introduce a temporary variable ¢ and copy the original value of a into this variable before executing step (1). The steps to do this are: a =a; a: After these two steps we have a t b 463 72h 463 We are better off than last time because now we still have the old value of a stored in . It is this value that we need for assignment to b. We can therefore make the aysignment Pee After execution of this step we have: a ‘ b 463 [721] ma | Rechicching with uur iaiget configuiation we see dai wand b have wow been interchanged as required. ‘The exchange procedure can now be outlined46 FUNDAMENTAL ALGORITHMS CHAP.2 Algorithm description 1. Save the original value of a in 2. 2. Assign to @ the original value of b. 3. Assign to 6 the original value of @ that is stored in 1. The exchange mechanism as a programming tooi is most usefully implemented as a procedure that accepts two variables and returns their exchanged values. Pascal implementation procedure exchange (var a,b: integer); var ¢: integer; begin {save the original vatue of a then exchange a and b} assert: a =a0\bD =bU} 1. The use of an intermediate temporary variable allows the exchange of two variables to proceed correctly 2. This example emphasizes that at any stage in a computation a variable always assumes the value dictated by the most recent assignment made to that variable. 3. Working through the mech: useful way of detecting design faults. 4. A more common application of exchange involves two array elements (e.g. a{i] and a[j}). The steps for such an exchange are: ha pi Applications Sorting algorithms. ‘Supplementary problems 2.1.1 Giventwo giasses marked A and B. Giass A is full of raspberry drink and glass B is full of lemonade, Suggest a way of exchanging the contents of glasses A and B.SEC, 2.2 COUNTING 47 2.1.2 Design an algorithm that makes the following exchanges: ectoe The arrows indicate that b is to assume the value of a, c the value of b, and so on. Design an aigorithm thar makes the foliowing exchanges: M a eons 2.1.4 Given two variables of integer type a and b, exhange their values without using a third temporary variable. 2.1.5 What happens when the arguments given to the procedure exhange are i and afi}? Algorithm 2.2 COUNTING Problem Given a set of n students’ examination marks (in the range 0 to 100) make a count of the number of students that passed the examination. A pass is awarded for all marks of 50 and above. Algorithm development Counting mechanisms are very frequently used in computer algorithms. Generally a count must be made of the number of items in a set which possess some particular property or which satisfy some particular condition or conditions. This ctass of problems is typified by the “examination marks” problem, As a starting point for developing a computer algorithm for this prob- lem we can consider how we might solve a particular example by hand. Suppose that we are given the set of marks 55, 42, 77, 63, 29, 57, 89 ‘To make a count of the passes for this set we can start at the left, cnamine the first mark (Le. 55), see if it is 50, and remember that one student has passed so far. The secand mark is then examined and no adjustment is made ‘one to our previous count. marks have heen tested he process continues ina similar manner until all48 FUNDAMENTAL ALGORITHMS CHAP, 2 In more detail we have: Marks Counting details for passes 55 previous count~ 0 current count ~ 1 Order in 42 previous count = 1 current count = 1 which marks 77 previous count=1 current count = 2 are 63 previous count 2 current count ~ 3 examined 29 previous count = 3 current count = 3 57 _ previous count = 3 current count =4 89 previous count = 4 current count = 5 .. Number of students passed = 5 After each mark has been processed the current count reflects the number of students that have passed in the marks list so far encountered. We must now ask, how can the counting be achieved? From our exampie above we see that every time we need to increase the count we build ‘on the previous value. That is. current_count = previous_count+ 1 When, for example, we arrive at mark 57, we have previous_count = 3 Current_count therefore becomes 4. Similarly when we get to the next mark (ie. 89) the current_count of 4 must assume the role of previous count. This means that whenever a new current_count is generaied it must then assume the role of previous_count before the next mark is considered. The two steps in this process can be represented by current_count := previous_count+1 a = current_couni rea) previous_count These (wo steps cau be repeatedly applied to obtain the count required. In conjunction with the conditional test and input of the next mark we execute step (1), followed by step (2), followed by step (1), followed by step (2) and so on. Because of the way in which previous count is employed in step (1) we can substitute the expression for previous_count in step (2) into step (1) to obtain the simpler expression current_count := current_count+1 1 1 (new value) (old value) The current_count on the RHS (right-hand side) of the expression assumes < Tole of previous count. As is statcment involves an assignment rather than an equality (which would be impossible) it is a valid computer state-SEC. 2.2 COUNTING 49 ment. What it describes is the fact that the new value of current_count is obtained by adding 1 to the old value of current_count. ‘Viewing the mechanism in this way makes it clear that the existence of the variable previous count in its own right is unnecessary. As a result we have a simpler counting mechanism. The essentiai steps in our pass-counting aigorithm can therefore be summarized as: while less then m marks have been examined do (a) read next mark, (6) if current mark satisfies pass requirement then add one to count. Before any marks have been examined the count must have the value zero. To complete the algorithm the input of the marks and the output of the number of passes must be included. The detailed algorithm is then as described below. Algorithm description 1. Prompt then read the number of marks to be processed. 2. Initialize count to zero. 3 While there are still marks to be processed repeatedly do (a) read next mark, ib) if it is a pass (ie. #50) then add one iv coun. 4, Write out total number of passes. Pascal implementation program passcount {input, output}; const passimark =50; var count {contains number of passes on termination}, i {current number of marks processed}, m {current mark}, 1n {total number of marks to be processed}: integer; begin {count the number of passes (>=50) in @ set of marks} writeln (enter a number of marks n on a separate line followed by the marks’); readin (n); assert: n> =0} court i= 0; f= 0; {invariant : count =| are >= passmark \. while i =passmark then count end; jassert: count =number of passes in the set of n marks read} writeln (number of passes =’, count) end. count+1 Notes on design 1 Initially, and each time through the loop, the variable count represents the number of passes so far encountered, On termination (when i= 1) count represents the total count represents the total incremented by 1 with each iteration, eventually the condition i1 can be generated iteratively. The instance where n= Qis a special case which cannot be generated iteratively. The sum of zero numbers is zero and so we can generate the first sum directly by the assignment s:=0 The core of the algorithm for summing # numbers therefore involves a special step followed by a set of x iterative steps. That is, 1. Compute first sum (s= 0) as special case. 2, Build each of the n remaining sums from its predecessor by an iterative process. 3. Write out the sum of # numbers.54 FUNDAMENTAL ALGORITHMS CHAP. 2 The only other considerations involve the input of n, the number of numbers to be summed, and the input of successive numbers with each iterative step. Our complete algorithm can now be outlined. Algorithm description Prompt and read in the number of numbers to be summed. Initialize sum for zero numbers. . While iess than n numbers have been summed repeatediy do (a) read in next number. (b) compute current sum by adding the number read to the most Tecent sum. 4, Write out sum of m numbers. eee Pascal implementationt program sum (input, output);t var i {summing loop index}, 1n (number of numbers to be summed}: integer; a {current number to be summed}, $ {sum of n numbers on termination}: real; begin {computes sum of n real numbers for n> =0} writein (input n an a separate line, followed by the nuinbers to be summed’); readin (n); Yinvariant: s=sum of first | numbers read Ni= a The harmonic mean defined by N a & is sometimes used 2s a mean of central tendency. Develup an algorithm to compute the harmonic mean of n data values. 2.3.5 Develop an algorithm to compute the sums for the first terms (120) of the following series: fa) s- 142434 (a) s7 142434 (b) = 143454 + (c) s=244+64 (A) 5=141/241/3+ - 2.3.6 Generate the first terms of the sequence 12.4 8 16 32 without using multiplication 2.3.7 Develop an algorithm that prints out # values of the sequence 1-11-11 -1 2.3.8 Develop an algorithm to compute the sum of the first x terms (x21) of the series S=1-34+5-749- Algorithm 2.4 FACTORIAL COMPUTATION Problem Given a number n, compute n factorial (written as !) where 130. Algorithm development We can start the development of this algorithm by examining the definitionSEC. 24 FACTORIAL COMPUTATION $7 of nl, We are given that m= 1x2X3x e+ x(n-)xn for na 1 and by definition Ol=1 in formuiating our design for this probiem we need to keep in mind that the computer’s arithmetic unit can only multiply evo numbers at a time. Applying the factorial definition we get Ol=1 Haixt 21= 1x2 Bt= 1x2x3 Am 1x2x3%4 We sce that 4! contains all the factors of 31. The only difference is the inclusion of the number 4, We can generalize this by observing that n! can always be obtained from (n—1)! by simply multiplying it by » (for 11). That is, nt=nx(n-1)l for n=1 Using this definition we can write the first few factorials as: 1!=1x0! 21= 2x1! 31=3x2! 41= 4x3! If we start with p= 0! = 1 we can rewrite the first few steps in computing n! as: ay From step (2) onwards we are actually repeating the same process over and over. For the general (i+1)'* step we have prs pei (+1) ‘This general step can be placed in a loop to iteratively generate n!. ‘This allows us to take advantage of the fact that the computer's arithmetic unit can ouly multiply two numbers al a tine, In many ways this problem is very much like the problem of summing a set of n numbers (algorithm 2.3). In the summation problem we performed a$8 PUNDAMENTAL ALGORITHMS cuar. 2 set of additions, whereas in this problem we need to generate a set of products. It follows from the general (i+1)"* step that all factorials for n> 1 can be generated iteratively. The instance where n=Qisa special case which must be accounted for directly by the assignment 1 (by definition of 01 The central part of the algorithm for computing n! therefore involves a special initial step followed by 7 iterative steps. Dp 1. Treat 0! as a special case (p := 1). 2. Build each of the m remaining products p from its predecessor by an iterative process. 3. Write out the value of n factorial. Algorithm description 1. Establish n, the factorial required where n=0. 2. Set product p for 0! (special case). Also set product count to zero. 3. While less than » products have been calculated repeatediy do {a) increment product count, {b) compute the #* product p by multiplying i by the most recent product 4. Return the result n!. This algorithm is most usefully implemented as a function that accepts as input a number n and returns as output the value of nt. In the Pascal implementation p has been replaced by the variable factor. Pascal implementation function nfactorial (n :integer): integer; var j {ioup index represemting ith factorial} integer; factor {i:integer; begin {computes and returns a! for n>=0} fassert:n > =0} factor := 1; {invariant : factor ~i\ after the ith iteration Ni j:= ton do fassert: nfactoriai =n} endSte. 2 4 PACTORIAL COMPUTATION 59 Notes on design 1. ‘The algorithm uses n muitiplications to compute n!, There is in fact a more efficient algorithm that computes a! in essentially log. steps (See note 5 below). After the i time through the loop the value of factor is i!. This condition holds for all 7. On termination (when i~ n) factor will corres pond to n!. Because é is incremented by 1 with each iteration, eventu- ally the condition = x wili be reached and the loop will terminate. The algorithm performs correctly for all n>0. However, no consideration is given to the finite size of numbers that can be represented in the computer. Careful definition of the problem and examination of some specific examples is central to the development of the algorithm. The simplest problem leads to a generalization that forms the basis of the algorithm, ‘The idea of accumulating products is very similar to that of accumula ting sums (see algorithm 2.3). A more efficient mechanism is based on the fact that it is possible to express n! in terms of (71/2)!. In principle, the mechanism is similar to Ghai used in caiculating the 2 Fibonacei number (ref. A. Shamir, “Factoring numbers in O(log n} arithmetic steps,” Inf. Proc, Letis., 8 28-31, 1979). See also algorithm 3.8 Applications Probability, statistical and mathematical computations Supplementary problems 2.4.6 For a given n, design an algorithm to compute 1/n! For a given x and a given 7, design an algorithm to compute x"/n!. Design an algorithm to determine whether or not a number n is a factorial number. Design an algorithm which, given sone integes 1, finds the largest factorial number present as a factor in n. Design an algorithm to simulate multiplication by addition. Your program should accept as input two integers (they may be zero, positive, or negative). ‘The binomial theorem of basic algebra indicates that the coefficient °C, of the r” power of x in the expansion of (x+1)" is given by rin=n)!60 FUNDAMENTAL ALGORITHMS CHAP. 2 Design an algorithm that evaluates all coefficients of x for a given value of 7. Algorithm 2.5 SINE FUNCTION COMPUTATION series expansion sin (x) Algorithm development This problem embodies some of the techniques we have seen in earlier algorithms. Studying the expression for sin (x) we see that powers and factorials following the sequence 1,3, are required. We can easily generate this odd sequence by starting with 1 and successively adding 2, Gur otfier problem is iv compute the general term x/i| which can be expressed as 1, XXXL Bp y*a* ‘The function we need to compute this will involve the following steps: fp i while jerror do begin {generate and accumulate successive terms of sine expression} (25542; term := ~term * x2 / ( * (1); tsin := tin +term eni sin := tsin {assert: sin =sine (x)Nabs (term)=1. ‘The first few terms are: 0,1, 1, 2,3, 5,8, 13, .. Each term beyond the first two is derived from the sum of its two nearest predecessors. Algorithm development From the definition we are given that: new term = preceding term+term before preceding term ‘The last sentence of the problem statement suggests we may be able to use the definition to generate consecutive terms (apart from the first two) iteratively. Let us detine: @ as the term before the preceding term bas the preceding term © new term ‘Then to start with we have: aie ft first Pibonace’ number b:=i second Fibonacci number and c:= ath third Fibanacci number (from definition)SEC. 2.6 GENERATION OF PIBONACCI SEQUENCE 65 When the new term ¢ has heen generated we have the third Fibonacci number. To gencrate the fourth, or next Fibonacci number, we need to apply the same definition again. Before we can make this next computation we need to make some adjustments, The fourth Fibonacci number is derived from the sum of the second and third Fibonacci numbers. With regard to the definition the second Fibonacci number has the roie of the term before tie preceding term and the third Fibonacci number has the role of “the preceding term”. Therefore, before making the next (i.e. the fourth) computation we must ensure that: (a) new term (i.e. the third) assumes the role of the preceding term. (b) and what is currently the preceding term must assume the role of the term before the preceding term. That is, [1] term before preceding term [2] preceding term [3] new term [4] term hefare preceding term hecames preced- ing term bi=e [5] preceding term becomes new term After making step [5] we are in a position where we can use the definition to generate the next Fihanacci number. A way to do this is to loop back to step [3]. Further investigation of steps [3] >[5] indicates they can be placed ina toop to iteratively generate Fibonacci numbers (for n>2). The essential mechanism we could use i:=2; while i 0} (invariant: after jth iteration i =2) + 2/‘irst f Fi, numbers have been generated Na = 1th Fib. no. Ab=ith Fib. no. Ais1. ‘Throughout the computation the variables a and b always contain the two most recently generated Fibonacci numbers. Therefore, whenever an addition is made to generate the next Fibonacci number, the requirements for that number to be a Fibonacci number are always will eventually be violated and the algorithm will terminate,68 FUNDAMENTAL ALGORITHMS CHAP. 2 3. The second algorithm is more efficient than the first algorithm hecause it makes only one assignment per Fibonacci number generated, The frst algorithm makes three assignments per Fibonacs! number gener ated. 4. With the present algorithm Fibonacci numbers are generated in pairs. When 7 is odd one more number will be generated than is required. It will not. however. be printed. This is a small sacrifice for the other gains. 5. A more advanced algorithm that can calculate the n‘* Fibonacci number in log.n steps will be discussed in algorithm (3.8). Applications The Fibonacci sequence has practical applications in botany, electrical nct- work theory, sorting and searching. Supplementary problems 2.6.1 implement the Fibonacci algorithm as a function thai aecepis as input two consecutive Fibonacci numbers and returns as output the next Fibonacci number. 2.6.2 The first few numbers of the Lucas sequence which is a variation on the Fibonacci sequence are: 13 4 7 11 1 2 .. Design an algorithm to generate the Lucas sequence. 26.3 Given a=0, b=1, and c=1 are the first three numbers of some scyucnve. All viher uuubers in the scquence aie generated frum the sum of their three most recent predecessors. Design an algorithm to generate this sequence 2.6.4 Given that two numbers d and e are suspected of being consecutive members of the Fibonacci sequence design an algorithm that will refute or confirm this conjecture. 2.6.5 The ascending sequence of all reduced fractions between 0 and 1 which have denominators 1 (vita) Mea | eon | Sent no=[ 222 err Design an algorithm to generate the Farey series for a given n. (See D. E. Knuth, The Art of Computer Programming, Vol. 1, Funda- mental Algorithms, Addison-Wesley, Reading, Mass., p. 157. Generate the sequence where each member is the sum of adjacent factorials, i.e. f= +0! faUtu fs= 3142! Note that by definition 0!= 1. Algorithm 2.7 REVERSING THE DIGITS OF AN INTEGER Problem Design an algorithm that accepts a positive integer and reverses the order of its digits. Algorithm development Digit reversal is a technique that is sometimes used in computing to remove bias from a set of numbers. It is important in some fast information-retrieval algorithms. A specific example clearly defines the relationship of the input to the desired output. For example, input: 27953 Output: 3597270 PUNDAMENTAL ALGORITHMS cuaP. 2 Although we might not know at this stage exactly how we are going to make this reversal one thing is clear—we are going to need to access individual digits of the input number. As a starting point we wilt concentrate on this aspect of the procedure. The number 27953 is actually 2x 105+ 7X10 49x 1 +5x 1043 To access the individual digits it is probably going to be easiest to start at one end of the number and work through to the other end, The question is at which end should we start? Because other than visually it is not easy to tell how many digits there are in the input number it will be best to try to establish the identity of the least significant digit (j.c. the rightmost digit), To do this we need to effectively “chop off” the least significant digit in the number. In other words we want to end up with 2795 with the 3 removed and identified, We can get the number 2° by 10 5 by integer division of the originai number ie. 27953 div 10-2795 This chops off the 3 but does not allow us to save it. However, 3 is the remainder that results from dividing 27953 by 10. To get this remainder we can use the mod function, That is, 27953 mod 10-3 1efure if we appiy tire following wo steps ri=n mod 10 (1)=>(r=3) n= n div 10 (2)=>(n = 2795) we get the digit 3, and the new number 2795, Applying the same two steps to the new value of 1 we can obtain the 5 digit. We now have a mechanism for iteratively accessing the individual digits of the input number. Our next major concern is to carry out the digit reversal, When we apply our digit extraction procedure to the first two digits we acquire first the 3. and then 5, In the final output they appear as: 3 followed by 5 (ur 35) If the original number was 53 then we could obtain its reverse by first extracting the 3, multiplying it by 10, and then adding 5 to give 35. That is, 3X14 5-335 ‘The last three digits of the input number are 953. They appear in the “reversed” number as 359. Therefore at the stage when we have the 35 and then catraci ihe 9 we cau ubiain and adding 9. That is, © scyuctiec 359 by utultiptying 35 by 16SEC, 2.7 REVERSING THE DIGITS OF AN INTEGER 71 35x 149-9359 Similarly 359% 1047-73597 and 3507 10+2-+35972 ‘The last number obtained from the multiplication and addition process is the “‘digit-reversed” integer we have been seeking. On closely examining the digit cxtraction, and the reversal process, it is evident that they both involve a set of steps that can be performed iteratively. now find a mechanism for building up the “reversed” integer it. Let us assume that the variable dreverse is to be used to build the reversed integer. At each stage in building the reversed integer its previous value is used in conjunction with the most recently extracted digit. Rewriting the multiplication and addition process we have just described in terms of the variable dreverse we get Iteration Value of dreverse dreverse*10+3 3 dreverse*10+5 35 dreverse*10+9 359 (1) dreverse [2] dreverse : [3] reverse ‘Therefore to build the reversed integer we can use the construct: dreverse (previous value of dreverse)*10 +(most recently extracted rightmost digit) The variable dreverse can be used on both sides of this expression. For the value of dreverse to be correct (i.c. dreverse ~ 3) after the first iteration it ‘must initially be zero. This initialization step for dreverse is also needed to ensure that the algorithm functions correctly when the input number to be reversed is zero, What we have not established yet is under what conditions should the iterative process terminate. The termination condition must in some way be related to the number of digits in the input integer. In fact as soon as all digits have been extracted and processed termination should apply. With each iteration the number of digits in the number being reversed is reduced by one, yielding the sequence shown in Table 2.1. Accumulative integer divi sion of the “number being reversed” by 10 produces the sequence 27953, 2795, 279, ... . In our example, when the integer division process is applied for a St time a zero results since 2 is less than 10. Since at this point in the computation the “reversed” number nas been fully construcied we can use the zero result to terminate the iterative process.72 FUNDAMENTAL ALGORITUMS. CHAP. 2 Table 2.1 Steps in digit reversal Number being reversed Reversed number being constructed Step 97988 3 ty) 2795 38 real 29 359 B 7 3597 i} 2 35972 [5] The central steps in our digit reversal algorithm are: 1. While there are still digits in the number being reversed da (a) extract the righmost digit from the number being reversed and append this digit to the right-hand end of the current reversed number representation; (b) remove the rightmost digit from the number being reversed. When we include input and output considerations and details on initializa~ tion and termination we arrive at the following algorithm description. Algorithm description 1. Establish n, the positive integer to be reversed. 2. Set the initial condition for the reversed integer dreverse. . While the integer being reversed is greater than zero do & {a) use the remainder function to extract the rightmost digit of the number being reversed; (b) increase the previous reversed integer representation dreverse by a factor of 10. and addto it the most recentiy extracted digit 10 give the current dreverse value; {c) use integer division by 10 to remove the rightmost digit from the number being reversed. This algorithm is most suitably implemented as a function which accepts as input the integer to be reversed and returns as output the integer with its digits reversed. Pascal implementation function reverse (n: integer): integer; var reverse: integer;SEC.2.7 REVERSING THE DIGITS OF AN INTEGER 73 {invariant: after jth iteration, n=al1), a(2), al), ... alk-f)A reverse =alk}, ak~1), ..., Hk—j +1} while» >0 do begin reverse := reverse * 101 mod 10; jassert: reverse =a(k), ak —1), .. (1) dreverse := reverse end Notes on design 1. 2. ‘The number of steps to reverse the digits in an integer is directly proportional to the number of digits in the integer. After the i time through the loop (/ is not explicitly defined in the algorithm) the variable dreverse contains the i leftmost digits in the reversed integer. This condition remains invariant for alt i, Also after i iterations the variable n is reduced by / digits. On termination when n has been reduced to zero digits the variable dreverse will contain the same number of digits as the input integer. The aigorithm will ter- minate because the variable n is reduced by one more digit with each iteration. The algorithm performs correctly for all values of n>0. In this design we see once again how a complete solution to a problem is built iteratively from a succession of partial solutions. This is the fundamental design framework for many algorithms. In designing this algorithm we have implicitly been working back from the solution to the starting point. This idea of working backwards is a very powerful and important concept in computing science which we wiii expioit much more expiicitiy in the design of more advanced algorithms. ‘A specific example helps to lead us to a method for building the reversed representation Applications Hashing and information retrieval, data base applications. Supplementary problems Design an algorithm that counts the number of digits in an integer. Design an algorithm to sum the digits in an integer. Design an algorithm that reads in a set of n single digits and converts them into a single decimal integer. For example, the algorithm should convert the set of 5 digits {2,7,4,9,3} to the integer 27493.74 FUNDAMENTAL ALGORITHMS CHAP. 2 Algorithm 2.8 BASE CONVERSION Probiem Convert a decimal integer to its corresponding octal represemation. Aigorithm deveiopment Frequently in computing it is necessary to convert a decimal number to the binary. octal or hexadecimal number systems. To design an algorithm for such conversions we need to understand clearly what the base change entaiis. Because initially we probably have no firm ideas on the mechanism for base conversion we will begin with some groundwork. We can start by trying to come to terms with exactly what is a decimal number and an octal number. For this exploration we can look at some specific examples. The decimal (i.c. base 10) number 275 by its representation, is seen to consist of: 5 units 5x1 7 teas 7x10 hundreds 2% 100 275 The decimal system uses the ten digits 0.1.2.3. .... 9 to represent numbers. The actual position of cach digit in a number determines its value. Similar conventions apply for the octal (or base 8) number system. The octal system uses only the eight digits 0.1.2.3. .... 7 to represent numbers. In the octal system the position of each digit determines its value in a similar (but different) way to the decimal system. Taking a few steps ahead for the purposes of illustration it can be shown that the octal representation of the decimal number 275 is 423. The octal number consists of: 3 units 3x1 2 eights 2x8 4 sixty-fours 4x64 275 decimal As another illustration in contrast to the decimal number 275 (written as 275,9) the octat number 275 {written as 275,) consists of:SEC, 28 BASE CONVERSION 75 5 units 5x1 7 eights 1x8 2sixty-fours 2x64 189 decimal We can see that 275, is much smaller in magnitude than 2751). With this groundwork complete we can now return to our base conver- sion problem. We have learned that in changing the base of a number we are not in any way changing the magnitude of the number but rather we are changing the way the number is represented. As a Starting point we will consider the conversion of a particular decimal number to its octal representation. This will hopefully reveal much of the mechanics of the algorithm we are seeking. Consider that we wish to convert the decimal number 93 to its octal representation. We know that 93 is made up of 3 units and 9 tens, In diagram form we have: 0 10. 2 30 40 fone O38 decimal The corresponding octal number will need to be divided up into “blocks” of units, eights, sixty-fours and so on, rather than units, tens, hundreds and so on, Using a diagram for the octal representation we get: 1 3 Sixty-four ——____>-— Fights > 0 8 16 24 32 40 48 56 64 72 80 88 9396 oe ry itl ee 2 Octal swuns From our diagram we can see that when we divide 93 decimal up into blocks of eight there are 5 units left over at the end. Our octal representation must therefore be of the form ...5. In dividing 93,, up into blocks of cight we also discover that there are 11 blocks of eight (i.e. 118 = 88) present in 93,,. At this stage we might be tempted to write down 115 as our octal representation of 93,,. There is a problem with this, however, because 11 is not a valid octal igit {the oniy octai digits that we can use are 0,1,2,3, ..., 7). We might76 FUNDAMENTAL ALGORITHMS CHAP. 2 therefore ask where do we go from here? With only eight digits the max- imum number of cights we can represent is 7. Remembering our previous examples we know that sixty-fours can be used in the representation. In fact 8 eights can (and must) be represented as 1 sixty-four. It follows that 11 eights can be represented as I sixty-four and 3 eights. Applying the posi- tionai conventions we have encountered earlier we can conciude that the octal representation of 93 decimal is 135. The octal number 135 consists of 5 units Sx1 3 eights 3x8 L sixty-four 1x64. 93 decimal Using a diagram we have arrived at the octal representation for 93... What we must now do is translate what was done with the diagram into an algorithm. Integer division of 93 by 8 will tell us how many eights are in 93,9; that is, 93 div 8-11 cights The number of sixty-fours in 93,, must also be found, This involves breaking the 11 eights into 1 sixty-four and 3 eights. To be systematic about the process we should first find the units, then the eights, then the sixty-fours, and then the five-hundred-and-twelves and so on. We have 93 units->11 eights and 5 units 11 eights—1 sixty-four and 3 eights 1 sixty-four—0 five-hundred-and-twelves and 1 sixty-four 8 [93 8 [41] 5 remainder 8 1 3 remainder 0 1 remainder In changing a decimal number to its octal representation we start by dividing the decimal number by 8. The remainder of this operation is the least significant digit (i.e. $) of the octal representation, The quotient (ie, 11) is then taken and divided by 8. The remainder of this operation is the second east significant digit of the octal representation. The process of dividing the quotient and using the remainder as the next least significant digit of the octal representation continues until a zero quotient is encountered. This representation (in the example above a zero quotient after 3 divistons by 8