INTRODUCTION TO ALGORITHMS NOTES1 More generally, the study of algorithms helps define the science of computing and provide

rithms helps define the science of computing and provide the focus for
by its study (Brookshear 2000). For example, the study of algorithms is the common theme through these
William S. Shu core questions in Computer Science:
1) Which problems can be solved by algorithmic processes? (Limitations of algorithms.)
2) How can the discovery of algorithms be made easier? (Discovery of algorithms.)
3) How can the techniques of representing and communicating algorithms be improved?
Books: (Representation and communication of algorithms.)
• Brookshear, J. G., Computer Science: An overview, 6th edition, Addison-Wesley, Reading, 4) How can our knowledge of algorithms and technology be be applied to provide better machines?
Massachusetts, 2000. Later editions available! (Execution of algorithms.)
• Berman, K. A. and Paul, J. L., “Algorithms: Sequential, Parallel and Distributed”, Thomson 5) How can the characteristics of algorithms be analysed and compared? (Analysis of algorithms.)
Course Technology, 2005 or later.
In this course, we give a really slow introduction to algorithms as expressed in terms of items (1) and (3)
Further References: of the above definition of Computer Science, but also touch on all the facets of algorithms raised by the
above questions. However, beyond a basic understanding and an intuitive grasp, the student is expected
• Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest. Introduction to Algorithms, 3rd
to reinforce in subsequent courses the ideas so introduced.
Edition, MIT Press, 2009 - ISBN 978-0-262-033384-8.
• Elaine Rich, “Automata, Computability, and Complexity: Theory and Applications”, Pearson
The delivery strategy of this course therefore seeks to introduce solid foundations for algorithms and
Prentice Hall, Upper Saddle River, NJ 07458, 2007.
their development (notably in algorithmic thinking, problem solving and correctness); disambiguate
• Shu, W. S., “Essential, Basic and Practical Pascal”, ANGWECAM, Yaounde, Cameroon, 2006. terminology and the use of concepts introduced, as they ramify into various aspects of computing; and
• Connolly Thomas & Begg Carolyn, “Database Systems: A practical Approach to Design, lay down a foundation for, and interfaces with, other possibly unrelated courses. Many of the answers to
Implementation and Management”, 4th Edition, 2004. Addison-Wesley Longman Ltd. exercises, homework, etc. are in the notes, and the student is well-advised to attempt them first, before
• Brass, Peter, “Advanced Data Structures”, Cambridge University Press, 2008. reading further.
One-off incidental accesses (during earlier development of notes):
• David Harel, “Algorithmics: The Spirit of Computing”, Addison-Wesley, 1987, 2nd Ed 1992. INTRODUCTORY NOTIONS ON ALGORITHMS
• David Gries, “The Science of Programming”, Springer-Verlag, New York, 1981. We motivate the need for algorithms, and its correctness, via an example of Euclid's algorithm as
• Edsger W. Dijkstra, W.H.J. Feijen, “A Method of Programming”, Addison-Wesley, 1988. presented by Walker (1996), and so set the tone (and bias) in the studies of algorithms for this course.
• Edsger W. Dijkstra, “A Discipline of Programming”, Prentice Hall, Englewood Cliffs, 1976. The example illustrates a number of things when seeking a solution. For example, we need:
• Assorted others (Knuth “bible”, etc.) • To be creative; to figure out ways in which we can obtain a solution. A strategy is to express the
problem as a simpler version of itself, usually after many tries. (c.f., HCF(2496396, 1193619) =
HCF(1193619, 109158)).
INTRODUCTION • To ensure that our reasoning for the solution is sound and provably correct. Notice the
Algorithms are fundamental to Computer Science. In fact, a widely accepted definitions (Norman E. mathematical basis of the reasoning.
Gibbs and Allen B. Tucker, “A Model Curriculum for a Liberal Arts Degree in Computer Science”, • To be able to have an answer and know whether or not it is correct. (e.g., I lied.)
Communications of the ACM, 29(3), March 1986.) is: “Computer Science is the study of algorithms, • To formulate our solution in a way that anyone who understands it can follow it. See the
especially their: summary of the solution in the last-but-one paragraph.
1) Mathematical properties: Correctness, efficiency, Complexity (time and space), etc. • To remember that there may be many variants of the same solution; we may have to choose one,
2) Hardware realisations: Logic gates, circuits, architecture, etc. with justification.
3) Software realisations: Programming and Programming methodologies, etc.
4) Applications to other disciplines: Mathematics, Physics, Engineering, Business, etc.” The above notwithstanding, we now examine, at a gentler pace, algorithms and their characteristics.

Thus, items (2) and (3) implement or represent the same “thing” (i.e., algorithm) that is described by Algorithms and their Characteristics
item (1). Item (4) uses algorithms to solve problems in specific disciplines. It demonstrates the We are familiar with procedures used everyday to carry out tasks. Some of them include:
usefulness of algorithms (and hence Computer Science) but is typically not the subject of study (or • Descriptions to a friend's house.
interest) in either. (Note: In this course, item 4 is “applications to problems in Computer Science”.) • Recipes to cook or to bake a cake.
• Instructions to set up an equipment or furniture.

While humans easily understand such procedures, giving instructions to machines (computers) are a bit
1© 1994-2018, William S. Shu
more complicated. We use the term “algorithms” for procedures given to machines. That is, the
All rights reserved. No part of these notes may be reproduced or transmitted in any form, or by any
instructions given to an agent (machine) so that it carries out a designated task. More generally, a
means electronic or mechanical, including photocopying, recording or any information storage or
[computing] agent is the entity—machine, robot, person(s), or whatever—that carries out (executes) the
retrieval system, without the prior written permission of the author. Permission is hereby granted to the
instructions in an algorithm.
applicable batches of students taking any course in Computer Science at the University of Buea,
Cameroon, to use this document for academic purposes only, and in as many copies as desired.

wss introduction to algorithms—notes of 08 October 2018 page 1 of 107 wss introduction to algorithms—notes of 08 October 2018 page 2 of 107
Motivating Example (Copyright © Dr A. N. Walker, 1996.) We use the maze adjacent to illustrate
Let us start with a problem. What is the HCF [highest common factor, same as GCD or greatest some of the things that help us with
common divisor] of 1193619 and 2496396? algorithmic thinking. The maze comprises
a number of blocks (light shade), and a
... pause ... number of paths R1 to R12 that can be
followed to reach various parts of the
Suppose I tell you it's 21. Do you believe me? You can easily verify that 21 is a factor of both these
blocks. An impenetrable barrier separates
numbers, how can we tell whether or not there are larger common factors?
the left and right sides of the maze, making
it impossible to move from one side to
... pause ...
another. For example, consider the problem
of moving from Point F to Point A. There
For this we need a process or algorithm for finding HCFs. You may already know one such algorithm;
are several way to solve this problem.
it consists of finding all the prime factors of the numbers concerned, then you can list all the common
prime factors, and 'obviously' the HCF is the product of these. But finding prime factors is a rather
Figure 1: Traversing a maze. © 2014, Denis L. Nkweteyim (Workshop on Algorithms, Data Structures,
time-consuming operation for large numbers; doing it efficiently needs very complicated computer
and Fundamental Programming Constructs at University of Buea, 13-16 August 2014.)
programs, and still takes forever and a day when the numbers are large enough to be of cryptographic
interest [but that's another story]. So, we look instead at a process known as Euclid's algorithm. This There are a number of definitions, mostly informal, of the term “algorithm”. The first we give is based
works as follows: on the characteristics [of instructions] of an algorithm (c.f., Shu 1996); the second based on algorithms
transforming data to produce results (c.f., Cormen et al., 2009); a third is based on properties exhibited
Divide 2496396 by 1193619. The result is 2, with remainder 109158. In other words, 2496396 = 2 × by an algorithm as a whole (c.f., Brookshear 2004); and the fourth, alluded to much later in the course,
1193619 + 109158. What has this gained us? Two things: sees algorithms as machines (solutions) to problems encoded (represented) as a formal [mathematical]
• It is easy to see from this equation that any number which divides both 2496396 and 1193619 language (c.f., Rich 2007). The earlier part of this course is biased towards the first and third definitions,
also divides both 1193619 and 109158, and vice versa. So, the set of divisors of either pair is with more emphasis on the third.
the same as the set of divisors of the other pair, and in particular, the greatest common divisor
is the same. [But note that common divisors of 2496396 and 109158, such as 2, do not Algorithm Definition 1 (by characteristics): An algorithm is a finite, well-ordered collection of well-
necessarily divide 1193619, because of the quotient.] We have found another pair of numbers defined, unambiguous steps (instructions) that can be carried out mechanically in a finite length of time
with the same HCF. and with a finite amount of effort in order to accomplish a given task.
• 109158 is simpler than 2496396.
The characteristics highlighted by the definition above include the following:
So, instead of finding the HCF of 2496396 and 1193619, we can instead find that of 1193619 and • Well-defined: An agent carrying out the task does so precisely and understands what must be
109158; the answer will be the same, and the problem is easier because the numbers are smaller. How done at each step. That is, with each instruction.
do we find the new HCF? Why, by the same method, of course; divide 1193619 by 109158, find the • Unambiguous: There is only one possible way of doing each step (or a set of them).
remainder and set up an even easier problem. Keep going until the answer is obvious. • Mechanically: An agent does not need to reason (imagine) in order to carry out a task; it is not
intelligent. (However intelligence is defined!)
What do I mean by the answer being 'obvious'? Let's continue this process and see:
• Finite time: Algorithm should not take forever to give a solution.
Divide 2496396 by 1193619; quotient 2 remainder 109158.
• Finite collection: There are a finite number of instructions in the algorithm.
Divide 1193619 by 109158; quotient 10 remainder 102039.
Divide 109158 by 102039; quotient 1 remainder 7119. • Well-ordered collection: Instructions (steps) in the algorithm are executed in a certain order.
Divide 102039 by 7119; quotient 14 remainder 2373. • Finite effort: Ensure each step is do-able.
Divide 7119 by 2373; quotient 3 remainder 0. • Accomplish Task: An algorithm solves the problem in question, not some other problem. It halts
Divide 2373 by 0; oops! after solving the problem; it cannot, after solving the problem, go on to solve other problem(s).

We can't divide by zero, but luckily the HCF of 2373 and 0 is 'obviously' 2373, so the answer is 2373
[and I lied -- note that 2373 = 21 × 113]. In summary, the process is: Basic Actions Solution 1: Solution 2:
To find the HCF of two non-negative integers, replace the larger by the remainder when it is divided by 1) Move <direction> along <path>. (1) Move West along R3. (1) Move West along R3.
the smaller. Repeat until the smaller number is zero, at which stage the answer is the other number. 2) Turn <direction> at <position>. (2) Turn Right at B. (2) Turn Right at C.
3) Stop at <position>. (3) Move Forward along R1. (3) Move Forward along R2.
Turning this process into a computer program is left as an exercise. [Note that after the first division, (4) Stop at A. (4) Turn Right at D.
you always know which is the smaller number, and before it doesn't matter except for wasting the first (5) Move Forward along R6.
division; note also that you should test for zero before doing the division!] There are many variants of (6) Turn Left at E.
this technique: one is to replace division by subtraction, which is easier to analyse and program but (7) Move Forward along R1.
gets very slow if the quotient is large; another uses binary shifts and subtractions, which can be made (8) Stop at A.
very efficient in computer machine codes but is harder to explain.

wss introduction to algorithms—notes of 08 October 2018 page 3 of 107 wss introduction to algorithms—notes of 08 October 2018 page 4 of 107
Note:
1) The task carried out by an algorithm will normally be given some data to use (input) and a result Equivalently, given the above explanations, an algorithm is a well-defined computational sequence that
(output) is expected. (See also Algorithm Definition 2.) transforms input into output. An algorithm thus seeks to solve a specific computational problem. A
2) When an algorithm stops executing and cannot continue further (c.f., when it accomplishes a task, computational problem specifies in general terms the desired relationship between the input and the
above), we say that the algorithm halts. In contrast, when an algorithm stops, it can resume later. output, and an algorithm describes a specific computational procedure to achieve it. This view of
3) A more formal definition of algorithm demands understanding more technical concepts such as algorithms is useful in the analysis [and design] of algorithms (see later).
“computation” (see later) and “termination”.
4) An algorithm that does not terminate is called a process; they are part of a class of algorithms called Thus, using our example above, our computational problem is getting from F to A, and solutions 1 and 2
pseudo-algorithms. (See pseudo-algorithms later.) A process, as used here, is a technical term, though are specific algorithms that describe the computational procedure.
some authors consider them as algorithms too.
Homework:
Example Q1) Suppose you are told to give the result of x squared where x is a (any) non-negative integer.
Consider the maze as described in Figure 1 above. Its basic operations (steps) are given above where (i) State the computational problem in terms of prose descriptions that clearly state:
<direction> is one of left, right, up, down, 90o left, etc.; <path> is one or more of the paths R1, R2, etc.; (a) in general terms what the computational problem is (including any constraints);
and <position> is any one of the labelled positions (e.g, A, B, C) or suitable descriptions, e.g., at (b) its input (which must satisfy any constraints expressed in the problem statement); and
junction, before E, etc. Solutions 1 and 2 are possible algorithmic solutions for the task “Move from (c) its output (which must reflect the type of results expected).
point F to point A”. (ii) Give possible algorithms that describe the problem in (i) in terms of: (a) multiplication of x; (b)
addition of x.
Algorithm Execution and their Results (iii) Which of the solutions in (ii) is more efficient? Explain.
Comparing solutions 1 and 2 we see that they both solve the problem. However, solution 2 takes more (iv) Is each algorithm correct? And does each terminate? Why or why not? [Hint: (1) Try executing your
time to solve the problem (based, e.g., on the number of steps or distances covered), and hence solution algorithms for sample inputs and check the results obtained. (2) Consider executing your algorithms
1 is more efficient. The resource needs of algorithms are typically expressed in terms of the time they using sample inputs with properties that are special to integers (e.g., positive, 0) and to multiplication
take to solve a problem, and the maximum amount of space needed to hold [possibly intermediate] (in a) and addition (in b), such as 0 and 1. (3) Then, try to generalise that the result would be correct
solutions to the problem. We desire algorithms to be efficient, since resources available to carry them out and terminate for all valid inputs.]
are generally limited. Thus, in order for algorithms to solve our problems, we expect them to:
1) Be correct: It does what it ought. The homework is only illustrative, and exercises the concepts introduced in this section. However, note
2) Be efficient: Use as little resource as possible or as is reasonable. That is, utilise resources optimally. that in the study of algorithms, a “computational problem” typically describes a general class of
3) Terminate: Stop executing instructions and give the results, or indicate failure to give the results. problems (e.g., sorting problem or search problem—see later) and algorithms are developed to solve
[variants] of this class (e.g., comparison-based sorting) or specific instances of them (e.g., selection sort).
Most of the characteristics of algorithms examined earlier can be categorised as contributing to one or
more of the above. However, there are limitations. Just as moving from point F to G is not possible given Note:
the barrier, some problems cannot be solved using algorithms (computers). The study of what problems 1) This data transformation view of algorithms is useful in the analysis of algorithms, as the algorithm
can be solved using algorithms comes under the “Theory of computation”; the study of how efficient an can be seen as the [mathematical] composition of computational problems/steps where the output of
algorithm can be (and what are its resource needs) comes under “Computational Complexity Theory”. some serve as input to others, and the details of problems/steps need not be considered further once
Termination and correctness of algorithms are often studied under algorithm analysis (or design) though they have been proven correct.
the concept of algorithm analysis almost always includes computational complexity. 2) Just as the notion of a computation problem is “general”, the notion of input is also considered
general. An algorithm cannot work (produce desired results) only for some of its valid (acceptable)
Informally, “computation” is the mathematical ability to mechanically carry out instructions in discrete input. The notion of input considers all possible valid inputs relevant to the problem:
steps. Thus, an algorithm prescribes a computation (i.e., something that can be computed), and its • the specific values (e.g., 1, “TWO”) or collection of values (e.g., integer).
execution is also a computation. • the order in which they are input: e.g., 1 then “TWO”, but not “TWO” then 1.
• the accepted combinations or sequences: e.g., accept any of [8, 6, 3], [3, 6, 8], [6, 3, 8] and
Algorithm Definition 2 (as data transformation): “An algorithm is a well-defined computational [6, 8, 3] as an unsorted list of integers; but only [3, 6, 8] for a list sorted in ascending order.
procedure that takes a set of values as input, and produces a value or set of values as output.” (Cormen et 3) Strictly, an instance of a computational problem is the input—the [valid] values that meet all the
al. 2009) constraints expressed in the problem statement—needed to compute a solution to the problem
(Cormen et al. 2004). Thus, each of the lists ([6, 3, 8], etc.) given in the preceding note is an instance
A number of [new] concepts introduced need explaining: of the sort problem.
• Input: The data values given to the algorithm, which are then manipulated to carry out a task. 4) The related notion of input size, the size of the input, gives a measure of how large a problem
• Output: The value or result from carrying out (executing) the algorithm. instance is—or more generally, the set of problem instances solved by a given algorithm—and hence
• Computational procedure: It is just a sequence of steps—steps (instructions) as explained in the amount of resources (time and space) needed. Thus, a given sorting algorithm should be expected
Algorithm Definition 1—but with emphasis that each step is computational (as defined above): to take more time when sorting a list with more items.
that is, mechanical, discrete and ordered. The computational procedure transforms input into 5) Output is an effect of sorts, such as change of values in computer memory, or activation of other
output. Notice the analogy to mathematical functions [that take arguments and return results]: processes. In practice, for analysis purposes, we see output as a data item.
How the computation does it is not specified, only that it is computational and well-defined.

wss introduction to algorithms—notes of 08 October 2018 page 5 of 107 wss introduction to algorithms—notes of 08 October 2018 page 6 of 107
Algorithms and their Properties moves in order to identify the winning one. Cosmologists estimate the age of the universe to be 1010
Hopefully, the prior definitions of algorithms and the related concepts introduced help give us a more years. Obviously, there is no point seeking a solution [using “brute force” algorithmic techniques].
precise, yet intuitive understanding of what an algorithm is. Here we define the notion of algorithm with Computational Complexity is the branch of computer science that seeks to establish and classify
a focus on its more abstract properties—determinism, correctness and termination—which help structure problems—strictly, their algorithms—based on their feasibility.
[at least the early parts of] this course. A more formal definition of Algorithm follows. 6) Yet also, there are problems that may have algorithmic solutions, but for which none is known. These
problems tend to require some notion of intelligence or the creation and manipulation of knowledge.
Algorithm Definition 3 (abstract intrinsic properties): An algorithm is a well-ordered set of Machine Learning and Artificial Intelligence are branches of Computer Science that deal with such
unambiguous, executable steps that defines a terminating process. problems. They may use techniques that are not algorithmic (e.g., heuristics) or not mechanical, etc.

A number of [new] concepts introduced need explaining, and some old ones better [re-]defined: Homework:
• Well-ordered: This means that there is an established structure that guides or guarantees the Q1) Is the following an algorithm? Justify your answer in terms of the concepts given in Algorithm
relative order in which instructions are executed. Typically, instructions are executed in sequence, Definition 3 above.
but they need not be. For example, some algorithms known as “parallel algorithms” are designed 1) Do something.
so that any one of a set of alternative sequences of steps execute to give the same result. 2) Do something.
• Executable steps: Each step is do-able; that is, it can be carried out by the agent. In other words, 3) Do something.
instructions can only be given in terms of basic steps (also see basic actions/primitives later) that 4) Repeat.
the agent can always carry out. 5) Halt.
• Unambiguous: The information (data or states) available at each execution step of an algorithm
are sufficient to uniquely and completely determine the actions required at that step. That is, each Q2) If your answer in Q1 states that the instructions do not constitute an algorithm, in what ways can
step in an algorithm does not require creative skills; it requires only the ability to follow you make it an algorithm?
directions.
• Process: A process is the activity of executing a set of instructions. Thus, the algorithm moves Q3) Read on “Control Constructs and Representation” of algorithms in Chapter 5 of (Brookshear 2000,
from one state (or set of states) to a guaranteed next state (or set of states) as determined by the or Chapter 6 of Brookshear 2004) and “Data and control constructs” (Shu 1996).
instruction executed. (See explanation under “unambiguous”.)
• Defines: An algorithm states what a “terminating process” is. It gives a specific description of From prior examples, we observed that different algorithms can be used to solve the same problem. In a
what the process achieves, and the process uniquely determines what the algorithm achieves. way, the different algorithms represent different approaches to solving the same problem: here,
• Termination: See prior explanations. difference is based on the choice of instructions and how they are ordered. More fundamentally though,
an algorithm is an abstract concept which is represented in terms of specific instructions for our task. It
Note: is an abstract concept, just as a number is an abstract concept. For example, the number two is an
1) We’ve explained words such as “process”, “termination”, and “correctness”, but their mathematical
abstract concept that is represented by the three letters “two” in English, the four letters “deux” in
interpretations are beyond the scope of this course. Further explanations will be given as needed. French, the Roman numeral “ii” and the Arabic symbol “2”.
2) Many ambiguities are often attributed to an algorithm, when they are actually from the algorithm’s
representation. Thus, the instruction “Convert the temperature in degrees Celsius to degrees We discuss representation of algorithms later. Here, Algorithm Definition 3 helps us focus on the more
Fahrenheit” is okay to a meteorologist, but a lay person may argue it is ambiguous. Actually, the abstract properties of algorithms—determinism, correctness and termination—which help structure [at
underlying algorithm (conversion to degree Fahrenheit) is not ambiguous. Its representation is; it is least the early parts of] this course. These [abstract] properties are properties of algorithms as a whole—
not sufficiently detailed to the lay person. that is, of computational sequences in general—as opposed to focus on its specific actions/steps
3) When discussing under operating systems (See CSC205, Introduction to Computer Science) we get (Algorithm Definition 1) or its outputs from inputs (Algorithm Definition 2).
other related definitions of a process, which must not be confused with our preferred definition. Now,
algorithm execution may be seen as taking the algorithm from one set of states (called the process An algorithmic process exhibits the following properties:
image) to the next: a process is thus defined as the execution of a process image. This view is not • Determinism: Determinism allows any agent to follow the same process on the same set of
interested in the individual instructions that are executed (or the particular images they generate), but [input] data and get the same result (output) each time. Such a process is said to be deterministic;
that the process is successfully executed under constraints imposed by the operating system. it is non-deterministic otherwise.
4) Computer Scientists say instructions that are do-able as explained above are effective or effectively • Termination: At some point (in finite time) the algorithm stops executing steps and give its
computable. Thus, the instruction sequence (Brookshear 2000) is not an algorithm: results. Termination is important to Theoretical Computer Science where one seeks to know
Step 1: Make a list of all positive integers. which problems have algorithmic solutions and which can never have. It is also important in
Step 2: Arrange this list in descending order (from largest to smallest). practical computing where, say, it rules out endless processes that never give meaningful results.
Step 3: Extract the first integer from the resulting list. • Correctness: We actually get correct results, the one that the algorithm is expected to give; the
since Step 1 is not effective, as one cannot list all the positive integers. Similarly, Step 2 is not algorithm solves the [computational] problem. An incorrect algorithm might not halt or, if it
effective as integers cannot be ordered. halts, gives an incorrect answer.
5) A related notion [to effective] is “feasible”. Some problems have algorithmic solutions, but they are Algorithms which do not demonstrate one or more of the above properties are also useful. We shall
too inefficient to be usable (useful). They are not feasible. For example, in the game of chess, players return to them later in the course.
alternately make moves; i.e., move their tokens—pawns, knight, rook, etc.—on the chess board. It is
estimated that it will take 1070 years (at 1018 moves per second!) to consider all possible sequences of

wss introduction to algorithms—notes of 08 October 2018 page 7 of 107 wss introduction to algorithms—notes of 08 October 2018 page 8 of 107
CONTROL AND DATA CONSTRUCTS Notation Meaning

Control Constructs while C do S1 od If condition C is true, we carry out action S1 (in fact, execute all instructions
In order to accomplish an algorithmic task, there are some basic actions that an agents can carry out, as while C do within do and od) then assert condition C again. Repeatedly test C then
well as basic constructs to control the order/sequence in which the instructions are carried out. The basic S1 execute S1 so long as C is true. If at any time C asserts to false, even
actions and basic control constructs are generally called [basic] primitives, though the term is sometimes od initially, do not execute S1; instead, go execute the next statement [in
used when referring only to basic actions. They are the set of building blocks from which algorithm sequence] after the iterative construct. (Call this notation the while-
representations can be constructed. Ambiguity is thus eliminated in most algorithms and the primitives construct.)
establish a uniform level of detail at which algorithms are described. From above, note that we can execute S1, zero, one or more times, based on whether C tests true
zero, one or more times accordingly.
The basic actions are specific to a [computing] agent, and must be defined before any algorithm can be
described for an agent. For our maze example the basic actions are the “Move”, “Turn” and “Stop” Note:
actions. For some other agent, it might be something else, or be the same thing represented differently. 1) The above notation (representation) is a convention—an agreement on how to represent constructs.
Of course, as with algorithms, the essence of an action can have different representations, but we require Others may agree differently, or may use other notations.
that they be in terms of the basic ones (which need not be uniquely represented). 2) We will add to our repertoire of constructs alternative constructs as needed.
3) Recall that an algorithm is abstract, and distinct from its representation, just as a number is abstract
In contrast, the basic control constructs are not unique to agents, though their representations could be. and distinct from its representations in: Arabic (e.g. 2), English (two), French (deux), and Roman
The three basic types of control constructs are: numerals (II).
• Sequence: Execute instructions one after the other. 4) Always make sure the constructs in our notation have the intended meanings and not those, for
• Choice or Selection: Execute at most one of a set of one more alternative instructions. When an instance, in the English language or in other notations. e.g., WHILE in “WHILE it is raining, shut the
alternative is selected, all the others are not executed. If there is only one instruction, it is window.” has the meaning (semantics) of IF in our notation: “IF it is raining THEN shut the door”.
optionally executed.
• Iteration: Do the same set of instructions repeatedly, again and again. i.e., repeat the instructions Home work:
possibly 0,1 or more times. Q1) Given the example maze, its basic actions and the notation adopted in class:
i) Give an algorithm to go round a block N times for some fixed number N between 5 and 10.
With these three basic constructs and its basic actions we can write any algorithm that a given agent can ii) Give your algorithm in (i), if there were no iterative constructs.
carry out. However, we need a way or notation to represent algorithms. e.g., in this course, we assume ii) Formulate three distinct algorithms based on the maze, different from any used in class or from any of
the following conventions. the exercises.
• Sequence. For sequential execution, assume instructions are executed left-to-right, top-to-bottom
Data and Data Constructs
and are optionally (but unambiguously) separated by semi colons (;) and/or by new lines. (i.e.,
Algorithms manipulate objects—i.e., data or values which are taken (input), or manipulated to give
instructions may start on a new line.) e.g., the following instruction sequences are the same:
results (output). This data must be constructed (organised) in a fashion suitable for use. Strictly, data
Sequence using Newline Sequence using semi-colons constructs permit us to create different kinds of data, and/or encode (represent) them for algorithm
Move West along R3 Move West along R3; Turn right at B; manipulation. Data constructs correspond to mathematical constructions such as Cartesian product,
Turn right at B disjoint union and [mathematical] functions. In practical terms, such constructs define data structures
• Choice. For choice, assume the notation: (relationships among data values).
Notation Meaning Data (or values) are stored somewhere (at a location or address) which, for convenience, is usually
if C then S1 else S2 if C then If condition C is true, we carry out action S1. If C is named. An assignment operation is the basic operation normally used to store the data [in its location or
if C then S1 else S2 end S1 false we do S2 instead, or nothing if S2 is absent. In all address]. A binding (or binding operation) is typically used to associate data and/or its location with a
else cases, the next action carried out is the one following name. The notions of binding and assignments will be considered again later.
if C then S1 else S2 endif S2 the choice. (Call this notation the if-construct.)
Data Types
if C then S1
Data is also associated with the notion of types. A [data] type is essentially a set of values (objects)
Keywords—words that should appear exactly as they are in the algorithm—are underlined, together with a set of permissible operations on them; that is, they determine what actions can be applied
though we sometimes use uppercase letters, e.g., THEN for then. A condition is a value that is to the data values. Thus, for example, numbers (integers) allow for standard arithmetic operations on
either true or false but not both [true and false] nor neither. Such values are called boolean them; eggs allow for whisking (by a baker); and in the maze problem, our agent can turn left or right.
values. For clarity we can indent actions such as S1 and S2, or add an end or endif keyword after
S2. Indentation highlights nesting of primitives, the inclusion of one primitive in another. !!! Digression !!!: How a Computer Works
(This subsection is a digression. It includes CSC 205 material in order to ease course delivery, just in
Note that, “if C then S1” is a degenerate form of the IF-THEN-ELSE construct where the S2 case class sessions on CSC 205/207 are not synchronised, or presentation is more convenient for CSC
action does not exist and so nothing is executed when C is false; the action after the choice 207. Students are advised to follow their notes for CSC 205 in relation to CSC 205 tests, classes, etc.)
construct is executed immediately.
• Iteration: For iteration, we assume the notation:

wss introduction to algorithms—notes of 08 October 2018 page 9 of 107 wss introduction to algorithms—notes of 08 October 2018 page 10 of 107
Block Structure of a Computer are non-destructive. Within a program, data from the CPU or from some other part of memory, say, may
Here, we see the block structure of a computer, how components are put together, and how data and be assigned to or stored in a given location in memory. Essentially, an assignment copies a value from
instructions are represented. The computer is typically used as illustrated adjacent. … one location to another: the original and final locations hold the same bit pattern.

The Von Neumann Machine Concept Since writes (or stores) are destructive, we must always make sure the value we overwrite is no longer
?? As indicated earlier, tasks given the computer have to be broken down into simpler, individual useful. If it is, we must first copy it to a safe location, before overwriting it. Similarly, since reads and
instructions (and data items) that the computer can assignments essentially copy values, we must erase (overwrite) values we don't want others to see.
understand and execute. Typically the instructions Block Structure of a Computer Typically, we write a string of 0's or 1's, or any other bit pattern that would not make sense if later used.
for a program are stored in a known location in If you want to swap or exchange the values found in two memory locations, call them x and y, you must
memory, and the location of its first instruction CPU first store one of them, say x, in a distinct location, call it t. Algorithm swap is simple:
stored in a special register called a program counter 1) Store (copy) the value in x to location t. [copy of] value in x is stored in a safe place.
(PC). The computer works by letting the CPU 2) Store (copy) the value in y to location x. value in x is overwritten by that in y.
(Central Processing Unit) execute the instruction 3) Store (copy) the value in t to location y. Value in y is now in x as copied from t.
Main Output
pointed to (indicated) by the PC and then Input
Memory Device
incrementing the PC (usually by a fixed value, e.g., Device Algorithm swap is very common in Computer Science. Optionally, t is erased, but the action is not
2). (The new program counter value is where the considered part of the Algorithm swap.
new instruction to execute should be.) The result of Secondary
the computation is stored in memory (if a value) or Memory Program's Data Access
sent to an [output] device. The next instruction, as Data (and instructions, though we speak here only of data) for use by a program are is read into or input
indicated by the PC, is executed, and the cycle System Unit to the program from the real world (the user) through an input device such as a computer keyboard. It is
repeats forever, or until the machine is shut down. A then stored in or written to a specific memory location under constraints as just explained in the section
special instruction, no-op (for no operation), that “In-memory Data Access”. Data presented to the user, say, by a program is said to be output, written or
does nothing, is executed if the CPU has nothing to do. printed (or displayed, if sent to a display screen). If the data is from main memory (as opposed to the
CPU or some other device), only a copy is taken, as explained earlier. Notice that reading data from the
Instructions that change the sequential execution order of instructions, as just described, do so by outside world is destructive, as it is stored in memory; writing to the outside world is non-destructive.
changing the value of the PC to that for the desired instruction. Also, should the PC automatically point
to a wrong address, the PC value is similarly adjusted. Algorithms, Programs and Programming Languages
A program is an algorithm executable on a computer. That is, the agent that executes the algorithm is a
Tasks as Done on a Computer Computer Carrying out a Task computer (See CSC 205 for what is a computer) or a processor. A processor executes processes;
The computer cyclically waits for a essentially, it carries out instructions/executes algorithms one step at a time. (Notice the circular
user to submit a task and any needed (2) Provide data for task. definition, based on the same root word! The reason is that we do not have—and do not expect to have in
data for task, then carry out the task, this course—the technical machinery needed to define a processor.)
produce a result for the task, then (1) Submit Computer
awaits the next task. e.g., to process task to do. system Syntax and Semantics
(3) Process
the class average, the teacher may do data for Programs are written in a programming language. A programming language is a collection of primitives
the following: The User (4) Output task. (basic actions and control and data constructs), the rules governing how the primitives are combined and
1) Invoke the program for student (0) results of the meanings attributed to them (i.e., to the primitives and their combinations) in order to express more
averages. task. complex ideas [as algorithms]. Syntax refers to the symbols representing primitives and how they can be
2) Give the number of students (n) combined; the rules constitute the grammar of the language. It describes how components are correctly
and their marks to the computer. put together. Semantics refers to the concept represented or the meaning of the primitives. More
3) The computer then computes the class average (i.e., sum of marks divided by number of students). precisely, it refers to the meaning associated with entities (actions, constructs, objects, etc.). For a
4) The computer displays the result. The computer then awaits the next task to do at (1) above. program, the basic actions (primitives) are the instructions recognised by the computer (or processor).
[Programming language] pragmas (short for pragmatics) refers to options and techniques that permit
The operating system helps to coordinate such processing activities. However, note any given task may one to efficiently use or implement a language. e.g., how best to implement variable bindings in a
be split into sub task which each go through the cycle, possibly omitting some steps. compiler.

In-memory Data Access Instructions and data values are expressed in a computer as strings of 0's and 1's. Any language over 0's
As indicated earlier, data (and instructions, though we speak here only of data) for use by the CPU are and 1's is too tedious for humans and so high-level languages, which are closer to human languages, are
stored in the main memory of the computer. A data item (or instruction) within a program is stored or often used. Even then, developing an algorithm can still be tedious, which calls for more suitable
written to a specific place in memory (the memory location) as a sequence of bits. Any data (bit pattern) representations, such as pseudocodes (see later), [for partial or as-yet-ill-formed ideas].
that was there before is overwritten; that is, it is replaced, and is forever lost. We say that writes—i.e.,
writing data to memory—are destructive. A data item is read from a memory location. Only a copy of Illustrating Syntax and Semantics
the data is taken from the location; the original data remains in memory, unchanged; we say that reads

wss introduction to algorithms—notes of 08 October 2018 page 11 of 107 wss introduction to algorithms—notes of 08 October 2018 page 12 of 107
Suppose we restrict the syntax of English sentences (P, for phrase) to a subject (S) followed by a verb nomenclature may vary with languages and paradigms. e.g., pure functional languages have no notion of
(V) followed by an object (O) and then by a period (.). Furthermore, let A stand for Article, N for noun. PL variables, and bindings are used virtually as in mathematics; and logic programming languages
Then, we may have the following as rules for our language, where the colon (:) means “[is] defined as”, instantiate variables (i.e., give names to values).
the arrow (→) means “[is] followed by”, and the comma (,) is used to separate alternatives as shown
below. Data types and type construction
Grammar Rules Asserting Sentences of the Grammar <types and type expressions; type constructors; type systems; polymorphism; basic and user-defined
Then, we can assert whether the P: S → V → O → . S V O Period types>
sentences adjacent are of S: A → N
English by trying to group O: A → N The monkey ate a peanut . We further discuss types later (?) but they are in programming languages (PLs) for reasons that include:
words as S, V and O in V: was, ate, saw, killed a elephant was a monkey . • Correct use of values: Everything in a machine is in terms of 0's and 1's and so typing helps
sequence. (S and O are defined N: mouse, elephant, The mouse killed a elephant . detect erroneous use as well as enforce correct use of values.
in terms of A and N, and each peanut, monkey The Monkey ate . • Program description: Types are useful in proofs of program properties, provide an expressive use
word stands for itself). A: a, The of program/programming concepts, and enable efficient implementation of programs.
• Documentation: Types serve to document correct use of values.
Programming Language Primitives
The [possibly] basic actions in programming languages are often called statements, and the data values A language with operations and rules for constructing types typically allows its users to define their own
they manipulate are often represented as expressions (c.f., arithmetic expressions) which are often types. Such types are called user-defined types. They could be as simple as renaming an existing type,
simplified to give specific data values. (Some languages, e.g., the C programming language, consider through restricting or extending the properties of existing types, to creating brand new types from
statements and expressions alike as expressions.) Programming languages typically have: existing types.
• Input statements: to communicate data into the program from the world (its surroundings).
• Output statements: to deliver its results to the world [outside it]. Data Structures and Constructors--- more technical material: to edit into CSC207 level or discard !!!!
• Control constructs: to properly sequence instructions (statements). A data structure, then, uses data constructors and other data structures and atomic values to construct
• Data constructs: to create and use data items within the program. other new data structures. A structured data type is thus a type whose objects are structured (i.e., have
• Memory access statements: to internally store (e.g., assignment statements) or retrieve (e.g., subcomponents); it would typically have rules for constructing and for using the type.
storage access expressions) data items they manipulates.
• Expression operations: to construct expressions used in programs. Data constructs have corresponding basic control constructs of sequence, choice and iteration and,
• Basic values: Values that could be assumed to already exist in a language. usually (but not necessarily), a control construct is used to access its corresponding data construct. e.g., a
• Programming language directives: Assorted directives to create/allocate, manage or use artefacts record is (mathematically) a tuple which is an ordered sequence of entities, usually of mixed numbers of
(e.g., procedures, memory locations) used in an algorithm. Note that these directives are not entries whose components can be accessed sequentially. (It can also be accessed randomly though). e.g.,
intended to solve the computational problem, per se, but are [instructions] to help secure standard Student no, mark and year are access in tuple.
things (memory, procedures, etc.) useful to solve the problem. They effectively help set up the
environment in which the algorithm may be developed or executed. Disjoint sums and tuples: Disjoint sum “tags” the source (origin) of elements in a set, so that elements in
the set are distinguished by their source as well. e.g., mark and year are integers, which can be tagged as
Variables M (for mark) and y for year. Thus, the set {M:70, M:3, y:2, y:3, y:4} actually has marks 70 and 3, giving
Recall that instructions in algorithms carry out actions on objects (called data or values). Data (or values) set marks {70,3}, and years {2,3,4}.
are stored somewhere called a location, which is usually an area of memory. A location has an address
(which is a bit pattern usually) that situates it in memory and used to access it. For convenience, a Without tags, the set will be {2,3,4,70}, where 3 is no longer duplicated. Hence we can have tuples
location may be given a name. A name is an identifier by which an object is recognised or labelled. An (student no, {M:mark, y:year}) where the second entry varies with the data source. e.g., the tuple (2561,
identifier is just a character string, usually of letters of the alphabet [of the language] and numbers. Note: mark: 3) says student 2561 has 3 marks and (2561, year: 1) says he/she is a year 1 student. We must
In the terminology of linguistics, a location is a referent, the thing referred to; and an address is a check the tag to know whether to treat the data in the second entry as mark or year. For tuples, tags are
reference, the thing used to refer to another. c.f., in “Joe gave his book to Jane.”, Joe is the referent and not necessary. They can be implicit since what is expected at the i-th tuple position is always known.
“his” is the reference, for it refers to Joe.
However, tuple values could be labelled, but such “tags” are usually for convenience (e.g., ease of
A variable, then, is a possibly named location. Informally, location, address and variable are used extracting/accessing tuple components, clarity in presentation, or more conveniently changing the tuple
interchangeably. Also a variable may be used to refer to (i.e., name) the value it holds. This order. For the latter, the order can always be restored by considering the names associated with values.
programming language notion of a variable is different from that in mathematics. A variable in Thus, labels (or names) are conceptually the same as tags, but are used for different purposes. Thus, our
mathematics is a value (not a location) that is not known specifically, and that can be assumed to change examples could then become: (student no: 2561, info: (year: 1)) and (student no: 2561, info: (mark: 3)),
[within constraints]. (In contrast, the value of a mathematics constant may not be known, but is must not where info: is the label for the second tuple entry. (The extra parentheses, as in (year: 1) is for grouping.)
be allowed to change!)
Agent CALCUL
More technically, the naming of values is known as binding, and setting a name to a specific value is We do not use any specific programming language for this course. We thus create one, as we proceed, for a
computational agent called “Agent CALCUL”.
often called instantiation. Note that assignment, the basic operation used to hold (store) data in a location
(or, equivalently at its address), presumes locations whereas instantiations do not. Furthermore,

wss introduction to algorithms—notes of 08 October 2018 page 13 of 107 wss introduction to algorithms—notes of 08 October 2018 page 14 of 107
 Agent CALCUL uses the arithmetic operations given below, with their usual meanings. x and y are
Basic Actions Predicates variables or numbers. The assignment operation stores values into variables. Read inputs values and
add x, y (or x + y) addition Is_integer write outputs values (results). sqrt(x) returns the positive square root of x. The predicates are of obvious
sub x, y (or x – y) subtraction Is_real meaning, and can be used together with relational (=, ≠, <, ≤, >, ≥) and boolean operations (e.g., AND,
mul x, y (or x * y) multiplication Is_positive NOT, and OR) or the boolean constants TRUE and FALSE. (A predicate is a condition that is either true or
div x, y (or x / y) division Is_zero false, but not both or neither; it normally takes one or more arguments.)
mod x, y (or x % y) modulo arithmetic Is_character
neg y (or – – y) negation Is_digit
Is_odd REPRESENTATIONS FOR ALGORITHM DEVELOPMENT
← assignment operation. Is_even As alluded to earlier, the same algorithm can be represented in assorted notations. Especially when
read (x) input: get a value into variable x. Procedural Abstraction discovering or developing algorithms, one needs suitable representations [for partial or as-yet-ill-formed
write(x) output: give/display the value in x. x is an ideas]. The most general and common representations include: pseudocode, flowcharts and dataflow
expression, variable, number, etc. PROC procname (arguments) ≡ diagrams.
sqrt(x) give the positive square root of the number x. statement sequence; END
return(x) return the value x from a procedural abstraction. FUN funcname (arguments) ≡ Pseudocode
statement sequence; END Here, representation is a cross between statements (basic actions, etc.) expressed in natural language
Some Control Constructs: (e.g., English) and a programming language(s). Pseudocode makes it easier to think in a natural
Choice: IF condition WHILE cond DO Sequence: language, but increasingly structure primitives in terms of a target programming language (PL). Thus
IF condition THEN Statement1 1) Statements on separate lines pseudocode is a mix of language constructs and natural language used to express an algorithm, usually
THEN Statement1 OD or optionally separated by (not always) in an algorithm development process. The notation adopted in this course is actually a
Statement1 ELSE --- semi-colon. pseudocode based mostly on the Pascal PL.
END Statement2 FOR i = 1 TO n DO 2) Use of the GOTO
END Statement1 instruction. More precisely, we explain the link between algorithm and algorithm development in of pseudocode as a
OD language for (virtual) agent(s). Because program (i.e. sequence of instructions) details are many and
Figure 2 – (a) Basic actions, predicates and some control constructs for agent CALCUL varied, likewise the problem details, one can express them more easily in natural language. However,
data /control constructs for our target agent should express the final desired solution. Thus, in solving a
REC ARRAY index type OF cell type; computational problem, one can express it in terms of control constructs of our target agent, while
field name 1: field type 1; ARRAY index range OF cell type; expressing complicated details in natural language. The partial solution, thus far, is for a virtual agent
field name 2: field type 2; --- that “understand” the natural language phrases used. The natural language components can be refined
... Data constructors: successively, until they are finally expressed in terms of the primitives of the target agent.
field name n: field type n; ×: product (product domain)
ENDR + : disjoint sum (sum domain)
The combination of basic actions, control and data constructs, and declaration directives for a given
→ : function (function domain)
VTAG
agent (virtual or target) constitutes a language. Therein, the algorithms for the agent are expressed. When
P**S: powerset (Power domain)
tagVar : tagVar type; R ≡ … R … : Recursive domain
the agent is a computer, or processor, the language is called a programming language, and the algorithm
vtag_1: ---
field name 1,1 : field type 1,1; Symbols for: (a) Flowchart diagrams; (b) Dataflow Diagrams
... Example arrays, and array access;
field name 1,p: field type 1,p; Example records
Example variant records
vtag_2: opera on A
operations
field name 2,1: field type 2,1; pointers and pointer access
... p->x ≡ field x of location pointed to by
field name 2,q: field type 2,q; p.
Data
arbitrary abstract data type (use of a Start
vtag_n: functional notation). choice
field name n,1: field type n,r; e.g., root(r) for root of tree r; a function Arrow
... that returns the root of tree r.
field name n,r: field type n,r; left(r) ≡ r.left (* field left of record r *)
ENDV Stop Halt
ary(i) ≡ ary[i]; (* i-th entry of array ary *) Output
input
ptr(p) == pointer to location p. ???

Figure 2 – (b) Data structures and data access constructs for agent CALCUL output

wss introduction to algorithms—notes of 08 October 2018 page 15 of 107 wss introduction to algorithms—notes of 08 October 2018 page 16 of 107
is called a program. More loosely, we may also talk of a program or PL for an arbitrary agent. Thus the
pseudocode is a mix of language constructs and natural language used to express an algorithm, usually “Control flow” is enforced by the way processes treat their data. Data availability, not instruction order,
(not always) in an algorithm development process. controls the evaluation order of actions and so the match between algorithmic constructs and execution
order is not immediate (obvious). Thus, in diagram (A), sequential execution is enforced by data items
Flowcharts m1 and m2 being available in order. However if data item m1 can be null (empty) or ignored by process
Flowcharts are a collection of graphic symbol showing different kinds of algorithmic actions P1, process P2 can still proceed on m2, or blocked if P1 never generates m2. Thus, the control structure
(primitives), with arrows showing the flow (sequencing) between statements. e.g., see box below. can be side-stepped by the process using them unless, perhaps, one explicitly constructs and uses
Input/output (I/O) enables data to be communicated to (and from) the program from (and to) the world processes dedicated for process control.
outside of it.
In (B) choice may be enforced if the possible values for m1 can be partitioned into two disjoint sets,
The following illustrates how basic control constructs may be implemented via flowcharts: {m'} and {m''}, such that process P1 executes on values in {m'} and process P2 on values in {m''}. If the
Except for a small class of problems flowcharts do not adequately guide the development of algorithms, sets are not disjoint, P1 and P2 could execute concurrently on the values common to both. Similarly,
can quickly get clustered and often does not reveal common desired ‘structure’ over algorithms (such as configuration (C) offers iteration, which can be side-stepped if, for example, process P2 never generates
iteration, nesting of constructs) and humans easily misinterpret them. As such, flowcharting is generally m3 [as explained for sequence].
discredited as a representation for algorithm development.
Example (Homework):
Note: Q1) Give the same algorithm in (a) pseudocode, (b) flowcharts, and (c) dataflow diagrams respectively
1) There are assorted symbols not discussed here. Some, e.g., link flowcharts together. to do the following: Read the marks of a given number of students, and then computes and prints the
2) The example of sequence, choice and iteration correspond to those given in pseudocode in class class average.
(Sequence, IF and WHILE).
3) More than one action may be given within a rectangle. Possible solution to homework: STUDENT ATTEMPTS PROBLEM FIRST, BEFORE LOOKING
AT SOLUTION; SOLUTION IS ALSO BIASED TO SPECIFIC PEDAGOGIC OUTCOMES!
Dataflow diagrams
Dataflow diagrams are like flowcharts except that the arrows show flow-of-data (not flow-of-control)
between two processes (activities), and the graphic symbols may be different or interpreted differently, Flowchart: Dataflow Diagram:
such as those shown above. Dataflow diagrams are suitable for data start
m Sum marks m Compute
flow machines. They are also be used in program development. sum
for n students average
To convert a temperature in degree Celsius to Fahrenheit. Read no of students n
Input temp in oC Convert from Input temp in oC
o
C to oF Avg

Read students mark Avg

Increment count of Print class
Add to sum so far
student average

Read n Pseudocode:
marks?

Read no of students, n;
DO
Read student marks;

average
Class
Compute average Add mark to running sum;
Increment count of students;
WHILE (student count < n);
Compute class average.
stop Print class average.

PL CONSTRUCTION: SYNTAX, SEMANTICS AND PRAGMATICS

So far, we have thought of programming languages in terms of how it can be used to express algorithms
that control the machine (a computing agent). We can think of programming languages more generally in
terms of how it is constructed (designed and implemented) as well as how it is used. [In practical terms,

wss introduction to algorithms—notes of 08 October 2018 page 17 of 107 wss introduction to algorithms—notes of 08 October 2018 page 18 of 107
this section is more appropriate for CSC 208 (Introduction to programming …) or after discussing other As with types of programming languages, the different paradigms of programming (not under this
concepts (e.g.,). It is put here for completeness, but need not be examined in CSC 207.] course, but alluded to later) are similarly best used to solve certain classes of problems. Hence the
languages that support them define the ways to formulate solutions for them. For instance, logic
Programming Language Usage programming paradigm is suitable for solving problems that directly exploit (use) techniques of logic
Earlier, we defined “programming language as a notation”. When using a PL, we may see it essentially (mathematics) such as inferences, resolution and unification.
as a convention (an agreed way adopted) for communicating an algorithm [from humans to machines].
More explicitly, it is a “language intended for use by a person to express a process by which a computer Domain Specific Languages (DSLs)
can solve a problem” (Hope and Jipping), but also to allows other humans to independently follow the The basic actions, control and data constructs, as well as composite constructs (c.f., procedures) over
process. Programming then becomes the art (process) of managing the complexities involved, so as to them, define a language in which to express the actions of an agent. (See section “Algorithms, programs
produce working programs (working code). and programming languages” above.) We can use this language to express the actions of any agent that
understands them, and so can develop algorithms for that agent. Languages for such specific contexts are
Types of Languages (for Application Areas) sometimes called Domain Specific Languages (DSL’s), though the term is normally used for problems at
Different languages are suitable for or designed to suitably solve different kinds of problems. For higher levels of abstraction than we have used in this course.
example, FORTRAN is designed to efficiently handle scientific computation; Pascal is for teaching; C
and C++ are for systems programming; Prolog and LISP for Artificial Intelligence problems; and PL Syntax
Haskell, ML and Ocaml are for certain mathematical problems. Syntax deals with how components of the language are correctly put together, as we saw earlier. Names
of syntactic forms are typically the non-terminal symbols of the grammar of the language described.
Hence the languages have features that support tasks suitable for them and provide/prefer techniques that Non-terminal symbols define (identify) the ordering (sequencing) of language components that are
help formulate solutions in them. For instance, logic programming tasks directly exploit (use) techniques considered correct in a language—that is, well-formed. The grammar of the language are the rules
of logic (mathematics) such as inferences, resolution and unification, and so are best implemented in describing (or defining) all the well-formed components of a language. Terminal symbols are the
logic programming languages. names/symbols in a grammar that appear as-is in a language.

Low level languages are used for tasks closely or critically linked to the machine hardware or where The syntax of a language are often presented graphically via syntax diagrams, and textually via the BNF
program efficiency is critical, such as in [hardware] drivers. (Drivers convert general computer (Backus Naur Form or Backus Normal Form) notation.
instructions and data to a form understood by specific [hardware] devices such as printers, and vice-
versa). Machine language is for particular machine model(s), while assembly languages, improve Syntax diagrams:
portability across [usually] related machine models. A syntax diagram is made up of arrows between components of a language, from a “start” arrow to a
“finish” arrow. A sequence of components is syntactically well-formed if there is a path from the start
Programming paradigms arrow to the finish arrow, recognising each component found. The diagram constructs correspond to the
notions of sequence, choice and iteration, and so can algorithmically describe any syntax.

Backus Naur Form:

Syntax Diagrams BNF/EBNF The classic notation for BNF has components in angled brackets (“<” and “>”) and uses concatenation
for sequence, vertical bar (“|”) for choice, asterisk for iteration, and parentheses (“(“ and “)”) for
S/N Notation Meaning Notation grouping. In the extended BNF (EBNF) notation, some symbols are added and others dropped. The
1 ──> a ──> b ──> Sequence: a followed by b. BNF: <a> <b> angled brackets are dropped, and the following added: “+” for iteration one or more times; “?” for option
EBNF: ab (zero or one); dot (“.”) to stand for any character; [xyz] and [x-y] to respectively stand for any one of the
┌── a ───>
characters x, y, or z, and any one in the range x to y inclusive.
2 Choice: a or b but not both. BNF: <a> | <b>
───┴── b ───>
EBNF: a | b (OR [ab] term syms) PL Semantics
3 ┌─<─ b ──┐ Iteration: zero, one or more BNF: <b>* Semantics deals with the meaning of each component of the language. Semantics is typically very
───┴────────┴───>
occurrences of b. EBNF: b* difficult to express formally (mathematically) and so it is often given in natural language, for people to
┌─<──────┐ One or more occurrences of BNF: <b> <b>* use, notwithstanding the inherent ambiguities introduced by natural language when writing programs.
───┴─> b ───┴───>
b. EBNF: b+
Since it is difficult to express meaning clearly and fully in natural language, developing or reasoning
4 ┌─> b ─┐ a followed by b or c, but BNF: <a> ( <b> | <c> )
──> a ─┴─> c ─┴──>
about PL's require more formal descriptions. Thus, we have broad categories to describe the meaning of
not both. EBNF: a ( b | c ) programs, programming language components, and the behaviour exhibited by a real or hypothetical
5 ┌─> b ─┐ a followed by b or c, then BNF: <a>(<b> | <c>) <d> machine. Three main classes of semantics are operational, axiomatic, and denotational semantics, with
──> a ─┴─> c ─┴─> d ─> other classes often seen as variants from one or more of these. Common ones are:
by d. EBNF: a (b | c) d
┌─────<────┐ • Operational semantics: assumes a real or hypothetical machine exists whose outcome (output)
6 a followed by one or more BNF: <a>(<b>|<c>) (<b >|<c>)*
│ ┌─> b ─┐ │ gives the desired meaning of all possible programs correctly written in that language. Any
──> a ─┴─┴─> c ─┴─┴─>
occurrences of b and/or c. EBNF: a (b | c) +
program that cannot be executed by the machine is not of the language.

wss introduction to algorithms—notes of 08 October 2018 page 19 of 107 wss introduction to algorithms—notes of 08 October 2018 page 20 of 107
 • Axiomatic semantics: describes meaning in terms of axioms, assertions that can be proven true • Symbol Table: Holds information on a program, so that it can be [subsequently] used by various
about constructs of a language, and inference rules—rules for reasoning about the language and components of the compiler.
its properties. Axioms are [mathematical] facts assumed to be true about a program or language • Syntax analyser: Checks out that language constructs are well-informed.
construct, and must hold true for any correct implementation of a programming language and/or • Semantic Analyser: Checks that program conforms to the semantics of (i.e., meaning associated
its programs. Assertions are often expressed as invariants—things [about a programming with constructs of) the programming language.
language, etc.] that do not change from program [run] to program [run]. • Code Generator: produces the target code, for execution.
• Denotational semantics: expresses semantics in terms of mathematical objects, such as numbers • Code Optimisation: Reorganise code and data so that they more efficiently use availableww
and functions, used to “denote” meaning. Such objects are typically enhanced with additional resources (typically expressed in terms of memory space and processor time).
mathematical attributes—e.g., to ensure all possible outcomes, even incorrect ones, can be
expressed—and are said to be of [specified] semantic domains. Efficient Use by Machine
• Algebraic semantics: describes semantics in terms of an abstract algebra—i.e., a set of values and Efficient use of language by the machine typically considers runtime structures to manage runtime
operations on them—whose properties are given by a set of axioms (c.f., axiomatic semantics) activities of a program. For instance, use of a runtime stack for procedure activations (i.e., PL procedure
and a set of equations over the operations. Strictly, this definition is that of an abstract algebraic calls), and the garbage collection of memory storage no longer needed. Portability of a language allows
specification—an approach to stating semantics algebraically. More generally, algebraic it to be used on different machines (with appropriate language systems), without modification. Strictly,
semantics expresses semantics in terms of manipulating algebraic entities in mathematics. this latter is a property of language implementation.

Note: [Note: As discussed later, a procedure is a named set of instructions that can be called at any time by
1) Mathematically, the correct meaning or semantics of a programming language is given by a [set of] using that name to execute that set of instructions. A stack is a kind of data structure that removes and
associations between syntactic components of the language and another set of objects called semantic uses its data values in the reverse order in which they are stored in it.]
objects (which may be grouped into semantic domains) The possible associations (matches) define
the meaning of each syntactic object. Such a set of possible associations between objects, not Ease of Use by Humans
necessarily between syntactic and semantic objects, is generally called an interpretation. Some principles of language design aid in the use of PLs in programming and programs. These include:
2) Computation model(s) give a general, abstract (non-physical) and mathematical description of how • Expressiveness: captures how easily (and concisely) one can express simple activities in a
computation is carried out. Some of these include Turing Machines, lambda calculus, partially- language, and render the programming process [by humans more] efficient. Thus, for instance,
recursive functions, etc. (We might refine our explanations later.) the programs of a language should be easy to read (readability): the language syntax should be
3) Some consider algebraic semantics as an axiomatic semantics abiding by algebraic laws; others easy to understand [by humans], and the language should allow for good human-readable
consider it low-level denotational semantics whose meaning is denoted by algebraic objects. commentary and naming conventions. Again, a language should ease problem solving for
4) Under operational semantics, a machine is anything that can carry out instructions or whatever in specific programming paradigms (i.e., specific approaches to the programming process—the
[discrete] steps. process used in producing programs).
• Regularity: expresses how well and systematically the features of a language are integrated.
Homework Regularity makes languages easier to learn and use. It may be expressed in terms of:
Give (a) the syntax diagram and (b) BNF notation for a bit, and hence those for a binary number. o Generality: closely related constructs are expressed as a single more general one. Thus, a
construct is used in the same way across all constructs of the language.
Programming Language Pragmatics o Orthogonality: Orthogonally expresses how constructs of the language, typically a small
Programming language pragmatics (Pragmas) addresses issues that affect how a language is constructed number of PL concepts, can be independently and consistently used, without any unexpected
so that it can be put into practical use. It considers options and techniques to efficiently use or implement restrictions or behaviour.
a language: how people can [more] conveniently use the language; how the language and programs o Uniformity: Similar things should look similar, be done in a similar fashion, and have similar
written in it can be implemented; and how their execution can better use [machine] resources.
meanings.
• Abstraction: expresses the ability to hide details of control and data constructs. (See later.)
Language Implementation
Abstraction allowed for simplicity, and makes it easy to prove the correctness of programs. It
It is mainly concerned with efficiency in language translation, efficiency of the resultant executable code
may also enable one to restrict language use to a subset of features (restrictability) or extend the
(optimisation), and implementability (how well one can realise the features of the language) e.g., until
language by adding features (extensibility).
recently, garbage collection was not efficiently implemented, and so was often not used..
• Other principles: Other language design principles include preciseness in language specification;
Language translation includes compilation and interpretation. In compilation, a program is translated reliability, the probability that the same result is obtained from repeated use; and security the
(converted) from a high-level language into machine code, which is a form that is executable on an removal (or avoidance) of intentional or accidental threats/damage.
actual or virtual machine. During interpretation, a program is translated and executed one statement at a
time by a “virtual machine” (an agent). Some languages (e.g., Java) are translated via a mixture of
interpreters and compilers. DEVELOPING ALGORITHMS (OR ALGORITHM DESIGN)

Compilation process: Compilers and hence the compilation process has components that include: Algorithmic Problem-solving
Algorithm design is understood as general solutions to problems and not just solutions to instances of the
• Lexical Analyser: Chunks basic language symbols into units (called lexemes or tokens) that are
problem. Thus we are more concerned with converting temperature from degrees Centigrade (oC) to
more easily handled by the syntax analyser.

wss introduction to algorithms—notes of 08 October 2018 page 21 of 107 wss introduction to algorithms—notes of 08 October 2018 page 22 of 107
degrees Fahrenheit (oF), and not just from 4oC, say, to oF. Again, in the travelling salesperson problem Advanced Design Techniques
(TSP), we seek the shortest tour among N cities, where each city is visited exactly once, except for the Other strategies (or techniques), which could be classified into one of the above or combinations thereof,
start city which is visited exactly twice—i.e. at the start and end of the tour. Again, we are interested in of algorithm design are increasingly popular, but we will not go into them, mainly because they exploit
the solution for an arbitrary N, as opposed to the solution for, say, 20 cities. TSP belongs to an important advanced technical or mathematical concepts. Nonetheless, some of them will be examined again when
class of computational problems in Computer Science. we look at different “models” (classes) of algorithms that possibly flout our definitions of algorithms.
• Greedy algorithms: Informally, they select the best solutions available to subproblems as soon as
To solve a problem algorithmically we split it into subproblems and develop algorithms for them. they are recognised, even if the final solution to the whole problem is not the best (optimal).
Seeking a solution is approached as follows (Brookshear 2000): • Dynamic programming: In obtaining and combining partial solutions for an eventual final
Phase 1: Understand the problem. solution, these algorithms re-use partial solutions that would have been recomputed. The partial
Phase 2: Get an idea of how the algorithm might solve the problem. and combined solutions are optimal—that is, the best of multiple solutions are always selected.
Phase 3: Formulate the algorithm and represent it as program. • Backtracking Algorithms: Reject possibly partial solutions that are wrong, and try others until
Phase 4: Evaluate the program for accuracy/correctness and for its potential as a tool to solve other success or failure (i.e., no further options possible). e.g.,when trying to exit a maze.
problems. • Constraint satisfaction algorithms: problem is expressed and solved as a set of constraints,
preferences, properties, or whatever, which any acceptable solution must adhere to.
To develop an algorithm (Phase 3 above): • Randomized algorithms: exploit likelihood of having a good, but not necessarily the most
• We split the problem into subproblems and seek algorithms accurate (or best) solution
for them. • Parallel algorithms: strategies for machines with multiple processors executing the same task at
• Algorithm can be task-specific or a building block problem once.
(i.e. generic problem whose solutions are used in other • Distributed algorithms: strategies for distributed machines—distinct machines “cooperating” to
problems (e.g., sorting, search)) carry out a task.
• Algorithm analysis is a facet of algorithm design that • Internet algorithms: to rapidly search the internet.
establishes how efficient an algorithm is.
• Given an algorithm, we seek better solutions (if any) that we Strategies for [Common] Computational Problems
can; e.g., binary search over linear search. Computational problems are sometimes grouped based on the type of problems solved (e.g., search
• We can also use problem solutions where available. problems) and/or the type of key data structures used, such as graphs or trees. Essentially, a problem is
formulated (expressed) as an instance of a problem class and solved using known algorithmic techniques
Broad Algorithm Design Strategies of that class. We mention, but not elaborate on, some of the problem classes:
Broad common approaches adopted to design algorithms: • Graph/Tree/String Algorithms: Problems are cast into problems for standard data structures and
• Incremental construction: Build solutions incrementally. Each step constitutes more or less operations of such data structures used, possibly augmented, to solve the problem. Such
directly to the final solution. e.g., in insertion sort a list of numbers, say, is scanned and the structures include: graphs (arbitrary set of nodes holding information, with relationships between
smallest value placed in position 1. The remaining numbers are similarly scanned and placed in them (the nodes) held in arcs linking them); trees (like graphs, but a sequence of nodes in it
positions 2, 3 and so on. A list is sorted until all entries are in ascending (or descending order). cannot be visited in a cycle (loop) and the links between nodes are typically visited in one
• Divide-and-conquer: This is a special case of recursion, which we see later. Essentially we split a direction only); and strings (sequences of symbols).
problem into two or more of the same type and repeatedly so, until known solutions are found • Search/Sort/Merge Algorithms: Algorithms to search for, sort or merge collections of objects.
[for sub-problems]. The solutions to subproblems are combined and repeatedly so, until the final Many problems directly or indirectly exploit these algorithms in their solutions.
solution [to the initial problem] is found. • Number-theoretic Algorithms: Algorithms to determine numbers (usually integers) with specific
mathematical properties, such as prime numbers or the factors of an integer.
The subproblems [except the known solution] deal with the smaller input size—informally, • Computational Geometry: Deals with the positioning and search of objects in some space.
smaller number of input elements to handle—e.g., in merge sort, a list of numbers, say, is split • Matrix Computations: Many problems are cast as matrices that should be solved using efficient
into two sublists to sort. Each sublist is split into two sublists and repeatedly so, until a list of one combinations of matrix operations.
element is found. A one-element list is already sorted (i.e., a known solution). Two sorted lists are
combined into one sorted list [via simple comparison of their values] and repeatedly so until the Note that given this viewpoint, a data structure is a method to implement a number of operations—such
final solution (sorted list) is obtained. Mergesort is illustrated adjacent. as searching for, or updating, a data item—on an abstract object that models (describes) some data (c.f,
Brass, 2008, preface and chapter 1).
Typically, most of the work is done when either splitting into subproblems or merging
subsolutions, but not both. In the mergesort example, the work is in merging subsolutions. In Exercise:
other sorting algorithms, such as quicksort (not discussed), the work is done in the splitting. Q1. Write an algorithm for agent CALCUL to carry out integer multiplication in terms of addition
• Algorithm Adaptation: Here we modify an existing algorithm for use in another context or operations only. Assume integers x and y are positive.
situation. e.g., we modify an algorithm to sort integers into one to sort characters. Note that
algorithm adaptation can be much more sophisticated than explained here, co-opting disparate Worked solution, as attempted with students:
algorithm fragments into a constituent whole. We want: x×y in terms of addition.
So we experiment with a few numbers (thereby considering instances of the problem):

wss introduction to algorithms—notes of 08 October 2018 page 23 of 107 wss introduction to algorithms—notes of 08 October 2018 page 24 of 107
Let: 1×2 ≡ 1+1 or 2; (i.e., 1, two times.)
3×2 ≡ 3+3 (i.e., 3, two times; or add 3 to itself once) array cell array values 2-D arrays solution
3×2 ≡ 2+2+2 (i.e., 2, three times; or add 2 to itself twice) Indices in single “row” cells sum ← 0 (* initialise sum *)
i←1 (* initialise num *)
We abstract into an algorithmic solution: 12 5 26 2 15 single while i <= n do
1 2 3 4 5 1 2 3 row cell sum ← sum + x[i]
• we generalise over our observations): x×y ≡ y+y+…+y where y is used x times; i.e., add y to
itself x-1 times. -2 3 4 5 i ← i +1
• The algorithmic construct that best implements above sequence of additions is iteration. cf index -1 9 3 7 od
• Since we do NOT know x, it is best to use the [indeterminate] while construct. 0 15 12 22
in maths array
1 16 8 4 1 i n m
indices
Then we refine our solution to the following, using operations and constructs for agent CALCUL:. x 9 1 7 3 6
x[1] ≡ 12 ≡ x1 row indices start from -2
x[3] ≡ 26 ≡ x3 Y[-2,1] ≡ 3 Y[0] ≡ (15, 12, 22) sum 17
subscript within Y[-2,3] ≡ 5 Y[-2][1] ≡ 3 Summed To sum Unused
Advanced Design Techniques Revisited square brackets Y[-2] ≡ (3, 4, 5) Y[-2][3] ≡ 5
(This material is NOT used for 2018/2019 batch of students; it adds more details to the section
“Advanced Design Techniques” above.)
Advanced design strategies (or techniques) are specific design approaches that are used when a COMMON DATA STRUCTURES
computational problem exhibits certain properties. We already discussed divide-and-conquer as a special
case of recursion. Data Structures
• Dynamic programming: It is used for optimisation problems—problems which can have many A data structure is a set of values with a relationship (i.e. structure) among them. The values could be
solutions, but for which we are seeking the optimal (best) one, e.g., minimum or maximum value. atomic (basic), or other data structures. Atomic values are typically values in the PL whose
In dynamic programming, a problem is split into overlapping subproblems, which subproblems subcomponents, if any, are not of interest. e.g., integers are represented as bits but their “bit structure” is
may also be in terms of subproblems. (A subproblem is overlapping if it is a subproblem to two of no interest in integer arithmetic. (It is of interest in bit-wise operations though.) A structured data type
or more other [sub]problems.) Optimal solution to a problem is built up from optimal solutions to defines a type associated with a data structure. (Recall a type defines a set of permissible values and
subproblems, starting from the smallest subproblem. Solutions to subproblems are reused, not operations.)
recomputed, wherever found.
• Greedy Algorithms: Greedy algorithms are used in optimisation problems as well. They always Common data structures are arrays, records and lists, and are found in most PLs. Sets are common in
make the choice that looks best at each decision-making time, even if the compounded choices problems but many PLs cannot implement them efficiently and so are not often provided. Some other
might not eventually lead to an optimal solution. data structures seen later (stacks, pointers, trees and even graphs) are often treated as abstract data types.
• Backtracking and branch & bound strategies: Backtracking algorithms essentially search for a
Arrays
solution via trial and error. If a solution is not found, the algorithm goes back (backtracks) to a
An array is a data structure that holds a set of values, each of the same type, and each value is accessed
previous point and then tries alternatives, until a solution is found, or failure. Branch-and-bound
via an index. (c.f., indexed sets in maths). e.g., let variable x be an array, with values (given in boxes) in
strategies prevent backtracking once certain alternatives are found: a correct solution can never
the diagram above. The indices are as shown, as well as representation of array values in agent actions
be found in the alternatives so discarded.
and in mathematics.
• Linear programming/Constraint satisfaction programming: In linear programming, one
expresses a problem as a linear function of certain variables (parameters)—the objective function
Conventions may vary with PLs. e.g., counting may be from 0 (as in C) instead of 1 or may start at an
—with constraints as equalities or inequalities on these variables. Constraint satisfaction
arbitrary position (as in Pascal). In the example above, x is a 1-dimensional (1-D) array or vector. A 2-D
problems have no such restrictions on their constraints; constraints may be absolute, may express
array is sometimes called a matrix (c.f., in maths) and may be seen either as a grid having rows and
a preference, may require exhibition of some property, etc.
columns, or a 1-D array of 1-D- arrays, as explained in box above/adjacent.

In general, an n-D array uses n values to access each cell entry but can be interpreted, if the language
allows, as a 1D array of (n-1)-D arrays, a 2-D array of (n-2)-D
arrays, etc. One could also have a 1D array of 1D array of (n-2)-D Record layout
arrays, etc. Arrays are useful when processing data items of the Record
same type in the same way. For example: Sum n numbers found in R R
an array x (indexed m >= n) can be given in pseudocode as above.
ID 12345
Records sex 'M' Fields
A record consists of a sequence of values of possibly different
GPA 3.99
types, where each value is accessed via distinct field names. (This
contrasts with access via indices in arrays.) e.g. a record of student

wss introduction to algorithms—notes of 08 October 2018 page 25 of 107 wss introduction to algorithms—notes of 08 October 2018 page 26 of 107
details R may hold the student's ID (integer), sex (character) and GPA (real). This may be represented 2) Data structures tend to be accessed using corresponding control constructs. e.g., records and arrays
diagrammatically as above. and lists are accessed sequentially and variant records and arrays via choice. (Arrays are random
access but are sequentially accessed by sequential control structures.)
Conventions may vary but to access a field location or value, the access is typically represented by the
field name separated from its record name by a period. Thus, R.sex represents access to the field “sex”
within record R. A field may also be a record (i.e., a composite value), in which case its fields are also ADDITIONAL CONTROL AND DATA CONCEPTS
separated by a period. For instance, suppose S is a record containing the field “person”, which field is We indicate more concepts in order to ease the development of algorithms. However, a number of
also a record containing the field sex. Then, access to the field sex is represented by: S.person.sex. notations may be introduced as needed or beyond class.

Records are used where we want to process a collection of values, especially in different ways, as Assignment
attributes, say, of some other entity. In memory, fields are seen as offsets from the base address (i.e., start This is an operation used to store values in variables (i.e., locations in memory). Values assigned are
address) of their record. That is, field names are seen as names of locations whose addresses are typically literals, constants, variables or expressions [that simplify to specific values]. Literals are found
determined as offsets (i.e., displacements) from the base address of their record. in the data type (e.g. 1, 2.5 or letter 'c') and stand for themselves totally. Constants are named literals;
they have a fixed value associated with a name through out the algorithm or program. Variables are
Variant Records named locations in memory and [the variable names also] stand for the values that they currently hold.
Thus, the term variable has 2 possible interpretations: a location in memory, and the value it holds, e.g.:
Variant Record: (a) variant objects; (b) representation and (c) layout x ← 25 (* store 25 in the variable [location] x *)
Object type and Representations of variants x←y (* store [value in variable] y in variable [location] x *)
attributes Memory layout of record data structure x ← 25+y (* compute sum of [value] y and 25 and store in variable [location] x *)
Tag Value Field Access
Memory Memory cells allocated to:
Square: s R.side Suppose PI is a constant set to 3.1459. Then, the following assignments are equivalent:
locations square rectangle triangle
side (of type integer) x ← 2*PI*r (* compute perimeter of circle with radius *)
1000 s r t
x ← 2*3.1459*r
Rectangle: length, r R.length; 1004 side length side1
width (integers) R.width 1008 width side2 GOTO (or goto) Instruction: This is a sequential construct used to transfer control to another, specific
Triangle: side1, t R.side1, 1012 side3 instruction in an algorithm. The use of goto's is strongly discouraged, as they often lead to algorithms
side2, side3 (reals) R.side2, R.side3 1016 that are hard to understand [by humans].
(a) (b) (c)
Determinate Loops: It allows iteration for a fixed (i.e., determinate) number of times. This is unlike the
These are records which hold alternative sets of mutually exclusive fields. i.e., alternative sets of WHILE loop which is indeterminate, executed for a varied, unknown and/or possibly infinite number of
attributes cannot describe the same entity instance at the same time. As such, each variant (i.e., times. Determinate loops are characterised by the FOR loop, for which we adopt the convention:
alternative) associated with a tag value (found in a tag field) is uniquely indicated when the variant is in FOR i IN lb (lower bound) TO ub (upper bound) DO S
the record. where i, the loop variable, takes successive values from lb up to ub inclusive. For each value of i
obtained, the loop body, S, is executed. No execution takes place if lb>ub initially. Variants may
Suppose a record R holds the dimensions of geometric objects as shown in (a) above, with tag values s, r decrement from lb down to ub or the range “lb to ub” may be replaced by a finite ordered set.
and t respectively. The access to fields of record R, for each tag, is also shown in (b). Also suppose
record R occupies memory locations 1000 to 1015 as shown in (c); locations from 1016 are not used by Further Indeterminate Loops: Some constructs such as:
R. Assume the memory size of all fields are the same (i.e., 4 bytes in this example), and the start repeat S until C
addresses of fields in each variant are as shown. Notice that the address ranges of fields in distinct test the loop condition C after executing the loop body S, and stop executing when C is true. In contrast:
variants overlap in memory: the record cannot hold two variants at the same time. e.g., variant square do S while C
occupies address range 1000–1003, while variant rectangle occupies 1000–1007. is like the repeat loop except that it stops executing when C is false (as opposed to true).
Sets Other Choice Constructs: Guarded (or multi-way) choice instructions, such as case statements in Pascal,
A set is a collection of values, which are seen as members of the set. Small sets (i.e., those with very are discussed as encountered.
[few] members) are often represented as a sequence of bits [e.g., in an integer, or a sequence of integers]
where the i’th bit is set/stored if the i-th value [in the set of possible values] is a member of the set, and
reset [to zero] if it is not. Necessarily, for efficient access, an ordering is imposed on the elements of the ABSTRACTIONS AND GENERALISATIONS
set. For larger sets, more elaborate data structures are used, such as binary trees (see later), to store the Loosely, abstraction enables one to identify and highlight certain things—attributes, ideas, concepts,
values of interest. For efficiency, an ordering is typically imposed on (or required of) the set elements. properties, etc.—that are of interest, similar or whatever, while at the same time suppressing, hiding or
ignoring those that are not. Generally, abstraction allows one to find, investigate and/or elaborate on
Notes: similarity or common patterns in possibly disparate entities. In computer science, abstractions tend to
1) Structured types as described above derive from Cartesian product (c.f., records) and disjoint sum hide (for later reconsiderations) rather than ignore.
(c.f., variant records) from mathematics, as well as other data constructs.

wss introduction to algorithms—notes of 08 October 2018 page 27 of 107 wss introduction to algorithms—notes of 08 October 2018 page 28 of 107
Generalisation broadens the scope of an application, discussion, abstraction or whatever, to be applied to • a name, formal parameters and information such as the type of its parameters and that of its
or cover a larger domain of objects, interests, etc. While abstraction enables disparate artefacts to be return value (if a function) [in what is termed a procedure header], and
treated alike, generalisation allows possible differences to be isolated and treated, possibly specially, in • a [procedure] body, which is the set of instructions it [the procedure] denotes, together with
an otherwise uniform treatment of objects, interests, etc. possibly other declarations (and definitions) within it.

This section discusses procedural abstraction, and generalisations through parameter passing and block We adopt the notations:
structuring of scopes. proc procName (formal parameter list) ≡ procedure body end
func funcName (formal parameter list): return type ≡ function body end
Naming Things return value
We name things when we give them labels by which they may be called, and often use the labels (i.e., for procedures and functions respectively, though the keywords func and proc may be omitted. The
the names) in lieu of the things in various situations. This is very convenient! e.g., our names on a course “procedure body”, or “function body”, is between ≡ (the equivalence sign) and the end keyword. “Return
list stand for us, when discussing course membership. It would be awkward to go stand on a line or type” is the type for the value returned by a function using the instruction “return value”. “Formal
present one's self whenever issues related to a course list are discussed, or have to give an accurate parameter list” is a comma-separated list of formal parameters enclosed in parentheses. If there are no
description of each student whenever he or she is referred to. Similarly, in programs (and algorithms) parameters, the parentheses are optional. For example, procedure Go Right may be defined as either of:
names are convenient to use. In fact, we abstract these things to only its relevant properties of interests proc Go Right ≡ move to junction; turn right end
[under discussion]. proc Go Right () ≡ move to junction; turn right end

Under assignment instructions, we already saw the convenience of naming constants. In most Note:
programming languages, names (labels) are called identifiers, because they are used to identify various 1) A procedure or function is called or invoked by simply giving its name where its needed. After the
entities. They are typically constructed from characters of the English alphabet and numbers. However, procedure body is executed (carried out), the instruction immediately following the call or invocation
names of instructions (algorithmic actions) in programming languages are often given as numbers (or point (i.e., where the procedure was called), is executed.
number plus a colon), and are specifically termed labels. In the following subsections, we focus on the 2) Procedures and functions are generally called subroutines (or subprograms, [within programs]). In
significance, not representation, of names (identifiers). this course, procedures are synonymous to subroutines, as you might have already figured out.
3) The technical notions of “binding”, “declaration” and “definition” are related though their precise
Procedural Abstraction Procedural Given Algorithm Modified meanings vary with were they are used. (Informally, the terms are used interchangeably in certain
Here we give a name to a sequence of Abstraction Algorithm contexts.)
instructions and call it a procedure. That a. Binding: A binding is an association between two objects, typically between a name and a
is, a [programming language] procedure proc Go Right ≡ algo M ≡ algo M ≡ programming language object such as a variable, value or procedure. Thus, constants and
is a named sequence of instructions; and Move to junction Move to junction Go Right procedural abstractions mentioned earlier use bindings.
its name is used wherever that sequence Turn right Turn right Go Right b. Declaration: A declaration states constraints on an association (binding) or identifies its attributes
of instructions is used. Conceptually, the Move to junction Enter house and/or how it (the association) can be used. (Informally, it states that an entity exists.) e.g., a
procedure name is replaced by the Turn right given name can be declared an integer variable or a structured data type. The declaration thus
sequence of instructions (the procedure Enter house restricts its use to those permissible to integer variables or to structured data types respectively.
body) whenever the name is found c. Definition: A definition states what an association means and describes precisely how it can be
during instruction execution. This is used. For instance, a variable definition associates a fixed location in memory to a name, and so
illustrated with procedure “Go Right” adjacent. identifies where its value (i.e., the variable's value) may be found and used, in ways permissible
to variables and their locations. In contrast, a declaration need not define the variable's location,
Procedural abstractions are convenient because they hide unnecessary details. Furthermore, they can though most PLs require the variable's type [in any declaration].
make algorithms more compact and hence more efficient (through repeated use of same code) and easier d. Procedure declaration and definition: Procedures are typically declared by giving their header,
to understand and use. Furthermore, it is a powerful tool for recursion (see later). and defined by giving additionally their body. Notice how the procedure header prescribes
(declares) what components of the procedure (parameters, return value, etc.) may be, and how
Procedures that can return a value are called [programming language] functions; they return values just they may be used; and the procedure body gives (defines) the meaning associated with the
as mathematical functions do. However, the term [PL] procedure is often used to include them as well. procedure name.
e. Variable definition: Variable definition typically has the semantics of assignment [of value to
Procedures are also used for generalisation—to make them handle a wider class [of similar] problems— location] rather than allocation of memory to it, as defined above.
by taking parameters or arguments. (c.f., arguments in mathematical functions). In effect, parameters f. Initialisation: provides an initial (or very first) value. In relation to variables, an initialisation is
permit us to use the same procedure in different contexts, with different values. Parameter passing is a also seen as a definition.
mechanism that permits us to transfer data from/to an algorithm's operating context to/from a procedure. Call-by-value: (a) functions; and (b) function call representation.
In effect, when a procedure (or function) is declared or defined, it may be given formal parameters Parameter Passing Mechanisms func f(x) ≡ return 2*x end proc g Call f proc f
which, conceptually, are dummy arguments (variables) that serve as place holders for the actual Parameter passing defines how proc g ≡ …
parameters (or arguments) that are used when the procedure is actually called or invoked. a 3 copy a to x→ 3 x
actual parameters are associated a←3
with formal parameters and then y ← 2+f(a)
A procedure is defined by giving it: used where the formal parameters end y 8 ← copy to 2+f(a) 6 f(a)

wss introduction to algorithms—notes of 08 October 2018 page 29 of 107 wss introduction to algorithms—notes of 08 October 2018 page 30 of 107
are used within the procedure. A parameter is an object or entity that assumes values, attributes (or program] only a bit at a time, and not read all the input at once. Also, one may avoid division by zero in
whatever) depending on the circumstances in which it is used. call-by-value and call-by-reference are the computation if the code that could induce division by zero is never executed. e.g., given y = 0, and
most common are the two most common parameter passing mechanisms. proc f(x) ≡ … if b then z ← x else … end … end
then the call f(3/y) ≡ f(3/0) does NOT raise an exception (i.e., error) if b never evaluates to false.
Call-by-value: Here, the value of the actual parameter is computed and stored in a location obtained for
its matching formal parameter, and is visible only within the procedure scope. Essentially, you copy the Procedural Scopes and Related Concepts
value of your actual parameter to the location created for your formal parameter. Changes to the formal Scope: The scope of a variable, name, procedure (or whatever) is the range of instructions over which it
parameter do not affect the actual parameter. can be recognised, identified or used. Thus, in most languages the scope of the formal parameters is the
procedure. Similarly, the range of declarations and definitions is the procedure within which they are
Call-by-reference: Here, the location (address) of the actual parameter is associated with the formal declared or defined. Such declarations and definitions are said to be local. Variables defined outside a
parameter. The formal parameter is then used as you would the location of the actual parameter. Changes given scope but used within it are said to be non-local; they are global if declared in the outermost
to the formal parameters do affect the actual parameters. e.g.: scope. Sometimes, the terms non-local and global are informally used as synonyms.
proc h(x,y) ≡ x←x+2; y←2*x end
where x is by value and y is by reference, and Block structure: Block structure is a strategy of partitioning algorithms so that their scopes may be
proc g ≡ … a ← 3; b ← 42; h(a, b); output a, b; end nested only in non-overlapping ways. That is, a procedure is [completely] defined within a single
procedure (not across procedures) and two or more procedures defined within another have distinct
In procedure h, parameter x assumes value 3, and 2 is added to it [to give 5], and y has the value 10 (≡ scopes. Block structure makes it easier to implement programming languages, but also permit scopes to
2*x = 2*5). In procedure g, variable b associates with y and so gets the value 10, since it is passed by be statically (textually) determined. Block-structure may be seen as a generalisation of scoping.
reference. Thus, b is output as 10 (not 42). However, variable a is output as 3, since the change to x in
procedure h does not affect parameter a. Side-effect: A side-effect is produced typically as a change to the value of a variable, whenever a
computation (sequence of instructions) produces an effect outside the scope of the values affected. That
Call-by-name: Call-by-name is historic, not common, but conceptually important. Here the computation is, whenever a computation is non-local to the scope of the values affected. An example is a global
(sequence of instructions) generating the actual argument is stored (associated) with the formal argument variable modified within a procedure; the scope of procedure masks that of the global variable, and so
[name], just as in procedural abstractions. Thus, the formal parameter is “executed” wherever its name is the global variable is non-local within the procedure. This concept may need further clarification later.
found. The stored computation is actually a procedure stump (or placeholder) called a thunk.
Conceptually, one textually substitutes (replaces) the formal parameter with its actual parameter Alias: An alias occurs when an object is attributed two or more names, e.g., in call-by-reference, the
wherever it is found in the procedure body. Thus, the value the thunk computes may change on each formal and matching actual parameters are aliases. Aliasing is more common via pointers. (See later).
execution if the values of variables it uses change between thunk executions. (Technically, the thunk
assumes the current context—the set of bindings of variables in scope—as it changes. Other Generalisation Concepts

For example, given procedures g and h in (a) and (b) below and assuming variable i is global, the step-
by-step execution h(i+3), as called from procedure g, is given in (c). Notice how x has been textually Substitution: replacement of like with like.
substituted by “i+3” throughout procedure h. Values computed for each line are given in comments.
PL environments: environment (the set of bindings (associations) of variables to locations)
Call-by-name interpretation
PL contexts in PL theory: (1) PL types that determine whether a variable makes sense. (2) “Secondly,
(a) Procedure g (b) Procedure h (c) Call-by-name: substitution & execution evaluation contexts for a language are terms with a hole that can be filled with another term.” It is like
Assume i is a global proc h(x) ≡ h(i+3) ≡ (* x ≡ i+3 *) having an arithmetic expression, say, with slots (“holes”) where other arithmetic expressions (“terms”)
variable. i← i+2 i ← 4+2 (* ≡ 6 *) are fitted and the resultant expression can then be evaluated.
if x>0, then if (i+3) > 0 then (* ≡ 6+3 > 0 ≡ TRUE *)
proc g(x) ≡ y ← i+2 y ← i+2 (* y ≡ 6+2 ≡ 8 *)
i←1 end end PROGRAM DEVELOPMENT
h(i+3) i ← 2*i i ← 2*i (* i ≡ 2*6 ≡ 12 *) Program/software development uses broad strategies to guide their designs. Strategies are broadly:
end y ← x+2 y ← (i+3)+i (* y ≡ 12+3+12 ≡ 27 *) • Top-down: split a problem into subproblems and solve the subproblems.
end
• Bottom-up: find solutions to simple subproblems and group them into a solution to the problem.
• Modular: split a problem into self-contained units called modules, which are charged with
Question addressing specific tasks as services provided to other modules and problems. (Modules are
What are the successive values of x in h(x) if the line “i ← i+2” is removed in the above example. variously defined.)

Call-by-need: This is like call-by-name but the thunk is executed once, when first met [on any execution Stepwise Refinement (SWR)
path], and the value obtained is stored and then used in all other places (as in call-by-value). Also called SWR is a top-down design strategy which we shall loosely describe but is also used for algorithm
lazy evaluation, this technique is useful where one needs to use only part (or subcomponents) of a value discovery [and design].
at a time, or use values only when they make sense. Thus, for example, one consumes the input [to a

wss introduction to algorithms—notes of 08 October 2018 page 31 of 107 wss introduction to algorithms—notes of 08 October 2018 page 32 of 107
Approach
Broadly, the top-down approach is: Stage 2)
1) Split problem into subproblems. That is: • let fxy = 2x2 – 5y and gxy = x2 + 2x + 2y2 + 1. (in my head!)
a. Identify main data structure(s). • WHILE (fxy and gxy are different) DO
b. Reexpress problem as an algorithm in terms of subproblems. read two numbers
c. Verify correctness of data structures and algorithm. print numbers and their product
2) A subproblem is solved if a solution is found (or an existing one is used). Otherwise, split it into new OD
[sub]problems as in (1).
3) Repeat steps (1) and (2) until all the subproblems are solved. Stage 3)
Rewrite above to:
Stepwise Refinement • let fxy = 2x2 – 5y and gxy = x2 + 2x + 2y2 + 1. (in my head!)
Advantages Disadvantages • read two numbers
• Complete view of initial problem at each level. • Generic, no strong guidelines for • WHILE (fxy and gxy are different) DO
• Can focus on one subproblem at a time and reuse user decisions. read two numbers (* <== problem! First no. read is NOT printed! *)
[subproblem] solutions. • Depends heavily on experience of print numbers and their product
• Used at system & program levels, as convenient. the user. OD

Stage 4)
Rewrite above to: SOLUTION A
Example let fxy = 2x2 – 5y and gxy = x2 + 2x + 2y2 + 1. (in my head!)
Write a program that repeatedly reads two numbers x and y and prints them together with their product. • read two numbers
Printing stops when the two numbers satisfy the equation 2 x 2 5 y =x2 +2 x+ 2 y 2 +1 . • WHILE (fxy and gxy are different) DO
2x2 – 5y and gxy = x2 + 2x + 2y2 + 1
Possible [algorithmic] solution: print numbers and their product
1) Read two numbers. read two numbers (* <== problem solved! First no. ALSO printed! *)
2) Print numbers and their product OD
3) Repeat (1) and (2) unless equation holds.
Alternatively: SOLUTION B.
Refining, using the WHILE construct we have: let fxy = 2x2 – 5y and gxy = x2 + 2x + 2y2 + 1. (in my head!)
Read x and y • read two numbers
WHILE equation does not hold DO • print Numbers and their product (* <== problem solved! First no. ALSO printed! *)
Print numbers and their products • WHILE (fxy and gxy are different) DO
Read numbers read two numbers
OD print numbers and their product
OD
We try to put on paper the thought processes and potential challenges involved before attaining a final
solution. The approach is to analyse, design and implement an algorithmic solution, iteratively, not Note that in SOLUTION B we can print when fxy and gxy are the same! This is an acceptable solution,
necessarily in order. given a possible [and reasonable] interpretation of “when” in the question.
1) Analysis of the problem: i.e. think about the problem, with a view to understanding it better.
2) Design: Propose a solution – how to go about solving the task. Stage 5)
3) Implementation: give code executable by an agent (agent CALCUL in this case). Solution A can now be refined into the Agent CALCUL program given:

Stage 1) Homework: Implement the algorithms (SOLUTIONS A and B) in Stage 4 for Agent CALCUL.
THOUGHT (in my head!):
• We want to print numbers read, together with their product. Stage 6) Possibly Improving the Solution?
• We need to read two numbers Identify values of x and y for which loop will terminate. (in my head!)
• We need to do the above repeatedly, ie., again and again. Solve for x and y (on scratchpad) such that the equation equals zero:
ACTION (written): x^2 – 5y – 2x – 2y^2 - 1 = 0
Therefore course of action of program written is some like: (x^2 – 2x + 1) = (2y^2 + 5y + 2)
• read two numbers (x – 1)(x – 1) = (2y +1)(y + 2)
• print numbers and their product
• do the above again and again Therefore: x = 1 and y = -1/2 or y = -2. Our loop condition can now become:
• stop when equation holds. (x == 1) AND ((y == -2) OR (y == -1/2))

wss introduction to algorithms—notes of 08 October 2018 page 33 of 107 wss introduction to algorithms—notes of 08 October 2018 page 34 of 107
which makes it easier for us to test and prove the loop termination condition.
Example use of recursion
Homework: Implement your improved solution for Agent CALCUL. 1. Explanation in terms of GOTO's.
1: IF (n>0) THEN
Stage 7) Some Correctness concerns (JUST FOR THE ARGUMENT!) write (n, "! ")
Integers and rational numbers can be represented exactly in computers, though reals cannot: only their N ← n-1
approximate values may be stored, and so they cannot be compared for equality. Given that y may be GOTO 1
fractional, if it is stored as a real number, it cannot be compared for equality! A solution is to modify the END
loop condition to test if the difference of two numbers is less than a very small absolute value, eps (e.g., write “TAKEOFF”
10-5, say). So the loop condition becomes:
abs(((2*x*x – 5*y) – (x*x + 2*x + 2*y*y + 1)) < eps 2. Explanation in terms of iteration.
Note that equivalent expressions from above analysis can be used, using eps in equality tests with reals! WHILE (n>0) DO
write (n, "! ")
Homework: Implement your revised solution for Agent CALCUL. N ← n-1
proc main() ≡
(* NEXT LOOP *)
Stage 8) Is the Improved Solution an Improvement? (JUST FOR THE ARGUMENT!) OD count_down(20);
We may observe that nothing in the [original] computational problem requires the equation to equal zero, write “TAKEOFF” END
only that both sides be equal. Thus, if x = 4 or -3 and y = 1, both sides equal 27, and the loop terminates.
3. Explanation in terms of Recurrent execution of instructions in proc count_down(n)≡
Careful thought shows that there are many more pairs of integers (and even more of reals) that can make pseudocode. IF (n>0) THEN
both sides of the equation equal. Our so-called “improved solution” actually relaxed the acceptable set of 3.1. That is, we print a number n; we then print the remaining write(n, "! ");
terminating values, and so made a larger number of pairs of x and y to be printed. numbers starting with n-1. If we can't print any number, we print
count_down(n-1);
“TAKEOFF”.
In effect, we introduced the following: first, we approximated our solution to gain a simpler loop 1. Print number n, if n > 0. ELSE
termination condition by making explicit the termination values of its variables; which is dangerous if 2. Print rest of numbers from n-1 if n-1 > 0. write("TAKE OFF");
the extraneous numbers must not be in our solution set. Second, to “fix” the approximation in this 3. Otherwise, write “TAKEOFF”. END
solution strategy, we need to enumerate all the required pairs of variables loop termination. That is, we END
need to first calculate an infinite list of pairs of values for x and y. The morale is: [Note: parameter n tracks the next number to print, and the
Be wary of improvements, unless they are provably correct! remaining numbers to print.] Program Output
20! 19! 18! 17! 16! 15!
3.2. We use the same code to print numbers from n-1. That is:
14! 13! 12! 11! 10! 9!
RECURSION 1. Print number n-1, if n-1 > 0.
Recursion is a powerful algorithm analysis and design tool. It provides a complete mathematical 2. Print rest of numbers from n-2 if n-2>0. 8! 7! 6! 5! 4! 3! 2! 1!
specification of an algorithm. That is, it not only states precisely in mathematical terms what the 3. Otherwise: output “TAKEOFF”. TAKE OFF
algorithm should do, it also prescribes how it should be done.
3.3. We use the same code to print numbers from n-2. That is: Contrived equivalent algorithm
The term is related to, but distinct from, the notion of recursion defined to be equivalent to computability 1. Print number n-2, if n-2 > 0. proc count_down(n)≡
in the Theory of Computation. Therein, the notion of recursion is used to mathematically define possibly 2. Print rest of numbers from n-3 if n-3>0. IF (n > 0) THEN
infinite data and algorithm structures, and hence used to formally define their semantics (meaning). 3. Otherwise, output “TAKEOFF”. write(n, "! ");
END
Recursion is the basis of many algorithm design strategies, and is generally more easily used in top- <... snip ...> n ← n–1;
down design strategies, though bottom-up ones, say, are often couched in them. (Dynamic programming
techniques, and relaxations thereto, are generally used in efficient bottom-up techniques; they too can be 3.(n-1). We use the same code to print numbers from 2. That is: count_down(n);
implemented top-down, with the aid of recursion.) 1. Print number 2, if 2 > 0. IF (n==-1) THEN
2. Print rest of numbers from 1 if 1 >0. write("TAKE OFF");
Illustrating Recursion 3. Otherwise, output “TAKEOFF”. END
Below we pass the number n = 20 to the recursive function. The function tests this parameter to check if END
it is greater than zero. If it is, the number is printed, and the function calls itself recursively, but this time 3.(n). We use the same code to print numbers from 1. That is:
with a parameter value of n-1. When n eventually gets to zero, a message is printed and program control 1. Print number 1, if 1 > 0.
returns to main. 2. Print rest of numbers from 0 if 0 >0.
3. Otherwise, output “TAKEOFF”.

Note: Here, we cannot “Print rest of numbers from 0” (instruction 2), since 0 is NOT greater than 0.

wss introduction to algorithms—notes of 08 October 2018 page 35 of 107 wss introduction to algorithms—notes of 08 October 2018 page 36 of 107
 that they use such data without making any assumptions about the data, except those necessary to do the
3.(n+1). We can use the same code to print numbers from 0. However, since n = 0 we can't print n, nor given task. For this (to do the task), a “concrete” representation is defined, independent of the programs
numbers from 0 (since 0 > 0 is false). Ignoring them, we can now do instruction 3. That is: that use the [abstract] data. Typically, PL procedures (or functions) are defined as the interface by which
3. Otherwise: write “TAKEOFF” operations are carried out on the data, either as constructors (to create new values of the data type) or as
selectors (to extract existing values or components of the data type). In essence, data abstraction seeks to
4. Explanation in terms of Procedural Abstraction. identify a basic set of operations for objects of a given type and use only those operations, and any
Once the procedure is named, step 2 can be implemented by invoking the name. operations expressed in terms of them, to manipulate objects of that type. (c.f., Abelson, 1996) These
PROC count_down (n) ≡ basic operations can similarly be used in further data abstractions.
IF (n>0) THEN
write (n, "! ") (* print number n, if n > 0 *) Note:
count_down(n-1) (* print rest of numbers from n-1, if n > 0 *) 1) Typically, constructors return values of the newly constructed type; the type has only one way of
ELSE (* otherwise *) being constructed. Selectors take as argument the value of a type and return one or more of the
write “TAKEOFF” components of the type.
END 2) Sometimes the terms constructor and observer [function] are used—instead of constructor and
END selector—with a slightly different meaning for “constructor”. As such, a constructor is seen as an
operation yielding the result of the new type [being defined], which is necessarily in the codomain of
Exploiting Programming Language Recursion the operation. Observers then are simply all the operations that are not constructors; the observer
Expressing a function or procedure in terms of itself, as we have just done, is called recursion. (Note that yields information about the state of a component of the [constructed] type; its codomain is of the
there are other meanings of the term “recursion”, e.g. it is synonymous to the mathematics notion of constructed type only if a component is of that type. (e.g., the component of a tree is still a tree.)
computation (discussed elsewhere) in some contexts. 3) A zero-argument function, often used to create a type [from nothing], is a nullary operation and is
more usually called a constant. Thus, a constant such as 5 is a nullary operation: it takes no argument
Basically, during recursion, we express a problem as a simple version (simpler in some sense) of itself; and always yields the same result, 5. Things are simpler when constants and functions are treated
the simpler version is expressed as an even simpler version of its self (this is the general case) until the uniformly.
problem is eventually expressed in terms of a known solution (this is the base case).
Abstract Data Types (ADTs)
In our example the general case is to print a positive number, and then print the rest of the numbers Constructing user defined types is the simplest of abstractions. However, the term abstract data types
starting from the number's predecessor. The base case is to print “TAKEOFF” when the number is zero. (ADT) is used where one defines permissible operations to use an object, masking the implementation
details of the objects and their operations from their user(s), and providing a well-defined and robust
In general, the general case must converge to the base case(s), for the algorithm to work. Otherwise the interface to the data objects. Hiding details is known as encapsulation: implementation details should
solution is said to diverge. If our example solution was written differently, contrived as in the box above, typically change without affecting how the ADT is used. e.g. a stack (see later), seen as an ADT, has the
the solution will diverge if the number input is negative: it will never decrement to a positive number! operations:
pop ≡ remove last element inserted;
Note that a “simpler problem” is one that has fewer issues to solve or address and so if the issues can push(x) ≡ add item x to the top of the stack.
only get fewer and fewer (not necessarily so from one subproblem to the next), eventually we will reach empty ≡ true if and only if (iff) the stack is empty (i.e., stack has no element); false otherwise.
a point where we have no issues to solve or address. full ≡ true iff stack is full (i.e., one can’t add any further element to it); false otherwise.
and has its structure defined in terms of:
In our example, printing (n-1) numbers is “simpler” to solve because we have one less number to print. top ≡ the top of a stack, where elements can be added or removed.
(the “problem size” reduces by 1 in each “simpler” version and so we will eventually have a zero bottom ≡ the bottom of a stack, where the very first element is added or (the last—not latest!—
problem size! element is removed).

Homework: The stack can be implemented in [terms of] an array where the index to the first array element is the
Q1) (a) Informally explain why the contrived algorithm is equivalent to the original one. (b) Suppose Joe index for bottom; and top points to (i.e., is an index to) the last element of the array. (Implementations of
decides to apply both algorithms to negative numbers. Why does the original algorithm not diverge, but top and bottom may vary. e.g., top could index the cell where the next stack element is inserted.)
the contrived one does, on negative numbers? (Hint: Consider the possible input instances.) Alternatively, the stack could be implemented as a linked list (see later). In either implementation, a user
Q2) Give a recursive definition of the factorial function. Explain how it should work in terms of should not know whether an array or a linked list is used. In practice, common programming languages
“simpler” problem size. Hence, write a program executable by Agent CALCUL. (PL’s) such as C and Pascal do not completely encapsulate: the types used for procedural arguments must
be known. Tricks include using an intermediate data type which does not change frequently with
implementation changes.
DATA ABSTRACTION AND ABSTRACT DATA TYPES
Data abstraction involves giving names to data so that one can separate the method of using an object Ideally, we need an abstraction (one with no implementation) that may be instantiated on demand as
(data) from how it is built. More precisely, data abstraction is an approach that permits us to “isolate how provided, for instance, by the inheritance mechanism of C++. An alternative, more conceptual tool for
a compound data object is used from the details of how it is constructed from more primitive data encapsulation is modules. The term module has assorted meanings but it is generally used to group data
objects” (Abelson et al, 2013). It structures programs so that they operate on “abstract data”; that is, so

wss introduction to algorithms—notes of 08 October 2018 page 37 of 107 wss introduction to algorithms—notes of 08 October 2018 page 38 of 107
and procedures; a function that collectively provides a service (via an interface), but hide/control what An invariant is something (an attribute, property or whatever [of the algorithm]) that does not change; it
aspects of its data and internal operations are visible. We do not consider modules further. does not vary [in well-defined respects] as the algorithm is executed; variants do vary with algorithm
execution. The invariant is chosen such that we can verify that our computation so far is correct, so long
Note: as the invariant, on inspection, does not change. Conversely, our algorithm must restore the invariant,
1) A service is a set operations that share a common purpose. Thus, a service is a main function whenever its computation changes it, so as to correctly assert (i.e., maintain) program correctness.
provided in order to satisfy goal(s) or objective(s).
2) Necessarily, a module consists of a collection of independent (i.e., standalone) operations that In contrast to invariants, a variant (which is also an attribute, property or whatever [of an algorithm]) is
interact with each other to provide specified services. Otherwise, a monolithic program would be a chosen so that its value varies (converges), with algorithm execution, to a fixed upperbound (or
module too. equivalently, to a fixed lower bound). In more mathematical terms, it tends to a finite limit. The idea
then, is to choose a variant that decreases (or increases) with algorithm execution, and is guaranteed to
stop increasing (or equivalently, decreasing) when algorithm execution stops. Thus, intuitively, the
COMMON ALGORITHMS: SEARCHING property tracks (or helps “count”) to a maximum (or minimum) number of algorithmic steps—however
We examine some algorithms commonly used to solve problems by developing them: search, sort and steps are defined—used in an algorithm.
merge algorithms. From an algorithmic point of view, we have taken into account correctness and
efficiency issues, though we later discuss some of these concepts a bit more. This section considers We further discuss invariants and variants, alongside recursion hints, after merge sort. (Other relevant
searching, and the next one sorting. Merging is considered under insertion sort, and later under file programming proof techniques are also mentioned.) We will NOT do formal proofs here (in this course),
processing. though we will think in these terms (variants, invariants) as well as in terms of another key programming
idea (identify guiding strategy/objective/or whatever) in order to demonstrate the correctness [and
Linear (Sequential) Search termination] of algorithms.
In linear (sequential) search, we look for an item in a list of elements (items). The list of items may or
may not be sorted. We consider first the unsorted case: In search for an invariant for the linear search algorithm, we observe that EITHER the sought item must
be in the range (or set) of values (elements) searched OR we are beyond that range. In an array, this
Analysis: range is given by an index to the first element to compare and the last element [to compare].
• Compare [sought] item with each element in the Elaboration of Linear Search Algorithm
list. The invariant then is: item is in the range of elements searched. Tacitly, as implied by the invariant, if the
a) Locate Sought Item
• Should be systematic, e.g start comparisons from item is NOT in the said range, then we must be beyond the range. To do the task (i.e., searching), for an
Get first element of list;
beginning of the list to end. If not systematic, we unsorted array, we can do nothing more than consider the elements in range [so far].
while [compared] item and element
might not be effective (get desired results) or [to test] are different do
efficient. e.g., we may loose track of which items For termination we can consider the size of the range still to search. This size (i.e., number of elements
Get next element to test
are already (or yet to be) compared. in the range) tends towards zero, as each element is considered. [Zero is the upper bound.] This number
od
• Above considerations are okay as an informal is zero only when the item sought is not found.
Declare if search is a success or not
analysis.
Using the informally developed algorithm, we can Elaboration of Binary Search Algorithm
b) Allow for Item not Found
More formally, we need to formulate problems in terms of say it terminates because the size of the range—
Get first element of list;
an algorithm that does not change. (See correctness and our variant—decreases by one on each iteration of Set lower and upper bounds of the search to first
Compare item with first element;
algorithm design later). Based on informal analysis, we the loop and it cannot go past zero. All other and last elements of array. (call them lo and hi).
while [compared] item and element
develop the solution as shown in box adjacent. (Notice that instructions too—“Get the first element”, the IF
[to test] are different AND
the second point of the informal analysis above suggests statement, etc.—terminate. while low is not greater than hi do
there are more elements [to test] do
the use of iteration). Get next element to test Set variable mid to mid-point of lo and hi
Note: if value at mid-point is value sought then
od
Home Work: While our theoretical analysis suggests that the Value is found; return its position in the array.
Implement the above linear search algorithm for agent algorithm terminates, we must make sure that else if value at midpoint < value sought then
If item and element are the same then
CALCUL. Check for its correctness. Can you do better? each instruction is do-able and terminates, and the Set lo to (mid-point + 1) (* why add 1? *)
Declare search a success
i.e., can you improve on the algorithm so that it is more combined set of instructions giving the algorithm else
else
efficient? terminates. Hoare’s axiomatic semantics, for set hi to (mid-point -1) (* why subtract 1? *)
Declare search a failure
example, provides mathematical machinery for end
end (* item and element are the same *)
More formal analysis such checks, but they (the machinery) are beyond od
More “formal” analysis—i.e., with a view to formal the remit of this course.
c) Allow for Possibly Empty Lists
mathematical proofs—require that we: If list not empty then if lo is greater than hi then
• Seek invariants (to assert what has to be done); and <... insert algorithm (b) here ...> Value is not found; return an invalid position
• Check the termination of our algorithm (through a else else
variant). Declare search of an empty list return mid (* this is redundant. Why? *)
end (* list not empty *) end

wss introduction to algorithms—notes of 08 October 2018 page 39 of 107 wss introduction to algorithms—notes of 08 October 2018 page 40 of 107
Linear Search of Sorted Elements comparison-based, bucket-based and swap-based. In comparison-sorting, we compare two items and
The analysis and development of algorithms for a sorted list of elements is similar to that for the find the smaller one (or bigger).
unsorted case, except that an item is not found once algorithm execution goes past its position in the list.
In bucket-based sorting (also called distribution sorting) items are put in one of a number of [disjoint]
Homework: buckets (c.f., pigeon holes) based on some attribute, and repeatedly so for each bucket, until all items are
State and justify the variant and invariants for the linear search of a list of sorted elements. [Hint: derive in sorted order. e.g., buckets may be initial letters of names of people, and items are the names to sort.
your invariant and variants from those for the unsorted list. Also exploit the fact that the list is sorted.] Items in the same bucket (initial letter) are similarly partitioned based on their second letter, and
repeatedly so, until entries in each [sub]buckets are unique (sorted). For optimal performance, buckets
Binary Search are expected to hold approximately the same number of data items.
Given our invariant for linear search (i.e., item Elaboration of Insertion Sort Algorithm
sought is in the range of element searched) we Example scenario: Sorting array of elements Swap-based methods exchange pairs of unsorted items so they be in order. The most popular of these is
can significantly improve our algorithm if the 1 5 bubble sort. Bubble sort sorts arrays elements by scanning the array (i.e., reading each of its values
elements in the array are sorted. A sought item is 25 6 14 5 29 exactly once [and acting on it]) and swapping adjacent elements that are out of order. Scanning is
in one half of the array or the other, not both. The ↑
repeated until all the elements are in [sorted] order.
half in which it is found is retained (and the other j
[notionally] discarded). The retained half is again 6 25 14 5 29 There are many other classifications but in this class we shall consider insertion sort (O(n2)), selection
halved and the process repeated until the range ↑ sort (O(n2)) and merge sort (O(n lg n)). (Others are also mentioned, but not directly studied).
contains only the desired item. The item is not j
found if it is not in the range [where it should 6 14 25 5 29 Insertion Sort
be]. This is the basis of the binary search ↑
j In insertion sorted, we start with an unsorted array, scan through it, and insert the smallest element found
algorithm (also called the binary chop so far in the first position. We do similar for second, third, …, to the n-th element found so far, until the
algorithm). 5 6 14 25 29 whole array is sorted. (We assume here that sorting is in ascending order—though the particular order
↑
j does not matter—and the element found is inserted in its correct position.)
The invariant for this algorithm is: The item
sought is at a position before or after the Values (variables) used The general strategy is to keep sorted the elements found so far. More formally, using the notion of
midpoint of the range considered. j ≡ Boundary element: first unsorted element. invariants, the situation is as follows, on observing that elements in the array are in one of two parts.
(lo, hi, mid locate array positions in binary search).
Note that an item is found if located at the mid- k ≡ Used to shuffle items up; see actual algorithm. Invariant: All elements in the first part of the array are sorted. [Necessarily, all those in the second part
point. (Adding checks to the mid-point would N ≡ Total number of elements. need not be sorted.] Thus, an element from
only complicate our invariant.) The binary search A ≡ Array to sort. the unsorted part needs to be moved to its Elaboration of Merge Sort Algorithm
algorithm can be given as adjacent. temp ≡ temporary [variable] to hold an item. correct position in the sorted part. 25 6 14 5 29 16 42 15 9
(Necessarily, the sorted part increases by 1,
Homework Insertion sort Algorithm (for Agent CALCUL) and the unsorted part decreases by 1.)
25 6 14 5 29 16 42 15 9
1) Make sure you understand how the binary for j ← 2 up to N do
search algorithm works. temp ← A[j] (* element to move to sorted part *) An approach to include an unsorted element
2) Implement the above algorithm so that it is is to take first element of the unsorted part 25 6 14 5 29 16 42 15 9
lo ← 1; hi ← j-1
executable by Agent of CALCUL. while lo <= hi do and add it at its correct position in the sorted
3) Develop a recursive version of the binary mid ← (lo + hi)/2 (* integer div.; e.g., 7/2=3 *) part. Of course, the element to the right of 25 6 14 5 29 16 42 15 9
chop algorithm. [Hint: Exploit the invariant if A[mid] > temp then this position must be shuffled up (or nudged
used in the non-recursive version, or develop hi ← mid – 1 up) the array to create the needed space.
yours.] Initially the first element is deemed sorted. 25 6 14 5 29
else
4) For questions (2) and (3) above, identify a lo ← mid +1
suitable variant (for each) to prove that the end Using our example array, the scenario may 6 25 14 5 29 16 42 9 15
algorithm works (terminates). Focus only on od (* while *) as shown above, where j is the boundary
the iteration, or recursion, as the case may be. od (* for *) element. i.e., the first unsorted element. We
identify other values and hence elaborate the 6 14 25 5 29 9 15 16 42
(* Shuffle elements up, to create space. We move the algorithm [for agent CALCUL] as shown
COMMON ALGORITHMS: SORTING rightmost element first, into position j; and adjacent. 5 6 14 25 29
Sorting is another procedure very common and preceding elements next, until we are at lo (i.e., at
important to computer science. There are many the position we want freed). Why? *)
classes of sorting algorithms; here we discuss 5 6 9 14 15 16 25 29 42
for k ← j-1 down to lo do A[k+1] ← A[k] od;
Exercise: Insert lines to show splits/merges of sublists.
A[lo] ← temp; (* insert moved item at correct
How does the algorithm behave with duplicate items?
position *).

wss introduction to algorithms—notes of 08 October 2018 page 41 of 107 wss introduction to algorithms—notes of 08 October 2018 page 42 of 107
Home work
Q1) Extract, in pseudocode, the algorithm (program) implemented for agent (CALCUL), mindful that POINTERS, REFERENCES AND DYNAMIC DATA STRUCTURES
the following activities are among those involved. We considered basic data structures as records and arrays which are found in almost all programming
1) Shuffling elements to create space. languages. Here we consider intuitively other data structures (lists, stacks, queues, trees and graphs)
2) Changing the frontier of sorted elements. which are often more conveniently handled as abstract data types (ADT’s) and implemented as dynamic
3) Moving item into sorted array. data structures.

In other words: Dynamic Data Structures

• Produce a high level algorithm [in pseudocode] that implements the insertion sort algorithm, Dynamic data structures are data structures whose size changes within a program and which may persist
using the abstract actions above, amongst others. beyond the program scope where it is created. This contrasts with static data structures whose sizes are
• Match each abstract action you eventually use to a segment of the program. known at compile time and persist only within the scope in which they are defined. Dynamic data
• Informally, show that insertion sort is correct and terminates. structures are typically implemented using pointers (in languages such as Pascal) or references (in
• Can we have a better (more efficient) algorithm if we strengthen our invariant so that it requires languages such as ML or Java). A pointer is the address [in memory] of a location; it is used to access the
that each sorted element is in its final sort position? That is, when we add an item to the sorted location or its value. The location is an anonymous variable (because it has no name) and is sometimes
part of the array, we do not move it again. called a reference variable (since it is referred to, i.e. associated to some object; c.f., binding earlier). A
pointer variable holds a pointer value (i.e., a memory address used to access a value) which is
Justify your answer. [Note: For this question, your reasoning is more important than the correctness of necessarily of a pointer type. A special pointer value called the nil pointer has no matching memory
your answer. e.g. your premises (and hence conclusions) may be wrong, but the inference rules (rules for address. It indicates that the pointer variable holding it has no valid value; else all bit patterns are valid
reasoning) should be used correctly.] memory addresses!
[Dynamic] Singly-linked List
Dynamic Data Type value link pointer
Q2) Suppose we strengthen the invariant for the selection sort to: All elements in the first part of the l field field nil ptr
array are sorted to their final position in the array. [Necessarily, all those in the second part need not be A dynamic data type is typically created from a record data structure
sorted.] which contains pointers to other data structures of the same type.
Fields (value fields), also called cells (value cells) of the records hold 25 16 80
1) Inspired by the insertion sort algorithm, develop a sorting algorithm based on this invariant. [Hint:
An element in its final position (in the array) will not be moved again, when other elements are the values we want to store in the dynamic data structure, while the
subsequently found. Thus, for example, the first element must be the smallest.] link fields (also called pointer fields, i.e., fields holding pointer values), define the structure of the
2) Find a suitable variant to prove termination. (You can alternatively do so after doing (3) below.) dynamic data structure. For example, a list l is made up of a node with a value field and a pointer to the
3) Implement your algorithm for agent CALCUL. node holding the next value of the list (the link field). In this case, the variable l that is represent the list
4) Informally prove that your algorithm is correct, and terminates. is actually a pointer variable.

Merge Sort and Merging Note that a value field may hold a pointer value but that field is not part of the structure of the dynamic
See separate handout for the merge sort algorithm. data structure. Conversely, a link field may also be considered as a value field for the problem being
solved; it however remains a part of the
Types of Variables (linked to Memory Layout)
Homework: dynamic data structure. The example list is
called a singly-linked list, for there is only one Description label Loca- Ptr Mem
Q1) Also read about the merge and sort algorithms in Brookshear section 12.5 (or 11.5 in 6th ed.). Limit tions links Addr
emphasis to understanding the algorithm, not its [computational] complexity, through we eventually set of links between nodes.
allude to this complexity. Pointer variable p p 1024 1000
Dynamic Variables
1004
Homework: Dynamic variables are allocated from the part
Q1) Find out about the uses of stacks and queues. of the memory [set aside for a program to use] … … … …
Q2) What are: (a) abstract data type, (b) dynamic data type, and (c) static data type. Explain how they called a heap. That way, dynamic variables can Anonymous variable 60 1024
may be used or implemented. be allocated and disposed (returned to the heap)
in any order, at any time, during a program run. Static variable, x x 40 1028
This contrasts with static variables that are Unused location 1032
allocated from a stack (see stacks later) and so
Anonymous pointer var 1004 1036
must be returned in the reverse order of
allocation. r is reference var for 25 r 25 1040

References
References are labels (names) associated to the objects of interest (objects that we want to manipulate) as
opposed to the location where the object is (as in pointer). Thus in the diagram above the name r refers to
the value 25 (the object of interest) and not the location (which is 1040). References are more convenient
because they avoid having the user manage memory and hence associated problems such as dangling

wss introduction to algorithms—notes of 08 October 2018 page 43 of 107 wss introduction to algorithms—notes of 08 October 2018 page 44 of 107
references (accessing a location already returned to memory, but whose address is still in the pointer
variable used to access it) and garbage (which is allocated memory that has no pointer to it from within
the program). Also, the binding of references is static (cannot be changed), unlike values in locations. FURTHER DATA STRUCTURES
Given our understanding of static and dynamic data structures, we can better explore data structures
Exercises further. Note that the same data structure may be variously classified—dynamic, recursive, ADT
Q1) Given the definition of variables, assignments and pointers as discussed above or earlier, what are (abstract data type)—based on certain properties in its definition and/or implementation. Thus, here, we
the results of following for the memory layout above? define a data structure as a “systematic way of organising and accessing data.” (William Ford and
(a) p ← 1024 William Topp, “Data Structures with C++ using STL”, 2nd ed, Prentice Hall, 2002.)
p^ + x – r [Hint: result here ≡ 75.]
(b) p ← 1036 Abstract Data Structures
(p^)^ ← x – r * r (This section is for completeness with respect to the notion of data abstraction, and for contrast to the
notion of abstract data types; it is not explicitly required for CSC 207, though the underlying thought
Notation: The caret (^) is used to denote a reference of a pointer. Thus, if q is pointer, then q^ is the processes are significant to algorithm development.)
location pointed to by q. More technically, ^ in q^ dereferences q; it returns the location (where locations Abstract data structures are data structures whose structures (relationship among components) and
are expected, as in the left hand side of assignment statements), and the value associated with the construction are not of particular interest, and are defined only in terms of the key operations [that define
location in expressions. Reference variables are dereferenced as needed, and the values associated to their purpose or functionality]. More technically, the key operations define their semantics (i.e., their
them returned. behaviour or what to expect).

Q2) (a) Give a diagrammatic representation of an unordered singly-linked list of 5 positive integers. A very common abstract data structure in Computer Science is the dictionary, which is just a “structure
(b) Determine any two adjacent numbers that are out of order. Then, using dashed or differently that stores objects, identified by keys, and supports the operations find, insert, and delete” (Brass 2008).
coloured lines, show how you can reverse their nodes. Number the lines [and any other needed The find operation locates an item in the dictionary, while insert and delete operations respectively
operations indicated] in the order in which they must be carried out. State any assumptions made. insert and remove items from it. Keys uniquely identify each item (See subsections “Hash Tables”,
“Relational Keys”) and are used to access the dictionary objects.
Q3) (a) Give a diagrammatic representation of a linear search tree containing integer numbers. In it,
the numbers in any left subtree are strictly less than the number in the root, and those in any right subtree A data structure is abstract in the sense that it is independent of [any particular] representation (i.e.,
are strictly more than that of the root. This type of tree is called a binary search tree (BST). Your BST possible implementations), memory layout, or their properties; it captures the essence of the data
has exactly three levels. structure—what it means and how it should behave. Hence it has to be implemented in terms of more
(b) is your binary tree balanced? That is, are all internal node positions filled? specific structures (other data structures) for it to be functionally useful. Also, additional operations, such
as those to create or delete the abstract data structure, or list its content, need not be explicitly defined,
A Dynamic Linked-list Example unlike in abstract data types. Notice, for instance, that nothing is said about creation or listing operations
We illustrate the construction of dynamic data structures using a linked list. A list is constructed of dictionaries, say.
(implemented) by providing cells for list elements, with associated pointer cells. A pointer cell holds a
[single] pointer at any time. (Recall a pointer is the address of a memory location, and can be used to An abstract data structure invariably defines an underlying abstract structure (independent of its
access that location.) Thus, a linked list—strictly, a singly-linked list—is implemented by considering the algorithm design or implementation) which is a structure that is imposed on a set of data items, and
following (Also see diagram as adjacent): hence used to organise desired operations on them (i.e., the data items). It is abstract in the sense just
• Each node (a value-pointer pair) is implemented using a record data structure. In the example, the explained above. However, such an abstract structure is typically a data structure with well studied
record has field names val and next. properties—such as a binary tree, a priority queue, or a hash table—whose salient operations assist, say,
• The address of the first node of the list is stored in a pointer variable (l in the example), which is in algorithm development or problem solving. It could also be one “conjured” by the programmer for
referred to as the list [instance]. Loosely, a list, and more generally any dynamic data structure, is very specific tasks.
denoted by the variable through which it is accessed. Thus, if l was instead a static variable with
record fields val and next (as for list nodes), and the pointer to the first list node is stored in its Common abstract structures include:
next field, l would still denote the list. • Dictionary: see explanation above.
• Accessing the next element in a list is done by taking the pointer value from the current node and • Lists, stacks, queues, trees, etc.: These “conventional” data structures are abstract when we
using it as the address for the node of the next element. consider their structures independent of their implementation. They can be further defined (i.e.,
• Let cur be a pointer [variable] to for the current node, the Linked-list construction/access refined) by their corresponding abstract data types. In this context, a “data structure” is an actual
node currently examined on the list. Then (assuming notation implementation and so must reckon with the layout for the data content held, and how to access,
adopted for records and pointers): update, create/destroy them.
o cur^.val is the value of current node.
o cur^.next is a pointer (link) to next [list] element. l cur Note:
o cur^.next^.val is the value of next [list] element. 1) Often, it is not clear whether we are referring to a given abstract structure or the method to realise the
o cur^.next^.next is the next element of the next [list]. 25 16 80 abstract structure. For example (Brass 2008), the term heap is seen by many as the abstract structure
element. In the example diagram this is its value? called a “priority queue,” but others see it as the first implementation of heaps in terms of an array
What is the value for l^.next^.next^.val? data structures.

wss introduction to algorithms—notes of 08 October 2018 page 45 of 107 wss introduction to algorithms—notes of 08 October 2018 page 46 of 107
2) The notion of abstract, in relation to data structure, may vary with programming languages. e.g., in
Java, its abstract data structures are independent of representation in the sense of being able to hold A recursive definition of a tree captures the above
different types of data, and need to be instantiated (defined to something more specific; see also the more succinctly: A binary tree is an empty binary tree Binary Tree (structure and some terminology)
subsection “Variables” under “Algorithms, Programs and Programming Languages”). Also, hash or a node with (pointing to) two other binary trees.
tables are concrete data structures. In C, the closest to this notion of abstraction is in the use of the (The two trees are called its left and right subtrees).
generic pointer type (void *), which can point to data structures of any type. Thus, concepts such as
linked lists and hash tables remain abstract in C, until they are instantiated. Note that the many An example tree is represented diagrammatically as
notions of “abstract” is too much of a digression off digressions, and must not be considered further. above. The terminology of genealogical trees is often
3) In algorithm development, an abstract data structure may sometimes include constraints due to used to describe trees and, unlike natural trees, tree
concerns for memory access, such as efficiency. e.g., should data reside in any nodes of a tree or data structures grow up-side-down! (Well, mostly
reside only at the leaves. But these constraints are seen as part of the abstract structure, not the upside down!) Notice that for any given node, its left
memory layout, say. Thus, conceptually, abstract structures may implement even more abstract ones. and right children are the root nodes for trees
4) The terms “abstract data structures” and “abstract structures” are often treated as synonyms, as is the obtained by removing the node. Such trees are called
case in this document. An “abstract data structure” may be seen as an “abstract structure” over data, subtrees. The binary tree has at most two subtrees,
with the latter applicable to things which need not be data (program instructions, concepts, etc.). The the left and right subtrees.
former expects to use specific data values, without concerns over their layout and access in memory.
An abstract data type, in contrast, comprehensively describes such sets of data values and operations Special operations must be carried out in order to use and update (insert, delete or modify) a binary tree.
on them. Informally, “abstract data type” and “data structure” are used interchangeably. How a tree is later used depends on the order in which its nodes are examined (or visited). Such visits,
when performed on all nodes of a tree, is called a [tree] traversal. Trees (and their traversals) are a
Recursively-defined Data Structures subject for further studies beyond this course. However, note that trees need not be binary; they can be
Some data structures can be recursively defined, just as algorithms (PL procedures) are recursively ternary, say, having at most 3 siblings, or n-ary with at most n siblings for some fixed n.
defined. That way, provably correct, problem-specific data structures can easily be constructed, typically
as dynamic data structures. Recursive data structures are data structures that are defined in terms of Table Data Structures
themselves—c.f., recurrence relations in mathematics—and other, base data structures. Base data We consider hash tables and associative arrays as their concepts are needed later, notably to explain file
structures are already-defined data structures (and atomic types) used, and so are not being defined at the and database concepts.
same time as the data structure being defined. c.f., base case in recursive algorithms. (Thus, a base data Hash Tables: (© Cormen 2009)
structure could be recursive.) Hash Tables
List [2, 4,3,25] as Hash tables are like arrays in that one uses
Lists element-list sequence: indices to access the values it holds. More
A list is a recursive data structure: it is either an empty list or an element elt list technically, in hash tables, the index values are
(i.e., a list element) followed by a list. The first value of a list is called its called keys, for they uniquely identify the values
head; the rest of the values is called its tail. elt list they are associated with. However, unlike arrays
where each index value (key) [of interest] is in a
Lists (and recursively defined data structures in general) are accessed with elt list contiguous range and has a matching cell
recursive or iterative algorithmic constructs. Thus, the list given as the [location] where its value is stored, hash tables are used in situations where one has a lot more possible
sequence 2, 4,3,25 may be given recursively as (2, (4, (3, (25, ◊)))) where ◊ elt list index values than cell locations.
is the empty list and parentheses group sublists. To see why, match the
elements in the list to elt in the diagram adjacent, from bottom to top. Thus, one has to map the actual keys of interest—given by the subset K of all possible keys U in the
diagram above—to specific cells in a 1-dimensional array. For instance, keys k1 and key k5 are mapped
Algorithmically, we may give a list of length n as: list(n) ≡ EITHER Empty list (if n=0) OR (Element, to the second and sixth cell positions respectively. (In a 1-D array, these keys would correspond to array
list(n-1)), where n is a non-negative integer and list(i) is a function that returns a list of length i. positions 1 and 6 respectively, assuming indexing from 1.) Unfortunately, two or more keys, e.g., k1 and
k4, may map to the same cell position, called a slot, giving rise to a collision. In the example, collision is
A list is used by starting from its first element (its head) and examining (visiting) successive elements handled by storing keys to the same slot in a doubly-linked list; a specified key is later obtained by
until the desired element is found. One either starts from the beginning of the list, or proceed from the searching the slot's list. (Other techniques of handling collision are beyond the scope of this course.)
last position of the list. When all the nodes of a list are visited exactly once, we say we walked the list.
A hash function is used to compute the array cell position (slot) from each key, and the collection of slots
Binary trees constitute a hash table. We say that a key (or the element associated with it) hashes to the slot, and the
A binary tree is a set of nodes (which hold data values) that are arranged so that each node points to at value used to index the slot is called its hash key.
most two other distinct nodes not already found in the tree. As a result, each node is pointed to by at
most one other node, except a single root node that does not have any other node pointing to it. (The Associative Arrays
empty tree, with no node, is a special binary tree.) Also, each node points to one or two other distinct Associative arrays are like arrays, except that arbitrary data types—such as strings, real numbers and
nodes, except leaf nodes, which are nodes that do not point to any other nodes. Furthermore, no two even values of distinct types—are used to index the array. (An array expects ordinal types, types whose
nodes can point to the same node, nor can two nodes point to each other, even via a chain of other nodes. values are successive, discrete and can be ordered, to access it. Conceptually, their implementation is like

wss introduction to algorithms—notes of 08 October 2018 page 47 of 107 wss introduction to algorithms—notes of 08 October 2018 page 48 of 107
for hash tables in that these arbitrary values are mapped to specific array index values, typically by the Queues
hashing technique, which are then used to access the associative array entry. A queue holds elements that are removed strictly in the same order in which they are inserted; the first
element in it is the first element out (FIFO). Unlike in a stack, the first element in is the last element out.
Data Structures and their ADT Implementations A queue has a head (or front) where elements are dequeued or served (i.e., removed), and a rear where
Here, we mention other data structures typically implemented as dynamic data structures, but also elements are enqueued or inserted. As with stacks, a queue can be empty or full. As for stacks under the
illustrate their implementation, as ADTs, in terms of array data structures. These include linked lists, section “Abstract Data Types”, front and rear can be variously defined.
stacks, queues, deques, and graphs.
In an array implementation of a queue, variable head indexes the first element (or entry) of the array and
Linked Lists List: Static vs Dynamic another variable, rear, always indexes the last element inserted into the array. A queue is empty when it
Lists were introduced under dynamic data structures, and again as a implementations has no element. It is full if there is no space to insert
recursive data structure. In non-recursive terms, a list is a data index 1 5 elements at the rear. Space created by removing elements Queues
structure that holds a sequence of values that are accessed in the order from the front of the array is reclaimed by, say,
in which they are found in the list. So in our example list [2, 25, 15, array 25 16 80 repositioning the [consecutive] elements of the queue to
40, 8], we must access 2 before 25, 25 before 15 and 15 before 40, and l the beginning of the array whenever the space at the rear is
so on. If we must access 15 again, we would typically start from 2 and exhausted.
25 16 80
access 25 (again) before accessing 15.
Alternatively, a queue may be implemented using a linked
A list may be implemented (stored) in an array, instead of as a linked list. In which case, the array list, as shown in the diagram above.
elements can only be accessed as one would access a list: adjacent elements are successive elements in
the array, the list head is the first element of the array and the last list element in the last entry of the Deques
array. Inserting a list element at a given array position entails shifting array elements right, to create A deque (pronounced dek) is a queue (or a stack) that can
space for the new element; and deleting an element at a given position entails shifting elements [to the be dequeued (or popped) form both ends: one can use it as
right of the element to delete] left, overriding it. a queue or a stack at the same time! This is useful in
situations where the same set of data may be treated at
Doubly-linked Lists some stage [of its processing] as a stack, and at another as
As indicated earlier, a singly-linked list has only one link between adjacent elements, and hence only one a queue.
way to access the elements. Doubly-linked lists are like singly linked ones except that is has
corresponding forward and backward links between nodes. Thus, for each arrow in list l above, such as Graphs
from node 25 to node 16, there is a matching backward arrow, as is the case from node 16 to node 25. In a graph data structure, components, are represented as vertices (or nodes) that contain information; the
relationship between the nodes are represented as arcs or edges (Arcs can also contain information.)
Exercise: Represent the above list as a dynamic data Stacks Unlike all other data structures considered so far, the relationship among nodes is arbitrary, without
structure, using the diagrammatic notation introduced constraints. e.g., lists have at most one successor and binary trees have at most 2 children. Normally, no
earlier. On a separate piece of paper, insert the backward node of a graph has an arc to itself.
links in l.
Graphs are one of the most general data
Doubly-linked lists are useful to access lists in the structures and can be used to represent Graphs: Representation and Terminology
forward and backward directions, without having to many things. They have many
always start from the head of the list. e.g., one can thus properties that can be exploited to solve
use the forward links from 16 to 80 and then the backward problems. Here we state a few of them,
ones back to 25. as well as some commonly used
terminology.
Stacks
As mentioned before, a stack holds elements that are Adjacent is a graph for some adjacent
removed strictly in the reverse order in which they were provinces in Cameroon (given by their
inserted. A stack has a top, where elements are pushed (inserted) or, popped (removed), and a bottom abbreviations). The following
where the first element is placed. A stack is empty when it has no elements, and full when it has no more terminology are illustrated using it:
space to accept an element. Array and linked list implementations of a stack are shown adjacent. • A path from node NW to node CE is (NW, OU, CE).
• The above graph is cyclic since there is at least one path from some node to itself [via other
Stacks may be implemented using an array as explained earlier (under the section “Abstract Data nodes]. If it was not the case, it would have been called an acyclic graph.
Types”). Alternatively, it can be implemented using a list data structure where the list head is the top • A subgraph of a graph contains a subset of its nodes and edges. The subgraph of Graph 1 that
element. The stack is empty if the list has no element; i.e., the list has only a nil pointer. The stack is full excludes NW, and the edge to NW, is strongly connected since any two nodes are neighbours.
when no more space can be allocated to it from available memory. • A connected graph has a path between any two pairs of nodes. Otherwise, the graph is
unconnected. Graph 1 is connected; Graph2 is not connected.

wss introduction to algorithms—notes of 08 October 2018 page 49 of 107 wss introduction to algorithms—notes of 08 October 2018 page 50 of 107
 • A directed graph, or digraph, has each arc associated with a direction, as indicated by its arrow
head. Thus, Graph 2 is directed (or oriented); paths exist only in the direction of the arrows.
• The fan-out of a node in a directed graph is the number of edges emanating from the node (or
equivalently, the number of other nodes it points to); its fan-in is the number of edges pointing to
the node. The fan-out and fan-in of a graph are respectively the maximum fan-out and fan-in that
a (any) node of the may graph have.
• In a weighted graph, each edge has an associated value.
• A tree is a graph that is connected and has no cycles. (Notice that a binary tree, as discussed
earlier, is oriented.)
• A spanning tree is a tree obtained from a connected graph by removing 0, 1 or more edges, until
all cycles are eliminated.

wss introduction to algorithms—notes of 08 October 2018 page 51 of 107