Data Structures and Algorithms

Instructors:

Data Structures and Algorithms

About the course (objectives, outline, recommended reading) Problem solving Notions of Algorithmics (growth of functions, efficiency, programming model, example analysis) Lists, stacks, queues.

Marius Joldo (lectures + lab. sessions),

Room D3, 71-73 Dorobanilor St. Tel. 0264-401276 (lab.) [email protected], [email protected]

Anca Ciurte, Andrei Vtavu (lab. sessions)

Course web site:

http://users.utcluj.ro/~jim/DSA Lectures: Friday 12-14, room P03, 26-28 G.Bariiu St. Labs: room D3, 71-73 Dorobanilor St. every week : 30413-4: Wednesday, 30411-2: Thursday, as scheduled

Schedule: as set in the timetable

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

Objectives
To get familiar with various ways to characterize properties of algorithms and data structures. To learn to compare and algorithms and data structures. To get familiar with standard algorithms and data structures so that you can find appropriate algorithms and data structures when they are available. To learn to design your own algorithms and data structures when no appropriate can be found in the literature and libraries.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 3

Evaluation
Activity during the semester 40%

Assigned programming tasks Activity during laboratory sessions

Present your work in class Criticize (constructively) the work learn how to evaluate work Deadlines MUST be met

Written exams

Midterm, in class 30%

After lecture 6?
+ problem)

of written

Final, during summer session 70% of written (theory

There may be bonuses ( <= 1 point)
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 4

Recommended books
Aho, Hopcroft, Ullman. Data Structures and Algorithms, Addison-Wesley, 427 pages, 1987.

Recommended books
Preiss. Data Structures and Algorithms with objectOriented Design Patterns in C++, John Wiley and Sons, 660 pages, 1999.

The classic book on the topic, still most valuable introductory book, though it misses a few topics. Uses extended Pascal.

Cormen, Leiserson, Rivest, (Stein): Introduction to Algorithms. MIT Press / McGraw Hill, (2nd edition),1028 pages, 1990 (2002).

An unusual algorithms and data structures book in that it stresses from the beginning the role of design patterns for the implementation of data structures as classes. An excellent source for good object-oriented design principles, with an accessible introduction to complexity analysis. Available online (see course web site)

Written for all levels, with introductory chapters in discrete math and offering specialized topics in later chapters. Uses pseudocode, which makes the presentation compact, easy to follow and timeless. Also useful as a reference book.

Knuth. The Art of Computer Programing, Addison Wesley, three volumes, reprinted several times. A classical book, extremely well written and documented by a Master of the field. Unfortunately uses the language of the hypothetical machine MIX for describing the algorithms which makes understanding more difficult.
5 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 6

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

Outline (preliminary)
Introduction: algorithmics and asymptotics. Lists, stacks and queues Trees, binary trees, binary search trees Basic ADTs for sets: dictionary, hash tables, mappings, priority queues Advanced ADTs for sets: 23 trees, AVL trees, unionfiind sets Directed graph ADTs: single source shortest path, all pairs shortest path, traversals Undirected graphs: minimum cost spanning trees (MST), MST algorithms, traversals, graph matching Algorithm analysis and design techniques: divide and conquer, dynamic programming, greedy algorithms, backtracking, local search algorithms Sorting Data Structures and algorithms for external storage. B-trees.
7

Problem Solving

First construct an exact model in terms of which we can express allowed solutions.

Finding such a model is already half the solution.

Any branch of mathematics or science can be called into service to help model the problem domain.

Once we have a suitable mathematical model, we can specify a solution in terms of that model.

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

Representation of real world objects

Issues when dealing with the representation of real world objects in a computer program how real world objects are modeled as mathematical entities, the set of operations that we define over these mathematical entities, how these entities are stored in a computer's memory (e.g. how they are aggregated as fields in records and how these records are arranged in memory, perhaps as arrays or as linked structures), and the algorithms that are used to perform these operations.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 9

Basic Notions
Model of Computation: An abstract sequential computer, called a Random Access Machine or RAM. Uniform cost model. Computational Problem: A specification in general terms of inputs and outputs and the desired input/output relationship. Problem Instance: A particular collection of inputs for a given problem. Algorithm: A method of solving a problem which can be implemented on a computer. Usually there are many algorithms for a given problem. Program: Particular implementation of some algorithm.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 10

Example of computational problem: sorting

Algorithm Design Algorithm

Statement of problem:

Input: A sequence of n numbers <a1, a2, , an> Output: A reordering of the input sequence <a1,
a2, , an> so that ai aj whenever i < j

Instance: The sequence <7, 5, 4, 10, 5> Algorithms:

Selection sort Insertion sort Merge sort (many others)

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 11 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 12

Analysis of algorithms
Determine the amount of some resource required by an algorithm (usually depending on the size of the input). The resource might be:

Growth of functions

running time memory usage (space) number of accesses to secondary storage number of basic arithmetic operations network traffic

Formally, we define the running time of an algorithm on a particular input instance to be the number of computation steps performed by the algorithm on this instance.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 13

Typically, problems become computationally intensive as the input size grows. We look at input sizes large enough to make only the order of the growth of the running time relevant for the analysis and comparison of algorithms. Hence we are studying the asymptotic efficiency of algorithms.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 14

A General Portable Performance Metric

Big-oh

Informally, time to solve a problem of size, n,

T(n) is O(log n) if T(n) = c log2n

Formally: O( g(n) ) is the set of functions, f, such that f(n) < c g(n)
for some constant, c > 0, and n > N (i.e. sufficiently large N) f(n) Alternatively, we may write lim c
n

O(g) is the set of functions that grow no faster than g.

g(n) describes the worst case behavior of an algorithm that is

O(g)
Examples: n lg n + n = O(n2) lgk n = O(n) for all k N
15 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 16

g(n)

and say

g is an upper bound for f

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

Big-omega
(g) is the set of
functions, f, such that f(n) > c g(n) for some constant, c, and n > N (g(n)): class of functions f(n) that grow at least as fast as g(n) g(n) describes the best case behavior of an algorithm that is (g)

Big-theta
(g(n)): class of functions f(n) that grow at same rate as g(n) Example: n2 / 2 - 3 n = (n2) We have to determine c1 > 0, c2 > 0, n0 N such that: c2 n2 n2 / 2 - 3 n c1 n2 for any n > n0 Example: Dividing by n2 yields: c2 1 / 2 3 / n c1 This is satisfied for c2 = 1 / 14, c1 = 1 / 2, n0 = 7.
17 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 18

Example: a n2 + b n + c = (n) provided a > 0

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

Basic Asymptotic Efficiency classes

1 log n n n log n n2 n3 2n n! constant logarithmic linear n log n quadratic cubic exponential factorial
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 19

A Simple Programming Model

Prog = statement; Prog': Time(statement) + Time (Prog') Prog = var = expr: 1 + Time(expr) Prog = expr1 op expr2: 1 + Time(expr1) + Time(expr2) for primitive operations Prog = array[expr]'': 1 + Time(expr) Prog = for i = 1 to n, do Prog: n times Time(Prog') Prog = function call: Time(function) Prog = while(expr), do Prog' : Time(expr) + Time(Prog') times
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 20

Theoretical analysis of time efficiency

Time efficiency of nonrecursive algorithms

Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

Steps in mathematical analysis of nonrecursive algorithms:

Basic operation: the operation that contributes

most towards the running time of the algorithm.
input size

Decide on parameter n indicating input size Identify algorithms basic operation Determine worst, average, and best case for input

T(n) copC(n)
running time execution time for basic operation Number of times basic operation is executed
21

of size n Set up summation for C(n) reflecting algorithms loop structure Simplify summation using standard formulas

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

22

Example (nonrecursive)

Time efficiency of recursive algorithms

Other issues regarding this algorithm implementation

Steps in mathematical analysis of recursive algorithms: Decide on parameter n indicating input size Identify algorithms basic operation Determine worst, average, and best case for input of size n Set up a recurrence relation and initial condition(s) for C(n) the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls). Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

23

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

24

Example (recursive)

Example (recursive) Analysis

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

25

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

26

Example (recursive) Analysis

Stacks, queues, linked lists

Stacks

A stack is a container of StackElements that are

inserted and removed according to the last-in-firstout (LIFO) principle. StackElements can be inserted at any time, but only the last (the most-recently inserted) StackElement can be removed. Inserting an item is known as pushing onto the stack. Popping off the stack is synonymous with removing an item.

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

27

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

28

The Stack Abstract Data Type

A stack is an abstract data type (ADT) that supports two main operations:

The Stack Abstract Data Type

push(x): Inserts StackElement x onto top of stack

Input: StackElement; Output: none

pop(): Removes the top StackElement of stack and returns it; if stack is empty an error occurs
Input: none; Output: StackElement

The following support operations should also be defined:

size(): Returns the number of StackElements in stack
Input: none; Output: integer Input: none; Output: boolean

isEmpty(): Return a boolean indicating if stack is empty. top(): return the top StackElement of the stack, without removing it; if the stack is empty an error occurs.
Input: none; Output: StackElement

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

29

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

30

Stacks - Relevance

Queues

Stacks appear in computer programs

Key to call / return in functions & procedures Stack frame allows recursive calls Call: push stack frame Return: pop stack frame
Stack frame

Function arguments Return address Local variables

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 31

A queue differs from a stack in that its insertion and removal routines follows the first-in-first-out (FIFO) principle. Elements may be inserted at any time, but only the element which has been in the queue the longest may be removed. Elements are inserted at the rear (enqueued) and removed from the front (dequeued)
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 32

The Queue Abstract Data Type

The queue supports two fundamental operations:

The Queue Abstract Data Type

Operations given for a circular queue

enqueue(o): Insert QueueElement o at the rear of the queue

Input: QueueElement; Output: none

dequeue(): Remove the QueueElement from the front of the queue and return it; an error occurs if the queue is empty

Input: none; Output: QueueElement Input: none; Output: integer

These support methods should also be defined:

size(): Return the number of QueueElements in the queue isEmpty(): Return a boolean value that indicates whether the queue is empty

Input: none; Output: boolean

front(): Return, but do not remove, the front QueueElement in the queue; an error occurs if the queue is empty

Input: none; Output: QueueElement

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

33

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

34

List example (doubly linked)

An empty list contains no elements Insertions occur at the front of the list Deletions done by key (e.g. deletion of element of key 18)

The List Abstract Data Type

The list supports three fundamental operations:

insert(x): Insert ListElement x at the front of the list

Input: ListElement; Output: none

delete(x): Remove the ListElement from the front of the list; an error occurs if the list is empty

Input: pointer to element to delete; Output: none

Input: key ot search; Output: pointer to ListElement or nil if not found

search(k): search for ListElement of key k on list

These support methods should also be defined:

size(): Return the number of ListElements in the lits

Input: none; Output: integer Input: none; Output: boolean

isEmpty(): Return a boolean value that indicates whether the list is empty

first(): Return, but do not remove, the first ListElement in the list; an error occurs if the queue is empty

Input: none; Output: ListElement

last(): Return, but do not remove, the last ListElement in the list; an error occurs if the queue is empty
Input: none; Output: ListElement

prev(x), next(x): ListElement preceeding/succeeding ListElement x on list

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 36

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

35

The List ADT. Operations

The List ADT. Operations

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

37

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

38

List examples with sentinels

An empty list always contains the sentinel (grey cell) Insertion is done at the front of the list List with elements of keys 6,18, and 11 inserted After insertion of 27 After deletion of 18

List ADT (w/ sentinels)

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

39

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos

40

Reading

AHU, chapters 1 & 2 CLR, chapters 2, 11.1-11.3 Preiss, chapters: Algorithm Analysis. Asymptotic Notation. Foundational Data Structures. Data Types and Abstraction. Stacks, Queues and Dequeues-4ups. Ordered Lists and Sorted Lists. Knuth, vol. 1, 2.1, 2.2 Notes
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 41