Scribd
Upload
45 views

Intro Algo

This document introduces algorithms and algorithm analysis. It defines an algorithm as a set of instructions to solve a problem, taking inputs and producing outputs. It discusses that algorithms can be specified in various ways like English, as a computer program, or pseudo-code. Algorithm analysis is important to determine an algorithm's efficiency and whether it will scale to large input sizes. Common metrics for analysis include best-case, worst-case, and average-case running times, which are often expressed using asymptotic notations like Big-O, Omega, and Theta.

Uploaded by

Ravi Stephen
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
45 views

Intro Algo

This document introduces algorithms and algorithm analysis. It defines an algorithm as a set of instructions to solve a problem, taking inputs and producing outputs. It discusses that algorithms can be specified in various ways like English, as a computer program, or pseudo-code. Algorithm analysis is important to determine an algorithm's efficiency and whether it will scale to large input sizes. Common metrics for analysis include best-case, worst-case, and average-case running times, which are often expressed using asymptotic notations like Big-O, Omega, and Theta.

Uploaded by

Ravi Stephen
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 22

Algorithms

Introduction

What is an Algorithm?

a clearly specified set of simple instructions to be followed to


solve a problem
Takes a set of values, as input and
produces a value, or set of values, as output

May be specified
In English
As a computer program
As a pseudo-code

Data structures

Methods of organizing data

Program = algorithms + data structures

Introduction

What are the criteria that an algorithm should satisfy?

Input - Zero or more inputs


Output - One or more outputs
Definiteness Clear and unambiguous
Finiteness Terminates after a finite number of steps
Effectiveness Each instruction is so basic and easy to
analyze

Introduction

Why we need algorithm analysis ?

Writing a working program is not good enough


The program may be inefficient!
If the program is run on a large data set, then the
running time becomes an issue

Example: Selection Problem


Given a list of N numbers, determine the kth
largest, where k N.
Algorithm 1:

(1) Read N numbers into an array


(2) Sort the array in decreasing order by some
simple algorithm
(3) Return the element in position k

Example: Selection Problem

Algorithm 2:
(1) Read the first k elements into an array and sort
them in decreasing order
(2) Each remaining element is read one by one

If smaller than the kth element, then it is ignored


Otherwise, it is placed in its correct spot in the array,
bumping one element out of the array.

(3) The element in the kth position is returned as


the answer.

Example: Selection Problem

Which algorithm is better when

N =100 and k = 100?


N =100 and k = 1?

What happens when N = 10,00,000 and k =


5,00,000?
There exist better algorithms

Algorithm Analysis

We only analyze correct algorithms


An algorithm is correct

Incorrect algorithms

If, for every input instance, it halts with the correct output
Might not halt at all on some input instances
Might halt with other than the desired answer

Analyzing an algorithm

Predicting the resources that the algorithm requires


Resources include
Memory
Computational time (usually most important)

Algorithm Analysis

Factors affecting the running time

computer
compiler
algorithm used
input to the algorithm
The content of the input affects the running time
typically, the input size (number of items in the input) is the main
consideration
E.g. sorting problem the number of items to be sorted
E.g. multiply two matrices together the total number of
elements in the two matrices

Worst- / average- / best-case

Worst-case running time of an algorithm


The longest running time for any input of size n
An upper bound on the running time for any input
guarantee that the algorithm will never take longer
Example: Sort a set of numbers in increasing order; and the
data is in decreasing order

Best-case running time

sort a set of numbers in increasing order; and the data is


already in increasing order

Average-case running time

Asymptotic notations
Big-oh(O)
Omega()
Theta()
Little-oh(o)

Asymptotic notation: Big-Oh


f(N) = O(g(N))
There are positive constants c and n 0 such
that
f(N) c g(N) when N n0

The growth rate of f(N) is less than or equal to


the growth rate of g(N)
g(N) is an upper bound on f(N)

Growth Rate

The idea is to establish a relative order among functions for large


n
c , n0 > 0 such that f(N) c g(N) when N n0
f(N) grows no faster than g(N) for large N

Big-Oh: example

Let f(N) = 2N2. Then

f(N) = O(N4)
f(N) = O(N3)
f(N) = O(N2) (best answer, asymptotically tight)

O(N2): reads order N-squared or Big-Oh N-squared

Some rules

When considering the growth rate of a function using


Big-Oh
Ignore the lower order terms and the coefficients of
the highest-order term
No need to specify the base of logarithm

Changing the base from one constant to another changes the


value of the logarithm by only a constant factor

If T1(N) = O(f(N) and T2(N) = O(g(N)), then

T1(N) + T2(N) = max(O(f(N)), O(g(N))),

T1(N) * T2(N) = O(f(N) * g(N))

Big-Omega
f(N) = (g(N))
There are positive constants c and n 0 such
that
f(N) c g(N) when N n0

The growth rate of f(N) is greater than or


equal to the growth rate of g(N).

Big-Omega

c , n0 > 0 such that f(N) c g(N) when N n0

f(N) grows no slower than g(N) for large N

Big-Omega: examples

Let f(N) = 2N2. Then

f(N) = (N)
f(N) = (N2)

(best answer)

Big-Theta
f(N) = (g(N)) iff
f(N) = O(g(N)) and f(N) = (g(N))
The growth rate of f(N) equals the growth rate
of g(N)
Example: Let f(N)=N2 , g(N)=2N2

Since f(N) = O(g(N)) and f(N) = (g(N)),


thus f(N) = (g(N)).

Big-Theta means the bound is the tightest


possible.

f(N) = (g(N))

the growth rate of f(N) is the same as the growth rate


of g(N)

Small - oh (o)

f(N) = o(g(N)) iff


f(N) = O(g(N)) and f(N) (g(N))

The growth rate of f(N) is always not greater


than growth rate of g(N)

Example: 3n+2 =o(n2) because 3n+2=O(n2)


and 3n+2 (n2)

Advantages of algorithm analysis

To eliminate bad algorithms early

You might also like