Computer
Science
GENG0032:
Know How Long A Program Takes
Algorithm Time-Complexity I
Abobakr Al-Shamiri
[email protected]
(Based on slides from Dr Corina Cîrstea)
Know How Long A Program Takes
1
Outline
1. TSP
2. Sorting
3. Time Complexity
4. Rates of Growth
2
Travelling Salesman Problem
Given a set of cities and a table of distances between cities, find the
shortest tour which goes through each city exactly once and returns
to the start.
Lon Car Dub Edin Reyk Oslo Sto Hel Cop Amst Bru Bonn Bern Rome Lisb Madr Par
London 0 223 470 538 1896 1151 1426 1816 950 349 312 503 743 1429 1587 1265 337
Cardiff 223 0 290 495 1777 1277 1589 1985 1139 564 533 725 927 1600 1492 1233 492
Dublin 470 290 0 350 1497 1267 1628 2026 1239 756 775 956 1207 1886 1638 1449 777
Edinburgh 538 495 350 0 1374 933 1314 1708 984 662 758 896 1243 1931 1964 1728 872
Reykjavik 1896 1777 1497 1374 0 1746 2134 2418 2104 2020 2130 2255 2617 3304 2949 2892 2232
Oslo 1151 1277 1267 933 1746 0 416 788 481 917 1088 1048 1459 2011 2739 2390 1343
Stockholm 1426 1589 1628 1314 2134 416 0 398 518 1126 1281 1181 1542 1978 2987 2593 1543
Helsinki 1816 1985 2026 1708 2418 788 398 0 881 1504 1650 1530 1856 2203 3360 2950 1910
Copenhagen 950 1139 1239 984 2104 481 518 881 0 625 769 662 1036 1538 2479 2076 1030
Amsterdam 349 564 756 662 2020 917 1126 1504 625 0 173 235 629 1296 1860 1480 428
Brussels 312 533 775 758 2130 1088 1281 1650 769 173 0 194 489 1174 1710 1315 262
Bonn 503 725 956 896 2255 1048 1181 1530 662 235 194 0 422 1067 1843 1420 400
Bern 743 927 1207 1243 2617 1459 1542 1856 1036 629 489 422 0 689 1630 1156 440
Rome 1429 1600 1886 1931 3304 2011 1978 2203 1538 1296 1174 1067 689 0 1862 1365 1109
Lisbon 1587 1492 1638 1964 2949 2739 2987 3360 2479 1860 1710 1843 1630 1862 0 500 1452
Madrid 1265 1233 1449 1728 2892 2390 2593 2950 2076 1480 1315 1420 1156 1365 500 0 1054
Paris 337 492 777 872 2232 1343 1543 1910 1030 428 262 400 440 1109 1452 1054 0
3
Example Tour
4
Brute Force
I wrote a program to solve TSP by
enumerating all tours and selecting
1
the shortest
0.8
I checked that it worked on some
problems with 10 cities 0.6
0.4
It takes just under half a second to
solve these problems 0.2
0
I set the program running on a 100 0 0.2 0.4 0.6 0.8 1
city problem – How long will it take
to finish?
5
How Many Possible Tours Are There?
For 100 cities how many
possible tours are there?
It doesn’t matter where we
start
Starting from Berlin there are
99 cites we can try next
6
Counting Tours (N cities)
Number of tours =
7
Counting Tours (N cities)
?
?
?
(N − 1)
?
Number of tours = (N − 1)
7
Counting Tours (N cities)
?
?
(N − 2)
?
Number of tours = (N − 1) ×(N − 2)
7
Counting Tours (N cities)
?
?
(N − 3)
?
Number of tours = (N − 1) ×(N − 2) ×(N − 3)
7
Counting Tours (N cities)
?
2
Number of tours = (N − 1) ×(N − 2) ×(N − 3) × · · · 2
7
Counting Tours (N cities)
Number of tours = (N − 1) ×(N − 2) ×(N − 3) × · · · 2 × 1 = (N − 1)!
7
How Long Does It Take?
The direction we go in is irrelevant
Total number of tours is (N − 1)!/2
Any more guesses how long it will take?
8
How Big is 99 Factorial?
99! = 99 · 98 · 97 · · · 2 · 1 =?
99!/2 ≈ 4.666 × 10155
How long does it take to check all possible tours?
We analysed about 200 000 tours in half a second
3.15 × 107sec = 1 year
Age of Universe ≈ 15 billion years
9
Answer
2.72 × 10132 ages of the universe!
10
Record TSP Solved – 15 112 and 24 978 Cities
11
In Case You’re Curious
Number of tours: 15111!/2 = 7.3 × 1056592
Current record: 24 978 cities with 1.9 × 1098992 tours
The algorithm for finding the shortest tour does not look at every
possible tour
12
Lessons
Even relatively small inputs can take you an astronomical time to
deal with using simple algorithms
As programmers we need to have an estimate for how long an
algorithm takes to run
For the 100 city TSP problem, if
we had 1087 cores, one for every particle in the Universe
we could compute a tour distance in 3 × 10−24 seconds, the
time it takes light to cross a proton
the brute force algorithm would still take 1039× the age of the
universe !!!
Smart algorithms can make a much bigger difference than fast
computers!
13
Outline
1. TSP
2. Sorting
3. Time Complexity
4. Rates of Growth
14
Sorting
Sorting is a very common problem, so we care about how long it
takes to solve it
Comparison between common sorting algorithms
Insertion sort – an easy algorithm to code
Shell sort – invented in 1959 by Donald Shell
Quick sort – invented in 1961 by Tony Hoare
These take an array of numbers and return a sorted array
15
Empirical Run Times
2000
Insertion sort
Shell sort
Median Time (ms)
1500 Quick sort
1000
500
0
0 5e+05 1e+06
Input size
16
Lessons
There is a right and wrong way to solve easy problems
You only really care when you are dealing with large inputs
Good algorithms are difficult to come up with
We would like to analyse/quantify the performance of an
algorithm and compare different algorithms without running them.
– How much better is Quick sort than Insertion sort?
17
Outline
1. TSP
2. Sorting
3. Time Complexity
4. Rates of Growth
18
Input Size
The running time of an algorithm depends on the size of its input
The input size is problem-dependent
number of elements in sorting
number of bits in integer multiplication
number of cities in TSP
number of nodes/edges in a graph
...
19
Running Time
We want to estimate the running time of algorithms
this depends on the hardware (how fast your computer is)
we want to abstract away differences in hardware !
We will take the running time of an algorithm on a given input to
be the number of ”steps”, or primitive operations (e.g. addition,
subtraction, comparison, . . . ) executed
each primitive operation takes a fixed amount of time to
execute
counting primitive operations gives a good measure of the time
taken by the algorithm, up to a constant factor
we can assign a cost to each line of pseudocode in our
algorithm by counting primitive operations
20
Example: minimum element in an array
MINIMUM(A) cost times
1 m = A[1] c1 1
2 i = 2 c2 1
3 while (i<=A.length) c3 n
4 m = min(m,A[i]) c4 n−1
5 i++ c5 n−1
ci counts the primitive operations in line i
For each array of size n, the algorithm takes
T (n) = c1 + c2 + c3 ∗ n + (c4 + c5) ∗ (n − 1)
primitive operations
Thus, T (n) = a ∗ n + b with a, b constants
Assume we count comparisons and assignments.
Q. How does T (n) change if line 4 is replaced by
if (A[i]<m)
m = a[i] ?
21
Time Complexity of an Algorithm
In general, the running time depends not only on the input size
but also on the input itself!
for a fixed input size, there is a best case and a worst case!
The worst-case running time (time complexity) of an algorithm is
a function T : N → N where
T (n) is the maximum number of elementary operations the
algorithm uses on inputs of size n
This gives a guarantee that the algorithm will not take longer.
The average-case running time (time complexity) is similar:
replace maximum by average
This definition assumes that all inputs are equally likely!
22
Average versus Worst Case
worst case
average case
time(I)
n=size(I)
23
Example: minimum element in an array (cont’d)
MINIMUM(A) cost times
1 m = A[1] c1 1
2 i = 2 c2 1
3 while (i<=A.length) c3 n
4 m = min(m,A[i]) c4 n−1
5 i++ c5 n−1
hence both the worst-case and the average-case running time is
T (n) = (c3 + c4 + c5) ∗ n + (c1 + c2 − c4 − c5)
24
Example: minimum element in an array (cont’d)
MINIMUM(A) cost times (worst-case) times (average-case)
1 m = A[1] c1 1 1
2 i = 2 c2 1 1
3 while (i<=A.length) c3 n n
4.1 if (a[i]<m) c4.1 n−1 n−1
4.2 m = a[i] c4.2 n−1 <n−1
5 i++ c5 n−1 n−1
the worst-case running time is as before
in the worst case, the values are in decreasing order
the average-case running time is slightly lower...
... but still roughly proportional to n
25
Outline
1. TSP
2. Sorting
3. Time Complexity
4. Rates of Growth
26
