D
J
J)
Y
COMPETITIVE
PROGRAMMING
PART -2
$
x
Scanned with CamScanner6 Greedy algorithms
6.1 Coin problem
62 Scheduling........
6.3 Tasks and deadlines
6.4 Minimizing sums
6.5 Data compression
7 Dynamic programming
7.1 Coin problem ........
7.2 Longest increasing subsequence
7.3 Pathsinagrid .......
7.4 Knapsack problems
7.5 Editdistance ........
7.6 Counting tilings
8 Amortized analysis
8.1 Two pointers method
8.2. Nearest smaller elements .
8.3. Sliding window minimum . beceeeee
9 Range queries
9.1 Static array queries... .
92 Binary indexed tree
9.3 Segment tree . . .
9.4 Additional techniques
10 Bit manipulation
10.1 Bit representation . . .
10.2 Bit operations . . .
10.3 Representing sets
10.4 Bit optimizations
10.5 Dynamic programming
II Graph algorithms
11 Basics of graphs
11.1 Graph terminology
11.2 Graph representation . . .
12 Graph traversal
12.1 Depth-first search
12.2 Breadth-first search .. . .
123 Applications
37
87
58
60
61
62
65
65
70
7
72
74
15
7
7
79
ceeeeeeeeeee BL
83
84
86
89
93
95
95
96
98
100
102
107
109
109
113
17
17
119
121
Scanned with CamScannerChapter 6
Greedy algorithms
A greedy algorithm constructs a solution to the problem by always making a
choice that looks the best at the moment. A greedy algorithm never takes back
its choices, but directly constructs the final solution. For this reason, greedy
algorithms are usually very efficient.
‘The difficulty in designing greedy algorithms is to find a greedy strategy that
always produces an optimal solution to the problem. The locally optimal choices
ina greedy algorithm should also be globally optimal. It is often difficult to argue
that a greedy algorithm works.
Coin problem
As a first example, we consider a problem where we are given a set of coins and
our task is to form a sum of money n using the coins. The values of the coins are
coins = (e1,¢2,...,c#l, and each coin can be used as many times we want. What
is the minimum number of coins needed?
For example, if the coins are the euro coins (in cents)
{1,2,5,10,20,50, 100,200}
and n = 520, we need at least four coins. The optimal solution is to select coins
200 + 200 + 100 +20 whose sum is 520.
Greedy algorithm
A simple greedy algorithm to the problem always selects the largest possible coin,
until the required sum of money has been constructed. This algorithm works in
the example case, because we first select two 200 cent coins, then one 100 cent
coin and finally one 20 cent coin. But does this algorithm always work?
It turns out that if the coins are the euro coins, the greedy algorithm always
works, ie,, it always produces a solution with the fewest possible number of coins.
‘The correctness of the algorithm can be shown as follow:
First, each coin 1, 5, 10, 50 and 100 appears at most once in an optimal
solution, because if the solution would contain two such coins, we could replace
57
Scanned with CamScannerthem by one coin and obtain a better solution. For example, if the solution would
contain coins 5 +5, we could replace them by coin 10.
In the same way, coins 2 and 20 appear at most twice in an optimal solution,
because we could replace coins 2+2+2 by coins 5 +1 and coins 20 +20 +20 by
coins 50+10. Moreover, an optimal solution cannot contain coins 2+2+1 or
20+20+10, because we could replace them by coins 5 and 50.
Using these observations, we can show for each coin x that it is not possible
to optimally construct a sum x or any larger sum by only using coins that are
smaller than x. For example, if x = 100, the largest optimal sum using the smaller
coins is 50+20+20+5+2+2=99. Thus, the greedy algorithm that always selects
the largest coin produces the optimal solution.
This example shows that it can be difficult to argue that a greedy algorithm
works, even if the algorithm itself is simple,
General case
In the general case, the coin set can contain any coins and the greedy algorithm
does not necessarily produce an optimal solution.
We can prove that a greedy algorithm does not work by showing a counterex-
ample where the algorithm gives a wrong answer. In this problem we can easily
find a counterexample: if the coins are {1,3,4) and the target sum is 6, the greedy
algorithm produces the solution 4 +1+1 while the optimal solution is 3+ 3.
It is not known if the general coin problem can be solved using any greedy
algorithm’. However, as we will see in Chapter 7, in some cases, the general
problem can be efficiently solved using a dynamic programming algorithm that
always gives the correct answer.
Scheduling
Many scheduling problems can be solved using greedy algorithms. A classic
problem is as follows: Given n events with their starting and ending times, find a
schedule that includes as many events as possible. It is not possible to select an
event partially. For example, consider the following events:
event starting time ending time
1 3
bamD
2 5
3 9
6 8
In this case the maximum number of events is two. For example, we can select
events B and D as follows:
THowever, itis possible to check in polynomial time if the greedy algorithm presented in this
chapter works for a given set of eains (53)
58
Scanned with CamScannerIt is possible to invent several greedy algorithms for the problem, but which
of them works in every case?
Algorithm 1
‘The first idea is to select as short events as possible. In the example case this
algorithm selects the following events:
es)
Cc)
bo)
However, selecting short events is not always a correct strategy. For example,
the algorithm fails in the following case:
CI
Co
Coe
If we select the short event, we can only select one event, However, it would be
possible to select both long events.
Algorithm 2
Another idea is to always select the next possible event that begins as early as
possible. This algorithm selects the following events:
a
co)
oo
However, we can find a counterexample also for this algorithm. For example,
in the following case, the algorithm only selects one event:
eee
co
Co
If we select the first event, it is not possible to select any other events. However,
it would be possible to select the other two events.
59
Scanned with CamScannerAlgorithm 3
‘The third idea is to always select the next possible event that ends as early as
possible. This algorithm selects the following events:
—
Be
Cc)
bo
It turns out that this algorithm always produces an optimal solution. The
reason for this is that it is always an optimal choice to first select an event that
ends as early as possible. After this, it is an optimal choice to select the next
event using the same strategy, ete., until we cannot select any more events.
‘One way to argue that the algorithm works is to consider what happens if we
first select an event that ends later than the event that ends as early as possible.
Now, we will have at most an equal number of choices how we can select the next
event. Hence, selecting an event that ends later can never yield a better solution,
and the greedy algorithm is correct.
Tasks and deadlines
Let us now consider a problem where we are given n tasks with durations and
deadlines and our task is to choose an order to perform the tasks. For each task,
we earn d-x points where d is the task’s deadline and x is the moment when we
finish the task. What is the largest possible total score we can obtain?
For example, suppose that the tasks are as follows:
task duration deadline
A 4 2
B 3 5
c 2 7
D 4 5
In this case, an optimal schedule for the tasks is as follows:
0 5 10
(CB a
In this solution, C yields 5 points, B yields 0 points, A yields -7 points and D
yields 8 points, so the total score is -10.
Surprisingly, the optimal solution to the problem does not depend on the
deadlines at all, but a correct greedy strategy is to simply perform the tasks
sorted by their durations in increasing order. The reason for this is that if we
ever perform two tasks one after another such that the first task takes longer
than the second task, we can obtain a better solution if we swap the tasks. For
example, consider the following schedule:
60
Scanned with CamScannerHere a > b, so we should swap the tasks:
yk)
6 a
Now X gives b points less and ¥ gives a points more, so the total score increases
by a—b > 0. In an optimal solution, for any two consecutive tasks, it must hold
that the shorter task comes before the longer task. Thus, the tasks must be
performed sorted by their durations.
Minimizing sums
We next consider a problem where we are given n numbers 1,2,
task is to find a value x that minimizes the sum
a, and our
Jays" + lag —x]° 40+ lan = 21°.
We focus on the cases ¢ = 1 and ¢=2.
Case c=1
In this case, we should minimize the sum
Jay x] + lay —x] +--+ Jay —2|
For example, if the numbers are [1,
which produces the sum
9,2,6], the best solution is to select x= 2
[1-2] + 12-2) +|9-2| + 12-2) + 16-2) = 12.
In the general case, the best choice for x is the median of the numbers, ie., the
middle number after sorting. For example, the list [1,2,9,2,6] becomes [1,2,2, 6,9]
after sorting, so the median is 2.
‘The median is an optimal choice, because if x is smaller than the median, the
sum becomes smaller by increasing x, and if x is larger then the median, the
sum becomes smaller by decreasing x. Hence, the optimal sohution is that x is
the median. If n is even and there are two medians, both medians and all values
between them are optimal choices.
2
In this case, we should minimize the sum
Case c
(ay =x) +(a2— 2) +--+ (an— 2)"
61
Scanned with CamScannerFor example, if the numbers are [1,2,9,2,6], the best solution is to select x= 4
which produces the sum
(1-4)? +(2-4)? + (9-4)? + (2-4)? + (6 - 4)? = 46,
In the general case, the best choice for x is the average of the numbers. In the
example the average is (1+2+9+2+6)5=4. This result can be derived by
presenting the sum as follows:
nx* -2x(ay +a +--+ ay) + (a4 +03 +---+a2)
‘The last part does not depend on x, so we can ignore it. The remaining parts
form a function nx? -2xs where | +az+---+ay. This is a parabola opening
upwards with roots x = 0 and x =2s/n, and the minimum value is the average of
the roots x= s/n, ie,, the average of the numbers ay,a2,...,d.
Data compression
A binary code assigns for each character of a string a codeword that consists
of bits, We can compress the string using the binary code by replacing each
character by the corresponding codeword. For example, the following binary code
assigns codewords for characters A-D:
character codeword
a 00
8 o1
c 10
o cn
This is a constant-length code which means that the length of each codeword is
the same. For example, we can compress the string AABACDACA as follows:
000001001011001000
Using this code, the length of the compressed string is 18 bits. However, we can
compress the string better if we use a variable-length code where codewords
may have different lengths. Then we can give short codewords for characters
that appear often and long codewords for characters that appear rarely. It turns
out that an optimal code for the above string is as follows:
character codeword
a 0
8 110
c 10
o 11
An optimal code produces a compressed string that is as short as possible. In this
case, the compressed string using the optimal cod
001100 101110100,
62
Scanned with CamScannerso only 15 bits are needed instead of 18 bits. Thus, thanks to a better code it was
possible to save 3 bits in the compressed string.
We require that no codeword is a prefix of another codeword. For example,
it is not allowed that a code would contain both codewords 10 and 1011. The
reason for this is that we want to be able to generate the original string from
the compressed string. If a codeword could he a prefix of another codeword, this
would not always be possible. For example, the following code is not valid:
character codeword
a 10
8 u
c 1011
D a
Using this code, it would not be possible to know if the compressed string 1011
corresponds to the string AB or the string C.
Huffman coding
Huffman coding? is a greedy algorithm that constructs an optimal code for
compressing a given string. The algorithm builds a binary tree based on the
frequencies of the characters in the string, and each character's codeword can be
read by following a path from the root to the corresponding node. A move to the
left corresponds to bit 0, and a move to the right corresponds to bit 1
Initially, each character of the string is represented by a node whose weight
is the number of times the character occurs in the string. Then at each step two
nodes with minimum weights are combined by creating a new node whose weight
is the sum of the weights of the original nodes. The process continues until all
nodes have been combined.
‘Next we will see how Huffman coding creates the optimal code for the string
AABACDACA, Initially, there are four nodes that correspond to the characters of the
string:
® O®® @
A 8 c D
‘The node that represents character A has weight 5 because character A appears 5
times in the string. The other weights have been calculated in the same way.
The first step is to combine the nodes that correspond to characters 8 and D,
both with weight 1. The result is:
2D. A. Huffman discovered this method when solving a university course assignment and
published the algorithm in 1952 [40],
63
Scanned with CamScannerAfter this, the nodes with weight 2 are combined:
Now all nodes are in the tree, so the code is ready. The following codewords
can be read from the tree:
character codeword
a 0
8 110
c 10
D a
64
Scanned with CamScannerChapter 7
Dynamic programming
Dynamic programming is a technique that combines the correctness of com-
plete search and the efficiency of greedy algorithms. Dynamic programming can
be applied if the problem can be divided into overlapping subproblems that can
be solved independently.
‘There are two uses for dynamic programming:
+ Finding an optimal solution: We want to find a solution that is as large
as possible or as small as possible.
* Counting the number of solutions: We want to calculate the total num-
ber of possible solutions,
We will first see how dynamic programming can be used to find an optimal
solution, and then we will use the same idea for counting the solutions.
Understanding dynamic programming is a milestone in every competitive
programmer's career. While the basic idea is simple, the challenge is how to apply
dynamic programming to different problems. This chapter introduces a set of
classic problems that are a good starting point,
Coin problem
We first focus on a problem that we have already seen in Chapter 6: Given a set,
of coin values coins ={¢1,¢2,...,¢g) and a target sum of money n, our task is to
form the sum n using as few coins as possible.
In Chapter 6, we solved the problem using a greedy algorithm that always
chooses the largest possible coin. The greedy algorithm works, for example, when
the coins are the euro coins, but in the general case the greedy algorithm does
not necessarily produce an optimal solution,
‘Now is time to solve the problem efficiently using dynamic programming, so
that the algorithm works for any coin set. The dynamic programming algorithm
is based on a recursive function that goes through all possibilities how to form
the sum, like a brute force algorithm. However, the dynamic programming
algorithm is efficient because it uses memoization and calculates the answer to
each subproblem only once.
65
Scanned with CamScannerRecursive formulation
The idea in dynamic programming is to formulate the problem recursively so
that the solution to the problem can be calculated from solutions to smaller
subproblems. In the coin problem, a natural recursive problem is as follows: what
is the smallest number of coins required to form a sum x?
Let solve(x) denote the minimum number of coins required for a sum x.
The values of the function depend on the values of the coins. For example, if
coins = {1,8,41, the first values of the function are as follows:
solve(0) =
solve(1)
solve(2) =
solve(3)
solve(4) =
solve(5)
solve(6) =
solve(7)
solve(8) =
solve(9)
solve(10)
For example, solve(10) = 3, because at least 3 coins are needed to form the
sum 10. The optimal solution is 3+3+4= 10.
‘The essential property of solve is that its values can be recursively calculated
from its smaller values. The idea is to focus on the first coin that we choose for
the sum. For example, in the above scenario, the first coin can be either 1, 3
or 4. If we first choose coin 1, the remaining task is to form the sum 9 using
the minimum number of coins, which is a subproblem of the original problem.
Of course, the same applies to coins 3 and 4. Thus, we can use the following
recursive formula to calculate the minimum number of coins:
ENR MEE HOS
"
cy
solve(x) = min(solve(x—1) +1,
solve(x—-3)+1,
solve(x—4)+1).
‘The base case of the recursion is solve(0) = 0, because no coins are needed to
form an empty sum. For example,
solve(10) = solve(7)+ 1 = solve(4) + 2 = solve(0)+3=3.
Now we are ready to give a general recursive function that calculates the
minimum number of coins needed to form a sum x:
co x<0
solve(x)=40 x=0
Mincccoirs SOlve(t—c)+1 x >0
First, if x <0, the value is co, because it is impossible to form a negative sum.
of money. Then, if x= 0, the value is 0, because no coins are needed to form an
66
Scanned with CamScannerempty sum. Finally, if x > 0, the variable c goes through all possibilities how to
choose the first coin of the sum.
Once a recursive function that solves the problem has been found, we can
directly implement a solution in C++ (the constant INF denotes infinity):
int solve(int x) {
if (x < @) return INF;
if (x == @) return @
int best = INF;
for (auto € : coins) (
best = min(best, solve(x-c)*1);
)
return best;
>
Still, this function is not efficient, because there may be an exponential
number of ways to construct the sum. However, next we will see how to make the
function efficient using a technique called memoization,
Using memoization
‘The idea of dynamic programming is to use memoization to efficiently calculate
values of a recursive function. This means that the values of the function are
stored in an array after calculating them. For each parameter, the value of the
function is calculated recursively only once, and after this, the value can be
directly retrieved from the array.
In this problem, we use arrays
bool readyiN];
int _value(N];
where ready[] indicates whether the value of solve() has been calculated,
and if it is, value[x] contains this value. The constant N has been chosen so that,
all required values fit in the arrays.
‘Now the function can be efficiently implemented as follows:
int solve(int x) (
if (x < @) return INF;
if (x == @) return @;
if (readytx]) return valueCx];
int best = INF;
for (auto € : coins) (
best = min(best, solve(x-c)+1);
y
valuelx]
ready[x]
return best;
67
Scanned with CamScanner‘The function handles the base cases x <0 and x =0 as previously. Then the
function checks from ready(-] if solve(x) has already been stored in value[.x], and
if it is, the function directly returns it. Otherwise the function calculates the
value of solve(x) recursively and stores it in value[-x]
This function works efficiently, because the answer for each parameter x is
calculated recursively only once. After a value of solve(x) has been stored in
valuefs!], it can be efficiently retrieved whenever the function will be called again
with the parameter x. The time complexity of the algorithm is O(nk), where n is
the target sum and k is the number of coins,
Note that we can also iteratively construct the array value using a loop that,
simply calculates all the values of solve for parameters 0...n:
valuele]
for (int x
value[x] = INF
for (auto € : coins) {
if (re >= 0) {
value(x] = min(value[x], value(x-c]+1);
x) (
d
?
In fact, most competitive programmers prefer this implementation, because
it is shorter and has lower constant factors. From now an, we also use iterative
implementations in our examples. Still, it is often easier to think about dynamic
programming solutions in terms of recursive functions,
Constructing a solution
Sometimes we are asked both to find the value of an optimal solution and to give
an example how such a solution can be constructed. In the coin problem, for
example, we can declare another array that indicates for each sum of money the
first coin in an optimal solution:
‘Then, we can modify the algorithm as follows:
int first(N]
value(@]
For (int x xen: x) (
value[x] = INF;
for (auto € : coins) (
if Gre >= @ 8&8 value(x-c]+) < valuefx]) {
valuelx] = valuefx-c]+1;
Firstlxd]
68
Scanned with CamScannerAfter this, the following code can be used to print the coins that appear in an
optimal solution for the sum n:
while (n> @) (
cout << first{n] << "\n'
n-= first(n];
Counting the number of solutions
Let us now consider another version of the coin problem where our task is to
calculate the total number of ways to produce a sum x using the coins. For
example, if coins = (1,3,4} and x = 5, there are a total of 6 ways:
e1s1eititl oB41t1
© 14143 sta
© 14341 eae
Again, we can solve the problem recursively. Let solve(x) denote the number
of ways we can form the sum x. For example, if coins = {1,3,4), then solve(5) = 6
and the recursive formula is
solve(x) =solve(x - 1)+
solve(x~3)+
solvelx—4).
‘Then, the general recursive function is as follows:
0 x<0
solve(x)=4 1 x=0
Leceoins SOlve(x—c) x>0
Ifx <0, the value is 0, because there are no solutions. If x = 0, the value is 1,
because there is only one way to form an empty sum. Otherwise we calculate the
sum of all values of the form solve(x~c) where c is in coins.
‘The following code constructs an array count such that count[x] equals the
value of solve(x) for 0
= @) {
count£x] += count{x-c];
69
Scanned with CamScannerOften the number of solutions is so large that it is not required to calculate the
exact number but it is enough to give the answer modulo m where, for example,
m= 10° +7. This can be done by changing the code so that all calculations are
done modulo m. In the above code, it suffices to add the line
count [x] %= m;
after the line
countLx] += countLx-c];
Now we have discussed all basic ideas of dynamic programming. Since
dynamic programming can be used in many different situations, we will now go
through a set of problems that show further examples about the possibilities of
dynamic programming.
Longest increasing subsequence
Our first problem is to find the longest increasing subsequence in an array
of n elements. This is a maximum-length sequence of array elements that goes
from left to right, and each element in the sequence is larger than the previous
element. For example, in the array
o 12845 6
6l|2[5]il7|4]als
the longest increasing subsequence contains 4 elements:
o 12345 67
6[2[5]1]7] 4/8
UN"
Let length(k) denote the length of the longest increasing subsequence that.
ends at position k. Thus, if we calculate all values of length(k) where 0= @) possibletxILk]
possible[x}{k] |= possible{x](k~
possibletx-wfk]]Ck-11;
?
However, here is a better implementation that only uses a one-dimensional
array possible[x] that indicates whether we can construct a subset with sum x.
‘The trick is to update the array from right to left for each new weight:
possible(@]
for (int k #1; k cen; ke) (
for (int «2M; x32 0; x--) {
if (possible(x]) possibleLx+wlk]] = true;
y
>
Note that the general idea presented here can be used in many knapsack
problems. For example, if we are given objects with weights and values, we can
determine for each weight sum the maximum value sum of a subset.
13
Scanned with CamScannerEdit distance
‘The edit distance or Levenshtein distance! is the minimum number of edi
ing operations needed to transform a string into another string. The allowed
editing operations are as follows:
* insert a character (e.g. ABC —- ABCA)
* remove a character (e.g. ABC — AC)
* modify a character (e.g. ABC — ADC)
For example, the edit distance between LOVE and NOVIE is 2, because we can
first perform the operation LOVE — MOVE (modify) and then the operation MOVE +
NOVIE (insert). This is the smallest possible number of operations, because it is
clear that only one operation is not enough.
Suppose that we are given a string x of length n and a string y of length m,
and we want to calculate the edit distance between x and y. To solve the problem,
we define a function distance(a, 6) that gives the edit distance between prefixes
x{0...a] and yl0...5]. Thus, using this function, the edit distance between x and
y equals distance(n~1,m- 1).
We can calculate values of distance as follows:
distance(a,b) = min(distance(a,b-1)+1,
distance(a—1,b)+1,
distance(a ~1,b— 1) + cost(a,b))
Here cost(a,b) = 0 if xa] = y[b], and otherwise cost(a,b) = 1. The formula
considers the following ways to edit the string x:
* distance(a,b —1): insert a character at the end of x
* distance(a—1,b): remove the last character from x
* distance(a~1,b~1): match or modify the last character of x
In the two first cases, one editing operation is needed (insert or remove). In the
last case, if x{a] = y[b], we can match the last characters without editing, and
otherwise one editing operation is needed (modify)
The following table shows the values of distance in the example case:
MOVIE
of1f2]3]4]5
tfafaf2lala[s
ol2|2fij2is/4
v[s[3f2lilels
ela{a[al2i2/2
Phe distance is named after V. I. Levenshtein who studied it in connection with binary codes
(49),
74
Scanned with CamScannerThe lower-right corner of the table tells us that the edit distance between
LOVE and MOVIE is 2. The table also shows how to construct the shartest sequence
of editing operations. In this case the path is as follows:
¥
1
w[wlo
wlele|<
wlelele
wo) efe|olm
m
and the total number of solutions is 781.
The problem can be solved using dynamic programming by going through
the grid row by row. Each row in a solution can be represented as a string that
contains m characters from the set (7,1, ,—. For example, the above solution
consists of four rows that correspond to the following strings:
*ncancan
* ucdunnu
+ cacguun
* cacgcsu
Let count(k,x) denote the number of ways to construct a solution for rows
1...k of the grid such that string x corresponds to row k. It is possible to use
dynamic programming here, because the state of a row is constrained only by the
state of the previous row.
ony
Scanned with CamScannerA solution is valid if row 1 does not contain the character Li, row n does not
contain the character /, and all consecutive rows are compatible. For example, the
rows UC UNMU and CIC UUM are compatible, while the rows CIN IN
and COCCI are not compatible.
Since a row consists of m characters and there are four choices for each
character, the number of distinct rows is at most 4”. Thus, the time complexity
of the solution is O(n4?”) because we can go through the O(4") possible states
for each row, and for each state, there are O(4”) possible states for the previous
row. In practice, it is a good idea to rotate the grid so that the shorter side has
length m, because the factor 42” dominates the time complexity.
It is possible to make the solution more efficient by using a more compact
representation for the rows. It turns out that it is sufficient to know which
columns of the previous row contain the upper square of a vertical tile. Thus, we
can represent a row using only characters 7 and D, where Cis a combination
of characters U, C and 4. Using this representation, there are only 2” distinct
rows and the time complexity is O(n 2”),
Asa final note, there is also a surprising direct formula for calculating the
number of tilings”:
fz tm xb
TL [] 4:€cos* = + cos’ ——
: nat ma
‘This formula is very efficient, because it calculates the number of tilings in O(nm)
time, but since the answer is a product of real numbers, a problem when using
the formula is how to store the intermediate results accurately.
Surprisingly, this formula was discovered in 1961 by two research teams [43, 67] that worked
independently:
16
Scanned with CamScannerChapter 8
Amortized analysis
‘The time complexity of an algorithm is often easy to analyze just by examining
the structure of the algorithm: what loops does the algorithm contain and how
many times the loops are performed. However, sometimes a straightforward
analysis does not give a true picture of the efficiency of the algorithm.
Amortized analysis can be used to analyze algorithms that contain opera-
tions whose time complexity varies. The idea is to estimate the total time used to
all such operations during the execution of the algorithm, instead of focusing on
individual operations.
Two pointers method
In the two pointers method, two pointers are used to iterate through the array
values. Both pointers can move to one direction only, which ensures that the
algorithm works efficiently. Next we discuss two problems that can be solved
using the two pointers method.
Subarray sum
As the first example, consider a problem where we are given an array of n positive
integers and a target sum x, and we want to find a subarray whose sum is x or
report that there is no such subarray.
For example, the array
1jaf2|sfifa
contains a subarray whose s
1{3|ajsjilile
‘This problem can be solved in O(n) time by using the two pointers method.
The idea is to maintain pointers that point to the first and last value of a subarray.
On each turn, the left pointer moves one step to the right, and the right pointer
moves to the right as long as the resulting subarray sum is at most x. If the sum
becomes exactly x, a solution has been found.
7
Scanned with CamScannerAs an example, consider the following array and a target sum x = 8:
ij3f2|sjilije|s
‘The initial subarray contains the values
1j3s|2}5|1jij2}3
T T
Then, the left pointer moves one step to the right. The right pointer does not
move, because otherwise the subarray sum would exceed x.
1/3]2]sjijij2|3
TT
Again, the left pointer moves one step to the right, and this time the right
pointer moves three steps to the right. The subarray sum is 2+5+1=8, 50a
subarray whose sum is x has been found.
1{3|[2]sjijij2|3
T T
‘The running time of the algorithm depends on the number of steps the right
pointer moves. While there is no useful upper bound on how many steps the
pointer can move on a single turn. we know that the pointer moves a total of
O(n) steps during the algorithm, because it only moves to the right.
Since both the left and right pointer move O(n) steps during the algorithm,
the algorithm works in O(n) time.
2SUM problem
Another problem that can be solved using the two pointers method is the following
problem, also known as the 2SUM problem: given an array of n numbers and a
target sum x, find two array values such that their sum is x, or report that no
such values exist
‘To solve the problem, we first sort the array values in increasing order. After
that, we iterate through the array using two pointers. The left pointer starts at
the first value and moves one step to the right on each turn. The right pointer
begins at the last value and always moves to the left until the sum of the left and
right value is at most x. If the sum is exactly x, a solution has been found.
For example, consider the following array and a target sum x = 12:
1j4|{5/6|7] 9/9 |10
‘The initial positions of the pointers are as follows. The sum of the values is
1+10=11 that is smaller than x
18
Scanned with CamScanner1|4|5/6|7]9]9 |10
T T
Then the left pointer moves one step to the right. The right pointer moves
three steps to the left, and the sum becomes 4+7=11
1|4]5/6|7] 9/9 |10
T T
After this, the left pointer moves one step to the right again. The right pointer
does not move, and a solution 5 +7 = 12 has been found.
1|4|5|6{7]9/ 9 |10
T T
‘The running time of the algorithm is O(nlogn), because it first sorts the array
in O(nlogn) time, and then both pointers move O(n) steps.
Note that it is possible to solve the problem in another way in O(n logn) time
using binary search. In such a solution, we iterate through the array and for
each array value, we try to find another value that yields the sum x. This can be
done by performing n binary searches, each of which takes O(logn) time.
A more difficult problem is the 3SUM problem that asks to find three array
values whose sum is x. Using the idea of the above algorithm, this problem can
be solved in O(n*) time!. Can you see how?
Nearest smaller elements
Amortized analysis is often used to estimate the number of operations performed
ona data structure. The operations may be distributed unevenly so that most
operations occur during a certain phase of the algorithm, but the total number of,
the operations is limited
As an example, consider the problem of finding for each array element the
nearest smaller element, ie., the first smaller element that precedes the
element in the array. It is possible that no such element exists, in which case the
algorithm should report this. Next we will see how the problem can be efficiently
solved using a stack structure.
We go through the array from left to right and maintain a stack of array
elements. At each array position, we remove elements from the stack until the
top element is smaller than the current element, or the stack is empty. Then, we
report that the top element is the nearest smaller element of the current element,
or if the stack is empty, there is no such element. Finally, we add the current
element to the stack.
‘As an example, consider the following array:
‘Ror a long time, it was thought that solving the 3SUM problem more efficiently than in O(n®)
time would not be possible. However, in 2014, it turned out [30] that this isnot the case.
79
Scanned with CamScannerFirst, the elements 1, 3 and 4 are added to the stack, because each element is
larger than the previous element. Thus, the nearest smaller clement of 4 is 3,
and the nearest smaller element of 3 is 1.
‘The next element 2 is smaller than the two top elements in the stack. Thus,
the elements 3 and 4 are removed from the stack, and then the element 2 is
added to the stack. Its nearest smaller element is 1
1j3|4aja]s|aiaiea
Then, the element 5 is larger than the element 2, so it will be added to the
stack, and its nearest smaller element is 2:
After this, the element 5 is removed from the stack and the elements 3 and 4
are added to the stack:
Finally, all elements except 1 are removed from the stack and the last element
2 is added to the stack:
The efficiency of the algorithm depends on the total number of stack opera-
tions. If the current element is larger than the top element in the stack, it is
directly added to the stack, which is efficient. However, sometimes the stack can
contain several larger elements and it takes time to remove them. Still, each
element is added exactly once to the stack and removed at most once from the
stack. Thus, each element causes O(1) stack operations, and the algorithm works
in O(n) time.
80
Scanned with CamScannerSliding window minimum
A sliding window is a constant-size subarray that moves from left to right
through the array. At each window position, we want to calculate some infor-
mation about the elements inside the window. In this section, we focus on the
problem of maintaining the sliding window minimum, which means that we
should report the smallest value inside each window.
‘The sliding window minimum can be calculated using a similar idea that
we used to calculate the nearest smaller elements. We maintain a queue where
each element is larger than the previous element, and the first element always
corresponds to the minimum element inside the window. After each window
move, we remove elements from the end of the queue until the last queue element
is smaller than the new window element, or the queue becomes empty. We also
remove the first queue element if it is not inside the window anymore. Finally,
we add the new window element to the end of the queue.
As an example, consider the following array:
2|ila|siaf4ajije
‘Suppose that the size of the sliding window is 4. At the first window position,
the smallest value is 1:
2|al4|s]3l4iif2
HHH)
‘Then the window moves one step right. The new element 3 is smaller than
the elements 4 and 5 in the queue, so the elements 4 and 5 are removed from the
queue and the element 3 is added to the queue. The smallest value is still 1
2|af4a]slial4iij2
After this, the window moves again, and the smallest element 1 does not
belong to the window anymore. Thus, it is removed from the queue and the
smallest value is now 3. Also the new element 4 is added to the queue.
2|ifa]s[sf4ajif2
‘The next new element 1 is smaller than all elements in the queue. Thus, all
elements are removed from the queue and it will only contain the element 1:
2|1|4|6]s}4}1]2
O)
81
Scanned with CamScannerFinally the window reaches its last position. The element 2 is added to the
queue, but the smallest value inside the window is still 1.
2[1]4]5]afajzf2
GHz)
Since each array clement is added to the queue exactly once and removed
from the queue at most once, the algorithm works in O(n) time.
82
Scanned with CamScannerChapter 9
Range queries
In this chapter, we discuss data structures that allow us to efficiently process
range queries. In a range query, our task is to calculate a value based on a
subarray of an array. Typical range queries are:
* sumg(a, ): calculate the sum of values in range [a,b]
* ming(a,): find the minimum value in range [a,b]
* maxp(a,): find the maximum value in range [a,6]
For example, consider the range [3,6] in the following array:
012345 67
1|3|[sjajejij3al4
In this case, sum,(3,6) = 14, ming(3,6) = 1 and max,(3,6) = 6.
A simple way to process range queries is to use a loop that goes through all
array values in the range. For example, the following function can be used to
process stm queries on an array:
int sum(int a, int b) {
int s = 0;
for (int dea; i
s = arrayli]
by it) ¢
)
»
This function works in O(n) time, where n is the size of the array. Thus, we
can process q queries in O(ng) time using the function. However, if both n and q
are large, this approach is slow. Fortunately, it turns out that there are ways to
process range queries much more efficiently.
83
Scanned with CamScannerStatic array queries
We first focus on a situation where the array is static, i.e., the array values are
never updated between the queries. In this case, it suffices to construct a static
data structure that tells us the answer for any possible query.
Sum queries
We can easily process sum queries on a static array by constructing a prefix
sum array. Each value in the prefix sum array equals the sum of values in the
original array up to that position, ie., the value at position k is sum4(0, k). The
prefix sum array can be constructed in O(n) time.
For example, consider the following array:
123
ij3|4aisljelilaj2
‘The corresponding prefix sum array is as follows
O12 845 67
1| 4 | 8 |16|22|23|27|29
Since the prefix sum array contains all values of sum,(0,), we can caleulate any
value of sum4(a, 6) in O(1) time as follows:
sum, (a,b) = sumy(0,5) ~ sum(0,a~ 1)
By defining sun,(0,-1)=0, the above formula also holds when a =0.
For example, consider the range [3,6]
O12 384 5 67
sielil4l2
*
= 19. This sum can be calculated from two
In this case sum,(3,6
values of the prefix sum array:
o 12345 67
1| 4 | 8 |16|22]23|27|29
‘Thus, sumg(3,6) = sumg(0,6) ~ sum,(0,2)=27-8=19.
It is also possible to generalize this idea to higher dimensions. For example,
‘we can construct a two-dimensional prefix sum array that can be used to calculate
the sum of any rectangular subarray in O(1) time. Each sum in such an array
corresponds to a subarray that begins at the upper-left corner of the array.
84
Scanned with CamScanner‘The following picture illustrates the idea’
ID c
B Al |
I
‘The sum of the gray subarray can be calculated using the formula
S(A)-S(B)-S(C)+S(D),
where $(X) denotes the sum of values in a rectangular subarray from the upper-
left corner to the position of X.
Minimum queries
Minimum queries are more difficult to process than sum queries, Still, there is
a quite simple O(n logn) time preprocessing method after which we can answer
any minimum query in O(1) time’. Note that since minimum and maximum
queries can be processed similarly, we can focus on minimum queries.
‘The idea is to precalculate all values of ming(a, 6) where 6 ~a +1 (the length
of the range) is a power of two. For example, for the array
o12 345 67
aj3fa|siefilal2
the following values are calculated:
a_b_ming(a,b)
0
1
2
ck ewnols
aaaaeale
b
i
2
3
4
5
6
7
beHonen
3
4
5
6
saoneennolas
aonkewroa
‘The number of precalculated values is O(n logn), because there are O(logn)
range lengths that are powers of two. The values can be calculated efficiently
using the recursive formula
ming(a,b) = min(ming(a,a +w—1),ming(a+w,b)),
This technique was introduced in [7] and sometimes called the sparse table method, There
are also more sophisticated techniques (22] where the preprocessing time is only O(n), but such
algorithms are not needed in competitive programming,
85
Scanned with CamScannerwhere 6-a +1 is a power of two and w = (b-a + 1)/2. Calculating all those values
takes O(n logn) time.
After this, any value of ming(a,b) can be calculated in O(1) time as a minimum,
of two precalculated values. Let k be the largest power of two that does not exceed
b-a+1, We can calculate the value of ming(a,b) using the formula
ming(a,b) = min(ming(a,a +h ~1),ming(b~k +1,)).
In the above formula, the range [a,b] is represented as the union of the ranges
[a,a+k-1]) and (6 -k+ 1,6), both of length k.
As an example, consider the range [1,6]
28 45 67
1{3[4]sjelil4[2
‘The length of the range is 6, and the largest power of two that does not exceed 6
is 4. Thus the range [1,6] is the union of the ranges [1,4] and [3,6]:
012345 67
1js|4jslelijsi2
Since ming(1,4)=3 and ning(3,6)= 1, we conclude that ming(1,6) = 1.
Binary indexed tree
Abinary indexed tree or a Fenwick tree can be seen as a dynamic variant,
of a prefix sum array. It supports two O(logn) time operations on an array:
processing a range sum query and updating a value
‘The advantage of a binary indexed tree is that it allows us to efficiently update
array values between sum queries. This would not be possible using a prefix sum
array, because after each update, it would be necessary to build the whole prefix
sum array again in O(n) time.
Structure
Even if the name of the structure is a binary indexed tree, it is usually represented
as an array. In this section we assume that all arrays are one-indexed, because it
makes the implementation easier.
Let p(k) denote the largest power of two that divides k. We store a binary
indexed tree as an array tree such that
tree{he] = sumy(k— plk)+1,k),
2he binary indexed tree structure was presented by P. M. Fenwick in 1994 [21]
86
Scanned with CamScanneri.e., each position & contains the sum of values in a range of the original array
whose length is p(k) and that ends at position &. For example, since p(6) = 2,
tree{6] contains the value of sumg(5,6).
For example, consider the following array’
1j3{4a|sjelija
‘The corresponding binary indexed tree is as follows:
12345 678
1|4| 4/16] 6|7| 4 |29
The following picture shows more clearly how each value in the binary indexed
tree corresponds to a range in the original array:
4
16
8
29
tsTe foe [74
oe rtd
oyo;o};o
5
I Je
FE
Using a binary indexed tree, any value of sum,(1,#) can be calculated in
Ollogn) time, because a range [1,k] can always be divided into O(logn) ranges
whose sums are stored in the tree.
For example, the range [1,7] consists of the following ranges:
45678
16| 6 [7 | 4 [29]
i
TELE
e]a)a)h
—_
|
‘Thus, we can calculate the corresponding sum as follows:
sumg(1,7) = sumg(1,4) + sumg(5, 6) + sumg(7,7) = 16-+7 +
To calculate the value of sum,(a,b) where a > 1, we can use the same trick
that we used with prefix sum arrays:
sumg(a,b) = sumy(1, 6) — sumg(1,@~ 1).
87
Scanned with CamScannerSince we can calculate both sumy(1,5) and sumg(1,a~1) in O(logn) time, the total
time complexity is O(logn).
‘Then, after updating a value in the original array, several values in the binary
indexed tree should be updated. For example, if the value at position 3 changes,
the sums of the following ranges change:
8
29
Te
iE
Since each array element belongs to O(logn) ranges in the binary indexed
tree, it suffices to update O(logn) values in the tree.
Implementation
‘The operations of a binary indexed tree can be efficiently implemented using bit.
operations. The key fact needed is that we can calculate any value of p(k) using
the formula
pik) = kB -k.
‘The following function calculates the value of sum,(1,)
int sum(int k) {
int 5 = 0;
while (k >= 1)
treetkl;
kak;
>
The following function increases the array value at position k by x (x can be
positive or negative):
void add(int k, int x) {
while (k = 1; k /= 2) {
tree(k] = tree[2*k]+treel2sk+1];
)
91
Scanned with CamScannerFirst the function updates the value at the bottom level of the tree. After this,
the function updates the values of all internal tree nodes, until it reaches the top
node of the tree.
Both the above functions work in O(logn) time, because a segment tree of n
elements consists of O(log n) levels, and the functions move one level higher in
the tree at each step.
Other queries
‘Segment trees can support all range queries where it is possible to divide a range
into two parts, calculate the answer separately for both parts and then efficiently
combine the answers. Examples of such queries are minimum and maximum,
greatest common divisor, and bit operations and, or and xor.
For example, the following segment tree supports minimum queries:
In this case, every tree node contains the smallest value in the corresponding
array range. The top node of the tree contains the smallest value in the whole
array. The operations can be implemented like previously, but instead of sums,
minima are calculated
‘The structure of a segment tree also allows us to use binary search for locating
array elements. For example, if the tree supports minimum queries, we can find
the position of an element with the smallest value in O(logn) time.
For example, in the above tree, an element with the smallest value 1 can be
found by traversing a path downwards from the top node:
Scanned with CamScannerAdditional techniques
Index compression
A limitation in data structures that are built upon an array is that the elements
are indexed using consecutive integers. Difficulties arise when large indices are
needed. For example, if we wish to use the index 10°, the array should contain
10° elements which would require too much memory.
However, we can often bypass this limitation by using index compression,
where the original indices are replaced with indices 1,2,8, ete. This can be done
if we know all the indices needed during the algorithm beforehand.
‘The idea is to replace each original index x with e(x) where c is a function that,
compresses the indices. We require that the order of the indices does not change,
so if a > k removes the & last bits from the number. For example, 14<<2= 56,
because 14 and 56 correspond to 1110 and 111000. Similarly, 49 >> 3 = 6, because
49 and 6 correspond to 110001 and 110.
Note that x << corresponds to multiplying x by 2', and x >> corresponds
to dividing x by 2* rounded down to an integer.
Applications
Anumber of the form 1 << & has a one bit in position k and all other bits are zero,
so we can use such numbers to access single bits of numbers. In particular, the
kth bit of a number is one exactly when x & (1 << &) is not zero. The following
code prints the bit representation of an int number x:
for (int i= 31; i >= @;
if (a(I
‘The following code goes through the subsets of a set x:
int b
do (
// process subset b
J while (b=(b-x)8x);
99
Scanned with CamScannerBit optimizations
Many algorithms can be optimized using bit operations. Such optimizations
do not change the time complexity of the algorithm, but they may have a large
impact on the actual running time of the code. In this section we discuss examples
of such situations.
Hamming distances
‘The Hamming distance hamning(a,5) between two strings a and of equal
length is the number of positions where the strings differ. For example,
hamming(01101, 11001) = 2.
Consider the following problem: Given a list of n bit strings, each of length k,
calculate the minimum Hamming distance between two strings in the list. For
example, the answer for [00111,01101, 11110] is 2, because
+ hamming(00111,01101)
© hanming(00111, 11110}
© hamming(01101, 11110)
A straightforward way to solve the problem is to go through all pairs of strings
and calculate their Hamming distances, which yields an O(n?A) time algorithm.
The following function can be used to calculate distances:
int haming(string a, string b) {
int d= @;
for (int i= @; i best[1< adjINI
The constant N is chosen so that all adjacency lists can be stored. For example,
the graph
Q {)
w
can be stored as follows:
'ad3L1]_push_back(2)
adj(2]-push_back(3)
ad}[2]_push_back(4)
adj(3]-push_back(4);
/adj(4) _push_back(1);
If the graph is undirected, it can be stored in a similar way, but each edge is
added in both directions.
For a weighted graph, the structure can be extended as follows:
vector> adjiN];
In this case, the adjacency list of node a contains the pair (b,w) always when
there is an edge from node a to node 6 with weight w. For example, the graph
5 7
Q-@—)
2\6 5
®
113
Scanned with CamScannercan be stored as follows:
adjC1].push_back((2,5));
adj£21.pushback((3,7));
adj{21.push_back((4,6});
.adj[3] push _back((4,5));
adj(4] push_back((1,2));
‘The benefit of using adjacency lists is that we can efficiently find the nodes
to which we can move from a given node through an edge. For example, the
following loop goes through all nodes to which we can move from node s:
for (auto u : adjls]) ¢
1 process node u
?
Adjacency matrix representation
An adjacency matrix is a two-dimensional array that indicates which edges
the graph contains. We can efficiently check from an adjacency matrix if there is
an edge between two nodes. The matrix can be stored as an array
int adENJIN];
where each value adjla][b] indicates whether the graph contains an edge from
node a to node 6. If the edge is included in the graph, then adjfal[5] = 1, and
otherwise adjla[6]=0. For example, the graph
1
o
can be represented as follows:
123 4
1)/0/1/)0/0
2/)0/0/1}1
3/0/0/0}1
4lifololo
If the graph is weighted, the adjacency matrix representation can be extended
so that the matrix contains the weight of the edge if the edge exists. Using this
representation, the graph
14
Scanned with CamScannercorresponds to the following matrix:
Rewe
elolelale
elelslole
elalalola
‘The drawback of the adjacency matrix representation is that the matrix
contains n? elements, and usually most of them are zero. For this reason, the
representation cannot be used if the graph is large.
Edge list representation
An edge list contains all edges of a graph in some order. This is a convenient,
way to represent a graph if the algorithm processes all edges of the graph and it
is not needed to find edges that start at a given node.
The edge list can be stored in a vector
vector> edges;
where each pair (a,b) denotes that there is an edge from node a to node b. Thus,
the graph
can be represented as follows:
ledges. push_back({1,2));
ledges. push_back({2,3));
edges push_back({2,4});
edges .push_back({3,4}) ;
edges. push_back({4,1});
If the graph is weighted, the structure can be extended as follows:
115
Scanned with CamScannervector> edges;
Each element in this list is of the form (a,b,w), which means that there is an
edge from node a to node 6 with weight w. For example, the graph
7
Q (2)
6 5
@)
5
can be represented as follows":
ledges. push_back({1,2,5));
ledges push_back({2,3,7));
edges. push_back({2,4,6));
‘edges. push_back({3,4,5));
edges .push_back({4,1,2));
Tin some older compilers, the function make_tuple must be used instead of the braces (for
example, nake_tuple(1,2,5) instead of (1,2,5))
116
Scanned with CamScannerChapter 12
Graph traversal
This chapter discusses two fundamental graph algorithms: depth-first search and
breadth-first search. Both algorithms are given a starting node in the graph, and
they visit all nodes that can be reached from the starting node. The difference in
the algorithms is the order in which they visit the nodes.
Depth-first search
Depth-first search (DFS) is a straightforward graph traversal technique. The
algorithm begins at a starting node, and proceeds to all other nodes that are
reachable from the starting node using the edges of the graph.
Depth-first search always follows a single path in the graph as long as it
finds new nodes. After this, it returns to previous nodes and begins to explore
other parts of the graph. The algorithm keeps track of visited nodes, so that it
processes each node only once.
Example
Let us consider how depth-first search processes the following graph:
We may begin the search at any node of the graph; now we will begin the search
at node 1
The search first proceeds to node 2:
Scanned with CamScannerAfter this, nodes 3 and 5 will be visited:
@—-@
{>
® ©
‘The neighbors of node 5 are 2 and 3, but the search has already visited both of
them, so it is time to return to the previous nodes. Also the neighbors of nodes 3
and 2 have been visited, so we next move from node 1 to node 4:
@—®
®
® ©
After this, the search terminates because it has visited all nodes.
‘The time complexity of depth-first search is O(n +m) where n is the number
of nodes and m is the number of edges, because the algorithm processes each
node and edge once.
Implementation
Depth-first search can be conveniently implemented using recursion. ‘The fol-
lowing function dfs begins a depth-first search at a given node. The function
assumes that the graph is stored as adjacency lists in an array
vector adiIN];
and also maintains an array
bool visitedtN];
that keeps track of the visited nodes. Initially, each array value is false, and
when the search arrives at node s, the value of visited{s] becomes true. The
function can be implemented as follows:
void dfs(int s) ¢
if (visitedfs}) return;
visited[s] = true;
// process node s
for (auto u: adjls]) €
dfs(u);
3
118
Scanned with CamScannerBreadth-first search
Breadth-first search (BFS) visits the nodes in increasing order of their distance
from the starting node. Thus, we can calculate the distance from the starting
node to all other nodes using breadth-first search. However, breadth-first search
is more difficult to implement than depth-first search.
Breadth-first search goes through the nodes one level after another. First the
search explores the nodes whose distance from the starting node is 1, then the
nodes whose distance is 2, and so on. This process continues until all nodes have
been visited.
Example
Let us consider how breadth-first search processes the following graph:
Q) 2
‘Suppose that the search begins at node 1. First, we process all nodes that can be
reached from node 1 using a single edge:
@
o
After this, we proceed to nodes 3 and 5:
Finally, we visit node 6:
Scanned with CamScannerNow we have calculated the distances from the starting node to all nodes of the
graph. The distances are as follows:
node distance
0
eapewn|
ewHwe
Like in depth-first search, the time complexity of breadth-first search is
O(n+m), where n is the number of nodes and m is the number of edges.
Implementation
Breadth-first search is more difficult to implement than depth-first search, be-
cause the algorithm visits nodes in different parts of the graph. A typical imple-
mentation is based on a queue that contains nodes. At each step, the next node
in the queue will be processed.
‘The following code assumes that the graph is stored as adjacency lists and
maintains the following data structures:
quevesint> a;
bool visited[N];
int distance(N];
‘The queue q contains nodes to be processed in increasing order of their
distance. New nodes are always added to the end of the queue, and the node at
the beginning of the queue is the next node to be processed. The array visited
indicates which nodes the search has already visited, and the array distance will
contain the distances from the starting node to all nodes of the graph.
‘The search can be implemented as follows, starting at node x:
visitedDd =
distance(x] = @
@.push(s) 5
while Cla.empty())
int 5 = @.front(); @.pop(s
/1 process node s
for (auto u: adifs)) {
if (visited[u]) continue;
visited[u] = true;
distancefu) = distance(s}+1;
q.push(u);
120
Scanned with CamScannerApplications
Using the graph traversal algorithms, we can check many properties of graphs.
Usually, both depth-first search and breadth-first search may be used, but in
practice, depth-first search is a better choice, because it is easier to implement.
In the following applications we will assume that the graph is undirected
Connectivity check
A graph is connected if there is a path between any two nodes of the graph. Thus,
we can check if a graph is connected by starting at an arbitrary node and finding
out if we can reach all other nodes.
For example, in the graph
P|
a depth-first search from node 1 visits the following nodes:
ST
Since the search did not visit all the nodes, we can conclude that the graph
is not connected. In a similar way, we can also find all connected components of
a graph by iterating through the nodes and always starting a new depth-first
search if the current node does not belong to any component yet.
Finding cycles
A graph contains a cycle if during a graph traversal, we find a node whose
neighbor (other than the previous node in the current path) has already been
visited. For example, the graph
contains two cycles and we can find one of them as follows:
121
Scanned with CamScanner@ (2)
®
@ ©
After moving from node 2 to node 5 we notice that the neighbor 3 of node 5 has
already been visited. Thus, the graph contains a cycle that goes through node 3,
for example, 3—+2—5 —3.
Another way to find out whether a graph contains a cycle is to simply calculate
the number of nodes and edges in every component. If a component contains ¢
nodes and no eycle, it must contain exactly c~ 1 edges (so it has to be a tree). If
there are c or more edges, the component surely contains a cycle.
Bipartiteness check
A graph is bipartite if its nodes can be colored using two colors so that there are
no adjacent nodes with the same color. It is surprisingly easy to check if'a graph
is bipartite using graph traversal algorithms.
The idea is to color the starting node blue, all its neighbors red, all their
neighbors blue, and so on. If at some point of the search we notice that two
adjacent nodes have the same color, this means that the graph is not bipartite.
Otherwise the graph is bipartite and one coloring has been found.
For example, the graph
Q) 2
@—®
is not bipartite, because a search from node 1 proceeds as follows:
We notice that the color or both nodes 2 and 5 is red, while they are adjacent,
nodes in the graph. Thus, the graph is not bipartite,
This algorithm always works, because when there are only two colors avail-
able, the color of the starting node in a component determines the colors of all
other nodes in the component. It does not make any difference whether the
starting node is red or blue.
Note that in the general case, it is difficult to find out if the nodes in a graph
can be colored using k colors so that no adjacent nodes have the same color. Even
when k =8, no efficient algorithm is known but the problem is NP-hard.
122
Scanned with CamScannerChapter 13
Shortest paths
Finding a shortest path between two nodes of a graph is an important problem
that has many practical applications. For example, a natural problem related to
a road network is to calculate the shortest possible length of a route between two
cities, given the lengths of the roads.
In an unweighted graph, the length of a path equals the number of its edges,
and we can simply use breadth-first search to find a shortest path. However, in
this chapter we focus on weighted graphs where more sophisticated algorithms
are needed for finding shortest paths.
Bellman-Ford algorithm
‘The Bellman-Ford algorithm! finds shortest paths from a starting node to all
nodes of the graph. The algorithm can process all kinds of graphs, provided that
the graph does not contain a cycle with negative length. If the graph contains a
negative cycle, the algorithm can detect this.
‘The algorithm keeps track of distances from the starting node to all nodes
of the graph, Initially, the distance to the starting node is 0 and the distance to
all other nodes in infinite. The algorithm reduces the distances by finding edges
that shorten the paths until it is not possible to reduce any distance.
Example
Let us consider how the Bellman-Ford algorithm works in the following graph:
0
Qy
"The algorithm is named after R. E. Bellman and L. R. Ford who published it independently
in 1958 and 1956, respectively (5, 24),
123
Scanned with CamScannerEach node of the graph is assigned a distance. Initially, the distance to the
starting node is 0, and the distance to all other nodes is infinite.
The algorithm searches for edges that reduce distances. First, all edges from
node 1 reduce distances:
After this, edges 2— 5 and 3 —4 reduce distances:
5
2
9 5
S
Of
%
Finally, there is one more change:
After this, no edge can reduce any distance. This means that the distances
are final, and we have successfully calculated the shortest distances from the
starting node to all nodes of the graph.
For example, the shortest distance 3 from node 1 to node 5 corresponds to the
following path:
124
Scanned with CamScannerImplementation
‘The following implementation of the Bellman—Ford algorithm determines the
shortest distances from a node x to all nodes of the graph. The code assumes
that the graph is stored as an edge list edges that consists of tuples of the form
(a,b,w), meaning that there is an edge from node a to node b with weight w
‘The algorithm consists of n- 1 rounds, and on each round the algorithm goes
through all edges of the graph and tries to reduce the distances. The algorithm
constructs an array distance that will contain the distances from x to all nodes
of the graph. The constant INF denotes an infinite distance.
for (int i n; itt) distance[i] = INF;
distance(x]
for Gint i Lend; in) (
for (auto e : edges) (
int a, by
tie(a, b, w) =e:
distancefb] = min(distance(b], distancelal+w) ;
d
The time complexity of the algorithm is O(nm), because the algorithm consists
of n~1 rounds and iterates through all m edges during a round. If there are no
negative cycles in the graph, all distances are final after n—1 rounds, because
each shortest path can contain at most n ~1 edges.
In practice, the final distances can usually be found faster than in n—1 rounds.
Thus, a possible way to make the algorithm more efficient is to stop the algorithm,
if no distance can be reduced during a round,
Negative cycles
‘The Bellman-Ford algorithm can also be used to check if the graph contains a
cycle with negative length. For example, the graph
contains a negative cycle 2— 3 — 4 — 2 with length ~4.
If the graph contains a negative cycle, we can shorten infinitely many times
any path that contains the cycle by repeating the cycle again and again. Thus,
the concept of a shortest path is not meaningful in this situation.
A negative cycle can be detected using the Bellman—Ford algorithm by running
the algorithm for n rounds. If the last round reduces any distance, the graph
contains a negative cycle. Note that this algorithm can be used to search for a
negative cycle in the whole graph regardless of the starting node.
125
Scanned with CamScannerSPFA algorithm
The SPFA algorithm ("Shortest Path Faster Algorithm”) [20] is a variant of the
Bellman-Ford algorithm, that is often more efficient than the original algorithm.
‘The SPFA algorithm does not go through all the edges on each round, but instead,
it chooses the edges to be examined in a more intelligent way.
‘The algorithm maintains a queue of nodes that might be used for reducing
the distances. First, the algorithm adds the starting node x to the queue. Then,
the algorithm always processes the first node in the queue, and when an edge
ab reduces a distance, node b is added to the queue.
‘The efficiency of the SPFA algorithm depends on the structure of the graph
the algorithm is often efficient, but its worst case time complexity is still O(nm)
and it is possible to create inputs that make the algorithm as slow as the original
Bellman-Ford algorithm.
Dijkstra’s algorithm
Dijkstra’s algorithm” finds shortest paths from the starting node to all nodes of
the graph, like the Bellman-Ford algorithm. The benefit of Dijsktra’s algorithm
is that it is more efficient and can be used for processing large graphs. However,
the algorithm requires that there are no negative weight edges in the graph.
Like the Bellman-Ford algorithm, Dijkstra'’s algorithm maintains distances
to the nodes and reduces them during the search. Dijkstra’s algorithm is efficient,
because it only processes each edge in the graph once, using the fact that there
are no negative edges.
Example
Let us consider how Dijkstra’s algorithm works in the following graph when the
starting node is node 1:
Like in the Bellman-Ford algorithm, initially the distance to the starting node is
O and the distance to all other nodes is infinite.
At each step, Dijkstra’s algorithm selects a node that has not been processed
yet and whose distance is as small as possible. The first such node is node 1 with
distance 0.
W. Dijkstra published the algorithm in 1959 [14); however, his original paper does not
mention how to implement the algorithm efficient
126
Scanned with CamScannerWhen a node is selected, the algorithm goes through all edges that start at
the node and reduces the distances using them:
In this case, the edges from node 1 reduced the distances of nodes 2, 4 and 5,
whose distances are now 5, 9 and 1.
‘The next node to be processed is node 5 with distance 1. This reduces the
distance to node 4 from 9 to 3:
A remarkable property in Dijkstra’s algorithm is that whenever a node is,
selected, its distance is final. For example, at this point of the algorithm, the
distances 0, 1 and 3 are the final distances to nodes 1, 5 and 4.
After this, the algorithm processes the two remaining nodes, and the final
distances are as follows:
127
Scanned with CamScannerNegative edges
‘The efficiency of Dijkstra’s algorithm is based on the fact that the graph does
not contain negative edges. If there is a negative edge, the algorithm may give
incorrect results. As an example, consider the following graph:
‘The shortest path from node 1 to node 4 is 1-- 3— 4 and its length is 1. However,
Dijkstra's algorithm finds the path 1 2— 4 by following the minimum weight
edges. The algorithm does not take into account that on the other path, the
weight -5 compensates the previous large weight 6.
Implementation
‘The following implementation of Dijkstra’s algorithm calculates the minimum.
distances from a node x to other nodes of the graph. The graph is stored as
adjacency lists so that adj[a] contains a pair (b,w) always when there is an edge
from node a to node 6 with weight w
An efficient implementation of Dijkstra’s algorithm requires that it is possible
to efficiently find the minimum distance node that has not been processed. An
appropriate data structure for this is a priority queue that contains the nodes
ordered by their distances. Using a priority queue, the next node to be processed
can be retrieved in logarithmic time.
In the following code, the priority queue q contains pairs of the form (d,x),
meaning that the current distance to node x is d. The array distance contains
the distance to each node, and the array processed indicates whether a node has
been processed. Initially the distance is 0 to x and co to all other nodes.
for (int
distancetx]
q.push((,x});
while Clq.empty())
int a = q.top().second; 9.pop();
If (processedta]) con
processedial
for (auto uw: adja) {
int b = u.first, w = u.second;
if Gdistancetal+w < distanceLb]) (
distencefb] = distanceLa]+w;
4g. push({-distancefb],b)) ;
it+) distancefi] = INF;
128
Scanned with CamScannerNote that the priority queue contains negative distances to nodes. The reason
for this is that the default version of the C++ priority queue finds maximum
elements, while we want to find minimum elements. By using negative distances,
we can directly use the default priority queue®, Also note that there may be
several instances of the same node in the priority queue; however, only the
instance with the minimum distance will be processed
‘The time complexity of the above implementation is O(n +m logm), because
the algorithm goes through all nodes of the graph and adds for each edge at most
one distance to the priority queue.
Floyd-Warshall algorithm
The Floyd-Warshall algorithm’ provides an alternative way to approach the
problem of finding shortest paths. Unlike the other algorithms of this chapter, it
finds all shortest paths between the nodes in a single run.
‘The algorithm maintains a two-dimensional array that contains distances
between the nodes. First, distances are calculated only using direct edges between
the nodes, and after this, the algorithm reduces distances by using intermediate
nodes in paths
Example
Let us consider how the Floyd—Warshall algorithm works in the following graph:
Initially, the distance from each node to itself is 0, and the distance between
nodes a and 6 is x if there is an edge between nodes a and 6 with weight x. All
other distances are infinite.
In this graph, the initial array is as follows:
{1 2 3 4 5
T)0 5o 9 I
2,55 0 2 w ow
30 2 0 7 &
4,90 7 0 2
5) 1 wo 2 0
OF course, we could also declare the priority queue as in Chapter 4.5 and use positive distances,
but the implementation would be a bit longer.
“The algorithm is named after R. W. Floyd and S. Warshall who published it independently in
1962 [23, 70}
129
Scanned with CamScanner‘The algorithm consists of consecutive rounds. On each round, the algorithm
selects a new node that can act as an intermediate node in paths from now on,
and distances are reduced using this node.
On the first round, node 1 is the new intermediate node. There is a new path
between nodes 2 and 4 with length 14, because node 1 connects them. There is
also a new path between nodes 2 and 5 with length 6.
123 45
i/o 50 9 1
2/5 0 214 6
Bfoo 2 0 7 &
4/9 14 7 0 2
5] 1 6 «@ 2 0
On the second round, node 2 is the new intermediate node. This creates new
paths between nodes 1 and 3 and between nodes 3 and 5:
On the third round, node 3 is the new intermediate round. ‘There is a new
path between nodes 2 and 4:
‘The algorithm continues like this, until all nodes have been appointed inter-
mediate nodes, After the algorithm has finished, the array contains the minimum
distances between any two nodes:
ork et
eae oc ale
wosamola
cr wmana
For example, the array tells us that the shortest distance between nodes 2
and 4 is 8. This corresponds to the following path:
130
Scanned with CamScanner