4 String Matching –

What’s behind Ctrl+F?

Sebastian Wild
24 February 2020

version 2020-03-03 14:31

Outline

4 String Matching
4.1 Introduction
4.2 Brute Force
4.3 String Matching with Finite Automata
4.4 The Knuth-Morris-Pratt algorithm
4.5 Beyond Optimal? The Boyer-Moore Algorithm
4.6 The Rabin-Karp Algorithm
4.1 Introduction
Ubiquitous strings
string = sequence of characters
� universal data type for . . . everything!
� natural language texts
� programs (source code)
� websites
� XML documents
� DNA sequences
� bitstrings
� . . . a computer’s memory � ultimately any data is a string

� many different tasks and algorithms

1
Ubiquitous strings
string = sequence of characters
� universal data type for . . . everything!
� natural language texts
� programs (source code)
� websites
� XML documents
� DNA sequences
� bitstrings
� . . . a computer’s memory � ultimately any data is a string

� many different tasks and algorithms

� This unit: finding (exact) occurrences of a pattern text.
� Ctrl+F
� grep
� computer forensics (e. g. find signature of file on disk)
� virus scanner

� basis for many advanced applications

1
Notations
� alphabet Σ: finite set of allowed characters; 𝜎 = |Σ| “a string over alphabet Σ”
� letters (Latin, Greek, Arabic, Cyrillic, Asian scripts, . . . )
� “what you can type on a keyboard”, Unicode characters
� {0, 1}; nucleotides {𝐴, 𝐶, 𝐺, 𝑇}; . . . comprehensive standard character set
including emoji and all known symbols

2
Notations
� alphabet Σ: finite set of allowed characters; 𝜎 = |Σ| “a string over alphabet Σ”
� letters (Latin, Greek, Arabic, Cyrillic, Asian scripts, . . . )
� “what you can type on a keyboard”, Unicode characters
� {0, 1}; nucleotides {𝐴, 𝐶, 𝐺, 𝑇}; . . . comprehensive standard character set
including emoji and all known symbols

� Σ𝑛 = Σ × · · · × Σ: strings of length 𝑛 ∈ ℕ0 (𝑛-tuples)

�
� Σ★ = 𝑛≥0 Σ
𝑛: set of all (finite) strings over Σ
�
� Σ+ = 𝑛≥1 Σ
𝑛: set of all (finite) nonempty strings over Σ
� 𝜀 ∈ Σ0 : the empty string (same for all alphabets)

2
Notations
� alphabet Σ: finite set of allowed characters; 𝜎 = |Σ| “a string over alphabet Σ”
� letters (Latin, Greek, Arabic, Cyrillic, Asian scripts, . . . )
� “what you can type on a keyboard”, Unicode characters
� {0, 1}; nucleotides {𝐴, 𝐶, 𝐺, 𝑇}; . . . comprehensive standard character set
including emoji and all known symbols

� Σ𝑛 = Σ × · · · × Σ: strings of length 𝑛 ∈ ℕ0 (𝑛-tuples)

�
� Σ★ = 𝑛≥0 Σ
𝑛: set of all (finite) strings over Σ
�
� Σ+ = 𝑛≥1 Σ
𝑛: set of all (finite) nonempty strings over Σ
� 𝜀 ∈ Σ0 : the empty string (same for all alphabets)
zero-based (like arrays)!

� for 𝑆 ∈ Σ𝑛 , write 𝑆[𝑖] (other sources: 𝑆 𝑖 ) for 𝒊th character (0 ≤ 𝑖 < 𝑛)

� for 𝑆, 𝑇 ∈ Σ★, write 𝑆𝑇 = 𝑆 · 𝑇 for concatenation of 𝑆 and 𝑇
� for 𝑆 ∈ Σ𝑛 , write 𝑆[𝑖..𝑗] or 𝑆 𝑖,𝑗 for the substring 𝑆[𝑖] · 𝑆[𝑖 + 1] · · · 𝑆[𝑗] (0 ≤ 𝑖 ≤ 𝑗 < 𝑛)
� 𝑆[0..𝑗] is a prefix of 𝑆; 𝑆[𝑖..𝑛 − 1] is a suffix of 𝑆
� 𝑆[𝑖..𝑗) = 𝑆[𝑖..𝑗 − 1] (endpoint exclusive) � 𝑆 = 𝑆[0..𝑛)
2
Clicker Question

True or false: Σ★ = Σ+ ∪ {𝜀}

A True B False

pingo.upb.de/622222
3
Clicker Question

True or false: Σ★ = Σ+ ∪ {𝜀}

A True � B False

pingo.upb.de/622222
3
String matching – Definition
Search for a string (pattern) in a large body of text
� Input:
� 𝑇 ∈ Σ𝑛 : The text (haystack) being searched within
� 𝑃 ∈ Σ𝑚 : The pattern (needle) being searched for; typically 𝑛 � 𝑚

� Output:
� �
� the first occurrence (match) of 𝑃 in 𝑇: min 𝑖 ∈ [0..𝑛 − 𝑚) : 𝑇[𝑖..𝑖 + 𝑚) = 𝑃
� or NO_MATCH if there is no such 𝑖 (“𝑃 does not occur in 𝑇”)

� Variant: Find all occurrences of 𝑃 in 𝑇.

� Can do that iteratively (update 𝑇 to 𝑇[𝑖 + 1..𝑛) after match at 𝑖)

� Example:
� 𝑇 = “Where is he?”
� 𝑃1 = “he” � 𝑖 = 1
� 𝑃2 = “who” � NO_MATCH

� string matching is implemented in Java in String.indexOf

4
4.2 Brute Force
Abstract idea of algorithms
Pattern matching algorithms consist of guesses and checks:
� A guess is a position 𝑖 such that 𝑃 might start at 𝑇[𝑖].
Possible guesses (initially) are 0 ≤ 𝑖 ≤ 𝑛 − 𝑚.
� A check of a guess is a pair (𝑖, 𝑗) where we compare 𝑇[𝑖 + 𝑗] to 𝑃[𝑗].

� Note: need all 𝑚 checks to verify a single correct guess 𝑖,

but it may take (many) fewer checks to recognize an incorrect guess.

� Cost measure: #character comparisons = #checks

� cost ≤ 𝑛 · 𝑚 (number of possible checks)

5
Brute-force method

1 procedure bruteForceSM(𝑇[0..𝑛), 𝑃[0..𝑚)) � try all guesses 𝑖

2 for 𝑖 := 0, . . . , 𝑛 − 𝑚 − 1 do
3 for 𝑗 := 0, . . . , 𝑚 − 1 do � check each guess (left to right);
4 if 𝑇[𝑖 + 𝑗] ≠ 𝑃[𝑗] then break inner loop stop early on mismatch
5 if 𝑗 == 𝑚 then return 𝑖 � essentially the implementation
return NO_MATCH
in Java!
6

a b b b a b a b b a b

� Example:
𝑇 = abbbababbab
𝑃 = abba

� 15 char cmps
(vs 𝑛 · 𝑚 = 44)
not too bad!

6
Brute-force method

1 procedure bruteForceSM(𝑇[0..𝑛), 𝑃[0..𝑚)) � try all guesses 𝑖

2 for 𝑖 := 0, . . . , 𝑛 − 𝑚 − 1 do
3 for 𝑗 := 0, . . . , 𝑚 − 1 do � check each guess (left to right);
4 if 𝑇[𝑖 + 𝑗] ≠ 𝑃[𝑗] then break inner loop stop early on mismatch
5 if 𝑗 == 𝑚 then return 𝑖 � essentially the implementation
return NO_MATCH
in Java!
6

a b b b a b a b b a b
a b b a
� Example:
a
𝑇 = abbbababbab
a
𝑃 = abba
a
� 15 char cmps a b b
(vs 𝑛 · 𝑚 = 44) a
not too bad! a b b a

6
Brute-force method – Discussion
Brute-force method can be good enough
� typically works well for natural language text
� also for random strings

but: can be as bad as it gets!

a a a a a a a a a a a
a a a b
a a a b
� Worst possible input: 𝑃 = 𝑎 𝑚−1 𝑏,
a a a b
𝑇 = 𝑎𝑛
a a a b
� Worst-case performance: (𝑛 − 𝑚 + 1) · 𝑚
a a a b
a a a b � for 𝑚 ≤ 𝑛/2 that is Θ(𝑚𝑛)
a a a b
a a a b

7
Brute-force method – Discussion
Brute-force method can be good enough
� typically works well for natural language text
� also for random strings

but: can be as bad as it gets!

a a a a a a a a a a a
a a a b
a a a b
� Worst possible input: 𝑃 = 𝑎 𝑚−1 𝑏,
a a a b
𝑇 = 𝑎𝑛
a a a b
� Worst-case performance: (𝑛 − 𝑚 + 1) · 𝑚
a a a b
a a a b � for 𝑚 ≤ 𝑛/2 that is Θ(𝑚𝑛)
a a a b
a a a b

� Bad input: lots of self-similarity in 𝑇! � can we exploit that?

� brute force does ‘obviously’ stupid repetitive comparisons � can we avoid that?
7
Roadmap
� Approach 1 (this week): Use preprocessing on the pattern 𝑃 to eliminate guesses
(avoid ‘obvious’ redundant work)
� Deterministic finite automata (DFA)
� Knuth-Morris-Pratt algorithm
� Boyer-Moore algorithm
� Rabin-Karp algorithm

� Approach 2 (� Unit 6): Do preprocessing on the text 𝑇

Can find matches in time independent of text size(!)
� inverted indices
� Suffix trees
� Suffix arrays

8
4.3 String Matching with Finite Automata
Clicker Question

Do you know what regular expressions, NFAs and DFAs are,

and how to convert between them?
A Never heard of this; are these new emoji?

B Heard the terms, but don’t remember conversion

methods.

C Had that in my undergrad course (memories fading a

bit).

D Sure, I could do that blindfolded!

pingo.upb.de/622222
9
Theoretical Computer Science to the rescue!
� string matching = deciding whether 𝑇 ∈ Σ★ · 𝑃 · Σ★

� Σ★ · 𝑃 · Σ★ is regular formal language

� ∃ deterministic finite automaton (DFA) to recognize Σ★ · 𝑃 · Σ★

� can check for occurrence of 𝑃 in |𝑇 | = 𝑛 steps!

10
Theoretical Computer Science to the rescue!
� string matching = deciding whether 𝑇 ∈ Σ★ · 𝑃 · Σ★

� Σ★ · 𝑃 · Σ★ is regular formal language

� ∃ deterministic finite automaton (DFA) to recognize Σ★ · 𝑃 · Σ★

� can check for occurrence of 𝑃 in |𝑇 | = 𝑛 steps!

Job done!

10
Theoretical Computer Science to the rescue!
� string matching = deciding whether 𝑇 ∈ Σ★ · 𝑃 · Σ★

� Σ★ · 𝑃 · Σ★ is regular formal language

� ∃ deterministic finite automaton (DFA) to recognize Σ★ · 𝑃 · Σ★

� can check for occurrence of 𝑃 in |𝑇 | = 𝑛 steps!

Job done! WTF!?

10
Theoretical Computer Science to the rescue!
� string matching = deciding whether 𝑇 ∈ Σ★ · 𝑃 · Σ★

� Σ★ · 𝑃 · Σ★ is regular formal language

� ∃ deterministic finite automaton (DFA) to recognize Σ★ · 𝑃 · Σ★

� can check for occurrence of 𝑃 in |𝑇 | = 𝑛 steps!

Job done! WTF!?

We are not quite done yet.

� (Problem 0: programmer might not know automata and formal languages . . . )

� Problem 1: existence alone does not give an algorithm!

� Problem 2: automaton could be very big!

10
String matching with DFA
� Assume first, we already have a deterministic automaton
� How does string matching work?

Example:
𝑇 = aabacaababacaa
a
𝑃 = ababaca
b,c a a Σ
b
a b a b a c a
0 1 2 3 4 5 6 7
c b,c
c
b,c
b,c

text: a a b a c a a b a b a c a a
state: 0
11
String matching with DFA
� Assume first, we already have a deterministic automaton
� How does string matching work?

Example:
𝑇 = aabacaababacaa
a
𝑃 = ababaca
b,c a a Σ
b
a b a b a c a
0 1 2 3 4 5 6 7
c b,c
c
b,c
b,c

text: a a b a c a a b a b a c a a
state: 0 1 1 2 3 0 1 1 2 3 4 5 6 7 7
11
String matching DFA – Intuition
Why does this work?
𝑇 = aabacaababacaa
a
𝑃 = ababaca
b,c a a Σ
� Main insight: a b a b
b
a c a
0 1 2 3 4 5 6 7
State 𝑞 means: c b,c
c
“we have seen 𝑃[0..𝑞) until here b,c
b,c
(but not any longer prefix of 𝑃)”

text: a a b a c a a b a b a c a a
state: 0 1 1 2 3 0 1 1 2 3 4 5 6 7 7

� If the next text character 𝑐 does not match, we know:

(i) text seen so far ends with 𝑃[0...𝑞) · 𝑐
(ii) 𝑃[0...𝑞) · 𝑐 is not a prefix of 𝑃
(iii) without reading 𝑐, 𝑃[0..𝑞) was the longest prefix of 𝑃 that ends here.

� New longest matched prefix will be (weakly) shorter than 𝑞

� All information about the text needed to determine it is contained in 𝑃[0...𝑞) · 𝑐!

12
NFA instead of DFA?
It remains to construct the DFA.
Σ Σ

a b a b a c a
� trivial part: 0 1 2 3 4 5 6 7

13
NFA instead of DFA?
It remains to construct the DFA.
Σ Σ

a b a b a c a
� trivial part: 0 1 2 3 4 5 6 7

� that actually is a nondeterministic finite automaton (NFA) for Σ★ 𝑃 Σ★

� We could use the NFA directly for string matching:

� at any point in time, we are in a set of states
� accept when one of them is final state

Example:
Previous versions of this example were missing states; this is the correct version:
text: a a b a c a a b a b a c a a
state: 0 0, 1 0, 1 0, 2 0, 1, 3 0 0, 1 0, 1 0, 2 0, 1, 3 0, 2, 4 0, 1, 3, 5 0, 6 0, 1, 7 0, 1, 7

But maintaining a whole set makes this slow . . .

13
Computing DFA directly
You have an NFA and want a DFA?
Simply apply the power-set construction
(and maybe DFA minimization)!

The powerset method has exponential state blow up!

I guess I might as well use brute force ...

14
Computing DFA directly
You have an NFA and want a DFA?
Simply apply the power-set construction
(and maybe DFA minimization)!

The powerset method has exponential state blow up!

I guess I might as well use brute force ...

Ingenious algorithm by Knuth, Morris, and Pratt: construct DFA inductively:

Suppose we add character 𝑃[𝑗] to automaton 𝐴 𝑗−1 for 𝑃[0..𝑗 − 1]
� add new state and matching transition � easy
� for each 𝑐 ≠ 𝑃[𝑗], we need 𝛿(𝑗, 𝑐) (transition from 𝑗 when reading 𝑐)

14
Computing DFA directly
You have an NFA and want a DFA?
Simply apply the power-set construction
(and maybe DFA minimization)!

The powerset method has exponential state blow up!

I guess I might as well use brute force ...

Ingenious algorithm by Knuth, Morris, and Pratt: construct DFA inductively:

Suppose we add character 𝑃[𝑗] to automaton 𝐴 𝑗−1 for 𝑃[0..𝑗 − 1]
� add new state and matching transition � easy
� for each 𝑐 ≠ 𝑃[𝑗], we need 𝛿(𝑗, 𝑐) (transition from 𝑗 when reading 𝑐)
� 𝛿(𝑗, 𝑐) = length of the longest prefix of 𝑃[0..𝑗)𝑐 that is a suffix of 𝑃[1..𝑗)𝑐
= state of automaton after reading 𝑃[1..𝑗)𝑐
≤ 𝑗 � can use known automaton 𝐴 𝑗−1 for that! State 𝑞 means:
“we have seen 𝑃[0..𝑞) until here
� can directly compute 𝐴 𝑗 from 𝐴 𝑗−1 ! (but not any longer prefix of 𝑃)”

seems to require simulating automata 𝑚 · 𝜎 times

14
Computing DFA efficiently
� KMP’s second insight: simulations in one step differ only in last symbol

� simply maintain state 𝑥, the state after reading 𝑃[1..𝑗 − 1].

� copy its transitions
� update 𝑥 by following transitions for 𝑃[𝑗]

Demo: Algorithms videos of Sedgewick and Wayne

https://cuvids.io/app/video/194/watch
15
String matching with DFA – Discussion
� Time:
� Matching: 𝑛 table lookups for DFA transitions
� building DFA: Θ(𝑚𝜎) time (constant time per transition edge).
� Θ(𝑚𝜎 + 𝑛) time for string matching.

� Space:
� Θ(𝑚𝜎) space for transition matrix.

fast matching time actually: hard to beat!

total time asymptotically optimal for small alphabet (for 𝜎 = 𝑂(𝑛/𝑚))

substantial space overhead, in particular for large alphabets

16
4.4 The Knuth-Morris-Pratt algorithm
Failure Links
� Recall: String matching with is DFA fast,
but needs table of 𝑚 × 𝜎 transitions.
� in fast DFA construction, we used that all simulations differ only by last symbol

� KMP’s third insight: do this last step of simulation from state 𝑥 during matching!
. . . but how?

17
Failure Links
� Recall: String matching with is DFA fast,
but needs table of 𝑚 × 𝜎 transitions.
� in fast DFA construction, we used that all simulations differ only by last symbol

� KMP’s third insight: do this last step of simulation from state 𝑥 during matching!
. . . but how?

� Answer: Use a new type of transition, the failure links

� Use this transition (only) if no other one fits.
� × does not consume a character. � might follow several failure links
Σ−𝑎 × × Σ

a b a b a c a
0 1 2 3 4 5 6 7
× × ×
×

� Computations are deterministic (but automaton is not a real DFA.)

17
Failure link automaton – Example
Example: 𝑇 = abababaaaca, 𝑃 = ababaca

Σ−𝑎 × × Σ
a b a b a c a
0 1 2 3 4 5 6 7
× × ×
×

𝑇: a b a b a b a a b a b

18
Failure link automaton – Example
Example: 𝑇 = abababaaaca, 𝑃 = ababaca

Σ−𝑎 × × Σ
a b a b a c a
0 1 2 3 4 5 6 7
× × ×
×

𝑇: a b a b a b a a b a b
𝑃: a b a b a × to state 3
(a) (b) (a) b a × to state 1
a b a b

𝑞: 1 2 3 4 5 3, 4 5 3, 1, 0, 1 2 3 4
(after reading this character)

18
Clicker Question

What is the worst-case time to process one character in a failure-

link automaton for 𝑃[0..𝑚)?

A Θ(1) C Θ(𝑚)

B Θ(log 𝑚) D Θ(𝑚 2 )

pingo.upb.de/622222
19
Clicker Question

What is the worst-case time to process one character in a failure-

link automaton for 𝑃[0..𝑚)?

A Θ(1) C Θ(𝑚) �
B Θ(log 𝑚) D Θ(𝑚 2 )

pingo.upb.de/622222
19
The Knuth-Morris-Pratt Algorithm
1 procedure KMP(𝑇[0..𝑛 − 1], 𝑃[0..𝑚 − 1]) � only need single array fail for
fail[0..𝑚] := failureLinks(𝑃)
failure links
2

3 𝑖 := 0 // current position in 𝑇
4 𝑞 := 0 // current state of KMP automaton � (procedure failureLinks later)
5 while 𝑖 < 𝑛 do
6 if 𝑇[𝑖] == 𝑃[𝑞] then
7 𝑖 := 𝑖 + 1; 𝑞 := 𝑞 + 1
8 if 𝑞 == 𝑚 then
9 return 𝑖 − 𝑞 // occurrence found
10 else // i.e. 𝑇[𝑖] ≠ 𝑃[𝑞]
11 if 𝑞 ≥ 1 then
12 𝑞 := fail[𝑞] // follow one ×
13 else
14 𝑖 := 𝑖 + 1
15 end while
16 return NO_MATCH

20
The Knuth-Morris-Pratt Algorithm
1 procedure KMP(𝑇[0..𝑛 − 1], 𝑃[0..𝑚 − 1]) � only need single array fail for
fail[0..𝑚] := failureLinks(𝑃)
failure links
2

3 𝑖 := 0 // current position in 𝑇
4 𝑞 := 0 // current state of KMP automaton � (procedure failureLinks later)
5 while 𝑖 < 𝑛 do
6 if 𝑇[𝑖] == 𝑃[𝑞] then Analysis: (matching part)
7 𝑖 := 𝑖 + 1; 𝑞 := 𝑞 + 1
� always have fail[𝑗] < 𝑗 for 𝑗 ≥ 1
8 if 𝑞 == 𝑚 then
9 return 𝑖 − 𝑞 // occurrence found � in each iteration
10 else // i.e. 𝑇[𝑖] ≠ 𝑃[𝑞] � either advance position in text
11 if 𝑞 ≥ 1 then (𝑖 := 𝑖 + 1)
12 𝑞 := fail[𝑞] // follow one × � or shift pattern forward
13 else (guess 𝑖 − 𝑗)
14 𝑖 := 𝑖 + 1
� each can happen at most 𝑛 times
15 end while
16 return NO_MATCH � ≤ 2𝑛 symbol comparisons!

20
Computing failure links
� failure links point to error state 𝑥 (from DFA construction)

� run same algorithm, but store fail[𝑗] := 𝑥 instead of copying all transitions

1 procedure failureLinks(𝑃[0..𝑚 − 1])

2 fail[0] := 0
3 𝑥 := 0
4 for 𝑗 := 1, . . . , 𝑚 − 1 do
5 fail[𝑗] := 𝑥
6 // update failure state using failure links:
7 while 𝑃[𝑥] ≠ 𝑃[𝑗]
8 if 𝑥 == 0 then
9 𝑥 := −1; break
10 else
11 𝑥 := fail[𝑥]
12 end while
13 𝑥 := 𝑥 + 1
14 end for
21
Computing failure links
� failure links point to error state 𝑥 (from DFA construction)

� run same algorithm, but store fail[𝑗] := 𝑥 instead of copying all transitions

1 procedure failureLinks(𝑃[0..𝑚 − 1])

2 fail[0] := 0
3 𝑥 := 0
Analysis:
4 for 𝑗 := 1, . . . , 𝑚 − 1 do
� 𝑚 iterations of for loop
5 fail[𝑗] := 𝑥
6 // update failure state using failure links: � while loop always decrements 𝑥
7 while 𝑃[𝑥] ≠ 𝑃[𝑗]
8 if 𝑥 == 0 then � 𝑥 is incremented only once per
9 𝑥 := −1; break iteration of for loop
10 else
� ≤ 𝑚 iterations of while loop in total
11 𝑥 := fail[𝑥]
12 end while � ≤ 2𝑚 symbol comparisons
13 𝑥 := 𝑥 + 1
14 end for
21
Knuth-Morris-Pratt – Discussion
� Time:
� ≤ 2𝑛 + 2𝑚 = 𝑂(𝑛 + 𝑚) character comparisons
� clearly must at least read both 𝑇 and 𝑃
� KMP has optimal worst-case complexity!

� Space:
� Θ(𝑚) space for failure links

total time asymptotically optimal (for any alphabet size)

reasonable extra space

22
Clicker Question

What are the main advantages of the KMP string matching (using
the failure-link automaton) over string matching with DFAs?
Check all that apply.
A faster preprocessing on pattern

B faster matching in text

C fewer character comparisons

D uses less space

E makes running time independent of 𝜎

F I don’t have to do automata theory

pingo.upb.de/622222
23
Clicker Question

What are the main advantages of the KMP string matching (using
the failure-link automaton) over string matching with DFAs?
Check all that apply.
A faster preprocessing on pattern �
B faster matching in text

C fewer character comparisons

D uses less space �

E makes running time independent of 𝜎 �
F I don’t have to do automata theory

pingo.upb.de/622222
23
The KMP prefix function
� It turns out that the failure links are useful beyond KMP

� a slight variation is more widely used: (for historic reasons)

the (KMP) prefix function 𝐹 : [1..𝑚 − 1] → [0..𝑚 − 1]:
𝐹[𝑗] is the length of the longest prefix of 𝑃[0..𝑗]
that is a suffix of 𝑃[1..𝑗].
� Can show: fail[𝑗] = 𝐹[𝑗 − 1] for 𝑗 ≥ 1, and hence

memorize this!
fail[𝑗] = length of the
longest prefix of 𝑃[0..𝑗)
that is a suffix of 𝑃[1..𝑗).

24
4.5 Beyond Optimal? The Boyer-Moore Algorithm
Motivation
� KMP is an optimal algorithm, isn’t it? What else could we hope for?

25
Motivation
� KMP is an optimal algorithm, isn’t it? What else could we hope for?

� KMP is “only” optimal in the worst-case (and up to constant factors)

� how many comparisons do we need for the following instance?

𝑇 = aaaaaaaaaaaaaaaa, 𝑃 = xxxxx
� there are no matches
� we can certify the correctness of that output with only 4 comparisons:

𝑇 a a a a a a a a a a a a a a a a
x
x
x
x

� We did not even read most text characters!

25
Boyer-Moore Algorithm
� Let’s check guesses from right to left!

� If we are lucky, we can eliminate several shifts in one shot!

26
Boyer-Moore Algorithm
� Let’s check guesses from right to left!

� If we are lucky, we can eliminate several shifts in one shot!

must avoid (excessive) redundant checks, e. g., for 𝑇 = 𝑎 𝑛 , 𝑃 = 𝑏𝑎 𝑚−1

� New rules:
� Bad character jumps: Upon mismatch at 𝑇[𝑖] = 𝑐:
� If 𝑃 does not contain 𝑐, shift 𝑃 entirely past 𝑖!
� Otherwise, shift 𝑃 to align the last occurrence of 𝑐 in 𝑃 with 𝑇[𝑖].
� Good suffix jumps:
Upon a mismatch, shift so that the already matched suffix of 𝑃 aligns with a
previous occurrence of that suffix (or part of it) in 𝑃.
(Details follow; ideas similar to KMP failure links)

� two possible shifts (next guesses); use larger jump.

26
Boyer-Moore Algorithm – Code

1 procedure boyerMoore(𝑇[0..𝑛 − 1], 𝑃[0..𝑚 − 1])

2 𝜆 := computeLastOccurrences(𝑃)
3 𝛾 := computeGoodSuffixes(𝑃)
4 𝑖 := 0 // current guess
5 while 𝑖 ≤ 𝑛 − 𝑚 � 𝜆 and 𝛾 explained below
6 𝑗 := 𝑚 − 1 // next position in 𝑃 to check � shift forward is larger of two
7 while 𝑗 ≥ 0 ∧ 𝑃[𝑗] == 𝑇[𝑖 + 𝑗] do heuristics
8 𝑗 := 𝑗 − 1
9 if 𝑗 == −1 then � shift is always positive (see
10 return 𝑖 below)
11 else � �
12 𝑖 := 𝑖 + max 𝑗 − 𝜆[𝑇[𝑖 + 𝑗]], 𝛾[𝑗]
13 return NO_MATCH

27
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
d o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
l d o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e
e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e
e
(r) e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e
e
(r) e
(m) e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e
e
(r) e
(m) r e

28
Bad character examples
𝑃 = a l d o
𝑇 = w h e r e i s w a l d o
o
o
a l d o

� 6 characters not looked at

𝑃 = m o o r e
𝑇 = b o y e r m o o r e
e
(r) e
(m) o o r e

� 4 characters not looked at

28
Last-Occurrence Function
� Preprocess pattern 𝑃 and alphabet Σ
� last-occurrence function 𝜆[𝑐] defined as
� the largest index 𝑖 such that 𝑃[𝑖] = 𝑐 or
� −1 if no such index exists

29
Last-Occurrence Function
� Preprocess pattern 𝑃 and alphabet Σ
� last-occurrence function 𝜆[𝑐] defined as
� the largest index 𝑖 such that 𝑃[𝑖] = 𝑐 or
� −1 if no such index exists

� Example: 𝑃 = moore
𝑃 = m o o r e
𝑐 m o r e all others 𝑇 = b o y e r m o o r e

𝜆[𝑐] 0 2 3 4 −1 e
(r) e

𝑖 = 0, 𝑗 = 4, 𝑇[𝑖 + 𝑗] = 𝑟, 𝜆[𝑟] = 3
� shift by 𝑗 − 𝜆[𝑇[𝑖 + 𝑗]] = 1
� 𝜆 easily computed in 𝑂(𝑚 + |Σ|) time.

� store as array 𝜆[0..𝜎 − 1].

29
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s

30
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s
h e l l s

30
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s
h e l l s
(e) (l) (l) (s)

30
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s
h e l l s
(e) (l) (l) (s)

2. 𝑃 = odetofood
i l i k e f o o d f r o m m e x i c o
o f o o d

30
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s
h e l l s
(e) (l) (l) (s)

2. 𝑃 = odetofood
i l i k e f o o d f r o m m e x i c o
o f o o d
(o) (d)

30
Good suffix examples
1. 𝑃 = sells␣shells
s h e i l a ␣ s e l l s ␣ s h e l l s
h e l l s
(e) (l) (l) (s)

2. 𝑃 = odetofood
i l i k e f o o d f r o m m e x i c o
o f o o d
(o) (d)

matched suffix

� Crucial ingredient: longest suffix of 𝑃[𝑗+1..𝑚−1] that occurs earlier in 𝑃.

� 2 cases (as illustrated above)
1. complete suffix occurs in 𝑃 � characters left of suffix are not known to match
2. part of suffix occurs at beginning of 𝑃
30
Good suffix jumps
� Precompute good suffix jumps 𝛾[0..𝑚 − 1]:
� For 0 ≤ 𝑗 < 𝑚, 𝛾[𝑗] stores shift if search failed at 𝑃[𝑗]
� At this point, had 𝑇[𝑖+𝑗+1..𝑖+𝑚−1] = 𝑃[𝑗+1..𝑚−1], but 𝑇[𝑖] ≠ 𝑃[𝑗]

31
Good suffix jumps
� Precompute good suffix jumps 𝛾[0..𝑚 − 1]:
� For 0 ≤ 𝑗 < 𝑚, 𝛾[𝑗] stores shift if search failed at 𝑃[𝑗]
� At this point, had 𝑇[𝑖+𝑗+1..𝑖+𝑚−1] = 𝑃[𝑗+1..𝑚−1], but 𝑇[𝑖] ≠ 𝑃[𝑗]

� 𝛾[𝑗] is the shift 𝑚 − 1 − ℓ for the largest ℓ such that

� 𝑃[𝑗+1 . . . 𝑚−1] is a suffix of 𝑃[0 . . . ℓ ] and 𝑃[𝑗] ≠ 𝑃[ℓ −𝑚+𝑗+1]
h e l l s
× (e) (l) (l) (s)

–OR–
� 𝑃[0 . . . ℓ ] is a suffix of 𝑃[𝑗+1, . . . , 𝑚−1]
o f o o d
(o) (d)

31
Good suffix jumps
� Precompute good suffix jumps 𝛾[0..𝑚 − 1]:
� For 0 ≤ 𝑗 < 𝑚, 𝛾[𝑗] stores shift if search failed at 𝑃[𝑗]
� At this point, had 𝑇[𝑖+𝑗+1..𝑖+𝑚−1] = 𝑃[𝑗+1..𝑚−1], but 𝑇[𝑖] ≠ 𝑃[𝑗]

� 𝛾[𝑗] is the shift 𝑚 − 1 − ℓ for the largest ℓ such that

� 𝑃[𝑗+1 . . . 𝑚−1] is a suffix of 𝑃[0 . . . ℓ ] and 𝑃[𝑗] ≠ 𝑃[ℓ −𝑚+𝑗+1]
h e l l s
× (e) (l) (l) (s)

–OR–
� 𝑃[0 . . . ℓ ] is a suffix of 𝑃[𝑗+1, . . . , 𝑚−1]
o f o o d
(o) (d)

� Computable (similar to KMP failure function) in Θ(𝑚) time.

� Note: You do not need to know how to find the values 𝛾[𝑗] for the exam,
but you should be able to find the next guess on examples.

31
Boyer-Moore algorithm – Discussion
Worst-case running time ∈ 𝑂(𝑛 + 𝑚 + |Σ|) if 𝑃 does not occur in 𝑇.
(follows from not at all obvious analysis!)

As given, worst-case running time Θ(𝑛𝑚) if we want to report all occurrences

� To avoid that, have to keep track of implied matches.
(tricky because they can be in the “middle” of 𝑃)
� Note: KMP reports all matches in 𝑂(𝑛 + 𝑚) without modifications!

On typical English text, Boyer Moore probes only approx. 25% of the characters in 𝑇!
� Faster than KMP on English text.

requires moderate extra space Θ(𝑚 + 𝜎)

32
Clicker Question

How does Boyer-Moore (BM) compare to Knuth-Morris-Pratt (KMP)?

Check all correct statements. They refer to the number of symbol
comparisons, ignoring preprocessing.

A BP ≤ KMP for all inputs

B BP ≤ KMP for some inputs

C KMP ≤ BM for all inputs

D KMP ≤ BM for some inputs

E BM ≤ KMP if there are no matches

pingo.upb.de/622222
33
Clicker Question

How does Boyer-Moore (BM) compare to Knuth-Morris-Pratt (KMP)?

Check all correct statements. They refer to the number of symbol
comparisons, ignoring preprocessing.

A BP ≤ KMP for all inputs

B BP ≤ KMP for some inputs �

C KMP ≤ BM for all inputs

D KMP ≤ BM for some inputs �

E BM ≤ KMP if there are no matches �

pingo.upb.de/622222
33
4.6 The Rabin-Karp Algorithm
Space – The final frontier
� Knuth-Morris-Pratt has great worst case and real-time guarantees

� Boyer-Moore has great typical behavior

� What else to hope for?

34
Space – The final frontier
� Knuth-Morris-Pratt has great worst case and real-time guarantees

� Boyer-Moore has great typical behavior

� What else to hope for?

� All require Ω(𝑚) extra space;

can be substantial for large patterns!
� Can we avoid that?

34
Rabin-Karp Fingerprint Algorithm – Idea
Idea: use hashing (but without explicit hash tables)

� Precompute & store only hash of pattern

� Compute hash for each guess

� If hashes agree, check characterwise

35
Rabin-Karp Fingerprint Algorithm – Idea
Idea: use hashing (but without explicit hash tables)

� Precompute & store only hash of pattern Example: (treat (sub)strings as decimal numbers)

𝑃 = 59265
� Compute hash for each guess 𝑇 = 3141592653589793238
� If hashes agree, check characterwise Hash function: ℎ(𝑥) = 𝑥 mod 97
� ℎ(𝑃) = 95.

35
Rabin-Karp Fingerprint Algorithm – Idea
Idea: use hashing (but without explicit hash tables)

� Precompute & store only hash of pattern Example: (treat (sub)strings as decimal numbers)

𝑃 = 59265
� Compute hash for each guess 𝑇 = 3141592653589793238
� If hashes agree, check characterwise Hash function: ℎ(𝑥) = 𝑥 mod 97
� ℎ(𝑃) = 95.

3 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 2 3 8
ℎ(31415) = 84
ℎ(14159) = 94
ℎ(41592) = 76
ℎ(15926) = 18
𝒉(59262) = 95

35
Rabin-Karp Fingerprint Algorithm – First Attempt
1 procedure rabinKarpSimplistic(𝑇[0..𝑛 − 1], 𝑃[0..𝑚 − 1])
2 𝑀 := suitable prime number
3 ℎ 𝑃 := computeHash(𝑃[0..𝑚 − 1)], 𝑀)
4 for 𝑖 := 0, . . . , 𝑛 − 𝑚 do
5 ℎ𝑇 := computeHash(𝑇[𝑖..𝑖 + 𝑚 − 1], 𝑀)
6 if ℎ𝑇 == ℎ 𝑃 then
7 if 𝑇[𝑖..𝑖 + 𝑚 − 1] == 𝑃 // 𝑚 comparisons
8 then return 𝑖
9 return NO_MATCH

� never misses a match since ℎ(𝑆1 ) ≠ ℎ(𝑆2 ) implies 𝑆1 ≠ 𝑆2

�
� ℎ(𝑇[𝑘..𝑘+𝑚−1]) depends on 𝑚 characters � naive computation takes Θ(𝑚) time

� Running time is Θ(𝑚𝑛) for search miss . . . can we improve this?

36
Rabin-Karp Fingerprint Algorithm – Fast Rehash
� Crucial insight: We can update hashes in constant time.
� Use previous hash to compute next hash
for above hash function!
� 𝑂(1) time per hash, except first one

37
Rabin-Karp Fingerprint Algorithm – Fast Rehash
� Crucial insight: We can update hashes in constant time.
� Use previous hash to compute next hash
for above hash function!
� 𝑂(1) time per hash, except first one

Example:
� Pre-compute: 10000 mod 97 = 9

� Previous hash: 41592 mod 97 = 76

� Next hash: 15926 mod 97 = ??

37
Rabin-Karp Fingerprint Algorithm – Fast Rehash
� Crucial insight: We can update hashes in constant time.
� Use previous hash to compute next hash
for above hash function!
� 𝑂(1) time per hash, except first one

Example:
� Pre-compute: 10000 mod 97 = 9

� Previous hash: 41592 mod 97 = 76

� Next hash: 15926 mod 97 = ??

Observation:
15926 mod 97 = (41592 − (4·10000)) · 10 + 6 mod 97
= (76 − (4·9 )) · 10 + 6 mod 97
= 406 mod 97 = 18

37
Rabin-Karp Fingerprint Algorithm – Code
� use a convenient radix 𝑅 ≥ 𝜎 (𝑅 = 10 in our examples; 𝑅 = 2 𝑘 is faster)

� Choose modulus 𝑀 at random to be huge prime (randomization against worst-case inputs)

� all numbers remain ≤ 2𝑅2 � 𝑂(1) time arithmetic on word-RAM

1 procedure rabinKarp(𝑇[0..𝑛 − 1], 𝑃[0..𝑚 − 1], 𝑅)

2 𝑀 := suitable prime number
3 ℎ 𝑃 := computeHash(𝑃[0..𝑚 − 1)], 𝑀)
4 ℎ𝑇 := computeHash(𝑇[0..𝑚 − 1], 𝑀)
5 𝑠 := 𝑅 𝑚−1 mod 𝑀
6 for 𝑖 := 0, . . . , 𝑛 − 𝑚 do
7 if ℎ𝑇 == ℎ 𝑃 then
8 if 𝑇[𝑖..𝑖 + 𝑚 − 1] = 𝑃
9 return 𝑖
if 𝑖 < 𝑛 − 𝑚 then
10
� �
11 ℎ𝑇 := (ℎ𝑇 − 𝑇[𝑖] · 𝑠) · 𝑅 + 𝑇[𝑖 + 𝑚] mod 𝑀
12 return NO_MATCH

38
Rabin-Karp – Discussion
Expected running time is 𝑂(𝑚 + 𝑛)

Θ(𝑚𝑛) worst-case;
but this is very unlikely

Extends to 2D patterns and other generalizations

Only constant extra space!

39
Clicker Question

Suppose we apply only the hashing part of Rabin-Karp (drop the

check if 𝑇[𝑖..𝑖 + 𝑚) = 𝑃, and only return 𝑖). Check all correct
statements about the resulting algorithm.
A The algorithm can miss occurrences of 𝑃 in 𝑇 (false
negatives).

B The algorithm can report positions that are not

occurrences (false positives).

C The running time is Θ(𝑛𝑚) in the worst case.

D The running time is Θ(𝑛 + 𝑚) in the worst case.

E The running time is Θ(𝑛) in the worst case.

pingo.upb.de/622222
40
Clicker Question

Suppose we apply only the hashing part of Rabin-Karp (drop the

check if 𝑇[𝑖..𝑖 + 𝑚) = 𝑃, and only return 𝑖). Check all correct
statements about the resulting algorithm.
A The algorithm can miss occurrences of 𝑃 in 𝑇 (false
negatives).

B The algorithm can report positions that are not

�
occurrences (false positives).

C The running time is Θ(𝑛𝑚) in the worst case.

D The running time is Θ(𝑛 + 𝑚) in the worst case. �

E The running time is Θ(𝑛) in the worst case.

pingo.upb.de/622222
40
String Matching Conclusion

Brute- Suffix
DFA KMP BM RK
Force trees*
Preproc.
— 𝑂(𝑚|Σ|) 𝑂(𝑚) 𝑂(𝑚 + 𝜎) 𝑂(𝑚) 𝑂(𝑛)
time
Search 𝑂(𝑛) 𝑂(𝑛 + 𝑚)
𝑂(𝑛𝑚) 𝑂(𝑛) 𝑂(𝑛) 𝑂(𝑚)
time (often better) (expected)
Extra
— 𝑂(𝑚|Σ|) 𝑂(𝑚) 𝑂(𝑚 + 𝜎) 𝑂(1) 𝑂(𝑛)
space

* (see Unit 6)

41