Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

5.3 S UBSTRING S EARCH

‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
Substring search

Goal. Find pattern of length M in a text of length N.

typically N >> M

pattern N E E D L E

text I N A H A Y S T A C K N E E D L E I N A

match
Substring search

3
Substring search applications

Goal. Find pattern of length M in a text of length N.

typically N >> M

pattern N E E D L E

text I N A H A Y S T A C K N E E D L E I N A

match
Substring search

4
Substring search applications

Goal. Find pattern of length M in a text of length N.

typically N >> M

pattern N E E D L E

text I N A H A Y S T A C K N E E D L E I N A

match
Substring search

Computer forensics.  Search memory or disk for signatures,

e.g., all URLs or RSA keys that the user has entered.

http://citp.princeton.edu/memory
5
Substring search applications

Goal. Find pattern of length M in a text of length N.

typically N >> M

pattern N E E D L E

text I N A H A Y S T A C K N E E D L E I N A

match
Substring search

Identify patterns indicative of spam.

・ PROFITS
・ L0SE WE1GHT
・ herbal Viagra
・ There is no catch.
・ This is a one-time mailing.
・ This message is sent in compliance with spam regulations.
6
Substring search applications

Electronic surveillance.
Need to monitor all
internet traffic.
(security)

No way!
(privacy)

Well, we’re mainly

interested in
“ATTACK AT DAWN”

OK. Build a
machine that just
looks for that.

“ATTACK AT DAWN”
substring search
machine

found
7
 Substring search applications

Screen scraping. Extract relevant data from web page.

Ex. Find string delimited by <b> and </b> after first occurrence of
pattern Last Trade:.

...
<tr>
<td class= "yfnc_tablehead1"
width= "48%">
Last Trade:
</td>
<td class= "yfnc_tabledata1">
<big><b>452.92</b></big>
</td></tr>
<td class= "yfnc_tablehead1"
http://finance.yahoo.com/q?s=goog width= "48%">
Trade Time:
</td>
<td class= "yfnc_tabledata1">
...

8
Screen scraping: Java implementation

Java library. The indexOf() method in Java's string library returns the index
of the first occurrence of a given string, starting at a given offset.

public class StockQuote

{
public static void main(String[] args)
{
String name = "http://finance.yahoo.com/q?s=";
In in = new In(name + args[0]);
String text = in.readAll();
int start = text.indexOf("Last Trade:", 0);
int from = text.indexOf("<b>", start);
int to = text.indexOf("</b>", from);
String price = text.substring(from + 3, to);
StdOut.println(price);
}
}
% java StockQuote goog
582.93

% java StockQuote msft

24.84
9
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
Brute-force substring search

Check for pattern starting at each text position.

i j i+j 0 1 2 3 4 5 6 7 8 9 10
txt A B A C A D A B R A C
0 2 2 A B R A pat
1 0 1 A B R A entries in red are
mismatches
2 1 3 A B R A
3 0 3 A B R A entries in gray are
for reference only
4 1 5 A B R A
entries in black
5 0 5 match the text A B R A
6 4 10 A B R A
return i when j is M
match

Brute-force substring search

12
Brute-force substring search: Java implementation

Check for pattern starting at each text position.

i j i+j 0 1 2 3 4 5 6 7 8 9 10
A B A C A D A B R A C
4 3 7 A D A C R
5 0 5 A D A C R

public static int search(String pat, String txt)

{
int M = pat.length();
int N = txt.length();
for (int i = 0; i <= N - M; i++)
{
int j;
for (j = 0; j < M; j++)
if (txt.charAt(i+j) != pat.charAt(j))
break;
index in text where
if (j == M) return i; pattern starts
}
return N; not found
}
13
Brute-force substring search: worst case

Brute-force algorithm can be slow if text and pattern are repetitive.

i ij j i+j 0 0 1 1 22 33 44 55
i+j 6
6 77 88 99 10

txttxt A A A B AA AC AA A D A A BA RA AB C
0 2 2 A B R A pat
0 4 4 A A A A B pat
1 0 1 A B R A entries in red are
1 4 5 A A A A B mismatches
2 1 3 A B R A
2 4 6 A A A A B entries in gray are
3 0 3 A B R A
for reference only
3 44 17 5 A AA AB A R AB
entries in black
4 54 08 5 match the text A AA A B RA AB
5 65 10
4 10 A A A BA RA AB
return i when j is M
Brute-force substring search (worst case)
match

Brute-force substring search

Worst case. ~ M N char compares.

14
Backup

In many applications, we want to avoid backup in text stream.

・Treat input as stream of data. “ATTACK AT DAWN”

・Abstract model: standard input. substring search

machine

found

Brute-force algorithm needs backup for every mismatch.

matched chars
mismatch

A A A A A A A A A A A A A A A A A A A A A B
A A A A A B
backup

A A A A A A A A A A A A A A A A A A A A A B
A A A A A B

shift pattern right one position

Approach 1. Maintain buffer of last M characters.

Approach 2. Stay tuned.
15
Brute-force substring search: alternate implementation

Same sequence of char compares as previous implementation.

・ i points to end of sequence of already-matched chars in text.
・ j stores # of already-matched chars (end of sequence in pattern).
i j 0 1 2 3 4 5 6 7 8 9 10
A B A C A D A B R A C
7 3 A D A C R
5 0 A D A C R

public static int search(String pat, String txt)

{
int i, N = txt.length();
int j, M = pat.length();
for (i = 0, j = 0; i < N && j < M; i++)
{
if (txt.charAt(i) == pat.charAt(j)) j++;
else { i -= j; j = 0; } explicit backup
}
if (j == M) return i - M;
else return N;
}

16
Algorithmic challenges in substring search

Brute-force is not always good enough.

Theoretical challenge. Linear-time guarantee. fundamental algorithmic problem

Practical challenge. Avoid backup in text stream. often no room or time to save text

Now is the time for all people to come to the aid of their party. Now is the time for all good people to
come to the aid of their party. Now is the time for many good people to come to the aid of their party.
Now is the time for all good people to come to the aid of their party. Now is the time for a lot of good
people to come to the aid of their party. Now is the time for all of the good people to come to the aid of
their party. Now is the time for all good people to come to the aid of their party. Now is the time for
each good person to come to the aid of their party. Now is the time for all good people to come to the aid
of their party. Now is the time for all good Republicans to come to the aid of their party. Now is the
time for all good people to come to the aid of their party. Now is the time for many or all good people to
come to the aid of their party. Now is the time for all good people to come to the aid of their party. Now
is the time for all good Democrats to come to the aid of their party. Now is the time for all people to
come to the aid of their party. Now is the time for all good people to come to the aid of their party. Now
is the time for many good people to come to the aid of their party. Now is the time for all good people to
come to the aid of their party. Now is the time for a lot of good people to come to the aid of their
party. Now is the time for all of the good people to come to the aid of their party. Now is the time for
all good people to come to the aid of their attack at dawn party. Now is the time for each person to come
to the aid of their party. Now is the time for all good people to come to the aid of their party. Now is
the time for all good Republicans to come to the aid of their party. Now is the time for all good people
to come to the aid of their party. Now is the time for many or all good people to come to the aid of their
party. Now is the time for all good people to come to the aid of their party. Now is the time for all good
Democrats to come to the aid of their party.

17
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
Knuth-Morris-Pratt substring search

Intuition. Suppose we are searching in text for pattern BAAAAAAAAA.

・Suppose we match 5 chars in pattern, with mismatch on 6 th char.
・We know previous 6 chars in text are BAAAAB.
・Don't need to back up text pointer! assuming { A, B } alphabet

text
A B A A A A B A A A A A A A A A
after mismatch
on sixth char B A A A A A A A A A pattern
brute-force backs B A A A A A A A A A
up to try this B A A A A A A A A A
and this B A A A A A A A A A
and this B A A A A A A A A A
and this B A A A A A A A A A
and this

but no backup B A A A A A A A A A
is needed

Text pointer backup in substring searching

Knuth-Morris-Pratt algorithm. Clever method to always avoid backup. (!)

20
Deterministic finite state automaton (DFA)

DFA is abstract string-searching machine.

・Finite number of states (including start and halt).
・Exactly one transition for each char in alphabet.
・Accept if sequence of transitions leads to halt state.
X
internal representation X
j 0 1 2 3 4 5
pat.charAt(j)
j 0 1 2A 3 B 4 A5 B A C
pat.charAt(j) A B A A1 B A If in state j reading char c:
1 3C 1 5 1
A 1 1 B
dfa[][j] 30 1 2 5 01 4 0 4if j is 6 halt and accept
0 2 0 4 0
04
dfa[][j] B
C 0 0 0 0 6else move to state dfa[c][j]
C 0 0 0 0 0 6

graphical representation

B,C A A A A
B,C A A B B
0 A
01 B 21 A 3 2B 4 A 5 C
A6
A B A 3 B 4 5 C 6
C
B,C
C C B,C
B,C C B,C

Constructing the DFA for KMP substring search for A B A B A C

Constructing the DFA for KMP substring search for A B A B A C 21
DFA simulation demo

A A B A C A A B A B A C A A

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 1 3 1 5 1
dfa[][j] B 0 2 0 4 0 4
C 0 0 0 0 0 6

B, C A A
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 22
DFA simulation demo

A A B A C A A B A B A C A A

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 1 3 1 5 1
dfa[][j] B 0 2 0 4 0 4
C 0 0 0 0 0 6

B, C A A
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
substring found
C

B, C 23
Interpretation of Knuth-Morris-Pratt DFA

Q. What is interpretation of DFA state after reading in txt[i]?

A. State = number of characters in pattern that have been matched.
length of longest prefix of pat[]
that is a suffix of txt[0..i]

Ex. DFA is in state 3 after reading in txt[0..6].

i
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5
txt B C B A A B A C A pat A B A B A C
suffix of txt[0..6] prefix of pat[]

B, C A A
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 24
Knuth-Morris-Pratt substring search: Java implementation

Key differences from brute-force implementation.

・Need to precompute dfa[][] from pattern.
・Text pointer i never decrements.

public int search(String txt)

{
int i, j, N = txt.length();
for (i = 0, j = 0; i < N && j < M; i++)
j = dfa[txt.charAt(i)][j]; no backup

if (j == M) return i - M;
else return N;
}

Running time.
・Simulate DFA on text: at most N character accesses.
・Build DFA: how to do efficiently? [warning: tricky algorithm ahead]
25
Knuth-Morris-Pratt substring search: Java implementation

Key differences from brute-force implementation.

・Need to precompute dfa[][] from pattern.
・Text pointer i never decrements.
・Could use input stream.
public int search(In in)
{
int i, j;
for (i = 0, j = 0; !in.isEmpty() && j < M; i++)
j = dfa[in.readChar()][j]; no backup

if (j == M) return i - M;
else return NOT_FOUND;
}
X

j 0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 1 3 1 5 1
dfa[][j] B 0 2 0 4 0 4
C 0 0 0 0 0 6

B,C A A
A B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B,C C B,C

Constructing the DFA for KMP substring search for A B A B A C 26

 Knuth-Morris-Pratt construction demo

Include one state for each character in pattern (plus accept state).

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A
dfa[][j] B
C

Constructing the DFA for KMP substring search for A B A B A C

0 1 2 3 4 5 6

27
 Knuth-Morris-Pratt construction demo

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 1 3 1 5 1
dfa[][j] B 0 2 0 4 0 4
C 0 0 0 0 0 6

Constructing the DFA for KMP substring search for A B A B A C

B, C A A
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 28
How to build DFA from pattern?

Include one state for each character in pattern (plus accept state).

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A
dfa[][j] B
C

0 1 2 3 4 5 6

29
How to build DFA from pattern?

Match transition. If in state j and next char c == pat.charAt(j), go to j+1.

first j characters of pattern next char matches now first j+1 characters of
have already been matched pattern have been matched

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 3 5
dfa[][j] B 2 4
C 6

0 A 1 B 2 A 3 B 4 A 5 C 6

30
How to build DFA from pattern?

Mismatch transition. If in state j and next char c != pat.charAt(j),

then the last j-1 characters of input are pat[1..j-1], followed by c.

To compute dfa[c][j]: Simulate pat[1..j-1] on DFA and take transition c.

Running time. Seems to require j steps. still under construction (!)

Ex. dfa['A'][5] = 1; dfa['B'][5] = 4

simulate BABA; simulate BABA;
j 0 1 2 3 4 5
take transition 'A' take transition 'B'
pat.charAt(j) A B A B A C
= dfa['A'][3] = dfa['B'][3]

simulation
B, C A A
of BABA j
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 31
How to build DFA from pattern?

Mismatch transition. If in state j and next char c != pat.charAt(j),

then the last j-1 characters of input are pat[1..j-1], followed by c.
state X

To compute dfa[c][j]: Simulate pat[1..j-1] on DFA and take transition c.

Running time. Takes only constant time if we maintain state X.

Ex. dfa['A'][5] = 1; dfa['B'][5] = 4; X'= 0

from state X, from state X, from state X,
take transition 'C' 0 1 2 3 4 5
take transition 'A' take transition 'B'
= dfa['A'][X] = dfa['B'][X] = dfa['C'][X] A B A B A C

B, C A A
X j
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 32
 Knuth-Morris-Pratt construction demo (in linear time)

Include one state for each character in pattern (plus accept state).

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A
dfa[][j] B
C

Constructing the DFA for KMP substring search for A B A B A C

0 1 2 3 4 5 6

33
 Knuth-Morris-Pratt construction demo (in linear time)

0 1 2 3 4 5
pat.charAt(j) A B A B A C
A 1 1 3 1 5 1
dfa[][j] B 0 2 0 4 0 4
C 0 0 0 0 0 6

Constructing the DFA for KMP substring search for A B A B A C

B, C A A
B
0 A 1 B 2 A 3 B 4 A 5 C 6
C
B, C
C

B, C 34
Constructing the DFA for KMP substring search: Java implementation

For each state j:

・Copy dfa[][X] to dfa[][j] for mismatch case.
・Set dfa[pat.charAt(j)][j] to j+1 for match case.
・Update X.
public KMP(String pat)
{
this.pat = pat;
M = pat.length();
dfa = new int[R][M];
dfa[pat.charAt(0)][0] = 1;
for (int X = 0, j = 1; j < M; j++)
{
for (int c = 0; c < R; c++)
dfa[c][j] = dfa[c][X]; copy mismatch cases
dfa[pat.charAt(j)][j] = j+1; set match case
X = dfa[pat.charAt(j)][X]; update restart state
}
}

Running time. M character accesses (but space/time proportional to R M).

35
KMP substring search analysis

Proposition. KMP substring search accesses no more than M + N chars

to search for a pattern of length M in a text of length N.

Pf. Each pattern char accessed once when constructing the DFA;
each text char accessed once (in the worst case) when simulating the DFA.

internal representation
j 0 1 2 3 4 5
Proposition. KMP constructs
pat.charAt(j) A B
dfa[][]Ain time
B A and
C space proportional to R M.
next[j] 0 0 0 0 0 3

Larger alphabets. Improved version of KMP constructs nfa[] in time and

mismatch transition
space proportional to M. (back up at least one state)
graphical representation

0 A 1 B 2 A 3 B 4 A 5 C 6

KMP NFA for ABABAC

NFA corresponding to the string A B A B A C 36
Knuth-Morris-Pratt: brief history

・Independently discovered by two theoreticians and a hacker.

– Knuth: inspired by esoteric theorem, discovered linear algorithm
– Pratt: made running time independent of alphabet size
– Morris: built a text editor for the CDC 6400 computer
・Theory meets practice.

SIAM J. COMPUT.
Vol. 6, No. 2, June 1977

FAST PATTERN MATCHING IN STRINGS*

DONALD E. KNUTHf, JAMES H. MORRIS, JR.:l: AND VAUGHAN R. PRATT

Abstract. An algorithm is presented which finds all occurrences of one. given string within
another, in running time proportional to the sum of the lengths of the strings. The constant of
proportionality is low enough to make this algorithm of practical use, and the procedure can also be
extended to deal with some more general pattern-matching problems. A theoretical application of the
algorithm shows that the set of concatenations of even palindromes, i.e., the language {can}*, can be
recognized in linear time. Other algorithms which run even faster on the average are also considered.

Key pattern, string, text-editing, pattern-matching, trie memory, searching, period of a

Don Knuth Jim Morris Vaughan Pratt

37
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu

Robert Boyer J. Strother Moore

 Boyer-Moore: mismatched character heuristic

Intuition.
・Scan characters in pattern from right to left.
・Can skip as many as M text chars when finding one not in the pattern.

i j 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
text F I N D I N A H A Y S T A C K N E E D L E I N A
0 5 N E E D L E pattern
5 5 N E E D L E
11 4 N E E D L E
15 0 N E E D L E
return i = 15
Mismatched character heuristic for right-to-left (Boyer-Moore) substring search

40
Boyer-Moore: mismatched character heuristic

Q. How much to skip?

Case 1. Mismatch character not in pattern.

before

txt . . . . . . T L E . . . . . .
pat N E E D L E

after

txt . . . . . . T L E . . . . . .
pat N E E D L E

mismatch character 'T' not in pattern: increment i one character beyond 'T'

41
Boyer-Moore: mismatched character heuristic

Q. How much to skip?

Case 2a. Mismatch character in pattern.

before

txt . . . . . . N L E . . . . . .
pat N E E D L E

after

txt . . . . . . N L E . . . . . .
pat N E E D L E

mismatch character 'N' in pattern: align text 'N' with rightmost pattern 'N'

42
 Boyer-Moore: mismatched character heuristic

Q. How much to skip?

Case 2b. Mismatch character in pattern (but heuristic no help).

before

txt . . . . . . E L E . . . . . .
pat N E E D L E

aligned with rightmost E?

txt . . . . . . E L E . . . . . .
pat N E E D L E

mismatch character 'E' in pattern: align text 'E' with rightmost pattern 'E' ?

43
Boyer-Moore: mismatched character heuristic

Q. How much to skip?

Case 2b. Mismatch character in pattern (but heuristic no help).

before

txt . . . . . . E L E . . . . . .
pat N E E D L E

after

txt . . . . . . E L E . . . . . .
pat N E E D L E

mismatch character 'E' in pattern: increment i by 1

44
Boyer-Moore: mismatched character heuristic

Q. How much to skip?

A. Precompute index of rightmost occurrence of character c in pattern

(-1 if character not in pattern).

N E E D L E
c 0 1 2 3 4 5 right[c]
A -1 -1 -1 -1 -1 -1 -1 -1
B -1 -1 -1 -1 -1 -1 -1 -1
right = new int[R];
for (int c = 0; c < R; c++) C -1 -1 -1 -1 -1 -1 -1 -1
right[c] = -1; D -1 -1 -1 -1 3 3 3 3
for (int j = 0; j < M; j++) E -1 -1 1 2 2 2 5 5
right[pat.charAt(j)] = j; ... -1
L -1 -1 -1 -1 -1 4 4 4
M -1 -1 -1 -1 -1 -1 -1 -1
N -1 0 0 0 0 0 0 0
... -1
Boyer-Moore skip table computation

45
Boyer-Moore: Java implementation

public int search(String txt)

{
int N = txt.length();
int M = pat.length();
int skip;
for (int i = 0; i <= N-M; i += skip)
{
skip = 0;
for (int j = M-1; j >= 0; j--)
compute
{
skip value
if (pat.charAt(j) != txt.charAt(i+j))
{
skip = Math.max(1, j - right[txt.charAt(i+j)]);
break;
} in case other term is nonpositive

}
if (skip == 0) return i; match
}
return N;
}

46
Boyer-Moore: analysis

Property. Substring search with the Boyer-Moore mismatched character

heuristic takes about ~ N / M character compares to search for a pattern of
length M in a text of length N. sublinear!

Worst-case. Can be as bad as ~ M N.

i skip 0 1 2 3 4 5 6 7 8 9
txt B B B B B B B B B B
0 0 A B B B B pat
1 1 A B B B B
2 1 A B B B B
3 1 A B B B B
4 1 A B B B B
5 1 A B B B B
Boyer-Moore-Horspool substring search (worst case)

Boyer-Moore variant. Can improve worst case to ~ 3 N character compares

by adding a KMP-like rule to guard against repetitive patterns.
47
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu

Michael Rabin, Turing Award '76

Dick Karp, Turing Award '85
Rabin-Karp fingerprint search

Basic idea = modular hashing.

・Compute a hash of pattern characters 0 to M - 1.
・For each i, compute a hash of text characters i to M + i - 1.
・If pattern hash = text substring hash, check for a match.
pat.charAt(i)
i 0 1 2 3 4
2 6 5 3 5 % 997 = 613

txt.charAt(i)
i 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
3 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3
0 3 1 4 1 5 % 997 = 508
1 1 4 1 5 9 % 997 = 201
2 4 1 5 9 2 % 997 = 715
3 1 5 9 2 6 % 997 = 971
4 5 9 2 6 5 % 997 = 442
match
5 9 2 6 5 3 % 997 = 929
6 return i = 6 2 6 5 3 5 % 997 = 613

Basis for Rabin-Karp substring search

50
 Efficiently computing the hash function

Modular hash function. Using the notation ti for txt.charAt(i),

we wish to compute
xi = ti R M-1 + ti+1 R M-2 + … + ti+M-1 R 0 (mod Q)

Intuition. M-digit, base-R integer, modulo Q.

Horner's method. Linear-time method to evaluate degree-M polynomial.

pat.charAt() // Compute hash for M-digit key

i 0 1 2 3 4 private long hash(String key, int M)
2 6 5 3 5
{
R Q
0 2 % 997 = 2 long h = 0;
1 2 6 % 997 = (2*10 + 6) % 997 = 26 for (int j = 0; j < M; j++)
2 2 6 5 % 997 = (26*10 + 5) % 997 = 265 h = (R * h + key.charAt(j)) % Q;
3 2 6 5 3 % 997 = (265*10 + 3) % 997 = 659 return h;
4 2 6 5 3 5 % 997 = (659*10 + 5) % 997 = 613 }
Computing the hash value for the pattern with Horner’s method

51
Efficiently computing the hash function

Challenge. How to efficiently compute xi+1 given that we know xi.

xi = ti R M–1 + ti+1 R M–2 + … + ti+M–1 R0

xi+1 = ti+1 R M–1 + ti+2 R M–2 + … + ti+M R0

Key property. Can update hash function in constant time!

xi+1 = ( xi – t i R M–1 ) R + t i +M

current subtract multiply add new (can precompute RM-1)

value leading digit by radix trailing digit

i ... 2 3 4 5 6 7 ...
current value 1 4 1 5 9 2 6 5
text
new value 4 1 5 9 2 6 5

4 1 5 9 2 current value
- 4 0 0 0 0
1 5 9 2 subtract leading digit
* 1 0 multiply by radix
1 5 9 2 0
+ 6 add new trailing digit
1 5 9 2 6 new value

52
Key computation in Rabin-Karp substring search
(move right one position in the text)
 Rabin-Karp substring search example

i 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
3 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3
0 3 % 997 = 3 Q

1 3 1 % 997 = (3*10 + 1) % 997 = 31

2 3 1 4 % 997 = (31*10 + 4) % 997 = 314
3 3 1 4 1 % 997 = (314*10 + 1) % 997 = 150
4 3 1 4 1 5 % 997 = (150*10 + 5) % 997 = 508 RM R

5 1 4 1 5 9 % 997 = ((508 + 3*(997 - 30))*10 + 9) % 997 = 201

6 4 1 5 9 2 % 997 = ((201 + 1*(997 - 30))*10 + 2) % 997 = 715
7 1 5 9 2 6 % 997 = ((715 + 4*(997 - 30))*10 + 6) % 997 = 971
8 5 9 2 6 5 % 997 = ((971 + 1*(997 - 30))*10 + 5) % 997 = 442 match
9 9 2 6 5 3 % 997 = ((442 + 5*(997 - 30))*10 + 3) % 997 = 929
10 return i-M+1 = 6 2 6 5 3 5 % 997 = ((929 + 9*(997 - 30))*10 + 5) % 997 = 613

Rabin-Karp substring search example

53
Rabin-Karp: Java implementation

public class RabinKarp

{
private long patHash; // pattern hash value
private int M; // pattern length
private long Q; // modulus
private int R; // radix
private long RM; // R^(M-1) % Q

public RabinKarp(String pat) {

M = pat.length();
R = 256;
a large prime
Q = longRandomPrime(); (but avoid overflow)

RM = 1; precompute RM – 1 (mod Q)
for (int i = 1; i <= M-1; i++)
RM = (R * RM) % Q;
patHash = hash(pat, M);
}

private long hash(String key, int M)

{ /* as before */ }

public int search(String txt)

{ /* see next slide */ }
}
54
Rabin-Karp: Java implementation (continued)

Monte Carlo version. Return match if hash match.

public int search(String txt)

check for hash collision
{
using rolling hash function
int N = txt.length();
int txtHash = hash(txt, M);
if (patHash == txtHash) return 0;
for (int i = M; i < N; i++)
{
txtHash = (txtHash + Q - RM*txt.charAt(i-M) % Q) % Q;
txtHash = (txtHash*R + txt.charAt(i)) % Q;
if (patHash == txtHash) return i - M + 1;
}
return N;
}

Las Vegas version. Check for substring match if hash match;

continue search if false collision.
55
Rabin-Karp analysis

Theory. If Q is a sufficiently large random prime (about M N 2),

then the probability of a false collision is about 1 / N.

Practice. Choose Q to be a large prime (but not so large to cause overflow).

Under reasonable assumptions, probability of a collision is about 1 / Q.

Monte Carlo version.

・Always runs in linear time.
・Extremely likely to return correct answer (but not always!).
Las Vegas version.
・Always returns correct answer.
・Extremely likely to run in linear time (but worst case is M N).

56
Rabin-Karp fingerprint search

Advantages.
・Extends to 2d patterns.
・Extends to finding multiple patterns.
Disadvantages.
・Arithmetic ops slower than char compares.
・Las Vegas version requires backup.
・Poor worst-case guarantee.

Q. How would you extend Rabin-Karp to efficiently search for any

one of P possible patterns in a text of length N ?

57
 for thisfor
rithms fingerprint
the same (the
task,hash
each value)
of theminhas
theattractive
text. Thefeatures.
algorithm is efficient
Brute because
force search the
is easy
fingerprints
to implementcan andbeworks
efficiently
well in computed and(Java’s
typical cases compared.
indexOf() method in String uses
brute-force search); Knuth-Morris-Pratt is guaranteed linear-time with no backup in
Substring
the search
Summary
input; cost
The
Boyer-Moore summary
table isatsublinear
the bottom(byof the page
a factor summarizes
of M) in typical the algorithms
situations; and that we
Rabin-
have is
Karp discussed for substring
linear. Each search. As is
also has drawbacks: often the might
brute-force case when
requirewe time
have proportional
several algo-
rithms
to MN; for the same task, eachand
Knuth-Morris-Pratt of them has attractive
Boyer-Moore features.
use extra Brute
space; and force search ishas
Rabin-Karp easy
a
Cost of searching
to implement
relatively for
long and
inner anloopM-character
works well in typical
(several pattern
cases
arithmetic in as
(Java’s indexOf()
operations, N-character
anopposed
method text.
in String
to character uses
com-
brute-force
pares search);
in the other Knuth-Morris-Pratt
methods. is guaranteed
These characteristics linear-timeinwith
are summarized no backup
the table below. in
the input; Boyer-Moore is sublinear (by a factor of M) in typical situations; and Rabin-
Karp is linear. Each also has drawbacks: brute-force might require time proportional
operation count backup extra
to MN; algorithm
Knuth-Morris-Pratt version
and Boyer-Moore use extra space; correct?
and Rabin-Karp has a
guarantee typical in input? space
relatively long inner loop (several arithmetic operations, as opposed to character com-
paresbrute
in theforce
other methods. These— characteristics
M N are1.1 summarized
N yes in the yes
table below.
1

full DFA
algorithm version5.6 ) 2operation
N count
1.1 N no
backup yes
correct?
extra
MR
(Algorithm in input? space
Knuth-Morris-Pratt guarantee typical
mismatch
3N 1.1 N no yes M
brute force — only
transitions MN 1.1 N yes yes 1

fullfull DFA
algorithm 3N N/M yes yes R
2N 1.1 N no yes MR
(Algorithm 5.6 )
Knuth-Morris-Pratt
Boyer-Moore mismatched char
mismatch
heuristic only M
3N N1.1
/MN yes
no yes
yes R
M
transitions only
(Algorithm 5.7 )
full
Montealgorithm
Carlo 3N N/M yes yes R
7N 7N no yes † 1
(Algorithm 5.8 )
Rabin-Karp†
Boyer-Moore mismatched char
heuristic only
Las Vegas 7MNN† N7 /NM yes†
no yes
yes 1R
(Algorithm 5.7 )
† probabilisitic guarantee, with uniform hash function
Monte Carlo
summary 5.8
7N 7N no yes † 1
(Algorithm
Cost for )substring-search implementations
Rabin-Karp† 58
Las Vegas 7N † 7N no † yes 1
 5.3 S UBSTRING S EARCH
‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu
Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

5.3 S UBSTRING S EARCH

‣ introduction
‣ brute force
‣ Knuth-Morris-Pratt
Algorithms
‣ Boyer-Moore
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Rabin-Karp
http://algs4.cs.princeton.edu