String Algorithms

Jaehyun Park
CS 97SI
Stanford University

June 30, 2015

Outline

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

String Matching Problem

String Matching Problem

Given a text T and a pattern P , find all occurrences of P

within T
Notations:

n and m: lengths of P and T

: set of alphabets (of constant size)
Pi : ith letter of P (1-indexed)
a, b, c: single letters in
x, y, z: strings

String Matching Problem

Example

T = AGCATGCTGCAGTCATGCTTAGGCTA

P = GCT

P appears three times in T

A naive method takes O(mn) time

Initiate string comparison at every starting point
Each comparison takes O(m) time

We can do much better!

String Matching Problem

Outline

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Hash Table

Hash Function

A function that takes a string and outputs a number

A good hash function has few collisions
i.e., If x 6= y, H(x) 6= H(y) with high probability

An easy and powerful hash function is a polynomial mod some

prime p
Consider each letter as a number (ASCII value is fine)
H(x1 . . . xk ) = x1 ak1 + x2 ak2 + + xk1 a + xk (mod p)
How do we find H(x2 . . . xk+1 ) from H(x1 . . . xk )?

Hash Table

Hash Table

Main idea: preprocess T to speedup queries

Hash every substring of length k
k is a small constant

For each query P , hash the first k letters of P to retrieve all

the occurrences of it within T

Dont forget to check collisions!

Hash Table

Hash Table

Pros:
Easy to implement
Significant speedup in practice

Cons:
Doesnt help the asymptotic efficiency

Can still take (nm) time if hashing is terrible or data is

difficult

A lot of memory consumption

Hash Table

Outline

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Knuth-Morris-Pratt (KMP) Algorithm

Knuth-Morris-Pratt (KMP) Matcher

A linear time (!) algorithm that solves the string matching

problem by preprocessing P in (m) time
Main idea is to skip some comparisons by using the previous
comparison result

Uses an auxiliary array that is defined as the following:

[i] is the largest integer smaller than i such that P1 . . . P[i] is
a suffix of P1 . . . Pi

... Its better to see an example than the definition

Knuth-Morris-Pratt (KMP) Algorithm

10

 Table Example (from CLRS)

[i] is the largest integer smaller than i such that P1 . . . P[i]

is a suffix of P1 . . . Pi
e.g., [6] = 4 since abab is a suffix of ababab
e.g., [9] = 0 since no prefix of length 8 ends with c

Lets see why this is useful

Knuth-Morris-Pratt (KMP) Algorithm

11

Using the Table

T = ABC ABCDAB ABCDABCDABDE

P = ABCDABD

= (0, 0, 0, 0, 1, 2, 0)

Start matching at the first position of T :

Mismatch at the 4th letter of P !

Knuth-Morris-Pratt (KMP) Algorithm

12

Using the Table

We matched k = 3 letters so far, and [k] = 0

Thus, there is no point in starting the comparison at T2 , T3
(crucial observation)

Shift P by k [k] = 3 letters

Mismatch at T4 again!

Knuth-Morris-Pratt (KMP) Algorithm

13

Using the Table

We matched k = 0 letters so far

Shift P by k [k] = 1 letter (we define [0] = 1)

Mismatch at T11 !

Knuth-Morris-Pratt (KMP) Algorithm

14

Using the Table

[6] = 2 means P1 P2 is a suffix of P1 . . . P6

Shift P by 6 [6] = 4 letters

Again, no point in shifting P by 1, 2, or 3 letters

Knuth-Morris-Pratt (KMP) Algorithm

15

Using the Table

Mismatch at T11 again!

Currently 2 letters are matched

Shift P by 2 [2] = 2 letters

Knuth-Morris-Pratt (KMP) Algorithm

16

Using the Table

Mismatch at T11 yet again!

Currently no letters are matched

Shift P by 0 [0] = 1 letter

Knuth-Morris-Pratt (KMP) Algorithm

17

Using the Table

Mismatch at T18

Currently 6 letters are matched

Shift P by 6 [6] = 4 letters

Knuth-Morris-Pratt (KMP) Algorithm

18

Using the Table

Finally, there it is!

Currently all 7 letters are matched

After recording this match (at T16 . . . T22 , we shift P again in
order to find other matches

Shift by 7 [7] = 7 letters

Knuth-Morris-Pratt (KMP) Algorithm

19

Computing

Observation 1: if P1 . . . P[i] is a suffix of P1 . . . Pi , then

P1 . . . P[i]1 is a suffix of P1 . . . Pi1

Observation 2: all the prefixes of P that are a suffix of

P1 . . . Pi can be obtained by recursively applying to i

Well, obviously...

e.g., P1 . . . P[i] , P1 . . . , P[[i]] , P1 . . . , P[[[i]]] are all

suffixes of P1 . . . Pi

Knuth-Morris-Pratt (KMP) Algorithm

20

Computing

A non-obvious conclusion:
First, lets write (k) [i] as [] applied k times to i
e.g., (2) [i] = [[i]]
[i] is equal to (k) [i 1] + 1, where k is the smallest integer
that satisfies P(k) [i1]+1 = Pi

If there is no such k, [i] = 0

Intuition: we look at all the prefixes of P that are suffixes of

P1 . . . Pi1 , and find the longest one whose next letter
matches Pi

Knuth-Morris-Pratt (KMP) Algorithm

21

Implementation

pi[0] = -1;
int k = -1;
for(int i = 1; i <= m; i++) {
while(k >= 0 && P[k+1] != P[i])
k = pi[k];
pi[i] = ++k;
}

Knuth-Morris-Pratt (KMP) Algorithm

22

Pattern Matching Implementation

int k = 0;
for(int i = 1; i <= n; i++) {
while(k >= 0 && P[k+1] != T[i])
k = pi[k];
k++;
if(k == m) {
// P matches T[i-m+1..i]
k = pi[k];
}
}

Knuth-Morris-Pratt (KMP) Algorithm

23

Outline

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Suffix Trie

24

Suffix Trie

Suffix trie of a string T is a rooted tree that stores all the

suffixes (thus all the substrings)

Each node corresponds to some substring of T

Each edge is associated with an alphabet

For each node that corresponds to ax, there is a special

pointer called suffix link that leads to the node corresponding
to x

Surprisingly easy to implement!

Suffix Trie

25

Example

(Figure modified from Ukkonens original paper)

Suffix Trie

26

Incremental Construction

Given the suffix tree for T1 . . . Tn

Then we append Tn+1 = a to T , creating necessary nodes

Start at node u corresponding to T1 . . . Tn

Create an a-transition to a new node v

Take the suffix link at u to go to u , corresponding to

T2 . . . Tn
Create an a-transition to a new node v
Create a suffix link from v to v

Suffix Trie

27

Incremental Construction

Repeat the previous process:

Take the suffix link at the current node
Make a new a-transition there
Create the suffix link from the previous node

Stop if the node already has an a-transition

Because from this point, all nodes that are reachable via suffix
links already have an a-transition

Suffix Trie

28

Construction Example

Given the suffix trie for aba

We want to add a new letter c

Suffix Trie

29

Construction Example

Suffix Trie

30

Construction Example

Suffix Trie

31

Construction Example

Suffix Trie

32

Construction Example

Suffix Trie

33

Construction Example

Suffix Trie

34

Construction Example

Construction time is linear in the tree size

But the tree size can be quadratic in n
e.g., T = aa . . . abb . . . b

Suffix Trie

35

Construction Example

To find P , start at the root and keep following edges labeled

with P1 , P2 , etc.

Got stuck? Then P doesnt exist in T

Suffix Trie

36

Outline

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Suffix Array

37

Suffix Array

Suffix Array

38

Suffix Array

Memory usage is O(n)

Has the same computational power as suffix trie

Can be constructed in O(n) time (!)

There is an approachable O(n log2 n) algorithm

But its hard to implement

If you want to see how it works, read the paper on the course
website
http://cs97si.stanford.edu/suffix-array.pdf

Suffix Array

39

Notes on String Problems

Always be aware of the null-terminators

Simple hash works so well in many problems

If a problem involves rotations of some string, consider

concatenating it with itself and see if it helps

Stanford team notebook has implementations of suffix arrays

and the KMP matcher

Suffix Array

40