CS 97SI: INTRODUCTION TO

PROGRAMMING CONTESTS
Jaehyun Park

Last Lecture: String Algorithms

String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array
Note on String Problems

String Matching Problem

Given a text and a pattern , find all the

occurrences of within
Notations:
and : lengths of and
: set of alphabets

Constant

size

th letter of (1-indexed)
, , : single letters in
, , : strings

String Matching Example

= AGCATGCTGCAGTCATGCTTAGGCTA
= GCT

A nave method takes () time

We

initiate string comparison at every starting point

Each comparison takes time

We can certainly do better!

Hash Function

A function that takes a string and outputs a number

A good hash function has few collisions
i.e.

If , () with high probability

An easy and powerful hash function is a polynomial

mod some prime
Consider

each letter as a number (ASCII value is fine)

1 + 2 + +
1 = 1
2
1 +
How do we find (2 +1 ) from (1 )?

Hash Table

Main idea: preprocess to speedup queries

Hash

every substring of length

is a small constant

For each query , hash the first letters of to

retrieve all the occurrences of it within
Dont forget to check collisions!

Hash Table

Pros:
Easy

to implement
Significant speedup in practice

Cons:
Doesnt
Can

help the asymptotic efficiency

take () time if hashing is terrible

lot of memory consumption

Knuth-Morris-Pratt (KMP) Matcher

A linear time (!) algorithm that solves the string

matching problem by preprocessing in time
Main

idea is to skip some comparisons by using the

previous comparison result

Uses an auxiliary array that is defined as the

following:
is the largest integer smaller than such that
1 [] is a suffix of 1

[]

Its better to see an example than the definition

 Table Example (from CLRS)

10

[]

[]: the largest integer smaller than such that

1 [] is a suffix of 1
e.g.

[6] = 4 since abab is a suffix of ababab

e.g. [9] = 0 since no prefix of length 8 ends with c

Lets see why this is useful

Using the Table

= ABC ABCDAB ABCDABCDABDE

= ABCDABD
= (0, 0, 0, 0, 1, 2, 0)
Start matching at the first position of :
12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Mismatch at the 4th letter of !

Using the Table

There is no point in starting the comparison at 2 , 3

We

matched = 3 letters so far

Shift by = 3 letters

12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Mismatch at 4 again!

Using the Table

We define 0 = 1
We

matched = 0 letters so far

Shift by = 1 letter

12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Mismatch at 11 !

Using the Table

[6] = 2 says 1 2 is a suffix of 1 6

Shift by 6 [6] = 4 letters
12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
||
ABCDABD
1234567
Again, no point in shifting by 1, 2, or 3 letters

Using the Table

Mismatch at 11 again!
12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Currently 2 letters are matched

We shift by 2 = 2 2 letters

Using the Table

Mismatch at 11 yet again!

12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Currently no letters are matched

We shift by 1 = 0 0 letters

Using the Table

Mismatch at 18
12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Currently 6 letters are matched

We shift by 4 = 6 6 letters

Using the Table

Finally, there it is!

12345678901234567890123
ABC ABCDAB ABCDABCDABDE
ABCDABD
1234567

Currently all 7 letters are matched

After recording this match (match at 16 22 ), we shift
again in order to find other matches
Shift by 7 = 7 7 letters

Computing

Observation 1: if 1 [] is a suffix of 1 ,
then 1 1 is a suffix of 1 1
Well,

obviously

Observation 2: all the prefixes of P that are a

suffix of 1 can be obtained by recursively
applying to
e.g.

1 , 1 , 1

suffixes of 1

are all

Computing

A non-obvious conclusion:
First,
e.g.

lets write () as [] applied times to

is equal to 1 + 1, where is the smallest

integer that satisfies 1 +1 =

If

there is no such , [] = 0

Intuition: we look at all the prefixes of that are

suffixes of 1 1 and find the longest one
whose next letter matches too

Implementation
pi[0] = -1;
int k = -1;
for(int i = 1; i <= m; i++) {
while(k >= 0 && P[k+1] != P[i])
k = pi[k];
pi[i] = ++k;
}

Pattern Matching Implementation

int k = 0;
for(int i = 1; i <= n; i++) {
while(k >= 0 && P[k+1] != T[i])
k = pi[k];
k++;
if(k == m) {
// P matches T[i-m+1..i]
k = pi[k];
}
}

Suffix Trie

Suffix trie of a string is a rooted tree that stores

all the suffixes (thus all the substrings)
Each node corresponds to some substring of
Each edge is associated with an alphabet
For each node that corresponds to , there is a
special pointer called suffix link that leads to the
node corresponding to
Surprisingly easy to implement!

Suffix Trie Example

(Figure modified from Ukkonens original paper)

Incremental Construction

Given the suffix tree for 1

Then

we append +1 = to , creating necessary

nodes

Start at node corresponding to 1

Create

an -transition to a new node

Take the suffix link at to go to , corresponding

to 2
Create

an -transition to a new node

Create a suffix link from to

Incremental Construction

We repeat the previous process:

Take

the suffix link at the current node

Make a new -transition there
Create the suffix link from the previous node

We stop if the node already has an -transition

Because

from this point, all nodes that are reachable

via suffix links already have an -transition

Construction Example
a
b

b
a

Given the suffix trie for aba

We want to add a new letter c

Construction Example
a
b

1. Start at the green node

and make a c-transition

b
a

2. Then follow the suffix link

Construction Example
a
b

b
a

3. Make a c-transition at

4. Make a suffix link from

Construction Example
a
b

Construction Example
c

Construction Example
c

Suffix Trie Analysis

Construction time is linear in the tree size

But

the tree size can be quadratic in

e.g.

= aaabbb

Pattern Matching

To find , start at the root and keep following

edges labeled with 1 , 2 , etc.
Got stuck? Then doesnt exist in

Suffix Array
Input string

BANANA

Get all suffixes

1
2
3
4
5
6

BANANA
ANANA
NANA
ANA
NA
A

Sort the suffixes

6
4
2
1
5
3

A
ANA
ANANA
BANANA
NA
NANA

Take the indices

6,4,2,1,5,3

Suffix Array

Memory usage is
Has the same computational power as suffix trie
Can be constructed in time (!)
But

its hard to implement

There is an approachable log 2 algorithm

If

you want to see how it works, read the paper on the

course website
http://cs97si.stanford.edu/suffix-array.pdf

Note on String Problems

Always be aware of the null-terminators

Simple hash works so well in many problems
Even

for problems that arent supposed to be solved by

hashing

If a problem involves rotations of a string, consider

concatenating it with itself and see if it helps
Stanford team notebook has implementations of
suffix arrays and the KMP matcher