Mo's Algorithm
Mo's Algorithm
Anudeep's blog
MOs Algorithm (Query square root decomposition)
Once again I found a topic that is useful, interesting but has very less resources online. Before writing this I did a small
survey and surprised that almost none of the Indian programmers knew this algorithm. Its important to learn this, all
the red programmers on codeforces use this effectively in div 1 C and D problems. There were not any problems on this
an year and half back, but from thenthere is a spike up! We can expect more problems on this in future contests.
1) State a problem
2) Explain a simple solution which takes O(N^2)
3) Slight modification to above algorithm. It still runs in O(N^2)
4) Explain an algorithm to solve above problem and state its correctness
5) Proof for complexity of above algorithm O(Sqrt(N) * N)
6) Explain where and when we can use abovealgorithm
7) Problems for practiceand sample code
State a problem
Given an array of size N. All elements of array <= N. You need to answer M queries. Each query is of the form L, R. You
need to answer thecount of values in range [L, R] which are repeated at least 3 times.
Example: Let the array be {1, 2, 3, 1, 1, 2, 1, 2, 3, 1} (zero indexed)
Query: L = 0, R = 4. Answer = 1. Values in the range [L, R] = {1, 2, 3, 1, 1} only 1 is repeated at least 3 times.
Query: L = 1,R = 8. Answer = 2. Values in the range [L, R] = {2, 3, 1, 1, 2, 1, 2, 3} 1 is repeated 3 times and 2 is repeated 3
times. Number of elements repeated at least 3 times = Answer = 2.
Explain a simple solution which takes O(N^2)
For each query, loop from L to R, count the frequency of elements and report the answer. Considering M = N, following
runs in O(N^2) in worst case.
1
2
3
4
5
6
7
add(position):
count[array[position]]++
if count[array[position]] == 3:
answer++
remove(position):
count[array[position]]-if count[array[position]] == 2:
answer-
currentL = 0
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
currentL = 0
currentR = 0
answer = 0
count[] = 0
for each query:
// currentL should go to L, currentR should go to R
while currentL &lt; L:
remove(currentL)
currentL++
while currentL &gt; L:
add(currentL)
currentL-while currentR &lt; R:
add(currentR)
currentR++
while currentR &gt; R:
remove(currentR)
currentR-output answer
Initially we always looped from L to R, but now we are changing the positions from previous query to adjust to current
query.
If previous query was L=3, R=10, then we will have currentL=3 and currentR=10 by the end of that query. Now if the next
query is L=5, R=7, then we move the currentL to 5 and currentR to 7.
add function means we are adding the element at position to our current set. And updating answer accordingly.
removefunction means we are deleting the element at position from our current set. And updating answer accordingly.
Edit: Have a look aShubajit Sahas comment. And also this.
Explain an algorithm to solve above problem and state its correctness
MOs algorithm is just an order in which we process the queries. We were given M queries, we will re-order the queries
in a particular order and then process them. Clearly, this is an offline algorithm. Each query has L and R, we will call
them opening and closing. Let us divide the given input array into Sqrt(N) blocks. Each block will be N / Sqrt(N) =
Sqrt(N) size. Each opening has to fall in one of these blocks. Each closing has to fall in one of these blocks.
A query belongs to Pth block if the opening of that query fall in Pth block. In this algorithm we will process the queries
of 1st block. Then we process the queries of 2nd block and so on.. finally Sqrt(N)th block. We already have an ordering,
queries are ordered in the ascending order of its block. There can be many queries that belong to the same block.
From now, I will ignore about all the blocks and only focus on how we query and answer block 1. We will similarly do for
all blocks. All of these queries have their opening in block 1, but their closing can be in any block including block 1. Now
let us reorder these queries in ascending order of their R value. We do this for all theblocks.
How does the final order look like?
All the queries are first ordered in ascending order of their block number (block number is the block in which its
opening falls). Ties are ordered in ascending order of their R value.
For example consider following queries and assume we have 3 blocks each of size 3.
{0, 3} {1, 7} {2, 8} {7, 8} {4, 8} {4, 4} {1, 2}
Let us re-order them based on their block number.
{0, 3} {1, 7} {2, 8} {1, 2} {4, 8} {4, 4} {7, 8}
Now let us re-order ties based on their R value.
{1, 2} {0, 3} {1, 7} {2, 8} {4, 4} {4, 8} {7, 8}
Now we use the same code stated in previous section and solve the problem. Above algorithm is correct as we did not
do any changes but just reordered the queries.
Proof for complexity of above algorithm O(Sqrt(N) * N)
We are done with MOs algorithm, it is just an ordering. Awesome part is its runtime analysis. It turns out that the
O(N^2) code we wrote works in O(Sqrt(N) * N) time if we follow the order i specified above. Thats awesome right, with
just reordering the queries we reduced the complexity from O(N^2) to O(Sqrt(N) * N), and that too with out any further
modification to code. Hurray! we will get AC with O(Sqrt(N) * N).
Have a look at our code above, the complexity over all queries is determined by the 4 while loops. First 2 while loops
can be stated as Amount moved by left pointer in total, second 2 while loops can be stated as Amount moved by
right pointer. Sum of these two will be the over all complexity.
Most important. Let us talk about the right pointer first. For each block, the queries are sorted in increasing order, so
clearly the right pointer (currentR) moves in increasing order. During the start of next block the pointer possibly at
extreme end will move to least R in next block. That means for a given block, the amount moved by right pointer is
O(N). We have O(Sqrt(N)) blocks, so the total is O(N * Sqrt(N)). Great!
Let us see how the left pointer moves. For each block, the left pointer of all the queries fall in the same block, as we
move from query to query the left pointer might move but as previous L and current L fall in the same block, the
moment is O(Sqrt(N)) (Size of the block). In each block the amount left pointer movies is O(Q * Sqrt(N)) where Q is
number of queries falling in that block. Total complexity is O(M * Sqrt(N)) for all blocks.
There you go, total complexity is O( (N + M) * Sqrt(N)) = O( N * Sqrt(N))
Explain where and when we can use abovealgorithm
As mentioned, this algorithm is offline, that means we cannot use it when we are forced to stick to given order of
queries. That also means we cannot use this when there are update operations. Not just that, there is one important
possible limitation: We should be able to write the functions add and remove. There will be many cases where add is
trivial but removeis not. One such example is where we want maximum in a range. As we add elements, we can keep
track of maximum. But when we remove elements it is not trivial. Anyways in that case we can use a set to add
elements, removeelements and report minimum. In that case the add and delete operations are O(log N) (Resulting in
O(N * Sqrt(N) * log N) algorithm).
There are many cases where we can use this algorithm. In few cases we can also use other Data Structures like
segment trees, but for few problems using MOs algorithm is a must. Lets discuss few problems in the next section.
Problems for practice and sample code
DQUERY SPOJ: Number of distinct elements in a range = number of elements with frequency >= 1. So it is the same
problem we discussed above.
Click here for sample code
Note: That code will give TLE on submission, it will give AC if fast I/O is added. Removed fast I/O to keep code clean.
Powerful array CF Div1 D: This is an example where MOs algorithm is a must. I cannot think of any other solution. CF
Div1 D means it is a hard problem. See how easy it is using MOs algorithm in this case. You only need to modify add(),
remove() functions in above code.
GERALD07 Codechef
GERALD3 Codechef
Tree and Queries CF Div1 D
Powerful Array CF Div1 D
Jeff and Removing Periods CF Div1 D
Sherlock and Inversions Codechef
I am sure there are more problems, if you know any of them, do comment, i will add them.
While this algorithmhas a special name MO, it is just smart square root decomposition.
Signing off! Wish you a happy new year
64 Replies
Previous
Next
Leave a Reply
Enter your comment here...
Hi Anudeep,thanks for the amazing post. Although not a bug but I would like to mention to the readers that the
code snippet given in the blog suffers from a logical counter intuitiveness that remove operation of an element
can happen before add operation on it. For example n = 8, and we have 2 queries (0,7) ans (1,4) then after
ordering the queries appear in (1,4) followed by (0,7) and the code snippet removes 0th element while increasing
the left pointer while it was not added previously. Now this may cause serious problem because in some
problems where this assumption is necessary to ensure certain quantity does not become negative. For
problems that require algebraic manipulations e.g powerful array , this is not a problem because algebraic
operations like addition and subtraction are order independent. For example in problem
http://codeforces.com/problemset/problem/375/D from codeforces you need to maintain a BIT of frequency of
occurrences of an element and if remove is permitted before addition than this frequency can go negative and
there by causing out of index or infinite loop problems. A simple get around is to first group the add while loops
followed by remove while loops instead of grouping them on the basis of left and right pointers to maintain the
logical intuitiveness. I think it will be wise to mention this in the blog.
Reply
Why my code getting runtime error ?? As i guess, itsindex out of bound, but Where is index out of bound
happening ?? Anyone can help ??
Dquery Problem Link : http://www.spoj.com/problems/DQUERY/
Ideone code link : http://ideone.com/kv6j17
Reply
algorithms involving segment trees seem far better than this i think
constructing segment tree takes O(N) time and then if Q queries are there then it could be done in just O(Q * lg
N) time.. Where N is the size of array
https://www.codechef.com/problems/CLOSEFAR
This problem brought me here !
Reply
I havent seen the editorial but I think that the problem can be solved using SQRT_Decmoposition and MOs
approach.
First Point : lets say there are e no. of edges, write down all the edges from 1 to e as 1,2,3, . . . , e
now do the sqrt decomposition of this list and for each chunk calculate how many connected components
will remain if the graph only had these edges.
After decomposition is over , be ready to answer the queries via MOs approach by sorting them in the
correct order.
Now assume that answer for [a,b] is known in previous step , the left point of this interval also falls in the
same chunk in which the left point of previous one falls ,this will look either like
[a+k,b+j] or [a-k,b+j] or [a+k,b] or [a-k,b], where j,k are positive, j takes at most e-1 and k takes at most
sqrt(e), hence guaranteeing the MOs suggested complexity.
The big issue is to find out how do we calculate the answer for this new interval, but this should not be an
issue if you know well how to use UNION-FIND Data Structure.
Reply
why I am getting WA
can anyone provide some testcase where my code gives wrong answer,I have tried many testcases but couldnt
find any mistake.
problem- http://www.spoj.com/problems/DQUERY/
code-https://ideone.com/4RJi9p
Reply
Another very useful and new algo thanks anudeep for such a nice effort .
Reply
Powerful array does not need fast IO, if you use cin.tie(NULL);
Reply
Can you please tell mistake in my add() and remove() function? My Solution
Thank You.
Reply
We can use a heap with square root decomposition to find the minimum of the range.
Reply
http://www.spoj.com/problems/ZQUERY/
This problem can be done using Mos algorithm.
Reply
very nice approach to MOs algorithm.i just dont understand why do we get wrong answer if we change the
code in DQUERY for the right pointer to
while(current_right right) {
remove(current_right]);
current_right;
}
and give the initial value of current_right =-1 and current_left=0
its give the correct answer for test cases but wrong on SPOJ
Reply
In your solution of DQUERY you fixed the block size according to max no of elements possible. So we do not
have to change the block size according to no of elements given in the question every time?
Like if n = 10 then also block size remains 555 ?
Reply
no need for fast i/o for dquery.. got acc using your concept just declare remove and add inline functions.
Reply
http://ideone.com/mpSL6G
Can you please tell me where am I going wrong ..
I implemented Mos Algorithm and yet I get a TLE
Is the implementation correct
COmpared with your code I found it almost identical.
Reply
hello Anudeep,
problem : powerful array can be solved online with the same complexity O((Q+N)*sqrt(N)). we can discuss it if u
want.
Reply
Hi, can you explain me how to modify ADD and REMOVE function to solve powerful array problem of codeforces.
Reply
How to find max. element in range. can you please explain the remove part in detail ?
I am not able to understand this we can use a set to add elements, remove elements and report minimum
Reply
http://www.spoj.com/problems/FREQUENT/
Sir i solved this problem using segmentree but i think it can be solved with MOs algorithm too . if it can be
solved using MOs algorithm , can you tell how to do so?
Reply
Hey, the links to the problems are not linked properly I guess.
Reply
Sir ,
Tanks for a nice tutorial !!
Reply
Yes, I added a problem from codechef which asks for the same. Thank you
Reply
Thank you.
Reply
Sir,
I think there is bug in your slide modification code which run in 0(n*n).
if suppose if have an array of 9 element indexing from 0
123123123
and there are
6 queries
query:1) L=0 R=8
answer should be 3 but came 2 //wrong answer
2) L=1 R=7
answer should be 1 and also came 1 //right answer
3) L=2 R=8
answer should be 1 and also came 1 //right answer
4) L=4 R=7
answer should be 0 but came 1 //wrong answer
5) L=2 R=8
answer should be 1 and also came 1 //right answer
3) L=2 R=7
answer should be 0 but came 1 //wrong answer
Please correct me if i am wrong.
Reply
Sir, u wrote it correctly in sample code but in blog u wrote it wrongly . so please update this
Reply
I know it, but to keep things simple i did not include the border cases clearly.
Reply
k sir.
and happy New Year
Reply
Yes, it can be wrong. I really did not look into the boundary conditions and implementation details, scope of
this article is to introduce and explain the algorithm and provide enough code so that the reader can
implement it on its own.
Reply
Hi Anudeep,
Thanks a lot for this blog, after attempting RRJAM in Dec cookoff I had been trying to read about sqrt
decomposition, fenwick trees, etc. I also looked at your solution, but couldnt follow it completely. I know its a lot
to ask, but if you could explain your RRJAM solution, it would really be helpful
PS: If you have any relevant resources to learns these related topics, pls do share them too.
Thanks
Reply
understand . Can you post something for the problem to help more like me facing the same issue.
Thank you
Reply
DQUERY Done
Reply
Out of curiosity, is this name used commonly in the literature? Or is this name used specifically by competitive
programming community? I searched online for a reference to MOs algorithm and could not find it.
Reply
What exactly you did not understand? Those lines summarize the paras above those and gives an example.
Reply
Hi Anudeep,
Firstly, good initiative with posting algorithms that are not very familiar to most. Please do continue
this.
I too didnt understand how (1,2) came after (2,8) after block wise ordering. Isnt the division of
inputs into blocks is based on L-value And then ties are sorted based on R-value?
Reply
Initial ordering is block based. As i told the block size is 3, L = 0,1,2 will be in the same block.
Reply
1) Block number.
2) On ties with block number, then on R value
if some one feel need here is link to my solution .. I think i have coded in such a manner that one can understand
.. after putting some efforts ..
https://www.hackerrank.com/contests/w12/challenges/white-falcon-and-tree/submissions/code/2445084
Reply
Hello Anudeep2011
Add this to the above list . This one is a tough problem which i solved a month ago using sqrt decomposition +
heavy light decomposition ..
Reply
Problem link :
https://www.hackerrank.com/contests/w12/challenges/white-falcon-and-tree
Reply
Categories
Algorithms (2)
Data Structures (1)
Machine Learning (1)
Segment trees (2)
SPOJ (1)
Uncategorized (1)
Recent Posts
Machine learning everywhere, why not in Competitive programming? April 18, 2016
Recent Comments
Shubajit Saha on MOs Algorithm (Query square root decomposition)
Aatmaram on Persistent segment trees Explained with spoj problems
Raiya Deep on MOs Algorithm (Query square root decomposition)
coded_mind on MOs Algorithm (Query square root decomposition)
Mudit Jain on Machine learning everywhere, why not in Competitive programming?
Contact Me
Ask about something or report me a bug or just say Hi. I will be happy to get back.