SORTING

INSERTION SORT:

PREREQUISITES​:
1) Comfort coding in C++.
2) Complexity

MOTIVATION:
Consider the following situation:
You are holding a deck of say 10 cards in your left hand. These 10 cards are sorted in
increasing order according to the number written on it. Now I give you another card. I
want you to see the number written on this card and accordingly insert it so that all the
11 cards(10+ the new card I gave you) are sorted according to the number written on it.
Think how would you do it?
Now, I give you another card. And tell you to insert this card appropriately so that all the
12 cards are now sorted.
Again I give you a card.
And so on…..

WHAT ARE WE EXACTLY DOING?

Every time I give you a new card, you would check the value written on this new card
with all the old card values and then place this new card in an appropriate place.

Let’s analyse what we are exactly doing more systematically with the help of an
example.

Say you are holding the following 5 cards in your left hand(sorted):
1 ; 3 ; 20 ; 100 ; 110
Now I give you a card with a value 2 written on it. The very obvious approach is to first
place this new card at the end of all the cards i.e after the last card (having value 110).
We get the following sequence:
1 ; 20 ; 100 ; 110 ; 2
But does this make our sequence sorted? No. To place the new card after 110, we had
to have this card’s value greater than equal to 110.
So, we swap the positions of 110 and 2 to get the following sequence:
1 ; 20 ; 100 ; 2 ; 110
But, even this does not make our sequence sorted. To place the new card here(3rd
index), it ought to have a value greater than 100(value at 2nd index). This is clearly not
the case. So we again swap the positions of 2 and 100 to get the sequence:
1 ; 20 ; 2 ; 100 ; 110
Does this make our sequence sorted? No. So swap the value of 2 with 20. We get the
following sequence:
1 ; 2 ; 20 ; 100 ; 110
Now does this make our sequence sorted? Yes.

So essentially when do we stop this process of swapping with the value of previous
index? The moment we find that the new card is present in an index such that its value
is greater than or equal to the value of the card present before it. If the new card
becomes the start of our sequence (in index 0), then we obviously stop as there are no
cards before it to be swapped with.
If you think carefully, intuitively this is what you do when you get a new card and have to
place it appropriately in a sorted deck of cards. We just wrote down the steps clearly as
this will help us to program it.

What we did is exactly insertion sort!

VISUALIZATION:
https://www.khanacademy.org/computing/computer-science/algorithms/insertion-sort/a/i
nsertion-sort
Instructions to visualise:
1. You will find the visualization box after the 3rd paragraph.
2. Your job is to click the”Next step” button and wait.
3. See the visualisation. Then again click “Next Step”.

STILL CAN’T GET THE IDEA:

You should have got the main idea by now! Incase you haven’t, don’t worry!
Go to the above link and read the entire text material.
Only thing is ignore the reference to selection sort in the first two paragraphs.

***In case you are interested feel free to look up selection sort on the internet. This
happens to be another sorting technique.

INSTEAD OF HAVING 10 SORTED CARDS IN YOUR HAND, YOU HAVE 0 SORTED

CARDS IN YOUR HAND!
Now instead of starting with 10 (or 5) cards let us start with 0 cards in your hand.
Again I give you a card. And then another. And another…..
You know what to do every time you get a new card, don’t you?

ANOTHER VISUALIZATION:
Some more visualizations in case you are not clear:
https://courses.cs.vt.edu/~csonline/Algorithms/Lessons/InsertionCardSort/index.html
Instructions to visualize:
1. Scroll down.
2. Click “start tutorial”
3. Once each part is done, you will find two buttons, “continue” and “replay”
4. Click “continue” if you want to proceed with the visualization.
5. Click “replay” if you want to again see the same segment.
6. Continue 5 or 6 till the end of visualization.
CODING INSERTION SORT IN C++:
Give this a try on your own first!
Then refer to this well commented code that implements insertion sort.
Link: ​https://ideone.com/R5aEya

COMPLEXITY ANALYSIS​:

Let’s do a worst case analysis of this algorithm.

Consider the following array:
{10,9,8,7,6,5,4,3,2,1}

Every time I am giving you a new card you have to go on till the 0th element swapping.
So how many swaps am I making?
In the first case: 0.
In the second case: 1
In the third case: 2
In the 4th case: 3
.
.
.
In the 10th case: 9
Can you recall a formula to calculate 1+2+3+....9. It is 9*(9+1)/2 = 45.
In general if we have N element we have to find the sum of : 1+2+3+.....(N-1)
= (N-1)*(N)/2
This in Big O Complexity we write O(N^2).

MERGE SORT:
PREREQUISITES:
1) Recursion
2) Functions
3) Complexity

Let us see another sorting technique by the name merge sort.

MERGING TWO SORTED ARRAYS TO GET A BIGGER SORTED ARRAY:

Let us consider the following array A:
{10,5,6,1,2,8,9,3,4,7}
We are required to sort this array.

Now I tell you the following. Let’s split this array into two equal parts, the first 5 elements
in 1 array and the next 5 elements in a separate array:
{10,5,6,1,2} and {8,9,3,4,7}
Now assume that magically I sort these two arrays and give it to you:
{1,2,5,6,10} and {3,4,7,8,9}
How can you use these two sub arrays (which I magically sorted and gave it to you) to
construct the main array A in sorted order?

We start by comparing the first elements in the two sub arrays. The one that is smaller
we put in an empty array B.

STEP 1:

1 2 5 6 10

^
​

3 4 7 8 9

^
​
After Step 1 the array B looks as:
1
We added 1 because 1<3. Then we move the ^ in the first array to the next element i.e
2 because the first element has already been added.

STEP 2:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Again we check the two elements indicated by the arrowheads and insert the smaller
one to the array B which looks as follows:

1 2

Since 2 is added we move the ^ in the first array to the next element i.e 5.

STEP 3:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Let’s insert the smaller of the two elements in array B:
1 2 3

Since we added 3 we move the ^ in the second array to the next element i.e 4.
STEP 4:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Let’s insert the smaller of the two element in B:

1 2 3 4

Since we added 4 we move the ^ in the second array to the next element i.e 7.

STEP 5:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Let’s compare the two elements indicated by the arrowheads in the two arrays and
insert the smaller one into array B:

1 2 3 4 5

We move the ^ in the first array to the next element.

STEP 6:
 1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Array B becomes:

1 2 3 4 5 6

STEP 7:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Array B becomes:

1 2 3 4 5 6 7

STEP 8:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Array B becomes:
1 2 3 4 5 6 7 8

STEP 9:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Array B becomes:

1 2 3 4 5 6 7 8 9

STEP 10:

1 2 5 6 10

​ ^
3 4 7 8 9

^
​
Since we have moved the ^ of the second array out of the array we have no other option
but add the remaining elements in the first array to Array B which now becomes:

1 2 3 4 5 6 7 8 9 10

Once this is done we copy array B into array A and we get sorted array A.
Great!
Can you come up with the code to do the above. Think about it. The solution will be
given at the end of this tutorial.

LET’S NOT DEPEND ON MAGIC!

We said that, when we wanted to sort array A, we divided it into two smaller arrays and
then magically we got these two smaller arrays sorted. Then we saw how, once we
have the two smaller arrays sorted, we could easily merge them to get the original
sorted array.
Now I say that let us not depend on any magic. We had splitted the array A =
{10,5,6,1,2,8,9,3,4,7} into two arrays {10,5,6,1,2} and {8,9,3,4,7}. Instead of assuming
that these two arrays got sorted magically let us sort them.
How do we do it?
Apply the same procedure.
Divide each of the smaller arrays into two halves further.
From the sub array {10,5,6,1,2} we get the following two sub subarrays: {10,5,6} and
{1,2}. Now instead of relying on magic again to sort these arrays, let us sort these
ourselves. Again divide the sub sub arrays further into two parts.
Can you picturize now what will this look like and eventually what do we come to?
We will have the following recursion tree:

This part has been intentionally left blank.

 10 5 6 1 2 8 9 3 4 7
|
V
-------------------------------------------------------------------
| |
V V

10 5 6 1 2 8 9 3 4 7

| |
| |
V V
--------------------------------------------------- -----------------------------------------------
| | | |
V V V V

10 5 6 1 2 8 9 3 4 7

| | | |
V V V V
--------------------- ------------------- ------------------- ----------------------
| | | | | | | |
V V V V V V V V

10 5 6 1 2 8 9 3 4 7

| |
V V
-------------- ---------------
| | | |
10 5 8 9
Let us consider the arrays that have only 1 element. Aren’t these arrays magically
sorted? So we saw how applying the process of dividing an array into two parts
eventually led us to a magically sorted array i.e an array of size 1.
Once we are at the bottom of the recursion tree can we not build up to the top? Yes,
absolutely yes. The bottom of the recursion tree would have an array of 1 element
which is sorted. We can move up and merge two sorted arrays just the way we saw in
the first half of the tutorial(where two arrays were sorted magically). This process
continues until we have the full array sorted.

VISUALIZATION:
Refer to this excellent post for visualizations:
https://www.commonlounge.com/discussion/98a5ec6dcb864b2eaf7af1e988e92d8f
***Do not miss the GIF.

COMPLEXITY ANALYSIS:
Now let us analyse the complexity of merge sort algorithm. Refer to the recursion tree.
How many times do you think we can keep on dividing an array of size 10. Let’s try it.
1) 10/2 = 5
2) 5/2 = 2
3) 2/2 = 1
4) ½ = 0
4. Yes? And what is the mathematical operator that does it? Log. Remember? So the
recursion tree will have Log (N) levels. For each layer how much time does it take to
merge? O(N). So our complexity is O(N*Log N).

STILL CAN’T GET THE IDEA? WANT A VIDEO TUTORIAL?:

Refer to this excellent video which follows the explanation in this tutorial:
https://www.youtube.com/watch?v=TzeBrDU-JaY

CODING MERGE SORT IN C++:

Great now that we know how merge sort works and why it works let us put all this into
code. Refer to this well commented solution: ​https://ideone.com/V6BTz7

CONCLUSION:
You may be wondering why do we need to learn sorting algorithms and why do
programmers lay so much stress on these algorithms. Well it turns out that sorting
algorithms appear in many other algorithms and problem solving paradigms.