CS 478: Design and Analysis of Algorithms

Loop Invariant
with Examples

Fall 2024
Mr. Ahsan Shah

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

What is an Algorithm?

• The word Algorithm means ” A set of finite rules or instructions

to be followed in calculations or other problem-solving
operations and also gives result in finite interval of time”
OR
” A procedure for solving a mathematical problem in a finite
number of steps that frequently involves recursive operations”.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

What is the need for algorithms?

• Algorithms are necessary for solving complex problems efficiently
and effectively.

• They help to automate processes and make them more reliable,

faster, and easier to perform.

• Algorithms also enable computers to perform tasks that

would be difficult or impossible for humans to do manually.

• They are used in various fields such as mathematics, computer science,

engineering, finance, and many others to optimize processes,
analyze data, make predictions, and provide solutions to problems.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

What are the Characteristics of an Algorithm?

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

How to Design an Algorithm?

• To write an algorithm, the following things are needed as a pre-requisite:

• The problem that is to be solved by this algorithm i.e. clear problem
definition.
• The constraints of the problem must be considered while solving the
problem.
• The input to be taken to solve the problem.
• The output is to be expected when the problem is solved.
• The solution to this problem is within the given constraints.
• Then the algorithm is written with the help of the above parameters such that
it solves the problem.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Algorithm Correctness

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Loop Invariant

• Definition:
• A loop invariant is a condition [among program variables] that is
necessarily true immediately before and immediately after each
iteration of a loop. (Note that this says nothing about its truth or
falsity part way through an iteration.)

• A loop invariant is some predicate (condition) that holds for

every iteration of the loop.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Loop Invariant vs Mathematical Induction

• When the first two properties hold, the loop invariant is true prior to every iteration of
the loop.

• Note the similarity to mathematical induction, where to prove that a

property holds, you prove a base case and an inductive step.

• Here, showing that the invariant holds before the first iteration is like
the base case, and showing that the invariant holds from iteration to
iteration is like the inductive step.

• The third property is perhaps the most important one, since we are using the loop
invariant to show correctness. It also differs from the usual use of mathematical
induction, in which the inductive step is used infinitely; here, we stop the “induction”
when the loop terminates.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Example

For example, let’s look at a simple for loop that looks like this:
int j = 9;
for(int i=0; i<10; i++)
j--;
In this example it is true (for every iteration) that i + j == 9.

A weaker invariant that is also true is that i >= 0 && i <10.

One may get confused between the loop invariant, and the loop conditional ( the condition which
controls termination of the loop ).

The loop invariant must be true:

• before the loop starts
• before each iteration of the loop
• after the loop terminates

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Usage

• Loop invariants capture key facts that explain why code works.
• A loop Invariant can help in the design of iterative algorithms
when considered an assertion that expresses important
relationships among the variables that must be true at the start
of every iteration and when the loop terminates. If this holds, the
computation is on the road to effectiveness. If false, then the
algorithm has failed.
• Loop invariants are used to reason about the correctness of
computer programs. Intuition or trial and error can be used to
write easy algorithms however when the complexity of the
problem increases, it is better to use formal methods such as
loop invariants.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Loop Invariant Condition

• A good loop invariant should satisfy three properties:

• Initialization: The loop invariant must be true before

the first execution of the loop.
• Maintenance: If the invariant is true before an iteration
of the loop, it should be true also after the iteration.
• Termination: When the loop is terminated the invariant
should tell us something useful, something that helps
us understand the algorithm.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Loop Invariant Condition

• For example: Consider an array A{7, 5, 3, 10, 2, 6} with 6

elements and we have to find maximum element max in the
array.

• In the above example after the 3rd iteration of the loop max
value is 7, which holds true for the first 3 elements of array A.
Here, the loop invariant condition is that max is always
maximum among the first i elements of array A.
Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi
 CS 478: Design and Analysis of Algorithms

Insertion Sort and Loop Invariant

for (i = 1 to n-1)
{
key = arr[i];
j = i-1;
while (j >= 0 and arr[j] > key)
{
arr[j+1] = arr[j];
j = j-1;
}
arr[j+1] = key;
}

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
2.7 7.4 0.5 1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 8 2 6 2 7 2 1 2 4 1

Iteration 0: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
2.7 7.4 0.5 1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 8 2 6 2 7 2 1 2 4 1

Iteration 1: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
2.7 0.5
7.4 7.4
0.5 1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 8 2
6 6
2 2 7 2 1 2 4 1

Iteration 2: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5
2.7 2.7
0.5 7.4 1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 8
6 6
8 2 2 7 2 1 2 4 1

Iteration 2: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 2.7 7.4 1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 6 8 2 2 7 2 1 2 4 1

Iteration 2: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 2.7 1.1
7.4 7.4
1.1 1.1 0.3 6.2 4.4 3.1 7.7
Value 6 8 2 2 7 2 1 2 4 1

Iteration 3: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1
2.7 2.7
1.1 7.4 1.1 0.3 6.2 4.4 3.1 7.7
Value 6 8
2 2
8 2 7 2 1 2 4 1

Iteration 3: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 2.7 7.4 1.1 0.3 6.2 4.4 3.1 7.7
Value 6 2 8 2 7 2 1 2 4 1

Iteration 3: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 2.7 1.1
7.4 7.4
1.1 0.3 6.2 4.4 3.1 7.7
Value 6 2 8 2
7 7
2 2 1 2 4 1

Iteration 4: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 1.1
2.7 2.7
1.1 7.4 0.3 6.2 4.4 3.1 7.7
Value 6 2 8
7 7
8 2 2 1 2 4 1

Iteration 4: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 1.1 2.7 7.4 0.3 6.2 4.4 3.1 7.7
Value 6 2 7 8 2 2 1 2 4 1

Iteration 4: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 1.1 2.7 0.3
7.4 7.4
0.3 6.2 4.4 3.1 7.7
Value 6 2 7 8 2 2 1 2 4 1

Iteration 5: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 1.1 0.3
2.7 2.7
0.3 7.4 6.2 4.4 3.1 7.7
Value 6 2 7 8
2 2
8 2 1 2 4 1

Iteration 5: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 1.1 0.3
1.1 1.1
0.3 2.7 7.4 6.2 4.4 3.1 7.7
Value 6 2 7
2 2
7 8 2 1 2 4 1

Iteration 5: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.5 0.3
1.1 1.1
0.3 1.1 2.7 7.4 6.2 4.4 3.1 7.7
Value 6 2 2 7 8 2 1 2 4 1

Iteration 5: step 3.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3
0.5 0.5
0.3 1.1 1.1 2.7 7.4 6.2 4.4 3.1 7.7
Value 6
2 2
6 2 7 8 2 1 2 4 1

Iteration 5: step 4.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 7.4 6.2 4.4 3.1 7.7
Value 2 6 2 7 8 2 1 2 4 1

Iteration 5: step 5.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 6.2
7.4 7.4
6.2 4.4 3.1 7.7
Value 2 6 2 7 8 2
1 1
2 2 4 1

Iteration 6: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 6.2 7.4 4.4 3.1 7.7
Value 2 6 2 7 8 1 2 2 4 1

Iteration 6: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 6.2 4.4
7.4 7.4
4.4 3.1 7.7
Value 2 6 2 7 8 1 2 2 4 1

Iteration 7: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 4.4
6.2 6.2
4.4 7.4 3.1 7.7
Value 2 6 2 7 8 1
2 2
1 2 4 1

Iteration 7: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 4.4 6.2 7.4 3.1 7.7
Value 2 6 2 7 8 2 1 2 4 1

Iteration 7: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 4.4 6.2 3.1
7.4 7.4
3.1 7.7
Value 2 6 2 7 8 2 1 2
4 4
2 1

Iteration 8: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 4.4 3.1
6.2 6.2
3.1 7.4 7.7
Value 2 6 2 7 8 2 1
4 4
1 2 1

Iteration 8: step 1.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 3.1
4.4 4.4
3.1 6.2 7.4 7.7
Value 2 6 2 7 8 2
4 4
2 1 2 1

Iteration 8: step 2.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 3.1 4.4 6.2 7.4 7.7
Value 2 6 2 7 8 4 2 1 2 1

Iteration 8: step 3.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 3.1 4.4 6.2 7.4 7.7
Value 2 6 2 7 8 4 2 1 2 1

Iteration 9: step 0.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

 CS 478: Design and Analysis of Algorithms

Insertion Sort
• Iteration i. Repeatedly swap element i with the one to its left if smaller.

• Property. After ith iteration, a[0] through a[i] contain first i+1 elements in
ascending order.

Array index 0 1 2 3 4 5 6 7 8 9
0.3 0.5 1.1 1.1 2.7 3.1 4.4 6.2 7.4 7.7
Value 2 6 2 7 8 4 2 1 2 1

Iteration 10: DONE.

Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi