Loop Invariant Condition with Examples

Definition:

A loop invariant is a condition [among program variables] that is necessarily true immediately before and immediately after each iteration of a loop. (Note that this says nothing about its truth or falsity part way through an iteration.)

A loop invariant is some predicate (condition) that holds for every iteration of the loop.

For example, let's look at a simple for loop that looks like this:

int j = 9;
for(int i=0; i<10; i++)  
j--;

In this example it is true (for every iteration) that i + j == 9.

A weaker invariant that is also true is that i >= 0 && i <= 10.

One may get confused between the loop invariant, and the loop conditional ( the condition which controls termination of the loop ).

The loop invariant must be true:

• before the loop starts
• before each iteration of the loop
• after the loop terminates

( although it can temporarily be false during the body of the loop ).

On the other hand the loop conditional must be false after the loop terminates, otherwise, the loop would never terminate.

Usage:

Loop invariants capture key facts that explain why code works. This means that if you write code in which the loop invariant is not obvious, you should add a comment that gives the loop invariant. This helps other programmers understand the code, and helps keep them from accidentally breaking the invariant with future changes.

A loop Invariant can help in the design of iterative algorithms when considered an assertion that expresses important relationships among the variables that must be true at the start of every iteration and when the loop terminates. If this holds, the computation is on the road to effectiveness. If false, then the algorithm has failed.

Loop invariants are used to reason about the correctness of computer programs. Intuition or trial and error can be used to write easy algorithms however when the complexity of the problem increases, it is better to use formal methods such as loop invariants.

Loop invariants can be used to prove the correctness of an algorithm, debug an existing algorithm without even tracing the code or develop an algorithm directly from specification.

A good loop invariant should satisfy three properties:

• Initialization: The loop invariant must be true before the first execution of the loop.
• Maintenance: If the invariant is true before an iteration of the loop, it should be true also after the iteration.
• Termination: When the loop is terminated the invariant should tell us something useful, something that helps us understand the algorithm.

Loop Invariant Condition: 
Loop invariant condition is a condition about the relationship between the variables of our program which is definitely true immediately before and immediately after each iteration of the loop. 
For example: Consider an array A{7, 5, 3, 10, 2, 6} with 6 elements and we have to find maximum element max in the array. 

max = -INF (minus infinite)
for (i = 0 to n-1)
  if (A[i] > max)
      max = A[i]

In the above example after the 3rd iteration of the loop max value is 7, which holds true for the first 3 elements of array A. Here, the loop invariant condition is that max is always maximum among the first i elements of array A. 

Loop Invariant condition of various algorithms: 
Prerequisite: insertion sort, selection sort, quick sort, bubblesort, binary search

Selection Sort: 
In selection sort algorithm we find the minimum element from the unsorted part and put it at the beginning. 

min_idx = 0
 
for (i = 0; i < n-1; i++)
{
   min_idx = i;
   for (j = i+1 to n-1)
      if (arr[j] < arr[min_idx])
         min_idx = j;

    swap(&arr[min_idx], &arr[i]);
}

In the above pseudo code there are two loop invariant condition: 

1. In the outer loop, array is sorted for first i elements.
2. In the inner loop, min is always the minimum value in A[i to j].

Insertion Sort: 
In insertion sort, loop invariant condition is that the subarray A[0 to i-1] is always sorted. 

for (i = 1 to n-1)
{
   key = arr[i];
   j = i-1;
   while (j >= 0 and arr[j] > key)
   {
      arr[j+1] = arr[j];
      j = j-1;
   }
   arr[j+1] = key;
}

Quicksort: 
In quicksort algorithm, after every partition call array is divided into 3 regions: 

1. Pivot element is placed at its correct position.
2. Elements less than pivot element lie on the left side of pivot element.
3. Elements greater than pivot element lie on the right side of pivot element.

quickSort(arr[], low, high)
{
    if (low < high)
    {
        pi = partition(arr, low, high);
        quickSort(arr, low, pi - 1);  // Before pi
        quickSort(arr, pi + 1, high); // After pi
    }
}

partition (arr[], low, high)
{
    pivot = arr[high];   
    i = (low - 1) 
    for (j = low; j <= high- 1; j++)
        if (arr[j] <= pivot)
            i++;   
            swap arr[i] and arr[j]
    swap arr[i + 1] and arr[high])
    return (i + 1)
}

Bubble Sort: 

In bubble sort algorithm, after each iteration of the loop largest element of the array is always placed at right most position. Therefore, the loop invariant condition is that at the end of i iteration right most i elements are sorted and in place.  

for (i = 0 to n-1)         
  for (j = 0 to j  arr[j+1])
       swap(&arr[j], &arr[j+1]);

Find the index of the minimum value in an array of integers:

We have an array A, which has n elements in it. We keep two variables, nextToCheck and smallestSoFar. We initially set smallestSoFar to 0, and compare A[1], A[2], ..., A[n-1] against A[smallestSoFar], If the current comparison shows a smaller value, we update smallestSoFar.

Here is the Loop Invariant:

smallestSoFar in [0,nextToCheck),
and for all k in [0,nextToCheck), A[k] >= A[smallestSoFar].

[a,b) means all integers from a to b, including a, but not including b.

We set the Loop Invariant true before the loop by setting:

nextToCheck = 1 ;
smallestSoFar = 0 ;

Each time through the loop we increment nextToCheck by one, and fix smallestSoFar to keep the Loop Invariant true:

 if ( a[smallestSoFar] > a[nextToCheck] ) {
    smallestSoFar = nextToCheck ;
 }
 nextToCheck ++ ;

The termination condition is nextToCheck==n, i.e., no more to check.

The combination of the loop invariant and the termination condition gives that smallestSoFar is the index of the smallest value in the array.

Binary search:

bsearch(type A[], type a) {
start = 1, end = length(A)

  while ( start <= end ) {
      mid = floor(start + end / 2)

      if ( A[mid] == a ) return mid
      if ( A[mid] > a ) end = mid - 1
      if ( A[mid] < a ) start = mid + 1

  }
  return -1

}

The invariant is the logical statement:

if ( A[mid] == a ) then ( start <= mid <= end )

This statement is a logical tautology - it is always true in the context of the specific loop / algorithm we are trying to prove. And it provides useful information about the correctness of the loop after it terminates.

If we return because we found the element in the array then the statement is clearly true, since if A[mid] == a then a is in the array and mid must be between start and end. And if the loop terminates because start > end then there can be no number such that start <= mid and mid <= end and therefore we know that the statement A[mid] == a must be false. However, as a result the overall logical statement is still true in the null sense. ( In logic the statement if ( false ) then ( something ) is always true. )