Corner Detection with Shi-Tomasi Corner Detection Method using OpenCV

Corner detection is a key technique in computer vision used to identify important points in an image where there is a significant change in intensity across multiple directions. These corners are important for tasks like feature matching, motion tracking and image stitching. Shi-Tomasi Corner Detection method works by scanning small regions of an image, identifying areas where the intensity changes noticeably in all directions.

Flat regions will have no change in any direction.

If there's an edge, then there will be no major change along the edge direction.

Why Use Shi-Tomasi Corner Detection?

The Shi-Tomasi method is used for detecting the most prominent corners that can be tracked across different frames in a video or image sequence. Let's see why it is beneficial:

1. Efficient and Fast: It is computationally simpler than Harris and faster in execution, making it suitable for real-time applications.
2. Accurate Corner Detection: It accurately detects the most prominent corners, even in noisy images.
3. Robust: The method is robust in detecting corners on textured images or images with simple backgrounds.

Mathematical Overview

The Shi-Tomasi method for corner detection is based on analyzing the intensity changes in a small window of pixels in an image. Let's see how it works:

1. Corner Intensity Change Formula: For a small window(W) at location (X, Y) in an image, the intensity I(X, Y) can be used to calculate the change in intensity across different directions. The change in intensity is calculated using the following formula:

f(X, Y) = Σ (I(X_k, Y_k) - I(X_k + ΔX, Y_k + ΔY))^2

where

• (X_k, Y_k) are the coordinates of the pixels within the window W.
• ΔX,ΔY represent the shifts in the x and y directions.

In simple terms, if we scan an image with a small window and notice a large change in intensity in any direction, we have a good chance of detecting a corner.

2. Simplification Using Taylor Expansion: Directly calculating f(X, Y) is computationally expensive, we simplify the process using the Taylor expansion. The simplified formula involves finding the eigenvalues (λ_1, λ_2) of the matrix where the corner response R is defined as:

R = min(λ1, λ2)

• λ1 and λ2 are the eigenvalues of the matrix derived from the intensity changes in both directions.
• R is the corner response where a smaller R shows a corner point.

3. Corner Detection:

• A region is identified as a corner if R is significantly high (compared to the surrounding areas).
• By comparing R values, the algorithm identifies the most prominent corners.

Using the goodFeaturesToTrack() Function in OpenCV

The OpenCV function cv2.goodFeaturesToTrack() helps us implement the Shi-Tomasi method for corner detection. This function detects the strongest corners in an image based on the Shi-Tomasi criterion.

Function Syntax:

cv2.goodFeaturesToTrack(gray_img, maxCorners, qualityLevel, minDistance)

Parameters:

1. gray_img: Input image in grayscale.
2. maxCorners: Maximum number of corners to detect. A negative value returns all detected corners.
3. qualityLevel: A parameter between 0 and 1 that defines the minimum quality of corners to be accepted. It is typically set to 0.01 for good results.
4. minDistance: Minimum distance between detected corners. It ensures that corners are not detected too close to each other.

Implementing Shi-Tomasi Corner Detection

Let’s now implement Shi-Tomasi Corner Detection using OpenCV and visualize the corners detected in an image. Here we will be using OpenCV, Numpy and Matplotlib libraries for this task.

• gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY): Converts the input image to grayscale, as corner detection algorithms typically work on single-channel images.
• corners = cv2.goodFeaturesToTrack(gray_img, 100, 0.01, 10): Detects the 100 most prominent corners based on intensity change, with a quality level of 0.01 and minimum distance of 10 pixels between corners.
• corners = np.int32(corners): Converts the floating-point corner coordinates to integers to allow for proper drawing on the image.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('/content/inputchess.png')

gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

corners = cv2.goodFeaturesToTrack(gray_img, 100, 0.01, 10)
corners = np.int32(corners)

for i in corners:
    x, y = i.ravel()
    cv2.circle(img, (x, y), 3, (0, 0, 255), -1)  

img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

plt.imshow(img_rgb)
plt.title('Shi-Tomasi Corner Detection')
plt.axis('off')  
plt.show()

Input :

Output:

Advantages of Shi-Tomasi Corner Detection

1. Efficient and Fast: This method is computationally less expensive than others like Harris, making it faster and suitable for real-time applications.
2. Real-Time Performance: It is well-suited for tasks like feature tracking in videos.
3. Simple Implementation: The method is simple to implement and works well for detecting a limited number of prominent corners.

Challenges of Shi-Tomasi Corner Detection

1. Limited Corners: This method is more suitable for detecting a smaller number of corners. If the image contains many corners, other methods like Harris may be more appropriate.
2. Sensitivity to Parameters: The quality of corner detection depends on the parameters such as qualityLevel and minDistance which need to be set correctly.
3. Low Contrast Issues: Shi-Tomasi may not perform well in images with low contrast or where there is little intensity variation.