Assignment 1.

1. Download and install the cameral calibration toolbox , written by Jean-Yves

Bouguet, from Canvas in the Assignment 1 module. The original source can be found
here: ( find the download in the Getting Started section ):

http://www.vision.caltech.edu/bouguetj/calib_doc/

2. Copy the toolbox folder and extract all contents. The default toolbox name is:

TOOLBOX_calib

Under MATLAB expand the Environment button and select Set Path. Add the path of
the extracted toolbox (on my Mac it is under the /Applications folder, but the exact
location is not important)

3. Also copy the folder “calib_example” consisting of five .tif images of a

checkerboard patter taken at different locations, which means the pattern was
photographed at different angles to the camera and different distances. This
method of camera calibration simplifies the procedure as the precise location of the
pattern is not needed, only identifying several points in the image ( that correspond
to known points in the image. The points are known because the pattern squares
have a known size ).

Note that the use of square patterns of known dimensions makes location of “corner
points” easier, and the user need only identify the outer points of the checkerboard
pattern. This assumes the lens does not distort the image substantially. If that is not
true, for example the biggest distortion normally found is a “radial distortion” when
the image will curve noticeably, particularly at the edge of the pattern. If this is
noticed, when the software identifies points that do not match the corner points, it
may be necessary to specify an initial value for radial distortion. While this often is
represented by a third order polynomial, for the initial value of radial distortion
only the first order term, called “kc” is needed.

Change, from within MATLAB, to the calib_example folder. You may use unix-like or
DOS like commands, such as cd to change directories, or ls or dir to display the
contents of a directoty.

4. From the MATLAB command prompt, type “calib_gui”. Once the toolbox path is
added to MATLAB’s path environment variable the command will run.

A small dialog box should appear with title “Camera Calibration Toolbox”. Select the
option of Standard. This will read all images into memory. The other option is for
memory limited systems. For the small number of files used in this assignment the
Standard option should be fine.
5. Click on the Read Image button and then type in the base name of the image files
used, in this case “Image”, then select the type, since these are tif files enter “t”

A set of thumbnail images should be shown, since we’re using five images you
should only see the five displayed.

Now click on the Extract grid corners button

Save each of the five images with the corners indicated. For the final image alone, an
initial value of kc will be beneficial. Save each image that is displayed for the value
you select. These images will be part of the document turned in for the assignment.

The toolbox will ask if all desired images are to be used, select the default answer,
which means not entry is required.

Select the defaults for wintx and winty, two separate entries. This will show the
given Window size, 11x11. This means the corner will be search in a window
around the indicated ( point clicked on ) plus dX and dY, so, in this case, the clicked
point plus and minus five points ( 11 points ) in each of the X and Y directions. Also
select the default for “automatic square counting”

Now the first image is presented, select the outer four corners, starting with the top
left then proceeding clockwise.

Now enter the square sizes uses, dX and dY. Both are 30 (for 30 mm), which is
different than the default. The corners for each square will be displayed. They
should be close to the actual corners.

Repeat for the remaining images. On the last image you should notice more radial
distortion, where it is possible to use an initial guess for kc, the first term in the
radial distortion model polynomial. In that image, try several different value of an
initial guess, starting with no guess then both a small positive and small negative
value for the value of kc. Note, if an initial guess is desired the software will need an
input of a numerical values, such as 1, than an initial guess, that is normally between
-1 and 1. These are usually fairly small, around .1 or -1, but may be larger for wide-
angle lenses or, especially, the so-called “fish-eye” lens.

6. After you are satisfied with the initial corner extraction, click on the Calibration
button on the toolbar. Record the initial error estimates ( a screen capture of the
MATLAB window will be sufficient )

7. Click on the Reproject on images button and save the display of the error plot.
Very likely, some errors will be small while there will be at least a few that are fairly
large, 0.5 pixels or larger.
8. Now click on the Recomp.corners button. This will repeat the corner estimation
routine with the current estimates as an initial guess. This will redo the numerical
corner estimation. Select the option to use all images.

Now redo the calibration step but clicking the Calibration button again. Record
these values ( another screen capture will be fine )

Click the Save button to save the calibration results.

9. Again, click on Reproject on images button. Save, or copy the plot showing the
error distribution.

10. Click on Analyze error and select one of the points near the edge of the plot.
Notice how the corner point that caused the error will be displayed in one of the
Figure windows.

11. Now click on the Show Extrinsic button. This will show the position of the
camera for each image. There are two views available, a camera centered view and
a world centered view. Save a copy of each view to include in your report.

12. Now, at the MATLAB command prompt, run the command:

“visualize_distortions” and save the plots for your report. There should be three
plots produced, a complete model, a tangential model, and a radial model. Notice
that the impact of the tangential model is small, and for these images the complete
model is very nearly the same as the radial model, which means the tangential error
could be ignored for this camera. This is a result of modern manufacturing methods
where lens and the image plane are not parallel. This is not common for modern
cameras while radial distortion still exists and can be significant for inexpensive
lenses, wide angle lenses, or high-magnification lenses.

Note: this is fine for this assignment as it is a means to demonstrate how camera
calibration can be done. Note that only a set of images, at different perspectives, is
required, along with the known size of each square. This is in contrast with earlier
methods that required precise knowledge of the movement of each image, that is the
distance and rotation angle previously needed to be known in precise detail.
Instead of that precision, as cost, known detail about the image, along with
numerical methods, are used to arrive at an accurate camera model.

For this assignment, this is sufficient. There is no reason to stop at only one
reprojection of the images. That is the purpose of the Analyze error button. If
desired, the sources of error could be investigated and those corner points that
contribute most to the error, could be re-estimated. While this involve a large
amount of clicking on the image and view of errors, this can be done with sufficient
time and using only common computer hardware and not specialize equipment.
This process could be repeated until observed error has been reduced to an
acceptable level.

Variations on this method exist, seeking to optimize the error in different ways. But
the basic idea is similar: use a closed form solution starting with a linear regression
(closed form) estimate, then refine that estimate using numerical methods ( such as
gradient descent ). The error that results can be determined by projecting the
estimated corner points back on the original images, as those are available. This can
be repeated, using the estimate as an initial guess for the next iteration.

Not part of this assignment: for some images the automatic corner detection may
fail, for lenses that have extreme radial distortion in particular. For those cases a
manual corner selection option exists in the commands (script files) available, not as
part of the GUI toolbox. This would involve much more selection of each corner in
the image, and fortunately is not needed here.

Create a document with the images and plots generated, and turn those in along
with answers to the following questions.

1. If the initial selection of corners is not accurate, how would that impact the final
result. Consider the fact that the initial corner estimate is used only in the closed
form, or linear regression, portion of the calibration routine. You should consider
two distinct cases: the initial corner selection is only slightly incorrect and if the
initial selection is significantly in error.

2. How could the “significant error” case be identified as part of the sequence of
steps followed above ?

3. How do you expect the error in the calibration model to change for the following
conditions:

a. Only two images are used, with only a slight rotation between them

b. A modest number of images are used, with many variations in image orientations,
such as rotations and translations.

c. A very large number images are used, with many different variations in image
orientations, rotations and translations. Keep in mind that several of these images
may be very similar to other images and that numerical methods are used as part of
the error reduction approach ( that is, to find the solutions of the camera model
equations ). Will the error always goes to zero as the number of images used
increases ? Why or why not?

d. If the entered square size is incorrect, the values of dX or dY, how could this be
identified during the calibration sequence used?
4. a. How important is even lighting for this calibration approach?

b. Why is a regular pattern of squares of known size used, why not just use the
outer corner points of each image ?

5. Increasing wintx and winty should make it easier to find corner points, and
therefore reduce error. Why is this statement not true for large values of wintx and
winty ? What could increase the error for large wintx and winty values depending
on the indicated (clicked) point ?

6. Read this blog post, from Cleve Molar of Mathworks, on one definition of a matrix
condition number, particularly the part on matrix inverses:
https://blogs.mathworks.com/cleve/2017/07/17/what-is-the-condition-number-
of-a-matrix/

How would the condition number of the matrix used in determining camera
calibration potentially be significant in finding the numerical solution?

7. Floating point machine arithmetic is not always exact. In particular, the

minimum difference between floating point numbers that a computer can resolve is
often called the “machine epsilon”. In MATLAB this is assigned the variable: eps.
Type “eps” at the command prompt in MATLAB and report the result. How is this
significant in determining the stopping condition of any numerical algorithm, such
as the gradient descent or Newton’s method used in camera calibration?