Photogrammetry

Lecture 2 Handling rotations, matrixes, and finding the inner and outer orientation

Overview of Photogrammetry Lectures

Four Lectures and one workshop session
Basic Maths of Photogrammetry Handling rotations, matrixes, finding the inner and outer orientation Relative and Absolute Orientations, Orthorectification, Block Adjustments, aero triangulation Softcopy Photogrammetry, Photogrammetry and GIS, Satellite Sensors.

Lecture Overview
Review of Matrix Algebra Transformations between coordinate systems Collinearity Equations Using control points to find the inner and outer orientation

Matrices
A Matrix is an array of numbers in two dimensions A Vector is an array of numbers in one dimension Matrices are convenient ways to represent sets of numbers and operations on them. Matrices can represent complex operations very simply and economically.
1 0 2 A = 1 1 0 0 1 1

0 1 2 B = 1 2 0 0 1 1

12 x = 12 5

10 y = 15 9

Vectors Matrices Matrices are usually represented by a boldface upper case letter. Vectors are usually represented by a boldface lower case letter.

Matrix Addition is achieved by Adding the corresponding elements. Matrices must have the same number of rows and columns.
a 00 A = a10 a 20 a 01 a 02 a11 a12 a 21 a 22

b 00 b01 b 02 B = b10 b11 b12 b 20 b21 b22 a 01 + b 01 a 11 + b11 a 21 + b21 a 02 + b02 a 12 + b12 a 22 + b22

a 00 + b00 A + B = a 10 + b10 a 20 + b 20

In scalar multiplication, each matrix element is multiplied by the same quantity

sa 00 sA = sa10 sa 20 sa 01 sa 02 sa11 sa12 sa 21 sa 22

 In Matrix Multiplication, the number of columns in the left hand matrix must match the number of rows in the right hand matrix. We proceed across the rows of the left hand matrix, and down the columns of the right hand matrix
a 00 a 01 a 02 A = a10 a11 a12 a 20 a 21 a 22

b00 b 01 b 02 B = b10 b11 b12 b20 b 21 b 22

a 00b00 + a 01b10 + a 02b20 a 00b01 + a 01b11 + a 02b21 a 00b02 + a 01b12 + a 02b22 AB = a10b 00 + a11b10 + a12 b20 a10b 01 + a11b11 + a12b21 a10b 02 + a11b12 + a12b22 a 20b00 + a 21b10 + a 22 b20 a 20b 01 + a 21b11 + a 22b 21 a 20b02 + a 21b12 + a 22b22

Matrix times a Vector:

a 00 a 01 a 02 x a 00 x + a 01 y + a 02 z Ax = a10 a11 a12 y = a10 x + a11 y + a12 z a 20 a 21 a 22 z a 20 x + a 21 y + a 22 z

Matrix Multiplication is not commutative.

AB BA

The Identity Matrix is defined as

1 0 0 I = 0 1 0 0 0 1

For a square matrix, it may be possible to find an inverse, denoted by the superscript -1. Then
AA 1 = A 1A = I

Any Introductory text on Linear Algebra will have details on finding the inverse.

 The transpose of a matrix is obtained by swapping the rows with the columns, and is denoted by a superscript T.
a 00 a 01 a 02 A = a10 a11 a12 a 20 a 21 a 22 a 00 a10 a 20 AT = a 01 a11 a 21 a 02 a12 a 22

Column 1 becomes Row 1

We will use matrix algebra to efficiently represent conversions between different coordinate systems.

Coordinate Systems - Review

Photo Coordinates - origin at centre of collimation. Denoted by (x,y) in notes. Image Coordinate System - origin at centre of Projection. Denoted by (xp,yp,-f) for the image point P (xp,yp,zp) for the actual point P related by central projection World Coordinate System - origin at specified location. Denoted by (X,Y,Z) In the next few slides, we will review how points can be transformed from one coordinate system to another

Translation
Z z y xp yp Y X0 Y0 X Z0 x
X = x + X0 X x X 0 Y = y + Y 0 Z z Z0

P zp

Rotation - 2D
Y y P x x cos y sin X X = x cos - y sin Y = x sin + y cos
X cos - sin x Y = sin cos y or X = Rx where r11 r 12 R= r 21 r 22

Z z

Rotation in 3D
There are a number of systems for representing rotations in three dimensions. They all must be specified with three independent parameters, and can be represented in matrix notation by a 9*9 matrix.

x X

 For any rotation in 3D,

X = Rx r 11 r12 r13 R = r 21 r 22 r 23 r 31 r 32 r 33

The elements r are functions of the three rotation parameters If we let cos(xX) represent the cosine of the angle between the axes x and X, we can write
r11 r 12 r 13 cos(xX) R = r 21 r 22 r 23 = cos(xY) r 31 r 32 r 33 cos(xZ) cos(yX) cos(yY) cos(yZ) cos(zX) cos(xY) cos(zZ)

 In Photogrammetry, a set of three angles of rotations around the three axes is often used to represent a three dimensional rotation. represents a counterclockwise rotation around the x axis, represents a counterclockwise rotation around the new y axis which was rotated by . represents a counterclockwise rotation around the new z axis, which was rotated by , and then by
z z y y x=x z

z = z y y x x

z y=y x x

 In the ,, system of specifying rotations, the rotation matrix has the following form
cos cos R = cos sin + sin sin cos sin sin - cos sin cos cos sin cos cos sin sin sin sin cos + cos sin sin - sin cos cos cos sin

Note that
R 1 = R T

Note also that the order of the rotations is important. A different overall rotation results from applying the rotational components in a different order.

Scaling
Y (x, y)
X 0 x Y = 0 y

(x,y)

In Three Dimensions and with different scale for each coordinate:

X 0 0 x Y = 0 0 y Z 0 0 z

General Transformation
A general transformation between coordinate systems may involve translation, rotation and scaling.
X 0 0 r11 r 12 r 13 x X0 Y = 0 0 r 21 r 22 r 23 y + Y 0 Z 0 0 r 31 r 32 r 33 z Z0

Such a transformation is a special case of an affine transformation, in which there is a one to one correspondence between points, lines and planes in each coordinate system. In an affine transformation, parallelism is retained. Affine transformations can be represented by
X a11 a 12 a13 x X0 Y = a 21 a 22 a 23 y + Y 0 Z a 31 a 32 a 33 z Z0

Collinearity Equations
Recall that the basis of locating a real world point P in an image was the fact that the projection centre, the image point, and the real world point lie on a straight line. The mathematical expression of this condition is one of the fundamental relationships of photogrammetry, and is referred to as the Collinearity Equations. Having developed a technique for dealing with transformations between coordinate systems, we can now develop the Collinearity Equations.

 Let the attitude of the photograph be given in reference to some world coordinate system X,Y,Z, by a rotation matrix R. That is any point (X,Y,0) would be rotated into a plane parallel to the photograph by the operation of R. Assume that the relationship between the image coordinate system and the photo coordinate system is completely specified by a translation. The central projection then becomes:
x x0 f = x z x x = x0 f z

and

y y0 f = y z y y = y0 f z

Where ( x,y) are the photo coordinates, (x0,y0) is the location of the principle point, and (x,y,z) is the location of the point in the image coordinate system, with origin at the centre of projection.

P
y0 x0

(xp,yp,-f)

(xp,yp,z p)

 We know that rotation R specifies the attitude of the photograph, and hence the image coordinate system. Therefore a rotation of R-1 will rotate the image coordinate system to be parallel with world coordinates. Since rotating the coordinate system is equivalent to an inverse rotation of a point, we will need to apply a rotation of R to the point (x,y,z) to obtain its coordinates in a new coordinate system parallel to the world coordinates. Then a simple vector addition can be used to find the world coordinates of the point.
X x X 0 Y = R y + Y 0 Z z Z 0

We can solve this equation for [x,y,z] T, and substitute back into the central projection equations

X X 0 X x X 0 x Y = R y + Y y = R T Y Y 0 0 Z z z Z 0 Z Z0

x = r11(X X0) + r 21( Y Y0) + r 31( Z Z0) y = r 12(X X0) + r 22( Y Y0) + r 32( Z Z0) z = r13(X X0) + r 23( Y Y0) + r 33( Z Z0)

Substituting into the central projection equations, we get:

x z r11( X X0) + r 21(Y Y0) + r 31( Z Z0) x = x0 f r 13( X X0) + r 23( Y Y0) + r 33(Z Z0) x = x0 f
y y0 f = y z r12( X X 0) + r 22 (Y Y 0) + r 32( Z Z0) y = y0 f r13(X X0) + r 23( Y Y0) + r 33( Z Z0)

(2.1)

(2.2)

 Equations (2.1) and (2.2) are known collectively as the collinearity equations, and give the photo coordinates of a point in terms of the
World Coordinates of P (X,Y,Z), the exterior orientation parameters
the location of the projection centre, (X0,Y0,Z0) and the attitude of the image plane, given by matrix R

the interior orientation parameters

the location of the principle point, (x0,y0) the focal length f

By using the collinearity equations we can take into account the deviations from normality in aerial photography. Improvements are still possible
lens distortion, misalignment between image plane and photocoordinates, atmospheric effects, etc

 The collinearity equations can also be framed to give the world coordinates X and Y as follows
X = X 0 + (Z Z0 ) Y = Y0 + ( Z Z0 ) r11 ( x x0 ) + r12 ( y y0 ) r13 f r31 ( x x0 ) + r32 ( y y0 ) r33 f r21 ( x x0 ) + r22 ( y y0 ) r23 f r31 ( x x0 ) + r32 ( y y0 ) r33 f

(2.3) (2.4)

If the inner and outer orientation are both known, and points are measured in a pair of overlapping photographs, we obtain (with above ratios denoted by k) (2.5) X = X01 + (Z Z01)k1 (2.6) Y = Y01 + (Z Z01)k2 (2.7) X = X02 + (Z Z02)k3 (2.8) Y = Y + (Z Z )k
02 02 4

 ki are functions of photo coordinates, the interior orientation parameters, and the elements of the rotation matrix for each camera. Camera One is located at (X01,Y01,Z01) and Camera Two is located at (X02,Y02,Z02). Equations (2.5) to (2.8) represent four equations with three unknowns, and so (X,Y,Z), the location in world coordinates of the imaged point can be computed. We have demonstrated how the collinearity equations can be used to perform stereo restitution in the case where inner and outer orientations are known, and the image plane can be oriented arbitrarily towards the world coordinate system. However we have not addressed how to find the inner and outer orientations. This is the subject of the next few sections.

Finding the Orientation Parameters

The Inner orientation parameters are often know, particularly with metric cameras. They are measured in a laboratory calibration process. However, if they are not known, as is the case when non-metric cameras are used (e.g. video, digital, 35mm), then they can be found. The principle for finding both interior and exterior orientation parameters involves the use of control points. Control points are easily identifiable points in the image, whose 3D coordinates are known exactly. The location of the control points can be established by
ground survey GPS topographic maps or GIS

The accuracy of the control points is a primary concern.

 Control points can be

special markings made on the ground easily identified locations such as building corners, fence intersections

For each control point, we can write the two collinearity equations:
xi = f ( x0 , f , , , , X0 , Y0 , Z0 , X i , Yi , Zi ) yi = f ( y0 , f , , , , X0 , Y0 , Z0 , Xi , Yi , Zi )

measured

unknown

known

Including the interior orientation parameters, there are 9 unknowns. Therefore we need at least 5 control points. If more control points are available, then we have redundant information, which can be used to improve the accuracy and reliability of the result.

 Unfortunately, the set of simultaneous equations is nonlinear, and a direct solution is not possible. The usual procedure is to linearize the equations using a Taylor Series Expansion, around an approximate solution. A least squares adjustment is the typical method for solving a set of redundant linear equations. If the interior orientation parameters are known, then there are only 6 unknowns. These are the exterior orientation parameters
the camera location (X0,Y0,Z0), and attitudes ,,

In this case a minimum of 3 control points is required. The problem of finding the exterior orientation for a single photograph using control points is called Space Resection.

Simultaneous orientation of stereo photographs

This procedure can give a better result than the previous method, because it also allows additional image points to be used. These additional points can be measured in both photographs, but their 3D coordinates are not known. For each control point there are four equations, with 12 unknowns (image coordinates are measured)
xi1 = f ( x0 , f , 1,1 , 1, X 01, Y01 , Z01 , X i , Yi , Zi ) yi1 = f ( y0 , f , 1,1 , 1, X01 , Y01, Z01, X i , Yi , Zi ) xi 2 = f ( x0 , f ,2 ,2 , 2 , X02 , Y02 , Z02 , X i , Yi , Zi ) yi 2 = f ( y0 , f , 2 , 2 , 2 , X 02 , Y02 , Z02 , Xi , Yi , Zi )

known

unknown

Known for control points, but not additional points

 For each additional point,

there are the 12 unknowns corresponding to the exterior orientation parameters. An additional 3 unknowns corresponding to the World Coordinates of the point. Each additional point adds four equations, and three unknowns to the system of equations, adding to the redundancy, and therefore accuracy of the solution.

This orientation procedure is considered by Krauss (1993) to be the most accurate, since any number of control and additional points can be used.

Two Step Orientation

So far we have found the exterior orientation parameters in a single step, by solving the collinearity equations for a number of control points. By taking more control points, we can increase the redundancy in the system of equations, and achieve a more reliable result. The method involves a numerical solution to the equations, and must be performed on a computer. Traditionally, exterior orientation has been performed as a two step process, following a procedure suited to analogue instruments.
Relative Orientation Absolute Orientation

Relative orientation
Eliminates the y-parallax between the two images of a stereo pair. Allows the construction of a 3D model coordinate system. Points can be precisely located within the model coordinate system. It is not known how the model coordinate system is related to the World Coordinate System Can be useful if 3D shape information is required, but absolute position or scale is not

Relative Orientation
Assume the left image is at model coordinates (X01,Y01,Z01) =(0,0,0) and ,, are 0. Assume the x coordinate of right image is Camera Two is X02 = b (base length) In relative orientation we determine the model coordinates Y02,Z02 of the right image and its three tilts For determining these 5 unknowns we need measurements to 5 or more corresponding image points in both images

z O1

Moving O2 parallel with the stereo base has no effect on the relative orientation

O2
Coplanarity: O1, O2 and P i must be coplanar

pi1

pi2

Pi

5 of these coplanarity conditions for 5 well distributed points P i are required to find the 5 elements of relative orientation

Relative orientation
Achieved by the intersection of five pairs of homologous rays at five well distributed points No control points are needed, since no reference is made to world coordinates Geometrically, each centre of projection and the model point must be coplanar. (Coplanarity condition)

Absolute orientation
We now rotate ,shift and scale the model into the world system If the model coordinate system is denoted by (x,y,z), and the world coordinate system by (X,Y,Z), then

X X mo Y = Y mo Z Z mo

+ mR

x y z

 (Xmo,Ymo,Zmo) is the origin of the model coordinate system R is a rotation matrix m is a scaling factor

Since R has three independent parameters, there are seven independent elements of absolute orientation. Therefore we need at least seven independent equations. There are three types of control points
Full Control point: (X,Y,Z) all known
Three equations

X = X mo + m (r11 x + r12 y + r13 z ) Y = Ymo + m ( r21 x + r22 y + r23z ) Z = Zmo + m ( r31x + r32 y + r33 z )
Planimetric Control Point: (X and Y known)
Two equations, one for X and one for Y

 Height control point: (Z known)

One equation for Z

Seven Equations can be derived from

Two full control points and one height control point Three planimetric control points and one height control point Must have at least one height or full control point

 A two step orientation procedure is required for analogue instruments, because the exterior orientation parameters cannot be directly solved, as can be done with a computer. In analogue instruments, the second diapositive is adjusted through five degrees of freedom, until y parallax is completely removed. This is repeated iteratively until y parallax is zero at the five selected points. The absolute orientation is then calculated from control points. Because of this historical reason, softcopy photogrammetry systems may still follow the procedure of computing a relative orientation, followed by an absolute orientation.
This can be useful if a 3D model is required, but exact world coordinates are not.

Example - Exterior Orientation

Find the exterior orientation of a photograph, given that
the interior orientation is f = 150mm, x0 = y0 = 0 the photography can be considered normal The following control points are available
X(m) 100 500 500 100 Y(m) 100 110 600 550 Z(m) 10 50 60 20 x(mm) y(mm) -46.88 -58.59 50.00 -60.00 50.85 63.56 -47.62 47.62

 The collinearity equations are:

x = x0 f y = y0 f r11(X X0) + r 21( Y Y0) + r 31(Z Z0) r13(X X0) + r 23( Y Y0) + r 33(Z Z0) r 12(X X0) + r 22( Y Y0) + r 32( Z Z0) r13( X X0) + r 23(Y Y0) + r 33(Z Z0)

Since we assume normal photography, there is no rotation, and so R becomes:

1 R = 0 0 0 1 0 0 0 1

The collinearity equations reduce to the simple forms:

xi = f

( Xi X0) (Zi Z0)

and

yi = f

( Yi Y0) ( Zi Z0)

Where the subscript i refers to each individual control point. In this case, without rotations, there are only three unknowns, so only two of the control points will be required. Can solve algebraically:
xi = f ( Xi X0) (Zi Z0) f (X i X0) xi

( Zi Z0) = Z0 = Zi +

f ( Xi X0) xi

 Solving for X0
Z1 + X0 = f (X1 X0) f (X 2 X0) = Z2 + x1 x2 x1 x2 (Z1 Z2 ) x2X1 x1X 2 + f ( x2 x1 ) x2 x1

Substituting control point information

X0 = - 46.88 50.00 (10 50) 50.00 100 -46.88 500 + 150 (50.00 -46.88 ) 50.00 -46.88 = 300m

Similarly for Y0
Y0 = y1 y2 ( Z1 Z 2 ) y2 Y1 y1Y2 + = 350 m f ( y2 y1 ) y2 y1

 Z0 can then be found from If we use i=1,

Z0 = 10 + 150 (100 300 ) = 650m - 46.88

Z0 = Zi +

f (X i X0) xi

The other control points can be used as a check. E.g. for i=3,
Z0 = 60 + 150 (500 300) = 650m 50.85

Getting a direct algebraic solution as shown becomes impossible when all the rotations are included as unknowns. This is due to the non-linearity of the resulting system of equations. Iterative numerical methods must then be used.

Summary
Review of Matrix Algebra
Identity, Inverse, Multiplication, addition

Transformations between coordinate systems

Translation, scaling, rotation

Collinearity Equations Using control points to find the inner and outer orientation
Simple Example Numerical methods must be used