1

A tutorial on the total least squares method for

fitting a straight line and a plane
Leonardo Romero Muoz, Moiss Garca Villanueva and Cuauhtmoc Gmez Surez,
Facultad de Ingeniera Elctrica, UMSNH, Mexico
AbstractThe classic least squares regression fits a line to data
where errors may occur only in the dependent variable, while the
independent variable is assumed to have no errors. The total least
squares regression fits a line where errors may occur in both
variables. This tutorial provides an introduction to the method of
total least squares supplementing a first course in statistics or
linear algebra.

I. INTRODUCTION
Detecting geometric features (lines, circles, surfaces, etc.)
from data points is a fundamental task in several fields of
science and engineering; for instance, metrology, computer
vision, mobile robotics, etc.
Let = { , , } be a set of measurements or points
where each point = , is represented by its rectangular
coordinates. A linear relation between
and
is usually
written as
(1)
=
+
where is the slope of the straight line and is the y-axis
intersection. In the classic Least Squares (LS) the abscissa
data ( , = 1, , ) are assumed to be known exactly while
the uncertainties of the ordinate data ( ) are used as weights
for fitting the line , , given by (1), to the set of
measurements .
The solution to fit a line using the least squares regression,
appears with complete derivations in textbooks at many levels:
calculus, linear algebra, numerical analysis, probability,
statistics, and others.
However, measured data are never free of uncertainty. This
means, in order to determine a best fit to a line, a method is
and
data
required which takes the uncertainties of the
into account [3]. The Total Least Squares regression (TLS)
was introduced by Golub and Van Loan [2] to deal with both
uncertainties. Despite its usefulness and its simplicity, TLS
has not yet appeared in numerical analysis, statistics or linear
algebra texts.
Introducing students to TLS is the purpose of this tutorial,
and it may complement the usual courses in numerical
analysis, statistics or linear algebra, or serve as a transition
from such courses to a more advanced and specialized course.
L. Romero Muoz, Facultad de Ingeniera Elctrica, Ciudad Universitaria,
Universidad
Michoacana,
58000,
Morelia,
Mxico
(e-mail:
[email protected]).
M. Garca Villanueva. Facultad de Ingeniera Elctrica, Ciudad
Universitaria, Universidad Michoacana, 58000, Morelia, Mxico (e-mail:
[email protected]).
C. Gmez Surez. Facultad de Ingeniera Elctrica, Ciudad Universitaria,
Universidad
Michoacana,
58000,
Morelia,
Mxico
(email:
[email protected])

Additional references to TLS are the introductory paper by

Yves Nievergelt [5]; an overview of the TLS methods, by Ivan
Markovsky and Sabine Van Huffel [4]; or the book by Sabine
Van Huffel and Joos Vandewalle, about the TLS problem [7].
II. PRELIMINARIES
A. Normal form of a line
There is a disadvantage of using equation (1) to represent a
line: vertical lines can not be represented, because . To
avoid this problem, a line in the plane is represented by its
normal form,
(2)
= ,
where and are the length and the angle of inclination of
the normal, respectively. As shown in Figure 1, the normal is
the shortest segment between the line and the origin of a given
coordinate frame. Using this form, points = , that are
on the line = , satisfy
= x cos

+ sin

(3)

Fig. 1. Line parameters in the normal form. The shortest distance from the
.
origin to the line is =

The relation between the normal form , to the

representation , given by eq. (1) can be obtained dividing
eq. (3) by sin (while 0, avoiding vertical lines) and
reordering terms:
cos
+
=
sin
sin
1
(4)
=
+
tan
sin
From this last equation, the parameters of the line , are
given by:
=
,
=
(5)

A condition for a minimum is that the partial derivatives of

with respect to the parameters of the line ( and ) vanish:

B. Orthogonal distance from a point to a line

The shortest distance from a given point = , to a
( , ), is easily computed as
line = , , denoted by
follows.
A line through the point , parallel to line , is given
by:
= cos + sin

=0.

cos

sin

( cos

The separation between this new line

with parameters
, and the line with parameters , is the difference
= , because both lines have the same (see Figure
2). So the desired distance, called orthogonal distance, is
( , ) =

= 0 first

Lets do

( cos

of

=0

)=0

( )

( )=0

( )
1

( ) + sin

) )

sin

( ) + sin

cos
cos

sin

( ) + sin

cos

(6)

=0

( ) =0

(10)

Expressions in square brackets are the well known means

and , defined as follows
=

(11)

Equation (10) reduces to

cos
Fig. 2. Orthogonal distance

III.

from point

to line .

THE TOTAL LEAST SQUARES REGRESSION

A. The problem definition

In the literature, the problem of fitting a straight line to
data with errors in both coordinates was first formulated by
Pearson as early as in 1901 [3]. Deming in 1943 [1] suggests
to minimize the sum
( )
( )
(, , , ) =
+
(7)
,

where ( , ) are the points coordinates with corresponding

uncertainties ( , , , ) and ( , ) denote its corresponding
point of the straight line . The best line minimizes . In the
case , = , = , = 1, , , the problem is reduced to
the so-called total least-squares problem and minimizing (7) is
equivalent to minimizing the orthogonal distance of the
measured points ( , ) to the fitting line. Therefore, this is
also often referred to as orthogonal regression [3]. In this case,
the best line minimizes
( , )
(; ) =
(8)
B. Finding the best line
Replacing eq. (6) into (8), the best line minimizes
1
(; ) =
( cos + sin )

+ sin
=0
= cos + sin

(12)

Comparing equations (3) and (12), we get an important

result:
The centroid of points given by , is a point of the line
with parameters , which minimizes eq. (9).
Replacing eq. (12) into (9), (; ) can be expressed as
1
(; ) =
(13)
[( ) cos + ( ) sin ]
In eq. (13) the only unknown parameter is
= 0, in order to find the right .
1
( [( ) cos + (
1

) cos

2 [(

) sin

[(
(
2(

) 2 cos
)(

sin

+(

) sin

(9)

2 cos
2(

sin

)(

) sin ] )

+(
+(

+ 2(

. So, lets do

=0

) sin ]
) cos ] = 0

)(

) cos

) 2 cos

sin

[(

=0

) ( ) ] +

)(cos

sin

)=0

(14)
(
Using the follo
owing trigonom
metric identitiees
sin 2 = 2 co
os

sin

, co
os 2 = cos

D. M
Matrix form to obtain the anggle
sin

(15)
(

Equation (14) reduces to

sin 2 [((
2(

)(
[((

sin 2
2 cos 2

) (

Equattion 13 can be rewritten in thhe matrix form

1
(; ) =

(17)

) ] +

) cos
c 2 =0

) (

) ] +

[(

)=0

)(

) )]
sin 2
2 [( )(
2
=
[( ) ( ) ]
cos 2
1
2 [( )( )]
= arctan
[(
2
[ ) ( ) ]

tan 2 =

(16)
(

Equations (16) and (12) get

g the desired
d line parametters
, . In practicce, equation (16) uses the four
f
quadrant arc
tanngent (atan2).
c
arctaan( ) but uses
u
2( , ) computes
thee signs of both
h and to determine
d
the quadrant
q
in wh
hich
thee resulting angle
a
lies. Fo
or example
2(2, 2)) =
1
135, whereaas
2(2,2)) = 45, a distinction
d
wh
hich
woould be lost with
w
a single-aargument arc tangent functiion.
Annother practicaal consideratio
on must be don
ne when eq. (12)
(
givves an < 0. In that case, the
t line , , where =
annd
= + , represents th
he same line , , but in this
t
reppresentation
0.

Fig. 3. Linne fitting minimiziing orthogonal disstances from pointss to line (TLS).

C. Example
Consider the data given in Table I. We want
w
to determ
mine
thee line of total least squares fo
or these points.
TAB
BLE I
AN EXAMPLE WITH
W
7 POINTS.

pooint
3
7

4
7

5
11

6
11

7
15
5

8
16

9
19
1

To calculate the line param

meters, we usse equations (1
11),
(166) and (12),
=7
3 + 4 + + 8 + 9
=
=6
7
7 + 7 + + 16
1 + 19
=
= 12..286
7
1
1
116
= arctan
= 0.43
361
2
97.4
429
= 0.24889
ws the line ass well as the points and th
heir
Figure 3 show
ortthogonal distances. For refeerence, Figure 4 shows the line
l
obbtained using th
he classical Leeast Square Meethod. In the saame
Figgure the vertical distances fro
om points to th
he line are show
wn.

Fig. 4. Linne fitting minimiziing vertical distances from points to line (LS).

where

2,

is a matrix of dimension

is a vecctor,

cos
sin

and denotes thhe Euclidean norm of vecctor

coordinaates [
] , defined ass
=

+ +

(18)

with
(19)

Now the goal is to find a vector which minimizes eq.

(17). In other words, a vector which minimizes the norm of
the linear map:
. Note that is a unit vector, because
= cos
+ sin
= 1.
To achieve this goal, the Euclidean norm also can be
expressed using de inner product of the vector and itself,
(20)

where
denotes the transpose of vector . Using the inner
product to compute the norm, eq. (17) can be written as
(

(;Z)=

)(

Let see the form of the matrix

(

)(

(23)

,
,

=
=

,
,

0
,
,
and second
Let
be a matrix which first column is
column is ; and let be a diagonal matrix with elements
and . Using these matrices we can write last equation in a
simpler form
(26)
=
The orthonormal matrix has an interesting property: its
inverse is its transpose (
=
= , where is the
identity matrix). Using this property and equation (26), the
matrix can be expressed in terms of and ,
(27)
=
Replacing the matrix , given by the equation (27), into
equation (22),
,

{ (

)
0
0

) +

) }

(29)

where is the angle between both vectors. Given that

and
and are the coordinates of unit vectors and
orthogonal vectors, we have
= cos
= cos( 2) = sin
where is the angle between vector
results into equation (28),
1
1

,
are
(30)
(31)

and . Replacing these

cos

sin

cos

+ (1 cos

)}

(32)

Let = cos , where is a value in the range [0,1], and

( = 0). Using
= 1 when vector is identical to vector
this new variable in Equation (32), we have finally
1
(; ) =
{
+ (1 )}
1
(; ) =
{( ) + }
(33)
The expression ( ) +
corresponding to a line
, with a negative slope =
(because
< )
and y-axis intersection
= . The maximum value of
(, ) is
when = 0, and the minimum value is
when = 1. Therefore,

,
,

= cos

(; ) =

(28)
To see the maximum and minimum value of , suppose
< . Taking into account that the inner product of
that
two vectors with coordinates and is defined as

)(

(; ) =

(; ) =

Because matrix has real elements, is symmetric ( =

0 for 0), matrix
) and is positive semidefinite (
0,
0; and two
has two real eigenvalues:
orthonormal eigenvectors (unit vectors with inner product
equal to zero).
Let
=[ ,
and
=[ ,
be the
, ]
, ]
coordinates of eigenvectors
and
. Eigenvalues and
eigenvectors are related by
=
=[ ,
(24)
, ]
=
=[ ,
(25)
, ]
Equations (24) and (25) can be joined into a single relation
,

(; ) =

, of dimension 2 2,

(; ) =

(21)

From this expression, using the properties of the transpose

and the associative law for the matrix product, we get a
quadratic form
1
(; ) =
(
)( )
1
(; ) =
(
)
1
(; ) =
(22)

(; ) =

Vector = , the eigenvector associated to the

of matrix
, minimizes
minimum eigenvalue
(, ).
From
= [u , , u , ] and
desired angle ,
= arctan(

= [cos
,

sin ] we get the

(34)

Using the four quadrant arc tangent (atan2) to compute

(eq. (34)), we can get the parameter of the line using eq.
(12),
= , +
(35)
,
Using equation (3) and previous results, we can see an
,
interesting property of vector

= cos + sin
+
, =
, +
0 = ( ) , + ( )
=( )

ending points of the desired line segment, corresponding to the

highest and the lowest point. The final steps are to undo the
rotation and translation made,

,
,

(36)

where = [
] , and = [ ] is the centroid of points.
must be orthogonal
Note that vector with coordinates
to vector , in order to satisfy equation (36).
E. Example (cont.)
Continuing the example from Section III-C, we can
compute the matrix ,
28.000 58.000
(37)
=
58.000 125.429

= ( )
= +

,
,

= ( )
= +

(44)
(45)

The line segment is the line between points and .

As an example, Figure 5 shows five line segments
computed from a set of points given by measurements of an
Infrared sensor of a small mobile robot, using the Line
Tracking algorithm [6] (with TLS). The robot rotates 360
taking measurements.

and its eigenvalues and eigenvectors (using the function svd of

the octave program):
= 0.97076,
= 152.45781
(38)
= [0.90641 0.42241]
= [0.90641 0.42241]
(39)
From eigenvector

, we can compute using equation (34),

0.42241
(40)
= 2.7055
= arctan
0.90641

Using eq. (12) with this value for , we get a negative

value for . In this case
= + , and we get the same
result for as in section III-C.
IV.

Fig. 5. Finding the best lines of a set of points given by an IR sensor of a small
mobile robot.

THE LINE SEGMENT

Sometimes it is useful to know the line segment associated

to the set of points , instead of only the infinite line
expressed by the parameters , . This section addresses this
problem.
First we move the origin of coordinates to the centroid of
points
= [ ] . For each point
=[
] , the
translated point is defined by
=

(41)

V. SOME EXTENSIONS
A. Weighted total least squares
In section III-A we consider the same uncertainty for all
for point
=
points. If we consider an uncertainty
1, , , the best line minimizes
( , )
(; ) =
(46)
Following a procedure similar to section III-B, we can get
the solution

Then we rotate points using a rotation matrix ( )

( ) = cos
sin

sin
cos

(42)

The matrix ( ) rotates points in the xy-Cartesian plane

counter-clockwise through an angle about the origin of the
Cartesian coordinate system. To perform the rotation using a
rotation matrix , the position of each point must be
represented by a column vector , containing the coordinates
of the point. A rotated vector is obtained by using the matrix
multiplication .
If we rotate all points an angle , the rotated points
follows a vertical line,
= ( )

= cos
1
=
2

(43)

Let
and
be the maximum and minimum values
respectively of all coordinates
, in
=[
], =
1, , . The points
= [0
] and
= [0
] are the

+ sin
2

[(

( )( )
) ( ) ]

(47)
(48)

where
=

(49)

(50)

= 1
are the weighted means; with individual weights
for each measurement. This approach is known as weighted
least squares.
B. Fitting a set of points to a plane
The method to find the best line in the total least squares

sense, can be extended easily to find the best plane of a set of

points in three dimensions.
A plane
is represented by four parameters =
, , , , where , , and are the length of the normal,
and the angle between the normal and the x-axis, y-axis and zaxis respectively. The normal is the shortest line segment
between the plane and the origin.
A point = , , that is on the plane satisfy
=

cos

+ cos

+ cos

The orthogonal distance from a point

(51)

to plane

is given

cos

(52)

by
( , )=

cos

cos

(53)

= 0, as before, we get a similar result: the

centroid of points given by , , is a point of the plane
which minimizes eq. (53),
+ cos

This article presents a tutorial about the method of Total Least

Squares to fit a line, developing a closed formula for
calculating the line parameters; and also the matrix
formulation, which can be easily extended to the case of fitting
a Cartesian plane to points in three dimensions.
It is written in a simple form which should be easy to
understand, even for students with a basic knowledge of
calculus and linear algebra. Also, the closed formulation is
useful for low computational resources, such as
microcontrollers for small mobile robots.
VII. REFERENCES
[2]

( , )

Doing

= cos

VI. CONCLUSION

[1]

In this case, the best line minimize

( ; )=

points.

+ cos

[3]
[4]
[5]
[6]

(54)

where
[7]

G. C. Deming. Data Reduction and Error Analysis for the Physical

Sciences. New York: Wile, 1943.
Gene H Golub. Some modified matrix eigenvalue problems. Siam
Review, 15(2):318-334, 1973.
Michael Krystek and Mathias Anton. A weighted total least-squares
algorithm for fitting a straight line. Measurement Science and
Technology, 18(11)_3438, 2007.
Ivan Markovsky and Sabine Van Huffel. Overview of total least-squares
methods. Signal processing, 87(10):2283-2302, 2007.
Yves Nievergelt. Tottal least squares: State-of-the-art regression in
numerical analysis. SIAM Rev., 36(2):258-264, June 1994.
L. Romero Muoz, M. Garcia Villanueva, and C.A. Lara Alvarez. An
extended line tracking algorithm. In 2013 IEEE International Autumn
Meeting on Power; Electronics and Computing (ROPEC), pages 1-5,
Now 2013.
Sabine Van Huffel and Joos Vandewalle. The total least squares
problem: computational aspects and analysis, volumen 9. Siam, 1991.

(55)
VIII. BIOGRAPHIES

To find the angles , and we use a similar procedure to

the matrix formulation. Replacing equations (52) and (54) into
(53),
1
(56)
( ; )=

where

3,

is a matrix of dimension

is a vector,
cos
cos
=
cos

(57)

Note
that
is
a
unit
vector,
because
|| ||= cos + cos + cos = 1.
The best plane is given by =
=[ ,
,
, ] , the
of matrix
eigenvector associated to the smallest eigenvalue
. From =
we can obtain , using equation (54),
=

(58)

Therefore the plane can be expressed by

=
where

=[

+
=(
,

] , and

=[

+
)

(59)
(60)

] is the centroid of

Leonardo Romero Munoz was born in Querndaro,

Michoacan, Mexico. He studied electrical engineering in the
Universidad Michoacana. He got a master's and Ph.D. in
Computer Science at ITESM Campus Morelos, Mexico, in
1990 and 2002 respectively. He is currently a Professor and
Researcher at the Faculty of Electrical Engineering of the
Universidad Michoacana. His research interests are
computer vision, robotics and probabilistic reasoning.
Moises Garcia Villanueva was born in Patzcuaro,
Michoacan, Mexico. He received the degree of Electrical
Engineer and Master in Electrical Engineering with option
in Computational Systems at the Faculty of Electrical
Engineering of the Universidad Michoacana in 1999 and
2001 respectively. He is currently a Professor and
Researcher in the same Faculty. His areas of interest
include pattern recognition, computer vision, robotics and data mining.
Cuauhtemoc Gomez Suarez received the degree of
Electronic Engineering and Master in Electrical
Engineering, with option in Computational Systems in the
Universidad Michoacana, Mexico. He currently serves as
a teacher of some courses at the Faculty of Electrical
Engineering of the same university. His areas of interest
are robotics and three-dimensional reconstruction.