Inverse Problems in Ordinary Differential Equations

Thesis submitted in partial fulfillment of the requirements

for the award of the degree of
Masters of Science
in
Mathematics and Computing

Submitted by
Amninder Singh
Roll No. 301603003

Under the guidance of

Dr. Kavita

June 2018
School of Mathematics
Thapar Institute of Engineering and Technology
Patiala 147004
Punjab, India
.
.
 ACKNOWLEDGEMENTS

The thesis entitled “Inverse Problems in Ordinary Differential Equations” contains my re-
search work carried out at the School of Mathematics at Thapar Institute of Engineering and
Technology in Patiala. First of all, I would like to thank the almighty for granting persever-
ance. Every project is successful largely because the effort of wonderful people who have
always given their valuable advice or lent a helping hand. I would like to express my gratitute
to my supervisor Dr. Kavita, Assistant Professor, School of Mathematics, Thapar Institute of
Engineering and Technology, Patiala, for her guidance, motivation, immense knowledge and
engagement through the learning process of this master thesis.

Besides my supervisor, I am thankful to Dr. Satish Kumar Sharma, Head, Associate Professor,
School of Mathematics, Thapar Institute of Engineering and Technology, Patiala, for providing
all the necessary facilities in the department. His support and sincere attitude towards depart-
ment have always been encouraging, motivating and helped me pave my way for the disserta-
tion. I would also like to thank Dr. A. K. Lal for his consistent support.

I owe my gratitude to my family, who have supported me spiritually throughout entire process,
both by keeping me harmonious and helping me putting pieces together. Without their support
and endless blessings, it is impossible for me to complete my education seamlessly. I will be
grateful forever for their love.

Last but not the least, I am heartfelt to my friends for the stimulating discussions, for the times
we work together, and for all the moments we have fun together. I greatly value their friendship
and deeply appreciate their belief in me. Thank you.

(Amninder Singh)

v
.
Contents

CERTIFICATE iii

ACKNOWLEDGEMENTS v

ABSTRACT ix

1 Inverse Problem 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 What is an inverse problem? . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 How inverse problems are used for differential equations? . . . . . . . . . . . . 7
1.4 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2 Regularization Techniques for Solving the Inverse Problem 13

2.1 Truncated Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . 14
2.2 Tikhonov Regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Regularized Least Square Fit Method . . . . . . . . . . . . . . . . . . . . . . 20

3 Inverse Problem in Ordinary Differential Equations 21

3.1 An Example Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Two Approaches for the Solution of Inverse Problems . . . . . . . . . . . . . 22
3.2.1 Data Fitting Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.2 Non-linear Equation Method . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 MATLAB Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3.1 Matlab coding for Data Fitting Method . . . . . . . . . . . . . . . . . 30
3.3.2 Matlab coding for Non-linear Equation Method . . . . . . . . . . . . . 31

vii
Conclusion 33
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

viii
 ABSTRACT

This thesis report includes solving the inverse problem in ordinary differential equation which is
helpful in various fields. For example, we have considered a second order ordinary differential
equation. In this second order ordinary differential equation, there are two types of equation.
The first one is linear equation and other one is non-linear equation. In beginning, we solve the
second order ordinary differential equation, then we implement the Least Square Fit Approach
on linear differential equation and Newton method on non-linear differential equation to find
the inverse problem. From the results, we have conclude that these methods are efficient for
solving inverse problems associated with differential equations.
Chapter 1 of this report describes about the inverse problem and it’s various use in many appli-
cations in differential equation. Chapter 2 describes the regularization techniques for solving
the inverse problems. Chapter 3 describes the various methods for solving the inverse problem
in ordinary differential equation. In the end, we presented the conclusion.

ix
x
Chapter 1

Inverse Problem

1.1 Introduction
We will discuss about the inverse problems and it’s various important applications in this in-
troductory section. In the previous years, the quickly developing area in applied mathematics
is the inverse problem. This growth has large applications in the field of sciences and industry.
So, we can say that inverse problem is a dynamic and widening branch of Mathematics. When
we are using the term ”inverse problem”, it is generally ask that ”inverse to what”, then we can
say that two problems are opposite to each other if the origination of one problem involves the
other problem. The one problem is the forward problem and another problem is the inverse
problem. However, if we will research that there is a physical world problem at the back of
the mathematical problem. There is a dissimilarity between direct(forward) problem and the
inverse problem. We can say that direct problem is that problem in which we know about the
future actions of physical system from the knowledge of existing state and real parameters of
that system. The inverse problem is the observation of existing state of the system from the
future knowledge or parameter identification.
Inverse problems are an analysis area which deals with inversion of the system. There are many
examples in our daily life in which inverse problems are widely used like in medical comput-
erized tomography (CT) scan, geophysics, astronomy, Image Formation, Elasticity Imaging,
Bio-mechanical imaging, Physics, Economics, Groundwater flow modeling, Navigation and
many other fields. So, in all these fields our aim is to determine some hidden factors from a set
of predictions that produced them. The various applications are given below:-

1
A. Medical Computerized Tomography (CT) Scan
It is the powerful method for analysis of human organs and biological system. The inverse
problems helps us to determine image structure from inside the body through the measurement
of X-rays. The X-rays are very high speed radiations that have passed through the body and
creates the photographic film.

B. Geophysics
In area of geophysics, the inverse problems have a crucial participation. We cannot determine
the interior structure of the earth but it is possible with the help of the inverse problem. In
inverse problem, there is transmission of seismic waves to find the position of an earthquake’s
point. Also, inverse problems helps to find the degree of compactness of earth from measure-
ments of the earth’s gravity field. So, this type of condition in which waves are transmit to
examine the material are possible through inverse problem.

C. Image Formation
To calculate about the making of an image, there is mapping in the direct problem of an object
from it’s image to it’s quantities. We know about the details of the direct problem. Thus, the
inverse problem is to calculating the real picture from the given input and the correct informa-
tion of the direct problem. In the case of explanation of deblurring pictures, we have an image
which is a fine picture, the given input is the blurred image, and direct problem is the whole
blurring process. The ill-posed problem is to determine the fine image from the blurred picture
and to provide the information of the complete blurring process.

D. Elasticity Imaging
It is a modern and optimistic method in medical imaging. It tells us about the distinction of
tissues of elastic modules. We know the characteristics of an object and it’s boundary input and
determine displacement field in the forward problem. In inverse problem, the whole condition
is opposite, we know about the displacement field and determine the boundary knowledge and
the material properties of the elastic modules.

2
E. Bio-mechanical imaging
It is feasible to see the image of physical characteristics of tissues in biomechanical imaging. It
includes the characteristics like viscoelastic, Young’s modulus, non-linear elastic properties. At
the same time, it has been set up that the physical characterstics of tissue which are changed if
the tissues are defected.
The maximum techniques in biomechanical imaging consists observing the distort tissue by
using a imaging treatment like ultrasound or Magnet Resonance Imaging and then using the
pre-deformation and post-deformation pictures to observe the internal problem of the tissue.
Then, this input is used to observe the spatial distribution of the tissue along with an appropri-
ate mechanical model.

F. Physics
There are lot of ill-posed problems which are appear in the branch of physics. These are given
below:
(a) Calculate the shape of a mountain from the travel time:- Let us consider that we have
a particle and we slide this particle in upward direction with starting energy on a frictionless
mountain and estimate that how much time for a particle is neccessary to return it’s starting
position. If we alters the initial energy and calculate the time need to return back, then we can
observe the shape of the mountain.
(b) Calculation of a potential from the period of oscillation:- Let we suppose that the motion
of a particle. Initially, we know the time of oscillations of the particle with how much energy is
required, then calculate the potential of the particle.

G. Economics
The demand theory of economics says about the forward problem. There is specified that in an
exchange economy consists of N customers, consists the mutual demand function, which is the
total of individual demand functions. So, there is a mapping of the cost system S to a item’s
quantity x(S). There is given a mapping in the inverse problem : S → x(S), it is commonly to
question that whether it is the total demand function of the market economy.

H. Groundwater flow modeling

We want to calculates the object boundary of an aquifer from calculations of pressure of a fluid

3
that dips the aquifer in groundwater flow modeling. But unluckily, very small amount of noise
in the data can creates the large amount of mistakes in the estimation. This uncertainity event
is termed as ill-posedness. So, our aim is to calculates the parameters of an aquifer and reduce
the errors in the estimation.

I. Navigation
When a sailor travels in a boat or submarine, then he finds the actual position in near to the
actual time. It is possible by taking a lot of observations. For example:- by using the position of
satellite, and also by taking the last location and using the knowledge such as keeping the data
of speed. When all the separate pieces of data collect with each other, then it gives the detail of
the movement of vessel.
The area of ill-posed theory is very vast. There are large number of applications of inverse
problem which are found in many diverse areas. The mathematical approaches called as regu-
larization that have been enlarged to handle with the inverse problems.

1.2 What is an inverse problem?

In the inverse problem, we have only knowledge of the effects and we are finding the causes,
which consists the idea of solving the inverse problems. However, if there is a minor changes in
the results, then there is a huge distiction in the causes. As a result, sometimes it is not possible
to calculate the real cause uniquely by observing only results. So, inverse problems starts with
the results and then determine the causes. It is the inverse of forward problem. We want to find
an effect from a cause in direct problem but in an inverse problem the effect is known and we
want to recover the causes.
Inverse problems are of two types:
(a) Well-posed problem
(b) Ill-posed problem

Well-posed problem:- According to Hadamard,” a problem is called well-posed if it fulfills

the subsequent given requirements:-

4
(a) the problem must have a solution (Existence),
(b) there is at most one solution to the problem (Uniqueness), and
(c) it must depend continuously on data and parameters (Stability).
If the problem violates any of these requirements, then it is called ill-posed problem.
In mathematical way, we can say that let X is a map defined from set A to set B i.e., X : A → B,
where A and B are Banach spaces or normed spaces, and X(s) = p where X is a non-linear
operator (generally). It becomes well-posed if X is a homeomorphism of A onto B. Also, we
can say that the solution exists for any p ∈ B is unique, and depends on p continuously, so that
X −1 is the continuous mapping. If some of these conditions do not happened, then the problem
becomes ill-posed.
Now, the question arises that why inverse problems are difficult than forward problem?
Actually, inverse problems are more harder than the forward problem because they tell us about
the parameters that we cannot directly observe. The one reason is that presence of various mod-
els that suitably holds the data. The necessary things must be include the solution occurrence,
solution uniqueness, and instability of the solution process.
1. Existence:- Actually, there may not be any model which fits absolutely in the data. This can
occur in practice due to input data contains so much errors.
2. Uniqueness:- If there exists proper solutions, then that solutions may not be unique, even for
an large amount of data points (i.e., there exists infinite answers).
3. Instability:- The procedure of evaluating an ill-posed solution is not stable because little
conversion in calculation can occurs the large change in the approximated model. In the inverse
problems, where this type of situation arises are termed to as ill-posed in the case of continu-
ous systems, or ill-conditioned in the case of discrete linear systems. It is generally possible to
stabilize the inversion process by adding extra constraints that bias the solution, a process i.e.,
generally termed as regularization.

An example
Let us take Fredholm first kind integral equation of convolution type in one dimension space:
Z 1
y(t) = p(t − t 0 )x(t 0 )dt 0 = (ρx)(t); 0 < t < 1, (1.2.1)
0
This equation is a one-dimensional kind of a representation which occurs in two-dimensional
optical imaging. Here, x, y, p denotes following things:-
x = light source intensity,

5
y = pixel intensity,
p = kernel which shows the blurring effects which occurs during picture formation.
It’s one-dimensional model is  2
−t
p(t) = cexp , (1.2.2)
2λ2
√
where c = 1/λ 2π and λ = 0.05 are positive values.
The forward problem is related with equation (1.2.1), given the source x and the kernel p which
determines the blurred picture y corresponding to a piecewise smooth source as shown in fol-
lowing Figure 1. Therefore, p is a smooth function, the estimation of y = λx by applying
standard numerical quadrature is straightforward.

Figure 1:One dimensional image data

In Figure 1, the source function x is represented by solid line, the blurred image y = λx is
represented by the dashed line, and the discrete noisy data d is represented by the circles. To
calculate the approximate integrals we are using the midpoint quadrature.

An inverse problem is associated with it as given below:

We know the kernel p and blurred image y, find the source x. In beginning, the exact solution
for this ill-posed problem looks straightforward. We can easily discretize the above equation
(1.2.1) to get a discrete linear system Px = d. If we are applying the midpoint quadrature, then

6
we get the values of P :

((i − j)k)2
 
[P]i j = kcexp ; 1 ≤ i ; j ≤ n, (1.2.3)
2λ2

where k = 1/n. If the matrix P is non-singular, so we can evaluate the discrete approximation
p−1 d to x. To find the exact quadrature value, n should have big value. But unluckily, matrix
P forms enlargely ill-conditioned as n becomes big, so noise in d may be largely amplified.
Certain noise because of quadrature, can be controlled but noise in the picture making device
cannot be controlled. As a result, straightforward solution approach looks not to success.
Therefore, we use the regularization techniques to control the errors which occurs due to noise.

1.3 How inverse problems are used for differential equations?

The two mathematical problems are differentiation and integration which are inverse to each
other. It is not clear that which one is direct problem and which one of these problems is an
inverse problem. However, the differential equation has the property of ill-posedness. In most
of the calculus courses, differentiation might be viewed as ”inverse problem”. The various oper-
ations like physical and social gives a new idea for the modeling of the inverse problems. There
are too many differential and integrable equations which are used in the formation of inverse
problems. If we know about the given essential data in these type of models, then we can solve
the inverse problem and get the approximate solution. Also, the behaviour of the model can be
calculated under many conditions. The important input data consists of knowledge of domain’s
shape and it’s size, initial input data and boundary material in the form of differential equations.
If these type of data which explains the working of the system satisfies, then we can use the
mathematical model possibly for learning the real model.
The differential equations are used for explaining the various processes whether they are natural
or the social. If we know the input data of the differential equations, then we can easily solve
that equations and also this type of differential equations fulfill the boundary conditions or ini-
tial input data. Also, there occurs so many difficulties in the field of science and engineering
very oftenly. The less coefficient of input in the equation are not known, we have to find the
conditions, if we know about some output values of the differential equation. So, these type of

7
difficulties are the inverse problems for the differential equation.

Inverse problem in linear order Ordinary Differential Equations.:- Suppose we take an

example which describes that the inverse problems are used for linear differential equation of
the quartic form:-
00
xiv (t) + p2 (t)x (t) + p0 (t)x(t) = 0. (1.3.1)
The above coefficients of the equation p3 (t) = p1 (t) = 0, f (t) = 0 and p2 (x), p0 (x) are not
known. Suppose x1 (t) = sint, x2 (t) = cost be the two outputs of the equation (1.3.1). Now
put x1 (t), x2 (t) in equation (1.3.1) which provides only one equation for the not known coeffi-
cients p2 (t)p0 (t) = 1. Therefore, coming output value is not single of this type of ill-posedness
problem. If there is stated another answers x1 (t) = et , x2 (t) = e2t of the equation (1.3.1),
then putting the values provide two equations for not-known coefficients p2 (t) + p0 (t) = 1
and 4p2 (t) + p0 (t) = 16 . This type of model has a unique solution of equations p2 (t) = 5,
p0 (t) = 4. This example shows that the necessary of analysis of knowledge in which we are
using the inverse problem for solving it.

Inverse Problems for Differential Equations of Elastodynamics:- Let us consider that the
differential equations of elastodynamics an let x = (x1 , x2 , x3 ) ∈ R3 be a three dimensional space
variable and let t ∈ R be a one-dimensional time variable and s(x,t) = (s1 (x,t), s2 (x,t), s3 (x,t))
is the displacement vector function of non-homogeneous anisotropic elastic material represent-
ing by density ρ and the elasticmoduli Ei jkl . The density ρ and the elasticmoduli Ei jkl are the
varying functions of position x = (x1 , x2 , x3 ). By adding the properties of strain-energy function
with Hooke law we search that Ei jkl satisfy the following property and strong convexity.

3 3 3 3
Ei jkl = Ekli j = Elki j = Ei jlk , ∑ ∑ ∑ ∑ Ei jkl εi j εkl > 0. (1.3.2)
i=1 j=1 k=1 l=1

The 3 × 3 real symmetric nonzero matrix i.e., (εi j )3×3 . The equations in the non-homogeneous
anisotropic elastic materials for the motion are given below:-

3 3 3
∂2 si ∂ ∂sk
ρ 2 = ∑ ∑ ∑ ∂x j (Ei jkl ∂xl ) + fi ; i = 1, 2, 3. (1.3.3)
∂t j=1 k=1 l=1

where fi are the elements of body forces i.e., f (x,t) = ( f1 (x,t) f2 (x,t), f3 (x,t)) acting per unit

8
volume on the particle exactly at location x at some time t. Now, we determine the approximate
solution of equation (1.3.3) for f = 0 :-

s(x,t) = S(t − T (x))B(x). (1.3.4)

Near a wavefront t = T (x), we suppose that the components of S = (S1 , S2 , S3 ) are changing
most quickly than B(x) or Ei jkl . Therefore, sucessive derivatives ∂S/∂t and ∂2 S/∂t 2 are also
changing still more fastly. Now, we substitute the equation (1.3.4) into (1.3.3), we get:
3 3
∂2 S k
 
∂T ∂T
ρδik − ∑ ∑ ∂x j ∂xl B = Di (SB), (1.3.5)
j=1 l=1 ∂t 2

where Di (SB) includes gradients, S is the elastic moduli, first-order derivatives of S and ampli-
tude function B(x).
Therefore, L.H.S. of (1.3.5) should be much smaller than ∂2 S/∂t 2 .We find that the matrix coef-
ficients of ∂2 S(t − T (x))/∂t 2 B(x) must be singular:
 3 3 
∂T ∂T
det ρδik − ∑ ∑ ∂x j ∂xl = 0. (1.3.6)
j=1 l=1

The above equation finds the certain wavefronts in an elastic medium, because this provides a
constraint on the function T (x). Also, in the non-homogeneous isotropic medium,

Ei jkl = νδi j δkl + µ(δik δ jl + δil δ jk ), (1.3.7)

where, 
0; i f i 6= j
δi j =
1; i f i = j

Mostly, ν = ν(x), µ = µ(x) are termed as Lame functions.In an inhomogeneous isotropic medium,
the special form equation (1.3.7) Ei jkl makes it possible to get following equation:-

ρ 2
  
ρ
∇T.∇T − ∇T.∇T − = 0. (1.3.8)
ν + 2µ µ

T (x) satisfies eikonal equation (1.3.8)

1
|∇T |2 = , (1.3.9)
EP2 (x)

9
or we can write this eikonal equation as follows:-
1
|∇T |2 = , (1.3.10)
ES2 (x)
p p
where EP2 = (ν + 2µ)/ρ denotes local P-wave speed and ES2 = µ/ρ denotes local S-wave
speed.

Inverse Kinematic Problem of Seismic Waves:- Let us consider that a point’s original location
x0 ∈ R3 at a time, t = 0 happens energetic. In homogeneous isotropic medium, wavefronts flow
from the origin, with radius EPt (for P-waves) and ESt for S-waves, reaching at generallocation
x at time t = |x − x0 |/EP and t = |x − x0 |/ES . We introduce the function T (x, x0 ) as the travel
time need for wavefront to arrive x from x0 . The functionT (x, x0 ) satisfies equation (1.3.9) for
P-waves and equation (1.3.10) for S-waves.
The first inverse problem was the inverse kinematic problem which was studied in geophysics.
Suppose that our Earth is an isotropic inhomogeneous elastic medium and the estimations of the
seismic waves, starting from a point location x0 and travelling in the interior of Earth, are pro-
vided for the points on the earth’s surface. So, these estimations of input involve the propagate
time T (x, x0 ) of seismic waves between the origin point x0 and any point on earths surface. By
using the estimation of the data, inverse kinematic problem calculates the speed of the seismic
waves inside the Earth. We can explain mathematically inverse kinematic problem as given
below:
Let D is the domain which is bounded by surface of the earth S, and suppose T (x, x0 ) is the
function of the moving time which is need by a signal with no estimation of speed E(x) > 0
to arrive x from x0 . We have to find C(x) for all x from D if function T (x, x0 ) is known for all
points x0 ∈ S1 and x ∈ S2 , where S1 ⊆ S and S2 ⊆ S are subsets of S.
Herglotz and Wiechert and Zoeppritz were the earliest mathematicians who discovered that the
inverse kinematic problem in considerations i.e.,
1 d
E(x) ; r = |x| ; (rn(r)) > 0, (1.3.11)
n(r) dr

x0 is the fixed point from S, if T (x, x0 ) is given for any x from S.Gerver and Markushevich have
d
displayed that the state dr (rn(r)) > 0 can be removed but in this case there are many solutions
for the inverse kinematic problems.

10
1.4 Literature Review
Till so far, many number of research papers have been published on inverse problem in ordinary
differential equation. Some of them are listed below which are related to our thesis work:

1. Using inverse problems as projects in an ordinary differential equations class

Paper: J. Graham, Using inverse problems as projects in an ordinary differential equa-
tions class, (1996)1-8. [4]
In this research paper, author tells about the inverse problem, mathematical features of
inverse problem. Solving the inverse problem is more creativity than solving the direct
problems. There is discussion of various applications of inverse problems in many fields
like medical, geophysical, non-destructive testing, etc. The second order linear and non-
linear differential equation is solved by the data fitting approach and non-linear equation
method.

2. Introduction of Inverse Problem and Its Applications to Science and Technology

Paper: A. Garg, K. Goyal, Kshama Goyal, Introduction of Inverse Problem and it’s
applications to Science and Technology, 2(2013)30-34. [11]
In this research paper, researchers describes the idea of ill-posed problem. By the side
of carefully describing the problem, they also gives some mathematical examples like
Fredholm integral equation, inhomogeneous Helmholtz equation etc. for recognized the
article and the problems which are connected with the study of inverse problems. They
also discuss about the various applications of ill-posed difficulties in the area of science
and technology.

3. Computational Methods for Inverse Problem

Book: Curtis R. Vogel, Computational Methods for Inverse Problem, (2002). [1]
In this book, there is study of inverse problems and various applications of inverse prob-
lem, regularization of inverse problem by various techniques, numerical solution to dis-
cretize problem, numerical optimzation methods and identification of differential equa-
tion. The important topic is selecting the regularization parameter from the statistical
perspective and also provides motivation for adjoint and higher order derivatives.

4. Numerical algorithms for inverse and ill-posed Problems

Book: A.. Denisov, Numerical algorithms for inverse and ill-posed Problems, (1987). [6]

11
 This book describes about inverse problem and ill-posed problems and numerical algo-
rithms for the solution of the problem. Also discussion of inverse and ill-posed problems
begins in science, ecology, engineering, medicine, etc. For the research of different pro-
cesses there are certain mathematical techniques and models. The author tells that the
inverse problem starts when some mathematical model parameters are not well-defined.
It describes the inverse problems in ordinary and partialy differential equations and many
numerical algorithms for solving the inverse problems .

5. Introduction to Inverse Problems for Differential Equations

Book: A. H. Hasanoglu, V. G. Romanov, Introduction to inverse problems for differential
equations, (2017). [3]
The textbook describes the idea of inverse problems arising in PDE’s. They tells the
nature of inverse problem in differential and integral equations which are depends on
well-defined mathematical models. The most unique properties of any inverse problem
are invisible in the characterstics of corelating direct problems solutions related to PDE’s
which are arising in mathematical models. In this book, the author deals with inverse
problems related to the second-order hyperbolic differential equations and for the elec-
trodynamic differential equations and elliptic differential equations.

6. A Survey on Inverse Problems for Applied Sciences

Paper: F. Yaman, V. G. Yakhno, and R. Potthast, A Survey on Inverse Problems for Ap-
plied Sciences, (2013). [5]
In this research paper, the researcher tells that inverse problem has engineering applica-
tions and we can solve them easily by mathematically methods. It decribes the ill-posed
problems for Time-Harmonic Acoustic and Electromagnetic Waves, Boundary Recon-
struction Problems of Acoustic Waves. It solves the inverse problems for differential
equations of elastodynamics and gives the idea regarding inversion based applications.

12
Chapter 2

Regularization Techniques for Solving the

Inverse Problem

We got important knowledge from the discrete linear system Px = d except ill-condition. Now,
let us consider a discrete linear model i.e.,

d = Pxtrue + η, (2.0.1)

with δ = ||η|| > 0.

where,

||.|| = standard euclidean norm,

xtrue = true discretized source,
η = noise in the data,
(δ)= noise level.

Next, let us suppose that P is an invertible, real-valued matrix. Then, it has a Singular Value
Decomposition (SVD):
P = Udiag(si )V T , (2.0.2)

with decreasing singular values values si > 0.Also,

P−1 = PT = V diag(s−1 T
i )U . (2.0.3)

Now, we need some below factors:-

column vectors vi of V termed as right singular vectors, and

13
column vectors ui of U, which are the left singular vectors. These vectors satisfy:

uTi u j = δi j ; vTi v j = δi j , (2.0.4)

Pvi = si ui ; PT ui = si vi , (2.0.5)

here, δi j is the Kronecker delta and U T = U −1 and V T = V −1 .

If P is symmetric and positive definite matrix, then unique values of si are the eigenvalues of P
and U = V has columns which involves the orthonormalized eigenvectors. Now, post-multiply
P−1 with equation (2.0.1),we get:

⇒ P−1 d = P−1 Pxtrue + P−1 η

⇒ P−1 d = xtrue + P−1 η

n
⇒ V diag(s−1 T −1 T
i U )d = xtrue + ∑ si (ui η)vi
i=1
n
⇒ P−1 d = V diag(s−1 T −1 T
i )U = xtrue + ∑ si (ui η)vi . (2.0.6)
i=1

When we divides the small singular values s−1

i , then there is some unstability. So, there are
three methods to remove the error. These are given below:-
2.1) Truncated Singular Value Decomposition (i.e., TSVD)
2.2) Tikhonov(-Phillips) Regularization
2.3) Least Square Fit method

2.1 Truncated Singular Value Decomposition

To overcome this error in equation (2.0.6), we can change the s−1
i i.e., multiplying equation
(2.0.6) by a regularizing filter function wα (s2i ) and the product wα (s2i )s−1 → 0 as s → 0. This
filters out the unique elements of K −1 d relating to small singular values and gets an approxima-
tion to xtrue with the following representation:
n
xα = V diag(wα (s2i )s−1 T
i )U d = ∑ wα (s2i )s−1 T
i (ui d)vi . (2.1.1)
i=1

14
We should keep the unique elements relating to large singular values for getting some degree of
accuracy. It is possible by considering wα (s2 ) ≈ 1 for large values of s2 . For example, of such
a filter function: 
1; i f s2 > α
wα (s2 ) = (2.1.2)
0; i f s2 ≤ α

The approximation of equation (2.1.1) becomes:

xα = ∑ s−1 T
i (ui d)vi , (2.1.3)
2
si >α

and this is called as the Truncated Singular Value Decomposition (i.e., TSVD) solution to
px = d.

15
2.2 Tikhonov Regularization
Let us consider the Tikhonov Filter Function:
s2
wα (s2 ) = , (2.2.1)
s2 + α

Put wα (s2 ) in equation (2.1.1), we get:

n
si (uTi d)
xα = ∑ 2
vi = (PT P + αI)−1 PT d. (2.2.2)
i=1 si + α

The equation (2.2.2) is a result of equation (2.0.2) and equation (2.0.5). This technique is known
as Tikhonov Regularization.

Figure 2:Plotting of Tikhonov’s Filter Function

In Figure 2, dotted line represents the filter function wα corresponding to TSVD regularization
and solid line represents the squared singular values s2 corresponding to Tikhonovs Regulariza-
tion. The input data for the regularization constant is α = 10−2 .

16
 Problem:- Numerically implement standard Tikhonov regularization for the test problem.
The true solution is given by:



0.75, 0.1 < x < 0.25,


0.25, 0.3 < x < 0.32,

ftrue =
sin4 (2πx), 0.5 < x < 1,





0, otherwise


Solution:- We solve this problem by using this formula d = Pxtrue + η, where ||η|| > 0 i.e.,η =
0.01(say).

⇒ d = Pxtrue + 0.01,

From the equation (1.2.3), we know that:

((i − j)k)2
 
[P]i j = kcexp , 1 ≤ i, j ≤ n
2λ2

(i − j)2
 
1 1
⇒ [P]i j = √ exp .
n λ 2π 2 × λ2 × n2
Also,we know that there are four inputs so the value of n = 4, λ = 0.05. Now put these values
in the above equation,we get:

(i − j)2
 
1
⇒ [P]i j = √ exp
4 × 0.05 2π 2 × (0.05)2 × (4)2

(i − j)2
 
1
⇒ [P]i j = √ exp
4 × 0.05 2π 2 × 0.0025 × 16
(i − j)2
 
1
⇒ [P]i j = exp
0.50142 0.08
(i − j)2
 
⇒ [P]i j = 1.9943 × exp .
0.08
So, we get the values of matrix [P]i j by putting the values of i and j, where 1 ≤ i ≤ 4 and
1 ≤ j ≤ 4. Also, we find the [P]i j . Then, substituting the values of [P]i j , xtrue and η = 0.01
in equation (1.2.3) and we get the values of d. Then, we substitute all values in the equation
(2.2.2) i.e., xα = (PT P + αI)−1 PT d for finding the values of xα . In this way , we can find the
solution of the problem.

17
The Matlab Code For Tikhonov Regularization is given below:-

clear variables
close all
clc
h=0.25;
gamma=0.05;
pi=22/7;
c=1/(gamma*sqrt(2*pi));
for i=1:4
for j=1:4
K(i,j)=h*c*(exp(-(((i-j)*h)ˆ2)/(2*(gamma)ˆ2)));
end
end
K
itta=0.01;
ftrue=[0.75;0.25;sin(2*pi*0.8)ˆ4;0];
d=(K*ftrue)+itta;
d
alpha=10ˆ(-2);
I=[1 0 0 0;0 1 0 0;0 0 1 0;0 0 0 1]
P=transpose(K);
P
falpha=(((P*K)+ (alpha*I))ˆ-1)*P*d
Output:

K=
1.9943 0.0000 0.0000 0.0000
0.0000 1.9943 0.0000 0.0000
0.0000 0.0000 1.9943 0.0000
0.0000 0.0000 0.0000 1.9943

18
d =
1.5057
0.5086
1.6373
0.0100

I =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1

P =
1.9943 0.0000 0.0000 0.0000
0.0000 1.9943 0.0000 0.0000
0.0000 0.0000 1.9943 0.0000
0.0000 0.0000 0.0000 1.9943

falpha =
0.7531
0.2544
0.8189
0.0050

19
2.3 Regularized Least Square Fit Method
Least squares are famous for solving the inverse problems because it helps in the easiest cal-
culations. The drawback of least square in a data set is that only a strong sensitivity to a small
number of large errors. First of all, we should know about the least square fit method, it is de-
sign of mathematical regression analysis that calculates best fit the line of a data set, by giving
a clear signification of the connection between the data points. Every data point is a entitled of
the connection between given self-contained variable and not well-known dependent variable.
We consider a linear discrete system Px = q, where P is m ∗ n matrix with m ≤ n. Suppose that
components of P and q are added by some error. In this case, error in the data is the total least
square :

min||(P, q) − (P̄, q̄)||F , (2.3.1)

subject to q̄ = P̄x. If components of P are exact and only q consists some error. Then, the
composition of least square problem is given below:

minx ||Px − q||2 . (2.3.2)

Our main target is on ill-posed problem where unique value of P decompose slowly to zero.
This type of problem arise in the discretization of ill posedness. In these problems, contains
noise to the small singular values of P. Due to this, it is compulsory to evaluate a regularized
solution in which error effect is filtered out. For least square problems, the filtering is done
by Tikhonov’s regularization method. For the least squares problems the general method of
Tikhonov’s regularization for Least Square Method is given below:-

min||Px − q||22 + µ||Nx||22 , (2.3.3)

where µ > 0 that we select to control the size of output vector and L is a matrix which describes
the semi-norm ||Nx||2 of output vector i.e., L represents the first or second degree differential
operator. If N is identity matrix, then tikhonov problem is termed to be in it’s standard form:

(PT P + µN T N)x = PT q. (2.3.4)

When µ increases, then semi-norm ||Nx||2 of output vector decreases monotonically while resid-
ual i.e., min||Px − q||2 increases monotonically.

20
Chapter 3

Inverse Problem in Ordinary Differential

Equations

When we are solving the inverse problem in differential equations, it consists more innova-
tion than finding standard direct problems which makes the most challenge to knowledge. By
solving inverse problems in ordinary differential equation enable us to superior recognized the
direct problem on which inverse problems are depend. Now, we consider the example problem
of inverse problem which is given in the form of ordinary differential equation.

3.1 An Example Problem

The inverse problem which is based on second order ordinary differential equation which is
used to prototype the harmonic motion. The problem posed will not be solved in full generality
but we want to generate the concept that what is an inverse problem which is consisted in the
model of harmonic motion. The problem becomes critically damped or overdamped in place
of the assumption. In case of underdamped requires of alterations to the approaches which are
given in the problem.
Formulate a method of recovering the coefficients in second order ordinary differential equation:

mx”(t) + bx0 (t) + kx(t) = f (t), (3.1.1)

from the measurements of x(t). Here we are free to select initial conditions and function(s)
in any manner which fits properly. We assume that the solution to homogeneous equation
approaches zero over time.

21
3.2 Two Approaches for the Solution of Inverse Problems
We are applying two approaches to the solution to get proper solution. There are two methods
to solve the second order ordinary differential equation:-
1.Data Fitting Method
2.Non-linear Equation Method

3.2.1 Data Fitting Method

We take f (t) = t 2 for the right hand side since this choice leads to a particular solution .

mx”(t) + bx0 (t) + kx(t) = t 2 , (3.2.1)

where m = 1, b = 10, k = 49.

Now, we are solving this differential Equation by putting the values of m, b, k in equation(3.2.1).
Therefore it becomes:
x00 (t) + 10x0 (t) + 49x(t) = t 2 . (3.2.2)

Complementary function:-To calculate the value of this function, we are using D instead of
derivative. So that it becomes in the following form:-

D2 + 10D + 49 = 0
√
−b ± b2 − 4ac
⇒D=
2a
p
−10 ± (10)2 − 4 ∗ 1 ∗ 49
⇒D=
2∗1
√
−10 ± 100 − 196
⇒D=
2
√
−10 ± −96
⇒D=
2
√
⇒ D = −5 ± 2 6i.
√ √
Therefore, C.F. = e(−5t) (c1 cos(2 6t) + c2 sin(2 6t)).
√ √
Here, e(−5t) is decreasing function and (c1 cos(2 6t) + c2 sin(2 6t)) is bounded function. So,
we neglect it due to very small term.

22
Particular Solution:-Now, we are solving the particular solution of equation (3.2.2).
We know that,
f (t) = at 2 + bt + c. (3.2.3)

Differentiate equation (3.2.3) w.r.t. to t, we get f 0 (t):-

⇒ f 0 (t) = 2at + b.

Also differentiate f 0 (t) w.r.t. to t, we get f 00 (t):

⇒ f 00 (t) = 2a.

Now, substitute the values of f (t), f 0 (t) and f 00 (t) in the following equation (3.2.4), and we
get:-
f 00 (t) + 10 f 0 (t) + 49 f (t) = t 2 (3.2.4)

⇒ 2a + 10(2at + b) + 49(at 2 + bt + c) = t 2

⇒ 2a + 20at + 10b + 49at 2 + 49bt + 49c = t 2

⇒ 49at 2 + (20a + 49b)t + 2a + 10b + 49c = t 2 .

Now, we are comparing the coefficients of t on both sides, so that we calculate the values of a,
b, and c respectively:-
49a = 1
1
⇒a= ,
49
20a + 49b = 0

⇒ 49b = −20a

Now, put the value of a = 1/49 in the above equation to find the value of b, we get:-

−20 1
⇒b= ×
49 49
20
⇒b=−
(49)2
2 × 10
⇒b=− ,
(49)2

23
 2a + 10b + 49c = 0.

We substitute the values of a = 1/49 and b = −20/(49)2 in the above equation and we get:-
   
1 20
⇒ 2× + 10 × − + 49c = 0
49 (49)2
2 200
⇒ 49c = − +
49 (49)2
2 200
⇒c=− 2
+
(49) (49)3
2 2 × (10)2
⇒c=− + .
(49)2 (49)3

Now, put all the above calculated values of a, b and c in the equation (3.2.3) and we get the
following equation (3.2.5):-

1 2 2 × 10 2 2 × (10)2
f (t) = t − t − + . (3.2.5)
49 (49)2 (49)2 (49)3

Therefore, we get the following particular solution x p (t) (in general) :-

1 2b 2m 2b2
x p (t) = t 2 − 2 t − 3 + 3 (3.2.6)
k k k k

⇒ f (t) = a2t 2 − a1t + a0 . (3.2.7)

where,
1 2b 2m 2b2
a2 = , a1 = − 2 , a0 = − 2 + 3 (3.2.8)
k k k k

If we suppose that the system runs up till x p (t) is superior, then we can apply the least squares
fit in the form of a2t 2 + a1t + a0 to input and evaluate the system for m, b, k (inverse values)
from the calculated values of a2 , a1 , a0 . Clearly,by using this method the basic conditions are
not required specially. When we perform least square fit on equation (3.2.7),we get:

n n n
2
∑ f (t) = a2 ∑ t − a1 ∑ t + a0 n. (3.2.9)
i=1 i=1 i=1

24
Now,we are multiplying the equation(3.2.9) with t,we get:-
n n n n
3 2
∑ t f (t) = a2 ∑ t − a1 ∑ t + a0 ∑ t. (3.2.10)
i=1 i=1 i=1 i=1

Also,multiply the equation(3.2.10) with t,we get the following equation(3.2.11) :-

n n n n
2
∑t f (t) = a2 ∑ t − a1 ∑ t + a0 ∑ t 2 .
4 3
(3.2.11)
i=1 i=1 i=1 i=1

Now, we calculate the values of t, f (t), t f (t), t 2 , t 3 , t 4 and t 2 f (t) in equations (3.2.9), (3.2.10)
and (3.2.11) from the below the table:

Table 1
t f (t) t f (t) t2 t3 t4 t 2 f (t)
0 0.0008 0 0 0 0 0
1 0.0129 0.0129 1 1 1 0.0129
2 0.0658 0.1316 4 8 16 0.2633
∑t = 3 ∑ f (t) = ∑ t f (t) = ∑ t 2 = ∑ t 3 = ∑ t 4 = ∑ t 2 f (t) =
0.0796 0.1446 5 9 17 0.2763

Put these all calculated values of t, f (t), t f (t), t 2 , t 3 , t 4 and t 2 f (t) in equations (3.2.9), (3.2.10)
and (3.2.11) respectively, we get:

3a0 − 3a1 + 5a2 = 0.08, (3.2.12)

3a0 − 5a1 + 9a2 = 0.14, (3.2.13)

5a0 − 9a1 + 17a2 = 0.27. (3.2.14)

After solving these three above equations ,we get the values of a0 , a1 , a2 respectively

a0 = 0.02167; a1 = 0.12; a2 = 0.075 (3.2.15)

Now put the values of equation (3.2.15) in equation (3.2.8),we get:

m = 6.605986; b = 10.677336; k = 13.34 (3.2.16)

25
3.2.2 Non-linear Equation Method
We take f (t) = F0 cos(ωt) on the right side for finding the particular solution in the following
non-linear equation.
mx”(t) + bx0 (t) + kx(t) = F0 cos(ωt), (3.2.17)

where m = 1, b = 10, k = 49. Now, we are solving the differential Equation and it becomes:

x00 (t) + 10x0 (t) + 49x(t) = F0 cos(ωt). (3.2.18)

Complementary function:-To calculate the value of this function, we are using D instead of
derivative. So, it becomes in the following form:-

D2 + 10D + 49 = 0
√
−b ± b2 − 4ac
⇒D=
2a
p
−10 ± (10)2 − 4 ∗ 1 ∗ 49
⇒D=
2∗1
√
−10 ± 100 − 196
⇒D=
2
√
−10 ± −96
⇒D=
2
√
⇒ D = −5 ± 2 6i.
√ √
Therefore, C.F. = e(−5t) (c1 cos(2 6t) + c2 sin(2 6t)).
√ √
Here, e(−5t) is decreasing function and (c1 cos(2 6t) + c2 sin(2 6t)) is bounded function. So,
we neglect it due to very small term.

Analytic Solution:-We are using the three different forcing functions in the form of F0 cos(ωt)
and the amplitude is calculated . The analytic solution must have an amplitude which is given
below equation:
F0
A(m, k, b) = p . (3.2.19)
(k − mω2 )2 + b2 ω2
We can get the system of non-linear equations from the three different values of ω = 1, 2, 3 ,
a 3 × 3 system of non-linear equation. Now, we are using the Newton’s method for solving the
system of equations for m, b, and k. The main advantage of this approach is that it can be easily
used in terms of RLC circuit. So, in this case the actual data can be collected. Also, the need of

26
initial conditions are not important in the analytic solution.

Also, we know that the values of m, b, and k respectively( i.e., m = 1; k = 49; b = 10 ). So, we
can perform the newton method (xn+1 = xn − [J(xn )]−1 ∗ f (xn )) easily on equation (3.2.19) :-
At initial point,    
1 f1
   
x0 = 
49 ; f (x0 ) =  f2 
  
10 f3
and ∂f ∂ f1 ∂ f1

1

 ∂∂mf2 ∂k
∂ f2
∂b
∂ f2 
 ∂m ∂k ∂b 

J(x0 ) = 
 ∂ f3 ∂ f3 ∂ f3 
 ∂m ∂k ∂b 

where,
For ω = 1,
1
f1 (m, k, b) = p ;
(k − m)2 + b2

For ω = 2
1
f2 (m, k, b) = p ;
(k − 4m)2 + 4b2

For ω = 3
1
f3 (m, k, b) = p .
(k − 9m)2 + 9b2

Now,

∂ f1 k−m
= ;
∂m ((k − m)2 + b2 ) 23
∂ f1 −(k − m)
= 3 ;
∂k ((k − m)2 + b2 ) 2
∂ f1 −b
= 3 ;
∂b ((k − m)2 + b2 ) 2

27
 ∂ f2 4(k − 4m)
= ;
∂m ((k − 4m)2 + 4b2 ) 32
∂ f2 −(k − 4m)
= 3 ;
∂k ((k − 4m)2 + 4b2 ) 2
∂ f2 −4b
= 3 ;
∂b ((k − 4m)2 + 4b2 ) 2
∂ f3 9(k − 9m)
= ;
∂m ((k − 9m)2 + 9b2 ) 32
∂ f3 −(k − 9m)
= 3 ;
∂k ((k − 9m)2 + 9b2 ) 2
∂ f3 −9b
= 3
∂b ((k − 9m)2 + 9b2 ) 2
, and  
∂ f1 ∂ f1 ∂ f1
 ∂∂mf ∂k
∂ f2
∂b 
∂ f2 
J(x0 ) = 
 ∂m
2
∂k ∂b 
.
∂ f3 ∂ f3 ∂ f3
∂m ∂k ∂b

Now, put the values of m = 1; k = 49; b = 10 in the above jacobian,we get:

 
0.00040 −0.00040 −0.000084
 
J(x0 ) =  0.0015 −0.00037 −0.00033 


0.00288 −0.00032 −0.00072

 
−4380.23012552301 7112.97071129706 −2749.08472803347
−1
 
[J(x0 )] = −3530.33472803347 1255.23012552301 −163.441422594142


−15951.8828451883 27894.0027894002 −12312.5871687587

Now,we are using the Newton’s Method :- xn+1 = xn − [J(xn )]−1 ∗ f (xn )

28
For n = 0:- x1 = x0 − [J(x0 )]−1 ∗ f (x0 )). i.e.,
     
1 −4380.23012552301 7112.97071129706 −2749.08472803347 0.02039
     
x1 = 49 − −3530.33472803347 1255.23012552301 −163.441422594142 ∗ 0.14250
    

10 −15951.8828451883 27894.0027894002 −12312.5871687587 0.02

   
1 865.7
   
x1 = 49  − 3389.4
   
10 1030
 
−864.7
 
x1 = −3379.4


−981

29
3.3 MATLAB Coding

3.3.1 Matlab coding for Data Fitting Method

The matlab code of the sample problem for the linear equation and we are performing the
least square fit by using the MATLAB command ”polyfit”. The second matlab code using the
MATLAB function ”ode45”.

datafit.m
tol=10e-6; t0=0; tf=5;
f=’yp’; y0=[1;0];
[T,U]=ode45(f,t0,tf,y0,tol,1)
q=length(T);
x=T(q-20:q); y=U(q-20:q,1);
p=polyfit(x,y,2);

prime.m
function yprime = yp(t,y)
yprime(1)=y(2);
yprime(2)=t*t-10*y(2)-49*y(1);

30
3.3.2 Matlab coding for Non-linear Equation Method
The first matlab code is used to evaluate the Jacobian of the function in the non-linear equation
method . The third matlab code computes the amplitudes of the analytic solution. The final
matlab code computes the solution via a simplified version of Newton’s method.

jacobian.m
function y = jac(m,b,k,w)
y=zeros(3);
for j=1:3
denom = ((k-m*w(j)*w(j))ˆ2+b*b*w(j)*w(j))ˆ(3/2);
y(j,1) = (k-m*w(j)ˆ2)*w(j)ˆ2/denom; % derivative wrt m
y(j,2) = -b*w(j)*w(j)/denom; % derivative wrt b
y(j,3) = (m*w(j)ˆ2-k)/denom;
end

Final.m
function y=F(m,b,k,w,amp)
y=zeros(3,1);
for j=1:3
y(j) = 1/((k-m*w(j)ˆ2)ˆ2+bˆ2*w(j)ˆ2)ˆ(1/2) - amp(j);
end

amplitude.m
function y=camp(m,b,k,w)
y=1/((k-m*wˆ2)ˆ2 + bˆ2*wˆ2)ˆ(1/2);

nonlinear.m
for j=1:3
amp(j) = camp(10,3,49,j);
end
w=1:3;
x=ones(3,1);
noise=2*rand(3,1)-1;
amp2=amp’+.001*noise*max(amp);

31
for i=1:10
x=x-inv(jac(x(1),x(2),x(3),w))*F(x(1),x(2),x(3),w,amp2)
end

32
 Conclusion

In this thesis report, we have studied that inverse problems which are used in differential equa-
tion in many fields like medical imaging, geophysics, image formation, physics, ground-water
flow model, navigation and so many other fields. We have solved the ordinary differential equa-
tion and then find the inverse problem of linear differential equation by using the method of
Least Square Fit and of non-linear differential equation by using the Newton method.

33
34
Bibliography

[1] C. R. Vogel, Computational Methods for Inverse Problem, SIAM, (2002).

[2] R. C. Aster, B. Borchers, C. H. Thurber, Parameter Estimation and Inverse Problems,

Elsevier, 2(2004).

[3] A. H. Hasanoglu, V. G. Romanov, Introduction to Inverse Problems for Differential Equa-

tions, Springer, (2017).

[4] J. Graham, Using inverse problems as projects in an ordinary differential equations class,
(1996), 1-8.

[5] F. Yaman, V. G. Yakhno and R. Potthast, A Survey on Inverse Problems for Applied Sci-
ences, Hindawi Publishing Corporation, (2013) 1-19.

[6] A. M. Denisov, Numerical algorithms for inverse and ill-posed Problems, (1987).

[7] T. Hohage, Lecture notes on Inverse Problems, (2002).

[8] M. Bertero and M. Piana, Inverse Problems in biomedical imaging: modeling and methods
of solution, Springer.

[9] J. Baumeister, Stable Solution of Inverse Problems, (1986).

[10] S. M. Tan and C. Fox, Introduction to Inverse Problems, 34-58.

[11] A. Garg, K. Goyal, Kshama Goyal, Introduction of Inverse Problem and Its Applications
to Science and Technology, 2 (2013), 30-34.

35