MIZAN TEPI UNIVERSITY

COLLEGE OF NATURAL AND COMPUTATIONAL SCIENCES

DEPARTMENT OF MATHEMATIC
TITLE: SOLVING HEAT EQUATION BY USING ARTIFICIAL NEURAL
NETWORK

BY: GROUP ONE

NO.
STUDENT NAME ID NUMBER
1. SINTAYEHU SITOTAW NSR/1503/13

2. AZEMERAW CHEKOL NSR/0299/13

3. TADESSE KASSEW NSR/1762/12

4. TESFAHUN BANTEGIZE NSR/1606/13

5. TILKSEW CHANE NSR/2204/13

The project proposal submitted to Mathematics Department, college of Natural and
Computational Science and Mizan Tepi University in partial fulfillment of the
requirement for the degree of bachelor science (B.Sc.) in Mathematics .
SUBMITTED TO MR.TENAW

SUBMISION DATE: JANUARY 2024

TEPI ETHIOPIA
 Acknowledgements
Above all, I would like to thank the almighty God that, he helped us to reach this level; I would
like to express my sincere and deepest thanks to my advisor, MR.Kassahun. Who provided
constructive advice and unreserved guidance to successfully complete this project proposal.

I would like also to express my sincere and deepest thanks to my family and Mizan-Tepi
University for giving me a chance to conduct this research project proposal.

Finally, I extend my thanks to our friends who asked and advised us in any time to conduct and
finalize this project proposal.
 Summary
Partial Differential Equations have important uses in many fields including physics and engineering. Due
to their importance, heavy research has been done to solve these problems efficiently and effectively.
However, some PDEs are still challenging to solve using classical methods, often due to the
dimensionality of the problem. In recent years, it has been thought that neural networks may be able to
solve these problems effectively. This research assesses how well a neural network can estimate a
simple example PDE, the heat equation, as well as the practicality of doing so.

.
 Table Content
Acknowledgment i

Summary ii
CHAPTER1: INTRODUCTION １
1.1 Background of the study…………………………………………………...1
1.2 statement of the problem………………………………………………………………………..2
1.3 Objective of the study……………………………………………………...2
1.3.1 General objective…………………………………………………………………………….2
1.3.2 Specific objectives…………………………………………………………………………..2
1.4 Significance of the study.......................................................……………..３
1.5 Expected outcomes………………………………………………………...3
CHAPTER 2: LITERATURE REVIEW ４
CHAPTER 3: RESEARCH METHODOLOGY 6
3.1 Mathematical Procedures……………………………………………………………………………….6
3.2 Study Design……………………………………………………………………………………………………7
CHAPTER 4: RESEARCH PLAN AND BUDGET BREAKDOWN ８
4.1 Research Plan………………………………………………………………………………..8
4.2 Budget Break Down……………………………………………………………………….8
References ９
 CHAPTER 1
INTRODUCTION

1.1 Background of the study

Partial Differential Equations (PDEs) have many important applications in scientific fields including
physics, engineering, finance and more. They are often used to model physical phenomena such as the
diffusion of heat or sound, the flow of fluids, electrodynamics, and other physical problems. Due to the
importance of PDEs, there already exists a number of methods to approximate the solutions to these
problems. However, many of these methods become less practical as the dimensionality of the
problems increase. Due to this, there is still a large amount of research being done on additional
methods for solving PDEs. One research area that has received a lot of attention lately is the use of
neural networks for solving PDEs. Neural networks are an important sector of machine learning and are
modeled after the human brain. Neural networks are promising for approximating the solutions of PDEs
due to their strong function approximation capabilities.

There are many important PDEs, but this paper will focus on the heat equation. The heat equation
describes the temperature distribution and diffusion of heat within an object. The majority of my
research dealt with the 1-dimensional heat equation because data was most readily available in this
dimension. Using the 1- dimensional heat equation also serves as a starting point for assessing the
ability of neural networks to approximate the heat equation. Because the 1-dimensional version is the
most simple, it should also be the easiest to model with a neural network, providing a base estimate for
how well neural networks can approximate the solution to PDEs.

This paper will report how accurately different types of neural networks can estimate the heat
equation. A variety of networks are used, including standard artificial neural networks with a variety of
activation functions, a radial basis function network (RBF network), and simple example of a ”physics
inspired neural network”.

Artificial neural network (ANN) is one of the popular areas of artificial intelligence (AI) research and
also an abstract computational model based on the organizational structure of the human brain [1]. The
simplest definition of ANN is provided by the inventor of one of the first neurocomputers, Dr. Robert
Hecht-Nielsen. He defines a neural network as a computing system made up of a number of simple,
highly interconnected processing elements, which process information by their dynamic state response
to external inputs.

ANN is a data modeling tool that depends upon various parameters and learning methods [2–8].
Neural networks are typically organized in layers. Layers are made up of a number of interconnected
“neurons/nodes,” which contain “activation functions.” ANN processes information through
neurons/nodes in a parallel manner to solve specific problems. ANN acquires knowledge through

1
learning, and this knowledge is stored within interneuron connections’ strength, which is expressed by
numerical values called “weights.”.

1.2 statement of problem

In this study we will consider the heat equation on partial differential equation.

This study is intended to answer the following basic equations;

 Can we write an efficient algorithm for the proposed method?

 Is the artificial neural network method efficient to solve heat equation.
 In the artificial neural network is efficient when we compare with the other method?


1.3 Objective of the study

1.3.1 General objective

The general objective of this study is to compute solution of heat equation by using artificial
neural network.

1.3.2 Specific objectives

The specific objectives of this study is:

• To write an efficient algorithm for artificial neural network method.

• To solve the heat equation by using artificial neural network.

2
• To comparing the efficiency of artificial neural network method.

1.4 Significance of the study

The result will be obtained from this study may

• Used to reference material for scholars who work in this research area.

• Offer a solution for system of heat equation offer a solution for system of artificial neural
network.

• To develop the algorithm of a deep artificial neural network method

1.5 Expected outcomes of the study

The expected outcomes of this study will be the formulations of heat equation for artificial neural
network. Finally, the result will be written in the form of a thesis for degree purpose and published in an
internet-indexed science journal.

3
 CHAPTER 2

LITERATURE REVIEW

Many physical phenomena in modern sciences have been described by using Partial Differential
Equations (PDEs) (Evans, Blackledge, & Yardley, 2012). Hence, the accuracy of PDE solutions
is challenging among the scientists and becomes an interest field of research (LeVeque &
Leveque, 1992). Traditionally, the PDEs are solved numerically through discretization process
(Burden, Faires, & Burden, 2015),. For instance, the well-known finite difference method (FDM)
and finite element method were utilized to solve many PDE linear and non-linear. Other
methods, such as the variational iteration method (VIM) and its variations were used to solve the
nonlinear PDE (He & Latifizadeh, 2020), and the finite difference-spectral method was
investigated to solve the fractal mobile and immobile transport (Fardi & Khan, 2021). These
methods typically end up with the algebraic systems that can be solved by using iterative
methods (Hayati & Karami, 2007). The big issue in using the iterative solvers for solving the
large scale of linear system of equations is that they potentially breakdown before getting a good
approximate solution (Maharani & Salhi, 2015). In fact, their accuracy is not promising. to get
rid of the breakdown problem, one has been done by using interpolation and extrapolation model
(Bakar & Salhi, 2019; Maharani et al., 2018; Maharani, Larasati, Salhi, & Khan, 2019), and
using prediction with support vector machine (Thalib, Bakar, & Ibrahim, 2021). However, the
problem is still not fully addressed since computationally, they quite expensive. With no
discretization process, artificial neural networks (ANNs) can be an alternative way. ANN is well-

4
known as one method under machine learning (ML) which is typically used for regressions and
classification problems. The development of ANN for solving PDE problems has been
investigated at the beginning of the 21st century. For instance (Malek & Beidokhti, 2006),
combined ANN and Nelder–Mead simplex method to find the numerical solutions of the high-
order of PDE. This hybrid method improved the ANN performances by approximating initial and
boundary conditions. Moreover (Sirignano & Spiliopoulos, 2018), used the Deep-Galerkin
method (DGM) embedded with ANN, for solving the high dimensional of PDE problems. While,
modified DGM by introducing ansatz method for binding the initial and boundary conditions.
ANN-based method for solving PDE, called Physics another Informed Neural Network (PINN),
was introduced by Raissi, Perdikaris, & Karniadakis (2017b), Raissi, Perdikaris, and Karniadakis
(2019) and Raissi et al. (2017b). PINN considers the physical laws of PDE to be embedded in
loss function as a regularization term. This method was improved by Guo, Cao, Liu, and Gao
(2020), in terms of the training effect by using the residual-based adaptive refinement (RAR)
method. This strategy will impact in increasing the number of residual points with the large
residuals of PDE until the residuals are less than the threshold. The ability of ANN in solving
PDE problems gives some advantages, including continuous and differentiable of the
approximate solutions, good interpolation characteristics and less memory (Chen et al., 2020).
Other advantages of ANN are that it can utilizeautomatic differentiation tools, such as
Tensorflow (Abadi et al., 2016) and PyTorch (Paszke et al., 2017; Rahaman et al., 2019), allow
researchers make more simpler methods in solving PDE problems (Chen et al., 2020). In this
study, we focus on three methods for solving PDEs based on ANN model, namely PyDEns
which modifies the DGM, NeuroDiffEq which is the ANN approximator with TAS applied
(Chen et al., 2020), and Nangs which based on the grid points for training data).This
modification simplifies the DGM original algorithm. Furthermore,

5
 CHAPTER 3

RESEARCH METHODOLOGY
3.1 Mathematical Procedures
In this study we focus on the heat equation.

Consider the second order partial differential equation of the form (McFall, 2010),

∂ u ∂ u ∂2 u ∂2 u
F(x, t, u, , , , )=f( x, t),( x, t)ϵ Ω, (1)
∂ x ∂ t ∂ x2 ∂ t 2

over the domain Ω ⸦ R2 with an initial condition

u(x,t 0)=u0 (x),xϵ σΩ, (2)

and a boundary condition U(x ,t)=ɡ(x, t), (x ,t) ϵ 𝜎Ω (3)

Generally, ANN to solve PDE (Equation (1)) is started by generating the weights to form a
linear combination with the inputs x, t and the bias, b i.

This form is then used to compute the hidden layer as described in Equation (4) as follows

h m
1=¿ ∑ ¿ ¿¿ x +w tit) +b 1, (4)
i=1

6
where h1 is the first hidden layer, w xi and w tiare the weights and b 1 is bias. The second hidden
layer, as expressed in Equation (5), is computed by feeding h1 into it and thus is processed to
yield the output layer.

n m
h2 =∑ ∑ v ijf (h1 ) +b 2 (5)
j=1 i=1

Where v ij are the weights, b 2 is the bias, and f is the activation function of the form

h1 −h1
e −e
f (h1 )= h −h . (6)
e +e 1 1

our aim here is to obtain the approximate solution unet (x ,t) which is written as follows,

n
unet (x ,t) = ∑ ƿj f (h2) (7)
j=1

Where ƿ jare the weights of the output layers . To control the accuracy of the approximate
solution, we compare it with the right-hand side of the PDE (Equation (1)), and this can be only
done by differentiate partially unet (x, t) as follows

k n k
∂ unet ∂ f ( h2)
k
(x , t)=∑ ƿ j k
, (8)
∂x j=1 ∂x

k n k
∂ unet ∂ f ( h2)
k
(x ,t)=∑ ƿ j k
, (9)
∂t j=1 ∂t

Where k=1,2.

3.2 Study Design

This study will involve the ANN implementation with python software. To compute the solution
of heat equation in partial differential equation using artificial neural network(ANN) method.
Also we will compare the deep artificial neural network(ANN) method solution using software.

7
 CHAPTER 4
RESEARCH PLAN AND BUDGET BREAKDOWN
4.1 Research plan
The periodical evaluation of the development of the activities of this proposal, designing a main
time schedule is worth while.

NO Activities November December January

1 Title Selection √
2 Preparing of Project √
3 Project Comment √
4 Submission of Project √

8
4.2 Budget breakdown
The action research requires fanatic. The researcher uses the finance for various activities,
which is summarized as follows.

Table 4.1: Budget Requirement Analysis

No. Item Quantity(amount) Unit Price

1 Internet Service 3day Free
2 Pen 1 25birr
3 Cover Page No no
4 Flash Disk No no
5 For Print No no
6 Note Book 1 50birr
Total
cost=75birr

9
 References

Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., … Isard, M. (2016). Tensorflow: A system for
large-scale machine learning. 12th fUSENIXg symposium on operating systems design and
implementation (fOSDIg 16) (pp. 265–283). Savanah, GA, US: USENIX. Aggarwal, R., & Ugail, H. (2019).On
the solution of poisson’s equation using deep learning. 2019 13th International Conference on Software,
Knowledge, Information Management and Applications (SKIMA) (pp. 1–8). Piscataway, NJ: IEEE.
doi:10.1109/SKIMA47702.2019. 8982518 Ahmat, N., Castro, G. G., & Ugail, H. (2014). Automatic shape
optimisation of pharmaceutical tablets using partial differential equations. Computers & Structures, 130,
1–9. doi:10.1016/j.compstruc.2013.09.001 Arnal, A., Monterde, J., & Ugail, H. (2011). Explicit polynomial
solutions of fourth order linear elliptic partial differential equations for boundary based smooth surface
generation. Computer Aided Geometric Design, 28(6), 382–394. doi:10.1016/j.cagd.2011.06.001 Bakar,
M. A., & Salhi, A. (2019). Rmeiemla: The recent advance in improving the robustness of lanczos-type
algorithms. AIP Conference Proceedings (pp. 03–09). Melville, NY: AIP Publishing LLC. Bakar, M. A., N.,
A., Larasati, N., Salhi, A., & Khan, W. M. (2019). Lanczos-type algorithms with embedded interpolation
and extrapolation models for solving large scale systems of linear equations. International Journal of
Computing Science and Mathematics, 10(5), 429–442. doi:10.1504/IJCSM.2019.103675 Blakely, R. J.
(1996). Potential theory in gravity and magnetic applications Cambridge: Cambridge University Press.
Burden, R., Faires, J., & Burden, A. (2015). Numerical analysis, brooks and cole. Boston, MA: Cencag
Learning. Chaudhry, E., Bian, S., Ugail, H., Jin, X., You, L., & Zhang, J. J. (2015). Dynamic skin deformation
using finite difference solutions for character animation. Computers & Graphics, 46, 294–305.
doi:10.1016/j.cag.2014.09.029 Chen, F., Sondak, D., Protopapas, P., Mattheakis, M., Liu, S., Agarwal, D.,
& Giovanni, M. D. (2020). Neurodiffeq: Apython package for solving differential equations with neural
networks. Journal of Open Source Software, 5(46), 1931. doi:10.21105/joss.01931 Elmahmudi, A., &
Ugail, H. (2019). The biharmonic eigen face. Signal, Image and Video Processing, 13(8),1639–1647.
doi:10.1007/s11760-019-01514-4 Elsherbeny, A. M., El-hassani, R. M., El-badry, H., & Abdallah, M. I.
(2018). Solving 2d-Poisson equation using modified cubic b-spline differential quadrature method. Ain
Shams Engineering Journal, 9(4), 2879–2885. doi:10.1016/ j.asej.2017.12.001 Evans, G., Blackledge, J., &
Yardley, P. (2012). Numerical methods for partial differential equations. Berlin, Germany: Springer
Science & Business Media. Fardi, M., & Khan, A. Y. (2021). A novel finite difference spectral method for
fractal mobile and immobile transport (FM and it) model based on capito-fabrizio derivative. Chaos,
Solitons and Fractals, 143(2021), 110573. doi: 10.1016/j.chaos.2020.110573 Goldberg, Y. (2016). A

10
primer on neural network models for natural language processing. Journal of Artificial Intelligence
Research, 57, 345–420. doi:10.1613/jair.4992 Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep
learning. Cambridge, MA: MIT Press. Gu, J., Zhang, Y., & Dong, H. (2018). Dynamic behaviors of
interaction solutions of (3 þ 1)-dimensional shallow water wave equation. Computers & Mathematics
with Applications, 76(6), 1408–1419. doi:10.1016/j.camwa. 2018.06.034 Guo, Y., Cao, X., Liu, B., & Gao,
M. (2020). Solving partial differential equations using deep learning and physical constraints. Applied
Sciences, 10(17), 5917. doi:10.3390/ app10175917 Hayati, M., & Karami, B. (2007). Feedforward neural
network for solving partial differential equations. Journal of Applied Sciences, 7(19), 2812–2817.
doi:10.3923/jas.2007. 2812.2817Haykin, S. (1999). Neural networks a comprehensive introduction. He,
J.-H., & Latifizadeh, H. (2020). A general numerical algorithm for nonlinear differential equations by the
variational iteration method. International Journal of Numerical Methods for Heat & Fluid Flow, 30(11),
4797–4810. doi:10.1108/HFF-01-2020-0029Helbing, G., & Ritter, M. (2018). Deep learning for fault
detection in wind turbines. Renewable and Sustainable Energy Reviews, 98, 189–198.
doi:10.1016/j.rser.2018.09. 012 Jagtap, A. D., Kawaguchi, K., & Karniadakis, G. E. (2020). Adaptive
activation functions accelerate convergence in deep and physics-informed neural networks. Journal of
Computational Physics, 404, 109136. doi:10.1016/j.jcp.2019.109136 Jha, R. K., Ugail, H., Haron, H., &
Iglesias, A. (2018).Multiresolution discrete finite difference masks for rapid solution approximation of
the Poisson’s equation. 2018 12th International Conference on Software, Knowledge, Information
Management & Applications (SKIMA) (pp.1–7). Piscataway, NJ: IEEE
doi:10.1109/SKIMA.2018.8631514Karlik, B., & Olgac, A. V. (2011). Performance analysis of various
activation functions in generalized mlp architectures of neural networks. International Journal of
Artificial Intelligence and Expert Systems, 1(4), 111–122.Kazhdan, M., Bolitho, M., & Hoppe, H. (2006).
Poisson surface reconstruction. Proceedings of the fourth Eurographics symposium on Geometry
processing (Vol. 7,p. 0).

11