Scribd
Upload
16 views4 pages

ODE Project B23404

Project

Uploaded by

b23404
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
16 views4 pages

ODE Project B23404

Project

Uploaded by

b23404
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Application of Neural Networks in Solving Second-Order

Differential Equations
Rohith Pranav
Roll Number: B23404
Course: MA-211 ODE
Instructor: Muslim Malik Sir
October 24, 2024

1 Introduction
This report presents a method for solving second-order ordinary differential equations (ODEs) using
Artificial Neural Networks (ANNs). Specifically, we solve the differential equation:

y ′′ (x) + 5y ′ (x) + 6y(x) = 0


subject to the boundary condition y(0) = 1. The neural network is trained to approximate the
solution, and we compare it with the known analytical solution y(x) = e−2x .

2 Approach
Neural networks provide an alternative to traditional numerical methods for solving ODEs. In this
project, the neural network receives x as input and predicts the corresponding value of y(x), such that
the prediction satisfies the ODE.

3 Mathematical Formulation and Model Design


3.1 The Differential Equation
The problem at hand is solving the following second-order differential equation:

y ′′ (x) + 5y ′ (x) + 6y(x) = 0


with boundary condition y(0) = 1.

3.2 Neural Network Architecture


The network is a fully connected feedforward neural network. It consists of:
• An input layer for x,
• Three hidden layers with 50 neurons each and ReLU activation functions,
• An output layer for the predicted y(x).

4 Loss Function
To ensure that the neural network predicts a solution that satisfies the ODE, we define a loss function
with two components:

1
• Residual Loss: Measures how well the predicted y(x) satisfies the ODE at different points in the
domain. The residual is computed as:
N
1 X ′′ 2
Residual Loss = (y (xi ) + 5y ′ (xi ) + 6y(xi ))
N i=1

• Boundary Condition Loss: Enforces the boundary condition y(0) = 1 by adding a term to the
loss function:
Boundary Loss = 1000 · (y(0) − 1)2

The total loss is then given by the sum of the residual loss and the boundary condition loss:

Total Loss = Residual Loss + 1000 · Boundary Loss

5 Training Process
5.1 Data Generation
We generate 100 points, uniformly spaced between x = 0 and x = 1, to train the model. The neural
network is trained to minimize the total loss over these points.

5.2 Training Algorithm


The model is trained using the Adam optimizer with a learning rate of 10−4 . The training runs for
40,000 epochs. TensorFlow’s automatic differentiation is used to compute the necessary derivatives for
the residual loss.

6 Code Implementation
Below is the Python code used to implement the neural network for solving the ODE:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

# True solution for comparison


def true_solution(x):
return np.exp(-2 * x)

# Define derivatives of the neural network’s output


def derivatives(model, x):
with tf.GradientTape(persistent=True) as tape2:
tape2.watch(x)
with tf.GradientTape() as tape1:
tape1.watch(x)
y_pred = model(x)
dy_pred = tape1.gradient(y_pred, x)
d2y_pred = tape2.gradient(dy_pred, x)
return y_pred, dy_pred, d2y_pred

# Build a fully connected neural network


def build_model():
return tf.keras.Sequential([
tf.keras.layers.InputLayer(input_shape=(1,)),
tf.keras.layers.Dense(50, activation=’relu’),
tf.keras.layers.Dense(50, activation=’relu’),
tf.keras.layers.Dense(50, activation=’relu’),
tf.keras.layers.Dense(1)

2
])

# Define the residual based on the ODE


def residual(x, y_pred, dy_pred, d2y_pred):
return d2y_pred + 5 * dy_pred + 6 * y_pred

# Define loss function combining residual and boundary condition


def loss_fn(model, x):
y_pred, dy_pred, d2y_pred = derivatives(model, x)
res = residual(x, y_pred, dy_pred, d2y_pred)
boundary_loss = tf.square(model(tf.convert_to_tensor([[0.0]], dtype=tf.float32)) - 1)
return tf.reduce_mean(tf.square(res)) + 1000 * boundary_loss

# Training step function


def train_step(model, x, optimizer):
with tf.GradientTape() as tape:
loss_value = loss_fn(model, x)
grads = tape.gradient(loss_value, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
return loss_value

# Training function for multiple epochs


def train_model(model, x, epochs=40000):
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001)
for epoch in range(epochs):
loss_value = train_step(model, x, optimizer)
if epoch % 5000 == 0:
print(f’Epoch {epoch}, Loss: {loss_value.numpy()}’)

# Input data
x_train = np.linspace(0, 1, 100).reshape(-1, 1)
x_train_tf = tf.convert_to_tensor(x_train, dtype=tf.float32)

# Build and train the model


model = build_model()
train_model(model, x_train_tf)

# Test and plot results


x_test = np.linspace(0, 1, 100).reshape(-1, 1)
x_test_tf = tf.convert_to_tensor(x_test, dtype=tf.float32)
y_pred, _, _ = derivatives(model, x_test_tf)

plt.plot(x_test, y_pred, label=’Predicted Solution’, color=’blue’)


plt.plot(x_test, true_solution(x_test), ’--’, label=’True Solution’, color=’orange’)
plt.xlabel(’x’)
plt.ylabel(’y(x)’)
plt.legend()
plt.title("Solution of y’’(x) + 5y’(x) + 6y(x) = 0")
plt.grid(True)
plt.show()

7 Results
After training, the neural network’s solution aligns closely with the true solution y(x) = e−2x . Below is
the plot comparing the predicted solution and the analytical solution:

3
Figure 1: Comparison of Predicted and True Solutions for y ′′ (x) + 5y ′ (x) + 6y(x) = 0

8 Conclusion
This project demonstrates how neural networks can be effectively used to solve differential equations.
The network learns to approximate the solution by minimizing the residual of the equation and enforcing
boundary conditions. The approach can be extended to more complex equations and systems, such as
non-linear ODEs or coupled differential equations.
Future work could focus on experimenting with deeper architectures, alternative activation functions,
and extending the approach to solve higher-order or non-linear differential equations.

You might also like