List of Optimization Problems in AI

B.T. Kien*

1 Introduction
Optimization problems are fundamental in artificial intelligence (AI) because they help improve
performance in various tasks, from training machine learning models to planning and decision-
making. Based on ChatGPT, we list some key optimization problems in AI and their contexts.

2 List of problems
2.1 Training Neural Networks (Deep Learning)
In deep learning, the goal is to minimize a loss function that quantifies how far a neural network’s
predictions are from the true values. This problem is often solved using Stochastic Gradient
Descent (SGD) and its variants.
Optimization Problem:

min L(f (X, θ), Y ), (2.1)

θ

where
θ are the parameters of the neural network (weights, biases),
f (θ, X) is the neural network with inputs
Y is the true label,
L is the loss function, such as mean squared error (MSE) or cross-entropy.
Challenges:
-Non-convexity: Neural networks often have non-convex loss landscapes, which means there
are many local minima.
-High dimensionality: The number of parameters (weights and biases) can be in the millions
for modern networks.

2.2 Reinforcement Learning (RL)

In reinforcement learning, an agent learns to make decisions by interacting with an environment.
The agent seeks to maximize cumulative rewards over time. The optimization problem in RL
involves learning a policy that maximizes the expected reward.
*
Department of Optimization and Control Theory, Institute of Mathematics, Vietnam Academy of Science
and Technology,18 Hoang Quoc Viet road, Hanoi, Vietnam; email: [email protected]

1
Optimization Problem:

T
hX i
max Eπ γ t rt (st , at ) , (2.2)
π
t=0

where
π is policy
st , at are the state and action at time t
rt (st , at ) is is the reward received at time t
γ ∈ [0, 1] is a discount factor that weighs future rewards.
Chalanges:
-Exploration vs. Exploitation: Balancing the exploration of new strategies and exploiting
known good strategies
- High-dimensional state/action spaces: Environments like robotic control or games (e.g.,
Go) have massive state/action spaces.

2.3 Support Vector Machines (SVM)

Support Vector Machines are a type of supervised learning algorithm used for classification. The
goal is to find a hyperplane that best separates the classes in the feature space while maximizing
the margin between the classes.
Optimization Problem: For a linearly separable case, the SVM optimization problem is:

1
min ∥W ∥2 s.t. yi (W · Xi + b) ≥ 0 i = 1, 2, .., n, (2.3)
W,b 2

where
W is the weight vector (defining the hyperplane),
b is the bias term,
Xi is training example and yi is its label, yi ∈ {−1, 1}
Chalenges:
-Non-linearity: When data is not linearly separable, kernel functions (e.g., RBF) are used,
leading to a more complex optimization.
-Scalability: For very large datasets, solving the quadratic programming problem can become
computationally expensive.

2.4 Generative Adversarial Networks (GANs)

In GANs, two networks (the generator and discriminator) compete against each other. The
generator creates synthetic data, and the discriminator evaluates the quality of the synthetic
data by comparing it to real data. The goal is for the generator to improve its ability to generate
realistic data, while the discriminator tries to better distinguish between real and fake data.
Optimization Problem (Minimax Game):

2
    
min max Ex∼pdata log D(x) + Ez∼pz log(1 − D(G(z))) , (2.4)
G D

where
D(x) is the discriminator’s output for real data,
G(z) is the generator’s output for random noise z
pdata is the real data distribution
pz is is the noise distribution.
Challenges:
-Mode collapse: The generator may collapse to generating only a few types of data points
(or even a single point).
- Training instability: GAN training is often unstable and requires careful balancing between
the generator and discriminator.

2.5 Constrained Optimization for AI Planning

In AI planning and robotics, optimization problems often involve constraints on the actions that
can be taken. These constraints might represent physical limitations (e.g., robot joint limits) or
task requirements.
Optimization Problem:

min f (x) s.t. gi (x) ≤ 0 i = 1, 2, ..., m, (2.5)

x∈X

where
x is represents a sequence of actions or control inputs,
f (x) is a cost function, such as minimizing energy consumption or time,
gi (x) are constraints representing system dynamics, safety, or feasibility.
Challenges:
- Nonlinear constraints: Often, the constraints are nonlinear, leading to a complex optimiza-
tion problem.
-Real-time constraints: Optimization has to be solved quickly in real-time applications like
autonomous driving or robotic control.

2.6 Optimization in Natural Language Processing (NLP)

In NLP, optimization is used to train models that understand and generate language. For
example, transformer models used in machine translation, text generation, or sentiment analysis
are trained by minimizing a loss function over a sequence of words.
Optimization Problem:

N
1 X
min L(yi , f (xi , θ)), (2.6)
θ N
i=1

3
where
xi is the input text,
yi is the target output (e.g., the translated sentence),
f (xi , θ) is the model (e.g., a neural network like BERT or GPT),
L is the loss function (e.g., cross-entropy loss for classification tasks).
Challenges:
-Sequence-to-sequence modeling: Optimizing models that generate sequences (e.g., transla-
tions, dialogue) is complex because of the dependencies between words in the sequence.
-Large-scale datasets: NLP models often require huge amounts of data and computational
power to optimize.

2.7 Hyperparameter Optimization

In machine learning, the performance of a model can depend significantly on its hyperparameters
(e.g., learning rate, batch size, number of layers in a neural network). Finding the optimal set
of hyperparameters is an optimization problem in itself.
Optimization Problem:

min E[Validation Error(λ)], (2.7)

λ∈Λ

where
λ represents the hyperparameters (e.g., learning rate, number of layers),
Λ is the hyperparameter search space, The validation error is the error on a held-out dataset.
Method:
-Grid search: Evaluate all combinations of hyperparameters in a predefined grid.
-Random search: Randomly sample hyperparameters from the search space.
-Bayesian optimization: Build a probabilistic model of the objective function and use it to
select the most promising hyperparameters to evaluate next.
Challenges:
-High-dimensional search space: The number of hyperparameters can be large, leading to a
high-dimensional optimization problem.
-Computational cost: Each evaluation of the objective function (e.g., training a model) can
be computationally expensive.

2.8 Bayesian Optimization for Expensive Function Evaluations

In AI applications where function evaluations are expensive (e.g., hyperparameter tuning, tun-
ing the architecture of neural networks), Bayesian optimization is used to find the optimal
parameters with fewer evaluations.
Optimization Problem:

min f (x), (2.8)

x∈X

4
 where
f (x) is expensive to evaluate (e.g., training a neural network),
A probabilistic model (e.g., a Gaussian process) is built to approximate f (x) and the optimiza-
tion algorithm iteratively updates this model to find the optimal x
Challenges:
-Exploration vs. exploitation: Balancing exploration of unknown regions of the search space
with exploitation of regions that seem promising.
-Scalability: Bayesian optimization typically does not scale well to high-dimensional spaces.

3 Conclusion
Optimization is at the heart of many AI problems, from training machine learning models to
solving real-time control problems in robotics. Techniques like stochastic gradient descent, rein-
forcement learning, and Bayesian optimization play a critical role in solving these optimization
problems. However, the challenges of non-convexity, high-dimensionality, and computational
cost make optimization in AI a complex and fascinating field of study.