1 Gradient-Adaptive Search Algorithm (CGASA)

The Gradient-Adaptive Search Algorithm (CGASA) is a novel optimization method

designed to tackle the inventory optimization problem in the presence of dynamic power
demand, uncertain cost parameters modeled by intuitionistic fuzzy numbers (IFNs), and
inventory deterioration. CGASA combines the benefits of gradient descent and local
search refinement while dynamically adjusting the balance between exploration and ex-
ploitation during the optimization process.
The key advantage of CGASA lies in its ability to adaptively adjust search parameters
based on the problem’s characteristics, allowing it to search for optimal solutions more
efficiently than traditional optimization methods, such as metaheuristics (e.g., Genetic
Algorithms, PSO).
The algorithm consists of the following core steps:

1. Initialization of the problem variables and fuzzy parameters.

2. Dynamic Gradient Descent to iteratively update inventory cycle times based

on partial gradients.

3. Local Search Refinement to adjust the search space and improve accuracy.

4. Adaptive Exploration-Exploitation Balance to adjust the focus between broad

exploration and fine-tuned exploitation.

5. Exit Condition to stop the algorithm based on convergence or a maximum number

of iterations.

1.1 Step-by-Step Breakdown

Step 1: Initialization[1, 2, 3]
In this step, the algorithm sets up the initial values for the optimization variables and
fuzzy parameters.

• Initialize Inventory Cycle Times (t1 , t2 ): The values of t1 and t2 , which rep-
resent the inventory depletion time and the total cycle time, are initialized based
on historical data or initial estimates. t1 is the time at which the inventory level
reaches zero, and t2 represents the total cycle time after replenishment.

• Set Intuitionistic Fuzzy Numbers (IFNs): The intuitionistic fuzzy numbers for
cost parameters (ordering, holding, shortage, and deterioration) are initialized. The
membership µ(x) and non-membership ν(x) functions define the fuzzy uncertainty
associated with these costs. These values are critical, as they capture the degree of
uncertainty and hesitation in the problem, which will be accounted for during the
optimization.

• Choose Initial Learning Rates (η1 , η2 ): The adaptive learning rates for the
gradient descent are initialized. These rates will evolve during the optimization
based on the gradient’s magnitude and the search progress.

1
Step 2: Dynamic Gradient Descent[4, 5, 6]
The primary optimization technique in CGASA is gradient descent, which is used to
iteratively adjust the inventory cycle times. This method computes the gradients of the
Total Average Cost (TAC) with respect to t1 and t2 and uses them to update the variables
in the direction that reduces TAC.
• Compute Partial Derivatives: The first step in gradient descent is to compute
the partial derivatives of the Total Average Cost (TAC) function with respect to t1
and t2 :
∂TAC ∂TAC
, . (1)
∂t1 ∂t2
These derivatives represent the rate of change of the cost function concerning each
inventory cycle time, providing valuable information on how to adjust the cycle
times to minimize the cost.
• Update t1 and t2 : Using the gradients, the algorithm updates the values of t1 and
t2 with the following formulas:
(k+1) (k) (k) ∂TAC (k+1) (k) (k) ∂TAC
t1 = t1 − η1 , t2 = t2 − η2 .
∂t1 ∂t2
Here, η1 and η2 are the adaptive learning rates for t1 and t2 , respectively. These
rates adjust based on the magnitude of the gradient, ensuring that the updates are
neither too small nor too large.

• Update Decision Variables:

Using the computed gradients, update t1 and t2 iteratively:
(k+1) (k) ∂T AC (k)
t1 = t1 − η1 · , (2)
∂t1
(k+1) (k) (k) ∂T AC
t2 = t2 − η2 · . (3)
∂t2

• Adapt Learning Rates: If the gradient is steep (large change in TAC), the
learning rates η1 and η2 can be increased to make larger updates. Conversely, if the
gradient is shallow, the learning rates are decreased to make finer adjustments.

Fine-Tuning Decision Variables[10,11,12]

Introduction of Perturbations: After applying gradient descent, refine the solutions
by introducing small perturbations δ1 and δ2 :
(k+1) (k+1) (k+1) (k+1)
t1 = t1 + δ1 , t2 = t2 + δ2 . (4)

Definition of δ1 and δ2 :
δ1 = λ1 · rand(−∆1 , ∆1 ), δ2 = λ2 · rand(−∆2 , ∆2 ),
where: - rand(−∆, ∆): Generates a random value within the range [−∆, ∆]. - λ1 , λ2 :
Scaling coefficients controlling the magnitude of the perturbation. - ∆1 , ∆2 : Exploration
ranges specific to the problem constraints.

2
Definition
∆ defines the maximum range within which random perturbations are generated. The
random value will lie between −∆ and ∆.

Factors to Consider
Problem Scale
• For larger-scale problems (e.g., where t1 , t2 are in the range of hundreds), ∆ should
be larger (e.g., 5–10).

• For small-scale problems (e.g., t1 , t2 between 0–10), ∆ should be smaller (e.g.,

0.1–1).

Optimization Phase
• Early Stage (Exploration): Use a larger ∆ to explore a wider search space.

• Later Stage (Exploitation): Gradually decrease ∆ to focus on fine-tuning around

the current solution.

Common Strategy
• Start with ∆1 = ∆2 = 0.5 to 1 (for small to moderate problems).

• Apply adaptive adjustment:

∆new = γ · ∆old
where γ is a decay factor (e.g., 0.9) to reduce ∆ over iterations.

Adaptive Adjustment of Perturbation Parameters

Dynamic Adjustment: Update ∆1 and ∆2 dynamically based on solution improve-
ment:
∆new
1 = ∆1 · γadjust , ∆new
2 = ∆2 · γadjust ,
where γadjust is an adjustment factor that:
- Increases exploration if the improvement rate is low.
- Decreases exploration as solutions converge.

Step 4: Exploration-Exploitation Adjustment[13, 14, 15]

Exploration-Exploitation Coefficient
Compute the exploration-exploitation coefficient θk :

∂T
\ AC i ∂T
\ AC i
∂t1
+ ∂t2
θk = . (5)
∥∆t1 ∥ + ∥∆t2 ∥

3
Numerator: Gradient Norms
The gradient norms for t1 and t2 are computed as:
!2
∂ T[AC i ∂ T[AC i
= ,
∂t1 ∂t1
!2
∂ T[AC i ∂ T[AC i
= .
∂t2 ∂t2

Denominator: Step Sizes

The step sizes for t1 and t2 are calculated as:
(k+1) (k)
∥∆t1 ∥ = t1 − t1 ,

(k+1) (k)
∥∆t2 ∥ = t2 − t2 .

Exploration-Exploitation Coefficient θk
Combine the gradient norms and step sizes to compute θk :
 2  2
∂T
\ AC i ∂T
\ AC i
∂t1
+ ∂t2
θk = .
(k+1) (k) (k+1) (k)
t1 − t1 + t2 − t2

Adjust Learning Rates:[7, 8, 9]

(k+1) (k+1)
Update Learning Rates: Adjust η1 and η2 based on the progress:
(
ηi (k) · γincrease , if ∥∇T AC(k)∥ > ∥∇T AC(k − 1)∥,
ηi (k + 1) = (6)
ηi (k) · γdecrease , otherwise.

Set γincrease = 1.1 and γdecrease = 0.9.

Adaptive Perturbation Adjustment

Dynamically adjust the perturbation ranges ∆1 and ∆2 for future iterations:

∆new
1 = ∆1 · γadjust , ∆new
2 = ∆2 · γadjust .

Step 5: Convergence and Final Output[16, 17, 18]

Convergence Check
Stop the iteration process when the following condition is satisfied:
(k+1) (k)
T ACi − T ACi < ϵ. (7)

4
In cases where the convergence condition is not met, the algorithm proceeds with adaptive
strategies in the next iteration. This includes dynamically adjusting learning rates, in-
creasing perturbation ranges to enhance exploration, and reinitializing decision variables
if stagnation persists. Additionally, hybrid local search techniques may be employed to
refine solutions. This adaptive mechanism ensures robustness and prevents premature
convergence to suboptimal solutions.

References
[1] Atanassov, K. (1986). Intuitionistic Fuzzy Sets. Fuzzy Sets and Systems, 20(1),
87–96. https://doi.org/10.1016/0165-0114(86)90034-3

[2] Dubois, D., & Prade, H. (1980). Fuzzy Sets and Systems: Theory and Applications.
Academic Press.

[3] Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, 8(3), 338–353. https:
//doi.org/10.1016/S0019-9958(65)90241-X

[4] Cauchy, A. (1847). Méthode générale pour la résolution des systèmes d’équations
simultanées. Comptes Rendus de l’Académie des Sciences, 25, 536–538.

[5] Bottou, L. (2010). Large-Scale Machine Learning with Stochastic Gradient Descent.
In Proceedings of COMPSTAT’2010 (pp. 177–186). Springer. https://doi.org/10.
1007/978-3-7908-2604-3_16

[6] Ruder, S. (2016). An Overview of Gradient Descent Optimization Algorithms. arXiv

preprint arXiv:1609.04747. https://arxiv.org/abs/1609.04747

[7] Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive Subgradient Methods for Online
Learning and Stochastic Optimization. Journal of Machine Learning Research, 12,
2121–2159. http://jmlr.org/papers/v12/duchi11a.html

[8] Kingma, D. P., & Ba, J. (2014). Adam: A Method for Stochastic Optimization.
arXiv preprint arXiv:1412.6980. https://arxiv.org/abs/1412.6980

[9] Zeiler, M. D. (2012). ADADELTA: An Adaptive Learning Rate Method. arXiv

preprint arXiv:1212.5701. https://arxiv.org/abs/1212.5701

[10] Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by Simulated
Annealing. Science, 220(4598), 671–680. https://doi.org/10.1126/science.220.
4598.671

[11] Glover, F. (1986). Future Paths for Integer Programming and Links to Artificial
Intelligence. Computers Operations Research, 13(5), 533–549. https://doi.org/
10.1016/0305-0548(86)90048-1

[12] Hansen, P., & Mladenović, N. (2001). Variable Neighborhood Search: Principles and
Applications. European Journal of Operational Research, 130(3), 449–467. https:
//doi.org/10.1016/S0377-2217(00)00147-7

[13] Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd
ed.). MIT Press. http://incompleteideas.net/book/the-book-2nd.html

5
[14] Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-Time Analysis of the Multi-
Armed Bandit Problem. Machine Learning, 47(2-3), 235–256. https://doi.org/
10.1023/A:1013689704352

[15] March, J. G. (1991). Exploration and Exploitation in Organizational Learning. Or-

ganization Science, 2(1), 71–87. https://doi.org/10.1287/orsc.2.1.71

[16] Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Cambridge University
Press. https://web.stanford.edu/~boyd/cvxbook/

[17] Nesterov, Y. (2004). Introductory Lectures on Convex Optimization: A Basic Course.

Springer. https://doi.org/10.1007/978-1-4419-8853-9

[18] Polyak, B. T. (1964). Some Methods of Speeding Up the Convergence of Iteration

Methods. USSR Computational Mathematics and Mathematical Physics, 4(5), 1–17.
https://doi.org/10.1016/0041-5553(64)90137-5

6
Summary of Newly Invented Sections in CGASA

Section What’s New? Impact

Adaptive Dynamic adjust- Prevents prema-
Exploration- ment using θk ture convergence
Exploitation
Balance
Dynamic Per- Adaptive per-
Escapes local
turbation Mech- turbation ranges
optima, en-
anism ∆1 , ∆2 hances solution
diversity
Gradient- Self-adjusting Faster con-
Adaptive Learn- learning rates vergence with
ing Rate based on gradi- improved stabil-
ent feedback ity
Non- Reinitialization Ensures robust-
Convergence and hybrid ness in challeng-
Handling Strat- search upon ing optimization
egy stagnation
Hybrid Gradi- Combining gra- Achieves both
ent and Local dient descent speed and accu-
Search with local refine- racy
ment

Table 1: Summary of Newly Invented Sections in CGASA