This article has been accepted for publication in IEEE Access.

This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2023.0322000

Optimal control method for congestion control

and delay reduction in deterministic networks
FIRST Q. GAO1 , SECOND J. LI1 , THIRD Y. LIU1 , and FOURTH Y. XING1
1
6G Research Center, China Telecom Research Institute, Beijing, China (e-mail: [email protected])
Corresponding author: First Q. GAO (e-mail: [email protected]).
This work was supported by the 2020 National Key R&D Program "Broadband Communication and New Network" special "6G Network
Architecture and Key Technologies" 2020YFB1806700.

ABSTRACT The proliferation of the types and number of the beyond fifth generation / the six generation
(B5G/6G) network services will contribute to severe network congestion collapse, which is detrimental to
deterministic service assurance for time-sensitive businesses. Faced with the congestion control problem in
deterministic networks, the regulation and control effect of network macroscopic congestion state on the node
various microscopic control strategies should be sufficiently considered from a global view. In this paper,
a congestion control system model is constructed, which includes node-level congestion states and control
strategies and network-level optimization objective. Continuous congestion state transition and real-time
control strategy selection are analyzed to minimize the network delay and congestion degree with the lowest
control cost. The analytical solution to optimal control strategy is derived based on the optimal control theory,
and the numerical solution is analyzed by proposing a congestion control discretization algorithm (CCDA).
Simulation results demonstrate that the congestion degree of each network node gradually decreases with the
obtained CCDA strategies. The impacts of model parameters such as congestion deterioration probability
and delay weight coefficients on the evolution of network total loss and optimal strategies are also analyzed.
Moreover, the network total loss of the proposed congestion control model is lower than the baseline
model, which can provide effective and referable theoretical guidance for congestion control problem in
deterministic networks.

INDEX TERMS Congestion control, deterministic network, optimal control theory.

I. INTRODUCTION which urgently calls for a novel deterministic delivery mode

provided for time-sensitive services.
T HE continuously evolving fifth generation (5G) network
and the future six generation (6G) network will provide
more ubiquitous connectivity, more extreme performance ex-
As a consequence, the Internet Engineering Task Force
(IETF) DetNet working group proposes the deterministic
perience, and more customized services for "to customer" network [4] to achieve end-to-end routing data flow on the
(2C) and "to business" (2B) fields. At the same time, the basis of IEEE 802.1 bridged networks and to expand the
explosion of service traffic leads to a large number of net- network scale of deterministic assurance. Typical determin-
work congestion and packet delay, which is not conducive istic network technologies mainly include Flexible Ethernet
to the quality of service (QoS) guarantee of time-sensitive (FlexE) [5], [6] for Layer-2 Ethernet, Time-sensitive Net-
services. In particular, industrial control, automatic driving working (TSN) [7] for local area network (LAN), Determinis-
and other vertical industry scenarios that have high real-time tic Internet Protocol (DIP) [8], [9] for Layer-3 large-scale net-
and security demands require ultra-low millisecond delay, works, Deterministic Networking (DetNet) [10] for wide area
microsecond jitter and more than 99.999% reliability [1]. network (WAN), and Deterministic Mobile Edge Computing
High-definition video surveillance scenarios have high re- (D-MEC) [11] for MEC networks, etc. Although applicable
quirements for video resolution and size, and need stable to different networks, these technologies are commonly dedi-
transmission with large bandwidth [2]. However, the tradi- cated to providing accurate, bounded network performance.
tional best-effort (BE) delivery mechanism can no longer sat- Service traffic aggregation and congestion collapse can be
isfy the punctual and accurate delivery demands of the future mitigated and time-critical services QoS can be deterministi-
services due to the long-tail effect of end-to-end delay [3], cally guaranteed by various deterministic mechanisms such as

VOLUME 11, 2023 1

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

resource reservation, multipath redundancy, and explicit path • To alleviate the network congestion and better guarantee
planning. the determinacy of the future network, a congestion
Existing work on deterministic networks mainly focuses on control system model is constructed in the deterministic
the technologies or mechanisms to reduce the network end- network scenario. The node-level congestion state evo-
to-end delay, increase the number of schedulable flows and lution, real-time control strategy selection and network-
mitigate network congestion, including the traditional queue level optimization objective are sufficiently considered.
scheduling mechanisms and the novel queue deterministic en- • The transition of congestion state and the optimal con-
hancement to improve the network determinacy. For example, gestion control strategy are analyzed based on the op-
at the queue input ports for enqueue congestion control, the timal control theory. In addition, a CCDA algorithm is
active queue management (AQM) actively discards packets proposed to jointly minimize the network end-to-end
that exceed the queue length threshold [12], whereas the delay and congestion degree with the lowest control cost.
explicit congestion notification (ECN) notifies the upstream • Simulation results demonstrate the CCDA strategies can
nodes to reduce the sending rate to mitigate traffic aggrega- reduce the congestion degree of network nodes and the
tion [13]. At the queue output ports for rate control, traffic network total loss. In addition, the impacts of model
flows are shaped based on different criterions such as token, parameters e.g., congestion deterioration probability and
credit value, gate control list (GCL), and priority [14]. Among delay weight coefficients are also analyzed.
multihop nodes for packet forwarding, the path and delay are The remainder of this paper is organized as follows. Re-
optimized deterministically through slot mapping based on lated work is provided in Section II. The congestion control
time synchronization and cycle mapping based on frequency system model in deterministic networks and optimization
synchronization [15]–[18]. problem formulation are provided in Section III. The optimal
However, existing work mainly concerns the influence of a control theory based solution and the CCDA algorithm are
certain technology on the partial-network performance opti- proposed to obtain the optimal congestion control strategies
mization from the microscopic technical level, and lacks the in Section IV. Simulation results are provided and discussed
overall-network multiple strategy decision-making from the in Section V. Finally, Section VI summarizes this paper.
macroscopic regulation level. A thorough solution to network
congestion could be achieved only with the cooperation of II. RELATED WORK
the various control strategies, which is inseparable from the Reducing network delay and congestion to improve the net-
macroscopic level regulation and control. Especially in the work determinacy has been studied in recent decades, and
case of a large number of congested nodes, the decision ig- we roughly divide them into the following four categories:
noring the overall-network real-time states may aggravate the enqueue congestion control, queue scheduling mechanism,
burden of the whole network, e.g., extra delay and resource dequeue shaping mechanism, multihop forwarding mecha-
waste. In addition, the node types and traffic flow types will nism. The research gap analysis is provided in Subsection II-
also affect the node strategy selection in deterministic net- E.
works where ordinary traffic flows and deterministic traffic
flows coexist. But the existing work mainly concentrates on A. ENQUEUE CONGESTION CONTROL
the QoS enhancement of deterministic services accompanied As a typical congestion control scheme to reduce the buffer
by relatively high cost consumption. Appropriately relaxing occupation and end-to-end delay, AQM drops the subsequent
the deterministic-applicable mechanisms for ordinary traffic packets when the packet length exceeds the queue length
flows may reserve more available resources for the network threshold. Based on this, an information compression model
especially deterministic traffic flows. This trade-off between for AQM design is proposed to deal with the heterogeneous
network performance and cost budget is definitely a challenge RTT and time-varying traffic load in [19], achieving low
for current research on deterministic network. latency and high scalability. To deal with head of line (HOL)
To solve the above problems, a real-time congestion control blocking, regional explicit congestion notification (RECN)
system model in deterministic network is constructed in this for source deterministic routing networks and flow-based
paper. The real-time prediction and macroscopic regulation implicit congestion management (FBICM) designed for dis-
of congestion control strategies according to the different tributed deterministic routing networks are proposed in [20].
states of nodes are carried out. The network end-to-end delay A deterministic latency congestion control system is provided
and congestion degree are jointly minimized with the lowest in [21] that integrates rate control, multiqueue management,
control strategy cost while taking distinguished node states and per-hop delay correction mechanisms, reducing the delay
and traffic flows into account. Simulation results are provided jitter by 90%. A two-timescale congestion window control
to demonstrate the evolution of network congestion degree mechanism is proposed in [22], considering distinguished
with node control strategies. The network total loss is reduced delay-aware priorities of flows and enqueue delay to reduce
compared with the baseline model, proving the effectiveness the average service delay compared with TCP Vegas. [23]
of the proposed congestion control model. proposes an online learning Crimson to continuously perceive
The main contributions of this paper are summarized as and adaptively satisfy the deterministic application QoS. Con-
follows. sistent performance can be achieved under diverse network
2 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

scenarios. The above schemes mainly implement congestion (send in time slot x, reach the next hop in x and send in
control in the enqueue process by means of intra-node oper- x + 1) which requires propagation delay less than T and is
ation or inter-node collaboration. suitable for LAN. To deal with the shortcomings of CQF,
CQF-3 introduces a third queue for caching to avoid frame
B. QUEUE SCHEDULING MECHANISM arrival slot errors. And to release the time synchronization
According to the granularity of queue scheduling, it can requirement of CQF, scalable deterministic forwarding (SDF)
be classified into: per-port scheduling, per-user scheduling, adds the explicit period identifier to the datagram [25]. How-
per-service-class scheduling, per-queue scheduling, per-flow ever, the above forwarding mechanisms are limited to LAN.
scheduling and per-packet scheduling, from coarse-grained to For WAN, deterministic internet protocol (DIP) requires fre-
fine-grained. Especially in TSN and DetNet, per-service-class quency synchronization with x → y → y + 1 [34], whereas
scheduling and per-flow scheduling are mainly deployed. For cyclic specified queue forwarding (CSQF) requires frequency
per-service-class scheduling, the priority is allocated based synchronization with x → y → y + n [35]. Multi-CQF,
on the service class, and reflected in the Ethernet Priority an extension of CSQF is evaluated in [36], the simulated
Code Point (PCP). However, the flows with the same PCP will annealing solution is developed and the latency is lower than
easily result in congestion. For per-flow scheduling, burst can CSQF.
be managed stream by stream and fine-grained QoS services
can be provided. The cost and scalability will get worse with
E. RESEARCH GAP ANALYSIS
the network expansion. Therefore, classified stream by stream
hybrid queue is proposed to integrate the advantages of per- Existing work mainly focuses on the low-level microscopic
service-class scheduling and per-flow scheduling [24], [25]. technologies to improve the network deterministic perfor-
mance. For example, proposing scheduling and forwarding
C. DEQUEUE SHAPING MECHANISM mechanisms to solve the congestion control problem, reduc-
Typical dequeue shaping mechanisms such as token bucket, ing the cost and complexity of congestion control from the
credit-based shaping (CBS) of IEEE 802.1Qav, time-aware hardware perspective, reducing the delay and jitter from the
shaping (TAS) of IEEE 802.1Qbv, asynchronous traffic shap- algorithm perspective, etc. Besides, existing work mainly
ing (ATS) of IEEE 802.1Qcr, frame preemption (FP) of IEEE concentrates on the enhancement of deterministic services
802.1Qbu can provide delay guarantee with different gran- accompanied by relatively high cost consumption. However,
ularity. For token bucket [26], tokens are put in at a certain a thorough solution to network congestion control could be
rate, packets with tokens can be sent whereas packets without achieved only with the appropriate cooperation and joint
tokens will be dropped or stored. It guarantees determinacy optimization of the various control strategies such as packet
in terms of bandwidth but has poor latency performance. dropping in AQM, rate limitation notification in ECN, packet
CBS [27] provides services with different priorities, and traf- queuing in CQF, etc. At the same time, it is required to take
fic flows are sent based on the constantly changing credit both deterministic services and important nondeterministic
value. TAS [28] uses GCL to control the periodic packet services into account, and balance their cost and performance.
transmission at the queue exit, which can realize microsecond In summary, the research for theoretical modeling and
per-hop-per-packet scheduling but needs strict time synchro- upper-level regulation is generally lacked, where node-level
nization. An enhanced TAS is proposed in [29] to guaran- congestion states and control strategies and network-level
tee determinacy for aperiodic key traffic neglected by TAS. optimization objective should be studied. Therefore, it is
ATS [30] relaxes time synchronization requirement and uses necessary to model the interplay between macroscopic net-
urgency-based scheduler. Emergency traffic can be prioritized work congestion states and microscopic congestion control
and ATS still has high bandwidth utilization when periodic decision-making to assist the network to predict the conges-
and sporadic traffic coexist. FP [31] allows high-priority tion situation and adjust the strategies in time. In addition,
eMAC traffic to interrupt low-priority pMAC traffic, based it has important theoretical guiding significance for the pre-
on which a frame truncation assisted FP is proposed in [32]. vention and mitigation of network congestion to consider the
By discarding the least important bytes, this enhanced-FP congestion control problem from the global perspective.
can transmit more payload bytes of image and video without
increasing delay of higher-priority services.
III. CONGESTION CONTROL SYSTEM MODEL IN
DETERMINISTIC NETWORKS
D. MULTIHOP FORWARDING MECHANISM
Queue forwarding mechanisms generally place a gate at the As illustrated in Fig. 1, the congestion control system model
entry and exit of the queue, respectively. The time is divided in the deterministic network includes two types of nodes, the
into equal slots T and periodic mapping between multiple ordinary nodes and the deterministic nodes. The type of node
hops needs to be maintained. The enqueuing and dequeuing i ∈ [1, N ] is denoted by vi (t) .
of packets are cyclic/rotated. Jitter can generally be controlled
within 2T . Cyclic queue forwarding (CQF) [33] requires strict

1, deterministic node,
time synchronization. The slot mapping is x → x → x + 1 vi (t) = (1)
0, ordinary node.
VOLUME 11, 2023 3

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

B. TRANSITION OF NODE CONGESTION STATE

BE flows and TS flows coexist in the deterministic network.
When congested, network nodes perform path replanning for
TS flows and dropping/path replanning for BE flows.
For congested node i, direct dropping may be performed
after packet arrival or path replanning may be performed be-
fore packet arrival. Note that due to explicit path planning and
resource reservation in deterministic networks, we assume
that when congestion occurs, there must be reserved paths
and resources for the forwarding of deterministic flows. The
Deterministic node Ordinary node
congestion control strategies for congested nodes are defined
Congested deterministic node Congested ordinary node as

TS flow
• Dropping rate δji (t). When node j is free and node i is
BE flow
congested, node i can drop the BE packets received from
Path replanning Dropping node j, so as to alleviate the traffic aggregation at node i.
After dropping, the packet sender will resend the packets
FIGURE 1. Congestion control system model in deterministic networks. at intervals, resulting in large waiting and resending
delay. However, the extra control cost is small or even
negligible. eji (t) = 0, δji (t) = 0; Cj (t) = 1, δji (t) = 0;
The traffic flows can be transmitted from node i to node j only Ci (t) = 0, δji (t) = 0 and vi (t) vj (t) pji (t) = 1,
when the path between them is reachable, denoted by eij (t). δji (t) = 0.

1, reachable, • Path replanning rate γji (t). When node j is free and
eij (t) = (2) node i is congested, the path of TS/BE packets sent from
0, unreachable.
node j to node i can be replanned to alleviate the traffic
The types of traffic flows include BE flows and time-sensitive aggregation at node i. The delay caused by this behavior
(TS) flows, where the traffic flow from node i to node j is is less than that of direct dropping, because the packets
denoted by pij (t). only need to be replanned from an intermediate node

1, TS flow, rather than resent from the source node. However, re-
pij (t) = (3) planning paths and calculating routes will consume large
0, BE flow.
extra control cost. eji (t) = 0, γji (t) = 0; Cj (t) = 1,
The ordinary nodes can only handle BE flows, whereas the γji (t) = 0 and Ci (t) = 0, γji (t) = 0.
deterministic nodes can handle both TS flows and BE flows.
The node congestion states are influenced by the traffic arrival For uncongested node i: Assuming that the packet arrival
rate, the node congestion control strategies and the congestion follows a Poisson distribution with parameter λp , then the
states of other nodes. Each node adjusts the packet processing congestion deterioration probability λi of node i is defined
measures (e.g., path replanning, packet dropping) to alleviate as the probability of sufficient packet arrival, where
the network congestion and to ensure the lowest user plane
λp
delay and control plane cost. X λkp
λi = Pr {X > λp } = 1 − e−λp , (5)
k=0
k!
A. NODE CONGESTION STATE
Two node congestion states are defined in the proposed sys- is taken as an example. It can be easily concluded through
tem model, congested state denoted by J and free (uncon- calculation that, as the packet arrival mean λp increases, the
gested) state denoted by F. congestion deterioration probability λi increases, which is
• J , the forwarding capacity and buffer space of the nodes consistent with the actual scenario.
in this state are severely limited, (80% of the incoming
packets can not be processed, just as an example.) The
packets arriving at this kind of nodes will experience
high delay.
• F, the nodes in this state can process the received packets
normally.
Let the variable Ci (t) denote the congestion state of node i FIGURE 2. Congestion state transition of network nodes.
at time t, the congestion probability of which is denoted by
ci (t).
Therefore, the evolution of the congestion probability of

1, congested state Ji (t) ,
Ci (t) = (4) node i over time, i.e., the congestion state differential equa-
0, uncongested state Fi (t) .
4 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

tion, can be expressed as (6). The congestion state transition The total objective loss G for the entire network, i.e., network
of network nodes is shown in Fig. 2. total loss, is the integral of the instantaneous objective loss
N over the time period [t0 , tf ].
· X
ci (t) = [1 − ci (t)] λi − ci (t) {δji (t) [1− Ztf Ztf X
N
j=1,j6=i
(6)
G= g (t) dt = [li (t) + fi (t)]dt. (12)
vj (t) vi (t) pji (t)] +γji (t)} eji (t) [1 − cj (t)] . i=1
t0 t0

C. OPTIMIZATION PROBLEM FORMULATION Network nodes adjust their congestion control strategies to
The optimization objective of the system is to minimize the minimize the network total loss G. Therefore, the optimiza-
additional delay and to alleviate the network congestion de- tion problem can be expressed as P1 :
gree with the lowest control cost.
The additional delay includes additional queuing delay P1 : min G, (13a)
{δji (t)},{γji (t)}
caused by network congestion, additional sending and prop- s.t. oji (t) 6 δji (t) + γji (t) , (13b)
agation delay after being dropped, and additional processing
delay for path replanning. Define li (t) as the additional delay ci (t) ∈ [0, 1] , (13c)
for node i at time t, which can be expressed as d1 > d2 > 0, (13d)
N
X 0 < b1 < b2 , (13e)
li (t) = aci (t) + d1 δji (t) Vji (t)Eji (t) ci (t) δji (t) > 0, (13f)
j=1,j6=i
(7) γji (t) > 0. (13g)
N
X
+d2 γji (t) Eji (t)ci (t) . (13b) indicates that the sum of the two strategies cannot be
j=1,j6=i lower than the amount of overflowed packets received from
For ease of expression, we define node j at node i. Since the dropping delay is larger than
the path replanning delay, the delay weight coefficient for
Vji (t) = 1 − vj (t) vi (t) pji (t) , (8) dropping d1 is larger than that of path replanning d2 as (13d).
Similarly, since the dropping cost is smaller than the path
and
replanning cost, the cost coefficient for dropping b1 is smaller
Eji (t) = eji (t) [1 − cj (t)] . (9)
than that of path replanning b2 as (13e).
a, d1 and d2 are the delay weight coefficients used to measure
the delay caused by congestion, dropping, and path replan- IV. OPTIMAL CONTROL THEORY BASED SOLUTION AND
ning, respectively. CCDA ALGORITHM
The control cost includes dropping cost and path replan- A. OPTIMAL CONTROL THEORY BASED SOLUTION
ning cost, proportional to the quadratic form of the congestion Optimal control theory seeks a control strategy that optimizes
control strategies. Define fi (t) as the control cost for node i the given system performance, subject to given constraints.
at time t, which can be expressed as Especially, Hamilton function method is widely used, which
N transforms the optimization of the system optimization ob-
b1 X 2 jective to the optimization of the constructed Hamilton func-
fi (t) = δji (t) Vji (t) Eji (t) ci (t)
2 tion [37]. Therefore, the Hamilton function with respect to
j=1,j6=i
N
(10) the instantaneous objective loss (11) and the congestion state
b2 X
+ γji2 (t) Eji (t)ci (t) , differential equation (6) is constructed by introducing the
2 costate variable µi (t) for each node i.
j=1,j6=i

where b1 and b2 are the cost coefficients for implementing N

X ·
N
X
dropping and path replanning strategies, respectively. H (t) = g (t) + µi (t) ci (t) = ci (t) [a
Therefore, the instantaneous objective loss g (t) at time t i=1 i=1
N N
for the entire network is defined as X X
N N
+d1 δji (t) Vji (t)Eji (t) + d2 γji (t) Eji (t)
X X j=1,j6=i j=1,j6=i
g (t) = [li (t) + fi (t)] = ci (t) [a
N N
i=1 i=1 b1 X 2 b2 X 2
N N + δji (t) Vji (t) Eji (t) + γji (t) Eji (t)]
X X 2 2
+d1 δji (t) Vji (t)Eji (t) + d2 γji (t) Eji (t) j=1,j6=i j=1,j6=i
N N
j=1,j6=i j=1,j6=i X X
N N + µi (t) {[1 − ci (t)] λi − ci (t) [δji (t) Vji (t)
b1 X 2 b2 X 2 i=1 j=1,j6=i
+ δji (t) Vji (t) Eji (t) + γji (t) Eji (t)] .
2 2 +γji (t)] Eji (t)} .
j=1,j6=i j=1,j6=i
(11) (14)
VOLUME 11, 2023 5

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

Therefore, the optimization problem P1 is equivalent to the Lemma 2: The costate variables {µi (t)} can be obtained by
optimization problem P2 . solving the costate differential equation (19) with boundary
condition (15e).
P2 : min H (t) , (15a) N
{δji (t)},{γji (t)} · X
µi (t) = −a − [d1 δji (t) Vji (t) +d2 γji (t)] Eji (t)
· ∂H (t) j=1,j6=i
s.t. ci (t) = , (15b)
∂ui (t) X N 
b1 2 b2

ci (t0 ) = ci0 , (15c) − δji (t) Vji (t) + γji2 (t) Eji (t)
2 2
j=1,j6=i
· ∂H (t)
µi (t) = − , (15d)
 
N
∂ci (t)  X 
+ µi (t) λi + [δji (t) Vji (t) + γji (t)]Eji (t)
µi (tf ) = 0, (15e)  
j=1,j6=i
oji (t) 6 δji (t) + γji (t) , (15f) N
X
ci (t) ∈ [0, 1] , (15g) + [d1 δij (t) Vij (t) + d2 γij (t)] cj (t) eij (t)
d1 > d2 > 0, (15h) j=1,j6=i
N  
0 < b1 < b2 , (15i) X b1 2 b2 2
+ δ (t) Vij (t) + γij (t) cj (t) eij (t)
δji (t) > 0, (15j) 2 ij 2
j=1,j6=i
γji (t) > 0. (15k) N
X
− [δij (t) Vij (t) + γij (t)] µj (t) cj (t) eij (t).
j=1,j6=i
It is easy to prove that
(19)
Proof: According to (15d), solve the partial derivative of
∂H (t) (14) with respect to {ci (t)}, (19) can be obtained.
> 0, (16a)
∂δij (t) Based on Lemma 1 and Lemma 2, the optimal congestion
∂H (t) control strategies can be obtained by Theorem 1.
> 0. (16b) Theorem 1: The optimal congestion control strategies
∂γij (t)  ∗
δji (t) and γji∗ (t) are

Therefore, the minimum of H (t) with respect to the con-  0,
 ci (t) = 0, or Vji (t) = 0, or
trol strategies exists. Since the system model is initial-state- ∗
δji (t) = Eji (t) = 0, or µi (t) 6 d1 , (20)
fixed and final-state-free, the state constraints and costate  µi (t)−d1 , c (t) V (t) E (t) 6= 0.

b1 i ji ji
constraints of this kind of model are (15b)-(15c) and (15d)-
(15e), respectively, which is according to the Hamilton opti- 
mal control theory. Moreover, the constraints (13b)-(13g) of  0,
 ci (t) = 0, or Eji (t) = 0,
P1 should be also applied to P2 , i.e., (15f)-(15k). γji∗ (t) = or µi (t) 6 d2 , (21)
 µi (t)−d2
The optimal control strategies δji∗ (t) and γji∗ (t) in , ci (t) Eji (t) 6= 0.
  
b2
optimization problem P2 should satisfy (17). Proof: Solve the partial derivatives of Hamilton function
with respect to {δji (t)} and {γji (t)} and make them 0 as
δji∗ (t) , γji∗ (t) 6 H ({δji (t)} , {γji (t)})
 

H (17) ∂H (t)
= d1 Vji (t) Eji (t) ci (t) + b1 δji (t) Vji (t) Eji (t) ci (t)
∂δji (t)
To solve δji∗ (t) and γji∗ (t) , the state variables {ci (t)}
 
−µi (t) ci (t) Vji (t) Eji (t) = 0.
and costate variables {µi (t)} need to be solved as Lemma 1 (22)
and Lemma 2, respectively. ∂H (t)
= d2 Eji (t) ci (t) + b2 γji (t) Eji (t) ci (t)
Lemma 1: The state variables {ci (t)} can be obtained by ∂γji (t) (23)
solving the congestion state differential equation (18) with −µi (t) ci (t) Eji (t) = 0.
boundary condition (15c).  ∗
 ∗ the optimal congestion control strategies δji (t) and
Then,
γji (t) can be solved.
N
· X From P2 and (18)-(21), we can see that the optimization
ci (t) = [1 − ci (t)] λi − ci (t) [δji (t) Vji (t)
(18) objective is the time integral function of the congestion state
j=1,j6=i
variables, the control strategies, and the costate variables.
+γji (t)] Eji (t) . These three variables are mutually dependent on each other,
whose close loop relationship makes it difficult to attain the
Proof: According to (15b), solve the partial derivative of explicit solution. Therefore, CCDA algorithm is proposed to
(14) with respect to {µi (t)}, (18) can be obtained. obtain the optimal control strategies in Section IV-B.
6 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

Algorithm 1 Congestion Control Discretization Algorithm (CCDA)

Input: ci0 , µi (K ), δji (t0 ), γji (t0 ), σ and other networknparameters o
Output: Optimal congestion control strategies δji∗ (k + 1) , Node 1
n o
Node 2
γji∗ (k + 1) . 0.4
Node 3
1. For k = 0, k < K , k + + Node 4

Congestion Probability
2. For i = 1, i 6 N , i + + 0.35
3. For j = 1, j 6= i & j 6 N , j + +
4. Calculate discrete state variables ci (k + 1) via (29);
0.3
5. Calculate discrete costate variables µi (K − k − 1) via (30);
6. Calculate optimal congestion control strategies δji∗ (k + 1) and
γji∗ (k + 1) according to Theorem 2; 0.25
7. If δji∗ (k + 1) + γji∗ (k + 1) < oji (k + 1)
8. γji∗ (k + 1) = oji (k + 1) − δji∗ (k + 1); 0.2
9. End
10. End
0.15
11. End

0 2 4 6 8 10
B. CCDA ALGORITHM Time
The total continuous time interval [t0 , tf ] is divided into K
FIGURE 3. The evolution of the congestion states.
discrete subintervals. The length of each subinterval is σ.
tf − t0
σ= . (24)
K probability of the four nodes is set as c10 = 0.45, c20 = 0.4,
The k-th subintervalis denoted byk, k ∈ [0, K − 1]. When σ c30 = 0.35, c40 = 0.3, respectively. The delay weight coeffi-
is extremely small, δji∗ (k) and γji∗ (k) can be considered cients are a = 3, d1 = 10, d2 = 4 and the cost coefficients are
as the optimal approximate solution as Theorem 2. Based on b1 = 2, b2 = 10. The congestion deterioration probability is
which, the optimal congestion control strategies can be finally λi = 0.384 with Poisson parameter λp = 5. The number of
obtained via Algorithm 1. subintervals is K = 10 and the subinterval length is σ = 0.01.
Theorem 2: The discrete optimal congestion control strategies The type of traffic flows among nodes pji (t) is randomly
(defined as CCDA strategies) can be expressed as assigned. The amount of overflowed data among nodes oji (t)
and the initial control strategies are both randomly assigned
0, ci (k + 1) = 0, or Vji (k + 1) = 0, or


 in [0, 10].
 Eji (k + 1) = 0, or µi (K − k − 1) 6 d1 ,


δji∗ (k + 1) = µi (K − k − 1) − d1 A. NODE CONGESTION STATE AND STRATEGY EVOLUTION
 , ci (k + 1) 6= 0, and


 b1 The evolution of congestion states of each node is shown in

Vji (k + 1) Eji (k + 1) 6= 0. Fig. 3. It is demonstrated that the congestion probability grad-
 (25) ually decreases over time through congestion control. The

 0, c i (k + 1) = 0 or Eji (k + 1) =0 dropping rates of deterministic node 2 and deterministic node
∗
γji (k + 1) = or µi (K − k − 1) 6 d2 , 3 are partially shown in Fig. 4. It is shown that the dropping
 µi (K −k−1)−d2 , c (k + 1) E (k + 1) 6= 0,

rate between deterministic nodes is lower than the dropping
b2 i ji
(26) rate between ordinary nodes and deterministic nodes most of
where ci (k + 1) and µi (K − k − 1) are respectively shown the time, e.g., in most cases, δ32 6 δ42 exists for node 2, and
as (29) and (30) in APPENDIX A. δ23 6 δ13 exists for node 3. However, at subinterval k = 6,
Proof: Please refer to APPENDIX A. since BE flow is transmitted from node 3 to node 2, and the
amount of overflowed data is more than that from node 4 to
V. SIMULATION RESULTS node 2, there exists δ32 > δ42 in this period. This is because
In this section, simulation results are provided to analyze the TS flows are not allowed to be dropped in the proposed model
evolution of congestion states and control strategies of nodes to guarantee the determinacy of them.
under the formulated model and proposed CCDA algorithm. In addition, the dropping rates of deterministic node 2
In addition, the CCDA strategies can reduce network total and ordinary node 4 are partially shown in Fig. 5. Results
loss compared with other strategies, proving the effectiveness and conclusions similar to those in Fig. 4 can be obtained:
of the formulated model and proposed CCDA algorithm in the dropping rate between deterministic nodes is lower than
reducing delay and mitigating congestion. the dropping rate between ordinary nodes and deterministic
The simulation is performed via MATLAB R2019b. Sim- nodes most of the time, e.g., in most cases, δ32 6 δ42 exists
ulation parameters are set as follows. The network includes for node 2, and δ24 6 δ14 exists for node 4. However, at
four nodes, where node 2 and node 3 are deterministic nodes, subinterval k = 0, since BE flow is transmitted from node 3
and node 1 and node 4 are ordinary nodes. The nodes are to node 2, and the amount of overflowed data is more than that
completely connected to each other. The initial congestion from node 4 to node 2, there exists δ32 > δ42 in this period.
VOLUME 11, 2023 7

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

1.2 500

42

32 490
1 Proposed
23 Random
480

Network Total Loss

13
0.8
Control Rate

470
0.6
460

0.4
450

0.2
440

0 430
0 2 4 6 8 10 0 2 4 6 8 10
Time Time

FIGURE 4. The evolution of dropping rate of deterministic nodes. FIGURE 6. Comparison of network total loss.

3 600

42 =3, i=0.353
p
2.5 32 =4, i=0.371
550 p
14 =5, i=0.384
p
=6, i=0.394
Network Total Loss

24 p
2 500
=7, i=0.401
Control Rate

1.5 450

1 400

0.5 350

0 300
0 2 4 6 8 10 0 2 4 6 8 10
Time Time
FIGURE 5. The evolution of dropping rate of deterministic node and FIGURE 7. Comparison of network total loss with different congestion
ordinary node. deterioration probability.

Another remarkable phenomenon is that, the dropping rate strategies. Compared with the random strategies, the network
of deterministic node 2 is lower than that of ordinary node total loss with the proposed control strategies is lower. This
4 in most cases. It is revealed that the deterministic nodes result indicates the proposed model can minimize delay and
and time sensitive flows are more subject to network delay mitigate congestion with the lowest control cost, because
than to strategy control cost. In other words, in order to ensure the joint reduction of node congestion degree, node control
the deterministic transmission of TS flows, the deterministic strategy cost and network delay is formulated as the reduction
nodes may attempt to avoid packet dropping despite having of network total loss. In addition, the network total loss
to sacrifice the strategy control cost. gradually rises because both the congestion control cost and
the network delay in the defined network total loss function
B. NETWORK TOTAL LOSS EVALUATION accumulate over time. However, the upward trend is more and
The comparison of network total loss with the proposed more gentle, indicating that the proposed model and algo-
CCDA strategies and random strategies is shown in Fig. rithm can achieve effective congestion control. In summary,
6. Note that we aim to take a more global view to study the constructed congestion control model in deterministic
the optimal selection of multiple congestion techniques or networks and the proposed CCDA algorithm can achieve the
mechanisms rather than a certain technique like related work goal of minimizing delay and congestion degree with the
in Section II, so maybe no related work is suitable for com- lowest control cost.
parison and we conservatively compare with the random
8 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

0.9
d =6 d =5
1 2
0.8 9.55
d1=8 d2=4

Path Replanning Rate of Node 1

d1=10 9.5 d2=3
0.7
Dropping Rate of Node 1

d =12 d2=2
1 9.45
0.6
9.4
0.5
9.35
0.4
9.3
0.3
9.25
0.2
9.2
0.1
9.15
5 6 7 8 9 10
0
0 2 4 6 8 10 Time
Time
FIGURE 9. Comparison of path replanning rate with different delay
FIGURE 8. Comparison of dropping rate with different delay weight weight coefficient for path replanning strategy.
coefficient for dropping strategy.

strategy. Therefore, in practical applications, preferentially

C. MODEL PARAMETER EVALUATION choosing the low-delay countermeasures can effectively re-
1) Congestion deterioration probability duce network congestion and network total loss. And it is also
The comparison of the network total loss with different required to trade off the delay performance and cost of these
congestion deterioration probability λi is shown in Fig. 7. countermeasures to make a comprehensive decision.
According to (5), as the packet arrival mean λp increases
from 3 to 7, λi will increase from 0.353 to 0.401, and the VI. CONCLUSIONS
network total loss increases consequently. This is because as In this paper, we investigate the network congestion alle-
λi increases, both the probability and the number of nodes viation problem in deterministic networks, and conduct the
from the uncongested state to the congested state increase. real-time prediction and macroscopic regulation of conges-
The implementation intensity of the control strategies and tion control. Firstly, a congestion control system model is
network congestion degree increase accordingly, so does the constructed, where the congestion states are affected by the
network total loss. Therefore, mechanisms such as traffic node strategies. Then, an optimization problem is formulated
shaping at the queue entrances, ECN at downstream nodes to jointly alleviate network congestion and minimize delay
can be deployed to control the packet arrival rate of congested with the lowest control cost. By proposing a CCDA algorithm
nodes, e.g., notifying upstream nodes to reduce the forward- based on the optimal control method, the CCDA strategies
ing rate within an acceptable range or limiting the dequeue of are derived, achieving the continuous state evolution and real-
unimportant packets to alleviate the network congestion. time strategy selection. Finally, simulation results show that
with the formulated model and proposed CCDA algorithm,
2) Delay weight coefficient the node congestion probability gradually decreases over
The comparison of dropping rate of node 1 with different time, and the network total loss is lower than the baseline
delay weight coefficient for dropping strategy d1 is shown model. It is revealed that this solution can effectively mitigate
in Fig. 8. It is demonstrated that with the increase of d1 , the network congestion, reduce network delay, and achieve the
dropping rate of node 1 decreases. In addition, the compar- trade off between control cost and network performance.
ison of path replanning rate of node 1 with different delay In summary, this research provides a general modeling
weight coefficient for path replanning strategy d2 is shown thought and a reasonable mathematical optimization method
in Fig. 9. Note that we just select partial subintervals to for the research of deterministic networks especially the con-
more clearly show the relationship among the four almost gestion state evolution analysis. In addition, it has a theo-
coincident lines. It is demonstrated that with the decrease of retical and practical guiding significance for the real-time
d2 , the path replanning rate of node 1 increases. In summary, selection of congestion control mechanisms and the global
the increase of delay weight coefficient for each control strat- regulation of network congestion situation.
egy will lead to the decrease of the implementation rate of For the future work, multilevel congestion states for net-
the corresponding strategy. This is because when the delay work nodes and the accurate mapping between actual network
weight coefficient increases, the delay caused by the control configurations and model parameters will be considered.
strategy of the same intensity will increase. To maintain the More realistic network topology may also be included in the
ideal delay and network total loss, the network nodes have simulation, so as to improve the authenticity and accuracy of
to reduce the implementation intensity of the corresponding the congestion control model.
VOLUME 11, 2023 9

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

APPENDIX A PROOF OF THEOREM 2 APPENDIX B MATHEMATICAL SYMBOLS

The discrete forms of (18) and (19) can be expressed as (27) Please refer to Table 1.
and (28), respectively.
ci (k + 1) − ci (k) REFERENCES
= [1 − ci (k + 1)] λi [1] Deterministic Networking Use Cases, RFC 8578, 2019.
σ [2] Q. Gao, J. Li, and Y. Liu, ‘‘Research on deterministic cyber-physical sys-
X N (27) tems architecture for 6G evolution,’’ Application of Electronic Technique,
−ci (k + 1) [δji (k) Vji (k) + γji (k)]Eji (k) . vol. 48, no. 12, pp. 11–14, Dec. 2022.
j=1,j6=i [3] A. Badar, D. Z. Lou, U. Graf, C. Barth, and C. Stich, ‘‘Intelligent edge
control with deterministic-IP based industrial communication in process
µi (K − k) − µi (K − k − 1) automation,’’ in 2019 15th International Conference on Network and Ser-
= −a + µi (K − k − 1) λi vice Management (CNSM), Halifax, NS, Canada, 2019, pp. 1–7.
σ [4] Deterministic Networking Architecture, IETF draft-finn-detnet-
N N
X X architecture-08, 2016.
−d1 δji (k) Vji (k)Eji (k) − d2 γji (k) Eji (k) [5] Z. Ding, W. Li, Y. Cheng, Y. Xu, Y. Dai, Y. Wang, and P. Yu, ‘‘Slice network
j=1,j6=i j=1,j6=i framework and use cases based on FlexE technology for power services,’’
in 2021 International Wireless Communications and Mobile Computing
N N
b1 X b2 X 2 (IWCMC), Harbin, China, 2021, pp. 57–62.
− δji2 (k) Vji (k) Eji (k) − γji (k) Eji (k) [6] T. Huang, S. Wang, Y. Huang, Y. Zheng, J. Liu, and Y. Liu, ‘‘Survey of the
2 2 deterministic network,’’ Journal on Communications, vol. 40, no. 6, p. 17,
j=1,j6=i j=1,j6=i
N
Jun. 2019.
X [7] X. Wang, H. Yao, T. Mai, Z. Xiong, F. Wang, and Y. Liu, ‘‘Joint routing
+µi (K − k − 1) [δji (k) Vji (k) + γji (k)]Eji (k) and scheduling with cyclic queuing and forwarding for time-sensitive
j=1,j6=i networks,’’ IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp.
N 3793–3804, Mar. 2023.
X [8] S. Wang, B. Wu, C. Zhang, Y. Huang, T. Huang, and Y. Liu, ‘‘Large-
+ cj (k) eij (k) [d1 δij (k) Vij (k) + d2 γij (k)] scale deterministic IP networks on CENI,’’ in IEEE INFOCOM 2021 -
j=1,j6=i IEEE Conference on Computer Communications Workshops (INFOCOM
N   WKSHPS), Vancouver, BC, Canada, 2021, pp. 1–6.
X b1 2 b2 [9] B. Liu, S. Ren, C. Wang, V. Angilella, P. Medagliani, S. Martin, and
+ cj (k) eij (k) δij (k) Vij (k) + γij2 (k) J. Leguay, ‘‘Towards large-scale deterministic IP networks,’’ in 2021 IFIP
2 2 Networking Conference (IFIP Networking), Espoo and Helsinki, Finland,
j=1,j6=i
N 2021, pp. 1–9.
X [10] T. Li and Y. Cai, ‘‘Joint routing and scheduling for deterministic net-
− µj (K − k) cj (k) eij (k) [δij (k) Vij (k) + γij (k)]. working: A segment routing approach,’’ in 2022 IEEE 23rd International
j=1,j6=i Conference on High Performance Switching and Routing (HPSR), Taicang,
(28) Jiangsu, China, 2022, pp. 189–194.
Therefore, ci (k + 1) and µi (K − k − 1) can be derived as [11] B. Wu, J. Wang, Y. Wang, W. Tan, and Y. Huang, ‘‘Achieving deterministic
service in mobile edge computing (MEC) networks,’’ in 2021 IEEE 29th
(29) and (30), respectively. International Conference on Network Protocols (ICNP), Dallas, TX, USA,
2021, pp. 1–6.
ci (k + 1) = [ci (k) + σλi ] [12] C. Pan, S. Zhang, C. Zhao, H. Shi, Z. Kong, and X. Cui, ‘‘A novel active
, N

queue management algorithm based on average queue length change rate,’’
 X 
IEEE Access, vol. 10, pp. 75 558–75 570, Jul. 2022.
1 + σλi + σ [δji (k) Vji (k) + γji (k)]Eji (k) . [13] S. Shahzad, E.-S. Jung, J. Chung, and R. Kettimuthu, ‘‘Enhanced explicit
 
j=1,j6=i congestion notification (EECN) in TCP with P4 programming,’’ in 2020
(29) International Conference on Green and Human Information Technology
µi (K − k − 1) = {µi (K − k) + σa (ICGHIT), Hanoi, Vietnam, 2020, pp. 35–40.
[14] G. Peng, S. Wang, Y. Huang, R. Huo, T. Huang, and Y. Liu, ‘‘Traffic
N
X shaping at the edge: Enabling bounded latency for large-scale deterministic
+ σEji (k) [d1 δji (k) Vji (k) + d2 γji (k)] networks,’’ in 2021 IEEE International Conference on Communications
j=1,j6=i Workshops (ICC Workshops), Montreal, QC, Canada, 2021, pp. 1–6.
[15] Y. Liu, Z. Cheng, J. Ren, and D. Yang, ‘‘Joint routing and scheduling for
N
X σ CQF,’’ in 2022 7th International Conference on Computer and Communi-
Eji (k) b1 δji2 (k) Vji (k) + b2 γji2 (k)
 
+ cation Systems (ICCCS), Wuhan, China, 2022, pp. 1–5.
2 [16] A. Jk, A. Sm, A. Pm, A. Jl, C. B. Shuang, B. Xc, and B. Xg, ‘‘Joint rout-
j=1,j6=i
N ing and scheduling for large-scale deterministic IP networks,’’ Computer
X Communications, vol. 165, pp. 33–42, Jan. 2021.
− σcj (k) eij (k) [d1 δij (k) Vij (k) + d2 γij (k)] [17] Y. Huang, S. Wang, T. Feng, J. Wang, T. Huang, R. Huo, and Y. Liu,
j=1,j6=i ‘‘Towards network-wide scheduling for cyclic traffic in IP-based determin-
N istic networks,’’ in 2021 4th International Conference on Hot Information-
X σ Centric Networking (HotICN), Nanjing, China, 2021, pp. 117–122.
cj (k) eij (k) b1 δij2 (k) Vij (k) + b2 γij2 (k)
 
− [18] S. Chen, J. Leguay, S. Martin, and P. Medagliani, ‘‘Load balancing for de-
2 terministic networks,’’ in 2020 IFIP Networking Conference (Networking),
j=1,j6=i
N Paris, France, 2020, pp. 785–790.
X [19] Z. Liu, J. Sun, S. Hu, and X. Hu, ‘‘An adaptive AQM algorithm based on a
+ σµj (K − k) cj (k) eij (k) [δij (k) Vij (k) + γij (k)]} novel information compression model,’’ IEEE Access, vol. 6, pp. 31 180–
j=1,j6=i 31 190, Jun. 2018.
N [20] J. Escudero-Sahuquillo, P. Garcia, F. Quiles, J. Flich, and J. Duato, ‘‘Cost-
X effective congestion management for interconnection networks using dis-
/ {1 + σλi + σ [δji (k) Vji (k) + γji (k)] Eji (k)} . tributed deterministic routing,’’ in 2010 IEEE 16th International Confer-
j=1,j6=i ence on Parallel and Distributed Systems, Shanghai, China, 2010, pp. 355–
(30) 364.

10 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

[21] J. Liu, J. Huang, W. Jiang, Z. Li, Y. Li, W. Lyu, W. Jiang, J. Zhang, and
J. Wang, ‘‘End-to-end congestion control to provide deterministic latency
over internet,’’ IEEE Communications Letters, vol. 26, no. 4, pp. 843–847,
Apr. 2022.
TABLE 1. Mathematical Symbols [22] X. Yang, W. Wang, J. Han, K. Xue, and Z. Zhang, ‘‘Congestion control with
deterministic service delay guarantee,’’ in 2022 IEEE 19th International
Symbol Quantity Remark Conference on Mobile Ad Hoc and Smart Systems (MASS), Denver, CO,
i node sequence i ∈ [1, N ] USA, 2022, pp. 555–563.
t time

continuous time t ∈ t0 , tf
 [23] L. Zhang, K. Zhu, Y. Cui, and Y. Jiang, ‘‘Qos-aware congestion control with
online learning,’’ China Communications, vol. 20, no. 2, pp. 168–178, Feb.
type of node
 i at time t, 2023.
vi (t) node type 1, deterministic node,
vi (t) = [24] A. Grigorjew, F. Metzger, T. Hoçfeld, J. Specht, F.-J. GÃűtz, F. Chen, and
0, ordinary node.
J. Schmitt, ‘‘Constant delay switching: Asynchronous traffic shaping with
eij (t) path reachability path reachabilityfrom node i to j at jitter control,’’ in 2022 IFIP Networking Conference (IFIP Networking),
1, reachable, Catania, Italy, 2022, pp. 1–9.
time t, eij (t) =
0, unreachable. [25] Y. Mo, Z. Yang, H. Liu, and T. He, ‘‘Global cyclic queuing and forwarding
pij (t) traffic flow type traffic flow type
 from node i to j at time mechanism for large-scale deterministic networks,’’ in 2021 IEEE 23rd Int
1, TS flow, Conf on High Performance Computing & Communications; 7th Int Conf
t, pij (t) =
0, BE flow. on Data Science & Systems; 19th Int Conf on Smart City; 7th Int Conf
congestion  state of node i at time t, on Dependability in Sensor, Cloud & Big Data Systems & Application
Ci (t) congestion state 1, congested state Ji (t) , (HPCC/DSS/SmartCity/DependSys), Haikou, Hainan, China, 2021, pp.
Ci (t) =
0, uncongested state Fi (t) . 275–282.
ci (t) congestion state vari- congestion probability of node i at [26] T. Fukui, Y. Sakaue, K. Minami, and T. Taniguchi, ‘‘Delay-based shaper
able time t with dynamic token bucket algorithm for deterministic networks,’’ IEEE
δji (t) dropping rate intensity of dropping strategy to be op- Access, vol. 10, pp. 114 424–114 433, Oct. 2022.
timized [27] IEEE standard for local and metropolitan area networks – virtual bridged
γji (t) path replanning rate intensity of path replanning strategy to local area networks amendment 12: Forwarding and queuing enhancements
be optimized for time-sensitive streams, IEEE Std 802.1Qav-2009 (Amendment to IEEE
λp packet arrival mean packet arrival follows a Poisson distri- Std 802.1Q-2005), 2010.
bution with parameter λp [28] IEEE standard for local and metropolitan area networks – bridges and
λi congestion deteriora- probability of sufficient packet arrival, bridged networks – amendment 25: Enhancements for scheduled traf-
tion probability e.g., λi = Pr {X > λp } = 1 − fic, IEEE Std 802.1Qbv-2015 (Amendment to IEEE Std 802.1Q-2014 as
λp
P λkp −λ
amended by IEEE Std 802.1Qca-2015, IEEE Std 802.1Qcd-2015, and
k!
e p IEEE Std 802.1Q-2014/Cor 1-2015), 2016.
k=0 [29] M. Kim, D. Hyeon, and J. Paek, ‘‘eTAS: Enhanced time-aware shaper for
li (t) delay refer to (7) supporting nonisochronous emergency traffic in time-sensitive networks,’’
Vji (t) none Vji (t) = 1 − vj (t) vi (t) pji (t), just IEEE Internet of Things Journal, vol. 9, no. 13, pp. 10 480–10 491, Jul.
for ease of expression
  2022.
Eji (t) none Eji (t) = eji (t) 1 − cj (t) , just for [30] IEEE standard for local and metropolitan area networks – bridges and
ease of expression bridged networks – amendment 34: Asynchronous traffic shaping, IEEE
a delay weight coeffi- a constant to weight the delay caused Std 802.1Qcr-2020 (Amendment to IEEE Std 802.1Q-2018 as amended by
cient by congestion IEEE Std 802.1Qcp-2018, IEEE Std 802.1Qcc-2018, IEEE Std 802.1Qcy-
d1 delay weight coeffi- a constant to weight the delay caused 2019, and IEEE Std 802.1Qcx-2020), 2020.
cient by dropping strategy [31] IEEE standard for local and metropolitan area networks – bridges
d2 delay weight coeffi- a constant to weight the delay caused and bridged networks – amendment 26: Frame preemption, IEEE Std
cient by path replanning strategy 802.1Qbu-2016 (Amendment to IEEE Std 802.1Q-2014), 2016.
fi (t) control cost refer to (10) [32] H. Wang, J. Zhao, X. Liu, and C. Yin, ‘‘Enhanced frame preemption in
b1 cost coefficient a constant to weight the control cost of image and video transmission over time sensitive networks,’’ in 2022 IEEE
dropping strategy 33rd Annual International Symposium on Personal, Indoor and Mobile
b2 cost coefficient a constant to weight the control cost of Radio Communications (PIMRC), Kyoto, Japan, 2022, pp. 1185–1190.
path replanning strategy [33] IEEE standard for local and metropolitan area networks – bridges and
PN bridged networks – amendment 29: Cyclic queuing and forwarding,
g (t) instantaneous g (t) = [li (t) + fi (t)], refer to IEEE 802.1Qch-2017 (Amendment to IEEE Std 802.1Q-2014 as amended
objective loss i=1
(11) by IEEE Std 802.1Qca-2015, IEEE Std 802.1Qcd(TM)-2015, IEEE Std
G total objective loss network total loss to be minimized, 802.1Q-2014/Cor 1-2015, IEEE Std 802.1Qbv-2015, IEEE Std 802.1Qbu-
refer to (12) 2016, IEEE Std 802.1Qbz-2016, and IEEE Std 802.1Qci-2017), 2017.
oji (t) overflowed packet a lower bound to constrain strategies [34] L. Qiang, B. Liu, D. Yu, and C. Wang, ‘‘Large-scale deterministic network
forwarding technology,’’ Telecommunication Science, no. 9, p. 8, Sep.
µi (t) costate variable each costate variable corresponds to a
2019.
state variable in optimal control theory
[35] Y. Huang, S. Wang, T. Huang and Y. Liu, ‘‘Cycle-Based Time-Sensitive
H (t) Hamilton function refer to (14)
and Deterministic Networks: Architecture, Challenges, and Open Issues,’’
δji∗ (t) optimal dropping rate optimized intensity of dropping strat-
IEEE Communications Magazine, vol. 60, no. 6, pp. 81-87, Jun. 2022.
egy
[36] K. Alexandris, P. Pop, and T. Wang, ‘‘Configuration and evaluation of
γji∗ (t) optimal path replan- optimized intensity of path replanning multi-CQF shapers in IEEE 802.1 time-sensitive networking (TSN),’’ IEEE
ning rate strategy Access, vol. 10, pp. 109 068–109 081, Oct. 2022.
k subinterval sequence discrete time k ∈ [0, K − 1] [37] B. Sun and B.-Z. Guo, ‘‘Convergence of an upwind finite-difference
σ subinterval length length of each discrete subinterval scheme for hamilton-jacobi-bellman equation in optimal control,’’ IEEE
t −t
σ= fK0 Transactions on Automatic Control, vol. 60, no. 11, pp. 3012–3017, Nov.
ci0 congestion probabil- congestion probability of node i at 2015.
ity subinterval k = 0

VOLUME 11, 2023 11

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3301964

Q. Gao et al.: Preparation of Papers for IEEE ACCESS

FIRST Q. GAO received the Master’s degree

from Beijing University of Posts and Telecom-
munications (BUPT) in 2022. She is currently a
researcher at China Telecom Research Institute.
Her research interests are in the area of mobile
communications and networks, core network ar-
chitecture design, and deterministic networks, with
current emphasis on the deterministic networks
and key technologies towards B5G/6G. (Email:
[email protected])

SECOND J. LI received the Master’s degree from

Beijing University of Posts and Telecommunica-
tions (BUPT). She is a professorate senior engineer
of China Telecom Research Institute. Her research
focuses on the standardization and technique evo-
lution of mobile networks, especially in the area
of architecture evolution and service deployment.
(Email: [email protected])

THIRD Y. LIU received the B.S. and Ph.D. degrees

from Beijing University of Posts and Telecom-
munications (BUPT) in 2006 and 2011, respec-
tively. She is currently a senior researcher at China
Telecom Research Institute and focuses on the
RAN architecture design towards 6G. Her re-
search interests include virtualization and intel-
ligence for wireless communications. (Email: li-
[email protected])

FOURTH Y. XING received the Master’s degree

from Capital Normal University in 1999. She is
currently a senior engineer at China Telecom Re-
search Institute. Her research interests include mo-
bile communication network architecture and key
technologies, network operation and management.
(Email: [email protected])

12 VOLUME 11, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4