Polymer Testing 93 (2021) 106893

Contents lists available at ScienceDirect

Polymer Testing
journal homepage: http://www.elsevier.com/locate/polytest

Long-term creep behavior prediction of polymethacrylimide foams using

artificial neural networks
Jianlin Zhong , Chunhao Yang *, Wuning Ma , Zhendong Zhang
Department of Mechanical Engineering, Nanjing University of Science and Technology, Nanjing, China

A R T I C L E I N F O A B S T R A C T

Keywords: To study the long-term creep behavior prediction of polymethacrylimide (PMI) foams, the gradation loading
Creep creep tests were proposed at four different temperatures in this paper. The Stepped Isostress (SSM) and TTSP
Polymethacrylimide form methods were combined to obtain the master curve under reference stress and temperature. The artificial Neural
Stepped isostress method
Networks (ANN) technique was used to build the long-term creep behavior prediction model of PMI materials.
Artificial neural networks
The effects of different activation functions, hidden layer structures, and other super parameters on the pre­
diction performance were investigated. The results suggest that the SSM plus TTSP method can be used to
construct the master curve, which could predict a larger time scale of material creep behavior based on a short-
term test. It is of great significance and feasible to predicts the long-term creep life of materials accurately using
advanced artificial intelligence technology. According to the statistical analysis, the logistic type activation
function has a more accurate and stable prediction performance on long-term creep behavior prediction of PMI.
To avoid overfitting, the number of hidden layers should be as small as possible, and the prediction performance
of a single hidden layer structure with 8 neurons is sufficient for long-term creep behavior prediction in the
engineering area. The statistical value of the correlation coefficient was greater than 0.995. The application
range of advanced artificial intelligence technology in this field can be further expanded in the preceding
research, such as in the prediction of long-term creep behavior on the composition level of the material.

1. Introduction platform and densification. Therefore, the traditional linear elastic

models are not suitable for PMI foam. Then, a phenomenological model
Polymethacrylimide (PMI) is formed by the copolymerization of was established to describe the compressive behavior of PMI foams at
methacrylate and methacrylonitrile, which has a higher specific strength different strain rates and temperatures. Siivola et al. [3] studied the
and specific stiffness than other traditional foam materials such as indentation response of PMI foams core and related sandwich structures
polyurethane and polyvinylchloride with the same density. Besides, PMI under different temperature and humidity conditions. From the uniaxial
forms also bring to an excellent capacity of heat resistance, fatigue compression test results, they found that the strength and stiffness of
resistance and gleyness. Therefore, as an advanced core material of PMI foams decrease with the increase of temperature and humidity, and
sandwich structures, PMI forms have been widely used in aerospace, the comprehensive influence was more serious than individual influ­
railway locomotives and ship engineering areas in recent years. ence. The quasi static and dynamic compression tests were made on
At present, extensive studies have been carried out on the mechan­ sandwich structure with PMI foams core by Qu et al. [4]. The influence
ical properties of sandwich structures with PMI foams core. According to factors including the density of PMI, thickness of Aluminum face sheets
statics test, Herranen et al. [1] found that core materials like PMI result and PMI core sheet were considered to analysis the mechanical response
in improvement material performance, such as compressive strength, of sandwich structures. The consititutive relation of PMI was fitting by
stiffness, modulus, temperature resistance and hot-wet resistance. ANN Cowper-Symonds function. The results showed that the density of PMI
model was also recommended to analysis the sensitivity of PMI foams in was the most influential factor of compressive strength, and the dynamic
their research. Based on the compression tests of PMI foams, Grace et al. mechanical properties increased with the increase of strain rate. Based
[2] found the stress-strain curves of PMI foams has strong nonlinear on “in-situ” Fourier transform infrared spectroscopy, Zhang et al. [5]
behavior, which can be divided into three phases: linear elastic, yield analyzed the preparation of PMI foams in temperature range

* Corresponding author.
E-mail addresses: [email protected], [email protected] (C. Yang).

https://doi.org/10.1016/j.polymertesting.2020.106893
Received 6 July 2020; Received in revised form 1 September 2020; Accepted 12 October 2020
Available online 16 October 2020
0142-9418/© 2020 Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
J. Zhong et al. Polymer Testing 93 (2021) 106893

180∘C–200∘C. A compression test was conducted to investigate the SSM experimental studied the tensile creep of unidirectional carbon/­
properties of a new type mental tube filling enhanced electromagnetic epoxy lamina, and three kinds of failure criterion were contrasted.
wave absorbent PMI foams by Yan et al. [6]. The results showed that Neural networks are composed of many connected simple processing
after enhanced by mental tube, the elastic module, compression strength units, similar to the structure of synaptic connections in biological brain.
and energy absorption efficiency were improved greatly. Due to the As an operational model, neural networks using distributed parallel
weak interface between PMI and electromagnetic absorbent, the main information processing algorithm, relie on a lot of nodes (neurons), and
failure modes were crack propagation and structure collapses. Zhou achieve the purpose of information processing by adjusting the
et al. [7] analyzed the quasi static compressive response of PMI foams connection relations among nodes [29]. It has the ability of self-learning
filled carbon fiber reinforced composite honeycombs using theoretical and self-adaptation. Artificial neural networks (ANN) are computational
and numerical methods. Both parameters of material and structure were model for simulate human brain neural networks in terms of structure,
considered, while the results showed that elastic buckling and ductile mechanism and function. Similar to biological neurons, ANN consist of
fracture were the failure mode under quasi static compression loads. The multiple connected nodes (artificial neurons), which can be used to
method provided could be used to predict the compressive strength of model the complex relationships between data. The connections be­
foam filled honeycomb structure. Since the multiplicity, time depen­ tween different nodes are endowed with different weights, each weight
dence, and temperature dependence of the molecular chain segment represents the impact among notes. Each node represents a specific
movement of the foam material, it becomes a typical viscoelastic ma­ function, and the information from the other nodes is synthesized with
terial, which can creep under normal temperature and pressure. its corresponding weights, then input into an activation function and get
Therefore, it is necessary to study the creep properties of PMI foam an active value (excitation or inhibition). From the system angle, ANN
under long-term pressure. Research on the creep properties under the are adaptive nonlinear dynamic systems with a large number of neurons
following conditions has important theoretical significance and guiding through extremely rich and improve connections. It has the advantages
value for comprehensively understanding the long-term mechanical of human brain neural network, and have been widely applied in pattern
properties of PMI, making full use of PMI materials, and preventing information processing, pattern recognition, optimization problem,
engineering accidents, which are barely reported up to now. intelligent processing technology, complex control, signal processing
Due to the closely relationship between the temperature and mate­ and other fields. Its main advantages include [30–33]: (1) High paral­
rial internal molecular motion, molecular motion needs a relatively lelism. (2) Highly nonlinear overall role. (3) High fault tolerance and
longer relaxation time at lower temperature. It takes a relatively longer associative memory. (4) Strong self-adaptation and self-learning. With
time for the response of material molecular motion to reach a certain the rapid development of machine learning technology in recent years,
macroscopic physical value under external forces. Increasing the tem­ neural networks have been gradually applied in many developing fields,
perature, material molecular motion can be more violent and the and its excellent model prediction performance has provided a new di­
relaxation time could be shorter. In other words, the effect of longer rection and power for the further development.
mechanical response on the macroscopic stage can be observed in a In this paper, a new master curve construction method, SSM plus
relatively shorter time. It can be seen that lengthening the action time TTSP method, was used to study the long-term creep properties of PMI,
and raising the temperature have the same effect on the material mo­ and the master curve under reference stress and temperature were
lecular motion. Therefore, the effect on the mechanical properties of proposed. Then, based on the machine learning technology, an ANN
foam materials is also equivalent. Time-temperature superposition prediction model of PMI long-term creep life was built. Finally, a
principle (TTSP) is intended to explain that changing the time scale and comparative analysis was made on the prediction performance and the
temperature scale are equivalent to the effects of material molecules training stability of the prediction model and the reasonable neural
[8–10]. According to TTSP, the mechanical behavior of PMI at different network structure, activation function and other super parameters were
time scales can be described by changing the temperature. By shifting considered. The statistical value of the correlation coefficient was
and superposition a series of mechanical properties curves measured by greater than 0.995, and the feasibility of the neural network model for
tests at various temperatures in short time range, the master curve of long-term creep behavior prediction of PMI was verified.
materials in a longer time range at reference temperature can be ob­
tained. With the continuous development of TTSP, time-stress super­ 1.1. Need for research
position principle (TSSP) [11–13] and time-temperature-stress
superposition principle (TTSSP) [14–16] emerge as the times require. Experiments and theories are mutual verification and promotion.
According to TSSP, both temperature and stress have influence on the The long-term compression creep test is the most accurate and direct
relaxation time and other parameters of material. So the same relaxation way to study the long-term creep life of materials. However, unlike the
phenomenon can be observed in both smaller stress with longer loading conventional time required for general mechanical experiments, the
time and larger stress with shorter loading time. While the TTSSP holds long-term creep test requires a quite longer experimental time, more
that both temperature and stress level affect the free volume of difficult data measurement, and fewer experimental methods. Based on
materials. various equivalent principles, multi-temperature and multi-load creep
In addition to TTSP, the stepped isothermal method (SIM) receives experiments are unavoidable, which also consumes a large amount of
much concern as one of the material properties accelerated character­ time. Especially in a similar analysis of the influence of the content of
ization methods in recent years. SIM based on same specimen on a components on the overall creep life of the material, the number of
gradation increased temperature and constant loading condition, then experimental conditions required to obtain a more accurate prediction
the creep curve under each temperatures can be obtained through a curve is huge, and the experiment always takes too long.
series of correction methods, finally the master curve of long-term creep To solve these problems, on the one hand, the long-term mechanical
can be achieved by TTSP. Based on SIM, Thornton et al. [17,18] studied properties of materials should be characterized based on the reasonable
the long-term creep behavior of polyester geogrids and yarns, then use of existing experimental data and further expansion and improve­
predicted the soil reinforcement performance. Alwis and Burgoyne [19, ment of the equivalent principles. On the other hand, with the rapid
20] applied SIM to the long-term mechanical properties characterization development of artificial intelligence, it is necessary to actively research
of high modulus yarns. Similar to the relationship between TTSP and the adaptation of advanced artificial intelligence technology to the
TSSP, the stepped isostress method (SSM) based on the same specimen prediction of long-term mechanical properties of materials, relying on
on a gradation increased load and constant temperature condition the emerging high-performance model prediction technology to gradu­
[21–26]. Giannopoulos and Burgoyne [27] predicted the long-term ally realize the prediction of single working condition experimental
creep performance of Kevlar 49 using SSM. Tanks et al. [28] using curve and reference working condition master curve. Starting from the

2
J. Zhong et al. Polymer Testing 93 (2021) 106893

prediction of the same material component, the creep behavior under

other working conditions in the interval and within the limited range
can be predicted based on the limited experimental data. In this paper,
the ANN model fitting and prediction technology were used to train and
predict the creep curve of the same PMI material at different tempera­
tures. The further application feasibility of this method in long-term
creep life analysis was demonstrated. This method can be gradually
extended to the prediction of materials with a different component. To
realize the material performance prediction with other component
contents through experiments on materials with several typical
component contents. Then analyze the influence of different compo­
nents on the long-term creep properties of materials more quickly.

2. Materials and methods

Fig. 2. The specimens’ distribution of the actual diameter and thickness.

2.1. Materials and specimens of PMI

Commercial closed-cell rigid halogen-free PMI foams were consid­ where εt , εe and εc are total strain, elastic strain and creep strain,
ered in this research. The foam attributes a fine and unique cell size respectively. Both εt and εc show temporal correlation.
grade (less than 0.3 mm). In order to ensure the consistency of the In this paper, the creep properties master curve of PMI foams under
thermal performance, all the specimens using in creep tests are taken reference temperature and reference stress was constructed by
from the same PMI foam sheet with a density of 160 kg/m3, which is combining SSM and TTSP. The operation framework is shown in Fig. 4.
processed by a CNC machine tool, as shown in Fig. 1a. The specimens are Firstly, using three major steps including vertical shifting, rescaling and
cylindrical shape with a diameter of 12.5 mm and thickness of 6 mm, as horizontal shifting on the step-wise creep curve at various temperatures,
shown in Fig. 1b. Due to the machining error in the CNC system, the and the master curves of PMI foams under reference stress at four
distribution of the actual size and theoretical size of the specimens were temperatures were obtained respectively. Then, the TTSP method was
statistically analyzed. The results are shown in Fig. 2. The solid line used to carry out a horizontal shifting of the master curves at each
represents the mean value of the real size, while the dash line represents temperature to realize the quadratic construction of the master curves.
the theoretical value. Finally, the master curve of PMI foams under the reference stress and
reference temperature was obtained.
2.2. Creep tests and SSM plus TTSP methods
2.2.2. Vertical shifting
2.2.1. Test method Through vertical shifting of the test curves, the elastic strain under
The loading sequence with stepwise jump stress and dwell time are each stage of stress was subtracted and the creep strain was retained.
used in the PMI creep tests. The stress of each stage is 1 MPa, 2 MPa, 3 Therefore, in this step, Eq. (1) is used to process the stress at all stages,
MPa, 4 MPa, 5 MPa and 6 MPa, respectively. And the dwell time of each and the elastic strain between stress is removed from the total strain.
stage is 4 h, as shown in Fig. 3a. The tests are conducted at 20∘C, 40∘C,
65∘C and 90∘C, respectively. The strain-time curve of PMI foams under 2.2.3. Rescaling
gradation loading at each temperature are shown in Fig. 3b. The purpose of rescaling is to transform the creep curve under step-
Assuming the thermal strains are negligible, the total strain can be wise stress into a group of independent creep curves under each stress
defined as the combination of elastic deformation and creep deforma­ step. In order to separate the creep curves at different stress steps, it is
tion, with reference stresses in the elastic region, using the following necessary to consider that the creep strain resulting from each stage of
equations: stress includes the creep accumulation of the previous stages. Therefore,
in order to obtain the creep curve under a certain stage of stress, the
εt = εe + εc (1)

Fig. 1. The material and specimens of PMI foams. (a) PMI specimen; (b) The size of specimen.

3
J. Zhong et al. Polymer Testing 93 (2021) 106893

Fig. 3. The strain-time curve of PMI material during the creep tests. (a) The loading sequence; (b) The strain-time curve of PMI.

[ ]B(i)
ε(i)
c =A
(i) (i)
t + C(i) (2)

where A(i) , B(i) and C(i) are power-law coefficients, i = 2, 3, ⋯, 6 refers to

the stress steps in the creep test. And the shifting value tre can be ob­
(i)

tained by Ref. [25].

(i)
tre = t(i) |ε(i) =0 (3)
c

2.2.4. Horizontal shifting of SSM

In this stage, the processing method is similar to that of TSSP. The
master curve of PMI under reference stress σ0 is obtained through hor­
izontal shifting the creep curve along the log-time axis. The shift factor is
a function of the stress. In order to determine the effects of stress on
creep rate and shifting factors, researchers mainly established Eyring’s
equation [23,24,35] and WLF equation depend on the relationship be­
tween the test temperature and the glass transition temperature. Ac­
cording to Eyring’s equation, the strain rate relationship can be
expressed as
( ) ( )
U NA V ∗ σ
ε̇ = ε̇0 exp − exp (4)
RT RT

where U is the activation energy, R is the universal gas constant, T is the

absolute temperature, NA is the Avogadro constant and V ∗ is the acti­
vation volume assumed constant. Assuming the activation volume as
Fig. 4. The operation framework of creep master curve combining SSM constant, the ratio of strain rate ε̇1 , ε̇2 under two stress level σ1 and σ2 at
and TTSP.
the same temperature T can be expressed as

theoretical loading start point should be obtained from the existing ( )

ε̇1 V∗
multi-stage test curve based on the Boltzmann superposition principle, ln = (σ 1 − σ 2 ) (5)
ε̇2 kT
which is called rescaling. The methods of rescaling can be carried out by
pure graphical method [23,28], power-law fitting [34] or Prony series
V∗
fitting [28]. Where the pure graphical method is relatively simple to log(ασ ) = (σ − σ0 ) (6)
2.303kT
operate, but it relies more on the subjective judgment of the researcher,
the result is not unique and prone to large errors. while the fitting where stress shift factor ασ is the ratio of time under test stress σ and
method focuses on the steady creep rate of the remaining creep curves reference stress σ0 .
except for the primary load. By fitting the steady creep curve, the ( )
theoretical loading time of the material is obtained. The fitting function ε0 (σ 0 , t) = ε σ ,
t
(7)
is selected according to the characteristics of the creep curve of different ασ
materials to some extent. In this research, the theoretical loading start
point was obtained by using the power-law function fitting the creep 2.2.5. Horizontal shifting of TTSP
curves under different stress except the initial stage and extended the After the previous steps, the creep master curves under the reference
fitting curves to the zero strain position. Then, shifting the curves on the stress were obtained focus on each corresponding temperature of the
time axis in order to change the theoretical loading start point to zero, test. In the final step, the temperature shifting factor αT of TTSP is used
using the following equations [25]: to horizontal shifting the creep curve. Eventually, get the creep master
curve of the PMI under reference temperature and reference stress.
Similarly, the temperature shifting factor αT can be obtained by

4
J. Zhong et al. Polymer Testing 93 (2021) 106893

( )
U 1 1 where f (1) is the activation function of the first hidden layer, which
log(αT ) = − (8)
2.303R T T0 would be determined by the precision of the prediction model. The ANN
will continue to propagate until output the matrix ̂ y . One neuron rep­
where T is the test temperature, while T0 is the reference temperature. resents strain that was used in the output layer ̂
y . The mean square error
(MSE) is used as a loss function to describe the error between ̂ y and
3. Prediction model of ANN actual value y, which can be expressed as
N ( )2
3.1. The structure of ANN model 1 ∑
L(̂
y , y) = ̂
y k − yk (11)
N k=1
At present, there are more than 40 kinds of ANN models. In the field
of machine learning, mainly feedforward neural networks (FNN), radial where N is the number of sample. The structural risk function of ANN on
basis function neural network (NRBFNN), recurrent neural networks the training set is:
(RNN), convolutional neural networks (CNN), and modular neural net­ ( )
works (MNN), etc are in common use. Each neuron in the FNN is divided 1 ∑N
1
R(w, b) = L ̂y k , yk + λ‖w‖2F (12)
into different groups according to the order of receiving information. N k=1 2
Each group can be regarded as a neural layer. The neurons in each layer
receive the output of the previous layer and output to the next layer. The where ||w||2F is the regularization term, using the Norm of Frobenius. λ >
information in the entire network propagates in one direction, and there 0 is the super parameter. The weight wi and bias bi in ANN model are
(j) (j)

is no reverse information propagation, which can be represented by a optimized by gradient descent method.
directed acyclic graph. The FNN can be regarded as a function and re­ The FNN training process using BP can be divided into the following
alizes the complex mapping from input space to output space through three steps:
multiple combinations of simple nonlinear functions.
In this research, ANN with Feedforward Neural Network (FNN) and ∑ (1)
1. The feedforward calculation of the input wi ⋅x + bi and activa­
(1)
Back Propagation (BP) was employed to achieve the prediction model of
tion value ai of each layer to the last;
PMI’s long-term creep behavior. According to the nonlinear creep
2. BP calculates the error term of each layer;
properties of PMI shown in tests, the prediction model contains an input
3. Calculate the partial derivative and then update the parameters of
layer, an output layer and at least one hidden layer, as shown in Fig. 5.
each layer.
Where three neurons xst ,xte and xt represent stress, temperature and
time, respectively were used in the input layer x.
3.2. Parameter initialization
x = [xst , xte , xt ]T (9)
Considering the nonlinear degree of the PMI creep curve, the number According to the creep test condition, the distribution range of each
of hidden layers j in the ANN model is chosen to be 1 or 2, the total input parameter can be obtained as shown in Table 1. The numerical
number of neurons i in each hidden layer is less than 8. The final number dimensions of each input parameter are close and the ANN model can
will be determined by the precision and parameter scale of the predic­ adapt to the value range of different characteristics by adjusting the
tion model. model parameters. Therefore, no normalized preprocessing was carried
Each neuron in hidden layers has a weight matrix wi and a bias bi
(j) (j)
out for input parameters in this study.
value. According to FNN, for the first hidden layer, the processing Due to the shallow ANN model relatively, the uniform distribution is
method of input parameters is used to initialize model parameters.
(∑ )
a1i = f (1) w(1) (1)
i ⋅ x + bi (10)
3.3. Network optimization

The ANN model is a non-convex function, coupled with the vanishing

gradient in the deep network, which is difficult to optimize. Besides, due
to the more parameters and large training data of the ANN model, the
training efficiency is generally low. Therefore, researchers have carried
out a large number of research on neural network optimization methods
[36–40].
At present, optimization methods can be roughly divided into two
categories: one is to adjust the learning rate to make the optimization
more stable; the other is to adjust the gradient direction to optimize the
training speed. And learning rate adjustment can be divided into fixed
decay learning rate, periodic learning rate, and adaptive learning rate.
The common optimization methods of ANN are shown in Table 2. It can
be seen that the Adam method is a synthesis of two types of methods.
The Adam method was used to achieve neural network optimization
in this paper.The Adam algorithm is adopted to adjust the learning rate α
and update direction dynamically. The renewal difference of parameters
is

Table 1
Range of input parameters.
Stress(MPa) Temperature(∘C) Log Time(s)

[1,5] [20,90] [0,6]

Fig. 5. ANN prediction model of PMI’s long-term creep behavior.

5
J. Zhong et al. Polymer Testing 93 (2021) 106893

Table 2 temperature, the total deformation of the high temperature creep curve
The optimization algorithm of ANN. increases significantly.
Category optimization algorithm The PMI creep test curves were processed by the method described in
Section 3.2, including vertical shifting, rescaling and horizontal shifting.
Fixed decay learning Piecewise constant decay, Inverse time decay, Exponential
rate decay, Cosine decay The treatment process of the creep curve at 20∘C was shown in Fig. 6.
Periodic learning rate Cyclic learning rate, Stochastic gradient descent with warm And the master creep curves of PMI foams under reference stress at four
restarts temperatures were obtained as shown in Fig. 7a.
Adaptive learning rate AdaGrad, RMSprop, AdaDelta Then, the TTSP method was used to horizontally shift the master
Gradient direction Momentum method, Nesterov accelerated gradient,
optimize Gradient clipping
curve at each temperature to achieve quadratic construction of the
Synthesis method Adam master curve, and finally, the master curve of PMI at reference stress and
reference temperature was obtained, as shown in Fig. 7b.

α
(13)
l ̂l
Δθl = − √̅̅̅̅̅̅̅̅̅̅̅̅̅̅ M
̂
Gl + ε 4.2. ANN prediction of PMI creep curve

√̅ The prediction performance of ANN model varies with the change of

where l is the iteration number, αl = α0 / l, α0 = 0.001 and
activation function and network structure [41]. Therefore, the predic­
̂ l = Ml , G ̂ l = Gl tive performance of ANN model with same structure but different acti­
M (14)
1 − βl1 1 − βl2 vation functions including ReLU, Logistic, Tanh, Softplus and Leaky
ReLU was compared firstly. Each activation function can be expressed
Ml = β1 Ml− 1 + (1 − β1 )gl
(15) as:
Gl = β2 Gl− 1 + (1 − β2 )gl ⊙ gl {
x, x ≥ 0
ReLU(x) = (16)
where gl is the gradient of step l, β1 = 0.9 and β2 = 0.99 are the average 0, x < 0
attenuation rate.
1
Logistic(x) = x
(17)
1 + e−
3.4. Regularization of ANN
ex − e− x
Tanh(x) = (18)
Due to the strong fitting ability of ANN, the error on the training data ex + e− x

in the actual training process can often be reduced to very low, which
leads to overfitting and poor model prediction accuracy. Therefore, Softplus(x) = log(1 + ex ) (19)
regularization methods are always used to improve the generalization {
ability of the network by limiting the complexity of the model, when LeakyReLU(x) =
x, x>0
(20)
γx, x≤0
training ANN. It mainly includes introducing some constraint rules,
adding a prior and early stop, etc. The initial ANN structure was set as 1 hidden layer with 8 neurons,
In traditional machine learning, the method of improving general­ same initial weight and bias, and 500 total iteration steps. The change of
ization ability is mainly to limit the complexity of the model, such as loss function during training was shown in Fig. 8a. The training speed of
using l1 and l2 regularization methods. When training ANN, especially each activation function was: Tanh > Logistic > Softplus > ReLU >
over-parameterization, the effects of l1 and l2 regularization are not as Leaky ReLU. To reduce the influence of the initial parameters and
significant as in shallow machine learning models. validation set selection, the uniform distribution random initial pa­
Early Stop is a simple and effective regularization method for ANN. rameters were determined and the validation sets were randomly
When using the gradient descent optimization method, The validation selected in proportion. The statistical analysis was considered about
set, a sample set independent of the training set is used, and the error on model predict performance with various activation function using cor­
the validation set is used to replace the expected error. The iteration relation coefficient R, coefficient of determination R2 and mean absolute
stops when the error on the validation set is no longer decreases. error MAE, which can be calculated by
The early stopping method is used to avoid over fitting in this ( )2
research. Firstly, 5% samples of training set were selected randomly as ∑
N ̂y k yk −
∑ ∑
̂y k yk
the validation set, using the MSE of validation set instead of expectation
R=[ ( )2 ] (21)
MSE. The iterative training was stopped when the MSE of validation set ∑ ∑ [ ∑ 2 ∑ ]
N ̂y 2k − yk
̂ N yk − ( yk )2
no longer dropped for 10 fold iterative steps.

4. Results and discussion ∑( )2

yk
yk − ̂
R2 = 1 − ∑ (22)
4.1. Accelerated creep testing results of PMI (yk − y)2

The whole process test curves of PMI foams at four temperatures ( )2

1∑
were shown in Fig. 3b by conducting stepwise loading creep tests. It can MSE = yk − ̂y k (23)
N
be seen from the curves that the overall creep trend of PMI was close at
the temperature between 20∘C and 90∘C. The total deformation could be 1 ∑⃒⃒ ⃒
MAE = yk⃒
yk − ̂ (24)
divided into instantaneous deformation and creep deformation. The N
instantaneous deformation produced when the load applied at all levels,
and PMI was in the stage of elasticity deformation. Then, creep defor­ where, N is the sample size, yk is the true value of sample k, ̂ y k is the
mation increased gradually, and PMI was in the stage of creep defor­ predictive value of sample k, y is the mean of true value. The results were
mation. As the creep time increased, after decay creep, the creep rate of shown in Fig. 8. It could be seen that the prediction model accuracy of
PMI gradually stabilizes and the curve became stable until the next load each activation function was: Logistic > Tanh > Softplus > ReLU >
was applied. As the stiffness of PMI decreases with the increase of ReLU, which ReLU and Leaky ReLU functions were relatively unstable.

6
J. Zhong et al. Polymer Testing 93 (2021) 106893

Fig. 6. The treatment process of PMI creep curve at 20∘C. (a) raw strain data; (b) vertical shifting; (c) rescaled creep curves; (d) horizontal shifting; (e) converted to
log(time) scale; (f) horizontal shifting of TSSP.

Fig. 7. The master curve of PMI at reference stress and reference temperature. (a) The master curve at four temperatures; (b) The master curve at 20∘C and 1 MPa.

After the comprehensive analysis, the Logistic function was selected as was consistent, increasing the number of hidden layers would reduce the
the activation function of the ANN prediction model of the PMI long- prediction performance of the model, but the prediction stability would
term creep characteristics. be improved. In general, the overall predicted performance and stability
Then, comparison studies of the ANN structure of the prediction of the network structure under working conditions 3 and 4 with 1 hidden
model were considered with Logistic as the activation function. The layer and 6 or 8 neurons were most suitable for the long-term creep
ANN structure was described in detail as shown in Table 3. Similarly, the characteristics prediction of PMI foams.
initial parameters were determined randomly with uniform distribution, Based on the results above, an ANN fitting prediction model using
and the validation sets were randomly selected in proportion. 100 Logistic as the activation function, a single hidden layer structure with 8
groups of training were carried out for each ANN model structure, and neurons was constructed. The master curve at reference stress in Fig. 7a
the prediction performance of the model was statistically analyzed. The are considered. The creep curves at three temperatures were set as a
parameter R, MSE, R2 and MAE were used to evaluate the prediction training set, and the creep curve at the other temperature was set as a
performance, as shown in Fig. 9. prediction set, respectively. Then 100 groups of training were carried
It could be seen according to Fig. 9 that, in most cases, if the ANN out and the results were shown in Fig. 10a. The solid lines refer to the
structure was chosen reasonably, the model showed good predictive reference stress creep master curves at various temperatures obtained
ability. The hidden layer structure of ANN had a great influence on the through experiments and SSM methods, while the shaded area refers to
long-term creep characteristics prediction performance of PMI foams. the envelope range of the prediction after 100 training. Fig. 10b showed
When the number of hidden layers was 1, the prediction performance the master curve after TTSP processing using the envelope curve. The
improved with the increasing number of neurons. When the number of black solid line refers to the master curve processed by the experimental
hidden layers was 2, the prediction performance improved slightly with results, while the red line refers to the main curve constructed by each
the increasing number of neurons. When the total number of neurons temperature curve of 100 predictions. The shaded area refers to the

7
J. Zhong et al. Polymer Testing 93 (2021) 106893

Fig. 8. Comparative analysis of activation functions. (a) The change curve of loss function; (b) The statistical results of R; (c) The statistical results of R2 ; (d) The
statistical results of.MAE

function and hidden layer structure on ANN model prediction perfor­

Table 3
mance was studied, and an ideal ANN model structure form with
The structure of ANN hidden layer.
comprehensive model complexity and prediction accuracy was ob­
Condition Number of Neuron Neuron Number of tained. The main conclusions of this paper are as follows:
number of number of

hidden hidden layer hidden layer model • By conducting the gradation loading creep test at different temper­
layers (1) (2) parameters
atures, the SSM plus TTSP method can be used to construct the
1 1 2 0 11 master curve, and realize the short-term test prediction of the ma­
2 1 4 0 21 terial creep behavior on a larger time scale.
3 1 6 0 31
• When using ANN technology to predict the long-term creep proper­
4 1 8 0 41
5 2 2 2 17 ties of materials, it is only necessary to train the experimental data to
6 2 4 2 29 obtain a more accurate prediction model, which can provide a
7 2 6 2 41 convenient and effective tool for the prediction and optimization of
8 2 4 4 41
long-term creep behaviors of materials.
• The prediction model accuracy of each activation function was: Lo­
envelope master curve range of the ANN prediction curve for each gistic > Tanh > Softplus > ReLU > ReLU, which ReLU and Leaky
temperature. ReLU functions were relatively unstable on similar creep prediction
It could be seen that except for the end of the 90∘C prediction curve, problem.
the ANN model had an excellent performance. It showed that the ANN • For general nonlinear prediction problems, to avoid overfitting, the
model had a certain amount of dispersion when predicting data outside number of hidden layers should be as small as possible. Usually the
the known temperature range. However, more accurate predictions prediction performance of a single hidden layer structure is sufficient
could be obtained through statistical methods such as taking the mean for engineering applications.
value from multiple training. The ANN prediction model can reduce the • Based on the limited experimental data, the ANN model prediction
experimental conditions to a certain extent and can accurately predict technology can predict the creep behavior of materials in the interval
the master curve of long-term creep. and other working conditions in a limited range. This method can be
used to reduce the experimental conditions to a certain extent and
5. Conclusions accurately predict the master curve of long-term creep.

In this paper, by combining SSM and TTSP, the method of con­ It can be seen from the conclusions above that it is important and
structing the master curve of long-term creep characteristics of materials feasible to accurately predict the long-term creep life of materials
was first obtained. Then the ANN method was considered to predict the through advanced artificial intelligence technology. However, the cur­
long-term creep behavior of PMI foams. The influence of activation rent research only carried out the prediction of the data in the interval

8
J. Zhong et al. Polymer Testing 93 (2021) 106893

Fig. 9. The statistics parameter of each ANN structure. (a) R; (b) MSE; (c) R2 ; (d).MAE

Fig. 10. The predicted comparison of the master curve at each temperature and the final master curve. (a) The master curve at each temperature; (b) The final
master curve.

and the limited interval based on the premise of a sufficient amount of Zhendong Zhang: Validation, Investigation.
working condition data. The current research is still in the initial
exploration stage. The scope of application should be expanded in Declaration of competing interest
further research. To predict the creep behavior from the level of each
component (or thermal parameter) of the material, obtain the influence The authors declare that they have no known competing financial
of different components on the long-term creep performance and realize interests or personal relationships that could have appeared to influence
the optimization of life extension in materials and engineering area. the work reported in this paper.
Furthermore, it is possible to directly predict the long-term creep life
based on short-term experimental data through the prediction model Acknowledgements
without resorting to various existing equivalent principles. The authors
believe and expect that artificial intelligence technology can bring new This study was funded by National Natural Science Foundation of
development and power to traditional mechanics. China (No. 11902160 and No. 12002169), the Natural Science Funds for
Young Scholar of Jiangsu Province (No. BK 20170837) and the funda­
CRediT authorship contribution statement mental research funds for the central universities (NO. 309181B8807).

Jianlin Zhong: Methodology, Software, Formal analysis, Visualiza­

tion, Writing - original draft. Chunhao Yang: Conceptualization,
Methodology, Writing - review & editing. Wuning Ma: Supervision.

9
J. Zhong et al. Polymer Testing 93 (2021) 106893

References [20] Alwis, Kapugama Geeganage Nadun Chaminda, PhD thesis, in: Accelerated Testing
for Long-Term Stress-Rupture Behaviour of Aramid Fibres, University of
Cambridge, 2003.
[1] Henrik Herranen, Pabut Ott, Eerme Martin, Jüri Majak, Meelis Pohlak, Jaan Kers,
[21] S.S. Yeo, Y.G. Hsuan, Evaluation of creep behavior of high density polyethylene
Mart Saarna, Georg Allikas, Aare Aruniit, Design and testing of sandwich structures
and polyethylene-terephthalate geogrids, Geotext. Geomembranes 28 (5) (2010)
with different core materials, Mater. Sci. 18 (1) (2012) 45–50.
409–421.
[2] Ihab Grace, Valery Pilipchuk, Raouf Ibrahim, Emmanuel Ayorinde, Temperature
[22] Frédéric Achereiner, Engelsing Kurt, Martin Bastian, Peter Heidemeyer,
effect on non-stationary compressive loading response of polymethacrylimide solid
Accelerated creep testing of polymers using the stepped isothermal method, Polym.
foam, Compos. Struct. 94 (10) (2012) 3052–3063.
Test. 32 (3) (2013) 447–454.
[3] Juho T. Siivola, Minakuchi Shu, Nobuo Takeda, Effect of temperature and humidity
[23] Ioannis P. Giannopoulos, Chris J. Burgoyne, Prediction of the long-term behaviour
conditions on polymethacrylimide (PMI) foam core material and indentation
of high modulus fibres using the stepped isostress method (SSM), J. Mater. Sci. 46
response of its sandwich structures, J. Sandw. Struct. Mater. 17 (4) (2015)
(24) (2011) 7660–7671.
335–358.
[24] Ioannis P. Giannopoulos, Chris J. Burgoyne, Accelerated and real-time creep and
[4] Qu Jia, Dianwei Ju, Shenrou Gao, Jiawei Chen, Research on the dynamic
creep-rupture results for aramid fibers, J. Appl. Polym. Sci. 125 (5) (2012)
mechanical properties of polymethacrylimide foam sandwich structure, Compos.
3856–3870.
Struct. 204 (1) (2018) 22–30.
[25] Rui Miranda Guedes, A systematic methodology for creep master curve
[5] J. Zhang, Ma Xu, B.Y. Huang, Song Lv, Y.F. Zhou, J.X. Zhang, Isothermal imidation
construction using the stepped isostress method (SSM): a numerical assessment,
kinetics of polymethacrylimide based on in-situ FTIR, Materiali in tehnologije 53
Mech. Time-Dependent Mater. 22 (1) (2018) 79–93.
(6) (2019) 865–871.
[26] Ke Chang Hung, Jyh Horng Wu, Effect of SiO2 content on the extended creep
[6] Leilei Yan, Wei Jiang, Chun Zhang, Yunwei Zhang, Zhiheng He, Keyu Zhu,
behavior of SiO2-basedwood-inorganic composites derived via the sol-gel process
Niu Chen, Wanbo Zhang, Bin Han, Xitao Zheng, Enhancement by metallic tube
using the stepped isostress method, Polymers 10 (4) (2018).
filling of the mechanical properties of electromagnetic wave absorbent
[27] Ioannis P. Giannopoulos, Chris J. Burgoyne, Stepped isostress method for aramid
polymethacrylimide foam, Polymers 11 (2) (2019) 372.
fibers, in: 9th International Symposium on Fibre Reinforced Polymer
[7] Hao Zhou, Rongzhong Liu, Yubing Hu, Pu Song, Rui Guo, Quasi-static compressive
Reinforcement for Concrete Structures, 2009, pp. 15–18.
strength of polymethacrylimide foam-filled square carbon fiber reinforced
[28] Jonathon Tanks, Katherine Rader, Stephen Sharp, Takenobu Sakai, Accelerated
composite honeycombs, J. Sandw. Struct. Mater. (2) (2020), 0–17.
creep and creep-rupture testing of transverse unidirectional carbon/epoxy lamina
[8] H.T. Jeong, E. Fleury, W.T. Kim, D.H. Kim, K. Hono, Study on the mechanical
based on the stepped isostress method, Compos. Struct. 159 (2017) 455–462.
relaxations of a Zr 36 Ti 24 Be 40 amorphous alloy by time-temperature
[29] Christopher M. Bishop, Pattern Recognition and Machine Learning (Information
superposition principle, J. Phys. Soc. Jpn. 73 (11) (2004) 3192–3197.
Science and Statistics), Springer-Verlag New York, Inc., 2006.
[9] Sean R. Tunis, Joshua Benner, Mark McClellan, Comparative effectiveness
[30] Miaoquan Li, Xuemei Liu, Aiming Xiong, Prediction of the mechanical properties of
research: policy context, methods development and research infrastructure, Stat.
forged TC11 titanium alloy by ANN, J. Mater. Process. Technol. 121 (1) (2002)
Med. 29 (19) (2010) 1963–1976.
1–4.
[10] Kwang Soo Cho, Time-temperature superposition, 2016, pp. 437–457.
[31] Maridass Balasubramanian, Marissa A. Paglicawan, Zhen-Xiu Zhang, Hyo
[11] Olesja Starkova, Jinglei Yang, Zhong Zhang, Application of time stress
Lee Sung, Xin Zhen-Xiang, Kim Jin Kuk, Prediction and optimization of mechanical
superposition to nonlinear creep of polyamide 66 filled with nanoparticles of
properties of polypropylene/waste tire powder blends using a hybrid artificial
various sizes, Compos. Sci. Technol. 67 (13) (2007) 2691–2698.
neural network-genetic algorithm (GA-ANN), J. Thermoplast. Compos. Mater. 21
[12] B. Schoeberle, M. Wendlandt, C. Hierold, Long-term creep behavior of SU-8
(1) (2008) 51–69.
membranes: application of the time-stress superposition principle to determine the
[32] Mohsen Ostad Shabani, Mazahery Ali, The ANN application in FEM modeling of
master creep compliance curve, Sensor Actuator Phys. 142 (1) (2008) 242–249.
mechanical properties of Al–Si alloy, Appl. Math. Model. 35 (12) (2011)
[13] Bharat Baldewa, Yogesh M. Joshi, Delayed yielding in creep, time stress
5707–5713.
superposition and effective time theory for a soft glass, Soft Matter 8 (3) (2012)
[33] M.O. Shabani, A. Mazahery, M.R. Rahimipour, A.A. Tofigh, M. Razavi, The most
789–796.
accurate ANN learning algorithm for FEM prediction of mechanical performance of
[14] Bo Wen, Luo, Hong Chu, Wang, Rong Guo Zhao, Application of time-temperature-
alloy A356, Met. Mater. 50 (1) (2012) 25–31.
stress superposition principle to nonlinear creep of poly(methyl methacrylate), Key
[34] Hadid Mohamed, Belhi Guerira, Mebarak Bahri, Abdelkrim Zouani, Assessment of
Eng. Mater. 340–341 (2007) 1091–1096.
the stepped isostress method in the prediction of long term creep of thermoplastics,
[15] Chengkai Jiang, Han Jiang, Zhongmeng Zhu, Jianwei Zhang, Shaoyun Guo,
Polym. Test. 34 (2014) 113–119.
Ying Xiong, Application of time-temperature-stress superposition principle on the
[35] Iani Ward, John Sweeney, An Introduction to the Mechanical Properties of Solid
accelerated physical aging test of polycarbonate, Polym. Eng. Sci. 55 (10) (2015)
Polymers. 01, 2004.
2215–2221.
[36] Yann N. Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho,
[16] Zhida Li, Hongjun Liu, Rongfeng Zhang, Hui Yi, Time-temperature-stress
Surya Ganguli, Yoshua Bengio, Identifying and attacking the saddle point problem
superposition principle of PMMA’s crazing damages under creep condition, J. Cent.
in high-dimensional non-convex optimization, in: Advances in Neural Information
S. Univ. Technol. 14 (S1) (2007) 318–323.
Processing Systems, vol. 27, Curran Associates, Inc., 2014, pp. 2933–2941.
[17] J.S. Thornton, J.N. Paulson, D. Sandri, Conventional and stepped isothermal
[37] Diederik Kingma, Ba Jimmy, Adam: a method for stochastic optimization,
methods for characterizing long term creep strength of polyester geogrids, in:
International Conference on Learning Representations 12 (2014).
Proceedings of the 6th International Conference on Geosynthetics, 1998,
[38] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski,
pp. 691–698.
Aapo Kyrola, Andrew Tulloch, Yangqing Jia, Kaiming He, Accurate, Large
[18] J.S. Thornton, S.R. Allen, R.W. Thomas, D. Sandri, The stepped isothermal method
Minibatch Sgd: Training Imagenet in 1 Hour, 2017, p. 6.
for time-temperature superposition and its application to creep data on polyester
[39] Li Zhang, Fulin Wang, Ting Sun, Bing Xu, A constrained optimization method
yarn, in: Proceedings of the 6th International Conference on Geosynthetics, 1998,
based on BP neural network, Neural Comput. Appl. 29 (2) (2018) 413–421.
pp. 698–706.
[40] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein, Visualizing the
[19] K.G.N.C. Alwis, C.J. Burgoyne, Accelerated technique to predict stress-rupture
loss landscape of neural nets, in: Advances in Neural Information Processing
behaviour of aramid fibres, in: Fibre-Reinforced Polymer Reinforcement for
Systems, vol. 31, Curran Associates, Inc., 2018, pp. 6389–6399.
Concrete Structures, World Scientific Publishing Company, 2003, pp. 111–120.
[41] Ramachandran Prajit, Zoph Barret, V. Le Quoc, Searching for Activation Functions,
CoRR, 2017.

10