SPE Polymers - 2022 - Tafreshi - Machine Learning Based Model For Predicting The Material Properties of Nanostructured PDF
SPE Polymers - 2022 - Tafreshi - Machine Learning Based Model For Predicting The Material Properties of Nanostructured PDF
DOI: 10.1002/pls2.10082
RESEARCH ARTICLE
KEYWORDS
data-driven modeling, machine learning, nanostructured aerogels, neural network
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided
the original work is properly cited.
© 2022 The Authors. SPE Polymers published by Wiley Periodicals LLC on behalf of Society of Plastics Engineers.
such as silica, have been extensively used in aerogel pro- The above studies in the field of organic aerogels sug-
cessing.[6] However, they suffer from brittleness and low gest the importance of both material and processing vari-
mechanical performance, and cannot be used for practi- ables for tailoring the nanostructure assembly of aerogels
cal applications. On the other hand, organic aerogels towards optimization of specific properties for target
such as polyimide (PI) aerogels with mechanically strong applications. From such studies, it is feasible to establish
integrity and flexible network have drawn attention in preliminary qualitative relationships, that is, empirical
recent years as an alternative for silica aerogels.[7–9] Com- models between variables and resultant properties. Nev-
pared with conventional organic aerogels, PI aerogels can ertheless, those will be only applicable to certain material
be up to 500-fold stronger.[7] types and experimental conditions. Moreover, if one
PI aerogels, as a representative class of organic aero- intends to concurrently improve and/or optimize more
gels, are typically synthesized via a sol–gel technique, than one property, empirical relationships will have
which consists of three major steps.[10] Firstly, polyamic major limitations. This is due to the sensitive relations
acid (PAA) precursor is formed from the reaction of between contributing factors, which make the structure
diamine and dianhydride monomers, which are carried to properties relations of organic aerogels very compli-
out in a basic aprotic solvent such as N- cated. Considering the rapidly growing field of organic
methylpyrrolidone (NMP). Subsequently, chemical imidi- aerogel research, there will be an emerging need for the
zation is performed using chemical dehydrating agents. design and development of novel organic aerogels with
This will be followed by the drying step to effectively optimum performance. Current attempts rely only on
remove the solvent within the network while maintain- experimental approaches which have inherent limita-
ing the solid structure to avoid shrinkage.[11] Considering tions as follows:
their synthesis route, in order to develop PI aerogels with
optimum performance, it is crucial to understand the i. The requirement of the equipment and its associated
structure to properties relations and identify the key costs, which make the fabrication process both com-
material and processing variables dictating the structural plex and costly. Apart from the materials cost, the
morphology and thus physical, thermal, mechanical, fabrication setups and required tools are extremely
optical, and electrical properties of resultant aerogels. So costly. Moreover, in order to dry the wet gels, high-
far, many groups have explored the effects of various sol– pressure chambers are required, which need to be
gel parameters, including both material and processing equipped with the process monitoring. The afore-
factors, on the resultant properties of PI aerogels, with mentioned expenses, therefore, make the entire pro-
the aim of optimizing target output for specific applica- cess uneconomical. Furthermore, to understand the
tions. Mosanenzadeh et al. could successfully tailor the structure to properties relations in aerogels and iden-
thermal insulation properties of PI aerogels.[12] A double tify key material and processing variables, in a full
dianhydride backbone strategy was employed, and parametric study, several characterizations are
through controlling the polymerization reaction, thermal required namely morphological, physical, chemical,
conductivity value was optimized to be as low as thermal, mechanical, electrical, electrochemical, and
19.7 mW m1 K1, which was 40% lower than previous optical investigations. These are the cornerstones in
studies.[12] In another study, the hydrophobicity of PI aerogel research and are the primary tools for ana-
aerogels was tailored and improved. It was shown that lyzing material behavior.[17] However, these impose
when combinations of rigid and flexible aromatic high requirements in terms of equipment and
diamines were synthesized, for PI aerogels with at least resources.[18] Another challenge is that characteriza-
50% rigid diamine, hydrophilicity was mitigated, and they tion studies are typically performed by individual
showed water-resistance.[13] Tuning the dielectric con- researchers and involve a human deciding what and
stant of PI aerogels was performed by Wu and co- how parameters are measured, along with the inter-
workers.[14] This was accomplished by incorporation of pretation of materials' behavior.[17,19]
2,2-bis-(trifluoromethyl)-4,4-diaminobiphenyl (TFMB) ii. Inefficiency due to the prolonged synthesis process
into the polymer backbone at different molar ratios. and drying stage is another limitation. Synthesis of
Compressive modulus, which represents the mechanical aerogels starts from the molecular precursors stage,
performance of PI aerogels, was also tailored and where monomers are prepared and reacted. Upon
improved. Meador et al. demonstrated that by altering fabrication of wet gels, aging is typically required for
the cross-linker and utilizing 1,3,5-benzenetricarbonyl 24–48 h to stabilize the gel network. This will be fol-
trichloride (BTC),[15] modulus of up to 74.8 MPa can be lowed by the lengthy washing process and solvent
achieved, which was greater than 2,4,6-tris(4-aminophe- exchange with a second solvent (e.g., ethanol) that is
nyl)pyridine (TAPP) cross-linked PI aerogels.[16] compatible with the supercritical drying process.
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
26 TAFRESHI ET AL.
F I G U R E 1 An overview of
experimental approach compared with
data-driven approach
After frequent ethanol to NMP exchange in a 3- to looking for patterns which will then be used for deriving
4-day period, wet gels are ready to be transferred to insights and decision-making.[32,33] Due to its high pre-
supercritical drying chambers. Drying is the final diction performance and low computational cost,
stage in the process of aerogel formation. In order to machine learning has been successfully implemented in
preserve the aerogel network and prevent shrinkage biology,[34] chemistry,[25,35,36] drug development,[37]
and structural damage, the liquid trapped in wet gel medicine,[38,39] and many optimization processes.[40–42]
pores needs to be subjected to continuous pressuriz- Despite the long history of data-driven machine learning
ing and heating to transform into a supercritical methods in these fields, data-driven research in the field
fluid.[6] This procedure needs to be performed metic- of porous materials, and specifically nanostructured aero-
ulously and may take up to 7–10 days. gels, only rose to prominence recently.[43] As an example,
iii. dependency on the expertise of the researcher is the Goodarzi et al. employed various machine learning
other drawback to consider. In fact, the role of models, namely Gaussian processes, K-nearest neighbor,
human factors and researcher expertise cannot be and radius nearest neighbor for predicting the thermal
neglected in the experimental approach.[20] The per- conductivity of selected aerogels. Root mean square error
formance of researchers in the aerogel field highly (RMSE) was considered for evaluating the performance
depends on acquiring domain-specific knowledge, and accuracy of models, and the RMSE of 0.0068 was
and since many procedures are skill-sensitive, it can achieved for the training dataset using the radius nearest
be argued that the quality of final aerogels depends neighbor model.[44]
on the expertise of the researcher. Further studies showed the application of machine
learning models, particularly artificial neural network, on
The intrinsic shortcomings of the traditional experi- optimizing the performance of glazing systems for low
mental research discussed above clearly highlight the energy buildings.[45–49] Aerogel glazing systems composed
requirement for developing new toolboxes to mitigate of exterior glazing layer, aerogel layer, and interior glazing
costs, risks, and time associated with the traditional layer were designed, and their heat transfer and solar radi-
approach (Figure 1). Recent developments in material ation transmission mechanisms were explored. The effect
informatics and data-centric approaches showed prom- of various parameters, including orientation angle, extinc-
ises in the application of machine learning-based models tion coefficient, thickness, RoC (multiplication for density
in material science for material design, material analysis, and specific heat capacity), and thermal conductivity on
and deriving invaluable insights into the key factors system outputs (total heat gain, heat flux, and indoor
underlying materials behavior.[21–28] Machine learning transmittance), was studied. It was found that neural net-
was proposed in 1959 by Samuel which drew inspiration work models were more accurate than traditional fitting
from the cognitive structure of the learning process in methods. These studies showed the successful implemen-
biological intelligence.[29] Humans, for instance, through tation of machine learning models in optimizing the per-
an iterative process learn from the outside world and use formance of devices constructed based on aerogels.
the extracted information for future decisions.[30,31] In a Nevertheless, they failed to study, predict, and optimize
similar manner, machine learning uses algorithms for the properties and performance of aerogel itself. The rela-
interpreting the extracted information from data and tionship between the synthesis parameters and material
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 27
properties has not yet been clarified, and it is crucial to T A B L E 1 PI aerogel synthesis parameters and their
identify the role of each parameter and provide a platform controllability in experimental research
for properties prediction of aerogel materials using Easily
advanced machine learning models. Furthermore, in the controllable in
above studies, due to the involvement of physical phenom- Parameter experimental
ena such as heat transfer and radiation transmission, using group Parameter design
hybrid approaches (combination of data-driven modeling Polymer Diamine chemical √
with physics-based models) was feasible. However, the backbone structure
prediction of the properties of aerogel materials involves Dianhydride chemical √
understanding the complex relationships between synthe- structure
sis parameters, intermolecular forces, resultant structure, Chemical crosslinker √
and properties. Although a knowledge-based description structure
of the prevailing phenomena can assist with feature selec- Single/hybrid √
tion, employment of physics-based models for aerogel backbone
properties prediction is challenging, and data-driven Intrinsic viscosity
methods are preferable.
Solubility parameters
Given this background, in this work, we applied an
Molecular weight
artificial neural network as an effective and high-
performance machine learning model to predict material Solvent Density
properties of nanostructured PI aerogels as a representa- Basicity
tive class of organic aerogels. Data mining and selection Surface tension
of parameters for the model were performed. Pre- Vapor pressure
processing of data was carried out and upon data parti-
Viscosity
tioning, a train dataset was used for training and refining
the machine learning model. Three different properties
Solution Polymer concentration √
namely compressive modulus, density, and porosity were Number of repeat units √
chosen as prediction targets. Results showed that there Viscosity
was an excellent correlation between true and predicted Gelation time
target values. To the best of our knowledge, multimater-
Processing Humidity
ial properties prediction through the application of the environment Temperature
machine learning model demonstrated here is the first of
its kind and can further provide the basis for establishing Pressure
a platform for data-driven materials innovation.
F I G U R E 2 (A) Physical appearance of PI aerogels; (B) morphology of PI aerogels based on single backbone; (C) morphology of PI
aerogels based on hybrid backbone. Source: Reproduced with permission from Reference [59] 2022, Royal Society of Chemistry
processing environment variables. Characteristics of the TABLE 2 Parameters selected for machine learning model
polymer backbone have perhaps the most significant
Feature
influence on the final structure and properties of aero- group Parameter Data/values
gels. The chemical structures and functional groups of
Polymer Diamine chemical ODA, DMBZ, BAPB,
the diamine, dianhydride, and cross-linker dictate their
backbone structure BAPP, PPDA
rigidity and/or flexibility, and inherent characteristics,
Dianhydride chemical BPDA, BTDA, PMDA
which play a considerable role in PI synthesis. For
structure
instance, studies showed that the electrophilicity of the
Chemical crosslinker BTC, TAB
dianhydride and nucleophilicity of the diamine have a
structure
prominent role in PAA formation.[56,57] Furthermore, the
Solution Polymer concentration 7, 8.5, 9, 10, 12, 15
aforementioned parameters can even affect the physical
appearance of the fabricated aerogels (a representative Number of repeat units 6, 10, 12, 15, 18, 20,
24, 25, 30, 40, 50,
sample is shown in Figure 2A). Another factor is the
60
choice of hybrid backbone. Recent studies showed that
by combining two different diamines and/or dianhy-
drides oligomers, there is an opportunity to achieve
hybrid backbones. In this approach, a combination of especially on mechanical modulus, and were widely inves-
rigid and flexible monomers at varying fractions is con- tigated in the literature. For instance, when polymer con-
sidered for tailoring the properties including the nano- certation varied from 5 to 10 wt.%, the compressive
structured assembly of aerogels.[12,58] Looking at modulus was reported to increase.[61] A similar trend was
representative morphology of different PI aerogels pre- observed by Meador and co-workers.[15] Other factors,
sented in Figure 2B,C, it is evident that these factors namely variation in viscosity and gelation time of solution,
could have significant impact on the resultant morphol- are rather complex and cannot be easily controlled in
ogy of nanostructured PI aerogels. Other factors, namely experimental design. As explained by Teo et al.,[62] gela-
viscosity, solubility parameters, and molecular weight of tion of PI takes place in a sequence of steps, from an
polymers, can be influential, though their effects have increase of oligomers molecular weight to polymerization-
been rarely examined. induced phase separation and formation of polymer-rich
Solvent properties including density, basicity, surface regions.[62] Sol–gel transition in PI aerogels is a compli-
tension, vapor pressure, and viscosity can affect the inter- cated phenomenon that is governed by various material
actions between the solvent and polymer chains and thus and processing parameters. Eventually, temperature,
significantly influence the imidization kinetics. Examina- humidity, and pressure of the processing environment can
tion of solvent-PAA interactions revealed that solvent can severely influence the synthesis and final product quality.
change the chain mobility and glass transition tempera- Nevertheless, as controlling them is not straightforward,
ture of the system, and greatly affect the imidization kinet- they are not commonly included in experimental design.
ics.[57,60] Therefore, differences in solvent properties are
expected to be influential. However, they are barely con-
sidered in experimental design, and in the majority of 2.2 | Dataset and basic exploratory data
cases, NMP was utilized as the main solvent due to its analysis
basic aprotic nature. Among solution parameters listed in
Table 1, polymer concentration and number of repeat The dataset used for the machine learning predictive model
units have a strong impact on PI aerogel properties, was prepared through data mining from the literature, and
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 29
TABLE 3 Display the head of the dataset with selected features and output
the selected parameters are listed in Table 2. Due to the observe the relationships between variables, 3D and 2D
strong impact of diamine, dianhydride, and cross-linker scatter plots are demonstrated in Figure 3. Dependency of
chemical structure on PI aerogel properties, these were compressive modulus (target of prediction) on other
selected from the polymer backbone category. Polymer con- parameters can be realized from the plots. In order to
centration and the number of repeat units were also consid- relate aerogels characteristics to resultant modulus, many
ered from the solution category, as they showed a studies refer to the Gibson and Ashby relationship[70]
significant effect on properties, especially mechanical mod- which demonstrates a strong correlation between density
ulus. In data mining, only data relating to single backbone and modulus of aerogels. However, such a dependency
chemistry were used and data was extracted from the litera- highly relies on the assumption that material compositions
ture.[15,58,63–69] The complete dataset is shown in Table S1. are similar. As can be seen from Figure 3B, although a gen-
Those with hybrid diamine and/or dianhydride will be eral trend exists, other materials and/or processing vari-
studied in future studies. Furthermore, PI aerogel compos- ables seem to have a strong contribution. For instance,
ites with the inclusion of fillers/fibers are not included in from Figure 3A, the impact of both polymer concentration
this study. The dataset consists of around 60 different data- and the number of repeat units on the modulus of aerogels
points and the data/values corresponding to each parame- is notable. Apart from compressive modulus, other aero-
ter are shown in Table 2. gels' characteristics, such as surface area are also highly
As an example, the first five rows of the dataset are dis- dependent on various synthesis parameters, and as can be
played in Table 3, where polymer concentration, number seen in Figure 3C, surface area data is highly scattered.
of repeat units (n), diamine, dianhydride, and cross-linker Overall, such a brief overview indicates that there is a com-
are the selected features (input variables), and the com- plex relationship between various PI aerogels synthesis
pressive modulus, density, and porosity are the target parameters and their resultant properties, which needs to
values (outputs) which we intend to predict. To get an be captured by sophisticated machine learning algorithms.
insight, some basic exploratory data analysis (EDA) was In order to prepare the structure and format of data
performed. Statistical analysis of numerical data was per- to be used in machine learning models, further prepro-
formed, and results are shown in Table 4, were mean, stan- cessing steps were also carried out. Moreover, given the
dard deviation (std), min, max, and different percentiles of volume of the dataset, in order to achieve better results
data are tabulated. To visualize the data distribution and and minimize bias of the model, k fold cross-validation
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
30 TAFRESHI ET AL.
F I G U R E 3 Data visualization: (A) 3D scatter plot of polymer concentration, number of repeat unit (n), and compressive modulus;
(B) 2D scatter plot of compressive modulus versus density; (C) 2D scatter plot of surface area versus density
technique was employed. Details of the preprocessing go through these layers and nodes, which perform a
steps, as well as k fold cross-validation procedure, can be series of linear equations, then run through the activation
found in Supplementary Information S1. function as illustrated in Figure 4A. Each input is multi-
plied by a weight factor guessed by the network, and with
the addition of a bias term, a prediction is made, as
3 | MACHINE L EARNING shown in Equation (1):
PREDICTIVE MOD E L !
X
Artificial neural networks (ANN) and response surface y¼f wi x i þ b ð1Þ
i
methodology (RSM) are two compatible tools for modeling
and prediction of numerical multivariate inputs. In a vari-
ety of fields, including material synthesis, comparative where x i is the ith input, wi is the ith weight, b is the bias,
studies have been conducted for prediction using RSM and f is the activation function, and y is the output.[30]
ANN models and it was found that ANN models, have bet- In supervised learning, the true value or label is part of
ter prediction capability than RSM or other regression the dataset. The model calculates the loss between the pre-
models.[71–75] The higher predictive capability of ANN diction and labels through backpropagation. The weights
models is attributed to its universal approximation abilities are updated accordingly to make more accurate predictions.
of nonlinear systems, whereas RSM models are confined to The complete process of the model is shown in Figure 4B.
second-degree polynomials.[74] Therefore, ANN is selected Neural networks are in fact inspired by how neurons in the
as the machine learning predictive model in this study. human brain process information. Features and importance
of information are refined in the synapses and dendrite;
then carried to other neurons by axons, eventually making
3.1 | Artificial neural network conclusions based on the information given.[30,76]
architecture
ANNs are a type of machine learning model that makes 3.2 | Topology of the neural network
predictions based on the trends given in the original data- model and hyper-parameter optimization
set. Generally, ANNs are comprised of the input layer,
hidden layers, and an output layer. In every layer, there Selection of neural network model parameters including
are one or more computation units or nodes. Data points number of layers, number of nodes per layer, activation
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 31
function, and learning rate can highly influence the Adam optimizer was used because of the findings of
model performance in predicting target properties. As Kingma and Ba, who demonstrated Adam optimizer was
stated in previous studies, identifying model parameters computationally efficient and straightforward to imple-
is not quite straightforward, as there is no fixed rule, and ment.[79] Furthermore, it has proved performance in
thus many researchers have relied on trial and error many applications, including the ones for material prop-
methods for the determination of parameters for their erty prediction.[80–82] Learning rate and number of hid-
specific case studies.[77] den layers are hyper-parameters of the model which can
In this study, the topology of the ANN model is cho- effectively contribute to the accuracy of the model.
sen as follows: the network consists of 12 input units cor- Therefore, in this work, hyper-parameter optimization
responding to the 12 features of the input data, as shown for the ANN was realized by varying the number of hid-
in Table S2. Furthermore, one hidden layer with 50 nodes den layers along with learning rate and observing their
and an output layer with one node are considered, which effects on the accuracy of model for both train and test
predicts the target properties. The activation function in datasets. As shown in Table 5, various trials were per-
each layer is the Rectified Linear Unit or ReLU, which formed, and model accuracy was evaluated using the
has recently been established to achieve better results by mean squared error (MSE) metric. It should be noted
preventing the vanishing and exploding gradient problem that while changing the number of layers and learning
in backpropagation.[78] Other factors affecting the final rate, all the other factors, such as number of nodes per
accuracy of the model are the training optimizer and layer, activation function, and other variables, were kept
hyper-parameters. constant. Also, the reported values correspond to the
The optimization algorithm can be significant in average values obtained from five different trials based
training the network due to the numerous local minima. on 5-fold cross-validation results. As an example, the
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
32 TAFRESHI ET AL.
T A B L E 5 Evaluation of ANN
Trial Learning rate Number of hidden layers MSE—train MSE—test
models with different architectures
1 0.0005 1 0.0754 0.6924
2 0.001 1 0.0185 0.5993
3 0.002 1 0.0070 0.9831
4 0.005 1 0.0010 1.0297
5 0.01 1 0.0007 1.4086
6 0.0005 2 0.0189 0.6609
7 0.001 2 0.0059 0.8268
8 0.002 2 0.0010 1.6191
9 0.005 2 0.0034 0.9490
10 0.01 2 0.0004 0.6332
11 0.0005 3 0.0101 0.9030
12 0.001 3 0.0036 0.9130
13 0.002 3 0.0023 0.6102
14 0.005 3 0.0022 1.1651
15 0.01 3 0.0009 1.1699
n 2
1X bi
performance of the model at each fold for number of MSE ¼ Yi Y ð2Þ
hidden layer of 1 and learning rate of 0.001 is demon- n i¼1
strated in Table S3.
Looking at results, it is noticed that for certain num- where n is the number of data points, Y i is the true value
ber of hidden layer, increasing the learning rate reduced of the property being predicted in the ith data point and
the train error and thus improving the train dataset pre- b i is the predicted value of the ith data point.
Y
diction. However, test error increased, which may reflect Accordingly, MAE can be calculated based on
the fact that the network could overfit the training data. Equation (3):
Therefore, it can be argued that using a very low/high
learning rate should be avoided. On the other hand, look- P
n
jyi x i j
ing at the effect of number of hidden layers on model i¼1
MAE ¼ ð3Þ
accuracy, although training error is minimized by adding n
more layers, the network started to fail to generalize for
the test data (overfitting).[83] More importantly, consider- where n is the total number of data points, x i is the true
ing the small dataset in this work, even with one hidden value of the property being predicted in the ith data point
layer, it is evident that the model can fit very well on and yi is the predicted value of the ith data point.
both train and test datasets, and adding more layers was Looking at Figure 5A,B, training loss in terms of MSE
restricted so that the chance of overfitting reduces. Over- and MAE metrics was calculated, and it can be clearly
all, the learning rate of 0.001 and one hidden layer were observed that the ANN model was successfully trained
selected as the model parameters. while overfitting and/or underfitting were restricted. Fur-
thermore, for the lower epochs, the loss of the network
during training was slightly higher than the test set. As
4 | R ES U L T S A N D D I S C U S S I O N explained in previous studies,[55] this can be attributed to
the larger size of the training set.
4.1 | Neural network model results for To visualize the accuracy, true versus prediction
compressive modulus prediction graphs for the target property (compressive modulus)
for both train and test datasets are plotted in
The performance of the ANN model was evaluated in Figure 6A,B. The diagonal lines inserted in these figures
terms of MSE and mean absolute error (MAE) values and demonstrate the ideal case where there is 100% agree-
plotted during each epoch in training as shown in ment between the predicted property values (predicted
Figure 5. The equation for calculating MSE is provided in by ANN model) and true property values (measured
Equation (2): experimentally). The results indicate that the ANN
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 33
FIGURE 5 (A and B) Training history for the ANN visualizing the MSE and MAE metrics, respectively
F I G U R E 6 True versus predicted data: (A) compressive modulus train set; (B) compressive modulus test set; (C) density train set;
(D) density test set; (E) porosity train set; (F) porosity test set
model yields predictions that are in good agreement generate meaningful results with future data. The corre-
with experimental data in both testing and training lation coefficient of the true and predicted values also
datasets. Thus, it is a generalized model and will represents the efficacy and validity of the ANN model.
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
34 TAFRESHI ET AL.
TABLE 6 Summary of MSE and Pearson correlation coefficient 4.3 | Recommended procedure for the
of all data use of neural network model
Pearson
MSE error correlation coeff. The results above showed that the developed model can
accurately predict the data. However, to further demon-
Compressive 0.0185 0.9943
Modulus—Train strate the capability of the model, it was employed to iden-
tify and explain the effects of synthesis parameters (input
Compressive 0.5993 0.6694
Modulus—Test
of the model) on resultant properties of the aerogels. For
demonstration, PPDA-PMDA PI aerogel crosslinked by
Density—Train 0.0033 0.9988
BTC was chosen as the material, and the polymer concen-
Density—Test 0.2221 0.9534
tration of 7 wt% and number of repeat units of 20 were
Porosity—Train 0.0084 0.9945 selected as the baseline. As shown in Table S4, to identify
Porosity—Test 0.1332 0.9990 the role of polymer concentration on the final properties
of aerogels, various synthesis parameters were considered
the input. Features were fed into the developed ANN
The Pearson correlation coefficient (R) was obtained model, and the compressive modulus (as the representa-
using the following equation: tive of the mechanical properties), and density (as the rep-
resentative of the physical properties) were calculated and
P P P n
n b n b plotted in Figure S2. Looking at results, it is evident that
n i¼1 Y i Y i i¼1 Y i i¼1 Y i
R ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
there is a strong relationship between polymer concentra-
h P P n 2 i P n 2 P n 2
n b b tion and predicted properties. Considering the fact that
i¼1 Y i i¼1 Y i
2
n i¼1 Y i n i¼1 Y i
these results are purely data-driven, it would be of interest
ð4Þ to confirm such a dependency based on similar experi-
mental works. Looking at previous experimental works, it
The mean squared error and Pearson correlation coef- was found that when mechanical and physical properties
ficient of the training set were equal to 0.0185 and of PI aerogels were characterized, there was direct depen-
0.9943, respectively. High correlation and low errors are dency between modulus, density, and polymer concentra-
desired for high accuracies, as is the case for this model. tion, thus confirming the obtained results from the ANN
model.[64,84] Therefore, it can be concluded that the devel-
oped model can successfully be used for studying and
4.2 | Neural network model results for understanding relations between various synthesis param-
density and porosity prediction eters and resultant properties without performing the
actual experiment.
The ANN was successfully trained and validated for From the knowledge and experience of the above-
the prediction of the compressive modulus of PI aero- described work, the developed machine learning model
gels. In order to demonstrate the versatility of the can be further used for prediction of new aerogel proper-
machine learning algorithm, two other major proper- ties, as well as synthesis of novel aerogel materials. As
ties, namely density, and porosity, were also trained demonstrated in Figure 7, upon material selection and
and predicted by the model. The original ANN for identifying the desired properties (based on target appli-
compressive modulus would not have been suitable to cation of individuals), via the developed model, users can
predict other properties, given the same input features. study the impact of many synthesis parameters on final
Therefore, density and porosity were each trained with properties of aerogels (without the necessity of synthesiz-
an identical ANN network with the same preprocessing ing each material and performing characterization stud-
techniques and model parameters. The only difference ies) and eventually decide on the optimum synthesis
is the labels of each datapoint which the network conditions. This can evidently save considerable time and
would optimize towards. A summary of MSE and Pear- cost for the development of nano-structured materials in
son correlation coefficients is shown in Table 6. The the future.
high accuracy can be further verified by the mean The data-driven approach employed in this study
squared error and the plot of true and predicted values showed promising prospects in predicting diverse proper-
shown in Figure 6C–F. The model was able to obtain ties of nanostructured aerogel materials. However, it can
slightly higher accuracy for density and porosity com- be argued that one of the challenges of this study is a lim-
pared to compressive modulus due to the smaller range ited number of data. Despite the case studies for typical
and standard deviation of data. data science projects in which various datasets in large
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 35
scale are readily available, in experimental cases (as is hyperparameter tuning, various model factors, including
the case in aerogels synthesis and other material science- the number of nodes in each layer, learning rate, and
related sectors), data collection faces great challenges. epoch, were identified. Model prediction results showed a
This is mainly due to the fact that data collection must be high correlation (Pearson correlation coefficient of up to
conducted based on the published experimental data in 0.99) along with low errors (MSE as low as 0.13), which
literature. Due to many factors such as prolong synthesis indicated high accuracy and performance of the machine
process and post-processing of samples, the available data learning model. Overall, this study can further be
for specialized cases such as PI aerogels synthesis is lim- expanded into material properties prediction of other
ited. The guideline provided in this study can act as a nanostructured materials and can pave the way for estab-
complement and facilitator to experimental approaches lishing a platform for data-driven materials innovation.
in the field of nanostructured aerogels. Future studies
can further extend available datasets for machine learn- ACKNOWLEDGMENT
ing researchers in this field. Furthermore, future studies The authors gratefully acknowledge the financial sup-
can be focused on hybrid approaches through application ports provided by the Natural Sciences and Engineering
of computational methods, such as density function the- Research Council of Canada (NSERC).
ory (DFT), combined with experimental approaches to
generate abundance of data. DA TA AVAI LA BI LI TY S T ATE ME NT
The data that support the findings of this study are available
from the corresponding author upon reasonable request.
5 | C ON C L U S I ON
ORCID
Considering the sensitive relations between contributing Hani E. Naguib https://orcid.org/0000-0003-4822-9990
material and processing parameters in nanostructured
aerogel synthesis, the structure to properties relations of RE FER EN CES
these materials are very complicated. Current attempts for [1] S. S. Kistler, Nature 1931, 127, 741.
the design and development of novel aerogels rely only on [2] S. S. Kistler, J. Phys. Chem. 2002, 36, 52.
the experimental approach, which suffers from numerous [3] S. S. Kistler, J. Phys. Chem. 2002, 39, 79.
limitations. In an attempt to address these shortcomings, [4] A. Mohaddespour, S. Atashrouz, M. T. Munir, S. Farag, K. T.
Jaradat, Chem. Eng. Sci. 2021, 229, 116123.
in this study, machine learning was employed for material
[5] X. Yue, W. Li, Z. Li, F. Qiu, J. Pan, T. Zhang, Chem. Eng. Sci.
properties prediction. Data mining and pre-processing
2020, 215, 115450.
steps were elaborated in detail, and basic EDA was dem- [6] S. Karamikamkar, H. E. Naguib, C. B. Park, Adv. Colloid Inter-
onstrated. Having been inspired by the human brain pro- face Sci. 2020, 276, 102101.
cessing structure, an artificial neural network was trained [7] B. Shi, B. Ma, C. Wang, H. He, L. Qu, B. Xu, Y. Chen, Compos.
on the prepared dataset. After many iterations and Part A Appl. Sci. Manuf. 2021, 143, 106283.
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
36 TAFRESHI ET AL.
[8] S. G. Mosanenzadeh, O. A. Tafreshi, S. Karamikamkar, Z. [36] Z. Shi, H. Liang, W. Yang, J. Liu, Z. Liu, Z. Qiao, Chem. Eng.
Saadatnia, E. Rad, M. Meysami, H. E. Naguib, Adv. Colloid Sci. 2020, 214, 115430.
Interface Sci. 2022, 304, 102646. [37] A. Lavecchia, Drug Discovery Today 2015, 20, 318.
[9] A. Paul, S. Konar, M. Chen, B. Peng, H. Li, J. Mater. Chem. [38] A. Rajkomar, J. Dean, I. Kohane, N. Engl. J. Med. 2019, 380,
2022, 100, 5088. 1347.
[10] O. A. Tafreshi, Z. Saadatnia, S. Ghaffari-Mosanenzadeh, T. [39] D. Hyun, A. Wiacek, S. Goudarzi, S. Rothlübbers, A. Asif, K.
Chen, S. Kiddell, C. B. Park, H. E. Naguib, Compos. Commun. Eickel, Y. C. Eldar, J. Huang, M. Mischi, H. Rivaz, IEEE Trans.
2022, 34, 101274. Ultrason. Ferroelectr. Freq. Control 2021, 68, 3466.
[11] Z. Saadatnia, S. G. Mosanenzadeh, T. Li, E. Esmailzadeh, [40] T. Vu-Huu, P. Phung-Van, H. Nguyen-Xuan, M. A. Wahab,
H. E. Naguib, Nano Energy 2019, 65, 104019. Comput. Math. Appl. 2018, 76, 1198.
[12] S. Ghaffari Mosanenzadeh, M. Alshrah, Z. Saadatnia, C. B. [41] H.-L. Minh, S. Khatir, M. A. Wahab, T. Cuong-Le, Eng. Struct.
Park, H. E. Naguib, Macromol. Mater. Eng. 2020, 305, 1. 2021, 242, 112412.
[13] H. Guo, M. A. B. Meador, L. McCorkle, D. J. Quade, J. Guo, B. [42] E. Samaniego, C. Anitescu, S. Goswami, V. M. Nguyen-Thanh,
Hamilton, M. Cakmak, ACS Appl. Mater. Interfaces 2012, 4, H. Guo, K. Hamdia, X. Zhuang, T. Rabczuk, Comput. Methods
5422. Appl. Mech. Eng. 2020, 362, 112790.
[14] T. Wu, J. Dong, F. Gan, Y. Fang, X. Zhao, Q. Zhang, Appl. [43] Z. Jensen, E. Olivetti, Computer Simulation of Porous Mate-
Surf. Sci. 2018, 440, 595. rials, Royal Society of Chemistry, Croydon, UK 2022, p. 265.
[15] M. A. B. Meador, C. R. Aleman, K. Hanson, N. Ramirez, S. L. [44] B. V. Goodarzi, A. R. Bahramian, J. Therm. Anal. Calorim.
Vivod, N. Wilmoth, L. McCorkle, ACS Appl. Mater. Interfaces 2021, 147, 1.
2015, 7, 1240. [45] S. Zheng, Y. Zhou, Adv. Theory Simul. 2019, 2, 1900092.
[16] D. Shen, J. Liu, H. Yang, S. Yang, Chem. Lett. 2013, 42, 1545. [46] Y. Zhou, S. Zheng, J. Cleaner Prod. 2020, 253, 119964.
[17] Z. Pei, K. A. Rozman, Ö. N. Dogan, Y. Wen, N. Gao, E. A. Holm, [47] Y. Zhou, S. Zheng, Renewable Energy 2020, 153, 375.
J. A. Hawk, D. E. Alman, M. C. Gao, Adv. Sci. 2021, 8, 2101207. [48] Y. Zhou, S. Zheng, Energy 2020, 193, 116718.
[18] J. Schmidt, M. R. G. Marques, S. Botti, M. A. L. Marques, NPJ [49] Y. Zhou, Iscience 2021, 24, 103420.
Comput. Mater. 2019, 5, 1. [50] A. Seko, A. Togo, I. Tanaka, Nanoinformatics, Springer,
[19] E. A. Holm, R. Cohn, N. Gao, A. R. Kitahara, T. P. Matson, B. Singapore 2018, p. 3.
Lei, S. R. Yarasi, Metall. Mater. Trans. A 2020, 51, 1. [51] A. Raza, S. Bardhan, L. Xu, S. S. Yamijala, C. Lian, H. Kwon,
[20] N. Charness, M. Tuffiash, Hum. Factors 2008, 50, 427. B. M. Wong, Environ. Sci. Technol. Lett. 2019, 6, 624.
[21] T. Mueller, A. G. Kusne, R. Ramprasad, Rev. Comput. Chem. [52] L. Cao, P. Zhu, Y. Zhao, J. Zhao, J. Hazard. Mater. 2018,
2016, 29, 186. 352, 17.
[22] J. Wei, X. Chu, X. Sun, K. Xu, H. Deng, J. Chen, Z. Wei, M. [53] M. Pardakhti, E. Moharreri, D. Wanik, S. L. Suib, R.
Lei, InfoMat 2019, 1, 338. Srivastava, ACS Comb. Sci. 2017, 19, 640.
[23] K. Rajan, Informatics for Materials Science and Engineering: [54] Y. Zhang, X. Xu, Polym. Chem. 2021, 12, 843.
Data-Driven Discovery for Accelerated Experimentation and [55] R. Abdusalamov, P. Pandit, B. Milow, M. Itskov, A. Rege, Soft
Application, Butterworth-Heinemann, Oxford, UK 2013. Matter 2021, 17, 7350.
[24] I. Pan, L. R. Mason, O. K. Matar, Chem. Eng. Sci. 2022, 249, [56] V. Ratta, Virginia Polytechnic Institute and State Universi-
117271. ty ProQuest Dissertations Publishing, 1999, 3000291.
[25] Y. Guan, D. Chaffart, G. Liu, Z. Tan, D. Zhang, Y. Wang, J. Li, [57] M. Ghosh, Polyimides: Fundamentals and Applications, Taylor
L. Ricardez-Sandoval, Chem. Eng. Sci. 2022, 248, 117224. and Francis, Oxfordshire, UK 2018.
[26] K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, A. Walsh, [58] S. G. Mosanenzadeh, Z. Saadatnia, F. Shi, C. B. Park, H. E.
Nature 2018, 559, 547. Naguib, Polymer (Guildf) 2019, 176, 213.
[27] C. Trinh, D. Meimaroglou, S. Hoppe, Processes 2021, 9, 1456. [59] O. A. Tafreshi, S. G. Mosanenzadeh, S. Karamikamkar, Z.
[28] J. Cai, X. Chu, K. Xu, H. Li, J. Wei, Nanosc. Adv. 2020, 2, 3115. Saadatnia, S. Kiddell, C. B. Park, H. E. Naguib, J. Mater. Chem.
[29] A. L. Samuel, IBM J. Res. Dev. 1959, 3, 210. C 2022, 10, 5088.
[30] G. Ciaburro, G. Iannace, J. Passaro, A. Bifulco, A. D. Marano, [60] X. Chen, J. Yang, J. Zhao, Polymer (Guildf) 2018, 143, 46.
M. Guida, F. Marulo, F. Branda, Appl. Acoust. 2020, 169, [61] Z. Zhang, Y. Pan, L. Gong, X. Yao, X. Cheng, Y. Deng, RSC
107472. Adv. 2021, 11, 10827.
[31] G. Ciaburro, B. Venkateswaran, Neural Networks with R: [62] N. Teo, S. C. Jana, Langmuir 2018, 34, 8581.
Smart Models Using CNN, RNN, Deep Learning, and Artificial [63] S. G. Mosanenzadeh, S. Karamikamkar, Z. Saadatnia, C. B.
Intelligence Principles, Packt Publishing Ltd, Birmingham, UK Park, H. E. Naguib, Sep. Purif. Technol. 2020, 250, 117279.
2017. [64] S. L. Vivod, M. A. B. Meador, C. Pugh, M. Wilkosz, K.
[32] C. Deng, X. Ji, C. Rainey, J. Zhang, W. Lu, Iscience 2020, 23, Calomino, L. McCorkle, ACS Appl. Mater. Interfaces 2020, 12,
101656. 8622.
[33] B. Mahesh, Int. J. Sci. Res. 2020, 9, 381. [65] R. P. Viggiano, J. C. Williams, D. A. Schiraldi, M. A. B.
[34] A. L. Tarca, V. J. Carey, X. Chen, R. Romero, S. Draghici, PLoS Meador, ACS Appl. Mater. Interfaces 2017, 9, 8287.
Comput. Biol. 2007, 3, e116. [66] M. A. B. Meador, E. J. Malow, R. Silva, S. Wright, D. Quade,
[35] J. P. Janet, H. J. Kulik, Machine Learning in Chemistry, Ameri- S. L. Vivod, H. Guo, J. Guo, M. Cakmak, ACS Appl. Mater.
can Chemical Society, Washington, DC 2020. Interfaces 2012, 4, 536.
26903857, 2023, 1, Downloaded from https://4spepublications.onlinelibrary.wiley.com/doi/10.1002/pls2.10082 by Cochrane Canada Provision, Wiley Online Library on [10/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TAFRESHI ET AL. 37
[67] M. A. B. Meador, E. McMillon, A. Sandberg, E. Barrios, N. G. [80] Q. Xie, M. Suvarna, J. Li, X. Zhu, J. Cai, X. Wang, Mater. Des.
Wilmoth, C. H. Mueller, F. A. Miranda, ACS Appl. Mater. 2021, 197, 109201.
Interfaces 2014, 6, 6062. [81] Z. Cao, Y. Dan, Z. Xiong, C. Niu, X. Li, S. Qian, J. Hu, Crystals
[68] H. Guo, O. S. Dewey, L. S. McCorkle, M. A. B. Meador, M. 2019, 9, 191.
Pasquali, ACS Appl. Polym. Mater. 2019, 1, 1680. [82] R. Cang, H. Li, H. Yao, Y. Jiao, Y. Ren, Comput. Mater. Sci.
[69] B. N. Nguyen, D. A. Scheiman, M. A. B. Meador, J. Guo, B. 2018, 150, 212.
Hamilton, L. S. McCorkle, ACS Appl. Polym. Mater. 2027, [83] P. Raut, A. Dani, presented at Adv. Comput. Technol. Appl.
2021, 3. Springer, Singapore 2020, 513.
[70] L. J. Gibson, MRS Bull. 2003, 28, 270. [84] B. N. Nguyen, M. A. B. Meador, D. Scheiman, L. McCorkle,
[71] M. Nazerian, F. Naderi, A. Partovinia, A. N. Papadopoulos, H. ACS Appl. Mater. Interfaces 2017, 9, 27313.
Younesi-Kordkheili, Forests 2021, 12, 1514.
[72] M. Khayet, C. Cojocaru, M. Essalhi, J. Membr. Sci. 2011,
368, 202.
[73] M. A. Karimi, P. Pourhakkak, M. Adabi, S. Firoozi, M. Adabi, SU PP O R TI N G I N F O RMA TI O N
M. Naghibzadeh, e-Polymers 2015, 15, 127. Additional supporting information can be found online
[74] S. Youssefi, Z. Emam-Djomeh, S. M. Mousavi, Drying Technol.
in the Supporting Information section at the end of this
2009, 27, 910.
article.
[75] K. A. Patel, P. K. Brahmbhatt, Procedia Technol. 2016, 23, 391.
[76] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT
Press, Cambridge, Massachusetts, US 2016.
How to cite this article: O. A. Tafreshi,
[77] E. Vatankhah, D. Semnani, M. P. Prabhakaran, M. Tadayon,
Z. Saadatnia, S. Ghaffari-Mosanenzadeh,
S. Razavi, S. Ramakrishna, Acta Biomater. 2014, 10, 709.
[78] C. Nwankpa, W. Ijomah, A. Gachagan, S. Marshall, arXiv Pre- S. Okhovatian, C. B. Park, H. E. Naguib, SPE
print. arXiv1811.03378 2018. Polym. 2023, 4(1), 24. https://doi.org/10.1002/pls2.
[79] D. P. Kingma, J. Ba, arXiv Preprint. arXiv1412.6980 2014. 10082