Artificial Intelligence in Chemical Engineering Solving Problems

- Artificial Intelligence in Chemical Engineering: Background

The current excitement about artificial intelligence (AI), particularly machine learning (ML), is
palpable and contagious. The expectation that AI is poised to “revolutionize,” perhaps even
take over, humanity has elicited prophetic visions and concerns from some luminaries.There
is also a great deal of interest in the commercial potential of AI, which is attracting significant
sums of venture capital and state-sponsored investment globally, particularly in China.
McKinsey, for instance, predicts the potential commercial impact of AI in several domains,
envisioning markets worth trillions of dollars. All this is driven by the sudden, explosive, and
surprising advances AI has made in the last 10 years or so. AlphaGo, autonomous cars,
Alexa, Watson, and other such systems, in game playing, robotics, computer vision, speech
recognition, and natural language processing are indeed stunning advances. But, as with
earlier AI breakthroughs, such as expert systems in the 1980s and neural networks in the
1990s, there is also considerable hype and a tendency to overestimate the promise of these
advances, as market research firm Gartner and others have noted about emerging
technology. It is quite understandable that many chemical engineers are excited about the
potential applications of AI, and ML in particular, for use in such applications as catalyst
design. It might seem that this prospect offers a novel approach to challenging,
long-standing problems in chemical engineering using AI. However, the use of AI in chemical
engineering is not new—it is, in fact, a 35-year-old ongoing program with some remarkable
successes along the way. This article is aimed broadly at chemical engineers who are
interested in the prospects for AI in our domain, as well as at researchers new to this area.
The objectives of this article are threefold. First, to review the progress we have made so far,
highlighting past efforts that contain valuable lessons for the future. Second, drawing on
these lessons, to identify promising current and future opportunities for AI in chemical
engineering. To avoid getting caught up in the current excitement and to assess the
prospects more carefully, it is important to take such a longer and broader view, as a “reality
check.” Third, since AI is going to play an increasingly dominant role in chemical engineering
research and education, it is important to recount and record, however incomplete, certain
early milestones for historical purposes. It is apparent that chemical engineering is at an
important crossroads. Our discipline is undergoing an unprecedented transition—one that
presents significant challenges and opportunities in modeling and automated
decision-making. This has been driven by the convergence of cheap and powerful
computing and communications platforms, tremendous progress in molecular engineering,
the ever-increasing automation of globally integrated operations, tightening environmental
constraints, and business demands for speedier delivery of goods and services to market.
One important outcome from this convergence is the generation, use, and management of
massive amounts of diverse data, information, and knowledge, and this is where AI,
particularly ML, would play an important role. So, what is AI? The term was coined in 1956 at
a math conference at Dartmouth College. Over the years, there have been many definitions
of AI, but I have always found the following to be simple, visionary, and useful: “Artificial
Intelligence is the study of how to make computers do things at which, at the moment,
people are better.” Note that this definition does not say which “things.” The implication is
that AI could eventually end up doing all “things” that humans do, and do them much
better—that is, achieve super-human performance as witnessed recently with AlphaGO and
AlphaGO Zero. This implication is sometimes called the central dogma of AI. Historically, the
term AI reflected collectively to the following branches:
• Game playing—for example, Chess, Go
• Symbolic reasoning and theorem-proving—for example, Logic Theorist, MACSYMA •
Robotics—for example, self-driving cars Vision—for example, facial recognition
• Speech recognition, Natural language processing–for example, Siri
• Distributed & evolutionary AI—for example, drone swarms, agent-based models •
Hardware for AI—for example, Lisp machines
• Expert systems or knowledge-based systems—for example, MYCIN, CONPHYDE
• ML—for example, clustering, deep neural nets, Bayesian belief nets Some of these are
application-focused, such as game playing and vision. Others are methodological, such as
expert systems and ML—the two branches that are most directly and immediately applicable
to our domain, and hence the focus of this article. These are the ones that have been
investigated the most in the last 35 years by AI researchers in chemical engineering. While
the current “buzz” is mostly around ML, the expert system framework holds important
symbolic knowledge representation concepts and inference techniques that could prove
useful in the years ahead as we strive to develop more comprehensive solutions that go
beyond the purely data centric emphasis of ML. Many tasks in these different branches of AI
share certain common features. They all require pattern recognition, reasoning, and
decision-making under complex conditions. And they often deal with ill-defined problems,
noisy data, model uncertainties, combinatorially large search spaces, nonlinearities, and the
need for speedy solutions. But such features are also found in many problems in process
systems engineering (PSE)—in synthesis, design, control, scheduling, optimization, and risk
management. So, some of us thought, in the early-1980s, that we should examine such
problems from an AI perspective. Just as it is today, the excitement about AI at that time was
centered on expert systems. It was palpable and contagious, with high expectations for AI’s
near-term potential. Hundreds of millions of dollars were invested in AI start-ups as well as
within large companies. AI spurred the development of special purpose hardware, called
Lisp machines (e.g., Symbolics Lisp machines). Promising proof-of-concept systems were
demonstrated in many domains, including chemical engineering (see below). In this phase, it
was expected that AI would have a significant impact in chemical engineering in the near
future. However, unlike optimization and model predictive control, AI did not quite live up to
its early promise. So, what happened? Why was not AI as impactful? Before addressing this
question, it is necessary to examine the different phases of AI, as I classify them, in chemical
engineering.

Big Data Analytics in Chemical Engineering

Big data analytics is the journey to turn data into insights for more informed business and
operational decisions. As the chemical engineering community is collecting more data
(volume) from different sources (variety), this journey becomes more challenging in terms of
using the right data and the right tools (analytics) to make the right decisions in real time
(velocity). This article highlights recent big data advancements in five industries, including
chemicals, energy, semiconductors, pharmaceuticals, and food, and then discusses
technical, platform, and culture challenges. To reach the next milestone in multiplying
successes to the enterprise level, government, academia, and industry need to
collaboratively focus on workforce development and innovation.
 - ADVANCES IN APPLICATIONS:
Chemical Process Industry The chemical process industry spans a large scale from
commodity chemicals, petrochemicals, refinery products, specialty chemicals, and life
sciences to consumer products. The scale of manufacturing increases from small-scale,
specialized products, such as life sciences or consumer products, to large-scale chemical
and petrochemical production facilities. Modern petrochemical and chemical complexes are
tightly integrated, with many manufacturing units geographically concentrated in one
location. As a result, incremental improvements in energy efficiency, reliability, and safety
would be amplified owing to the economy of scale. A recent survey by
PricewaterhouseCoopers reveals that 88% of chemical industry executives acknowledge
that data analytics will be crucial for retaining the competitive advantage in five years (14).
As a result, big data analytics is expected to be a key growth area in the industry.
Historically, the chemical process industry is one of the earliest adopters of computer-based
control. Safe and efficient plant operation requires constant monitoring of thousands of
process variables. Collection of process data for monitoring and control became routine and
provided a platform for exploration and development of data-driven methods and
applications. Database servers designed specifically for storing process data, such as
OSISoft PITM and AspenTech IP21TM, enabled easy and reliable access to process data by
engineers and researchers. The chemical process industry has been a pioneer in adopting
data-driven tools owing to this advantage (7). This section first highlights the ongoing
data-driven analytics efforts that have already seen success in the industry and then
discusses the emerging technologies that could potentially revolutionize analytics-driven
decision making.

Machine Learning in Chemical Engineering: A Perspective

The chemical industry must convert to using renewable energy and feedstock supply,
otherwise chemical production might become the largest driver of global oil consumption by
2030. However, renewable resources fluctuate over time and space, requiring dynamic
operation and a new paradigm for identifying new process routes and the design of flexible
plants. At the same time, the chemical companies are facing increased competition and
must ensure optimal operation and short development cycles for new processes. Facilitating
this radical change poses difficulties as conventional methods for process synthesis and
operation may not be sufficient. To make optimal decisions in complex environments, models
are conventionally developed based on mechanistic understanding and optimized. However,
the development of physicochemical models is expensive, and many phenomena cannot be
fully described by computationally tractable models.

Machine learning (ML) has the potential to overcome the limitations of mechanistic modeling
as ML methods can learn complex behaviors, the model development is cheaper, and it can
be advantageous for optimization. Chemical engineering (CE) already experienced two big
waves of ML applications between the 1980s and 2008, i.e., expert systems and (shallow)
artificial neural networks (ANNs). These waves had limited impact due to several reasons: i)
lack of data, of data accessibility, of computation power, and of programming
environments/paradigms, ii) competing successful emerging technologies for CE, in
particular, mechanistic modeling, optimization, and model predictive control.

Today, we have cheap and powerful computing, easy-to-use programming environments

(e.g., Python & TensorFlow), and a large open-source community in ML. At the same time,
ML has seen a surge in automatic feature learning by deep ANNs. This development
together with advances in hardware – most importantly GPU computing – led to
breakthrough results in image recognition, especially when based on convolutional neural
networks (CNNs), and in game playing and generally to a technology push in ML.

CE currently undergoes a transformation towards digitization and full automation of industry

and research. This leads to an ever-increasing availability of data and the need for
automated optimal decision-making based on data, allowing for more sustainable process
operations. We thus have a technology push and industry pull situation, where ML opens up
new possibilities to overcome pressing challenges in CE. In this perspective, we first review
ML methods already established in CE. Then, we identify six emerging ML challenges with
great potentials in CE.

Emerging Machine Learning Challenges in Chemical Engineering

1) Optimal Decision Making

Optimal decision making is a prominent topic in CE, for process synthesis, control, as well as
solvent, catalyst, or adsorbent selection. All these decisions need to be made based on
existing information which can be in the form of data and mechanistic knowledge, e.g.,
models. As shown in Fig.1 , optimal decision making based on data can be done by training
of data-driven or hybrid models and subsequent optimization with embedded data-driven
models.

Fig.1

Within PSE, various data-driven models have been used for regression and subsequent
process optimization. As illustrated in Fig. 2, the complexity of applied data-driven models
ranges from linear approximations to deep ANNs. For a long time, the literature focused on
linear models to approximate simulation and experimental data . Since the 1990s, shallow
ANNs have been used extensively. Shallow ANNs can theoretically approximate any
nonlinear, smooth function given a sufficient number of neurons to any given positive
accuracy on a training data set. In many PSE applications, ANNs are fitted to complete
processes (black-box approach, e.g.,), or ANNs are combined with mechanistic model
equations (hybrid modeling approach, e.g.,). Subsequently, the obtained process models
can be optimized, e.g., to identify process design. Today, the re-emergence of ML is mostly
driven by deep ANNs and big data. The deep ANNs are believed to become more important
in PSE because abundant data becomes available, e.g., through smart manufacturing, high
throughput experiments, and simulation studies. However, applications of deep ANNs are
still limited in process design in PSE.

Fig. 2

Description:
Overview of data-driven models embedded in optimization problems in CE: linear, convex
region linear surrogate model , nonlinear basis functions, e.g., ALAMO , piecewise
polynomial function, spline function, Gaussian process, support vector machine, ensemble
tree model (e.g., random forest, gradient boosted trees), ANNs with ReLU activation , and
other ANNs with more complex activation functions. Note that the models are ordered by
their estimated ability to learn complex dependencies.

While optimization of problems with linear models can be solved globally on a large scale,
e.g., for structural optimization, linear models cannot learn high-dimensional nonlinear
problems accurately. On the other hand, the consideration of more complex data-driven
models like ANNs and GPs, which could reflect high dimensional nonlinear problems better,
has long been limited to local or stochastic solution approaches. Although some tailor-made
data-driven models can be solved using state-of-the-art global solvers, these are also limited
to low-dimensional problems. A few researchers in CE and ML have developed tailored
optimization approaches for problems with ML models embedded. Mistry et al. proposed a
tailored algorithm for problems with gradient boosted trees embedded. Grimstad and
coworkers proposed an algorithm for the optimization of piecewise polynomial functions and
spline functions . Some previous works also used general-purpose global solvers to solve
optimization problems with complex surrogate models embedded but observed
computational burdens. Recently, Schweidtmann and Mitsos proposed an efficient
reduced-space optimization formulation for global optimization of deep ANNs. Notably, ANNs
with rectified linear unit (ReLU) activations have recently been reformulated as mixed-integer
linear programs (MILPs). In the MILP formulations, binary variables are introduced to divide
the domain of the piecewise linear ReLU activation functions into two linear sub-domains.
Similarly, tree models can be reformulated as MILPs. However, the number of integer
variables and constraints grows linearly with the model complexity (e.g., number of nodes in
the ANN) Overall, optimal decision-making based on data has already seen some work on
optimal process synthesis and optimal process operation. However, there are still severe
limitations in integrating learning and optimization frameworks that exhibit complex ML
models.

Promising future research in optimal decision making includes ML-assisted/embedded

optimization and ML-assisted control. In both areas, the complexity of real-life processes and
the inclusion of non-Markovian effects promises to discover insufficiencies of ML methods, to
nurture new developments, and to open promising new avenues of research in both fields.
This area has excellent synergy potential with the other five.