A brief about

Machine Learning
Prof. Kailash Singh
Department of Chemical Engineering
MNIT Jaipur
 Introduction
• Machine Learning (ML) is the field of study that gives computers the capability
to learn.
• Machine learning is data driven technology. Large amount of data is generated
by organizations on daily bases.
• Machine can learn itself from past data and automatically improve.
• From the given dataset it detects various patterns on data.
• It is similar to data mining as it also deals with the huge amount of data.
• ML is a subdomain of artificial intelligence (AI) that focuses on developing
systems that learn—or improve performance—based on the data they ingest.
• AI is a broad word that refers to systems or machines that resemble human
intelligence.
2
Contd…
• Arthur Samuel coined the term “Machine Learning ” in 1959.
• He defined machine learning as “the field of study that gives computers
the ability to learn without being explicitly programmed”
• A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance at
tasks T, as measured by P, improves with experience E.
• Example:Handwriting recognition
• Task T : Recognizing and classifying handwritten words within images
• Performance P : Percent of words correctly classified
• Training experience E : A dataset of handwritten words with given classifications
3
Contd…
• A robot driving learning problem
• Task T : Driving on highways using vision sensors
• Performance P : Average distance traveled before an error
• Training experience E : A sequence of images and steering commands
recorded while observing a human driver

4
ML in Chemical Engineering
• The conventional modeling and simulation process traditionally used
in chemical engineering is composed of the following steps:
1. Observe the physical phenomenon;
2. Assume hypotheses;
3. Generate a mathematical model from the fundamental equations;
4. Generate data from the model;
5. Compare the model with experimental data.
• The construction of model by ML reverses this order.

Ref.: Lavor et al. (2024), “Machine learning in chemical engineering: Hands-on activities,”
Education for Chemical Engineers, https://doi.org/10.1016/j.ece.2023.09.005 5
Inversion towards programming

6
AI in Engineering
• Industrial processes usually generate a large volume of data.
• Often it is needed to deal with very complex processes where usual
mathematical modelling based on phenomenological models
become nearly impossible to perform.
• In such kind of problems, Data-Driven Modeling comes in handy,
requiring a lot of data.
• It may be possible to include hybrid of phenomenological as well as AI
model.

7
 Background of AI in Engineering
• AI has made tremendous advancements in the last 15 years or so.
• AlphaGo, autonomous cars, Alexa, Watson, and other such systems,
in game playing
• Robotics, computer vision, speech recognition, and natural language
processing are stunning advances.
• But, as with earlier AI breakthroughs, such as expert systems in the
1980s and neural networks in the 1990s, there is a considerable hype
to overestimate the promise of these advances.

Ref.: V. Venkatasubramanian (2019), “The Promise of Artificial Intelligence in Chemical Engineering: Is It Here,
Finally?”, AIChE Journal. 8
Perspective in Chemical Engineering
• Many chemical engineers are excited about the potential applications of AI
and ML for various uses in such applications as catalyst design.
• However, the use of AI in chemical engineering is not new—it is, in fact, a
35-year-old ongoing program with some remarkable successes.
• Chemical Engineering is undergoing an unprecedented transition—one that
presents significant challenges and opportunities in modeling and
automated decision-making.
• This has been driven by the convergence of cheap and powerful computing
devices, tremendous progress in molecular engineering, the ever-
increasing automation of integrated operations, tightening environmental
constraints, and business demands for speedier delivery of goods and
services to market.
9
Contd…
• Artificial Intelligence is the study of how to make computers do things
at which, at the moment, people are better.
• AI could eventually end up doing all “things” that humans do, and do
them much better—that is, achieve super-human performance as
witnessed recently.

10
AI applications in various disciplines
• Game playing: Chess, Go
• Robotics: self-driving cars, etc.
• Vision: Facial recognition
• Speech recognition
• Natural language processing
• Expert systems or knowledge-based systems—for example, MYCIN,
CONPHYDE
• Clustering

11
Phase-I: Expert Systems era
• The Expert Systems Era (from the early 1980s through the mid-
1990s), saw the first broad effort to exploit AI in chemical
engineering.
• Expert systems, also called knowledge-based systems, rule based
systems, or production systems, are computer programs that mimic
the problem-solving of humans with expertise in a given domain.
• Expert problem-solving typically involves large amounts of
specialized knowledge, called domain knowledge, often in the form
of rules of thumb, called heuristics, typically learned and refined over
years of problem-solving experience.

12
Contd…
• Expert system rapidly narrows down the search by recognizing
patterns and by using the appropriate heuristics.
• A number of impressive expert systems were developed, starting with
MYCIN, an expert system for diagnosing infectious diseases
developed at Stanford University during 1972–82.
• The first expert system application in chemical engineering was
CONPHYDE, developed in 1983 by Bañares-Alcántara, Westerberg,
and Rychner at Carnegie Mellon for predicting thermo-physical
properties of complex fluid mixtures.
• Another expert system DECADE was developed in 1985 at CMU for
catalyst design.
13
• The first course on AI in PSE was developed and taught at Columbia
University in 1986 and it was subsequently offered at Purdue University
for many years.
• In 1986, Stephanopoulos published an article titled, “Artificial Intelligence
in Process Engineering”, in which he discussed the potential of AI in process
engineering and outlined a research program to realize it.
• The first AIChE session on AI was organized by Gary Powers (CMU) at the
annual meeting held in Chicago in 1985.
• The first international conference, Intelligent Systems in Process
Engineering (ISPE’95), sponsored by the Computer Aids for Chemical
Engineering (CACHE) Corporation, was organized by Stephanopoulos,
Davis, and Venkatasubramanian, in July 1995.
14
Phase II: Neural Networks Era (1990-2008)
• NN era was started in 1990 onward. NN acquired large amount of
data easing maintenance and development of models.
• Backpropagation algorithm gained popularity for training feedforward
NN to learn hidden patterns in input-ouput patterns. This algorithm
used gradient descent search method to adjust the weights of
connections between nodes iteratively.

15
Contd…
• Researchers made substantial progress on addressing challenging
problems in modeling, fault diagnosis, control, and product design.
• There were hundreds of articles in this domain during this phase.
• Despite the surprising success of neural networks in many practical
applications, some especially challenging problems in vision, natural
language processing, and speech understanding remained beyond
the capabilities of the neural nets of this era.
• Some researchers thought of need of neural nets with more hidden
layers to solve such problems.

16
Current Scenario
• The progress of AI over the last decade or so has been very exciting, and
the resource limitations mentioned above are largely gone now.
• Implementation difficulties have been greatly diminished.
• people have started to trust and accept recommendations from AI-assisted
systems such as ChatGPT, Google Gemini, Alexa, etc.
[Practice question for students: Make a report on the AI assisted systems
available today]
• In 1985, the most powerful computer was the CRAY-2 supercomputer. Its
computational speed was 1.9 gigaflops and it consumed 150 kW of power.
• Today Apple Watch performs at 3 gigaflops, while consuming just 1 W of
power. Normal desktops usually perform at 100 gigaflops.

17
Contd…
• There have been dramatic advances in software—in the performance of
algorithms and in high-level programming environments such as MATLAB,
Mathematica, Python, Julia, and TensorFlow.
• We have also seen great progress in wireless communication technologies.
• The other critical development is the availability of tremendous amounts of
data, “big data”, in many domains, which made possible the stunning
advances in ML.
• history would recognize that there were three early milestones in AI: One is
Deep Blue (IBM) defeating Gary Kasparov in chess in 1997, the second
Watson (IBM) becoming Jeopardy champion in 2011, and the third is the
surprising win by AlphaGO in 2016.

18
Phase III: Deep learning and the data science
era (~2005 to present)
• The era of Data Science or Predictive Analytics was made possible by
three important ideas:
• deep or convolutional neural nets (CNNs),
• reinforcement learning, and
• Statistical ML
• These technologies are behind the recent AI success stories in game
playing, natural language processing, robotics, and vision.
• Unlike neural nets of the 1990s, which typically had only one hidden
layer of neurons, deep neural nets have multiple hidden layers. Such
an architecture has the potential to extract features hierarchically for
complex pattern recognition.

19
Contd…
• The breakthrough came in 2006 by using a layer-by-layer training
strategy coupled with considerable increase in processing speed.
• A procedure called convolution in the training of the neural net made
feature extraction feasible.
• Convolution is a filtering technique, well-known in signal processing.
• Another architectural innovation was the recurrent neural network.
• The second important idea, reinforcement learning, can be thought
of as a scheme for learning a sequence of actions to achieve a desired
outcome, such as maximizing an objective function.

20
Contd…
• The third key idea, statistical ML, is the combination of mathematical
methods from probability and statistics with ML techniques.
• This led to a number of useful techniques such as LASSO, Support
Vector Machines, random forests, clustering, and Bayesian belief
networks.

21
Advice for future chemical engineers
• Developing AI methods is not just a matter of tracking new
developments in computer science and merely applying them in
chemical engineering.
• While there are some easy gains to be made, many intellectually
challenging problems in our industry are not amenable to AI solutions
yet. Nor can they be solved by computer scientists, as they lack our
domain knowledge.
• Since transport phenomena problems are solved by using differential
equations, mathematicians would address these core chemical
engineering problems.

22
Contd…
• To find the solution in chemical engineering, has to be well educated
in AI fundamentals, going beyond merely running ML code.
• Our UG and PG students need to become familiar with applied AI
techniques if we want keeps ourselves in AI race.
• So today’s engineering graduates should be well adapted to AI
techniques but be well acquainted with fundamentals of chemical
engineering to rightly apply these techniques.

23
 The future of chemical engineering in the era
of Generative AI
• AI promises to revolutionize the way we work by automating labor-
intensive tasks and allowing complex processes to be completed in a
fraction of the time.
• Recently, ChatGPT provided a boost to pharmaceutical companies by
interacting with other AI tools and providing a more user-friendly
interface.
• This allowed ChatGPT to answer questions based on information
provided by the other tools rather than making users spend time
searching for the information themselves.
• By being integrated into the wider platform of AI tools, ChatGPT has
been able to improve the value of the entire system.
Ref.: Jin Xuan and Thorin Daniel (2023) The Future of Chemical Engineering in the Era of Generative AI, The Chemical Engineer.
24
Contd…
• Large language models (LLMs) have been used by students to solve
chemical engineering problems such as turbine efficiency and
reaction time.
• If these tools are taken from the classroom into industry, a whole
new set of opportunities for chemical engineering design will open
up.
• The necessity of coding ability in chemical engineering has been
established for some time, but many chemical engineers have limited
coding ability.
• Generative AI can automate the creation of small functions and
portions of programs by generating chunks of code.
25
Limitations of using ChatGPT in chemical
engineering
• While AI is powerful, it is limited in several significant ways.
• Many implementations of generative AI are hampered by a lack of
suitable data and an inability to accept different input data types.
• Recent attempts to design generative AI for P&ID automation have
shown promise but are hindered by the need for improved
comprehension of chemical engineering-specific terms and a lack of
available P&IDs to learn from.
• If complex chemical engineering models are to be successfully
developed, much more data is required which may prove difficult to
obtain.
26
• Chemical engineers of the future may use a suite of AI technologies to design
a chemical plant in a fraction of the time it currently takes.
• Future AI will integrate several different kinds of models into one combined
framework, able to parse information and generate results in one go.
• Unit design requires flowchart and P&ID creation, before an economic
analysis occurs. These steps could be condensed with the use of a set of AI
models.
• While the use of AI may speed up the design process, the results will still
need to be understood and the human aspect cannot be removed.
• So, these tools will not replace the skills of an engineer but allow them to
work with greater efficiency and flexibility.
• Overall, the chemical engineers are supposed to use AI tools with complete
understanding of the process.
27
An introduction to Machine Learning

28
Types of Machine Learning
• Supervised learning
• Unsupervised learning
• Semi-supervised learning
• Reinforcement learning

Ref.: Trinh, C.; Meimaroglou, D.; Hoppe, S. “Machine Learning in Chemical Product Engineering: The State of
the Art and a Guide for Newcomers”, Processes, 2021, 9, 1456. https://doi.org/10.3390/pr9081456
29
(a) Supervised learning
• The model is trained on labelled dataset. It learns by inputs and their
desired outputs and then finds patterns and connections between the
input and the output.
• The goal is to learn a general rule that maps inputs to outputs.
• The training process continues until the model achieves the desired
level of accuracy on the training data.
• There are two categories of supervised learning:
• Classification
• Regression

30
Classification algorithms
• Classification deals with predicting categorical target variables, which represent
discrete classes or labels.
• For example, classifying emails as spam or not spam,
or predicting whether a patient has a high risk of heart disease.
• Classification algorithms learn to map the input features to one of the predefined
classes.
• Some classification algorithms are:
• Logistic Regression
• Support Vector Machine
• Random Forest
• Decision Tree
• K-Nearest Neighbors (KNN)
• Naive Bayes

31
Regression algorithms
• Regression deals with predicting continuous target variables, which
represent numerical values.
• For example, predicting the price of a house based on its size, location, and
amenities, or forecasting the sales of a product.
• Regression algorithms learn to map the input features to a continuous
numerical value.
• Some regression algorithms are:
• Linear Regression
• Polynomial Regression
• Ridge Regression
• Lasso Regression
• Decision tree
• Random Forest

32
Applications of Supervised Learning
• Image classification: Identify objects, faces, and other features in images.
• Natural language processing: Extract information from text, such as sentiment,
entities, and relationships.
• Speech recognition: Convert spoken language into text.
• Recommendation systems: Make personalized recommendations to users.
• Predictive analytics: Predict outcomes, such as sales, customer churn, and stock
prices.
• Medical diagnosis: Detect diseases and other medical conditions.
• Autonomous vehicles: Recognize and respond to objects in the environment.
• Email spam detection
• Weather forecasting
33
(b) Unsupervised Machine Learning

• Unsupervised learning algorithm discovers patterns and relationships using

unlabeled data.
• The primary goal of Unsupervised learning is often to discover hidden
patterns, similarities, or clusters within the data, which can then be used
for various purposes, such as data exploration, visualization, dimensionality
reduction, and more.
• For example, we have mixed data of images of elephant, cats, dogs. Classify
the items based on their similarities.
• The most common problems are related to dimensionality reduction and
clustering
• Dimensionality reduction is used for the compression of large data sets.
Principal component analysis (PCA) is the most popular algorithm of this
family.

34
Contd…
• Clustering refers to the process of identification of existing clusters in
the input data. The clusters are groups of data that present a relative
similarity with respect to a specific characteristic.
• K-means clustering is a popular clustering algorithm, mainly due to its
ease in application and its low level of mathematical complexity.
• Other unsupervised learning algorithms are:
• Autoencoders (AE)
• Hierarchical clustering analysis (HCA),
• Independent component analysis (ICA)
• Gaussian mixture model (GMM)

35
Applications of Unsupervised Learning
• Clustering: Group similar data points into clusters.
• Anomaly detection: Identify outliers or anomalies in data.
• Dimensionality reduction: Reduce the dimensionality of data while preserving its
essential information.
• Recommendation systems: Suggest products, movies, or content to users based
on their historical behavior or preferences.
• Topic modeling: Discover latent topics within a collection of documents.
• Density estimation: Estimate the probability density function of data.
• Image and video compression: Reduce the amount of storage required for
multimedia content.
• Community detection in social networks: Identify communities or groups of
individuals with similar interests or connections.
• Content recommendation: Classify and tag content to make it easier to
recommend similar items to users. 36
(c) Semi-supervised Learning
• In semi-supervised learning, the data set is generally composed of a
small amount of labeled data and a majority of unlabeled data.
• It’s particularly useful when obtaining labeled data is costly, time-
consuming, or resource-intensive.
• This technique is mostly applicable in the case of image data sets
where usually all images are not labeled.

37
Semi-supervised learning methods
• Graph-based semi-supervised learning: This approach uses a graph to represent the
relationships between the data points. The graph is then used to propagate labels from
the labeled data points to the unlabeled data points.
• Label propagation: This approach iteratively propagates labels from the labeled data
points to the unlabeled data points, based on the similarities between the data points.
• Co-training: This approach trains two different machine learning models on different
subsets of the unlabeled data. The two models are then used to label each other’s
predictions.
• Self-training: This approach trains a machine learning model on the labeled data and
then uses the model to predict labels for the unlabeled data. The model is then retrained
on the labeled data and the predicted labels for the unlabeled data.
• Generative adversarial networks (GANs): GANs are a type of deep learning algorithm
that can be used to generate synthetic data. GANs can be used to generate unlabeled
data for semi-supervised learning by training two neural networks, a generator and a
discriminator.

38
Applications of Semi-Supervised Learning
• Image Classification and Object Recognition: Improve the accuracy of models by
combining a small set of labeled images with a larger set of unlabeled images.
• Natural Language Processing (NLP): Enhance the performance of language
models and classifiers by combining a small set of labeled text data with a vast
amount of unlabeled text.
• Speech Recognition: Improve the accuracy of speech recognition by leveraging a
limited amount of transcribed speech data and a more extensive set of unlabeled
audio.
• Recommendation Systems: Improve the accuracy of personalized
recommendations by supplementing a sparse set of user-item interactions
(labeled data) with a wealth of unlabeled user behavior data.
• Healthcare and Medical Imaging: Enhance medical image analysis by utilizing a
small set of labeled medical images alongside a larger set of unlabeled images.

39
(d) Reinforcement Learning
• It focuses on learning a series of actions for optimizing an overall reward—
for example, winning at a game of chess.
• RL is centered around the concept of learning by interaction. It means that
RL learns from interactions with an environment to maximize a reward
function.
• Correct labels for learning a series of actions are not known upfront in RL—
instead, they need to be learned through interactions with the
environment
• With RL, the model (also called an agent) interacts with its environment,
and by doing so generates a sequence of interactions that are together
called an episode.

40
Contd…
• For example, imagine that we want to teach a computer to play the
game of chess and win against human players.
• The labels (rewards) for each individual chess move made by the
computer are not known until the end of the game, because during
the game itself, we don't know whether a particular move will result
in winning or losing that game.
• It would be a positive reward given if the computer won the game,
because the agent had achieved the overall desired outcome; and
vice versa, a negative reward if the computer had lost the game.

41
Application of Reinforcement Learning
• Robotics: Robots with pre-programmed behavior are useful in
structured environments, such as the assembly line of an automobile
manufacturing plant, where the task is repetitive in nature.
• A master chess player makes a move. The choice is informed both by
planning, anticipating possible replies and counter replies.
• An adaptive controller adjusts parameters of a petroleum refinery’s
operation in real time.

42
 Comparison
Reinforcement learning
Unsupervised
Supervised learning learning

Definition Makes predictions Segments and groups Reward-punishment

from data data system and
interactive
environment

Types of data Labeled data Unlabeled data Acts according to a

policy with a final
goal to reach (No or
predefined data)

Types of problems Regression and Association and Series of actions

classification Clustering
43
 Reinforcement
Unsupervised
learning
Supervised learning learning

Supervision Extra supervision No No supervision

Algorithms Linear Regression, K – Means Q – Learning,

Logistic Regression, clustering, SARSA
SVM, KNN and so
C – Means, Apriori
forth

Aim Calculate outcomes Discover underlying Learn a series of

patterns action

Application Risk Evaluation, Recommendation Self-Driving Cars,

Forecast Sales System, Anomaly Gaming, Healthcare
Detection
44
 Stephen Hawking quote
• "I fear that AI may replace
humans altogether. If people
design computer viruses,
someone will design AI that
improves and replicates itself”.
• “AI will eventually reach a level
where it will essentially be a
new form of life that will
outperform humans.”

45