Usama wahab Khan

MVP,MCT, CTO @Evolution Technologies

Usama Wahab Khan
Father, data Scientist, Developer/Nerd, Traveler

Twitter : @usamawahabkhan
LinkedIn : Usamawahabkhan
Lab Overview
Prompt
• Azure Playground
Engineering

• Model Deployment
• Creating environment
• Build basic App OpenAI
Working With
• Use OpenAI PythonSDK
Open API,SDK
• Chat
• Completions
• Embeddings

• GPT Functions
• Working Vector store
Advance
• Langchain
Options
• Semantic kernel
• Hugging face and LLama
Agenda
Content Access
Introduction to AI
What is AI

Artificial intelligence (AI) is the field of

computer science that aims to create machines
and systems that can perform tasks that
normally require human intelligence and
abilities. AI can be applied to various domains,
such as natural language processing, computer
vision, robotics, gaming, healthcare, education,
and more.
Types of AI

While there are various forms of AI as it’s a broad concept, we can divide it into the following three categories based on
AI’s capabilities:

Weak AI(Narrow intelligence), which is also referred to as Narrow AI, focuses on one task. There is no self-awareness or
genuine intelligence in case of a weak AI.

iOS Siri is a good example of a weak AI combining several weak AI techniques to function. It can do a lot of things for
the user, and you’ll see how “narrow” it exactly is when you try having conversations with the virtual assistant.

Strong AI (General intelligence ), which is also referred to as True AI, is a computer that is as smart as the human
brain. This sort of AI will be able to perform all tasks that a human could do. There is a lot of research going on in this
field, but we still have much to do. You should be imagining Matrix or I, Robot here.

Artificial superintelligence , Artificial superintelligence (ASI) is a form of AI that is capable of surpassing human
intelligence by manifesting cognitive skills and developing thinking skills of its own.
Major Sub-Fields of Artificial Intelligence
Subfields of Artificial Intelligence
AI has several subfields, the prominent of which are:

• Machine Learning (ML)

This subfield of AI aims to develop algorithms that enable machines to learn from data without
explicit programming. Due to the rise of big data and the need to process and make sense of vast
amounts of information, machine learning has become increasingly important in recent years. ML,
in itself, has multiple subfields, like deep learning and supervised and unsupervised learning.

• Natural Language Processing (NLP)

Natural language processing aims to teach machines to understand and generate human language.
Speech recognition, language translation, and text analysis are some methods used in NLP.
• Neural Networks
A neural network is an algorithm inspired by the arrangement and function of the human brain. In
areas such as image recognition and speech recognition, these models are used to model complex
relationships between inputs and outputs.
Subfields of Artificial Intelligence
• Expert Systems
In contrast to conventional procedural code, expert systems solve problems in a specific domain by
reasoning about knowledge, primarily represented as ‘if-then’ rules. It refers to machines mimicking
a human expert’s decision-making intelligence.
• Robotics
As a field of study, robotics focuses on designing, developing, and programming robots that interact
with the external environment, just like humans. The field of robotics brings together knowledge
from mechanical, electrical, and computer engineering.
• Computer Vision
Computer vision aims to teach machines to interpret and understand visual data. It includes tasks
such as object recognition, facial recognition, and image and video analysis.
Introduction to Machine learning

Machine learning is a branch of artificial intelligence that enables computer systems to learn from data and
improve their performance without explicit programming. Machine learning algorithms can be classified into
three main types:
1. supervised and unsupervised. Supervised learning involves training a model with labeled data, such as
images with captions or text with sentiment labels. The model then learns to predict the correct output for
new inputs based on the learned patterns.
2. Unsupervised learning involves finding hidden structures or patterns in unlabeled data, such as clustering
similar customers or detecting anomalies. Machine learning applications span across various domains, such
as natural language processing, computer vision, recommender systems, self-driving cars, and more.
3. Reinforcement learning :
Introduction to Deep learning

Deep learning is a branch of machine learning that uses artificial neural networks to learn
from large amounts of data and perform tasks such as image recognition, natural language
processing, speech synthesis, and computer vision. Deep learning networks can have multiple
layers of neurons, each of which can perform nonlinear transformations on the input data.
The deeper the network, the more complex and abstract features it can learn. Deep learning
has achieved remarkable results in various domains, such as beating human champions in
games like Go and chess, generating realistic images and text, and enhancing medical
diagnosis and treatment.
Introduction to Deep learning
Introduction to Neural Network
A neural network is a computational model that mimics the
structure and function of biological neurons. A neural
network consists of a collection of interconnected nodes,
called artificial neurons, that process information and
transmit signals to other nodes. Each node has a set of
weights and a bias that determine how it responds to the
inputs it receives. A node computes a weighted sum of its
inputs, adds the bias, and applies a nonlinear activation
function to produce an output.

Neural networks can learn from data by adjusting their

weights and biases through a process called training.
Training involves presenting the network with a set of input-
output pairs, called training examples, and comparing the
network's output with the desired output.
Introduction to Neural Network

In a regular Neural Network there are three types of layers:

1. Input Layers: It’s the layer in which we give input to our model. The number of neurons in
this layer is equal to the total number of features in our data (number of pixels in the case
of an image).
2. Hidden Layer: The input from the Input layer is then feed into the hidden layer. There can
be many hidden layers depending upon our model and data size. Each hidden layer can
have different numbers of neurons which are generally greater than the number of
features. The output from each layer is computed by matrix multiplication of output of the
previous layer with learnable weights of that layer and then by the addition of learnable
biases followed by activation function which makes the network nonlinear.
3. Output Layer: The output from the hidden layer is then fed into a logistic function like
sigmoid or softmax which converts the output of each class into the probability score of
each class.
Introduction to Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are a type of artificial neural network that can process
images and other types of data with spatial structure. CNNs are composed of layers that
perform different operations on the input data, such as convolution, pooling, activation, and
normalization. Convolution is the core operation of CNNs, where a small filter (also called a
kernel) slides over the input data and produces a feature map that captures the local patterns
in the data

CNNs have many applications in computer vision, natural language processing, speech
recognition, and other domains. They are especially good at tasks that require recognizing
objects, faces, scenes, or words in images or audio signals. CNNs can also be used for
generative tasks, such as image synthesis, style transfer, or text generation. CNNs are trained
using gradient-based optimization methods, such as stochastic gradient descent or Adam,
and often require large amounts of labeled data and computational resources.
Introduction to Recurrent Neural Networks (RNNs)

• Recurrent Neural Networks (RNNs) are a type of artificial neural network that can process
sequential data, such as natural language, speech, or video. Unlike feedforward neural networks,
which have fixed inputs and outputs, RNNs have recurrent connections that allow them to
maintain an internal state or memory of the previous inputs. This enables them to learn from the
temporal dependencies and patterns in the data, and to generate outputs that depend on the
entire input sequence.

• RNNs are widely used for natural language processing tasks, such as machine translation, text
summarization, sentiment analysis, and speech recognition. They can also be applied to other
domains that involve sequential data, such as time series forecasting, music generation, and
video analysis. RNNs are often combined with other neural network architectures, such as
convolutional neural networks (CNNs) or attention mechanisms, to enhance their performance
and capabilities.
Introduction to Recurrent Neural Networks (RNNs)
Introduction to Generative deep learning models

• Generative deep learning models are a class of artificial neural networks that can produce
realistic data samples from a given input. For example, generative models can create images of
faces, text summaries of documents, or music compositions. Generative models are different
from discriminative models, which are trained to classify or predict an output given an input.
Generative models aim to learn the underlying distribution of the data and generate new
samples from it.
• There are different types of generative models, such as variational autoencoders (VAEs),
generative adversarial networks (GANs), and autoregressive models. VAEs are based on the idea
of encoding the input data into a latent space and decoding it back to the original space. GANs
consist of two competing networks: a generator that tries to fool a discriminator that tries to
distinguish between real and fake samples. Autoregressive models generate data one element at
a time, conditioning on the previous elements.
• Generative models have many applications in computer vision, natural language processing,
speech synthesis, and more. They can be used for data augmentation, image inpainting, style
transfer, text generation, and more. Generative models are also useful for exploring the latent
space of the data and discovering new patterns or features.
Introduction to Generative AI
Generative AI models are a class of machine learning algorithms that can create new data from existing data.
They can be used for various applications, such as image synthesis, text generation, speech synthesis, music
composition, and more. There are different types of generative AI models, depending on how they learn from the
data and how they generate new data. Some of the most common types are:
• Generative adversarial networks (GANs): These are composed of two competing neural networks, one called
the generator and the other called the discriminator. The generator tries to create realistic data that can fool
the discriminator, while the discriminator tries to distinguish between real and fake data. The two networks
learn from each other and improve their performance over time.
• Variational autoencoders (VAEs): These are a type of autoencoder, which is a neural network that can encode
and decode data. VAEs can learn a probabilistic representation of the data, which means they can capture the
variability and uncertainty in the data. They can also generate new data by sampling from the learned
distribution.
• Autoregressive models (ARMs): These are a type of neural network that can generate data sequentially, by
predicting the next element in a sequence given the previous elements. They can model complex
dependencies and patterns in the data, such as natural language or music. Some examples of ARMs are
recurrent neural networks (RNNs) and transformers.
Introduction to Generative Adversarial Networks (GANs)

• Generative Adversarial Networks (GANs) are a class of neural networks that can learn to generate
realistic and diverse data from a latent space.
• GANs consist of two components: a generator and a discriminator.
• The generator tries to produce data that resembles the true data distribution, while the
discriminator tries to distinguish between the real and fake data.
• The two networks are trained in an adversarial manner, competing against each other until they
reach an equilibrium. GANs have been applied to various domains, such as image synthesis, text
generation, style transfer, and anomaly detection.
How do GANs work?
• Generative Adversarial Networks (GANs) can be broken down into three parts:
1. Generative: To learn a generative model, which describes how data is generated in terms of a probabilistic
model.
2. Adversarial: The training of a model is done in an adversarial setting.
3. Networks: Use deep neural networks as artificial intelligence (AI) algorithms for training purposes.
• In GANs, there is a Generator and a Discriminator. The Generator generates fake samples of data(be it an
image, audio, etc.) and tries to fool the Discriminator. The Discriminator, on the other hand, tries to
distinguish between the real and fake samples. The Generator and the Discriminator are both Neural
Networks and they both run in competition with each other in the training phase. The steps are repeated
several times and in this, the Generator and Discriminator get better and better in their respective jobs after
each repetition. The work can be visualized by the diagram given below:
How do GANs work?
introduction to Encoder-Decoder Architecture

• Encoder-decoder architecture is a common framework for building neural network

models that can process sequential data, such as natural language, speech, or
images. The basic idea is to use one neural network, called the encoder, to encode
the input sequence into a fixed-length vector representation, and another neural
network, called the decoder, to generate the output sequence from the vector
representation. Encoder-decoder models can be trained end-to-end using
supervised learning, and can handle variable-length inputs and outputs.

• Encoder-decoder architecture has many applications in natural language

processing, such as machine translation, text summarization, question answering,
and dialogue generation. In this article, we will introduce the general structure and
components of encoder-decoder models, and discuss some of the challenges and
extensions of this framework.
When do we use an encoder decoder model?

1-Image Captioning
Encoder decoder models allow for a process in which a machine learning model generates a
sentence describing an image. It receives the image as the input and outputs a sequence of
words. This also works with videos.

2-Translation
This model reads an input sentence, understands the message and the concepts, then translates it
into a second language. Google Translate is built upon an encoder decoder structure
When do we use an encoder decoder model?

3-Sentiment Analysis
These models understand the meaning and emotions of the input sentence and output a
sentiment score. It is usually rated between -1 (negative) and 1 (positive) where 0 is neutral.
It is used in call centers to analyse the evolution of the client’s emotions and their reactions
to certain keywords or company discounts.
 Introduction to Image generation in AI

Define the
Image generation is a subfield of artificial intelligence problem
(AI) that focuses on creating realistic and diverse images

from various sources of input, such as text, sketches, or Deploy and Collect and
monitor the preprocess
other images. Image generation can have many model the data
applications, such as enhancing visual content, life cycle of an image generation
generating art, designing products, or creating training

data for other AI models.

Evaluate and Choose and
refine the train the
model model
Introduction to Large language models
Large language models are computational systems that can process natural language data at a massive scale.
They are trained on huge amounts of text, such as books, articles, web pages, social media posts, and more, to
learn the patterns and structures of human language. These models can then perform various tasks, such as
answering questions, generating text, summarizing documents, translating languages, and more.
Large language models have become increasingly popular and powerful in recent years, thanks to advances in
deep learning, hardware, and data availability. Some of the most well-known examples of large language models
are GPT-3, BERT, and T5. These models have millions or billions of parameters, which are the numerical values
that determine how the model processes the input and produces the output.
Introduction to Large language models
Large language models have many applications and benefits, such as improving natural language understanding,
enabling new forms of human-computer interaction, and creating novel content. However, they also pose many
challenges and risks, such as ethical, social, environmental, and technical issues. For example, large language
models may generate biased, inaccurate, or harmful content, consume a lot of energy and resources, or be
misused for malicious purposes.
Therefore, it is important to understand the principles and practices of large language models, as well as their
limitations and implications. This course will provide an introduction to large language models, covering their
history, architecture, training, evaluation, and applications. It will also discuss some of the current research topics
and open questions related to large language models.
Importance of Large Language Models in software engineering and
data engineering
Large Language Models have many applications in software engineering and data engineering, such as:
• Code generation and documentation: Large Language Models can produce executable code or documentation
from natural language specifications, reducing the time and effort required for programming tasks.
• Code analysis and debugging: Large Language Models can analyze code and detect errors, bugs, or
vulnerabilities, as well as suggest fixes or improvements, enhancing the quality and security of software
• Importance of LLMs in software engineering and data engineering
products.
• Data extraction and transformation: Large Language Models can extract relevant information from
unstructured or semi-structured data sources, such as text, images, or audio, and transform them into
structured formats, such as tables, graphs, or databases, facilitating data analysis and visualization.
• Data synthesis and augmentation: Large Language Models can synthesize new data or augment existing data
with additional features, such as labels, captions, or summaries, enriching the data sets and improving the
performance of downstream tasks.
Application of Large language models

Popular large language model (LLM) use cases include various applications for large language models across
various industries, showcasing their versatility and potential to enhance efficiency and decision-making
processes. Let us explore few unique use cases of large language models (LLM)

1. LLM Use Cases In Customer Experience And Support

Chatbots powered by LLMs enable companies to offer efficient and personalized customer service these chatbots
can talk to you like a human, helping you with everything from product information to troubleshooting.
2. Sentiment Analysis For Understanding Customer Feedback
LLMs enable sentiment analysis, allowing companies to gain insights from customer feedback. By analyzing
customer reviews, social media posts, and other textual data, LLMs can determine the sentiment expressed
towards products, services, or brand experiences.
3. LLM Use Cases In software development
LLMs can be used to assist software developers and engineers in tasks such as code generation, documentation,
debugging, testing, and refactoring. For example, GitHub Copilot
Foundation Models

From data to tasks with foundation model

Foundation Models
Foundation models are large-scale machine learning models that can perform a variety of tasks across different
domains, such as natural language processing, computer vision, and speech recognition. They are often pre-
trained on massive amounts of data and then fine-tuned for specific applications. Foundation models have shown
impressive results in many areas, but they also pose new challenges and risks, such as ethical, social, and
technical issues.
How to help LLM to solve previously unseen tasks
Q&A
Usama Wahab Khan
Twitter : @usamawahabkhan
LinkedIn : Usamawahabkhan
Thank you ☺