Scribd
Upload
27 views15 pages

Ai - W6L11

Uploaded by

sajidajalil63
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
27 views15 pages

Ai - W6L11

Uploaded by

sajidajalil63
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

ARTIFICIAL INTELLIGENCE

WEEK 06
LECTURE 11
TOPICS TO COVER IN THIS LECTURE

• Features: Organization of features


• Feature templates
• Sparsity in feature vectors
• Feature vector representation
FEATURES

• Features are a critical part of machine learning


which often do not get as much attention as they
deserve. Ideally, they would be given to us by a
domain expert.
• The prediction is driven by the score w (x).
• In regression, we predict the score directly, and in
binary classification, we predict the sign of the score.
• So far, we have fixed (x) and used learning to set w.
Now, we will explore how (x) affects the prediction.
ORGANIZATION OF FEATURES

Overall, the organization of features directly impacts the performance, efficiency, and
interpretability of machine learning models. Properly managing features helps in building
robust models that generalize well to unseen data. If you have specific aspects you want
to delve deeper into, let me know!

• Task: predict whether a string is an email address


• How would we go about creating good
features?

• Here, we used our prior knowledge to dene


certain features (contains @) which we
believe are helpful for detecting email
addresses.

• But this is ad-hoc: which strings should we


include? We need a more systematic way to
FEATURE TEMPLATES
• Feature templates are often used in the context of
natural language processing (NLP) and other domains
where you want to define a structure for extracting
features from raw data.
• They serve as blueprints for how to transform data into a
format that a machine learning model can understand.

• A feature template specifies how to extract relevant


information from input data. It outlines the rules or
patterns for identifying features.
• A feature template is a group of features all computed in a
• A feature template also allows us to group a set of related
features (contains @, contains a, contains b).
• This reduces the amount of burden on the feature engineer
since we don't need to know which particular characters are
useful, but only that existence of certain single characters is a
useful cue to look at.
• We can write each feature template as a English description
with a blank (--- ), which is to be labelled in with an arbitrary
string. Also note that feature templates are most natural for
combining binary features, ones which take on value 1 (true) or
0 (false).
• Note that an isolated feature (fraction of alphanumeric
characters) can be treated as a trivial feature template with no
blanks to be labelled.
• As another example, if x is a k k image, then f pixel Intensityij :
SPARSITY IN FEATURE
VECTORS
• Sparsity in feature vectors refers to the condition where a
significant number of elements in a vector are zero (or some
default value). This is common in many datasets, especially in
fields like natural language processing, image processing, and
recommendation systems.
• For example, in a vector of size 1000, if only 50 components are
non-zero, it is a sparse vector.
• Common in High-Dimensional Data:
• Storage and Efficiency: sparse data structures, which only
store non-zero elements and their indices.
• Impact on Algorithms: linear models and support vector
machines, can benefit from sparsity, as they may converge faster
when dealing with sparse data.
FEATURE VECTOR REPRESENTATIONS
• Arrays assume a fixed ordering of the features and represent the
feature values as an array. This representation is appropriate
when the number of non-zeros is significant (the features are
dense).
• Arrays are especially efficient in terms of space and speed (and
you can take advantage of GPUs Graphics Processing Unit). In
computer vision applications, features (e.g., the pixel intensity
features) are generally dense, so array representation is more
common.

• USE of map: a map (also known as a dictionary, associative array,


or hash table) is a collection of key-value pairs where each key is
unique and is used to access its corresponding value.
• when we have sparsity (few non zeros), it is typically more
effcient to represent the feature vector as a map from strings to
doubles rather than a fixed-size array of doubles.
• The features not in the map implicitly have a default value of
zero. This sparse representation is very useful in natural
• language processing, and is what allows us to work eectively over
trillions of features.
• In Python, one would dene a feature vector (x) as the dictionary
• "endsWith "+x[-3:]: 1.
• Maps do incur extra overhead compared to arrays, and therefore
maps are much slower when the features are not sparse.
QUESTIONS?

You might also like