Computer Vision – Image

Representation (Histograms)
(Slides borrowed from various
presentations)
Image representations

• Templates
– Intensity, gradients, etc.

• Histograms
– Color, texture, SIFT descriptors, etc.
 Image Representations: Histograms

Global histogram
• Represent distribution of features
Space Shuttle
– Color, texture, depth, …
Cargo Bay
Images from Dave Kauchak
Cumulative Histograms
 Image Representations: Histograms
Histogram: Probability or count of data in each bin

• Joint histogram Marginal histogram

– Requires lots of data • Requires independent features
– Loss of resolution to • More data/bin than
avoid empty bins joint histogram
Images from Dave Kauchak
Histograms: Adaptive Binning
Marginal Histogram Example
Marginal Distribution vs Histogram?
 Histogram Clusters: Signatures

EASE Truss
Assembly

Space Shuttle
Use the same cluster centers for all images
Cargo Bay
Images from Dave Kauchak
Modelling Multi-dimensional Data
Hough transform
• What kind of a histogram does Hough
Transform use?
• What is its weakness?
• Can you improve it?
Computing histogram distance

histint(hi , h j ) = 1 - å min(hi (m), h j (m) )

K

m =1
Histogram intersection (assuming normalized histograms)

K [ h ( m) - h ( m)]2
1
c 2 (hi , h j ) = å i j

2 m=1 hi (m) + h j (m)

Chi-squared Histogram matching distance

Cars found by color histogram matching using chi-squared

Histograms: Implementation issues
• Quantization
– Grids: fast but applicable only with few dimensions
– Clustering: slower but can quantize data in higher
dimensions

Few Bins Many Bins

Need less data Need more data
Coarser representation Finer representation

• Matching
– Histogram intersection or Euclidean may be faster
– Chi-squared often works better
– Earth mover’s distance is good for when nearby bins
represent similar values
What kind of things do we compute
histograms of?

• Color

L*a*b* color space HSV color space

• Texture (filter banks or HOG over regions)

What kind of things do we compute
histograms of?
• Histograms of oriented gradients

SIFT – Lowe IJCV 2004

Orientation Normalization
• Compute orientation histogram [Lowe, SIFT, 1999]

• Select dominant orientation

• Normalize: rotate to fixed orientation

0 2p
T. Tuytelaars, B. Leibe
But what about layout?

All of these images have the same color histogram

Spatial pyramid

Compute histogram in each spatial bin

•
Spatial pyramid
Extension of a bag of features
representation
• Locally orderless representation at several levels of resolution

level 0

Lazebnik, Schmid & Ponce (CVPR 2006)

•
Spatial pyramid
Extension of a bag of features
representation
• Locally orderless representation at several levels of resolution

level 0 level 1

Lazebnik, Schmid & Ponce (CVPR 2006)

•
Spatial pyramid
Extension of a bag of features
representation
• Locally orderless representation at several levels of resolution

level 0 level 1 level 2

Lazebnik, Schmid & Ponce (CVPR 2006)

Feature Vectors & Representation Power
• How to analyze the information content in my
features? Activation heat-map , PCA, correlations

• Is there a way to visualize the representation

power of my features?
Mean, Variance, Covariance and
Correlation
Covariance Explained
Covariance Cheat Sheet
Organize your data into a set of (x, y)
• First we need to calculate the means of both x
and y independently.
Plug your variables into the formula
• Now, you have everything you need to find
the covariance of x and y. Plug your value for
n, your x and y averages, and your individual x
and y values into the appropriate spaces to
get the covariance of x and y -> Cov(x, y)
Covariance is unbounded
• Depending on the different units of x and y,
the range of values that x and y take can be far
away from each other.
• In order to normalize the covariance value, we
can use the below formula:

• This is called Correlation.

Know that a correlation of 1 indicates
perfect positive correlation.
• When it comes to correlations, your answers will always be
between 1 and -1. Any answer outside this range means that
there has been some sort of error in the calculation.
• Based on how close your correlation is to 1 or -1, you can
draw certain conclusions about your variables.
• For instance, if your covariance is exactly 1, this means that
your variables have perfect positive correlation.
• In other words, when one variable increases, the second
increases, and when one decreases the other decreases.
• This relationship is perfectly linear for variables with perfect
positive correlation — no matter how high or low the
variables get, they'll have the same relationship.
Know that a correlation of -1 indicates
perfect negative correlation.
• If your correlation is -1, this means that your
variables are perfectly negatively correlated.
• In other words, an increase in one will cause a
decrease in the other, and vice versa.
• As above, this relationship is linear. The rate at which
the two variables grow apart from each other
doesn't decrease with time.
Know that a correlation of 0 indicates no
correlation.
• If your correlation is equal to zero, this means that
there is no correlation at all between your variables.
• In other words, an increase or decrease in one will
not necessarily cause an increase or decrease in the
other with any predictability.
• There is no linear relationship between the two
variables, but there might still be a non-linear
relationship.
Know that another value between -1 and
1 indicates imperfect correlation.
• Most correlation values aren't exactly 1, -1, or 0.
• Usually, they are somewhere in between. Based on how close
a given correlation value is to one of these benchmarks, you
can say that it is more or less positively correlated or
negatively correlated.
• For example, a covariance of 0.8 indicates that there is a high
degree of positive correlation between the two variables,
though not perfect correlation. In other words, as x increases,
y will generally increase, and as x decreases, y will generally
decrease, though this may not be universally true.
An Example
Do everything this time using matrices
Data Means Deviations

Covariance
Matrix ??
Covariance Matrix

• We can interpret the variance and covariance statistics in matrix V to understand

how the various test scores vary and covary.
• Shown in red along the diagonal, we see the variance of scores for each test. The
art test has the biggest variance (720); and the English test, the smallest (360). So
we can say that art test scores are more variable than English test scores.
• The covariance is displayed in black in the off-diagonal elements of matrix V.
• The covariance between math and English is positive (360), and the covariance
between math and art is positive (180). This means the scores tend to covary in a
positive way. As scores on math go up, scores on art and English also tend to go
up; and vice versa.
• The covariance between English and art, however, is zero. This means there tends
to be no predictable relationship between the movement of English and art scores.
How to use Covariance of Features
• Can we use this valuable covariance
information among different features of our
data for something valuable?
Principal Component Analysis
• Finds the principal components of the data
 Projection onto Y axis

The data isn’t very spread out here, therefore it doesn’t have a large variance. It is
probably not the principal component.
 Projection onto X axis

On this line the data is way more spread out, it has a large variance.
Eigenvectors and Eigenvalues in PCA
• When we get a set of data points, we can deconstruct the
set’s covariance matrix into eigenvectors and eigenvalues.
• Eigenvectors and values exist in pairs: every eigenvector has a
corresponding eigenvalue.
• An eigenvector is a direction, in the example above the
eigenvector was the direction of the line (vertical, horizontal,
45 degrees etc.)
• An eigenvalue is a number, telling you how much variance
there is in the data in that direction, in the example above the
eigenvalue is a number telling us how spread out the data is
on the line.
• The eigenvector with the highest eigenvalue is therefore the
principal component.
Example
Example: 1st Principle Component
Example: 2nd Principle Component
PCA and Dimensionality Reduction
How to select the reduced dimension?
• Can I automatically select the number of
effective dimensions of my data?

• What is the extra information that I need to

specify for doing this automatically?

• Does PCA consider your data’s labels?

Some limitations of PCA
Some limitations of PCA
Data Representation Learning