MN907 – Multimedia Compression

Scalar quantization and

predictive coding
Giuseppe Valenzise
Laboratoire des Signaux et Systemes
Université Paris-Saclay,
CNRS, CentraleSupelec, France
[email protected]

MN907 Multimedia compression

07/11/2022
 MN907 – Multimedia Compression

Outline
• Definitions
• Uniform quantization
• Optimal scalar quantization
• Predictive coding / DPCM

Scalar quantization and prediction

 MN907 – Multimedia Compression

Basic tools for compression

• Entropy coding
• A.k.a. variable length coding (VLC)
• Removes statistical redundancies
• Lossless
• Quantization
• Introduce approximations
• Used with prediction and transform to reduce perceptual redundancy
• Enables to obtain high compression gains
• Prediction
• Alternative (and sometimes additional) method to concentrate information
• E.g., motion estimation in video
• Transform
• Concentrate information into a few coefficients
• Typically used with quantization to remove perceptual redundancy

Scalar quantization and prediction

 MN907 – Multimedia Compression

Quantization (lossy coding)

• We have seen that a discrete source can be coded without loss of

information using a number of bits per symbol that approaches the entropy
rate of the sources

• Continuous source:
• The outcome of each random variable could be any real number
• Infinite number of bits required!
• Some error (distortion) must be tolerated to represent symbols with a finite number
of bits

Scalar quantization and prediction

 MN907 – Multimedia Compression

Quantization (lossy coding)

• Quantization is the fundamental (nonlinear and non-invertible) operation to
convert a continuous random variable into a discrete one

• Scalar quantization:
• Discretize a scalar random variable
• Vector quantization:
• Discretize a vector random variable, e.g., a block of pixels at the same time

• Quantization can also be applied to already discrete sources to obtain

coarser representations
• Trade bitrate for reconstruction error

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization saturation/overload

region granular region
saturation/overload
region
Let 𝑥 be a continuous random
variable with p.d.f. 𝑓𝑥 ⋅
A scalar quantizer is described by:
• A set of decision
regions/intervals 𝐼𝑘 (𝑘 =
1,2, … 𝑀), defined by thresholds
𝑎𝑘 , such that
𝐼𝑘 ∩ 𝐼𝑗 = ∅ and ∪ 𝐼𝑘 = ℝ
• A set of reconstruction levels 𝑦𝑘
(the codebook/dictionary)
• The quantization rule
𝑞 𝑥 = 𝑦𝑘 if 𝑥 ∈ 𝐼𝑘

Scalar quantization and prediction

 MN907 – Multimedia Compression

Quantization error

• Quantization is not reversible and

produces a quantization error:

𝑒 =𝑦−𝑥 =𝑞 𝑥 −𝑥

• The quantization error is typically called

quantization noise, and modeled as an
additive noise

Scalar quantization and prediction

 MN907 – Multimedia Compression

Pulse Code Modulation (PCM)

• Scalar quantization is often referred to as PCM coding

• Originally PCM (Pulse Code Modulation) encoders were designed for the
digital transmission of telephone signals, and they included:
• sampling of the continuous time signal
• quantization of the samples
• and digital modulation

Scalar quantization and prediction

 MN907 – Multimedia Compression

Uniform scalar quantization

• A quantizer is uniform if:
• the quantization regions in the granular region have all the same length, i.e.:

𝑎𝑘+1 − 𝑎𝑘 = Δ, for 𝑘 = 2, 3, … , 𝑀 − 1

• the reproduction levels in the granular region are the midpoints of the corresponding
quantization regions, i.e.:

1 Δ
𝑦𝑘 = 𝑎𝑘+1 + 𝑎𝑘 = 𝑎𝑘 + , for 𝑘 = 2, 3, … , 𝑀 − 1
2 2

Δ Δ
and the reproduction levels in the overload region are 𝑦1 = 𝑎2 − and 𝑦𝑀 = 𝑎𝑀 +
2 2
• Δ is called the quantization step of the uniform quantizer
Scalar quantization and prediction
 MN907 – Multimedia Compression

Uniform quantization: Mid rise quantizer

• Even number of levels
• The origin lies in the
middle of a rising part
of the staircase
function

Scalar quantization and prediction

 MN907 – Multimedia Compression

Uniform quantization: Mid tread quantizer

• Odd number of levels
• The origin lies in the
middle of a tread of the
staircase function

Scalar quantization and prediction

 MN907 – Multimedia Compression

Uniform quantization: Mid tread quantizer with deadzone

• Odd number of levels
• The origin lies in the
middle of a tread of the
staircase function
• Special case:
• Mid tread uniform
quantizer with
deadzone
• The decision interval
around the origin can
have a larger size
(deadzone)

Scalar quantization and prediction

 MN907 – Multimedia Compression

Uniform quantization: example

Scalar quantization and prediction

 MN907 – Multimedia Compression

Distortion of the uniform quantizer

• Squared error between original and quantized signal:

2
𝑑 𝑥, 𝑦 = 𝑦 − 𝑥 = 𝑒2

• Mean squared error (MSE):

+∞

𝔼 𝑒 2 = න 𝑞 𝑥 − 𝑥 2 𝑓𝑥 𝑥 𝑑𝑥
−∞

Scalar quantization and prediction

 MN907 – Multimedia Compression

Distortion of the uniform quantizer

• If the input random variable 𝑥 is zero mean:
• the output 𝑦 is also zero mean
• the average distortion is the variance of the quantization error:
𝑀 𝑎𝑘+1 𝑀 𝑎𝑘+1

𝜎𝑒2 = 𝔼 𝑒 2 = ෍ න 𝑞 𝑥 −𝑥 2
𝑓𝑥 𝑥 𝑑𝑥 = ෍ න 𝑦𝑘 − 𝑥 2
𝑓𝑥 𝑥 𝑑𝑥
𝑘=1 𝑎𝑘 𝑘=1 𝑎𝑘

The error can be summed over the

disjoint decision regions

Scalar quantization and prediction

 MN907 – Multimedia Compression

Distortion of the uniform quantizer – input with uniform pdf

𝑀Δ 𝑀Δ
• If 𝑥 has uniform p.d.f. 𝑓𝑥 𝑥 = 1/(𝑀Δ) on the interval − 2 ,+ 2 :

𝑀 𝑎𝑘+1
1
𝜎𝑒2 =෍ න 𝑥 − 𝑦𝑘 2 𝑑𝑥
𝑀Δ
𝑘=1 𝑎𝑘 The quantization
Δ error is also a
𝑀 +
2 uniform random
1
= ෍ න 𝑒 2 𝑑𝑒 variable on the
𝑀Δ
𝑘=1 Δ Δ Δ
−
2 interval − , +
2 2

1 Δ3 Δ2
= 𝑀 =
𝑀Δ 12 12

Scalar quantization and prediction

 MN907 – Multimedia Compression

Rate-distortion function: input with uniform pdf

𝑀Δ 𝑀Δ 𝜎𝑥2
• If 𝑥~𝑈 − ,+ then we can rewrite 𝜎𝑒2 =
2 2 𝑀2
𝑀Δ 2
where 𝜎𝑥2 = is the quantizer input variance
12

• Let’s assume that the output symbols of the quantizer are coded with a
fixed length code with a bit rate 𝑅 = log 2 𝑀
• The error quantizer variance can be written as:

2 2
2
Δ 𝜎𝑥
𝜎𝑒 = = 2 = 𝜎𝑥2 2−2𝑅
12 𝑀

Scalar quantization and prediction

 MN907 – Multimedia Compression

Rate-distortion function: input with uniform pdf

• The error quantizer variance can be written as:

2 2
2
Δ 𝜎𝑥
𝜎𝑒 = = 2 = 𝜎𝑥2 2−2𝑅
12 𝑀

• We can express this relation in logarithmic units through the signal-to-

(quantization) noise ratio:

𝜎𝑥2
𝑆𝑁𝑅 = 10 log10 2 = 10 log10 22𝑅 ≅ 6.02 𝑅
𝜎𝑒
i.e. the distortion decreases by 6 dB for each additional bit spent.

Scalar quantization and prediction

 MN907 – Multimedia Compression

High rate approximation for a uniform quantizer

• The result obtained for a uniform pdf is valid with good approximation
• when 𝑀 is large (i.e., the quantization step is small)
• the random variable 𝑥 has a generic smooth pdf (the pdf is approximately constant
over each decision interval)
• the overload distortion is negligible

• In general, for any smooth input distribution 𝑓𝑥 , the high-rate/high-

resolution distortion-rate function of a uniform quantizer is:

𝜎𝑒2 ≈ 𝛾 2 𝜎𝑥2 2−2𝑅

𝑎𝑀−1 −𝑎2 2
where 𝛾 2 = is called the loading factor
𝜎𝑥2

Scalar quantization and prediction

 MN907 – Multimedia Compression

Performance of uniform quantization

• Uniform quantization is
optimal if the input is
uniformly distributed
• When the input distribution is
not uniform, there is a penalty
in using uniform quantization
• The penalty depends on the
loading factor
• Trade-off between the granular
error and the overload error of
the quantizer
• For Gaussian distributions,
typically 𝛾 = 4

Scalar quantization and prediction

 MN907 – Multimedia Compression

Optimal quantization
• Problem: given an input signal with p.d.f. 𝑓𝑥 , find the quantizer that
minimizes the quantization error for a given rate
• Solve the optimization problem:

𝑀 𝑎𝑘+1

arg min 𝜎𝑒2 = 𝔼 𝑒 2 = ෍ න 𝑦𝑘 − 𝑥 2

𝑓𝑥 𝑥 𝑑𝑥
𝑦𝑘
𝑘=1 𝑎𝑘

Scalar quantization and prediction

 MN907 – Multimedia Compression

Optimal quantization
• Solve the optimization problem: 𝑎𝑘+1
𝑀

arg min 𝜎𝑒2 = 𝔼 𝑒 2 = ෍ න 𝑦𝑘 − 𝑥 2

𝑓𝑥 𝑥 𝑑𝑥
𝑦𝑘
𝑘=1 𝑎𝑘
• By setting to zero the partial 𝑎derivatives of 𝜎𝑒2 w.r.t. 𝑦𝑘
𝑘+1
2
𝜕𝜎𝑒
= 2 න 𝑦𝑘 − 𝑥 𝑓𝑥 𝑥 𝑑𝑥 = 0
𝜕𝑦𝑘
𝑎𝑘
we get: 𝑎𝑘+1
‫𝑎׬‬ 𝑥 𝑓𝑥 𝑥 𝑑𝑥 Centroid
𝑘
𝑦𝑘 = 𝑎𝑘+1 = 𝔼 𝑥 ȁ𝑥 ∈ 𝐼𝑘 condition
‫𝑎׬‬ 𝑓𝑥 𝑥 𝑑𝑥
𝑘

Scalar quantization and prediction

 MN907 – Multimedia Compression

Properties of the optimal quantizer

• The quantization error is zero mean, i.e., 𝔼 𝑒 = 0
• The quantization error is uncorrelated with the quantizer
output, i.e., 𝔼 𝑒 ⋅ 𝑞(𝑥) = 0
• 𝔼 𝑥2 = 𝔼 𝑦 − 𝑒 2
= 𝔼 𝑦 2 + 𝔼 𝑒 2 − 2𝔼 𝑦𝑒 = 𝔼 𝑦 2 + 𝔼 𝑒 2
• 𝜎𝑥2 = 𝜎𝑦2 + 𝜎𝑒2
• 𝜎𝑦2 ≤ 𝜎𝑥2
• The quantization error is correlated with the quantizer input:
𝔼 𝑒 ⋅ 𝑥 = 𝔼 𝑦 − 𝑥 𝑥 = 𝔼 𝑦 2 − 𝔼 𝑥 2 = −𝔼 𝑒 2 = −𝜎𝑒2
• At high rates (small quantization step) the correlation is small

Scalar quantization and prediction

 MN907 – Multimedia Compression

Lloyd-Max optimal quantizer

• Iterative procedure to design a generic optimal quantizer (in MSE sense) for
a given pdf
• Given a pdf 𝑓𝑥 ⋅ and the number of quantization levels 𝑀, it allows to find
the optimal thresholds 𝑎𝑘 and the reconstruction levels 𝑦𝑘 that minimize
the quantization error variance

Scalar quantization and prediction

 MN907 – Multimedia Compression

Lloyd-Max optimal quantizer

• Given a pdf 𝑓𝑥 ⋅ and the number of quantization levels 𝑀, it allows to find
the optimal thresholds 𝑎𝑘 and the reconstruction levels 𝑦𝑘 that minimize
2 𝑀 𝑎𝑘+1
the quantization error variance 𝜎𝑒 = σ𝑘=1 ‫𝑎׬‬ 𝑦𝑘 − 𝑥 2 𝑓𝑥 𝑥 𝑑𝑥
𝑘
• Setting the partial derivatives of 𝜎𝑒2 w.r.t. 𝑦𝑘 and 𝑎𝑘 to zero, we obtain:

𝑦𝑘−1 +𝑦𝑘
• 𝑎𝑘∗ = Nearest neighbor condition
2

𝑎
‫𝑘 𝑎׬‬+1 𝑥𝑓𝑥 𝑥 𝑑𝑥
𝑦𝑘∗ = 𝑘
𝑎 Centroid condition
‫𝑘 𝑎׬‬+1 𝑓𝑥 𝑥 𝑑𝑥
𝑘

Scalar quantization and prediction

 MN907 – Multimedia Compression

Lloyd-Max algorithm
1. Choose an initial set of 𝑀 values for the reproduction levels 𝑦𝑘
2. Optimize the partition for the given levels using the nearest-neighbor
rule, obtaining the thresholds 𝑎𝑘
3. Optimize the set of reproduction levels for the partition found above
using the centroid criterion, obtainig a new set of 𝑦𝑘
4. Repeat steps 2 and 3 until convergence is achieved

• Similar to the k-means algorithm for clustering

Scalar quantization and prediction

 MN907 – Multimedia Compression

Lloyd-Max algorithm – example for Gaussian input

Scalar quantization and prediction

 MN907 – Multimedia Compression

High-rate approximation for LM quantizer

• Similarly as for the case of uniform quantization, it is possible to derive an
approximated distortion-rate function for the Lloyd-Max quantizer at high
bitrate [Panter&Dite, 1951]

𝐷 𝑅 ≅ 𝜖𝑥2 𝜎𝑥2 2−2𝑅

1 3
with 𝜖𝑥2 𝜎𝑥2 = ‫׬‬ℝ 𝑓𝑥 (𝑥)𝑑𝑥
12

Scalar quantization and prediction

 MN907 – Multimedia Compression

High-rate approximation for LM quantizer

𝐷 𝑅 ≅ 𝜖𝑥2 𝜎𝑥2 2−2𝑅

• 𝜖𝑥2 is called the shape factor, as it depends only on the shape of the input
pdf
• Some example values for 𝜖𝑥2 :
• 1 for uniform distribution
• 4.5 for Laplacian distribution
3𝜋
• ≅ 2.721 for Gaussian distribution
2

Scalar quantization and prediction

 MN907 – Multimedia Compression

Performance of the LM quantizer

LM quantizer Uniform quantizer

Scalar quantization and prediction

 MN907 – Multimedia Compression

Is it possible to do better than LM quantizer?

• So far, we have assumed that the output of the quantizer is coded with a
fixed-length code, using 𝑅 = log 2 𝑀 bits
• If the entropy of 𝑌 = 𝑦1 , … , 𝑦𝑀 is lower than log 2 𝑀, then LM is
suboptimal
• Solution: Entropy-Constrained Scalar Quantization (ECSQ)

𝑀 𝑎𝑘+1
Minimize 𝜎𝑒2 = σ𝑘=1 ‫𝑎׬‬ 𝑦𝑘 − 𝑥 2
𝑓𝑥 𝑥 𝑑𝑥
𝑘

with the constraint 𝐻 𝑌 = − σ𝑀

𝑘=1 𝑝𝑘 log 2 𝑝𝑘 ≤ 𝑅,

𝑎𝑘+1
where 𝑝𝑘 = 𝑃 𝑥 ∈ 𝐼𝑘 = ‫𝑎׬‬ 𝑓𝑥 𝑥 𝑑𝑥
𝑘
Scalar quantization and prediction
 MN907 – Multimedia Compression

Entropy-Constrained Scalar Quantization

• It is possible to solve the ECSQ problem using Lagrange multipliers and an
iterative algorithm similar to Lloyd-Max

• ECSQ provides a coding gain over the LM quantizer

• For a Gaussian r.v.:
2
• 𝜖𝐿𝑀 = 2.721
2
• 𝜖𝐸𝐶𝑆𝑄 = 1.423

• At high rates and for smooth pdf, the uniform quantizer is optimal

Scalar quantization and prediction

 MN907 – Multimedia Compression

Predictive coding

Scalar quantization and prediction

 MN907 – Multimedia Compression

Limits of simple scalar quantization

• Scalar quantization alone assumes that symbols are statistically
independent (memoryless source)
• In practice, this is never the case for signals of practical interest (such as
sounds, images, etc.)

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization on an image

• Original, 24 bpp

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization on an image

• Uniform quantizer, 21 bpp

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization on an image

• Uniform quantizer, 18 bpp

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization on an image

• Uniform quantizer, 9 bpp

Scalar quantization and prediction

 MN907 – Multimedia Compression

Scalar quantization on an image

• Uniform quantizer, 3 bpp

Scalar quantization and prediction

 MN907 – Multimedia Compression

Predictive coding + scalar quantization

• Rather than coding each symbol independently (memoryless coding)
predict the present symbol based on the information provided by the
previous symbols and code (quantize) the prediction residual (i.e., the
difference between the symbol and its prediction)

• Due to correlation, the prediction residual assumes small values with high
probability, thus has a smaller variance with respect to the source

• Since we have seen that the quantization error variance is proportional to

the variance of the quantizer input, better rate-distortion performance

Scalar quantization and prediction

 MN907 – Multimedia Compression

Differential Pulse Code Modulation (DPCM)

• Code the prediction residual instead of the original signal
• In order to guarantee the synchronization between encoder and decoder
(no drift), the prediction must be done on the quantized reconstruction

Scalar quantization and prediction

 MN907 – Multimedia Compression

Differential Pulse Code Modulation (DPCM)

• Basic property of DPCM:
• The quantization error on the residual 𝑑𝑛 equals the reconstruction error on the
source sample 𝑥𝑛

Scalar quantization and prediction

 MN907 – Multimedia Compression

DPCM coding gain

• The performance of a predictor is evaluated by the prediction gain ratio

𝜎𝑥2
𝐺𝐷𝑃𝐶𝑀 = 2
𝜎𝑑
𝜎𝑥2
𝐺𝐷𝑃𝐶𝑀 𝑑𝐵 = 10 log10 2
𝜎𝑑

• There is a coding gain if the variance of the prediction residual is smaller

than the variance of the original signal

Scalar quantization and prediction

 MN907 – Multimedia Compression

Example: AR(1) process

• Consider a sequence 𝑥 𝑛 that can be modeled as an auto-regressive
process of order 1:
𝑥 𝑛 =𝜌⋅𝑥 𝑛−1 +𝑧 𝑛
where 𝑧(𝑛) is a zero-mean stationary noise with variance 𝜎𝑧2
• The power of 𝑥 𝑛 is:
2
2 2
𝜎𝑧
𝜎𝑥 = 𝔼 𝑥 𝑛 =
1 − 𝜌2
• The optimal predictor in MSE sense is obtained by solving the Wiener-Hopf
equations
𝑥ො 𝑛 = 𝜌𝑥 𝑛 − 1
• The error signal is 𝑑 𝑛 = 𝑥 𝑛 − 𝜌𝑥 𝑛 − 1 = 𝑧(𝑛)

Scalar quantization and prediction

 MN907 – Multimedia Compression

Example: AR(1) process

• The DPCM coding gain turns out to be:

𝜎𝑥2 𝜎𝑧2 1 1
𝐺𝐷𝑃𝐶𝑀 = 2= 2 2
=
𝜎𝑑 1 − 𝜌 𝜎𝑧 1 − 𝜌2

• As the inter-samples correlation 𝜌 increases and tends to 1, the coding gain

increases

Scalar quantization and prediction