AMERICAN UNIVERSITY OF SCIENCE AND TECHNOLOGY

FACULTY OF ENGINEERING
DEPARTMENT OF COMPUTER AND COMMUNICATIONS ENGINEERING

QUANTIZATION
Digital Communications Systems

CCE 411

AUST – 2022

Dr. Gaby Abou Haidar

 GABY ABOU HAIDAR, PH.D.

Quantization
 Discrete sources are a subject of interest in their own right(for text, computer files, etc.) and also serve as the
inner layer for encoding analog source sequences and waveform sources.

 This section treats coding and decoding for a sequence of analog values.

 Source coding for analog values is usually called quantization.

 Quantization is the process of representing a large, possibly infinite, set of values with a smaller set.
 GABY ABOU HAIDAR, PH.D.

Quantization
 The second step in digitizing an analog signal is to quantize the amplitude of the sampled signal xs(t).

 Quantization is the process of mapping a set of continuous amplitudes (infinite number of levels) into a finite number
of discrete values.

 Obviously, this is a many-to-one mapping and, thus, in general we cannot recover exactly the analog signal from its
quantized version. We can, however, through appropriate design, minimize this distortion. We will refer to the
distortion introduced by quantization as quantization noise.

 Let us assume that the analog signal to be quantized has amplitudes in the range −Vp ≤ V ≤ Vp volts, and that we map
all voltages in [−Vp;Vp] into L discrete levels.

 The obvious question (which does not in general have an obvious answer) is: how do we choose these L discrete levels
such that the quantization noise is minimized?

 In general, quantization is of two kinds: scalar quantization and vector quantization.

 Scalar quantization consists of two types: uniform quantization and non-uniform quantization.
 GABY ABOU HAIDAR, PH.D.

Scalar Quantization
 A scalar quantizer partitions the set R of real numbers into M subsets R1,...,RM, called quantization regions.

 Assume that each quantization region is an interval. Each region Rj is then represented by a representation point aj ∈ R

 When the source produces a number u ∈ Rj, that number is quantized into the point aj.

 A scalar quantizer can be viewed as a function {q(u)} : R → R that maps analog real values u into discrete real values
q(u) where q(u) = aj for u ∈ Rj

Thus an L−level quantizer is specified by L−1 interval endpoints, b1,...,bL−1, and L representation points, a1,...,aL.

Quantization regions and representation points.

 GABY ABOU HAIDAR, PH.D.

Quantization Noise
 It needs to be mentioned at this stage that the quantization process introduces noise represented by the error or
difference between the input signal u and the quantized output signal q(u).

 This error is called quantization noise, and it introduces distortion. When this error is made sufficiently small, the
original message signal and the quantized signal become practically indistinguishable to the human ear or eye
depending on whether u is a voice or picture signal.

 This means that the analog message can be approximated by a signal constructed at discrete amplitudes which are
selected on a minimum error basis from an available set.

 Clearly, the quantization noise can be reduced further by reducing the spacing between the adjacent quantization
levels or step-size.

 An important question that needs to be answered in order to design a quantizer is the following: for a given value of
L, how can the regions and representation points be chosen to minimize the mean-square quantization error?
 GABY ABOU HAIDAR, PH.D.

Mean Squared Quantization Error (MSQE)

 To answer the previous question, the probabilistic model for U1,U2 ,... is important.
 For example, if it is known that each Uk is discrete and has only one sample value in each interval, then the
representation points would be chosen as those sample values.
 Suppose now that the random variables {Uk} are i.i.d (independent and identically distributed) analog random
variables with the pdf fU(u).
 For a given set of points {aj}, Q(U) maps each sample value u ∈ Rj into aj.
 The distortion of the quantization noise is given by the random variable D = U −Q(U).
 The mean-square distortion, or mean-squared quantization error (MSQE) is then:

 In order to minimize the error over the set of aj , it is simply necessary to choose aj to minimize the
corresponding integral, assuming that the regions are considered fixed.
 If the regions are not fixed, then we need to minimize MSQE over the set of bj as well.
 GABY ABOU HAIDAR, PH.D.

Mean Squared Quantization Error (MSQE)

 In order to design an L−level quantizer, it is important to notice that the distortion equation contains in general
(2L−1) unknowns, L quantization levels aj and (L−1) quantization interval boundaries bj.

 Taking the derivatives of the distortion with respect to the (2L−1) parameters and setting to zero every single
equation, we obtain the following conditions for the optimum quantization levels aj and quantization interval
boundaries bj: The MSQE can be written as:

Differentiating with respect to bj yields

 GABY ABOU HAIDAR, PH.D.

Mean Squared Quantization Error (MSQE)

which results in:

To determine the quantized values aj, we differentiate with respect to aj and define b0 =−∞ and bL = +∞. Thus, we
obtain:

which results in:

 GABY ABOU HAIDAR, PH.D.

Optimum Uniform Quantizer

 Thus, for the optimum uniform quantizer, we have:

1. The optimum quantization interval boundaries are at the midpoints of the optimum quantization values:

2. The optimum quantization values are the centroids of the quantization intervals:
 GABY ABOU HAIDAR, PH.D.

Optimum Uniform Quantizer

 Although the previous rules are very simple, they do not result in analytical solutions to the optimal quantizer
design.

 The usual method of designing the optimal quantizer is to start with a set of quantization regions and then
using the second criterion, to find the quantized values.

 Then, we design new quantization regions for the new quantized values, and alternating between the two
steps until convergence (when the distortion does not change much from one step to the next).

 This iterative numerical method is known as the Max-Lloyd Algorithm.

 Based on this method, one can design the optimal quantizer for various source statistics.
 GABY ABOU HAIDAR, PH.D.

The Max-lloyd Algorithm

 The Max-Lloyd algorithm is an algorithm for finding the endpoints {bj} and the representation points {aj} to meet the
previous necessary conditions.
 The algorithm is almost obvious given the necessary conditions; the contribution of Lloyd and Max was to define the
problem and develop the necessary conditions.
 The algorithm simply alternates between the optimizations of the previous subsections, namely optimizing the
endpoints {bj} for a given set of {aj} , and then optimizing the points {aj} for the new endpoints.
 The Max-Lloyd algorithm is as follows. Assume that the number M of quantizer levels and the pdf FU(u) are given.

1) Choose an arbitrary initial set of M representation points a1 < a2 < ··· < aM.

2) For each j; 1≤ j ≤ M −1, set bj = 1/2(aj +aj+1).

3) For each j; 1 ≤ j ≤ M, set aj equal to the conditional mean of U given U ∈ (bj−1,bj) (where b0 and bM are taken to be
−∞ and +∞ respectively).

4) Repeat steps (2) and (3) until further improvement in MSE is negligible; then stop.
 GABY ABOU HAIDAR, PH.D.

The Max-lloyd Algorithm

 The MSQE decreases (or remains the same) for each execution of step (2) and step (3).

 Since the MSQE is nonnegative, it approaches some limit.

 Thus if the algorithm terminates when the MSE improvement is less than some given ε > 0, then the algorithm must
terminate after a finite number of iterations.

 The problem with the Max-Lloyd algorithm is that the algorithm might reach a local minimum of MSQE instead of the
global minimum.

 This algorithm is a type of hill-climbing algorithm; starting with an arbitrary set of values, these values are modified
until reaching the top of a hill where no more local improvements are possible.

 A reasonable approach in this sort of situation is to try many randomly chosen starting points, perform the Max-
Lloyd algorithm on each and then take the best solution.

 This is somewhat unsatisfying since there is no general technique for determining when the optimal solution has
been found.
 GABY ABOU HAIDAR, PH.D.

Minimum Mean-square Quantization Error (MMSQE)

 Let us now derive an expression for the minimum distortion (MMSQE) incurred by an optimum scalar quantizer.
Expanding the quadratic term using the identity method we have:

Where is the variance of U (we are assuming without loss of generality that U is zero-mean), and is
the probability of quantization values. The discrete random variable Q(U) takes values from the set of L optimum
quantization values with respective probabilities pj
 GABY ABOU HAIDAR, PH.D.

Minimum Mean-square Quantization Error (MMSQE)

 Then, the mean of Q(U) is the same as the mean of U. This can be shown easily:

 Since U is zero-mean, this means Q(U) is zero-mean and thus the sum in the last equality is in fact the variance
of Q(U); thus the MMSQE (Minimum mean-square quantization error) for the optimum quantizer is:
 GABY ABOU HAIDAR, PH.D.

Example:
 GABY ABOU HAIDAR, PH.D.

Example: (Cntd..)
 GABY ABOU HAIDAR, PH.D.

Uniform Quantization
 This section analyzes the performance of uniform scalar quantizers.
 For a uniform scalar quantizer, every quantization interval Rj has the same length |Rj| = ∆.
 In other words, R (or the portion of R over which f (u) > 0), is partitioned into equal intervals, each of length ∆.
U

Uniform scalar quantizer

 A uniform or linear quantizer has all the quantization levels uniformly distributed in the interval [−Vp ;Vp] (except
possibly the two intervals at the boundaries when the range of possible amplitudes is infinite).
 In other words, the interval [−Vp ;Vp] is subdivided into L quantization intervals, and the quantization amplitudes are
assigned at the center of each quantization interval.
 GABY ABOU HAIDAR, PH.D.

Uniform Quantization
output w2(t)
V

-V V
input w1(t)

-V
Region of operation
For M=2n levels, step size :
 = 2V /2n = V(2-n+1)
 GABY ABOU HAIDAR, PH.D.

Example: (Uniform Quantizer)

Consider a 4-level quantization (L =4).

 For a uniform quantizer, the quantization intervals and their corresponding quantized amplitudes are
as shown in the figure below.

 Any amplitude x within a quantization interval is assigned to the voltage in the middle of that interval
(except again possibly for the boundary intervals).
 GABY ABOU HAIDAR, PH.D.

Quantizer Bit-rate
 Thus, the input-output characteristic of a uniform quantizer is a stair case type characteristic and the spacing
between two adjacent quantization levels ak−1 and ak is called a “quantum” or “step-size”, and is denoted by ∆.

 A look at non-uniform quantization will be taken later.

(Quantizer Bit-rate) If the number of possible quantizer’s outputs is L (L-level quantizer), then the quantizer bit
rate is:

Alternatively, we refer to the L−level quantizer as an R−bit quantizer.

 GABY ABOU HAIDAR, PH.D.

Uniform Quantizer (Types)

Uniform quantizers are usually of two types:
1. Midrise quantizer: zero is not an output level. See Fig (a).
2. Midtread quantizer (Dead-Zone quantizer): zero is an output level. See Fig (b).
 GABY ABOU HAIDAR, PH.D.

Quantization Noise
 An important performance measure for quantizers is the signal-to-quantization noise ratio (SQNR), defined by:

 where for stationary stochastic processes:

 The previous definitions can be extended to non-stationary processes:

 For the optimum quantizer of a zero-mean source:

 GABY ABOU HAIDAR, PH.D.

Example:
 GABY ABOU HAIDAR, PH.D.

Quantization Noise
 Therefore, it can be seen here that to reduce the quantization noise, ∆ needs to be reduced.
 With the assumption that the quantization levels need to cover the entire dynamic range of the analog message, the
reduction of ∆ is equivalent to an increase in the number of quantization levels.
 Also, for every additional bit of quantization, we improve the SQNR performance by about 6 dB (not a small amount).
 It needs to be noted here that each quantization level is to be represented by a binary code word formed by a specific
number of binary digits, or bits.
 This representation permits the transmission of the quantization levels in binary form.
 Let R be the number of bits per sample used in the construction of the binary code.
 Then, we can write: L = 2R, under the assumption of a fixed length coding.
 Also, the average power of the quantization noise becomes:
 GABY ABOU HAIDAR, PH.D.

Non-Uniform Quantization(Companding)
 As long as the statistics of the input signal are close to the
uniform distribution, uniform quantization works fine.

 However, in coding for certain signals such as speech, the

input distribution is far from being uniformly distributed.

 For a speech waveform in particular, there exists a higher

probability for smaller amplitudes and lower probability
for larger amplitudes.

 If we use a uniform quantizer for such signals, the

distortion will be high.

 In fact, speech signals are modeled as having a Gamma

or Laplacian distribution, peaking about zero (that does The effect of increasing the quantization levels on
not mean that 0 Volt has the peak highest probability). reconstructing an image
 GABY ABOU HAIDAR, PH.D.

“Compressing-and-Expanding” Is Called “Companding.”

Non uniform quantizer
Discrete Uniform digital
samples Compressor Quantizer signals

••••
Channel
••••

Decoder Expander output

received
digital
signals
 GABY ABOU HAIDAR, PH.D.

Non-uniform Quantization(Companding)

 Therefore, it makes sense to design a quantizer with

more quantization regions at lower amplitudes and
less quantization regions at larger amplitudes.

 The resulting quantizer will be a nonuniform

quantizer having quantization regions of various
sizes.

3-bit non-uniform quantizer. (a) Laplacian pdf. (b) Input-output characteristic

 GABY ABOU HAIDAR, PH.D.

Non-uniform Quantization(Companding)
 For example, the range of voltages covered by voice signals,
from the peaks of loud talk to the weak passages of weak
talk, is on the order of 1000 to 1.
 By using a non-uniform quantizer with the feature that the
step size increases as the separation from the origin of the
input-output amplitude characteristic is increased, the large
end-step of the quantizer can take care of possible
excursions of the voice signal into the large amplitude ranges
that occur relatively in frequently.
 In other words, the weak passages that need more
protection are favored at the expense of the loud passages.
 In this way, a nearly uniform percentage precision is
achieved throughout the greater part of the amplitude range
of the input signal, with the result that fewer steps are
needed than would be the case if a uniform quantizer were
used.
 This figure illustrates a comparison between uniform and
non-uniform quantization for a speech voltage signal.
 GABY ABOU HAIDAR, PH.D.

Companding of Voice Signal

 We saw previously that uniform quantizers are easy to built and implement. However, their performance can be poor
for practical sources.
 Non-uniform, optimum quantizers on the other hand have optimum performance, but their optimality assumes
perfect knowledge of the source statistics and they are not robust to variations in these statistics.
 In many practical systems, the need for robustness and good performance (although not optimal) can be met by pre-
distorting the source signals through an invertible non-linearity in order to make the amplitudes at the output of the
non-linearity be more uniform. In this case, a simple uniform quantizer can be used.
 The process of pre-distorting the signal at the transmitter is known as (signal compression) and is performed using a
compressor.
 To restore the reconstructed quantized and compressed signal to its correct amplitude levels, a device, called an
expander, is used at the receiver.
 The expander law is the inverse of the compressor law. The two operations together are typically referred to as
companding.
 GABY ABOU HAIDAR, PH.D.

Companding Laws
 The use of a nonuniform quantizer is equivalent to passing the message signal through a compressor and then
applying the compressed signal to a uniform quantizer.
 A particular form of compression law that is used in practice is the so called µ-law defined by:

 where the logarithm is the natural logarithm; m and v are

respectively the normalized input and output voltages, and
µ is a positive constant.

 For convenience of presentation, the input to the quantizer

and its output are both normalized so as to occupy a
dimensionless range of values from zero to one, here we
have plotted the µ-law for varying µ.

 Practical values of µ tend to be in the vicinity of 255

 GABY ABOU HAIDAR, PH.D.

Companding Laws
 The case of uniform quantization corresponds to µ =0. For a given µ, the reciprocal slope of the compression
curve, which defines the quantum steps, is given by the derivative of |m| which respect to |v|; that is:

 We see therefore that the µ-law is neither strictly linear nor strictly logarithmic, but it is approximately linear at low
input levels corresponding to µ|m| << 1, and approximately logarithmic at high input levels corresponding to µ|m|
>> 1.

 The µ-law used for signal compression is used in the United States, Canada, and Japan.

 In Europe, another compression law known as the A-law is used for signal compression.
 GABY ABOU HAIDAR, PH.D.

Companding Laws
The A-law is defined by:

 Typical values of A used in practice tend to be in the vicinity of 100.

 The case of uniform quantization corresponds to A =1.
 The reciprocal slope of this second compression curve is given by the derivative of |m| with respect to |v|, as shown by:

 To restore the reconstructed quantized and compressed signal to its correct amplitude levels, a device, called an
expander, is used at the receiver. The expander law is the inverse of the compressor law.

 Note that the µ−law is used in T1 digital telephony systems(using twisted cables)that achieve a bit rate of 1.544Mbits/s,
and the A−law is used in E1 digital telephony systems (using coaxial cables or twisted cables) that achieve a bit rate of
2.048 Mbits/s.
 GABY ABOU HAIDAR, PH.D.

µ-law and A-law

A - law compressor
w1 (t )  1, A0
 A w1 (t ) 1
 0  w1 (t ) 
 1  ln A A
w2 (t )  
1  ln  A w1 (t )  1
  w1 (t )  1
 1  ln A A

 - law compressor
(very popular internationally)

w1 (t )  1
ln 1   w1 (t ) 
w2 (t ) 
ln 1   
In the U.S.,   255 is used.
 GABY ABOU HAIDAR, PH.D.

Example:
 GABY ABOU HAIDAR, PH.D.

Example:
 GABY ABOU HAIDAR, PH.D.

Example:
 GABY ABOU HAIDAR, PH.D.

“One man’s “magic” is another

man’s engineering.
“Supernatural” is a null word”

Robert A. Heinlein (Sci-FI writer)