21-04-2023

Unit 5
• Concept of finite word length in DSP

• Fixed and floating point numbers

• Representation of negative number

• Effect of truncation

• Finite word length effect in realization of IIR and FIR system.

• Fundamentals of adaptive filters and applications

• System identification

• Adaptive channel equalization

• Echo cancelation in data transmission

21-04-2023 1

Concept of finite word length in DSP

21-04-2023 2

1
 21-04-2023

• Important digital signal processing (DSP) operations like convolution and correlation

involve sum of product expression.

• Numeric representation and type of arithmetic have profound influence on the performance

of DSP system.

• The two forms of representation of numbers are

fixed-point representation

floating-point representation.

21-04-2023 3

• Fixed-point arithmetic represents numbers in a fixed range with a finite number of bits of
precision.
• Numbers outside the specified range will either saturate or wrap around.
• In general, fixed-point representation is preferred for high speed and lower cost.
• Floating-point arithmetic represents every number in two parts:
(i) a mantissa and
(ii) an exponent.
• Floating-point representation has higher dynamic range, and there is no need for scaling, which
makes it attractive for complex algorithms.

21-04-2023 4

2
 21-04-2023

• The implementation of digital filters involves the use of finite precision arithmetic.
• This leads to quantization of the filter coefficients and the results of the arithmetic
operations.
• Such quantization operations are non-linear and cause a filter response substantially
different from the response of the underlying infinite precision model.

21-04-2023 5

DIGITAL REPRESENTATION OF NUMBERS

• Signals are represented in form of numbers and stored in the system in form of binary numbers.
• The numbers can be broadly classified as
(i) Fixed point number
(ii) Floating point number

21-04-2023 6
Different forms of number representations

3
 21-04-2023

BINARY NUMBER SYSTEM

• Bit, which stands for ‘Binary digit’, is the fundamental unit of binary number system.
• Bit can take a value of ‘0' or ‘1’.
• Combining several bits allows one to represent a range of values.

Unsigned integer

An unsigned integer is represented in terms of binary number as

where‘ A’ represents the unsigned number.

An N-bit number with all 1s is equivalent to 2N-1.
21-04-2023 7

Signed integer

Signed integers can be represented in any one of the three representations:

(i) Sign-magnitude representation

(ii) One’s complement representation and
(iii) Two’s complement representation.

(a)Sign-magnitude representation
In sign-magnitude representation, the leftmost bit is the sign bit and the remaining bits are the
magnitude bits.
Sign bit ‘0’ indicates that the number is positive, and the sign bit as ‘1’ indicates that the number
is negative.
The sign-magnitude representation of the binary number is given in Figure

‘S’ represents the sign bit.

The other bits represent the magnitude.
Sign-magnitude representation of binary numbers
21-04-2023 8

4
 21-04-2023

-
Advantage: The main advantage of sign magnitude representation is that it is easier to understand and simple to implement.
Drawback: The main drawback of sign-magnitude representation is that there are two representations for ‘0’.

For example:
00000000 = + 0 10

and
10000000 =- 0 10

21-04-2023 9

Different 4-bit binary number representations

21-04-2023 10

5
 21-04-2023

Question

21-04-2023 11

Find the equivalent decimal number for the following sign-magnitude

binary representation: (i) 01010010 and (ii) 11010111

21-04-2023 12

6
 21-04-2023

One’s complement representation

The one’s complement form of negative binary number is the complement of its positive counterpart

21-04-2023 13

Two’s complement representation

• In two's complement representation, the sign bit (MSB) is ‘1’ for negative and ‘0’ for
positive number.
• The two’s complement of a number is obtained by adding a ‘1’ to one’s complement.
• The main advantage of two’s complement representation is that there is only one
representation for ‘ 0’.

21-04-2023 14

7
 21-04-2023

FIXED-POINT NUMBER REPRESENTATION

• This is another representation of binary number.
• Here, the binary point is fixed, and it cannot move.
• A fixed-point number type is simply defined by indirectly fixing the binary point to be at some
position of a numeral.
• To define a fixed-point type conceptually, the following two parameters are required:
(i) width of the number representation (w) and
(ii) binary point position within the number (b).
The notation or the fixed-point number representation is fixed(w,b).

Here, fixed denotes fixed-point representation,

‘ w’ indicates the width of the number representation and
‘ b' represents binary point position within the number.
21-04-2023 15

Example:
The unsigned binary number given to the computer is (10010001)2. Find the decimal
value of the fixed-point representation fixed(8,2)

Solution
The given unsigned binary number is 10010001.
The fixed-point representation is fixed(8,2).
Here 8 represents the width of the number representation, which means that it is an 8-bit
binary number, and 2 denotes the
binary point position from the least significant bit (LSB).
The fixed-point representation of the given binary pattern (100100. 01)2 The decimal value is
(36.25)10, which is explained in table

21-04-2023 16

8
 21-04-2023

21-04-2023 17

Fixed-point arithmetic operation

The fixed-point arithmetic operations of two binary numbers are

(i) binary addition and
(ii) binary multiplication

Binary addition: The basic arithmetic operations of addition and multiplication depend on the
representation of the two binary numbers.
Addition of two binary numbers is using one’s complements and two’s complements by adding bit
by bit.
For example, addition of two numbers (5/8) and (-1/8) is as follows:

21-04-2023 18

9
 21-04-2023

Using sign magnitude: An addition of two binary numbers using sign-magnitude

representation with example is given below. The addition of two numbers (5/8) and (-1/8) and
its sign-magnitude representation are 0.101 and 1.001, respectively

The answer 1.110 represents (-6/8), but the correct answer is 0.100 (i.e. 4/8).
Hence, binary addition using sign-magnitude representation is more complex and can involve
sign checks, complementing, and the generation of carry.

21-04-2023 19

Using one’s complement: The one’s complement representation of (5/8) and (-1/8) are
0101 and 1110, respectively, using 4-bit fixed-point representation.

The answer is 10.011, where1 is the carry, 0 is the sign bit and Oil represents 3/8, which is
not correct.
Hence, based on the carry, we have to adjust the final answer. If carry exists during addition
operation, the
carry MSB is carried around to LSB.

Now the correct answer is (1/2), and its corresponding one’s complement representation is 0.100.

21-04-2023 20

10
 21-04-2023

21-04-2023 21

21-04-2023 22

11
 21-04-2023

FLOATING-POINT NUMBER REPRESENTATION

In fixed-point binary representation, the binary point is fixed and it cannot move,
whereas in floating-point representation, the binary point can move.
The floating-point number is represented by

where s stands for sign, which is ‘0’ for positive and ‘1’ for negative,
‘M’ represents the fractional part, which is also termed as ‘significand’,
‘B’ refers to the base and
‘E’ stands for exponent.

21-04-2023 23

Example:
Express the decimal number 81 in floating-point representation (base 2)

21-04-2023 24

12
 21-04-2023

IEEE 754 standard

21-04-2023 25

IEEE 754 single-precision floating-point format

IEEE 754 double-precision floating-point format

21-04-2023 26

13
 21-04-2023

Special cases in IEEE 754 single precision representation

The sign bit is ‘ 0’ for positive numbers and ‘ 1’ for negative numbers.
The significand or mantissa contains the binary fraction.
The exponent field uses a biased integer representation. The bias value is 127 for single-precision representation and 1023
for double-precision number .
The IEEE double-precision floating-point format is shown in Figure.
Special cases:
Some of the special cases of IEEE 754 single-precision format are given below:
(i) If the exponent is 255,mantissa is zero and the sign bit ‘ 1’, thenit represents + ∞.
(ii) If the exponent is 255,mantissa is zero and the sign bit ‘ 0’, thenit represents -∞.
(iii) If the exponent is 255 and the mantissa is non-zero, thenit represents NaN (not a number)
21-04-2023 27
.
(iv) If the exponent is zero and the mantissa is zero, then it represents zero

EXAMPLE: Convert the decimal number 5.25 to 32-bit IEEE single-precision format

21-04-2023 28

14
 21-04-2023

21-04-2023 29

EXAMPLE: The 32-bit single-precision floating-point number is ‘010000010

11000000000000000000000’. Find the corresponding decimal value

21-04-2023 30

15
 21-04-2023

21-04-2023 31

FIXED-POINT QUANTIZATION ERROR

• Quantization is the process in which a quantity X is approximated into a quantity Q( X).
• It is approximately equal to X, but it has some distortion errors.
• Quantization occurs when a real number is represented by nearest integers so as to reduce the
word length of a binary representation of X by reducing the number of bits after the binary
point.
• For example, the binary representation of 3.85 is (11.110110011...); after reducing the number
of bits (11.1101), the value is 3.8125.
• The difference between X and Q(X ) is called as ‘quantization error’.
• The quantization is commonly encountered in DSP in the form of
i) Rounding
(ii) Two’s complement truncation

21-04-2023
(iii) Magnitude truncation. 32

16
 21-04-2023

21-04-2023
Quantization characteristics curve 33

Fixed-point quantization errors in rounding

Fixed-point quantization errors in two’s complement truncation

Fixed-point quantization errors in magnitude truncation

21-04-2023 34

17
 21-04-2023

COEFFICIENT QUANTIZATION

• For a given set of filter specifications, the system function of the filter H( z) is obtained with an
assumption that the filter coefficients can be represented with infinite precision.
• However, when implementing the filter in the real world, a finite number of bits are used to
represent each coefficient of H(z ).
• The representation of the filter coefficients from infinite precision to finite number precision may
give coefficient quantization.
• This coefficient quantization can somehow change the location of the filter poles and zeros.
• As a result, after implementing a filter, the frequency response of the filter will be quite different
from that of the original design.

21-04-2023 35

• The error in the pole and zero locations depends on several factors.
• These factors and methods to design filters that exhibit smaller sensitivity to the coefficient
quantization.
• The coefficients obtained by design methods are real or complex.
• For real-world DSP, these coefficients are often stored in a finite length register.
• Often the coefficients are rounded in order to accommodate it in the finite length register.
• This causes a rounding error that will influence the filter characteristics.
• The frequency response of the quantised filter coefficients will differ from the desired frequency
response.
• Sensitivity of the filter response to coefficient quantization is dependent on the type of filter
structures.
21-04-2023 36

18
 21-04-2023

EXAMPLE
Design a seventh-order Chebyshev type I low-pass filter that passes frequencies between 0 and
0.2 with 3 dB of ripple in the passband. Use cheby1 MATLAB function to find the filter
coefficients

21-04-2023 37

21-04-2023 38

19
 21-04-2023

The unquantised coefficients of a Chebyshev type I low»pass filter

21-04-2023 39

• Now the IIR filter coefficients are quantised with the limited number of bits.
• The integer and fractional part of the denominator coefficients are quantised with 6-bit
representation and additional 1 bit for sign.
• Similarly, the numerator coefficients of the IIR filter are quantised with 14 plus 1 totally 15 bits.
• The 14 bits are used to represent the fractional part of the coefficients (in this case, all the
coefficients are fractional value and there is no integer), and a bit is used to represent the sign

21-04-2023 40

20
 21-04-2023

• Quantising the coefficients has adversely affected the frequency response.

• Two poles are moved out of the unit circle, and the quantised filter is unstable.
• This example shows that after designing a filter, it is necessary to examine the effect of the
coefficient quantization. If the quantised filter does not meet the target specifications, then the
filter has to be redesigned.
21-04-2023 41

Effects of coefficient quantization on IIR filter

The transfer function of IIR filter is expressed as

• The overall sensitivity of each pole to quantization errors introduced by all the coefficients can be
approximated by

• The poles are closely clustered and very sensitive to quantization error in filter coefficents.
• The same concept will be applicable to zeros as well.
• 21-04-2023
To reduce the effect of quantization error, the poles are not closely clustered and should be42
separated.

21
 21-04-2023

Analysis of sensitivity to coefficient quantization

• Sensitivity of the filter reponse to coefficient quantization is dependent on the type of filter structures.
• In general the digital fitters are implemented in direct form I , direct form II , cascade, parallel and
lattice structure.
• The use of second order section in filter design decouples the effect of quantization error on
closely ' clustered poles,
• For this, let us consider the second-order filter, which is having pair of complex poles
• The filter transfer function is written us

21-04-2023 43

• Based on the availability of the bits, the constant terms cos and - must be computed and rounded
properly.
• Let us consider that we use a 4-bit quantiser including a sign bit b0.b1b2b3 Both the terms cos and -
can take on the numbers from 1.000 to 0.111 (i.e-1 to 0.875).

Direct form I structure of pair of complex pole second-order Infinite Impulse response filter
21-04-2023 44

22
 21-04-2023

21-04-2023 45

it is evident that poles and

zeros of a second-order filter
can only occur at the
intersection of the lines
representing cos and in
the form of semicircles,

21-04-2023 46
Pole-zero plot for second-order infinite impulse response filter

23
 21-04-2023

Cascade form of a higher order filters

• The reduction of sensitivity to coefficient quantization can be done by splitting higher

order filters into lower order filters.
• For this, cascade form can be used to split the higher order filters into multiple lower order
filters.
• Finding the cascade form of a higher order filter involves tedious mathematics; for this,
tf2sos a MATLAB built-in command, is used, which stands for transfer function to second-
order section

21-04-2023 47

• Each row of sos denotes the transfer function of one of the second-order sections.
• Here, the first three numbers of each row represent the numerator coefficient of the
corresponding second-order section, and the second three numbers give its denom inator
coefficient.
• For example, for the first second-order section obtained from the first row, the transfer
function can be drawn as

21-04-2023 48

24
 21-04-2023

21-04-2023 49

The overall transfer function of the filter is

• Now, the quantization can be applied on these coefficients with 1 bit for the sign representation and
6 bits each for the magnitude of the coefficients of integer and fractional part.
• The MATLAB code for the quantization of these filter coefficients is shown in Figure

21-04-2023 50

25
 21-04-2023

• A higher order filter as a cascade of second-order sections can significantly reduce the
sensitivity to the coefficient quantization.
• Implementing a higher order filter as a cascade of second-order sections can
significantly reduce the sensitivity to the coefficient quantization.
• Also the pole of the system lies in the unit circle; hence, the cascade section is stable
21-04-2023 51

COEFFICIENT QUANTIZATION OF FIR FILTER

• The quantization of the FIR filter coefficients also affects the magnitude response of the
original filters.
• The FIR filter has linear phase characteristics; hence, phase response of the FIR filter will not
be affected by the quantization errors.
• The MATLAB code for the effects of coefficient quantization on FIR filter is given in Figure.
• Using fir1 built-in command, the low-pass filter coefficients are computed.
• In this example, the normalised upper cutoff frequency is chosen as 0.2 radians, and the
number of filter coefficients is chosen as 12.
• The original FIR filter coefficients are quantised by 5, 7 and 15 bits, and its magnitude
and phase response are shown in Figure.
• Cascade structure is preferred to reduce the sensitivity of the FIR filter coefficient
quantization.
21-04-2023 52

26
 21-04-2023

21-04-2023 53

• While increasing the

number of bit
representation, the
quantized filter
magnitude response
approaches to the original
filter response,
• From this, it is possible to
infer that the
quantization of the FIR
filter coefficients affects
the magnitude response of
21-04-2023 54
the filter.

27
 21-04-2023

Cascade structure of FIR filter

• Effects of quantization error in FIR filter can be reduced by using cascade structure of FIR filter
coefficients.
• Here, the higher order FIR filter coefficients are split into cascaded second-order filter sections.
• These split filter coefficients are quantised by low-bit quantiser.
• The MATLAB code for the FIR filter cascade second-order section is given in Figure.
• In the simulation of the MATLAB code, the numerator coefficients are quantised by 7 bits including
the sign bit and the denominator coefficients are quantised by 5 bits including the sign bit.

21-04-2023 55

• The magnitude and phase response of the quantized filter coefficients are on par with the
magnitude and phase response with the original filter coefficients, which are shown in Figure.
• This example illustrates the fact that the cascade section of second-order filter will reduce the
sensitivity of the FIR filter coefficient quantization.

MATLAB code for cascade structure

21-04-2023 56

28
 21-04-2023

Frequency response of quantised finite impulse response coefficients with cascade

implementation

21-04-2023 57

• Fundamentals of adaptive filters and applications

• System identification

• Adaptive channel equalization

• Echo cancelation in data transmission

21-04-2023 58

29
 21-04-2023

• Adaptive signal processing is concerned with the design, analysis and implementation of
adaptive discrete time system whose structure adapts itself in response to the external
environment or changes in the incoming data.
• As the name suggests, adaptive signal processing deals with design of digital filter which can
adapt against changing external parameters, statistical parameters of the system or disturbances
affecting input data.
• Adaptive signal processing finds application in
(i) Echo cancellation
(ii) Equalization of data communication channel
(iii) System identification etc.

21-04-2023 59

• An adaptive system interacts with the environment, learns through the

interaction and adjusts its behaviour for optimal performance.
• An adaptive filter is one whose coefficients vary with respect to time according
to certain rules.

21-04-2023 60

30
 21-04-2023

MOTIVATION FOR ADAPTIVE FILTER

• When the signal of interest and the noise reside in separate frequency bands, conventional linear
filters can be used to extract the desired signal.
• When there is a spectral overlap between signal of interest and noise or whenever the signal or
interfering signal’s statistics change with time, a fixed coefficient filter will be inappropriate.
• In real life the signal statistics are not fixed and change with respect to the time.
• In this situation, there is a need for design of the adaptive filter.
• Adaptive fillers have the ability to self-adjust their coefficients in order to minimise the error
function.

21-04-2023 61

• The error function is termed as cost function.

• The cost function has to be minimised according to optimising algorithms.
• Adaptive filter finds applications in the field of
signal processing,
communications,
radar,
sonar,
seismology,
biomedical engineering,
navigation systems etc.
• The adaptive algorithms are expected to be computationally efficient, numerically robust and
fast convergent.
21-04-2023 62

31
 21-04-2023

WIENER FILTER

• The Wiener filter was introduced by Norbert Wiener in 1949.

• The Wiener filter is an optimal filter with respect to minimum mean square error (MMSE)
sense.
• The Wiener filter produces an estimate of the desired signal from an observed noisy process
assuming that the noise is additive in nature.
• It is assumed that the signal and the additive noise are stationary in nature.
• Here it is assumed that the system is a linear time invariant (LTI) system and the
measurements are applied to the input of the LTI system, and the system is designed to
produce as its output the MMSE estimate of the process of interest.

21-04-2023 63

General framework of the Wiener filter

• s[n] represents the original signal which is uncorrupted.

• The signal s[n] is corrupted by additive noise n[n] to yield the signal x[n].
• The impulse response of the filter is denoted by h[n].
• The parameters of the filter have to be designed in such a way that the output of the filter
y[n] should resemble the desired signal d[n] such that the error e[n] is minimum.

21-04-2023
FIGURE General framework of a Wiener filter 64

32
 21-04-2023

If h[n] is a two tap linear phase filter with the coefficients b0 and b1 the above structure will be modified as shown in
Figure

transversal structure.

FIGURE Two-tap Wiener filter

the expression for the impulse response of the filter can be given as

The expression for the output signal is obtained as

21-04-2023 65

Hence this may deduce a framework for the error signal as:

The expression for the squared error can be described by taking the square of the error signal

Assuming the input signal to be a statistical stationary signal, we may take the expectation of the squared
error to obtain the mean square error as:

21-04-2023 66

33
 21-04-2023

Taking the expectation operator inside the expression we get

If r(l) denotes the autocorrelation of x[n] and p(l) denotes the cross-correlation between the desired signal
d[n] and the observed signal x[l], the above expression can be rewritten in the reduced form its

The cost function with respect to the filler coefficients b0 and b1 is expressed as

21-04-2023 67

• The optimal filter coefficient is obtained by minimising the cost function.

• The cost function can be minimised by taking partial derivative of J ( b0 , b1 ) with respect
to b0 and b1.
• Taking partial derivative of J (b0 , b1) with respect to b0 is given by

Similarly, taking partial derivative of J(b0 , b1) with respect to b1 is given by

The expression for the gradient operator is given from expression

21-04-2023 68

34
 21-04-2023

• In the above expression b0 and b1 are the coefficients of the filter ‘h’.
• The expression in vector form is given by

To find the optimal filter coefficient the gradient must vanish, hence

The expression for optimal filter is given by

• The above expression is termed as Wiener-Hopf expression which is named after American-born
Norbert Wiener and Austrian-born Eberhard Hopf.
• The expression for optimal filter depends on autocorrelation matrix (R) of the observed signal (x[n])
and the crosscorrelation vector (p) between observed signal (x[n]) and the desired signal (d[n] ).

21-04-2023 69

Limitations of the Wiener filter

The following are the limitations of the Wiener filter

(i) The autocorrelation matrix of the observed sequence (R) and the cross-correlation between the
desired and the observed signal (p) are not known a priori.
(ii) When the signals are non-stationary, ‘R’ and ‘p’ change with time.
(iii) The matrix inversion is really challenging.
(iv) Adaptive algorithms are used to compute the solutions which are closer to the optimal solution
without computing 'R' and ‘p' explicitly

21-04-2023 70

35
 21-04-2023

ADAPTIVE FILTER

• Filters are basically used to extract or enhance the desired

information in the signal. An adaptive filter is a time-variant
filter whose coefficients are adjusted in a way to satisfy
some predetermined optimisation criterion.
• The term adaptive implies that the filter can operate in
changing environments.
• Adaptive filters were first introduced by Widrow and Hopf in
the 1960s. FIGURE Block diagram of an adaptive filter

• Some of the applica tions which require adaptiveness include

(i) Signal detection in noisy environment
(ii) Coding of speech signal
(iii) Tracking of moving objects etc.
21-04-2023 71

Some of the significant characteristics of adaptive

filler are summarized below:
• Adaptive filters can operate satisfactorily in a time-
varying environment without user intervention.
• Adaptive filters improve their performance during
operation by learning statistical characteristics from
current signal observations.
FIGURE Block diagram of an adaptive filter
• Adaptive filters can track variations in the signal
operating environment.

21-04-2023 72

36
 21-04-2023

The symbol x[n] represents the input signal.

h[n) represents the impulse response of the filter,
• y [n] is the filtered signal,
• d[n] represents the desired signal.
• e[n] represents the error signal
21-04-2023 73

System identification

• System identification is the experimental approach for modelling a process from the
input/output data.
• The system identification using adaptive algorithm is shown in Figure.

FIGURE System identification using adaptive algorithm

21-04-2023 74

37
 21-04-2023

• The unknown system is modelled by an FIR filter with adjustable coefficients.

• The same input x[n] is applied to the system to be identified and the adaptive filter which is
denoted by w[n].
• The difference between system output (d[n]) and output of the adaptive filter (y[n)) results
in the error signal (e[n]), which is fed as input to the adaptive algorithm which corrects or
updates the tap weight of the filter.
• This process is repeated through several iterations until the estimation error e[n] becomes
sufficiently small in some statistical sense.
• The resultant FIR filter response now represents that of the previously unknown system.

21-04-2023 75

38