PAPER NAME – ADAPTIVE SIGNAL PROCESSING

PAPER CODE – PE-EC702A

SUBMITTED BY – SOUMYADIP MONDAL.
UNIVERSITY ROLL – 12000321081.
REPORT ON – WINER FILTER DERIVATION

Department of Electronics & Communication Engineering

(Affiliated to Maulana Abul Kalam Azad University of Technology)
Dr. B. C. Roy Engineering College, Durgapur
Durgapur-713206, India
2024
 Table of Contents
 Introduction
 Mathematical Background
 Stochastic Processes and Random Variables
 Mean Square Error (MSE) Criterion
 Derivation of Wiener Filter
 Applications of Wiener Filter
 Conclusion
 References
 INTRODUCTION
In the field of signal processing, one of the key challenges is to extract or estimate useful
information from observed signals, which are often contaminated with noise or other
unwanted interference. This problem becomes particularly important when working with
real-world data, where noise is almost always present. The Wiener filter, introduced by
Norbert Wiener in the 1940s, offers an optimal solution for signal estimation in the presence
of such noise. The filter is designed to minimize the mean square error (MSE) between the
estimated output and the desired signal, making it highly effective in noise suppression and
signal enhancement.
The Wiener filter operates on the principle that both the signal of interest and the noise are
modeled as random processes. Under the assumption that the statistical properties of these
processes, such as their means and variances, are known and stationary over time, the
Wiener filter is able to find the optimal filter coefficients that yield the best estimate of the
desired signal. The filter works by leveraging the correlation between the input (noisy) signal
and the desired (clean) signal to suppress the noise while preserving the essential
characteristics of the original signal.
The Wiener filter can be implemented in both time and frequency domains, offering
flexibility in its application across various fields, including image processing, audio signal
enhancement, communication systems, and control systems. In the time domain, the
Wiener filter works by minimizing the MSE using autocorrelation and cross-correlation
functions. In the frequency domain, the filter is expressed in terms of the power spectral
densities of the signal and noise, allowing it to address problems involving frequency-
dependent noise.
This report delves into the detailed derivation of the Wiener filter, starting from the basic
problem setup, followed by the mathematical steps leading to the optimal filter coefficients.
We will explore the Wiener-Hopf equations, which form the foundation of the filter
derivation, and demonstrate how these equations can be solved to obtain the filter that
minimizes the MSE. Additionally, we will discuss the frequency domain representation of the
Wiener filter and highlight its practical applications.
The objective of this report is to provide a thorough understanding of the Wiener filter's
derivation and its significance in practical applications. By the end, we will gain insight into
the fundamental principles behind Wiener filtering, the mathematical derivation process,
and the wide range of real-world problems where this powerful tool is used.

2 MATHEMATICAL BACKGROUND
To fully understand the derivation of the Wiener filter, it is essential to review some
foundational concepts related to stochastic processes, random variables, and the mean
square error (MSE) criterion. These mathematical concepts form the basis for defining and
solving the Wiener filter problem.
2.1 Stochastic Processes and Random Variables
A stochastic process is a collection of random variables indexed by time. In Wiener filtering,
both the signal of interest s(n) and the noise v(n) are modeled as stochastic processes. This
means that for each time n, s(n) and v(n) are random variables that follow certain probability
distributions.
The signal s(n) is the underlying process that we aim to estimate. It is a random process, and
its exact values at each time instant are not known deterministically.
The noise v(n), which corrupts the signal, is also modeled as a random process. It is assumed
to be uncorrelated with the signal s(n).
Key characteristics of stochastic processes include:
Mean E[s(n)]: The average or expected value of the process at time n. If the mean of the
process does not change with time, the process is said to be stationary.
E[s(n)]= μs, where μs is constant.
Variance Var[s(n)]: The measure of how much the values of the process fluctuate around the
mean.
Autocorrelation function Rss(m): This function describes how the values of the process at
different times are related to each other.

2.2 Mean Square Error (MSE) Criterion

The Mean Square Error (MSE) is a critical concept in Wiener filtering. It is a measure of the average
squared difference between the true signal s(n) and the estimated signal s’(n) The objective of the
Wiener filter is to minimize this error.

Let:
 s(n) be the true, desired signal.
 s’(n) be the estimate of the signal, which is produced by applying the Wiener filter to
the noisy observation x(n).
The MSE is defined as:

MSE=E [ ( s ( n )−^s ( n ) ) ]
2
This expression represents the expected value of the squared difference between the actual
signal s(n) and its estimate ^s ( n )
Minimizing the MSE
The Wiener filter aims to minimize the MSE by finding the optimal filter coefficients that
best approximate the true signal from the observed data. Minimizing the MSE involves
taking the derivative of the MSE with respect to the filter coefficients and setting it to zero,
which leads to the Wiener-Hopf equations.
Let us assume signal ^s ( n ) is generated by filtering the observed signal x(n), which includes
both the desired signal s(n) and the noise v(n). If the filter is represented by a set of
coefficients h(k), the output of the filter is
M −1
^s ( n )= ∑ h ( k ) x ( n−k )
k=0

Where h(k) are the filter coefficients, and M is the length of the filter. The MSE can then be
expressed as a function of these coefficients, and the goal is to find the coefficients that
minimize the MSE.
2.3 Cross-Correlation and Autocorrelation Functions
To derive the Wiener filter, it is essential to use autocorrelation and cross-correlation
functions, which measure the statistical relationships between the signals and the noise.

Autocorrelation function R xx m : The autocorrelation of the observed signal x(n) gives us

information about the statistical relationship between the values of x(n) at different time
instants. It is defined as:

R xx ( m) =E [ x ( n ) x ( n−m ) ]

Where E[⋅] denotes the expectation operator. This function describes how the values of x(n)
at different time instants are correlated.
Cross-correlation function Rsx(m): The cross-correlation between the desired signal s(n) and
the observed signal x(n) measures how s(n) and x(n)are related. It is defined as:

R xx ( m) =E [ s ( n ) x ( n−m) ]

This function plays a crucial role in determining how the observed noisy signal x(n) can be
used to estimate the desired signal s(n).
The Wiener filter uses these correlation functions to compute the optimal filter coefficients
that minimize the MSE. By exploiting the relationship between the autocorrelation of the
observed signal and the cross-correlation between the observed and desired signals, the
Wiener filter can find the best possible estimate of the desired signal.
 3 DERIVATION OF WINER FILTER
Let x(n) represent the observed noisy signal, which is a combination of the desired signal s(n)
and noise v(n):
x(n) = s(n) + v(n)
We want to find a filter h(n) that produces an estimate ^s ( n ) of the signal s(n) from the
observation x(n) :
M −1
^s ( n )= ∑ h ( k ) x ( n−k )
k=0

Where h(k) represents the filter coefficients and M is the length of the filter.
Now we need to define the Mean Square Error (MSE), which is the expected value of the
squared difference between the true signal s(n) and the estimated signal ^s ( n ) :

MSE=E [ ( s ( n )−^s ( n ) ) ]
2

Expanding this expression:

MSE=E [ s ( n )2 ]−2 E [ s ( n ) s^ ( n ) ] + E [ ^s ( n )2 ]

M −1
Using the filter output ^s ( n )= ∑ h ( k ) x ( n−k ), we express the cross-correlation between s(n)
k=0
and ^s ( n ) , as well as the autocorrelation of x(n).
 Cross- correlation between s(n) and x(n) :
The cross-correlation function Rsx(m)between the desired signal s(n)s(n)s(n) and the
observed signal x(n) is:

R sx ( m )=E [ s ( n ) x ( n−m ) ]

 Autocorrelation of x(n):
The autocorrelation function Rxx (m) of the observed signal x(n) is:
R xx ( m) =E [ x ( n ) x ( n−m ) ]

The Wiener-Hopf equations represent a system of linear equations that can be solved for the
filter coefficients h(k). These equations involve the cross-correlation between the desired
signal and the observed signal, Rsx(k), and the autocorrelation of the observed signal, Rxx(k)
In matrix form, these equations can be written as:
R xx ( h )=R xx

Where:
R xx is the autocorrelation matrix of the observed signal,
h is the vector of filter coefficients,
R sx is the cross-correlation vector between the desired and observed signals.

APPLICATION OF WINER FILTER

1. Noise Reduction in Signal Processing

One of the primary applications of the Wiener filter is noise reduction in signals. When a
signal is corrupted by additive noise, the Wiener filter can estimate the original signal by
minimizing the error between the actual signal and the filtered output.

2. Image Restoration
In image processing, the Wiener filter is commonly used for image deblurring and noise
reduction. When an image is blurred due to motion or other factors, and noise is added, the
Wiener filter can restore the image by estimating the original pixel values.

3. Channel Equalization in Communication Systems

In digital communication systems, signals transmitted over a channel are often corrupted by
noise and channel distortions. The Wiener filter can be used as an equalizer to remove these
distortions and recover the transmitted signal at the receiver.

4. Echo Cancellation
Wiener filters are also employed in echo cancellation systems, particularly in
telecommunications. Echo occurs when a portion of the transmitted signal is reflected back
to the sender, leading to a delayed and distorted version of the original signal.

5. Estimation in Control Systems

In control systems, the Wiener filter is used for estimating the state of a system from noisy
observations. This helps in designing optimal controllers that can handle uncertainties and
noise in sensor measurements.
 CONCLUSION
The Wiener filter is a powerful tool for signal estimation and noise reduction, designed to
minimize the mean square error (MSE) between the true signal and its estimate. Its
formulation based on statistical properties of the signal and noise allows it to achieve
optimal performance in various practical scenarios where noise or interference is present.
The Wiener filter finds widespread applications in areas such as signal processing, image
restoration, and communications. It helps in tasks like noise reduction, deblurring, echo
cancellation, and equalization, making it indispensable in fields like telecommunications,
audio processing, image analysis, and control systems. By leveraging autocorrelation and
cross-correlation functions, the Wiener filter provides an efficient method for estimating
signals in the presence of noise or distortion.
In summary, the Wiener filter offers an optimal balance between filtering out noise and
preserving the desired signal, making it a key component in many modern signal processing
systems. Its versatility and statistical foundation allow it to adapt to a wide range of
applications where signal estimation is critical.
 REFERENCES
 Dspcookbook: Winer filter derivation https://dspcookbook.github.io
 OCW: Winer filter https://ocw.mit.edu
 Wikipedia: Winer filter https://en.wikipedia.org