Different Evaluation Metrices used in Engineering

Presented by Ahmad Shah Hafeez ur Rehman

Different metrics provide means for evaluating and comparing system performances in various contexts. A variety of metrics are available as analytic tools but must be carefully and properly applied in order to obtain accurate and useful results. Each area has got its own metrices.

For example
Efficiency and Mechanical advantage (Machines) Directivity, beamwidth and gain etc (Antennas) Bandwidth, gain ,gain bandwidth product (Amplifers) and rolloff factor BER,bandwidth ,bps,SNR,CNR (Communication systems) MIPS(million of instructions per second) (computers) Execution time and memory (Softwares)

TRANSMISSION IMPAIRMENT
Signals travelling through a medium(guided or unguided) suffers from impairments.This arises due to imperfections in the medium/channel.
The received signal is a bit different from what is sent.

Causes of impairment

Attenuation
loss of energy -> weaker signal When a signal travels through a medium it loses energy overcoming the resistance of the medium Amplifiers are used to compensate for this loss of energy by amplifying the signal.

Measurement of Attenuation
To show the loss or gain of energy the unit decibel is used. dB = 10log10 (P2/P1) P1 - input signal power P2 - output signal power

Amplifying the attenuated signal

Distortion
Means that the signal changes its form or shape Each frequency component has its own propagation speed traveling through a medium. The different components therefore arrive with different delays at the receiver. That means that the signals have different phases at the receiver than they did at the source

Distortion

Noise
Any unwanted signal that corrupts the signal of interest

Noise
There are different types of noise
Thermal - random noise of electrons in the wire creates an extra signal Induced - from motors and appliances, devices act as transmitter antenna and medium as receiving antenna. Crosstalk - same as above but between two wires. Impulse - Spikes that result from power lines, lighning, etc.

Noise spectral density (No)

Noise spectral density (No) is defined as the amount of (white) noise energy per bandwidth unit (Hz).

No = N / B

No is often expressed as No = k T
where K is the Boltzmann's constant in Joules per Kelvin [J/K] T is the receiver system noise temperature in Kelvin [K]

Ambient thermal noise is typically calculated at 290K, or 16.85C

Signal to Noise Ratio (SNR)

SNR

Obviously, we want as high an SNR as possible

The received SNR may be different at different points in the receiver, as various components, such as amplifiers, mixers, filters, etc., all add small amounts of noise to the total noise power. The sum of the noise contributions of the various components in the receive chain is often called the Noise Figure (NF) of the receiver. Digital processing can add noise in the form of quantization errors and other effects, and while these noise sources contribute to the total noise that may be seen at a detector, they are not the same as the thermal noise.

Capacity of a System
The bit rate of a system increases with an increase in the number of signal levels we use to denote a symbol. A symbol can consist of a single bit or n bits. The number of signal levels = 2n. As the number of levels goes up, the spacing between level decreases which increasing the probability of an error occurring in the presence of transmission impairments.

Nyquist Theorem
Nyquist gives the upper bound for the bit rate of a transmission system by calculating the bit rate directly from the number of bits in a symbol (or signal levels) and the bandwidth of the system (assuming 2 symbols/per cycle and first harmonic). Nyquist theorem states that for a noiseless channel: C = 2 B log22n C= capacity in bps B = bandwidth in Hz

Shannons Theorem
Shannons theorem gives the capacity of a system in the presence of noise.

C = B log2(1 + SNR)

CNR
The ratio of the received modulated carrier signal power C to the received noise power N

CNR

Carrier power

=
Noise power

Difference between SNR and CNR

carrier-to-noise ratio, often written CNR or C/N, is the signal-to-noise ratio (SNR) of a modulated signal. CNR and SNR represent quite different measurement parameters, one in the RF domain (Figure 1) and the other in the baseband domain

Carrier-to-interference ratio (CIR)

The ratio of received modulated carrier power S or C to the average received co-channel interference power I.

C / I = C / (I1+ I2 + In)

Allows analysis and rating of channel robustness in the presence of co-channel interference.

Carrier-to-Noise Density (C/N0)

The ratio of carrier power divided to the noise power spectral density

Energy per Bit (Eb)

Energy per information bit (i.e. the energy per bit net of FEC overhead bits)

Eb= C / R

where
C is the carrier power, and R is the actual information bit rate.

Using the Eb rather than overall carrier power (C) allows comparing different modulation schemes easily.

Energy per Bit to Noise Spectrum Density (Eb/No)

Eb/No is the ratio of the Energy per Bit divided by the noise power density.

Eb/No
Allows comparing bit error rate (BER) performance (effectiveness) of different digital modulation schemes. Both factors are normalized, so actual bandwidth is no longer of concern.

Modulation schemes are compared through BER plots against Eb/No.

BER
bit error ratio (BER) is the number of bit errors divided by the total number of transferred bits during a studied time interval.
Sent Bits Received Bits 1101101101 1100101101
error

BER =

# of Wrong Bits # of Total Bits

1 10

0.1

BER, Coding Scheme and Eb/No

Coding gain
For a given bit-error probability, the reduction in the Eb/N0 that can be realized through the use of code.
PB
Coded A F C B D E Uncoded

Eb / N 0 (dB)

BER
BER is normally displayed in Scientific Notation. The more negative the exponent, the better the BER. Better than 1.0E-6 is needed after the FEC for the system to operate.
D e ci m a l 1 0. 1 0. 0 1 0 . 00 1 0 . 00 0 1 0 . 0 00 0 1 0 . 0 00 0 0 1 0 . 0 0 00 0 0 1 0 . 0 0 00 0 0 01 0. 0 0 0 00 0 0 01 S c i e n ti fi c N o ta ti o n 1. 0 E + 0 0 1 . 0 E -0 1 1 . 0 E -0 2 1 . 0 E -0 3 1 . 0 E -0 4 1 . 0 E -0 5 1 . 0 E -0 6 1 . 0 E -0 7 1 . 0 E -0 8 1 . 0 E -0 9

Lower and Better BER

D e c i m a l S c i e n ti fi c N o ta ti o n 0 . 0 0 00 1 0. 0 0 0 00 9 0. 0 0 0 00 8 0. 0 0 0 00 7 0. 0 0 0 00 6 0. 0 0 0 00 5 0. 0 0 0 00 4 0. 0 0 0 00 3 0. 0 0 0 00 2 0. 0 0 0 00 1 1 . 0E -05 9 . 0E -06 8 . 0E -06 7 . 0E -06 6 . 0E -06 5 . 0E -06 4 . 0E -06 3 . 0E -06 2 . 0E -06 1 . 0E -06

Noise and Intermittents

Errors caused by noise or intermittent causes can have the same BER, but very different effects. Errors that are spread out are due to noise problems Errors that are grouped are due to intermittent problems such as ingress or loose connectors.
Spaced Errors Burst Errors 1101101011010011100 1111101011101101101

This Example Shows the Same Error Rate But the Burst Errors are More Difficult to Correct

Pre and Post FEC BER

The Forward Error Correction (FEC) can correct errors up to a point, after which errors are passed on to the decoding circuitry. Its important to know the Pre and Post FEC BER to know how hard the FEC is working to correct errors. The harder its working, the closer the system is to failure.
Pre FEC BER (Before Correction)

Post FEC BER (After Correction)

BER
BER is a very important measurement for determining the health of digital systems. Determining the PRE and POST FEC BER can tell you how hard the FEC is working giving an indication of system margin. Error seconds or severely
CM1000 Cable Modem System Analyzer

errored seconds can indicate intermittent errors.

AT2000 Spectrum Analyzer

packet error rate (PER)

The packet error rate (PER) is the number of incorrectly received data packets divided by the total number of received packets

Throughput
A network with bandwidth of 10 Mbps can pass only an average of 12,000 frames per minute with each frame carrying an average of 10,000 bits. What is the throughput of this network? Solution We can calculate the throughput as

The throughput is almost one-fifth of the bandwidth in this case.

Thanks