Different Evaluation Metrics used in Engineering

Presented by Ahmad Shah Hafeez ur Rehman

Introduction
Different metrics provide means for evaluating and comparing system performances in various contexts. A variety of metrics are available as analytic tools but must be carefully and properly applied in order to obtain accurate and useful results. Each study area has got its own metrics.
Efficiency and Mechanical advantage Directivity, Beam-width and gain etc Bandwidth, gain, gain bandwidth product and roll-off factor BER, bandwidth , bps, SNR,CNR MIPS(Million of instructions per second), LoCs (Machines) (Antennas) (Amplifiers) (Communication systems) (Computers)

Transmission Constraints
Signals travelling through a medium whether guided or unguided, suffers from impairments. Impairments arise due to imperfections in the medium/channel or the transmitting or receiving devices. The received signal is a erroneous/distorted w. r. t the transmitted signal.

Causes of impairment

Attenuation
Loss of energy results in weaker signal When a signal travels through a medium it loses energy in overcoming the resistance of the medium. Amplifiers/Repeaters are used to compensate for this loss of energy by amplifying the signal.

Amplifying the attenuated signal

Measurement of Attenuation
To show the loss or gain of energy the unit decibel is used.

dB = 10 log10 (Po/Pi)

Distortion
Means that the signal changes its form or shape. Each frequency component has its own propagation speed traveling through a medium.

The different components therefore arrive with different delays at the receiver.
Signals have different phases at the receiver than they did at the source.

Distortion

Noise
Any unwanted signal that corrupts the signal of interest

Noise
There are different types of noise: Thermal : random noise of electrons in the wire creates an extra signal.

Induced : from motors and appliances, devices act as transmitter antenna and medium as receiving antenna.
Crosstalk : same as above but between two wires. Impulse : Spikes that result from power lines, lightning etc.

Noise Spectral Density (No)

Noise spectral density (No) is defined as the amount of (white) noise energy per bandwidth unit (Hz).

No = N / B
No is often expressed as

No = k T
where K is the Boltzmann's constant in Joules per Kelvin [J/K] T is the receiver system noise temperature in Kelvin [K]

Signal to Noise Ratio (SNR)

SNR

Obviously, we want as high an SNR as possible

SNR
The received SNR may be different at different points in the receiver, as various components, such as amplifiers, mixers, filters, etc., all add small amounts of noise to the total noise power. The sum of the noise contributions of the various components in the receive chain is often called the Noise Figure (NF) of the receiver. Digital processing can add noise in the form of quantization errors and other effects, and while these noise sources contribute to the total noise that may be seen at a detector, they are not the same as the thermal noise.

Capacity of a System
The bit rate of a system increases with an increase in the number of signal levels we use to denote a symbol. A symbol can consist of a single bit or n bits.

The number of signal levels = 2n.

As the number of levels goes up, the spacing between level decreases which increasing the probability of an error occurring in the presence of transmission impairments.

Nyquist Theorem
Nyquist gives the upper bound for the bit rate of a transmission system by calculating the bit rate directly from the number of bits in a symbol (or signal levels) and the bandwidth of the system (assuming 2 symbols/per cycle and first harmonic). Nyquist theorem states that for a noiseless channel:

C = 2 B log22n
C= capacity in bps B = bandwidth in Hz

Shannons Theorem
Shannons theorem gives the capacity of a system in the presence of noise.

C = B log2(1 + SNR)

CNR
The ratio of the received modulated carrier signal power C to the received noise power N

Difference between SNR and CNR

Carrier-to-noise ratio, often written CNR or C/N, is the signal-to-noise ratio (SNR) of a modulated signal. CNR is a measurement for modulated signals while SNR is usually used for baseband signals.

Carrier-to-Interference ratio (CIR)

The ratio of received modulated carrier power C to the average received co-channel interference power I.

C / I = C / (I1+ I2 + In)
CIR allows analysis and rating of channels robustness in the presence of co-channel interference.

Carrier-to-Noise Density (C/N0)

The ratio of carrier power divided to the noise power spectral density.

Energy per Bit (Eb)

Energy per information bit (i.e. the energy per bit net of FEC overhead bits)

Eb= C / R

where C: carrier power, and R is the actual information bit rate. Using the Eb rather than overall carrier power (PC) allows comparing different modulation schemes easily.

Energy per Bit to Noise Spectrum Density (Eb/No)

Eb/No is the ratio of the Energy per Bit divided by the noise power density.

Eb/No
Eb/No is the measure of signal to noise ratio for a digital communication system Allows comparing bit error rate (BER) performance (effectiveness) of different digital modulation schemes. Both factors are normalized, so actual bandwidth is no longer of concern. Modulation schemes are compared through BER plots against Eb/No.

BER
Bit Error Ratio (BER) is the number of bit errors divided by the total number of transferred bits during a studied time interval.
Sent Bits 1101101101 Received Bits 1100101101
error

BER

No. of Bits in error No. of Total Bits

1 10

= 0.1

BER
BER is normally displayed in Scientific Notation. The more negative the exponent, the better the BER. Better than 1.0E-6 is needed after the FEC for the system to operate.
D e ci m a l 1 0. 1 0. 0 1 0 . 00 1 0 . 00 0 1 0 . 0 00 0 1 0 . 0 00 0 0 1 0 . 0 0 00 0 0 1 0 . 0 0 00 0 0 01 0. 0 0 0 00 0 0 01 S c i e n ti fi c N o ta ti o n 1. 0 E + 0 0 1 . 0 E -0 1 1 . 0 E -0 2 1 . 0 E -0 3 1 . 0 E -0 4 1 . 0 E -0 5 1 . 0 E -0 6 1 . 0 E -0 7 1 . 0 E -0 8 1 . 0 E -0 9

Lower and Better BER

D e c i m a l S c i e n ti fi c N o ta ti o n 0 . 0 0 00 1 0. 0 0 0 00 9 0. 0 0 0 00 8 0. 0 0 0 00 7 0. 0 0 0 00 6 0. 0 0 0 00 5 0. 0 0 0 00 4 0. 0 0 0 00 3 0. 0 0 0 00 2 0. 0 0 0 00 1 1 . 0E -05 9 . 0E -06 8 . 0E -06 7 . 0E -06 6 . 0E -06 5 . 0E -06 4 . 0E -06 3 . 0E -06 2 . 0E -06 1 . 0E -06

Noise and Intermittents

Errors caused by noise or intermittent causes can have the same BER, but very different effects. Errors that are spread out are due to noise problems Errors that are grouped are due to intermittent problems such as loose connectors.
Spaced Errors Burst Errors 1101101011010011100 1111101011101101101

This Example Shows the Same Error Rate But the Burst Errors are More Difficult to Correct

Packet Error Rate (PER)

The packet error rate (PER) is the number of incorrectly received data packets divided by the total number of received packets.

Throughput
The throughput is a measure of how fast we can actually send data through a network. A link may have a bandwidth of B bps, but we can only send T bps through this link with T always less than B. The bandwidth is a potential measurement of a link; the throughput is an actual measurement of how fast we can send data.