Un
Dep
Dep
zers for Fr
A
nder the able
partment of
partment of
Motilal N
requency D
Transm
A Term Pap
Bha
Sha
Vih
e guidance o
f Electronic
MNN
f Electronic
Nehru Natio
Deem
Allaha
Discrimin
mission Sys
per Report Su
By
arat Tandon
ardul Verma
hag Mishra
of Lecturer
cs & Commu
IT Allahaba
to the
cs & Commu
onal Institu
med Universi
abad 2110
ation Rece
stem
ubmitted
Mr. Asim M
unication E
ad
unication E
ute of techno
ity
004
eiver of M
Mukherjee
Engineering
Engineering
ology
MSK Opticcal
i
Department of Electronics and Communication Engineering
Motilal Nehru National Institute of Technology
Allahabad
CERTIFICATE
This is to certify that the term paper entitled MLSE Equalizers for Frequency Discrimination
Receiver of MSK Optical Transmission System submitted by Bharat Tandon (20055036),
Shardul Verma (20055038), Vihag Mishra (20055017) for the course of Optical Communication
in Electronic and Communication Engineering at Motilal Nehru National Institute of
Technology, is a bonafide record of their own work and has been successfully completed under
my supervision.
Date:
Mr. Asim Mukherjee
Lecturer
Department of Electronics and Communication Engineering
Motilal Nehru National Institute of Technology
Allahabad  211004
ii
Abstract
This work illustrates the maximumlikelihood sequence estimator (MLSE) equalizers based on
either Viterbi algorithm or template matching temple matching (TM) for the equalization of
impairments imposed on the minimum shift keying (MSK) modulation formats in long haul
transmission without optical dispersion compensation. The TMMLSE equalizer is proposed as a
simplified alternative for the ViterbiMLSE equalizer.
The performance analysis is done on the following three factors: (i) Number of states, (ii)
distance covered on Standard Single Mode Fiber (SSMF) and (iii) Speed on the link. The
performance of 16state ViterbiMLSE equalizers for optical frequency discrimination receiver
(OFDR)based optical MSK transmission systems for PMD mitigation is also numerically
investigated.
iii
Acknowledgement
We are extremely grateful to Mr. ASIM MUKHERJEE for his revered guidance and
cooperation, which led to the completion of this Term Paper. Without his constant appraisal and
efforts, this task would have been a mere dream. He was always there to help us throughout this
project. He provided us with all the necessary resources and guidance during the project which
helped us to complete the project successfully.
Finally, we deem it a great pleasure to thank one and all that helped us directly or indirectly in
carrying out this term paper project work. We are also thankful to our colleagues and friends for
their support.
Date: 11
th
November, 2008
Place: Allahabad
Bharat Tandon
Shardul Verma
Vihag Mishra
iv
Contents
Sl no. Chapter Page no
Certificate i
Abstract ii
Acknowledgement iii
1 Introduction
01
2 Minimum Shift Keying
2.1 Minimum Shift Keying (MSK)
2.2 Advantages of MSK
03
04
04
3 MLSE and State Trellis implementation of Viterbi
Algorithm
3.1: Maximum Likelihood Sequence Estimator (MLSE)
3.2: State Trellis Structure
3.3: Viterbi Algorithm
3.4: Explanation of Viterbi Algorithm
3.5: A Viterbi Example
3.6: Viterbi in MLSE
06
07
07
08
09
10
11
v
4
5
6
7
Implementation of Viterbi Algorithm: Hidden Markov
Process
4.1: MATLAB code
MLSE Equalization in Optical Communications
5.1: Template Matching Algorithm
5.2: Intersymbol Interference (ISI) in Optical Systems
5.3: Construction of State Trellis Structure
5.4: MLSE Equalizers for OFDRBASED MSK Systems
5.4.1 OFDR as Optical FrontEnd
5.4.2 Comparison of MLSE Equalizer based on Viterbi
Algorithm and Template Matching Algorithm
5.5 Noise Distribution
5.6 Performance Analysis
5.7 Suggestions on Improvement of Performance
Scopes and Conclusion
6.1 Conclusion
6.2 Future Proposals
REFERENCES
13
14
15
16
16
16
17
17
17
18
19
20
21
22
22
23
1
Chapter 1
Introduction
2
The paper MLSE Equalizers for Frequency Discrimination Receiver of MSK Optical
Transmission System by Le N. Binh, Member, IEEE, Member, OSA, Thanh Liem Huynh, K. K.
Pang, Life Senior Member, IEEE, and T. Sivahumaran, Student Member, IEEE describes
maximumlikelihood sequence estimator(MLSE) equalizers based on either Viterbi algorithm or
template matching temple matching (TM) for the equalization of impairments imposed on the
minimum shift keying (MSK) modulation formats in long haul transmission without optical
dispersion compensation. The TMMLSE equalizer is proposed as a simplified alternative for the
ViterbiMLSE equalizer. It is verified that the ViterbiMLSE equalizer can operate optimally
when noise approaches a Gaussian distribution. Simulation results of the performances of the two
MLSE equalizers for optical frequency discrimination receiverbased optical MSK systems are
described. The transmission performance is evaluated in terms of:
1) The chromatic dispersion (CD) tolerance for both ViterbiMLSE and TMMLSE
equalizers;
2) Transmission distance limits of ViterbiMLSE equalizers with various number of states;
3) The robustness to fiber polarization mode dispersion (PMD) of ViterbiMLSE equalizers;
4) Performance improvements for ViterbiMLSE equalizers when utilizing sampling
schemes with two and four samples per bit over the conventional single sample per bit.
The potential of ViterbiMLSE equalizers in fiberoptic communications is reflected by the
effectiveness in combating the severe ISI caused either by fiber dispersion impairments or by
narrowband filtering of optical and electrical filters. Thus, ViterbiMLSE equalizers have been
adopted into optical fiber communications.
3
Chapter 2
Minimum Shift Keying
4
2.1: Minimum Shift Keying (MSK)
In digital modulation, minimumshift keying (MSK) is a type of continuousphase frequency
shift keying that was developed in the late 1960s. MSK is encoded with bits alternating between
quarternary components, with the Q component delayed by half the symbol period. It encodes
each bit as a half sinusoid. This results in a constantmodulus signal, which reduces problems
caused by nonlinear distortion. In addition to being viewed as related to OQPSK, MSK can also
be viewed as a continuous phase frequency shift keyed (CPFSK) signal with a frequency
separation of onehalf the bit rate.
Figure 1: Signal using MSK Modulation
The resulting signal is represented by the formula where a
I
(t) and a
Q
(t) encode the even and odd
information respectively with a sequence of square pulses as of duration 2T.
s(t) = a
I
(t)cos (t/2T) cos (2f
c
t)  a
Q
(t)sin (t/2T) sin (2f
c
t)
2.2: Advantages of MSK
Most of the recent studies have focused either on onoff keying (OOK)/amplitudeshift keying
(ASK) or DPSK modulation formats. Very few works have been reported on the performance of
MLSE equalizers for optical transmission systems employing minimum shift keying (MSK)
format.
In OOK and DPSK transmission systems, optical filters are used to reduce the effects of the
accumulated amplified spontaneous emission (ASE) noise. However, these filters usually have
large bandwidths (with reference to the signal bandwidth) in order to avoid filtering highpower
5
spectral sidebands of optical signals. As a result, the ASE noise is not greatly suppressed; thus
degrading the optical signaltonoise ratio (OSNR) level for the same signal power. In addition,
the distributions of OOK or DPSK noisecorrupted received signals do not follow Gaussian
profiles. This avoids the ViterbiMLSE equalizer from its optimal performance, and hence, the
exact noise distribution is required increasing the complexity of the MLSE receiver.
These problems, however, can be effectively mitigated by using MLSE equalizers in
combination with MSK format and narrowband optical filtering. A novel receiver scheme, the
optical frequency discrimination receiver (OFDR) scheme, has been proposed for 40 Gb/s optical
MSK systems and also reported. This receiver scheme enables a breakthrough dispersion
tolerance of up to 340 ps/nm for 1dB power penalty. Combined with OFDR, the Viterbi
MLSE equalizer serves as the postdetection electronic equalizer while OFDR serves as the
optical frontend. In addition, narrowly filtered noise is shaped back to a Gaussian distribution,
thus enabling the optimal performance for Viterbi algorithm as well as reducing the complexity
of the receiver.
The performance of the scheme is evaluated in terms of:
1) the CD tolerance for both ViterbiMLSE and TMMLSE equalizers;
2) transmission distance limits of ViterbiMLSE equalizers with various number of states;
3) the robustness to fiber polarization mode dispersion (PMD) of ViterbiMLSE equalizers; and
4) performance improvements for ViterbiMLSE equalizers when utilizing sampling schemes
with two and four samples per bit over the conventional single sample per bit.
6
Chapter 3
MLSE and State Trellis
implementation of Viterbi
Algorithm
7
3.1: Maximum Likelihood Sequence Estimator (MLSE)
MLSE is a wellknown technique in wireless and wireline communications for equalization and
detection of transmitted digital signals. A MLSE receiver determines a sequence as the most
likely transmitted sequence a when the conditional probability P
r
(ya) is maximized, where y is
the received noisecorrupted signal. If y is corrupted by a noise vector n which is modeled as an
additive Gaussian source (i.e., y=a+n), the maximization of the conditional probability is
equivalent to the minimization operation of the Euclidean distance d, given by
d `y
k
a
k

2
k
where k represents the index for the elements in the sequence. MLSE can be implemented
effectively by utilizing the Viterbi algorithm which is based on a state trellis structure.
3.2: State Trellis Structure
Figure 2: FSM modeling Block diagram
The FSM performs an onetoone mapping from a signal symbol a to a coded signal symbol c, as
shown in Figure 2. The FSM can be a convolutional encoder or particularly in fiberoptic
communications, an optical fiber channel. A state trellis structure is defined with the FSM.
8
Figure 3: The state trellis structure
Demonstration of a state trellis structure at epoch of the FSM is given in Figure 3. For a
simplified explanation, several assumptions are made. These assumptions and according
notations are described as follows:
1) Current state B is the starting state of only two branches which connect to states E and F,
denoted as: b
BE
and b
BF
. In general, b
B*
represents all the possible branches starting from
state B in the state trellis. In Figure 3, these two branches b
BE
and b
BF
correspond to two
possible binary information symbols of 0 or 1, respectively. The notation c(B,a=0)
represents the encoded symbol at the FSM output for the branch BE(b
BE
) and corresponds
to a 0 symbol (a=0). Similarly, in the case of branch BF, b
BF
, i.e., c(B,a=1)
corresponds to a 1 symbol (a=1). Similar notations are applied for all branches in the
trellis structure.
2) There are only two branches ending at state E which are from states A and B and denoted
as: b
AE
and b
BE
. In general, b
*E
represents all possible branches ending at state E in the
trellis structure.
3) The number of states 2
2
L between f
1
and f
2
, where f
D
= f
1
f
2
= R/2,
2
represents the GVD parameter of the
fiber and L is the fiber length. If the differential delay is fully compensated, optical light waves
in two paths arrive at two photodiodes simultaneously. It should be noted that these two
photodiodes are not connected. Optically filtered signals on each arm are then converted to
electrical domain by the photodiodes. These two separately detected electrical signals are then
sampled before being fed to the MLSE equalizer.
5.4.2: Comparison of MLSE Equalizer based on Viterbi Algorithm and Template Matching
Algorithm
The processing time and complexity of ViterbiMLSE equalizers are exponentially increased
with the number of states. Thus, the implementation of ViterbiMLSE equalizers at high speeds
such as 40 Gb/s may be beyond the reach of the processing speed of the present semiconductor
technology. However, with a small number of states, commercial chips for implementing
ViterbiMLSE equalizers are available at 10 Gb/s operating rate.
18
Template Matching MLSE equalization technique is a single shot template matching algorithm.
The values of each element in the weighting vector for calculation of metric m in TM Algorithm,
over which, minimization is performed, is selected with the intention that when the template is
compared with the received samples, less weighting is given to the samples further away from
the middle sample.
The computational time is mainly due to the scanning process throughout the rows of the matrix.
However, the proposed TMMLSE equalizer only requires the scanning of 2
+2
sequence patterns
instead of 2
2
+1
, leading to a considerable reduction of the processing time. Thus, the TMMLSE
equalizer is promising for high speed electronic processing in the novel application for digital
photonic transmission systems.
5.5: Noise Distribution
Calculations of branch and state metrics in the Viterbi algorithm optimally perform when the
noise distribution complies with a normal/Gaussian distribution. If noise distribution departs
from the Gaussian distribution, the minimization process in the Viterbi algorithm gives sub
optimum solutions. Thus, for optical OOK and DPSK systems in which large bandwidth optical
filters are used, more complex stages are required for estimating the exact noise distributions. It
is wellknown that noise distribution after the photodiodes in OOK and DPSK systems approach
the Maxwellian distribution.
Figure 5: Noise distribution following Gaussian profile due to narrowband optical filtering.
19
In optical MSK systems, the narrowband optical filter plays the main role in shaping the noise
distribution back to the Gaussian profile. Thus, the statebased Viterbi algorithm can achieve its
optimum performance and the equalizer structure is less complex than that of the OOK or DPSK
systems. In addition, a significant reduction in the computational time processing is also
accomplished.
5.6: Performance Analysis
With a small number of states (64 states), the uncompensated optical channel can equivalently
reach up to 928 km SSMF for 10 Gb/s transmission or 58 km SSMF for 40 Gb/s.
. Figure 6: Simulation setup for CD performance of ViterbiMLSE and TMMLSE equalizers for
40 Gb/s OFDRbased optical MSK systems
For 40 Gb/s MLSE equalizer for CD Mitigation:
1. Power level is kept low to avoid any nonlinearity, controlled by EDFA1.
2. EDFA2 acts as a load and controls OSNR.
3. At 40Gb/s bit rate, the onesided bandwidth B is equal to 5.2 GHz with onesided
bandwidths as 13% of the signal bandwidth for the filter.
4. For constraint length =5, trace back length=10(5)=50 and G has dimension [2
11
,11]
For 40Gb/s ViterbiMLSE Equalizers for PMD Mitigation:
The transmission link consists of a number of spans which are comprised of 100km SSMF (D =
+17 ps/(nm.km), =0.2 dB/km) and 10km DCF (D = 170 ps/(nm.km), =0.9 dB/km) . Input
power into each span (P
0
) is 3 dBm. The EDFA1 has a gain of 19 dB, hence providing input
20
power into the DCF to be 4 dBm which is lower than the nonlinear threshold of the DCF. The
transmission link consists of a number of spans which are comprised of 100km SSMF 17 ps nm
km 0.2 dB km and 10km DCF 170 ps nm km 0.9 dB km . Input power into each span is 3 dBm.
The EDFA1 has a gain of 19 dB, hence providing input power into the DCF to be 4 dBm which
is lower than the nonlinear threshold of the DCF. The 10 dB gain of EDFA2 guarantees the input
power into next span unchanged at 3 dBm value.
The advantages of using multiple samples per bit over the conventional single sample per bit in
the performance of Viterbi MLSE equalizers are numerically investigated and obtained by the
Monte Carlo method. It can be observed that the increase from 2 to 4 samples per bit does not
offer any gain in the performance of the Viterbi MLSE equalizer. Thus, two samples per bit is
preferred in order to reduce the complexity of the equalizer.
5.7: Suggestions on Improvement of Performance
One solution to improve the performance of the Viterbi MLSE equalizer without increasing
much the complexity is to implement sampling schemes with multi samples within one bit
period. Although, a greater number of electronic samplers are required, they only need to operate
at the bit rate. Moreover, it can be observed that there is no noticeable improvement with more
than two samples per bit period. Hence, the complexity ViterbiMLSE equalizers can be
affordable while its performance is significantly improved.
21
Chapter 6
Scopes and Conclusion
22
6.1: Conclusion
It has been shown that noise after tight optical filtering follows a Gaussian distribution. Thus, the
Viterbi algorithm can obtain its optimum performance without requiring more complex
algorithms for estimating the exact noise distribution. The combination of OFDRbased optical
MSK systems and ViterbiMLSE equalizers, particularly with the use of multisample sampling
schemes, was found to be highly effective in combating the ISI effects.
The proposed equalizers significantly extend the reach of uncompensated transmission link. With
a small number of states (64 states), the uncompensated optical link can equivalently reach up to
approximately 928km SSMF for 10Gb/s transmission or 58km SSMF for 40 Gb/s. The results
are comparable to the distance of 1040km SSMF as reported for OOK/IM systems, in which up
to 8192 trellis states are required.
This work has proved the improvement in performance of ViterbiMLSE equalizers by using the
sampling schemes with multiple samples per bit compared to the conventional single sample bit.
The novel schemes in which ViterbiMLSE equalizers are integrated with OFDRbased optical
MSK systems significantly extend the uncompensated transmission distance as well as provide
high robustness to ISI impairment.
6.2: Future Proposals
In addition to bringing out the features of this work, the project also includes a novel algorithm
and a fully working MATLAB code for implementation of Hidden Markov Model based on
Viterbi Algorithm and finding out the sequences obtained on a predefined set of Transition and
Emission Matrices upto any number of specified states in the trellis.
Another aspect that cannot be left untouched is the implementation of Viterbi based system on a
FPGA and analysis of the design using Cadence and Synopsys Tools. The works on these titles is
still under stress and may be carried out in future. It will help enhance the speed of the complex
Viterbi Algorithm.
Also, the implementation of TM algorithm on multi samples per bit, if possible, can bring about
a revolution in the speed of the systems for MLSE based equalizers for OFDR in MSK
transmission systems.
23
REFERENCES
[1] MLSE Equalizers for Frequency Discrimination Receiver of MSK Optical Transmission
System; Le N. Binh, Member, IEEE, Member, OSA, Thanh Liem Huynh, K. K. Pang, Life
Senior Member, IEEE, and T. Sivahumaran, Student Member, IEEE
[2] Optical Fiber Communications, Second Edition; Gerd Keiser
[3] Digital and Analog Communication Systems; K.Sam Shanmugam