An Application of GA for Symbol Detection in

MIMO Communication Systems

Sajid Bashir
Centre for Advanced
Studies in
Engineering Islamabad,
Pakistan

Adnan Ahmed Khan

Centre for Advanced
Studies in
Engineering Islamabad,
Pakistan

Abstract
Multi-Input
Multi-Output
(MIMO)
based
communication system architecture promises increased
capacity and high data rates. Increase in the number of
transmit antennas and using higher order complex
modulation schemes achieves even higher performance
but with exponentially increasing complexity at the
receiver end. This paper explores the application of
genetic algorithm (GA) for reducing complexity in solving
this NP Hard problem. This approach is particularly
attractive as GA is well suited for physically realizable,
real-time applications, where low complexity and fast
convergence is of absolute importance. While an optimal
Maximum Likelihood (ML) detection using an exhaustive
search method is prohibitively complex, simulation
results show that the GA optimized MIMO detection
algorithm results in near optimal Bit Error Rate (BER)
performance, with significantly reduced complexity.
Results also suggest that the GA based MIMO detection
out-performs the Vertical Bell labs Layered Space Time
(V-BLAST) detector in BER performance without severely
increasing the systems complexity

1. Introduction
Wireless communication using MIMO antenna
architecture also referred to as spatial multiplexed system
is one of the most significant technological developments
of last decade, which promises to play a key role in
realizing the tremendous growth in the field of
communication. MIMO systems achieve transmit and
receive diversity by employing a Multi-Element Antenna
(MEA) structures at both the transmitter and receiver. The
challenge is to design signals to be sent by the transmit
array and algorithms for processing those seen at the
receive array, so that the quality of the transmission (i.e.,
bit error rate) and/or its data rate are improved. These
gains lead to increased reliability, lower power
requirements (per transmit antenna) and higher composite
data rates (either higher rates per user or more users per
link). The most substantial benefit offered by MIMO
technology is attainment of these gains without the need

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

Muhammad Naeem
Engineering Sciences
Department, Simnon
Fraser University,
Canada

Syed Ismail Shah

IQRA Univeristy
Islamabad,
Pakistan

for additional spectral resources. An information theoretic

analysis on the capacity of MIMO systems was presented
in [1] assuming flat, quasi static and spatially independent
Rayleigh fading environment with perfect Channel State
Information (CSI) at the receiver. The study concluded
that the capacity of multiple antenna channels increases
linearly with the smaller of the number of transmit and
receive antennas.
On the other hand MIMO systems demand tremendous
signal processing power at receiver side. Simultaneous
transmission of signals from multiple transmitters
complicates detection of composite signal at the receiver.
Many MIMO detection algorithms for spatial
multiplexing [2] are proposed and can be divided into
linear and non-linear categories. Linear detectors are
found to be computationally less complex with degraded
performance when compared to non-linear methods.
Maximum Likelihood (ML) and Vertical Bell labs
Layered Space Time (V-BLAST) detectors [3] are two
famous non-linear MIMO detection methods. The ML
detector outperforms VBLAST in terms of BER
performance, while the later has significant advantage in
terms of complexity. Worst-case complexity of
computing exact ML solution is generically exponential,
due to NP-hardness [1] and cannot be solved in
polynomial time. This paper focuses on non-linear
detectors and makes effort to improve their performance
using well known heuristic approach of GA. An
Algorithm which works reasonably well on many cases,
but for which there is no proof that it is always fast (e.g.,
evolutionary techniques) are classified as Heuristic
Algorithms. GA is also member of this family of
algorithms that has found numerous applications as a
search tool in various fields, [8][9] are example of GA
being applied in field of communications.
Excessively high complexity increasing exponentially
with increase in number of antennas is a major drawback
restricting ML detectors practical implementation in
spatial multiplexed systems. Performance analysis at
higher signal to noise ratios (SNR) rules out the choice of
VBLAST because of significant comparative degradation
in BER performance. Seeking the optimal solution, we
report a GA assisted MIMO detection algorithm with a

reasonable performance complexity tradeoff. GA based

detector operates on an initial population of potential
solutions and produces better approximations to the
solution using ML equation as an objective function.
Each iteration results in a new set of possible solutions
created by selecting individuals based on fitness level by
applying the so-called genetic operators; crossover and
mutation. Direct application of GA, i.e., full power (or
pure) GA, requires fine-tuning of its related parameters,
and extensive computation, yet as a MIMO detector it
might have lower computational complexity than
optimum detector based.
The rest of the paper is organized as follows. Section-2
provides the wideband spatial multiplexing system model
and sets-up MIMO detection problem for flat, quasi static
and spatially independent Rayleigh fading environment.
A brief overview of existing MIMO detectors is given in
section-3. Section-4 details the proposed detection
algorithm. Performance of the proposed detector is
reported in next sections followed by conclusions.

X1

INPUT
DATA
STREAM

DEMULTIPLXER

Xn

P
N

n1

P
N

nm

r1

rm

DETECTOR

DETECTOR
OUTPUT

rM

XN

P
N

Eq.1 illustrates the time selective fading case with one

transmit antenna and one receive antenna. Because the
channel under consideration is slowly fading, the
coefficients will be independent of time, i.e., h[l] = h. In
MIMO case MEAs are deployed at both the transmitter
and the receiver. Consider the MIMO system shown in
fig.1 where N different signals are transmitted and arrive
at an array of M (N M) receivers via a rich-scattering
flat-fading environment. The block transmission is
assumed to contain one symbol i.e. L=1. Baseband
equivalent model of received signal vector at each
sampling instant can be represented as:

r=

2. MIMO detection in flat fading channels

2.1 System model
The channel model assumed is that of quasi-static
fading. In this case the path attenuations do not vary over
the duration of the block transmission with length L,
thus the channel becomes an LTI system within this
block. The attenuations also do not vary over the
spectrum of the transmitted signal, and its impulse
response is a scaled Dirac delta function. Thus the
process can be modeled mathematically as a single-tap
filter with complex coefficient h[l], or equivalently by
real path attenuation [l] and phase shift [l]:
r [l ] = [l ]e j [ l ] x[l ] + n[l ]
(1)
= h[l ] x[ l ] + n[l ]

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

(2)

Where r is an Mx1 vector of received symbols at each

antenna, x is an Nx1 vector of symbols transmitted by
each antenna, and n is an Mx1 vector of complex
Additive White Gaussian Noise (AWGN) random
variables seen at each receive antenna. The channel
matrix H is an MxN matrix, whose elements hij represent
the complex fading coefficients experienced by a signal
transmitted from transmit antenna j to receive antenna
i. P is total transmit energy for one transmit antenna
system and is normalized for N transmit antenna system.
Each element of x is determined from the same set S
comprised of C=2b constellation points where b is the
number of bits per symbol. The receiver is also assumed
to have perfect knowledge of the channel coefficients.
This is a reasonable assumption when the fading is slow
enough to allow estimation of the CSI with negligible
error, as in the case of fixed wireless systems.

nM

Figure 1. MIMO system model with N transmit and M

receive antennas

P
Hx + n
N

n
r

x
Detector

Figure 2. Linear system model for MIMO communication

system

2.2 Problem formulation

Transmitted symbols from a known finite alphabet =
{x1,,x}of size = CN are passed to the channel. The
detector chooses one of the possible transmitted symbol
vectors from . The optimal detector for a MIMO
system returns x = x* , the symbol vector whose
(aposteriori) probability of having been sent, given
observed vector r, is the largest:

x * = arg

max
P(x r)
x

(3)

x * = arg

max P ( r x ) P ( x )

(4)

x
P (r )
This is known as Maximum Aposteriori Probability
(MAP) detection rule. Making the standard assumption
that symbol vectors x are equiprobable i.e. P(x) was
sent is constant, the MAP detector rule can be written as:

max
x* = arg
P(r x)
x

(5)

A detector that always returns an optimal solution

satisfying eq.5 is called Maximum Likelihood (ML)
detector. Assuming the additive noise n to be white and
Gaussian, the ML detection problem can be expressed as
minimization of the squared Euclidean distance to a target
vector r over N-dimensional finite discrete search set:

min
P
xML = arg
r
H.x
x
N

3.2 Non-Linear MIMO detectors

3.2.1.

VBLAST

A popular nonlinear combining approach is the

vertical Bell labs layered space time algorithm
(VBLAST) [3] This detection method is also called
Ordered Successive Interference Cancellation (OSIC). It
uses the detect-and-cancel strategy similar to that of
decision-feedback equalizer. Either ZF or MMSE can be
used for detecting the strongest signal component used
for interference cancellation. The performance of this
procedure is generally better than ZF and MMSE.
VBLAST provides a suboptimal solution with lower
computational complexity than ML. However, the
performance of VBLAST is degraded due to error
propagation.

(6)

Thus conventional ML detection scheme needs to

examine CN symbol combinations resulting in an
exponentially increasing computational complexity with
increases in C and/or N. High speed processing
requirements of real time applications demand a
comparatively simplified detection scheme. We report a
GA assisted spatial multiplexing systems symbol detector
that views MIMO symbol detection issue as a
combinatorial optimization problem and try to
approximate near optimal solution iteratively to counter
ML complexity.

3.2.2.

ML detector

Maximum Likelihood detector is optimal but

computationally very expansive. ML detection is not
practical in large MIMO systems.
Define cost function,
cost, variables
Select parameters

START

GENERATE
INITIAL
POPULATION

DECODE
CHROMOSOME

FITNESS
EVALUATION

OPITMALITY
TEST

3. Existing MIMO detectors

3.1 Linear MIMO detectors

YES

SOLUTION
FOUND WITH
BEST
INDIVIDUAL S

NO
MUTATION

CROSSOVER

SELECTION

GENERATE NEW POPULATION

A straightforward approach to recover x from r is to

use an N x M weight matrix W to linearly combine the
elements of r to estimate x, i.e. x = Wr.

Figure 3. The GA flow diagram

3.1.1.

4. GA based detection for MIMO systems

Zero-Forcing (ZF)

The ZF algorithm attempts to null out the interference

introduced from the matrix channel by directly inverting
the channel with the weight matrix [1].

3.1.2.

Minimum mean squared error (MMSE)

A drawback of ZF is that nulling out the interference

without considering the noise can boost up the noise
power significantly, which in turn results in performance
degradation. To solve this, MMSE minimizes the mean
squared-error, i.e. J(W) = E{(x- x )*(x- x )}, with respect
to W [4], [5].

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

GA is an inspiration based on principles of natural

genetics and selection. Algorithm starts by defining
optimization variables, optimization cost and the cost
function. Convergence/Fitness test follows different
components of algorithm, as explained.

4.1 Initialization of GA
All potential solutions of a problem are
level to simplify the following GA
operations. In MIMO detection x
ML search space are coded as binary

encoded in bit
recombination
that form the
strings called

chromosomes. It is common to select the initial

population randomly from solution space or derive from a
linear/non-linear detector output. Each chromosome is a
combination of the probable solution for all transmit
antennas. Normally, the population size Npop is taken as
the product between N, the number of transmit antennas,
and C, the number of all possible solutions of each
transmit antenna.
(7)
N pop = 2 b N

by cost. This new population of Nkeep chromosomes forms

mating pool of good parents.

4.2 Fitness evaluation using cost function

4.6 Crossover

Each member of the population at each generation is

evaluated, and according to its fitness value (output of the
cost function), it is assigned a probability to be selected
for reproduction. Following equation acts as the cost
function to evaluate the fitness of each chromosome:

Crossover operation is applied to pairs of selected

parents and creates offspring. Process of mating by
applying crossover operator creates two offspring by
combining subparts of bit strings of two selected parents
[6]. The crossover point is randomly selected along the
chromosome lengths and portions up to that point are
exchanged between two parents. Probability of crossover
pc is user controlled and usually set to a high value.

f = r Hx

(8)

The optimal solution of eq.8 should yield a minimum

value. ML detector evaluates x results in
computation time that varies exponentially with the
number of transmit antennas. As will be explained, the
use of GA in a MIMO system can reduce the computation
times of eq.8 significantly. The best chromosome in a
generation should have the least value of the objective
function. If the value of the best chromosome in the
present generation is larger than its counterpart in the
previous generation, the chromosome with the largest
value of the objective function in the present generation
will be replaced by the best chromosome of the previous
generation. This operation ensures that at least the useful
information contained in the present generation is passed
on to the next.

4.3 Optimality test

If the optimal criterion is satisfied, that is, when any
one of all f in the population is less than a predetermined
threshold, or if the generation number has exceeded a
predefined value, which is also commonly taken as the
product between N and C, then go to 4.8. Otherwise, go
to Step 4.4.

4.4 Selection
This process uses fitness value and serves to provide
chromosomes for the subsequent recombination
operations. Of Npop chromosomes in a generation, only
Nkeep= Npop * psel survive for mating, and bottom Npop
Nkeep are discarded to make room for new offspring.
Deciding how many chromosomes to keep is arbitrary
and normally 50% (Psel = 0.5) of population survives
Selection process after chromosomes have been sorted

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

4.5 Reproduction
This step is intended to replace the chromosome with
largest objective function value by the best chromosome
of the same generation. The Offspring to replace the bad
parents is created by the processes of Crossover and
Mutation.

4.7 Mutation
These offspring are mutated through mutation operator
and new members of next generation are produced.
Mutation operator simply alters each bit of the binary
chromosome randomly with a user-controlled probability
pm. Generally, the crossover probability pc is close to 1
and the mutation probability pm is close to 0.
New individuals replace members of previous
populations with worst fitness values. Algorithm iterates
until optimization of objective function is achieved. For
given predefined number of generations and population
size, the computation times of eq.8 vary linearly with
(Npop X Ngen) for GA based detector, which is much
smaller than the factor of 2NXbin the ML detection. The
improvement is clearly significant.

4.8 Decision making

If the defined optimality criterion is met the algorithm
terminates its execution and output is generated.
Otherwise further refining of solution is carried out
through step 4.2 by producing next generation.

5. Simulation and numerical results

BER performance analysis of reported GA-MIMO
detector and its comparison with other MIMO detectors is
presented in this section. Improvement in performance of
spatial multiplexing system using reported detector is
shown with simulation results and a comparison of
computational complexity is also presented in last section.

5.1 BER Performance Analysis

Performance of GA-MIMO detector is analyzed for a
4x4 (MxN) and a 6x6 MIMO system using 4-QAM and
16-QAM schemes. Population size Npop and number of
generations Ngen are dependent on system configuration
i.e. number of transmit antennas used in spatial
multiplexing system and the QAM size. For each case the
parameters of GA-MIMO detector are different and noted
against in the simulation results. For example for a 4x4,
4-QAM system, Npop equals 14 and it grows to 26 for
16-QAM system. Ngen is kept as 10 and 18 for the two
systems respectively. The SNR (Eb/No) is average signal
to noise ratio per antenna / 2 where average power per
antenna is and 2 is noise variance. Simulation
environment assumes quasi static Rayleigh flat-fading
channel with no correlation between sub-channels.
Fig.4 presents the bit error rate (BER) versus Eb/No
performance of reported GA detector compared with ML,
VBLAST and ZF detectors in 4x4 spatial multiplexing
system using 4-QAM. As shown by simulation results of
fig.4, at BER of 10-3 proposed detector has a performance
gain of approx 5-dB compared to VBLAST.
Fig.5 shows BER performance comparison of two 4x4
MIMO systems using different modulation schemes.
Using higher order modulation scheme increases the
complexity of optimization problem. To counter this
increased complexity Npop and Ngen are increased from 14
to 26 and from 10 to 18 for 16-QAM system respectively.
Relative improvement in performance compared to
VBLAST for case of 16-QAM seems little but as shown
in fig.6 a further tuning of GA parameters enhances
relative BER performance of reported detector. This
results in a linear increase in computational complexity as
discussed in next section. Thus to achieve a balance
between BER performance and computational complexity
careful selection of values for GA parameters is vital.
Performance analysis of MIMO systems with different
configurations using identical modulation scheme is
presented in fig.7. Systems with 4x4 and 6x6
configurations using 4-QAM are used. Ngen is taken as 10
and 16 whereas Npop is taken as 14 and 20 for 4x4 and
6x6 systems respectively. Relative improvement in
performance for the two systems using GA-MIMO
detector at 10-4 BER is also shown. It is evident that
relative improvement for higher order system is more
than for a lower order system. For example at 10-4 BER,
the performance gain for 4x4 system is approx 4 dB
whereas for 6x6 system it is approx 7 dB.

5.2 Computational complexity comparison

Computational complexity of reported GA detector is
examined and compared with ML and VBLAST detectors

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

in this section. As the hardware cost of each algorithm is

implementation-specific, we try to provide a rough
estimate of complexity in terms of number of complex
multiplications. The computational complexity is
computed in terms of the N, M and C i.e. constellation
size.
ML detection requires N(M+1)CN complex
multiplications, as seen from (6). Where CN(NM) is for
matrix multiplication and CN(N) is for square operation.
For VBLAST, pseudo-inverse of matrix (HHH)-1HH
requires 4N3+2MN2 computations [7]. The pseudoinverse matrix is calculated N times with decreasing
dimension. In addition, the complexity of ordering and
interference

canceling

is

N 1

[N (N

i) + 2 N ] .

i=0

Therefore, total complexity of VBLAST (VBLAST) results

in:
N

(4i
i =0

N 1

+ 2 Mi 2 ) + ( N ( N i ) + 2 N )

(9)

i =0

1
5 2
7

VBLAST = N 4 + + M N 3 + + M N 2 + NM (10)

3
2 3
2

For proposed detector, before applying GA operators,

fitness of each chromosome in initial population (Npop) is
assessed. Based on the parameters used for simulation i.e.
selection probability ps=0.5, crossover operator with
pc=0.9 is applied to pairs of Nkeep chromosomes which
undergo logical AND/OR operations. Hence number of
crossover operations is equal to 0.5*Npop(1-ps)*pc.
Mutation of one bit i.e. logical NOT operation is applied
to crossover results with probability pm=0.1 on a
population of size Npop-Nkeep and these logical operations
are performed Ngen times. However the most expensive
computational process of multiplication results in
complexity of:
(11)
Genetic = N pop N gen (MN )

Since the Proposed detector takes initial solution guess

as the output of ZF-VBLAST therefore, total
computational complexity of proposed detector results in:
GAMIMO = Genetic + VBLAST (12)
Computation complexity of reported GA-MIMO
detector for different system configurations along with
the performance gains are mentioned in table-1. The
results are derived from simulations results for BER
performance analysis discussed in previous section. The
complexity estimate in table-1 is only meaningful in the
order of magnitude sense since it is based on the number
of complex multiplications.

5.3 Complexity-Performance trade off

For a 4x4 MIMO system BER performance of
different detectors shown in fig.4. As shown in table.1 a
gain of 5 dB is achieved for proposed detector over

VBLAST at a BER of 10-4. The complexity of GAMIMO detector for achieving this performance gain is
57.6% to that of ML detector.
The complexity performance trade off is found to be
better when the proposed GA-MIMO detector is used
with higher order system configurations. As shown with
fig.5 and fig.6 for a 4x4 system with 16-QAM the gain in
BER performance increases from 1 dB to 6 dB with a
corresponding increase in relative complexity from
0.626% to 1.32%. Performance gain of 7 dB is observed
for a 6x6 system with 4-QAM with a relative complexity
of 8.5%. These numeric figures suggest that GA-MIMO
detector achieves a complexity/performance trade off
balance for system configurations where practical
implementation of ML detector is not realizable and
performance of VBLAST is severely degraded.

[4] S. Haykin, Adaptive Filter Theory, 3rd ed. Prentice-Hall,

1996.
[5] A. Sayed, Fundamentals of Adaptive Filtering. WileyIEEE Press, 2003.
[6] Randy L.Haupt and Sue Ellen Haupt, Practical Genetic
Algorithms, Second Edition. A John Wiley & sons, 2004.
[7] G. H. Golub and C. F. V. Loan, Matrix Computations, 3rd
ed. John Hopkins University Press, 1996.
[8] M. Y. Alias, S. Chen and L. Hanzo, Genetic Algorithm
Assisted Minimum Bit Error Rate Multiuser Detection in
Multiple Antenna Aided OFDM, 2004.
[9] Kai Yen and Lajos Hanzo, Genetic Algorithm Assisted
Joint Multiuser Symbol Detection and Fading Channel
Estimation for Synchronous CDMA Systems, IEEE
Journal on selected areas in communication, vol. 19, No. 6,
June 2001.
0

10

6. Conclusion

7. References

-1

BER

10

-2

10

-3

10

-4

10

20
Eb/No

30

40

10

ML 4-QAM
VBLAST 4-QAM
GA-MIMO 4-QAM
ML 16-QAM
VBLAST 16-QAM
GA-MIMO 16-QAM

-1

10

-2

10

10

[2] H. Bolcskei, D. Gesbert, C. and Papadias , Space-Time

Wireless Systems: From Array Processing to MIMO
Communications, Cambridge University Press, 2005.

10

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

10

System with Ngen=10 and Npop=14

[1] I. Emre Telatar. Capacity of multi-antenna Gaussian

channels. Technical report, October 1995.

[3] G. J. Foschini, Layered space-time architecture for

wireless communication in a fading environment when
using multiple antennas, Bell Labs Technical Journal, vol.
1, pp. 41-59, Autumn 1996.

Figure 4. Bit Error Vs Eb/No for a 4x4 4-QAM MIMO

BER

In this paper, power of GA is explored when used as a

search tool. The algorithm is applied to MIMO detection
problem and it achieves reduced computational
complexity for complex system configurations. This
natural optimization method shows promising results
when compared with traditional ML detection for systems
with increased number of antennas and using higher order
complex modulation schemes. The optimality is however
compromised slightly but reduction of computations show
a major gain. Although VBLAST detector has a reduced
complexity, its BER performance is much inferior to the
proposed detector. The simulation results show that the
proposed detector significantly out-performs the
VBLAST detector without severely compromising the
systems complexity.
Simulation results also show that a trade off can be
made between complexity and optimality. Employing
parallel processors for proposed algorithm will further
reduces this complexity. GA based solution of MIMO
detection problem has proven to be very cheap and shows
a promise for Heuristic Algorithms in MIMO detection
for complex modulation schemes.

ML
ZFE
VBLAST
GA-MIMO

-3

-4

10

20
Eb/No

30

40

Figure 5. BER Analysis for 4x4 system using 4-QAM and

16-QAM, with values of Npop taken as 14 and 26 & Ngen
taken as 10 and 18 respectively.

10

-1

10

ML Detector
(4X4 System)
ZFE
VBLAST
GA-MIMO

-1

10

--------- 6x6 System

-2

-2

10
BER

BER

10

-3

10

-3

10

-4

10

VBLAST
POPULATION SIZE = 30
POPULATION SIZE = 40
ZF
POPULATION SIZE = 36
10
20
Eb/No

-4

10

30

40

Figure 6. Effect of Population Size on BER performance

for a 4x4 system using 16-QAM with Npop = 26

10

15
20
25
Eb/No (dB)

30

35

Figure 7. BER Performance Comparison for a 4x4 and

6x6 System using 4-QAM with Ngen= [10 16] and Npop= [14
20] for the two systems.

Table 1. Complexity and performance analysis

System
Configuratoin

Population
Size
(Npop)

Number of
Generatoins
(Ngen)

ML

GA-MIMO =
Genetic+VBLAST

4x4 MIMO
System with
4-QAM

14

10

5120

2240 + 712
= 2952

26

18
1310720

7488 + 712
= 8200
13192
15688
17352

172032

11520 + 3048
= 14568

4X4 MIMO
System with
16-QAM
6X6 MIMO
System with
4-QAM

30
36
40
16

26

20

Third International Conference on Natural Computation (ICNC 2007)

0-7695-2875-9/07 $25.00 2007

Performance/Complexity
Comparison
ML & GA-MIMO
Complexity
Relative
Performance Improvement
to ML
relative to VBLAST
GA MIMO
(at BER = 10-4)
100
ML
57.6%

5 dB

0.626 %

1 dB

1%
1.19%
1.32%

2 dB
4 dB
6 dB

8.5%

7 dB