A hybrid precoding design in mmWave massive
MIMO systems
Homeyra Rahbari Kahjough† , Reza Abdolee‡ , Behzad Mozaffari Tazehkand†
Emails:h_rahbari@alum.sharif.edu, rabdolee@csub.edu, mozaffary@tabrizu.ac.ir
†University of Tabriz, Iran ‡California State University, Bakersfield
Abstract—Millimeter wave (mmWave) communication using
massive multiple input multiple output (MIMO) techniques is
one of the key enabling technologies for high capacity 5G
cellular networks. However, the hardware complexity and the
high power consumption in massive-MIMO array hinder its
integration. Hybrid precoding technique, which combine large-
dimensional analog preprocessing with low-dimensional digital
processing, is a promising approach for reducing the hardware
cost and power consumption in massive MIMO systems. In this
paper, we propose a novel algorithm for hybrid minimum mean-
square error (MMSE) precoding, inspired by two algorithms
namely the effective alternating minimization (AltMin) and
orthogonal matching pursuit (OMP) algorithm. These algorithms
optimize the digital and analog precoders. In addition, the
flexible partially-connected architecture that used in the proposed
structure will be more energy efficient, since it requires fewer
phase shifters. Numerical simulation results demonstrate that
the proposed hybrid precoder with low-complexity scheme can
significantly perform near-optimal fully-connected structure. It
can provide an appropriate trade-off between performance and
complexity compared to the conventional structures.
Index Terms—Hybrid precoding, mmWave MIMO systems,
RF chain, phase shifter, alternating minimization approach, low-
complexity.
I. INTRODUCTION
Millimeter wave (mmWave) multiple input multiple out-
put (MIMO) systems which use hybrid analog and digital
precoding are one of the promising techniques for future
communication networks. In mmWave MIMO systems due to
the use of small wavelengths, a large number of antennas can
be packed in a small area. The number of radio frequency (RF)
chains increases when the large antenna arrays are used, that
results in higher cost and power consumption of the system.
To reduce the number of RF chain in mmWave massive
MIMO system, the hybrid precoding scheme was proposed
[1–3] is promising solution to these problems. In this scheme
analog precoders in the RF domain are combined with digital
precoders in baseband to reduce the number of RF chains.
The decomposition of optimal precoder into hybrid pre-
coders is realized using an algorithmic precoding solution
based on the concepts of compact sensitivity such as or-
thogonal matching pursuit (OMP) [4]. It takes an optimal
unconstrained precoder as input and approximates it as a
linear combination of beam steering vectors applied at analog
RF. They have developed sparse precoding in receiver-side
by designing hybrid minimum mean-square error (MMSE)
combiners via basis pursuit. Reference [5] compares the
performance of hybrid precoder design with optimal digital
and simplified beam steering systems. They have proposed
the gradient pursuit (GP) algorithm as a novel solution that
is cost-effective and fast, also perform well compared to the
OMP algorithm.
In [6], a hybrid precoding algorithm with low computa-
tional complexity has been developed by exploiting the semi-
unitary optimum precoder. Effective alternating minimization
(AltMin) algorithms was proposed in [7] in which the hybrid
precoders are designed using a matrix factorization scheme.
The authors of [8] have proposed a hardware efficient im-
plementation for hybrid precoding, named the fixed phase
shifter (FPS) implementation. In this technique, a dynamic
switch network which is adaptive to the channel states, provide
connections from a small number of phase shifters to the
antenna array. The hybrid precoder and switch mode are
determined based on an effective alternating minimization
(AltMin) algorithm.
A. Contributions
The main purpose of this paper is to find a low-complex
hybrid precoder with near-optimal performance. We provide
an algorithmic hybrid precoding solution inspired from the
concept of orthogonal matching pursuit (OMP) algorithm and
alternating minimization (AltMin) algorithm in combination.
The proposed algorithm takes an optimal unconstrained pre-
coder as input and approximates it as a linear combination of
beam steering vectors that can be applied to RF chain. Further,
we extend this sparse precoding approach by designing hybrid
minimum mean-square error (MMSE) precoders. To have low
RF-complexity technique for mmWave MIMO systems, we
use the flexible partially-connected architecture in computing
the analog RF precoding matrix. Our results indicate that
the performance gains of the proposed scheme with fewer
number of phase shifters is near to the fully-connected and
conventional structures.
We organize the remainder of this paper as follows. In
section II, we introduce the system model and formulate
the corresponding optimization problem. Section III discusses
a new hybrid precoding design algorithm. In this section,
we represent the proposed hybrid precoding algorithm based
on flexible partially-connected architecture and analyze its
computational complexity. Simulation results are presented in
section IV, followed by concluding remarks in section V.
B. Notations:
The lowercase and uppercase bold letters are used for
vectors and matrices, respectively. [.]∗ signifies the conjugate
transpose. IN is the N ×N identity matrix and the expectation
operator and the real part of a complex variable are represented
by E[·] and Re[.]. T r(.) and vec(.) indicate the trace and
vectorization.
II. FORMULATION OF THE HYBRID PRECODER
DESIGN PROBLEMS
A. System Model
In this section, we describe a system model for the downlink
hybrid precoding system. We assume that the BS is equipped
with M antennas to transmit data to K users and there are
NtRF RF chains at the transmitter.
The channel vector of k th user is denoted as hk ∈
M ×1
C , k = 1, . . . , K whose entries are i.i.d. CN (0, δk ) where
δk represents large scale fading. To model favorable propaga-
tion, these channel vectors should be pair-wisely orthogonal.
The downlink channel from BS to all users is defined as
H = [h1 , h2 , . . . , hK ].
In the hybrid precoding structure, Va = ture. The implementation of conventional hybrid precoding is
RF
[va,1 , va,2 , . . . , va,NtRF ] ∈ CM ×Nt is the analog RF
RF
precoder and Vd = [vd,1 , vd,2 , . . . , vd,K ] ∈ CNt ×K is
the digital precoder.
Let s ∈ CK×1 , where E{ss∗ } = IK is the signal
vector transmitted by the BS antenna array and n is Gaussian
noise signal with zero mean and unit variance. The sampled
baseband signal received by all users is given by:
y = H∗ Va Vd s + n (1)
The received signal by the k th user can be written as
yk = h∗k Va Vd s + nk (2)
th
where nk is the additive noise at the k user.
B. Millimeter Wave Channel Model
We consider a clustered channel model to characterize the
sparse scattering features. Thus, the narrow-band mmWave
channel matrix can be written as
Ncl Nray
1 X X
H= p αkl a∗t (φta,kl , φte,kl )ar (φra,kl , φre,kl ) (3)
Ncl Nray k=1 l=1
The channel consists of Ncl clusters and each cluster contains
Nray propagation paths. In this equation, αkl , the complex
gain of the lth ray in the k th cluster, is assumed to be a
Gaussian random variable with zero mean and unit variance.
The azimuth (elevation) angle of departure φta,kl (φte,kl ) and
azimuth (elevation) angle of arrival φra,kl (φre,kl ) are of a
truncated Laplacian distribution. The vectors a∗t (φta,kl , φte,kl )
and ar (φra,kl , φre,kl ) are, respectively, the transmit and receive
antenna array response vectors [9].
C. Architecture of Analog Precoding
The analog part of the hybrid structure is equipped by basic
analog components such as analog adders and analog phase
shifters which can only change the phase of signals. This
causes some constraints on designing the analog precoding
matrix depending on the structure of the analog precoder.
The motivation behind the RF domain precoder design is
coordination of the channel elements to obtain great power
efficiency. Hence, the (ı, )th element of Va is given by
1
va (ı, ) = aı, eφı, = √ eφı, (4)
M
where aı, denotes the magnitude of the (ı, )th element of Va
and φı, is the phase of the (ı, )th element of H. The ideal
phase shifters harvest continuous phases without quantization,
however such phase shifters are difficult to implement.
1) Fully-Connected Architecture: In the fully-connected
architecture as shown in Fig. 1 (a), there is a connection
from each RF chain to all of the antenna elements via a
large number of phase shifters. Therefore, each RF chain
is connected to all the antennas at BS. This connection
requires NtRF analog phase shifters and one RF adder for
each antenna. However, this structure can provide preferable
spectral efficiency compare with the partially-connected struc-
less feasible in practical implementation [10].
2) Partially-Connected Architecture: In the partially-
connected structure [11], each RF chain is connected to a
subset with M/NtRF antennas, where each antenna element
is equipped with one analog phase shifter. Therefore, in
the partially-connected architecture the analog precoding ma-
trix needs to have M analog phase shifters. The partially-
connected structure that is more appropriate for practical
application, is illustrated in Fig. 1 (b).
3) Flexible Partially-Connected Architecture: In the flex-
ible partially-connected structure as presented in Fig. 1 (c),
each RF chain is connected to M/NtRF analog phase shifters.
Unlike partially-connected method, the antennas associated
with each RF chains are not fixed. Each phase shifter is
connected to all of the antennas through M independent on/off
switches.
D. Architecture of Digital Precoding
Inter-user interference at the baseband, is neutralized by
applying zero-forcing precoding given by
Vd = H∗eq (Heq H∗eq )−1 (5)
where Heq = H∗ Va is a low dimension K × NtRF matrix.
The final digital processed signal is:
y = H∗ Va Vd s + n
= Heq H∗eq (Heq H∗eq )−1 s + n = s + n (6)
where H  CM ×K is the channel matrix. Assume that the
transmitted signals of different users are independent from
each other and from the noise signals. The received signal
at the k th user can be written as:
 K
X
yk = h∗k Va Vd,k sk + h∗ı Va Vd,ı sı + nk (7)
ı=1,ı6=k
2
where nk ∼ CN (0, σn,k ) denotes the additive white Gaus-
th
sian noise at the k user. Hence, the downlink signal-to-
interference-plus-noise-ratio (SINR) at the k th user can be
represented as:
|h∗k Va Vd,k |2
SIN Rk = PK (8)
2
|h∗ı Va Vd,ı |2 + σn,k
ı=1,ı6=k

Gaussian symbols are transmitted over the mmWave channel,

the spectral efficiency achieved is given by:
K
X
Rsum−rate = log2 (1 + SIN Rk ) (9)
k=1

(a) Fully-connected
E. Problem Formulation
In this paper, we will mainly focus on the precoder design.
Instead of maximizing the spectral efficiency, we design Va
and Vd to maximize the mutual information achieved by Gaus-
sian signaling over the mmWave channel [4]. The precoder
optimization problem can be stated as:

{Va,opt , Vd,opt } = arg min kVopt − Va Vd k (10)

Va ,Vd

s.t. kVa Vd k2 = K
where Vopt stands for the optimal fully digital precoder,
and Va , Vd respectively, are the analog and digital precoders
to be optimized.

III. HYBRID PRECODING DESIGN

In this section, we introduce a novel hybrid precoder
(b) Partially-connected
structure to optimize the digital and analog precoders. The
proposed hybrid precoding algorithm as described in Algo-
rithm 1, uses the flexible partially-connected structure (low-
complexity architecture) that reduces total number of phase
shifters compared to the fully-connected structure. In addition,
the proposed algorithm is combined with the sparse MMSE
solution to find linear MMSE estimators with complex struc-
tural constraints for mmWave systems.
The hybrid precoder optimization problem can further be
stated as follows:

{Va,opt , Vd,opt } = arg min kVopt − Va Vd k (11)

Va ,Vd

s.t. Vaı , ∀ı ∈ {at (φta,kl ), 1 ≤ k ≤ Ncl , 1 ≤ l ≤ Nray }

kVa Vd k2 = K
The constraint of Vaı can be inserted directly into the opti-
mization objective to obtain Vd,opt as follows
Vd,opt = arg min kVopt − At Vd k (12)
Vd
(c) Flexible partially-connected
s.t. kdiag(Vd Vd∗ )k = NtRF
Fig. 1: Architectures of analog precoding kAt Vd k2 = K
where At = [at (φta,11 , φte,11 ), ..., at (φta,Ncl Nray , φte,Ncl Nray )]
is an t × Ncl Nray matrix consisting of array response vectors.  Pı
=1 x̃

 Pn ı
, x̄ı < 0 and x̄ı ∈ [2x̃ı , 2x̃ı+1 ]
x̃
A. Proposed Architecture x̄ı = =ı+1
, x̄ı > 0 and x̄ı ∈ [2x̃ı , 2x̃ı+1 ] (22)
 n−ı
+∞, otherwise

In the proposed hybrid precoding implementation based on
the flexible partially-connected structure, M phase shifters and
in total M 2 adaptive switches with binary states are required. Algorithm 1 Proposed Algorithm: Spatially Sparse MMSE
Accordingly, the analog precoding matrix Va can be expressed precoding under flexible partially-connected structure
as:
Require: Vopt , SN R
Ṽa = SVa (13)
1: Va = ∅
M ×M ı=0
where S ∈ {0, 1} is the switch matrix, and the boolean 2: Vres = Vopt
constraints are induced by the switches with binary states. By 3: FOR ı ≤ NtRF do
replacing the digital precoder matrix Vd with Vd = αVD 4: ψı−1 = A∗t Vres ı−1
, {At = [at (φta,kl , φte,kl )], 1 ≤ k ≤
where VD is a semi-unitary matrix, the precoder optimization Ncl , 1 ≤ l ≤ Nray }
∗
problem can be written as: 5: `= arg min (ψı−1 ψı−1 )k,l
∗ ∗
k ∈ {1, . . . , N cl }
kVopt − Ṽa Vd k2 = T r(Vopt Vopt ) − T r(Vopt Ṽa Vd ) l ∈ {1, . . . , Nray }
− T r(Vd∗ Ṽa∗ Vopt ) + T r(Vd∗ Ṽa∗ Ṽa Vd ) 6: Va = [Va |A`t ]
∗ ∗
2
= kVopt k − 2αRe(T r(VD )Vopt Ṽa )∗ 7: VD = √ 1
SN R
(VD ∗
Va∗ Vopt Vopt Va VD +
RF
2 2 Nt ∗ −1 ∗ ∗ ∗
+ α kṼa VD k (14) SN R Va Va ) VD Va Vopt
∗
IN RF −Vopt Va VD
ı
We can derive an upper bound for the last term by utilizing 8: Vres = t
∗ V V k
kIN RF −Vopt a D
t
the semi-unitary property of Ṽa VD 9: END FOR
∗ 10: REPEAT
kṼa VD k 2
= T r(VD Ṽa∗ Ṽa VD )
∗
11: Fix VD and optimize α and S according to (20) and (21),
= T r(VD Va∗ S∗ SVa VD ) respectively.
2
< kSk (15) 12: Fix α and S and update VD with (23);
13: until a stopping criterion triggers.
Thus, we will have:
14: For the digital
√ precoder at the transmit end, normalize
∗ K
kVopt k2 − 2αRe(T r(VD Vopt SVa )) + α2 kSk2 (16) Vd = α kVa V Dk
VD
15: RETURN Ṽa = SVa , Vd
So, the precoder optimization problem can be reformulated as:
∗
min {α2 kSk2 − 2αRe(T r(VD Vopt SVa )} (17) For the digital precoder at the transmit end, Vd is normal-
α,S,VD
ized to reduce the complexity of the algorithm. After updating
∗ the analog precoder Va , the digital precoder Vd = αVD will
VD V D = IK
be optimized with the semi-orthogonal constraint. The solution
When the switch matrix S and α are fixed, the optimization which is a typical semi-Orthogonal Procrustes Problem (OPP),
problem can be written as: can be expressed as
∗
max{αRe(T r(VD Vopt SVa ))} (18) VD = VU∗ (23)
VD
∗
It is shown recently in [8] that the subproblem of updating S where αVopt SVa = UΣV∗ follows the singular value
and α can be recast as: decomposition (SVD) and Σ is a diagonal matrix.
∗
min kRe(T r(Vopt VD Va ∗ )) − αSk (19)
α,S B. Computational Complexity Comparison
The optimal solution to (19) is given by: In this subsection, we compare the computational complex-
?
α = arg min f(x̃ı )f(x̄ı ) (20) ities for the three involved architectures, namely the fully-
{x̃ı x̄ı }n
ı
connected, partially-connected and flexible partially-connected
(the proposed architecture). Their computational complexities
where, f (α) is a quadratic function within each interval
of these architectures are listed, in TABLE I. It is worth not-
[2x̃ı , 2x̃ı+1 ].
ing that our proposed architecture reduces the computational
complexity compare to the fully-connected structure.
∗
Va ∗ ) > α

1Re(T r(Vopt VD 2 1M ×M , α>0
S? = (21)
∗
1Re(T r(Vopt VD Va ∗ ) < α
2 1M ×M , α<0 IV. SIMULATION RESULTS
where x = ∗
vec{Re(Vopt VD Va ∗ )} and 1 is the indicator In this section, we evaluate the performance of the proposed
function and 1m×n denotes an m × n matrix with all entries hybrid structure. In the clustered channel model, we use Ncl =
equal to one. In addition, x̃ı is the ıth smallest entry in x, 3 and Nray = 10, and the simulation results are averaged
 TABLE I: Computational complexity
Architecture Phase shifters Adders Switches
Fully-connected M NtRF M -
Partially-connected M - -
Proposed architecture 2M M M2
over 10000 channel realizations. This paper demonstrates the
average rate and bit error rate (BER) performance of the
proposed algorithm for downlink transmission scenario when
M = 64 and K = 16. The results of other algorithms are also
provided for comparison. We assume that the number of RF
chains equals the number of users K.
Fig. 2 compares the BER performance of the proposed
structure and other structures (OMP [4], AltMin [7], Fully-
digital structure). Fig. 3 illustrates the average transmission
rate versus SNR achieved by the proposed structure (Proposed
ALG), the hybrid precoder design via OMP algorithm (OMP)
[4], alternating minimization algorithm (AltMin) [7] and our
proposed architecture under two configurations in analog pre-
coding (Fully, Flexible Partially). We observe that the rate of
the proposed structure (Proposed ALG-Fully) is superior than
the other algorithms and it is near to the rate of fully-digital
structure.
Fig. 2: BER performance
V. CONCLUSIONS
In this paper, we have studied hybrid precoding schemes for
narrow-band downlink massive MIMO systems and demon-
strated the advantages of the recently proposed hybrid pre-
coder. The analog precoder of the proposed algorithm uses the
flexible partially-connected configuration. We showed that the
same framework can be applied to the problem of designing
practical MMSE precoders for mmWave systems. Simulation
Fig. 3: Average transmission rate versus SNR
results have shown that the proposed algorithm outperforms
the other algorithms and the performance gain of this struc-
ture under the flexible partially-connected configuration with
reducing the number of phase shifters for a given total number
of RF chains, is near to the performance of fully-connected
architecture.
R EFERENCES
[1] Y. Y. Lee, C. H. Wang, and Y. H. Huang, “A hy-
brid RF/baseband precoding processor based on parallel-
index-selection matrix-inversion-bypass simultaneous or-
thogonal matching pursuit for millimeter wave MIMO
systems,” IEEE Transactions on Signal Processing,
vol. 63, no. 2, pp. 305–317, 2015.
[2] M. M. Molu, P. Xiao, M. Khalily, K. Cumanan, L. Zhang,
and R. Tafazolli, “Low-complexity and robust hybrid
beamforming design for multi-antenna communication
systems,” IEEE Transactions on Wireless Communica-
tions, vol. 17, no. 3, pp. 1445–1459, 2018.
[3] X. Zhang, A. F. Molisch, and S.-Y. Kung, “Variable-
phase-shift-based RF-baseband codesign for MIMO an-
tenna selection,” IEEE Transactions on Signal Process-
ing, vol. 53, no. 11, pp. 4091–4103, 2005.
[4] O. El Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and
R. W. Heath, “Spatially sparse precoding in millimeter
wave MIMO systems,” IEEE transactions on wireless
communications, vol. 13, no. 3, pp. 1499–1513, 2014.
[5] A. Kaushik, J. Thompson, and M. Yaghoobi, “Sparse
hybrid precoding and combining in millimeter wave
MIMO systems,” IET Radio Prop. Tech. 5G, pp. 1–7,
Durham, UK, Oct. 2016.
[6] C. Rusu, R. Méndez-Rial, N. González-Prelcicy, and
R. W. Heath, “Low complexity hybrid sparse precoding
and combining in millimeter wave MIMO systems,”
in IEEE International Conference on Communications
(ICC), pp. 1340–1345, IEEE, 2015.
[7] X. Yu, J. C. Shen, J. Zhang, and K. B. Letaief, “Al-
ternating minimization algorithms for hybrid precoding
in millimeter wave MIMO systems,” IEEE Journal of
Selected Topics in Signal Processing, vol. 10, no. 3,
pp. 485–500, 2016.
[8] X. Yu, J. Zhang, and K. B. Letaief, “A hardware-
efficient analog network structure for hybrid precoding
in millimeter wave systems,” IEEE Journal of Selected
Topics in Signal Processing, vol. 12, no. 2, 2018.
[9] M. R. Akdeniz, Y. Liu, M. K. Samimi, S. Sun, S. Rangan,
T. S. Rappaport, and E. Erkip, “Millimeter wave channel
modeling and cellular capacity evaluation,” IEEE journal
on selected areas in communications, vol. 32, no. 6,
pp. 1164–1179, 2014.
[10] L. Liang, W. Xu, and X. Dong, “Low-complexity hybrid
precoding in massive multiuser MIMO systems,” IEEE
Wireless Communications Letters, vol. 3, no. 6, pp. 653–
656, 2014.
[11] F. Sohrabi and W. Yu, “Hybrid analog and digital beam-
forming for mmwave OFDM large-scale antenna arrays,”
IEEE Journal on Selected Areas in Communications,
vol. 35, no. 7, pp. 1432–1443, 2017.