IEEE SIGNAL PROCESSING LETTERS, VOL. 26, NO.
2, FEBRUARY 2019
Bilateral Angle 2DPCA for Face Recognition
Shuisheng Zhou
Abstract—Two-dimensional principal component analysis
(2DPCA), as a state-of-the-art method for dimensionality reduc-
tion, has been widely used in face recognition. However, it is very
sensitive to outliers since it minimizes the sum of squared F-norm,
which is least-squares loss in nature. Recently, angle 2DPCA was
presented to alleviate this problem by minimizing the sum of F-
norm, which is corresponding to L1 loss. But a vital unresolved
problem of angle 2DPCA is that it needs many more coefficients
for image representation because it works only in the row direction.
In this letter, we first give a new angle 2DPCA called Sin-2DPCA by
minimizing the relative error, which has a better explanation than
the original one. Furthermore, in order to obtain better perfor-
mance with fewer reduced coefficients, we project the input image
to a lower dimension from right and left simultaneously, and then,
the bilateral angle 2DPCA (BA2DPCA) is proposed. The experi-
mental results on two benchmark face recognition datasets with
outlier noises illustrate that the Sin-2DPCA has the similar perfor-
mance with original angle 2DPCA, and BA2DPCA can obtain the
highest performance in all compared algorithms with the minimal
number of representation coefficients.
Index Terms—Face recognition, dimensionality reduction,
2DPCA, F-norm, angle 2DPCA.
I. INTRODUCTION
RINCIPAL Component Analysis (PCA) [1] has been
P widely used in dimensionality reduction, pattern recog-
nition and computer vision. However, applying PCA to face
image representation and recognition, we need transform each
image, which is usually represented as a matrix, into a 1D vec-
tor column by column or row by row, so it cannot well exploit
the spatial structure information embedded in pixels and their
neighbors of image. Moreover, concatenating 2D matrices into
1D vectors often leads a high-dimensional vector space, where
it is difficult to evaluate the covariance matrix accurately due to
its large size and relatively small number of training samples. To
tackle this problem, Yang et al. [2] proposed two-dimensional
principal component analysis (2DPCA) which is based on 2D
image directly rather than 1D vector. It has been successfully ap-
plied in computer vision and signal process community [3], [4].
2DPCA performed on the 2D images is essentially equivalent
to the PCA performed on the rows of the images. Only working
in the row direction of images makes it a lower compression
rate than that of PCA. To address this problem, some bilateral
Manuscript received September 26, 2018; revised December 22, 2018; ac-
cepted December 22, 2018. Date of publication December 27, 2018; date of
current version January 11, 2019. This work was supported by the National
Natural Science Foundation of China under Grants 61772020 and 61472303.
The associate editor coordinating the review of this manuscript and approving it
for publication was Prof. Joao Paulo. (Corresponding author: Shuisheng Zhou.)
The authors are with the School of Mathematics and Statistics, Xid-
ian University, Xi’an 710126, China (e-mail:, sszhou@mail.xidian.edu.cn;
546142538@qq.com).
Digital Object Identifier 10.1109/LSP.2018.2889925
1070-9908 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
317
and Danqing Zhang
projection-based approaches have been developed. Kong et al.
[5] proposed B2DPCA that constructs two subspace to encode
the row and column vectors of image matrices respectively.
Zhang et al. [6] presented (2D)2 PCA that simultaneously con-
sidering the row and column directions of images. Xu et al. [7]
constructed two projection transformation matrices by defining
two image covariance matrices and proposed complete 2DPCA
which similar as (2D)2 PCA in [6]. Kim et al. [8] developed a
method which obtains the linear transform matrix by using two
covariance matrices in 2DPCA [2].
PCA and 2DPCA are both based on minimizing the sum
of squared F-norm, which is equivalent to the least squared
loss or squared L2-norm. It is commonly known that the least
squared loss is not robust in the sense that outlying measure-
ments can arbitrarily skew the solution from the desired solution.
Compared to the squared F-norm, L1-norm is more robust to
outliers. Kwak [9] used L1-norm to measure the variance and
developed PCA-L1. Following that, 2DPCA-L1 [10] was pro-
posed to well exploit the spatial structure embedded in image.
One drawback of L1-norm is that it has no rotational invari-
ance, which has been emphasized in the learning algorithms
[11]. Rotational invariance can help avoid performance degra-
dation. Based on this content, some F-norm based methods have
been proposed [12]–[14], because F-norm has rotational invari-
ance property. Wang et al. [12], [14] proposed F2DPCA which
uses F-norm to measure the reconstruction error. In [13], F-
norm is employed to measure the variance under the projected
subspace.
For F-norm based methods, minimizing the reconstruction
error is not already equal to maximizing the variance. Therefore,
the aforementioned methods do not consider the relationship
between reconstruction error and variance.
Recently, to overcome this drawback, Gao et al. [15] proposed
an angle 2DPCA (Next we will call it as Tan-2DPCA and see
Section II-C for details), which employs F-norm as the distance
metric and obtains the optimal projection matrix by minimizing
the ratio between reconstruction error and variance. Tan-2DPCA
is not only robust to outliers but also rotational invariance. The
experimental results in [15] show that Tan-2DPCA has high
effectiveness.
In this letter, we first generalize the Tan-2DPCA in [15] to
Sin-2DPCA by minimizing the sine of the angle between the
input and its variance, whereas the original angle 2DPCA is
minimizing the tangent of the angle between the input and its
variance. We can show that the two algorithms are compara-
ble, but the new one explains the robustness of angle 2DPCA
models more precisely because it is just minimizing the recon-
structed relative error of the image. Furthermore, enlightening
by the aforementioned bilateral projection methods, we propose
a bilateral angle 2DPCA (BA2DPCA).
The notations and assumptions in this letter are described
as follows, assuming that we have a set of N training sam-
ples {Ai ∈ Rm ×n , i = 1, 2, . . . , N }, which is centered, i.e.
318 IEEE SIGNAL PROCESSING LETTERS, VOL. 26, NO. 2, FEBRUARY 2019

outliers, thus the solution of Eq. (1) or Eq. (4) is not robust in
the sense that outlying measurements skew the solution from
the desired solution [15].
To improve the robustness of 2DPCA and keep rotational in-
variance, an angle 2DPCA [15] was proposed by minimizing the
ratio between reconstruction error and variance under F-norm
metric. It aims to extract the low-dimensional representations
Fig. 1. Reconstruction error and variance of the ith image (i = 1, . . . , N ).
Yi = Ai R by solving the following optimization problem.
N N
Ai = 0. tr(·) is the trace operator of a matrix, Ir is an
i=1
 Ei F
r × r identity matrix, and  · F denotes the Frobenius norm. min (5)
R  R=I r
i=1
Ai RF

II. THE RELATED WORKS with Ei = Ai − Ai RR . Here we call it as Tan-2DPCA since
E i F
Now we review the related methods, including 2DPCA [2], the item A i RF
in Eq. (5) is just the tangent of angle αi between
2DPCA-L1 [10], (2D)2 PCA [6], and Tan-2DPCA [15]. the input and its variance (see Fig. 1).
There is no close form solution for problem (5). An iterative
A. 2DPCA and 2DPCA-L1 algorithm is adopted to solve it in [15]. Experimental results
show that the algorithm works well, whereas its convergence is
2DPCA [2] aims to seek column orthogonal projection matrix still an open problem.
R ∈ Rn ×r by solving the following programming. Compared with 2DPCA [2], Tan-2DPCA has two advantages.
N
 N
 One is that, in F-norm based angle 2DPCA, distance in attribute
 
max tr (Ai R) Ai R = max Ai R2F . (1) dimensions is measured in F-norm, while the summation over
R  R=I r R  R=I r different data points uses 1-norm, thus it improves the robust-
i=1 i=1
ness. The other is that the objective function of Tan-2DPCA

If let Ei = Ai − Ai RR be the reconstruction error of the ith takes the reconstruction error and variance into account simul-
training image, we have Ei 2F + Ai R2F = Ai 2F , which taneously, thus more effective information of face images is
can be illustrated in Fig. 1. Hence, the programming in Eq. (1) captured to enhance the recognition rate.
is equivalent to the following problem in Eq. (2).
N
 III. SIN-2DPCA
min Ei 2F . (2) In the fact that sin x ≈ tan x if the angle x is small, by Fig. 1
R  R=I r
i=1 we can also minimize
The optimal solution of Eq. (1) or Eq. (2) is composed of the N
 Ei F
orthogonal unit eigenvectors of covariance matrix N 
i=1 Ai Ai min (6)
corresponding to the first r largest eigenvalues. R  R=I r
i=1
A i F
Furthermore in [10], Li et al. proposed 2DPCA-L1 to improve
instead of (5), which is corresponding to minimizing the sine of
the robustness of model (1) by solving
angle αi (i = 1, . . . , N ). We called the model Sin-2DPCA.
N E i F
 Since A is just the relative reconstruction error of the
i F
max Ai RL 1 , (3) input image Ai , the new model precisely explains why the angle
R  R=I r
i=1
2DPCA is robust to outliers. Such as for a given outlier Ai with
and an algorithm generalized from PCA-L1 [9] is given. large Ai F , to make its absolute error Ei F small by 2DPCA
in Eq. (2), the solution may highly be skewed. Similar things
B. (2D)2 PCA will happen when 2DPCA-L1 or (2D)2 PCA is used. However
for angle 2DPCAs like the new model Eq. (6), we overcome
(2D)2 PCA [6] was introduced to alleviate the problem that E i F
this shortcoming by minimizing its relative error A i F
. Then
many more coefficients are needed for image representation in
2DPCA. Its idea is to project image Ai by the transformation the outliers will not skew the solution greatly.
Bi = L Ai R, where L ∈ Rm ×l and R ∈ Rn ×r denote the left By simple algebra as in [15], we have
and right projection transformation matrix, respectively, and N N
Bi ∈ Rl×r is the resulted feature matrix of Ai . In [6], R is ob- Ei F  Ei 2F  
= = tr G(R) − G(R)RR
tained by the above classical 2DPCA method, and L is obtained i=1
Ai F i=1 Ai F Ei F
by another 2DPCA process since: (7)
N
   N
 N 
with G(R) = i=1 A A i Ai
. There is still no close form
max tr (L Ai ) (L Ai ) = max A 2
i LF . i F E i F
L  L=I l R  R=I l solution. By [15], an iterative algorithm is adopted as
i=1 i=1
(4)  
Rt+1 = arg min tr G(Rt ) − G(Rt )Rt R , (8)
R  R=I r
C. Tan-2DPCA
for given Rt . Based on [15, Th. 2], (8) is solved by Rt+1 =
It can be seen that both 2DPCA and (2D)2 PCA employ UW with UΛW be compact SVD of G(Rt )Rt .
squared F-norm as the distance metric, which is sensitive to The pseudo code of Sin-2DPCA is listed in Algorithm 1.
ZHOU AND ZHANG: BILATERAL ANGLE 2DPCA FOR FACE RECOGNITION 319

Algorithm 1: Sin-2DPCA.

Input: Ai (i = 1, 2, . . . , N ) satisfying Ni=1 Ai = 0.
1: Set R0 ∈ Rn ×r satisfying R0  R0 = Ir , t = 0;
2: while not converge do
 A i A i
3: G(Rt ) = N i=1 A i F E i F with
Ei = Ai − Ai Rt R t ;
4: Rt + 1 = UW with the SVD Fig. 2. Some noised images (in second row) in the Extended YaleB dataset,
G(Rt )Rt = UΛW ; and the first row is the corresponding ground truth.
5: t ← t + 1.
6: end while
Output: Transformation matrix R = Rt .

IV. BILATERAL ANGLE 2DPCA

As discussed in Section III, compared with traditional
2DPCA, angle 2DPCA is more robust to outliers. Neverthe-
less, angle 2DPCA works just in the row direction of image.
That is, angle 2DPCA learns a projection matrix by a set of
training images only reflecting the information between rows of
images and disregards the information embedded in columns of
images. In addition, it needs more coefficients for image repre- Fig. 3. Comparisons of accuracies on Extended Yale B dataset with noise.
sentation, thus larger storage is needed for large-scale datasets.
Inspired by the work in [6], we use two projection matrices R
Algorithm 2: Bilateral Angle 2DPCA ( BA2DPCA).
on the right and L on the left to overcome the drawbacks of the
angle 2DPCAs. Input: Ai ∈ Rm ×n (i = 1, . . . , N ) satisfying
N
Specifically, the angle 2DPCA is employed firstly on the i=1 Ai = 0.
training samples Ai ∈ Rm ×n (i = 1, 2, . . . , N ) to obtain one 1: Put Ai (i = 1, . . . , N ) as the inputs of Algorithm 1 to
projection matrix R ∈ Rn ×r on the right. Projecting the image obtain the right transformation matrix R ∈ Rn ×r and
Ai onto R yields Yi = Ai R ∈ Rm ×r which is called the right calculate the feature matrix Yi = Ai R for Ai .
feature matrix of Ai . Furthermore, we use angle 2DPCA again 2: Put Yi (i = 1, . . . , N ) as the inputs of Algorithm 1 to
with the input training samples Yi ∈ Rr ×m (i = 1, 2, . . . , N ) obtain the left transformation matrix L ∈ Rm ×l , and
to get another projection matrix L ∈ Rm ×l similarly. Then the calculate the feature matrix B  
i = Yi L for Yi .
matrix Yi is mapped to an r × l feature matrix B 
i = Yi L. Output: Transformation matrices L and R, the feature
Through the two processes, the input larger image Ai was trans- matrices Bi = L Ai R ∈ Rl×r (i = 1, 2, . . . , N ).
formed into a smaller feature matrix Bi . Namely,

Bi = L Ai R, i = 1, 2, . . . , N. (9)
We call R the right projection matrix and L the left projection
matrix, and the corresponding algorithm is called the Bilateral
Angle 2DPCA (BA2DPCA).
Since l × r is much smaller than m × n, so BA2DPCA can
use fewer coefficients to represent the input image. The exper-
imental results illustrate that, compared the related algorithms
2DPCA, (2D)2 PCA and angle 2DPCAs, BA2DPCA can ob-
tain higher performance with a fewer number of representation
coefficients.
The BA2DPCA is described in Algorithm 2, where Sin-
2DPCA was used because of its reasonable explanation.
V. EXPERIMENTS AND ANALYSIS
In this section, we do some experiments to compare Sin-
2DPCA and BA2DPCA with four state-of-art algorithms, i.e.
2DPCA [2], 2DPCAL1 [10], Tan-2DPCA [15] and (2D)2 PCA
[6] in terms of recognition accuracy and image reconstruction. In
our experiments, two well-known face data sets, Extended Yale
B and CMU PIE, are used. All algorithms are implemented with
MATLAB R2016b and run on a computer with CPU i5-6500,
3.20 GHz, and 8GB RAM.
Since the resulted feature matrices of different algorithms
have different dimensions, to make a fair comparison, we use
the same size of the features for all methods. Such as, if the
size of the resulted feature matrix is mr for one side methods
(2DPCA, 2DPCAL1, Tan-2DPCA and Sin-2DPCA), then we set
l and r for the two sides methods ((2D)2 PCA and BA2DPCA)
satisfying l ∗ r ≈ m ∗ r and let l = r for simplicity. For all
algorithms, 1-nearest neighbor (1-NN) is used for classification.
A. Experiments on Extended Yale B Dataset
The first set of experiments is on the Extended Yale B data
set [16], which consists of 2414 frontal-face pictures of 38 in-
dividuals with different illuminations.
In the experiments, each image is normalized to 48 × 42 pix-
els. 20% of the total images are randomly selected and occluded
with a rectangular noise whose size account for 5%-15% of the
image as Fig. 2 shows. The noise is the gray level noise image
whose pixels are random generated between 0 and 255. 50% of
total images are randomly chosen for training, and the rest for
testing.
Fig. 3 plots the curve of accuracies (averaged on ten times)
versus a series of the different representation coefficients.
320
TABLE I
COMPARISONS ON EXTENDED YALE B DATABASE
Fig. 4. Some noised images(in second row) in the CMU PIE dataset, and the
first row is corresponding ground truth.
The first finding in Fig. 3 is that the new proposed Sin-2DPCA
nearly has the same accuracies as the Tan-2DPCA in [15] since
sin α ≈ tan α with small |α|. However, Sin-2DPCA saves a
little training time because Eq. (6) is simpler than Eq. (5). The
second finding in Fig. 3 is that, for all one-side algorithms,
the angle-type algorithms are better than others in our outliers
setting and the proposed BA2DPCA outperforms all compared
algorithms.
In Table I, we list the average classification accuracies (aver-
aged on ten times) with standard deviations (Std) obtained by
the related algorithms, including on the original(clean) dataset
(in the third column) and on the noised dataset (in the fourth
column). The settings of r or/and l are in the second column.
Here they are given because the larger value of r or/and l will
not improve the performance of the algorithms much. In the last
column, the differences of the accuracies on the clean dataset
and on the noised dataset are given.
In Table I, on the one hand, the comparison of one side al-
gorithms shows that, on the clean dataset, the differences of
them can be negligible, while on the noised dataset, angle based
methods are evidently better than 2DPCA and 2DPCAL1. This
is probably because the angle 2DPCA model minimizes the rela-
tive errors, which is robust to outliers. Furthermore, the values in
the last column show that the performance of all algorithms has
been decreased by noise. However, the angle-type algorithms
have a small decrement. This again supports the robustness of
the angle models. On the other hand, it is observed that the pro-
posed BA2DPCA is the best one in all compared algorithms. It
has not only the minimal number of representation coefficients,
but also the highest accuracy and the least decrement.
B. Experiments on CMU PIE Dataset
To further examine the performance of the proposed methods,
we conduct experiments on a subset (Pose C9) of the CMU PIE
dataset [17], which has 1632 frontal-face including 68 individ-
uals with 24 different illuminations. Each image is rescaled to
64 × 64 pixels. The noise is added as the preceding subsection
and Fig. 4 shows some samples. All images are split into 50%
training and 50% testing.
IEEE SIGNAL PROCESSING LETTERS, VOL. 26, NO. 2, FEBRUARY 2019
TABLE II
COMPARISONS ON CMU PIE DATASET
Fig. 5. Two reconstructed images experiments on the Pose C9 dataset. The
upper row is the clean image and its reconstructed images, and the lower row is
the noised image and its reconstructed images. The algorithms used are 2DPCA,
2DPCAL1, (2D)2 PCA, Tan-2DPCA, Sin-2DPCA and BA2DPCA successively
from left to right. The PSNR/SSIM scores of the reconstructed image are pasted
nearby.
The experimental results are given in Table II, where all the
settings are similar as those in Table I.
The results in Table II again support the findings of the pre-
ceding experiments. All angle-type algorithms outperform oth-
ers in our setting, and the proposed BA2DPCA again has the
highest accuracy among all methods with a minimal number of
representation coefficients and the least decrement.
Finally, we do experiments to compare the reconstruction
performance of the related algorithms in representing images.
Fig. 5 gives the reconstructed images on one noised sample from
the stained Pose C9 dataset and its ground truth. We use the same
compressing rate to make a fair comparison and set l = r = 18
in the bilateral projection-based approaches and r = 5 for one
side approaches.
From the reconstructed images and their PSNR/SSIM scores
in Fig. 5, we can see that the proposed BA2DPCA yields the
highest quality and the best PSNR/SSIM scores in all algo-
rithms, since it maintains more information with the similar
amount representation coefficients. It also shows that, for clean
image, the differences of the reconstructed images by the related
algorithms are small, while for the noised image (outlier) their
differences are large.
VI. CONCLUSION
In this letter, we first propose a new angle 2DPCA model
minimizing the sine of the angle between the inputs and their
variances, which is exactly the relative reconstruction error. This
model gives a new explanation of the robustness of the angle
2DPCA in [15]. Furthermore, it is extended to a bilateral pro-
jection model and BA2DPCA is proposed. The new BA2DPCA
reduces the dimensionality of original image matrix from row
and column simultaneously. The Experimental results on two
benchmark datasets indicate that the proposed methods can ob-
tain the higher performance with a fewer number of representa-
tion coefficients.
ZHOU AND ZHANG: BILATERAL ANGLE 2DPCA FOR FACE RECOGNITION
REFERENCES
[1] I. T. Jolliffe, Principal Component Analysis. New York, NY, USA:
Springer-Verlag, 2002.
[2] J. Yang, D. Zhang, A. F. Frangi, and J.-Y. Yang, “Two-dimensional PCA: A
new approach to appearance-based face representation and recognition,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 1, pp. 131–137,
Jan. 2004.
[3] D. Wang and H. Lu, “Object Tracking via 2DPCA and L1-regularization,”
IEEE Signal Process. Lett., vol. 19, no. 11, pp. 711–714, Nov. 2012.
[4] Y. Jeong and H. S. Kim, “New speaker adaptation method using 2-
DPCA,” IEEE Signal Process. Lett., vol. 17, no. 2, pp. 193–196, Feb.
2010.
[5] H. Kong, X. Li, L. Wang, E. K. Teoh, J. G. Wang, and R. Venkateswarlu,
“Generalized 2D principal component analysis,” in Proc. Int. Joint Conf.
Neural Netw., Montreal, Canada, 2005, vol. 1, pp. 108–113.
[6] D. Zhang and Z. Zhou, “(2D)2 PCA: Two-directional two-dimensional
PCA for efficient face representation and recognition,” Neurocomputing,
vol. 69, no. 1–3, pp. 224–231, Jun. 2005.
[7] A. Xu, X. Jin, Y. Jiang, and P. Guo,“Complete two dimensional PCA for
face recognition,” in Proc. 18th Int. Conf. Pattern. Recognit., Washington,
DC, USA, 2006, vol. 3, pp. 481–484.
[8] Y. G. Kim, Y. J. Song, U. D. Chang, D. W. Kim, T. S. Yun, and J. H. Ahn,
“Face recognition using a fusion method based on bidirectional 2DPCA,”
Appl. Math. Comput., vol. 205, no. 2, pp. 601–607, Nov. 2008.
321
[9] N. Kwak, “Principal component analysis based on L1-norm maximiza-
tion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1672–
1680, Sep. 2008.
[10] X. Li, Y. Pang, and Y. Yuan, “L1-norm-based 2DPCA,” IEEE Trans. Syst.,
Man, Cybern. B, Cybern., vol. 40, no. 4, pp. 1170–1175, Aug. 2010.
[11] A. Y. Ng, “Feature selection, L1 vs. L2 regularization, and rotational
invariance,” in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, 2004,
vol. 19, pp. 379–387.
[12] Q. Wang and Q. Gao, “Two-dimensional PCA with F-norm minimization,”
in Proc. 31st Conf. Artif. Intell., Feb. 2017, pp. 180–186.
[13] T. Li, M. Li, Q. Gao, and D. Xie, “F-norm distance metric based robust
2DPCA and face recognition,” Neural Netw., vol. 94, pp. 204–211, 2017.
[14] Q. Wang, Q. Gao, X. Gao, and F. Nie, “Optimal mean two-dimensional
principal component analysis with F-norm minimization,” Pattern Recog-
nit. vol. 68, pp. 286–294, Aug. 2017.
[15] Q. Gao, L. Ma, Y. Liu, X. Gao, and F. Nie, “Angle 2DPCA: A new
formulation for 2DPCA,” IEEE Trans. Cybern., vol. 48, no. 5, pp. 1672–
1678, Jun. 2017.
[16] A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to
many: Illumination cone models for face recognition under variable light-
ing and pose,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6,
pp. 643–660, Jun. 2001.
[17] T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, and expres-
sion database,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12,
pp. 1615–1618, Dec. 2003.