FA.

docx
Factor Analysis

The methods we have employed so far attempt to repackage all of the variance in the p
variables into principal components. We may wish to restrict our analysis to variance that is common
among variables. That is, when repackaging the variables variance we may wish not to redistribute
variance that is unique to any one variable. This is Common Factor Analysis. A common factor is
an abstraction, a hypothetical dimension that affects at least two of the variables. We assumed that
there is also one unique factor for each variable, a factor that affects that variable but does not affect
any other variables. We assume that the p unique factors are uncorrelated with one another and with
the common factors. It is the variance due to these unique factors that we shall exclude from our FA.
The most common sort of FA is principal axis FA, also known as principal factor analysis.
This analysis proceeds very much like that for a PCA. We eliminate the variance due to unique
factors by replacing the 1s on the main diagonal of the correlation matrix with estimates of the
variables communalities. Recall that a variables communality, its SSL across components or factors,
is the amount of the variables variance that is accounted for by the components or factors. Since our
factors will be common factors, a variables communality will be the amount of its variance that is
common rather than unique. The R
2
between a variable and all other variables is most often used
initially to estimate a variables communality.
For the beer data, I used the following SAS statement:
PROC FACTOR PRIORS=SMC MIN=1 REORDER ROTATE=VARIMAX;
Specifying PRIORS=SMC replaced the 1s with R
2
s in the correlation matrix. Here are selected
parts of the output:
COST SIZE ALCOHOL REPUTAT COLOR AROMA TASTE
SMC .74 .91 .87 .50 .92 .86 .88
POST-ROTATION .74 .91 .86 .41 .91 .87 .89
COMMUNALITY
Note that the R
2
s are generally quite large, indicating considerable shared variance. That for
REPUTAT is less impressive, indicating that REPUTAT had considerable unique variance.
Here are the eigenvalues from the reduced correlation matrix (with R
2
s in the main diagonal).
Factor 1 2 3 4 5 6 7
Eigenvalue 3.13 2.47 .19 .03 .00 -.06 -.09
Cumulative .55 .99 1.02 1.03 1.03 1.02 1.00
The sum of these eigenvalues is 5.68. That means that 5.68/7 = 81% of the variance was
distributed among seven factors (recall that our PCA distributed all of the variance, that is, the sum of
its seven eigenvalues was seven).
Do not be concerned about the small negative eigenvalues. While some authors warn that
negative or near zero eigenvalues indicate multicollinearity (very high R
2
s), which may make the
results unstable [fluctuating wildly with only minor changes in the correlations], necessitating the
removal of redundant variables, I dont think this is often a problem in FA, where we are seeking to
collect somewhat redundant variables into factors. Of course, if you do something stupid like have
more variables than subjects or have a perfect linear combination among the variables, for example,
SAT total score, SAT verbal score, and SAT math score, you will have a singular matrix that cant be
inverted and the FA will crash. The SAS users guide says, with data that do not fit the common

Copyright 2013, Karl L. Wuensch - All rights reserved.

2
factor model perfectly, you can expect some of the eigenvalues to be negative. If a principal factor
analysis fails to yield any negative eigenvalues, the prior communality estimates are probably too
large.
The retained factors should account for nearly all of the common variance. Notice that the first
two factors do account for 99% of the common variance. The cumulative figures here are cumulative
proportion of common variance, not total variance. For example, for factor 1, the proportion is
3.13/5.68 = .55, not 3.13/7 = .45. A two factor solution seems clearly indicated.
Here are the loadings for a two factor solution after the Varimax rotation:
COLOR TASTE AROMA SIZE ALCOHOL COST REPUTAT
FACTOR1 .95 .94 .93 .07 .03 -.05 -.44
FACTOR2 .07 -.02 .02 .95 .93 .86 -.46
This solution is not much different from that we obtained when we did a PCA (PRIORS = ONE)
and retained two components. The total SSL for the two factors is 5.60, with Factor 1 accounting for
2.88/5.60 = 51% of that variance, factor 2 accounting for the remaining 49%.
I did try a three factor rotated solution just to be sure that a two factor solution was a good
choice. The third factor loaded well only on REPUTAT (.46) and had a SSL of only 0.32 after
rotation. REPUTAT loaded as well (but negatively) on the first two factors as on the third, so Im
happy with the two-factor solution.

Iterated Principal Factors Analysis
This is a refinement of the FA we just covered. For an iterated principal axis solution we first
estimate communalities, usually with R
2
s, and obtain p factors. The communalities from that
solution (variables SSLs) are then inserted into the main diagonal of the correlation matrix in place of
the R
2
s, and factors extracted again. The variables SSLs from this second solution are then
inserted into the main diagonal replacing the communalities from the previous iteration, etc. etc., until
the change from one iteration to the next iteration is trivial.
On SAS one can specify the maximum number of iterations by the option MAXITER = n. The
default is 30. Iteration will stop sooner if the convergence criterion is met. One may set the criterion
with the option CONV = n, or take the .001 default. Specify METHOD = PRINIT to get the iterated
analysis.
Here are selected parts of the output from:
PROC FACTOR METHOD=PRINIT PRIORS=SMC MIN=1 REORDER ROTATE=VARIMAX;
COMMUNALITIES
---------------------------------------------------------------------------------------------------------------
ITERATION CHANGE COST SIZE ALCOHOL REPUT COLOR AROMA TASTE
PRIORS .738 .912 .866 .499 .922 .857 .881
1 .086 .741 .910 .865 .413 .910 .874 .891
2 .021 .743 .912 .866 .392 .904 .884 .897
| | | | | | | | |
| | | | | | | | |
| | | | | | | | |
6 .001 .745 .914 .866 .385 .893 .895 .902
7 .0008 .745 .914 .866 .385 .892 .896 .902
Change is the largest change in a variables communality. For example, on the first iteration,
the largest change was in REPUTAT, from .499 to .413 = .086.
3
Note that by the seventh iteration the communalities are changing so little that the .001
convergence criterion has been satisfied and the iterative procedure stops. The communalities on
the iteration 1 row are those that were obtained in our previous noniterative principal factors analysis.
Note that the iterative process has reduced the communalities of REPUTAT and COLOR and
increased the remaining communalities.
The final rotated loadings are:
COLOR TASTE AROMA SIZE ALCOHOL COST REPUTAT
FACTOR1 .94 .95 .95 .07 .03 -.05 -.43
FACTOR2 .06 -.02 .02 .95 .93 .86 -.45
The loadings are nearly identical to those obtained in the previous noniterated solution, so, for
these data, little was gained by the iterations.

The Residual Correlation Matrix
Having extracted common factors, one can turn right around and try to reproduce the
correlation matrix from the factor loading matrix. We assume that the correlations between variables
result from their sharing common underlying factors. Thus, it makes sense to try to estimate the
correlations between variables from the correlations between factors and variables. The reproduced
correlation matrix is obtained by multiplying the loading matrix by the transposed loading matrix. This
results in calculating each reproduced correlation as the sum across factors (from 1 to m) of the
products (r between factor and the one variable)(r between factor and the other variable). For
example, for our 2 factor iterative solution the reproduced correlation between COLOR and TASTE =
(r for color - Factor1)(r for taste - Factor 1) + (r for color - Factor 2)(r for taste - Factor2) = (.95)(.94) +
(.06)(-.02) = .89. The original r between color and taste was .90, so our two factors did indeed capture
the relationship between Color and Taste.
The residual correlation matrix equals the original correlation matrix minus the reproduced
correlation matrix. We want these residuals to be small. The option RES in PROC FACTOR will
produce a residual correlation matrix. For our beer data, it looks like this:
COST SIZE ALCOHOL REPUTAT COLOR AROMA TASTE
COST .25 .01 -.03 -.04 .00 .02 .00
SIZE .01 .09 .01 .07 .05 .01 -.02
ALCOHOL -.03 .01 .13 -.03 -.02 .00 .00
REPUTAT -.04 .07 -.03 .61 .06 .03 -.04
COLOR .00 .05 -.02 .06 .11 .02 .01
AROMA -.02 .01 .00 -.03 .02 .10 -.03
TASTE .00 -.02 .00 -.04 .01 .03 .10
Each variables uniqueness (1 minus its communality) can be found on the main diagonal of
this matrix. Notice that REPUTAT is relatively unique. The residuals are generally small. SAS also
gives the Root Mean Square for each variables residuals and overall. This is the square root of (a
sum of squared residuals divided by the number of residuals). For example, for REPUTAT,
. . . . . . 04 07 03 06 03 04
6
2 2 2 2 2 2

. The overall value was .03. The values for COST....TASTE were
.02, .03, .02, .05, .03, .02, .02.
SAS also gives a matrix of Partial Correlations Controlling Factors. That is, what is the
correlation between one variable and another variable after the effects of all factors have been
4
partialled from both variables. In other words, what variance are two variables sharing with one
another that is not captured by the factors. For our data, here is that matrix:
COST SIZE ALCOHOL REPUTAT COLOR AROMA TASTE
COST 1.00 .09 -.18 -.10 .02 -.13 -.01
SIZE .09 1.00 .14 .28 .47 .09 -.25
ALCOHOL -.18 .14 1.00 -.12 -.16 .03 .03
REPUTAT -.10 .28 -.12 1.00 .25 .10 -.18
COLOR .02 .47 -.16 .25 1.00 .15 .10
AROMA -.13 .09 -.03 -.10 .15 1.00 -.28
TASTE -.01 -.25 .03 -.18 .10 .28 1.00
SAS also gives the Root Mean Square of the off-diagonal partials for each variable and overall.
The overall value was .18, the values for COST....TASTE were .11, .26, .12, .19, .24, .15, & .18. The
partials are generally small, with the exception of that between COLOR and SIZE. If you go back to
the original simple correlation matrix you will see that there is a .18 correlation between COLOR and
SIZE. COLOR is well correlated with AROMA and TASTE (which load on Factor 1) but not with
COST and ALCOHOL (which load on Factor 2), SIZE is well correlated with COST and ALCOHOL
but not with AROMA and TASTE. Thus, that .18 correlation between COLOR and SIZE was not
captured by either factor.

Nonorthogonal (Oblique) Rotation
Recall the plot presented earlier (Figure 3) that indicated that the data would be better fit with
axes that were not perpendicular. This can be done by means of an oblique rotation, but the factors
will now be correlated with one another. Also, the factor loadings (in the pattern matrix) will no
longer be equal to the correlation between each factor and each variable. They will still be
standardized regression coefficients (Beta weights), the As in the
j m mj j j j
U F A F A F A X
2 2 1 1

formula presented at the beginning of the handout on principal components analysis. The
correlations between factors and variables are presented in a factor structure matrix.
PROC FACTOR OPTION ROTATE = PROMAX will produce a VARIMAX rotation followed by
an OBLIQUE rotation. I ran a PROMAX with PRIORS = SMC and no iterations on the beer data,
retaining two factors (see the file PROMAX.SAS, which uses a data set other than our beer data).
The loadings obtained were:
COLOR TASTE AROMA SIZE ALCOHOL COST REPUTAT
FACTOR1 .95 .95 .94 .03 -.01 -.08 -.43
FACTOR2 .03 -.06 -.02 .95 .93 .86 -.45
The correlations between factors and variables were:
COLOR TASTE AROMA SIZE ALCOHOL COST REPUTAT
FACTOR1 .95 .94 .93 .11 .07 -.01 -.46
FACTOR2 .10 .02 .06 .95 .93 .86 -.48
The semipartial correlations between each factor (partialled for all other factors) and each
variable were:
COLOR TASTE AROMA SIZE ALCOHOL COST REPUTAT
FACTOR1 .95 .94 .93 .03 -.01 -.08 -.43
FACTOR2 .03 -.06 -.02 .95 .93 .86 -.45
Notice that this solution is not much different from the previously obtained VARIMAX solution,
so little was gained by allowing the factors to be correlated.
5
The SAS output also tells us that the correlation between Factor 1 and Factor 2 is .08, very
small. We are also given the Variance Explained By Each Factor Eliminating Other Factors, the sum
of the squared semipartial correlations for each factor, which were 2.85 and 2.70 for our data, and the
Variance Explained By Each Factor Ignoring Other Factors, the sum of the squared correlations
(with variables) for each factor, which were 2.90 and 2.75 for our data.

Exact Factor Scores
One may wish to define subscales on a test, with each subscale representing one factor.
Using an "exact" weighting scheme, each subject's estimated factor score on each factor is a
weighted sum of the products of scoring coefficients and the subject's standardized scores on the
original variables.
The regression coefficients (standardized scoring coefficients) for converting scores on
variables to factor scores are obtained by multiplying the inverse of the original simple correlation
matrix by the factor loading matrix. For our beer data, the standardized scoring coefficients are:
TASTE AROMA COLOR SIZE ALCOHOL COST REPUTAT
FACTOR1 .41 .40 .22 -.07 .04 .03 .01
FACTOR2 .11 .03 -.20 .61 .25 .16 -.04
To obtain a subjects factor scores you multiply es standardized scores (Zs) on the variables
by these standardized scoring coefficients. For our variables, the necessary descriptive statistics
(along with subject #1s Z-scores) are:
TASTE AROMA COLOR SIZE ALCOHOL COST REPUTAT
MEAN 67.25 44.75 51.00 43.50 46.50 47.25 48.25
STDDEV 24.68 26.43 27.51 34.53 33.01 35.07 24.78
Z
SUBJ1
-.294 .955 -.036 1.057 .712 1.219 -1.14
Subject # 1s Factor scores are:
Factor 1: (-.294)(.41) + (.955)(.40) + (-.036)(.22) + (1.057)(-.07) + (.712)(.04) +
(1.219)(.03) + (-1.14)(.01) = 0.23.
Factor 2: (-.294)(.11) + (.955)(.03) + (-.036)(-.20) + (1.057)(.61) + (.712)(.25) +
(.16)(1.219) + (-1.14)(-.04) = 1.06
SAS will not only compute the scoring coefficients for you, it will also output the factor scores
of your subjects into a SAS data set that you can input into other SAS procedures. You must specify
the number of factors to retain. I used the following on our beer data:
proc factor simple corr msa method=prinit reorder rotate=varimax priors=smc
nfactors=2 res preplot plot out=drinkme;
var cost size alcohol reputat color aroma taste; run;
DATA put; SET drinkme; if factor1 ne . ;
FILE 'C:\StatData\factbeer.dat'; PUT factor1 factor2 ses group; run;
Warning: if your first SAS PROC statement takes up two lines and your data are in the
program file, you must insert a line containing only a semicolon between the last line of data and the
first PROC statement (otherwise SAS will read the first PROC line as a data line).
The input data included two variables (SES and Group) not included in the factor analysis.
The out=drinkme option creates a new data set, drinkme, which contains the factor scores
(factor1 and factor2) as well as the original variables in the input data set. The DATA put step
6
takes that drinkme data set and eliminates any observation where there is not a score for factor1
(these will be for subjects who were missing data on one or more of the variables used in the factor
analysis). The FILE and PUT statements create a data file, factbeer.dat, on the C drive in the
StatData directory. Each line contains data for one subject: Score on factor1, score on factor2, SES,
and Group. We can then use this data file for additional analysis, with SAS or with another statistical
program.
Obtain and run the file FactOut.sas from my SAS Programs page. It will produce not only a
listing, but it will also output to C:\StatData\ the factbeer.dat file. You will need to edit the FILE
statement so that it points to an appropriate folder on your computer.
Look at the Factbeer.dat file that is created. On each row you see the subjects score on
Factor 1, then score on Factor 2, then score on SES and then score on Group. If you look back at
Factbeer.sas you will see that the last subject had missing data on the aroma variable. That subject
is excluded from the new data set.
Look at the listing. We did an iterated principal factors analysis, using SMCs for the initial
communality estimates. We asked for simple statistics, correlations, and Kaisers MSA. We asked
that the variables be reordered by their loadings and that a varimax rotation be done. We asked for
residual correlation matrices and for plots of the factors both before and after rotation. We specified
that only two factors should be retained (you must specify the number of factors to retain when
outputting factor scores this way). In this listing you will find many of the statistics discussed in this
document. Do note that some of the statistics presented in this document were from runs with
parameters set at values other than those specified in the Factbeer.sas program (for example, doing
PCA instead of FA).
After you have created the factbeer.dat file, obtain and run from my SAS Programs page the
file Factor-MR-T.sas. You will need to edit the INFILE statement so that it correctly points to the
location of factbeer.dat on your computer. This SAS program does a multiple regression analysis
predicting subjects SES from their factor scores and does t-tests comparing the two groups of
subjects on the factor scores. Do note that the scores for factor 1 are not correlated with those for
factor 2. Accordingly, in the multiple regression the squared semipartial correlation coefficients are
identical to squared zero-order correlation coefficients and the
2
2
2
1
2
Y Y
r r R .
You should also read the document SPSS Discriminant Analysis on Factor Scores Produced
By SAS (Factor-Discrim.doc) and follow the instructions there for using SPSS to do a discriminant
function analysis predicting subjects group membership from their SES and their factor scores.
Estimated factor scores are our estimates, on the basis of subjects scores on the observed
variables, of the amount of some underlying factor that they have. True factor scores are the
amounts of that underlying factor that they really have. The correlation between the estimated factor
scores and the true factor scores is known to be equal to the variance of the estimated factor scores,
and this is what SAS gives you as Squared Multiple Correlations of the Variables With Each
Factor (look at the variances reported for the factor scores in the output of the Factor-MR-T.sas
program). Of course, the estimated factor scores are a linear function (a weighted combination) of the
observed scores on the variables, that is, the R
2
between estimated factor scores and scores on the
observed variables is 1.00. Since the estimated factor scores are a linear function of the observed
scores on the variables, we can also interpret the variance of the estimated factor scores as the
correlation between the observed scores on the variables and the true factor scores. Some then treat
this as a measure of the stability or internal consistency of the solution - a high R
2
(.70 and up)
indicates that the derived factor is indeed substantially related to the observed variables, a low R
2

that the derived factor is poorly defined by the observed variables.
7

Unit-Weighted Factor Scores
If one were using FA to define subscales on a test, e might decide to define subscale 1 as
being an unweighted sum of all of a subjects scores (for example, 1....7 on a Likert item) on any item
loading heavily (>.4) on Factor 1, etc. etc. Such a method is sometimes referred to as a "unit-
weighting scheme." For our data, if I answered the Color, Taste, Aroma, Size, Alcohol, Cost,
Reputation questions with the responses 80, 100, 40, 30, 75, 60, 10, my Subscale 1 (Aesthetic
Quality) subscale score would be 80 + 100 + 40 - 10 = 210 [note that I subtracted the Reputation
score since its loading was negative] and my Subscale 2 (Cheap Drunk) score would be 30 + 75 + 60
- 10 = 155.
Certain types of unit-weighting schemes may actually be preferable to exact factor scores in
certain circumstances (Grice, 2001; Grice & Harris, 1998). I recommend Grice, J. W. (2001). A
comparison of factor scores under conditions of factor obliquity, Psychological Methods, 6, 67-83.
Grice reviewed the literature on this topic and presented results of his own Monte Carlo study (with
oblique factor analysis). The early studies which suggested that unit-weighted factor scores are
preferable to exact factor scores used data for which the structure was simple (each variable loading
well on only one factor) and the loadings not highly variable (most loadings either very high or very
low). Grice & Harris, 1998, have shown that unit-weighted factor scores based on the loadings
perform poorly under conditions of non-simple structure and variable loadings, which are typical of the
conditions most often found in actual practice. They developed an alternative unit-weighting scheme
which produced factor scores that compared favorably with exact factor scores -- they based the
weightings on the factor score coefficients rather than on the loadings.
Grice's (2001) article extended the discussion to the case of oblique factor analysis, where one
could entertain several different sorts of unit-weighting schemes -- for example, basing them on the
pattern matrix (loadings, standardized regression coefficients for predicting), the structure matrix (
correlations of variables with factors), or the factor score coefficients. Grice defined a variable as
salient on a factor if it had a weighting coefficient whose absolute value was at least 1/3 as large as
that of the variable with the largest absolute weighting coefficient on that factor. Salient items'
weights were replaced with 1 or 1, and nonsalient variables' weights with 0. The results of his Monte
Carlo study indicated that factor scores using this unit-weighting scheme based on scoring
coefficients performed better than those using various other unit-weighting schemes and at least as
good as exact factor scores (by most criteria and under most conditions). He did note, however, that
exact factor scores may be preferred under certain circumstances -- for example, when using factor
scores on the same sample as that from which they were derived, especially when sample size is
relatively small.

Comparing Two Groups Factor Structure
Suppose I wish to compare the factor structure in one population with that in another. I first
collect data from randomly selected samples from each population. I then extract factors from each
group using the same criteria for number of factors etc. in both groups. An eyeball test should tell
whether or not the two structures are dissimilar. Do I get the same number of well defined factors in
the two groups? Is the set of variables loading well on a given factor the same in Group A as in
Group B (minor differences would be expected due to sampling error, of course - one can randomly
split one sample in half and do separate analyses on each half to get a picture of such error).
Consider the following hypothetical loading matrices:

8

Group A Group B
Variable Factor1 Factor2 Factor3 Factor1 Factor2 Factor3
X1 .90 .12 .21 .45 .49 -.70
X2 .63 .75 .34 .65 -.15 .22
X3 .15 .67 .24 .27 .80 .04
X4 -.09 -.53 -.16 -.15 .09 .67
X5 -.74 -.14 -.19 .95 -.79 .12
SSL 1.78 1.32 0.28 1.62 1.53 1.00

One may use a simple Pearson r to compare two factors. Just correlate the loadings on
the factor in Group A with the loadings on the maybe similar factor in Group B. For Factor 1 in Group
A compared with Factor 1 in Group B, the A(B) data are .90(.45), .63(.65), .15(.27), -.09(-.15),
& -.74(.95). The r is -0.19, indicating little similarity between the two factors.
The Pearson r can detect not only differences in two factors patterns of loadings, but also
differences in the relative magnitudes of those loadings. One should beware that with factors having
a large number of small loadings, those small loadings could cause the r to be large (if they are
similar between factors) even if the factors had dissimilar loadings on the more important variables.
CC, Tuckers coefficient of congruence. Perform a procrustean rotation of the solution for
the one group, which makes it as similar as possible to the solution in the other group. Multiply each
loading in the one group by the corresponding loading in the other group. Sum these products and
then divide by the square root of (the sum of squared loadings for the one group times the sum of
squared loading for the other group). CC of .85 to .94 corresponds to similar factors, and .95 to 1 as
essentially identical factors.
See Comparing Two Groups Factor Structures: Pearson r and the Coefficient of Congruence
for an example of computing r and Tuckers CC.
Cross-correlated factor scores. Compute factor scoring coefficients for Group 1 and,
separately, for Group 2. Then for each case compute the factor score using the scoring coefficients
from the group in which is it located and also compute it using the scoring coefficients from the other
group. Correlate these two sets of factor scores Same Group and Other Group. A high correlation
between these two sets of factor scores should indicate similarity of the two factors between groups.
Of course, this method and the other two could be used with random halves of one sample to assess
the stability of the solution or with different random samples from the same population at different
times to get something like a test-retest measure of stability across samples and times.
RMS, root mean square. For each variable square the difference between the loading in the
one group and that in the other group. Find the mean of these differences and then the square root
of that mean. If there is a perfect match between the two groups loadings, RMS = 0. The maximum
value of RMS (2) would result when all of the loadings are one or minus one, with those in the one
group opposite in sign of those in the other group.
Cattells salient similarity index (s). I have not often seen this method used in recently
published articles. If you would like to learn a little about it, see my document Cattells s .
See Factorial Invariance of the Occupational Work Ethic Inventory -- An example of the use of
multiple techniques to compare factor structures.

9
Required Number of Subjects and Variables
"How many subjects do I need for my factor analysis?" If you ask this question of several
persons who occasionally employ factor analysis, you are likely to get several different answers.
Some may answer "100 or more" or give you some other fixed minimum N. Others may say "That
depends on how many variables you have, you should have at least 10 times as many subjects as
you have variables" (or some other fixed minimum ratio of subjects to variables). Others may say
"get as many subjects as you can, the more the better, but you must have at least as many subjects
as you have variables." I would say "that depends on several things." So what are those several
things? Here I shall summarize the key points of two recent articles that address this issue. The first
also addresses the question of how many variables should be included in the analysis. The articles
are:
Velicer, W. F., & Fava, J. L. (1998). Effects of variable and subject sampling on factor pattern
recovery. Psychological Methods, 3, 231-251.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis.
Psychological Methods, 4, 84-99.
Velicer & Fava briefly reviewed the literature and then reported the results of their empirical
study. The main points were as follows:
Factors that do not have at least three variables with high loadings should not be interpreted
accordingly, you need to have at least three variables per factor, probably more. Since not all
of your variables can be expected to perform as anticipated, you should start out with 6 to 10
variables per factor.
If the loadings are low, you will need more variables, 10 or 20 per factor may be required.
Assuming that some variables will have to be discarded, you should start with 20 to 30
variables per factor.
The larger the number of subjects, the larger the number of variables per factor, and the larger
the loadings, the better are your chances of doing a good job at reproducing the population
factor pattern. Strength in one of these areas can compensate for weakness in another for
example, you might compensate for mediocre loadings and variable sample size by having a
very large number of subjects. If you can't get a lot of subjects, you can compensate by having
many good variables per factor (start with 30 60 per factor, but not more variables than
subjects).
MacCullum et al. demonstrated, by both theoretical and empirical means that the required
sample size or ratio of subjects to variables is not constant across studies, but rather varies as a
function of several other aspects of the research. They concluded that rules of thumb about sample
size are not valid and not useful. Here are the key points:
Larger sample size, higher communalities (low communalities are associated with sampling
error due to the presence of unique factors that have nonzero correlations with one another
and with the common factors), and high overdetermination [each factor having at least three
or four high loadings and simple structure (few, nonoverlapping factors)] each increase your
chances of faithfully reproducing the population factor pattern.
Strengths in one area can compensate for weaknesses in another area.
When communalities are high (> .6), you should be in good shape even with N well below 100.
With communalities moderate (about .5) and the factors well-determined, you should have 100
to 200 subjects.
10
With communalities low (< .5) but high overdetermination of factors (not many factors, each
with 6 or 7 high loadings), you probably need well over 100 subjects.
With low communalities and only 3 or 4 high loadings on each, you probably need over 300
subjects.
With low communalities and poorly determined factors, you will need well over 500 subjects.
Of course, when planning your research you do not know for sure how good the communalities
will be nor how well determined your factors will be, so maybe the best simple advise, for an a priori
rule of thumb, is "the more subjects, the better." MacCallum's advise to researchers is to try to keep
the number of variables and factors small and select variables (write items) to assure moderate to
high communalities.
Out With the Reputation Variable
Over and over we have seen that the reputation variable really does not belong in the factor
analysis. Lets see what happens when we delete it. Run the program FactBeer.sas from my SAS
Programs page. You can also find there annotated output from this program.
Closing Comments
Please note that this has been an introductory lesson that has not addressed many of the less
common techniques available. For example, I have not discussed Alpha Extraction, which extracts
factors with the goal of maximizing alpha (reliability) coefficients of the Extracted Factors, or
Maximum-Likelihood Extraction, or several other extraction methods.
I should remind you of the necessity of investigating (maybe even deleting) outlying
observations. Subjects factor scores may be inspected to find observations that are outliers with
respect to the solution [very large absolute value of a factor score].
I deliberately kept the number the variables low and the number of cases low, for practical and
pedagogical purposes. For actual research this is strongly discouraged.
References

Cattell, R. B., Balcar, K. R., Horn, J. L., & Nesselroade, J. R. (1969). Factor matching procedures: an
improvement of the s index; with tables. Educational and Psychological Measurement, 29, 781 792. doi
10.1177/001316446902900405
Grice, J. W. (2001). A comparison of factor scores under conditions of factor obliquity.
Psychological Methods, 6, 67 - 83.
Grice, J. W., & Harris, R. J. (1998). A comparison of regression and loading weights for the
computation of factor scores. Multivariate Behavioral Research, 33, 221 - 247.

The url for this document is http://core.ecu.edu/psyc/wuenschk/MV/FA/FA.docx.

Polychoric and Tetrachoric Factor Analysis with data from Likert-type or dichotomous items.

Return to Wuenschs Statistics Lessons Page

Copyright 2013, Karl L. Wuensch - All rights reserved.