You are on page 1of 20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Factoranalysis
FromWikipedia,thefreeencyclopedia

Factoranalysisisastatisticalmethodusedtodescribevariabilityamongobserved,correlatedvariablesin
termsofapotentiallylowernumberofunobservedvariablescalledfactors.Forexample,itispossiblethat
variationsinfourobservedvariablesmainlyreflectthevariationsintwounobservedvariables.Factor
analysissearchesforsuchjointvariationsinresponsetounobservedlatentvariables.Theobserved
variablesaremodelledaslinearcombinationsofthepotentialfactors,plus"error"terms.Theinformation
ofobservedvariables.Factoranalysisoriginatedinpsychometricsandisusedinbehavioralsciences,social
sciences,marketing,productmanagement,operationsresearch,andotherappliedsciencesthatdealwith
largequantitiesofdata.
Factoranalysisisrelatedtoprincipalcomponentanalysis(PCA),butthetwoarenotidentical.Latent
variablemodels,includingfactoranalysis,useregressionmodellingtechniquestotesthypotheses
controversyinthefieldovertheequivalenceorotherwiseofthetwotechniques(seeexploratoryfactor
analysisversusprincipalcomponentsanalysis).

Contents
1Statisticalmodel
1.1Definition
1.2Example
1.3Mathematicalmodelofthesameexample
1.4Geometricinterpretation
2Practicalimplementation
2.1Typeoffactoranalysis
2.2Typesoffactoring
2.3Terminology
2.4Criteriafordeterminingthenumberoffactors
2.5Rotationmethods
3Factoranalysisinpsychometrics
3.1History
3.2Applicationsinpsychology
4Exploratoryfactoranalysisversusprincipalcomponentsanalysis
4.1ArgumentscontrastingPCAandEFA
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

1/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

4.2Varianceversuscovariance
4.3Differencesinprocedureandresults
5Factoranalysisinmarketing
5.1Informationcollection
5.2Analysis
6Factoranalysisinphysicalandbiologicalsciences
7Factoranalysisinmicroarrayanalysis
8Implementation
9Seealso

Statisticalmodel
Definition
Supposewehaveasetof observablerandomvariables,

withmeans

Supposeforsomeunknownconstants and unobservedrandomvariables

and
,where
,wehave

,where

Here,the areindependentlydistributederrortermswithzeromeanandfinitevariance,whichmaynotbe
thesameforall .Let
,sothatwehave

Inmatrixterms,wehave

Ifwehave observations,thenwewillhavethedimensions
,
,and
.Eachcolumnof
and denotevaluesforoneparticularobservation,andmatrix doesnotvaryacrossobservations.
Alsowewillimposethefollowingassumptionson :
1.

and areindependent.

2.
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

2/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

3.

(tomakesurethatthefactorsareuncorrelated).

Anysolutionoftheabovesetofequationsfollowingtheconstraintsfor isdefinedasthefactors,and
Suppose

.Thennotethatfromtheconditionsjustimposedon ,wehave

or

or

Notethatforanyorthogonalmatrix ,ifweset
and
,thecriteriaforbeingfactors
transformation.

Example
Thefollowingexampleisforexpositorypurposes,andshouldnotbetakenasbeingrealistic.Supposea
psychologistproposesatheorythattherearetwokindsofintelligence,"verbalintelligence"and
"mathematicalintelligence",neitherofwhichisdirectlyobserved.Evidenceforthetheoryissoughtinthe
randomlyfromalargepopulation,theneachstudent's10scoresarerandomvariables.Thepsychologist's
whosharesomecommonpairofvaluesforverbalandmathematical"intelligences"issomeconstanttimes
theirlevelofverbalintelligenceplusanotherconstanttimestheirlevelofmathematicalintelligence,i.e.,it
isacombinationofthosetwo"factors".Thenumbersforaparticularsubject,bywhichthetwokindsof
intelligencearemultipliedtoobtaintheexpectedscore,arepositedbythetheorytobethesameforall
thattheaveragestudent'saptitudeinthefieldoftaxonomyis
{10thestudent'sverbalintelligence}+{6thestudent'smathematicalintelligence}.
Twostudentshavingidenticaldegreesofverbalintelligenceandidenticaldegreesofmathematical
intelligencemayhavedifferentaptitudesintaxonomybecauseindividualaptitudesdifferfromaverage
aptitudes.Thatdifferenceiscalledthe"error"astatisticaltermthatmeanstheamountbywhichan
individualdiffersfromwhatisaverageforhisorherlevelsofintelligence(seeerrorsandresidualsin
statistics).

https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

3/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Theobservabledatathatgointofactoranalysiswouldbe10scoresofeachofthe1000students,atotalof
inferredfromthedata.

Mathematicalmodelofthesameexample
Inthefollowing,matriceswillbeindicatedbyindexedvariables."Subject"indiceswillbeindicatedusing
lettersa,bandc,withvaluesrunningfrom1to
whichisequalto10intheaboveexample."Factor"
indiceswillbeindicatedusinglettersp,qandr,withvaluesrunningfrom1to whichisequalto2inthe
aboveexample."Instance"or"sample"indiceswillbeindicatedusinglettersi,jandk,withvaluesrunning
from1to .Intheexampleabove,ifasampleof
studentsrespondedtothe
questions,theithstudent'sscorefortheathquestionaregivenby .Thepurposeoffactoranalysisisto
characterizethecorrelationsbetweenthevariables ofwhichthe areaparticularinstance,orsetof
observations.Inorderthatthevariablesbeonequalfooting,theyarestandardized:

wherethesamplemeanis:

andthesamplevarianceisgivenby:

Thefactoranalysismodelforthisparticularsampleisthen:

or,moresuccinctly:

where
istheithstudent's"verbalintelligence",
istheithstudent's"mathematicalintelligence",
Inmatrixnotation,wehave
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

4/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Observethatbydoublingthescaleonwhich"verbalintelligence"thefirstcomponentineachcolumnof
tothemodel.Thus,nogeneralityislostbyassumingthatthestandarddeviationofverbalintelligenceis1.
Likewiseformathematicalintelligence.Moreover,forsimilarreasons,nogeneralityislostbyassumingthe
twofactorsareuncorrelatedwitheachother.Inotherwords:

where istheKroneckerdelta(0when
independentofthefactors:

and1when

).Theerrorsareassumedtobe

Notethat,sinceanyrotationofasolutionisalsoasolution,thismakesinterpretingthefactorsdifficult.See
intelligenceareuncorrelated,thenwecannotinterpretthetwofactorsasthetwodifferenttypesof
intelligence.Eveniftheyareuncorrelated,wecannottellwhichfactorcorrespondstoverbalintelligence
andwhichcorrespondstomathematicalintelligencewithoutanoutsideargument.
"fundamentaltheorem"maybederivedfromtheaboveconditions:

Thetermontheleftisjustthecorrelationmatrixoftheobserveddata,andits diagonalelementswillbe
rightisthe"reducedcorrelationmatrix"andwillbeequaltothecorrelationmatrixexceptforitsdiagonal
valueswhichwillbelessthanunity.Thesediagonalelementsofthereducedcorrelationmatrixarecalled
"communalities":

Thesampledata willnot,ofcourse,exactlyobeythefundamentalequationgivenabovedueto
factors
fitisdefinedastheminimumofthemeansquareerrorintheoffdiagonalresidualsofthecorrelation
matrix:[2]

https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

5/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Thisisequivalenttominimizingtheoffdiagonalcomponentsoftheerrorcovariancewhich,inthemodel
equationshaveexpectedvaluesofzero.Thisistobecontrastedwithprincipalcomponentanalysiswhich
considerableeffortwasdevotedtofindingapproximatesolutionstotheproblem,particularlyinestimating
thecommunalitiesbyothermeans,whichthensimplifiestheproblemconsiderablybyyieldingaknown
highspeedcomputers,theminimizationproblemcanbesolvedquicklyanddirectly,andthecommunalities
arecalculatedintheprocess,ratherthanbeingneededbeforehand.TheMinResalgorithmisparticularly
suitedtothisproblem,butishardlytheonlymeansoffindinganexactsolution.

Geometricinterpretation
Theparametersandvariablesoffactoranalysis
canbegivenageometricalinterpretation.The
data( ),thefactors(
)andtheerrors( )
canbeviewedasvectorsinan dimensional
Euclideanspace(samplespace),representedas
, and respectively.Sincethedatais
standardized,thedatavectorsareofunitlength
(
).Thefactorvectorsdefinean
dimensionallinearsubspace(i.e.a
hyperplane)inthisspace,uponwhichthedata
vectorsareprojectedorthogonally.Thisfollows
fromthemodelequation

andtheindependenceofthefactorsandthe
errors:
.Intheaboveexample,the
hyperplaneisjusta2dimensionalplanedefined
bythetwofactorvectors.Theprojectionofthe
datavectorsontothehyperplaneisgivenby

GeometricinterpretationofFactorAnalysisparametersfor3
theunitvector ,whichisprojectedontoaplanedefined
bytwoorthonormalvectors and
.Theprojection
vectoris andtheerror isperpendiculartotheplane,
sothat
.Theprojectionvector maybe
representedintermsofthefactorvectorsas
.Thesquareofthelengthofthe
projectionvectoristhecommunality:
.If
anotherdatavector wereplotted,thecosineoftheangle
between and wouldbe
:the(a,b)entryinthe

andtheerrorsarevectorsfromthatprojected
pointtothedatapointandareperpendicularto
thehyperplane.Thegoaloffactoranalysisistofindahyperplanewhichisa"bestfit"tothedatainsome
sense,soitdoesn'tmatterhowthefactorvectorswhichdefinethishyperplanearechosen,aslongasthey
areindependentandlieinthehyperplane.Wearefreetospecifythemasbothorthogonalandnormal(
)withnolossofgenerality.Afterasuitablesetoffactorsarefound,theymayalsobe
arbitrarilyrotatedwithinthehyperplane,sothatanyrotationofthefactorvectorswilldefinethesame
hyperplane,andalsobeasolution.Asaresult,intheaboveexample,inwhichthefittinghyperplaneistwo
dimensional,ifwedonotknowbeforehandthatthetwotypesofintelligenceareuncorrelated,thenwe
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

6/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

cannotinterpretthetwofactorsasthetwodifferenttypesofintelligence.Eveniftheyareuncorrelated,we
cannottellwhichfactorcorrespondstoverbalintelligenceandwhichcorrespondstomathematical
intelligence,orwhetherthefactorsarelinearcombinationsofboth,withoutanoutsideargument.
Thedatavectors haveunitlength.Thecorrelationmatrixforthedataisgivenby
.The
correlationmatrixcanbegeometricallyinterpretedasthecosineoftheanglebetweenthetwodatavectors
and .Thediagonalelementswillclearlybe1'sandtheoffdiagonalelementswillhaveabsolute
valueslessthanorequaltounity.The"reducedcorrelationmatrix"isdefinedas
.
Thegoaloffactoranalysisistochoosethefittinghyperplanesuchthatthereducedcorrelationmatrix
reproducesthecorrelationmatrixasnearlyaspossible,exceptforthediagonalelementsofthecorrelation
matrixwhichareknowntohaveunitvalue.Inotherwords,thegoalistoreproduceasaccuratelyas
possiblethecrosscorrelationsinthedata.Specifically,forthefittinghyperplane,themeansquareerrorin
theoffdiagonalcomponents

istobeminimized,andthisisaccomplishedbyminimizingitwithrespecttoasetoforthonormalfactor
vectors.Itcanbeseenthat

Thetermontherightisjustthecovarianceoftheerrors.Inthemodel,theerrorcovarianceisstatedtobea
diagonalmatrixandsotheaboveminimizationproblemwillinfactyielda"bestfit"tothemodel:Itwill
yieldasampleestimateoftheerrorcovariancewhichhasitsoffdiagonalcomponentsminimizedinthe
meansquaresense.Itcanbeseenthatsincethe areorthogonalprojectionsofthedatavectors,their
lengthwillbelessthanorequaltothelengthoftheprojecteddatavector,whichisunity.Thesquareof
theselengthsarejustthediagonalelementsofthereducedcorrelationmatrix.Thesediagonalelementsof
thereducedcorrelationmatrixareknownas"communalities":

Largevaluesofthecommunalitieswillindicatethatthefittinghyperplaneisratheraccuratelyreproducing
thecorrelationmatrix.Itshouldbenotedthatthemeanvaluesofthefactorsmustalsobeconstrainedtobe
zero,fromwhichitfollowsthatthemeanvaluesoftheerrorswillalsobezero.

Practicalimplementation
Typeoffactoranalysis
Exploratoryfactoranalysis(EFA)isusedtoidentifycomplexinterrelationshipsamongitemsandgroup
relationshipsamongfactors.[3]
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

7/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Confirmatoryfactoranalysis(CFA)isamorecomplexapproachthatteststhehypothesisthattheitems
areassociatedwithspecificfactors.[3]CFAusesstructuralequationmodelingtotestameasurementmodel
unobservedvariables.[3]Structuralequationmodelingapproachescanaccommodatemeasurementerror,
andarelessrestrictivethanleastsquaresestimation.[3]Hypothesizedmodelsaretestedagainstactualdata,
asthecorrelationbetweenthelatentvariables.[3]

Typesoffactoring
Principalcomponentanalysis(PCA):PCAisawidelyusedmethodforfactorextraction,whichisthe
firstphaseofEFA.[3]Factorweightsarecomputedinordertoextractthemaximumpossiblevariance,with
successivefactoringcontinuinguntilthereisnofurthermeaningfulvarianceleft.[3]Thefactormodelmust
thenberotatedforanalysis.[3]
samemodelasPCA,whichusestheprincipalaxismethod.Canonicalfactoranalysisseeksfactorswhich
havethehighestcanonicalcorrelationwiththeobservedvariables.Canonicalfactoranalysisisunaffected
byarbitraryrescalingofthedata.
Commonfactoranalysis,alsocalledprincipalfactoranalysis(PFA)orprincipalaxisfactoring(PAF),
seekstheleastnumberoffactorswhichcanaccountforthecommonvariance(correlation)ofasetof
variables.
Imagefactoring:basedonthecorrelationmatrixofpredictedvariablesratherthanactualvariables,where
eachvariableispredictedfromtheothersusingmultipleregression.
Alphafactoring:basedonmaximizingthereliabilityoffactors,assumingvariablesarerandomlysampled
fromauniverseofvariables.Allothermethodsassumecasestobesampledandvariablesfixed.
Factorregressionmodel:acombinatorialmodeloffactormodelandregressionmodeloralternatively,it
canbeviewedasthehybridfactormodel,[4]whosefactorsarepartiallyknown.

Terminology
arethecorrelationcoefficientsbetweenthecases(rows)andfactors(columns).AnalogoustoPearson'sr,
equalsthesumoftheirvariancesasthevarianceofastandardizedvariableis1.)Thisisthesameas
dividingthefactor'seigenvaluebythenumberofvariables.
orhighertoconfirmthatindependentvariablesidentifiedaprioriarerepresentedbyaparticularfactor,on
thefactor.However,the.7standardisahighoneandreallifedatamaywellnotmeetthiscriterion,which
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

8/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

iswhysomeresearchers,particularlyforexploratorypurposes,willusealowerlevelsuchas.4forthe
theory,notbyarbitrarycutofflevels.
Inobliquerotation,onegetsbothapatternmatrixandastructurematrix.Thestructurematrixissimplythe
byafactoronbothauniqueandcommoncontributionsbasis.Thepatternmatrix,incontrast,contains
coefficientswhichjustrepresentuniquecontributions.Themorefactors,thelowerthepatterncoefficients
asarulesincetherewillbemorecommoncontributionstovarianceexplained.Forobliquerotation,the
researcherlooksatboththestructureandpatterncoefficientswhenattributingalabeltoafactor.Principles
ofobliquerotationcanbederivedfrombothcrossentropyanditsdualentropy.[5]
varianceinthatvariableaccountedforbyallthefactors,andthisiscalledthecommunality.The
communalitymeasuresthepercentofvarianceinagivenvariableexplainedbyallthefactorsjointlyand
maybeinterpretedasthereliabilityoftheindicator.
Spurioussolutions:Ifthecommunalityexceeds1.0,thereisaspurioussolution,whichmayreflecttoo
smallasampleortheresearcherhastoomanyortoofewfactors.
Uniquenessofavariable:Thatis,uniquenessisthevariabilityofavariableminusitscommunality.
Eigenvalues:/Characteristicroots:Theeigenvalueforagivenfactormeasuresthevarianceinallthe
variableswhichisaccountedforbythatfactor.Theratioofeigenvaluesistheratioofexplanatory
importanceofthefactorswithrespecttothevariables.Ifafactorhasaloweigenvalue,thenitis
contributinglittletotheexplanationofvariancesinthevariablesandmaybeignoredasredundantwith
moreimportantfactors.Eigenvaluesmeasuretheamountofvariationinthetotalsampleaccountedforby
eachfactor.
methods,eigenvaluesafterextractionwillbelowerthantheirinitialcounterparts.SPSSalsoprints
extractioneigenvalues,thoughtheirtotalwillbethesame.
Factorscores(alsocalledcomponentscoresinPCA):arethescoresofeachcase(row)oneachfactor
(column).Tocomputethefactorscoreforagivencaseforagivenfactor,onetakesthecase'sstandardized
sumstheseproducts.Computingfactorscoresallowsonetolookforfactoroutliers.Also,factorscoresmay
beusedasvariablesinsubsequentmodeling.(ExplainedfromPCAnotfromFactorAnalysisperspective).

Criteriafordeterminingthenumberoffactors
Usingoneormoreofthemethodsbelow,theresearcherdeterminesanappropriaterangeofsolutionsto
investigate.Methodsmaynotagree.Forinstance,theKaisercriterionmaysuggestfivefactorsandthe
screetestmaysuggesttwo,sotheresearchermayrequest3,4,and5factorsolutionsdiscusseachin
termsoftheirrelationtoexternaldataandtheory.

https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

9/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Comprehensibility:Apurelysubjectivecriterionwouldbetoretainthosefactorswhosemeaningis
comprehensibletotheresearcher.Thisisnotrecommended.
Kaisercriterion:TheKaiserruleistodropallcomponentswitheigenvaluesunder1.0thisbeingthe
eigenvalueequaltotheinformationaccountedforbyanaveragesingleitem.TheKaisercriterionisthe
defaultinSPSSandmoststatisticalsoftwarebutisnotrecommendedwhenusedasthesolecutoffcriterion
forestimatingthenumberoffactorsasittendstooverextractfactors.[6]Avariationofthismethodhasbeen
createdwherearesearchercalculatesconfidenceintervalsforeacheigenvalueandretainsonlyfactors
whichhavetheentireconfidenceintervalgreaterthan1.0.[7][8]
for90%(sometimes80%)ofthevariation.Wheretheresearcher'sgoalemphasizesparsimony(explaining
variancewithasfewfactorsaspossible),thecriterioncouldbeaslowas50%
Screeplot:TheCattellscreetestplotsthecomponentsastheXaxisandthecorrespondingeigenvaluesas
theYaxis.Asonemovestotheright,towardlatercomponents,theeigenvaluesdrop.Whenthedrop
ceasesandthecurvemakesanelbowtowardlesssteepdecline,Cattell'sscreetestsaystodropallfurther
componentsaftertheonestartingtheelbow.Thisruleissometimescriticisedforbeingamenableto
researchercontrolled"fudging".Thatis,aspickingthe"elbow"canbesubjectivebecausethecurvehas
multipleelbowsorisasmoothcurve,theresearchermaybetemptedtosetthecutoffatthenumberof
factorsdesiredbyhisorherresearchagenda.
Horn'sParallelAnalysis(PA):AMonteCarlobasedsimulationmethodthatcomparestheobserved
eigenvalueswiththoseobtainedfromuncorrelatednormalvariables.Afactororcomponentisretainedif
theassociatedeigenvalueisbiggerthanthe95thofthedistributionofeigenvaluesderivedfromtherandom
data.PAisoneofthemostrecommendablerulesfordeterminingthenumberofcomponentstoretain,but
onlyfewprogramsincludethisoption.[9]
basedonafactorwithaverysmalleigenvaluecancorrelatestronglywithdependentvariables,inwhich
casedroppingsuchafactorfromatheoreticalmodelmayreduceitspredictivevalidity.
Velicers(1976)MAPtest[10]involvesacompleteprincipalcomponentsanalysisfollowedbythe
examinationofaseriesofmatricesofpartialcorrelations(p.397).ThesquaredcorrelationforStep0
(seeFigure4)istheaveragesquaredoffdiagonalcorrelationfortheunpartialedcorrelationmatrix.OnStep
1,thefirstprincipalcomponentanditsassociateditemsarepartialedout.Thereafter,theaveragesquared
offdiagonalcorrelationforthesubsequentcorrelationmatrixisthencomputedforStep1.OnStep2,the
firsttwoprincipalcomponentsarepartialedoutandtheresultantaveragesquaredoffdiagonalcorrelationis
againcomputed.Thecomputationsarecarriedoutforkminusonestep(krepresentingthetotalnumberof
variablesinthematrix).Thereafter,alloftheaveragesquaredcorrelationsforeachsteparelinedupandthe
stepnumberintheanalysesthatresultedinthelowestaveragesquaredpartialcorrelationdeterminesthe
numberofcomponentsorfactorstoretain(Velicer,1976).Bythismethod,componentsaremaintainedas
longasthevarianceinthecorrelationmatrixrepresentssystematicvariance,asopposedtoresidualorerror
variance.Althoughmethodologicallyakintoprincipalcomponentsanalysis,theMAPtechniquehasbeen
showntoperformquitewellindeterminingthenumberoffactorstoretaininmultiplesimulation
forguidance.
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

10/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Rotationmethods
Theunrotatedoutputmaximisesvarianceaccountedforbythefirstandsubsequentfactors,andforcingthe
orthogonaloroblique(allowingthefactorstocorrelate).
Varimaxrotationisanorthogonalrotationofthefactoraxestomaximizethevarianceofthesquared
differentiatingtheoriginalvariablesbyextractedfactor.Eachfactorwilltendtohaveeitherlargeorsmall
identifyeachvariablewithasinglefactor.Thisisthemostcommonrotationoption.However,the
orthogonality(i.e.,independence)offactorsisoftenanunrealisticassumption.Obliquerotationsare
inclusiveoforthogonalrotation,andforthatreason,obliquerotationsareapreferredmethod.[14]
Quartimaxrotationisanorthogonalalternativewhichminimizesthenumberoffactorsneededtoexplain
EquimaxrotationisacompromisebetweenVarimaxandQuartimaxcriteria.
Directobliminrotationisthestandardmethodwhenonewishesanonorthogonal(oblique)solutionthat
is,oneinwhichthefactorsareallowedtobecorrelated.Thiswillresultinhighereigenvaluesbut
diminishedinterpretabilityofthefactors.Seebelow.
Promaxrotationisanalternativenonorthogonal(oblique)rotationmethodwhichiscomputationally
fasterthanthedirectobliminmethodandthereforeissometimesusedforverylargedatasets.

Factoranalysisinpsychometrics
History
CharlesSpearmanpioneeredtheuseoffactoranalysisinthefieldofpsychologyandissometimescredited
withtheinventionoffactoranalysis.Hediscoveredthatschoolchildren'sscoresonawidevarietyof
seeminglyunrelatedsubjectswerepositivelycorrelated,whichledhimtopostulatethatageneralmental
thefieldofintelligenceresearch,whereitisknownasthegtheory.
RaymondCattellexpandedonSpearman'sideaofatwofactortheoryofintelligenceafterperforminghis
owntestsandfactoranalysis.Heusedamultifactortheorytoexplainintelligence.Cattell'stheory
similaritycoefficients.Hisresearchledtothedevelopmentofhistheoryoffluidandcrystallized
factoranalysisandpsychometrics.Hebelievedthatalltheoryshouldbederivedfromresearch,which
supportsthecontinueduseofempiricalobservationandobjectivetestingtostudyhumanintelligence.
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

11/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Applicationsinpsychology
Factoranalysisisusedtoidentify"factors"thatexplainavarietyofresultsondifferenttests.Forexample,
intelligenceresearchfoundthatpeoplewhogetahighscoreonatestofverbalabilityarealsogoodon
otherteststhatrequireverbalabilities.Researchersexplainedthisbyusingfactoranalysistoisolateone
factor,oftencalledcrystallizedintelligenceorverbalintelligence,whichrepresentsthedegreetowhich
someoneisabletosolveproblemsinvolvingverbalskills.
Factoranalysisinpsychologyismostoftenassociatedwithintelligenceresearch.However,italsohasbeen
thepostulatedfactors.

Reductionofnumberofvariables,bycombiningtwoormorevariablesintoasinglefactor.For
example,performanceatrunning,ballthrowing,batting,jumpingandweightliftingcouldbe
combinedintoasinglefactorsuchasgeneralathleticability.Usually,inanitembypeoplematrix,
factorsareselectedbygroupingrelateditems.IntheQfactoranalysistechnique,thematrixis
transposedandfactorsarecreatedbygroupingrelatedpeople:Forexample,liberals,libertarians,
conservativesandsocialists,couldformseparategroups.
Identificationofgroupsofinterrelatedvariables,toseehowtheyarerelatedtoeachother.For
example,CarrollusedfactoranalysistobuildhisThreeStratumTheory.Hefoundthatafactorcalled
perception".Thismeanssomeonewithahigh"g"islikelytohavebothahigh"visualperception"
capabilityandahigh"auditoryperception"capability,andthat"g"thereforeexplainsagoodpartof

"...eachorientationisequallyacceptablemathematically.Butdifferentfactorialtheoriesprovedto
differasmuchintermsoftheorientationsoffactorialaxesforagivensolutionasintermsof
anythingelse,sothatmodelfittingdidnotprovetobeusefulindistinguishingamongtheories."
(Sternberg,1977[15]).Thismeansallrotationsrepresentdifferentunderlyingprocesses,butall
rotationsareequallyvalidoutcomesofstandardfactoranalysisoptimization.Therefore,itis
impossibletopicktheproperrotationusingfactoranalysisalone.
Factoranalysiscanbeonlyasgoodasthedataallows.Inpsychology,whereresearchersoftenhave
torelyonlessvalidandreliablemeasuressuchasselfreports,thiscanbeproblematic.
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

12/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Interpretingfactoranalysisisbasedonusinga"heuristic",whichisasolutionthatis"convenient
thesameway,andfactoranalysiscannotidentifycausality.

Exploratoryfactoranalysisversusprincipalcomponentsanalysis
Whileexploratoryfactoranalysisandprincipalcomponentanalysisaretreatedassynonymoustechniques
insomefieldsofstatistics,thishasbeencriticised(e.g.Fabrigaretal.,1999[17]Suhr,2009[18]).Infactor
analysis,theresearchermakestheassumptionthatanunderlyingcausalmodelexists,whereasPCAis
simplyavariablereductiontechnique.[19]Researchershavearguedthatthedistinctionsbetweenthetwo
techniquesmaymeanthatthereareobjectivebenefitsforpreferringoneovertheotherbasedontheanalytic
goal.Ifthefactormodelisincorrectlyformulatedortheassumptionsarenotmet,thenfactoranalysiswill
systempermitsgoodinitialmodelformulations.Principalcomponentanalysisemploysamathematical
PCAistodetermineafewlinearcombinationsoftheoriginalvariablesthatcanbeusedtosummarizethe
datasetwithoutlosingmuchinformation.[20]

ArgumentscontrastingPCAandEFA
notequivalenttofactoranalysis:
1. Itissometimessuggestedthatprincipalcomponentsanalysisiscomputationallyquickerandrequires
resourceshaverenderedthispracticalconcernirrelevant.
certaincases,wherebythecommunalitiesarelow(e.g.,.40),thetwotechniquesproducedivergent
results.Infact,Fabrigaretal.arguethatincaseswherethedatacorrespondtoassumptionsofthe
commonfactormodel,theresultsofPCAareinaccurateresults.
whereby100%ormoreofthevarianceinameasuredvariableisestimatedtobeaccountedforbythe
model.Fabrigaretal.suggestthatthesecasesareactuallyinformativetotheresearcher,indicatinga
misspecifiedmodeloraviolationofthecommonfactormodel.ThelackofHeywoodcasesinthe
PCAapproachmaymeanthatsuchissuespassunnoticed.
4. ResearchersgainextrainformationfromaPCAapproach,suchasanindividualsscoreonacertain
componentsuchinformationisnotyieldedfromfactoranalysis.However,asFabrigaretal.
contend,thetypicalaimoffactoranalysisi.e.todeterminethefactorsaccountingforthestructure
ofthecorrelationsbetweenmeasuredvariablesdoesnotrequireknowledgeoffactorscoresand
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

13/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Varianceversuscovariance
Factoranalysistakesintoaccounttherandomerrorthatisinherentinmeasurement,whereasPCAfailsto
doso.ThispointisexemplifiedbyBrown(2009),[21]whoindicatedthat,inrespecttothecorrelation
matricesinvolvedinthecalculations:

"InPCA,1.00sareputinthediagonalmeaningthatallofthevarianceinthematrixisto
beaccountedfor(includingvarianceuniquetoeachvariable,variancecommonamong
variables,anderrorvariance).Thatwould,therefore,bydefinition,includeallofthe
varianceinthevariables.Incontrast,inEFA,thecommunalitiesareputinthediagonal
meaningthatonlythevariancesharedwithothervariablesistobeaccountedfor
(excludingvarianceuniquetoeachvariableanderrorvariance).Thatwould,therefore,by
definition,includeonlyvariancethatiscommonamongthevariables."
Brown(2009),Principalcomponentsanalysisandexploratoryfactoranalysis
Definitions,differencesandchoices

betweenvariablesexist,whereasPCAshouldbeusedifthegoaloftheresearcheristoexplorepatternsin
theirdata.

Differencesinprocedureandresults
ThedifferencesbetweenprincipalcomponentsanalysisandfactoranalysisarefurtherillustratedbySuhr
(2009):
PCAresultsinprincipalcomponentsthataccountforamaximalamountofvarianceforobserved
variablesFAaccountforcommonvarianceinthedata.[18]
correlationmatrixwiththeuniquefactors.[18]
PCAminimizesthesumofsquaredperpendiculardistancetothecomponentaxisFAestimates
factorswhichinfluenceresponsesonobservedvariables.[18]
ThecomponentscoresinPCArepresentalinearcombinationoftheobservedvariablesweightedby
eigenvectorstheobservedvariablesinFAarelinearcombinationsoftheunderlyingandunique
factors.[18]
InPCA,thecomponentsyieldedareuninterpretable,i.e.theydonotrepresentunderlying
accuratemodelspecification.[18]

Factoranalysisinmarketing
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

14/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Thebasicstepsare:
Identifythesalientattributesconsumersusetoevaluateproductsinthiscategory.
Usequantitativemarketingresearchtechniques(suchassurveys)tocollectdatafromasampleof
potentialcustomersconcerningtheirratingsofalltheproductattributes.
Inputthedataintoastatisticalprogramandrunthefactoranalysisprocedure.Thecomputerwill
yieldasetofunderlyingattributes(orfactors).
Usethesefactorstoconstructperceptualmapsandotherproductpositioningdevices.

Informationcollection
respondenttorateaproductsampleordescriptionsofproductconceptsonarangeofattributes.Anywhere
fromfivetotwentyattributesarechosen.Theycouldincludethingslike:easeofuse,weight,accuracy,
durability,colourfulness,price,orsize.Theattributeschosenwillvarydependingontheproductbeing
codedandinputintoastatisticalprogramsuchasR,SPSS,SAS,Stata,STATISTICA,JMP,andSYSTAT.

Analysis
Theanalysiswillisolatetheunderlyingfactorsthatexplainthedatausingamatrixofassociations.[22]
Factoranalysisisaninterdependencetechnique.Thecompletesetofinterdependentrelationshipsis
examined.Thereisnospecificationofdependentvariables,independentvariables,orcausality.Factor
analysisassumesthatalltheratingdataondifferentattributescanbereduceddowntoafewimportant
dimensions.Thisreductionispossiblebecausesomeattributesmayberelatedtoeachother.Therating
giventoanyoneattributeispartiallytheresultoftheinfluenceofotherattributes.Thestatisticalalgorithm
deconstructstherating(calledarawscore)intoitsvariouscomponents,andreconstructsthepartialscores
intounderlyingfactorscores.Thedegreeofcorrelationbetweentheinitialrawscoreandthefinalfactor

Bothobjectiveandsubjectiveattributescanbeusedprovidedthesubjectiveattributescanbe
convertedintoscores.
Factoranalysiscanidentifylatentdimensionsorconstructsthatdirectanalysismaynot.
Itiseasyandinexpensive.

Usefulnessdependsontheresearchers'abilitytocollectasufficientsetofproductattributes.If
importantattributesareexcludedorneglected,thevalueoftheprocedureisreduced.
Ifsetsofobservedvariablesarehighlysimilartoeachotheranddistinctfromotheritems,factor
analysiswillassignasinglefactortothem.Thismayobscurefactorsthatrepresentmoreinteresting
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

15/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

relationships.
Namingfactorsmayrequireknowledgeoftheorybecauseseeminglydissimilarattributescan
correlatestronglyforunknownreasons.

Factoranalysisinphysicalandbiologicalsciences
Factoranalysishasalsobeenwidelyusedinphysicalsciencessuchasgeochemistry,ecology,
hydrochemistry.,[23]astrophysics,cosmology,aswellasbiologicalsciencessuchasmolecularbiologyand
biochemistry.
Ingroundwaterqualitymanagement,itisimportanttorelatethespatialdistributionofdifferentchemical
parameterstodifferentpossiblesources,whichhavedifferentchemicalsignatures.Forexample,asulfide
mineislikelytobeassociatedwithhighlevelsofacidity,dissolvedsulfatesandtransitionmetals.These
signaturescanbeidentifiedasfactorsthroughRmodefactoranalysis,andthelocationofpossiblesources
canbesuggestedbycontouringthefactorscores.[24]
Ingeochemistry,differentfactorscancorrespondtodifferentmineralassociations,andthusto
mineralisation.[25]

Factoranalysisinmicroarrayanalysis
FactoranalysiscanbeusedforsummarizinghighdensityoligonucleotideDNAmicroarraysdataatprobe
levelforAffymetrixGeneChips.Inthiscase,thelatentvariablecorrespondstotheRNAconcentrationina
sample.[26]

Implementation
Factoranalysishasbeenimplementedinseveralstatisticalanalysisprogramssincethe1980s:SAS,BMDP
andSPSS.[27]ItisalsoimplementedintheRprogramminglanguage(withthefactanalfunction),OpenOpt,
andthestatisticalsoftwarepackageStata.RotationsareimplementedintheGPArotationRpackage.

Seealso
Designofexperiments
Formalconceptanalysis
Higherorderfactoranalysis

WikimediaCommonshas
mediarelatedtoFactor
analysis.

Independentcomponentanalysis
Nonnegativematrixfactorization
Perceptualmapping
Productmanagement
Qmethodology
Recommendationsystem
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

16/20

1/26/2015

Recommendationsystem

FactoranalysisWikipedia,thefreeencyclopedia

Varimaxrotation
GeneralizedStructuredComponentAnalysis
[28]==References==

1. ^Bartholomew,D.J.Steele,F.Galbraith,J.Moustaki,I.(2008).AnalysisofMultivariateSocialScienceData.
StatisticsintheSocialandBehavioralSciencesSeries(2nded.).Taylor&Francis.ISBN1584889608.
2. ^abcHarman,HarryH.(1976).ModernFactorAnalysis.UniversityofChicagoPress.pp.175,176.ISBN0
226316521.
3. ^abcdefghiPolitDFBeckCT(2012).NursingResearch:GeneratingandAssessingEvidenceforNursing
4. ^Meng,J.(2011)."UncovercooperativegeneregulationsbymicroRNAsandtranscriptionfactorsin
glioblastomausinganonnegativehybridfactormodel"
(http://www.cmsworldwide.com/ICASSP2011/Papers/ViewPapers.asp?PaperNum=4439).International
ConferenceonAcoustics,SpeechandSignalProcessing.
5. ^Liou,C.Y.Musicus,B.R.(2008)."CrossEntropyApproximationofStructuredGaussianCovariance
Matrices"(http://ieeexplore.ieee.org/xpl/articleDetails.jsp?
Processing56(7):33623367.doi:10.1109/TSP.2008.917878
(https://dx.doi.org/10.1109%2FTSP.2008.917878).
6. ^Bandalos,D.L.BoehmKaufman,M.R.(2008)."Fourcommonmisconceptionsinexploratoryfactoranalysis"
StatisticalandMethodologicalMythsandUrbanLegends:Doctrine,VerityandFableintheOrganizationaland
SocialSciences.Taylor&Francis.pp.6187.ISBN9780805862379.
7. ^Larsen,R.,&Warne,R.T.(2010).Estimatingconfidenceintervalsforeigenvaluesinexploratoryfactor
analysis.BehaviorResearchMethods,42,871876.doi:10.3758/BRM.42.3.871
8. ^abWarne,R.T.,&Larsen,R.(2014).EvaluatingaproposedmodificationoftheGuttmanrulefor
determiningthenumberoffactorsinanexploratoryfactoranalysis.PsychologicalTestandAssessment
Modeling,56,104123.
9. ^*Ledesma,R.D.ValeroMora,P.(2007)."DeterminingtheNumberofFactorstoRetaininEFA:Aneasyto
usecomputerprogramforcarryingoutParallelAnalysis"(http://pareonline.net/getvn.asp?v=12&n=2).Practical
AssessmentResearch&Evaluation12(2):111.
10. ^Velicer,W.F.(1976)."Determiningthenumberofcomponentsfromthematrixofpartialcorrelations".
Psychometrika41:321327.doi:10.1007/bf02293557(https://dx.doi.org/10.1007%2Fbf02293557).
11. ^Ruscio,JohnRoche,B.(2012)."Determiningthenumberoffactorstoretaininanexploratoryfactoranalysis
usingcomparisondataofknownfactorialstructure".PsychologicalAssessment24:282292.
doi:10.1037/a0025697(https://dx.doi.org/10.1037%2Fa0025697).
tomakemorejudiciousestimations.PracticalAssessment,ResearchandEvaluation,18(8).Availableonline:
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

17/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

tomakemorejudiciousestimations.PracticalAssessment,ResearchandEvaluation,18(8).Availableonline:
http://pareonline.net/getvn.asp?v=18&n=8

14. ^Russell,D.W.(December2002)."Insearchofunderlyingdimensions:Theuse(andabuse)offactoranalysis
inPersonalityandSocialPsychologyBulletin"(http://psp.sagepub.com/content/28/12/1629.short).Personality
andSocialPsychologyBulletin28(12):162946.doi:10.1177/014616702237645
(https://dx.doi.org/10.1177%2F014616702237645).
15. ^Sternberg,R.J.(1977).MetaphorsofMind:ConceptionsoftheNatureofIntelligence.NewYork:Cambridge
UniversityPress.pp.85111.
16. ^RichardB.Darlington(2004)"FactorAnalysis"(http://comp9.psych.cornell.edu/Darlington/factor.htm).
RetrievedJuly22,2004.
17. ^abFabrigaretal.(1999)."Evaluatingtheuseofexploratoryfactoranalysisinpsychologicalresearch."
(http://www.statpower.net/Content/312/Handout/Fabrigar1999.pdf).PsychologicalMethods.
18. ^abcdefSuhr,Diane(2009)."Principalcomponentanalysisvs.exploratoryfactoranalysis"
(http://www2.sas.com/proceedings/sugi30/20330.pdf).SUGI30Proceedings.Retrieved5April2012.
19. ^SASStatistics."PrincipalComponentsAnalysis"(http://support.sas.com/publishing/pubcat/chaps/55129.pdf).
SASSupportTextbook.
20. ^Meglen,R.R.(1991)."ExaminingLargeDatabases:AChemometricApproachUsingPrincipalComponent
Analysis".JournalofChemometrics5(3):163179.doi:10.1002/cem.1180050305/
(https://dx.doi.org/10.1002%2Fcem.1180050305%2F).
21. ^Brown,J.D.(January2009)."PrincipalcomponentsanalysisandexploratoryfactoranalysisDefinitions,
differencesandchoices."(http://jalt.org/test/PDF/Brown29.pdf).Shiken:JALTTesting&EvaluationSIG
22. ^Ritter,N.(2012).Acomparisonofdistributionfreeandnondistributionfreemethodsinfactoranalysis.Paper
presentedatSouthwesternEducationalResearchAssociation(SERA)Conference2012,NewOrleans,LA
(ED529153).
23. ^Subbarao,C.Subbarao,N.V.Chandu,S.N.(December1996)."Characterisationofgroundwater
contaminationusingfactoranalysis".EnvironmentalGeology28(4):175180.doi:10.1007/s002540050091
(https://dx.doi.org/10.1007%2Fs002540050091).
24. ^Love,D.Hallbauer,D.K.Amos,A.Hranova,R.K.(2004)."Factoranalysisasatoolingroundwaterquality
management:twosouthernAfricancasestudies".PhysicsandChemistryoftheEarth29:113543.
doi:10.1016/j.pce.2004.09.027(https://dx.doi.org/10.1016%2Fj.pce.2004.09.027).
25. ^Barton,E.S.Hallbauer,D.K.(1996)."TraceelementandUPbisotopecompositionsofpyritetypesinthe
ProterozoicBlackReef,TransvaalSequence,SouthAfrica:Implicationsongenesisandage".ChemicalGeology
133:173199.doi:10.1016/S00092541(96)000757(https://dx.doi.org/10.1016%2FS0009
2541%2896%29000757).
26. ^Hochreiter,SeppClevert,DjorkArnObermayer,Klaus(2006)."Anewsummarizationmethodfor
affymetrixprobeleveldata"(http://bioinformatics.oxfordjournals.org/content/22/8/943.full).Bioinformatics22
(8):9439.doi:10.1093/bioinformatics/btl033(https://dx.doi.org/10.1093%2Fbioinformatics%2Fbtl033).
PMID16473874(https://www.ncbi.nlm.nih.gov/pubmed/16473874).
27. ^MacCallum,Robert(June1983)."AcomparisonoffactoranalysisprogramsinSPSS,BMDP,andSAS".
Psychometrika48(48).doi:10.1007/BF02294017(https://dx.doi.org/10.1007%2FBF02294017).
28. ^Ishida,E.E.O&deSouza,R.S..Hubbleparameterreconstructionfromaprincipalcomponentanalysis:
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

18/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

minimizingthebias.Astronomy&Astrophysics,Volume527,id.A49(2011)

id=rQ2vdJgohH0C)(3rded.).ContinuumInternational.ISBN9780826480002.
Fabrigar,L.R.Wegener,D.T.MacCallum,R.C.Strahan,E.J.(September1999)."Evaluatingthe
useofexploratoryfactoranalysisinpsychologicalresearch"
(http://psycnet.apa.org/journals/met/4/3/272/).PsychologicalMethods4(3):272299.
doi:10.1037/1082989X.4.3.272(https://dx.doi.org/10.1037%2F1082989X.4.3.272).
Psychometrika,Vol.71,No.1,pp.173191,March2006.
Katz,JeffreyOwen,andRohlf,F.James.Primaryproductfunctionplane:Anobliquerotationtosimple
structure.MultivariateBehavioralResearch,April1975,Vol.10,pp.219232.
Katz,JeffreyOwen,andRohlf,F.James.Functionplane:Anewapproachtosimplestructurerotation.
Psychometrika,March1974,Vol.39,No.1,pp.3751.
Katz,JeffreyOwen,andRohlf,F.James.Functionpointclusteranalysis.SystematicZoology,September
1973,Vol.22,No.3,pp.295301.
Thompson,B.(2004).Exploratoryandconfirmatoryfactoranalysis:Understandingconceptsand
applications.WashingtonDC:AmericanPsychologicalAssociation.ISBN1591470935.

FactorAnalysis.RetrievedJuly23,2004,
RaymondCattell.RetrievedJuly22,2004,fromhttp://www.indiana.edu/~intell/rcattell.shtml
ExploratoryFactorAnalysisABookManuscriptbyTucker,L.&MacCallumR.(1993).Retrieved
June8,2006,from:http://www.unc.edu/~rcm/book/factornew.htm
Garson,G.David,"FactorAnalysis,"fromStatnotes:TopicsinMultivariateAnalysis.Retrievedon
April13,2009fromhttp://www2.chass.ncsu.edu/garson/pa765/statnote.htm
FactorAnalysisat100(http://www.fa100.info/index.html)conferencematerial
FARMSFactorAnalysisforRobustMicroarraySummarization,anRpackage
(http://www.bioinf.jku.at/software/farms/farms.html)software
Retrievedfrom"http://en.wikipedia.org/w/index.php?title=Factor_analysis&oldid=639737268"
Categories: Factoranalysis Psychometrics Multivariatestatistics Latentvariablemodels
https://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis

19/20

1/26/2015

FactoranalysisWikipedia,thefreeencyclopedia

Marketresearch Productmanagement Marketing Educationalpsychology

Thispagewaslastmodifiedon26December2014,at22:10.