Factor Analysis

19-2

What is factor analysis ?

Factor analysis is a general name denoting a class of Procedures primarily used for data reduction and summarization. Variables are not classified as either dependent or independent. Instead, the whole set of interdependent relationships among variables is examined in order to define a set of common dimensions called Factors.

19-3

Purpose of Factor Analysis

To identify underlying dimensions called Factors, that explain the correlations among a set of variables. -- lifestyle statements may be used to measure the psychographic profile of consumers.

To identify a new, smaller set of uncorrelated variables to replace the original set of correlated variables for subsequent analysis such as Regression or Discriminant Analysis. -- psychographic factors may be used as independent variables to explain the difference between loyal and non loyal customers.

19-4

Assumptions
Models are usually based on linear relationships Models assume that the data collected are interval scaled Multicollinearity in the data is desirable because the objective is to identify interrelated set of variables. The data should be amenable for factor analysis. It should not be such that a variable is only correlated with itself and no correlation exists with any other variables. This is like an Identity Matrix. Factor analysis cannot be done on such data.

19-5

An Example
A study conducted to determine customers perception and attributes of an airline. A set of 10 statements were constructed and respondents were asked to rate in a 7 point scale ( 1= completely agree, 7 = completely disagree ) Statements were as follows: 1. The Airline is always on time 2. The seats are very comfortable 3. I love the food they provide 4. Their air-hostesses are very courteous 5. My boss/friend flies with the same airline 6. The airlines have younger aircrafts 7. I get the advantage of a frequent flyer program 8. It suits my schedule 9. My mom feels safe when I fly in this airline 10. Flying by this airline compliments my lifestyle and social standing in the society

19-6

Example Contd..
Do the ten different statements indicate 10 different factors which influence a customer to fly by this airline ? OR Is there any correlations between these statements so that we can identify only a few factors such that some of these statements can be associated to these factors.

19-7

Factor Analysis basic ideas

Each of the statement indicated in the example is considered as a Variable. Hence for each respondent there will be a score against each variable. Ex: V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 respondent 1 2 2 4 3 5 3 5 7 6 2 We can attach suitable weights to each of the variable scores and a Weighted sum of these can be calculated. Ex: weight for V1 = 0.3 , weight for V2 = 0.1 etc Hence a score called Factor Score can be calculated as Factor Score ( Resp 1) = W1x2 + W2x2+ W3x4+w4x3+.

Similarly factor score can be calculated for each respondent. If there were 20 respondents, we would get a table containing 20 factor scores.

19-8

Factor Analysis basic ideas contd

The weights which are assigned to each of the variables are not taken arbitrarily but are chosen such that the variance in the factor scores obtained is the maximum. Once the first set of weights are obtained, a new set of weights are obtained so that the new set of factor scores shows the maximum variance but keeping in mind that these set of factor scores are uncorrelated with the first set of factor scores. This process is repeated till all the variance is explained by these factors. The first set of factor scores obtained is now correlated with the data for the variable 1 to 10 . This is called factor loadings. Thus factor loading is the correlation between the factor scores and the variables.

19-9

Factor Analysis basic ideas contd

An example would clarify what we have discussed so far. A file in excel data sheet can now be looked at to understand what we have just discussed.

The factors thus extracted are done using a technique called Principal Component Analysis.

19-10

Determining the number of factors

It is possible to extract as many factors as there are variables but the very purpose of factor analysis will be defeated and hence a smaller number of factors need to be found. Question is --- how many?

Several procedures are available: -- Determine based on Eigenvalues. An eigenvalue represents the amount of variance associated with the factor. Generally only factors with an Eigenvalue of >1.0 is included.

19-11

Determining the number of factors

Determination based on Scree Plot.

A scree plot is a plot of the eigenvalues against the number of factors. Typically, the plot has a distinct break with a gradual trailing off with the rest of the factors. This trailing off is referred to as Scree.

19-12

Scree Plot
3.0 2.5 Eigenvalue 2.0 1.5 1.0 0.5 0.0 1

3 4 5 Component Number

19-13

Determining the number of factors

Determination based on percentage of Variance.
The number of factors extracted is determined so that the cumulative percentage of variance reaches a satisfactory level. The amount of variance explained can vary with situation but above 60% is considered satisfactory.

19-14

How to check suitability for Factor Analysis

Kaiser-Meyer-Olkin ( KMO ) measure of sampling adequacy . This index compares the magnitude of observed correlation coefficients to the magnitude of partial correlation coefficients. Typically it should be > 0.5 is considered as good enough for conducting Factor analysis for the data under consideration. Bartlett test of sphericity : It is a test used to examine the hypothesis that the variables are uncorrelated in the population. If the hypothesis can be rejected then the data is suitable for factor analysis.

19-15

Conducting Factor Analysis

RESPONDENT NUMBER 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 V1 7.00 1.00 6.00 4.00 1.00 6.00 5.00 6.00 3.00 2.00 6.00 2.00 7.00 4.00 1.00 6.00 5.00 7.00 2.00 3.00 1.00 5.00 2.00 4.00 6.00 3.00 4.00 3.00 4.00 2.00 V2 3.00 3.00 2.00 5.00 2.00 3.00 3.00 4.00 4.00 6.00 4.00 3.00 2.00 6.00 3.00 4.00 3.00 3.00 4.00 5.00 3.00 4.00 2.00 6.00 5.00 5.00 4.00 7.00 6.00 3.00 V3 6.00 2.00 7.00 4.00 2.00 6.00 6.00 7.00 2.00 2.00 7.00 1.00 6.00 4.00 2.00 6.00 6.00 7.00 3.00 3.00 2.00 5.00 1.00 4.00 4.00 4.00 7.00 2.00 3.00 2.00 V4 4.00 4.00 4.00 6.00 3.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 4.00 5.00 2.00 3.00 3.00 4.00 3.00 6.00 3.00 4.00 5.00 6.00 2.00 6.00 2.00 6.00 7.00 4.00 V5 2.00 5.00 1.00 2.00 6.00 2.00 4.00 1.00 6.00 7.00 2.00 5.00 1.00 3.00 6.00 3.00 3.00 1.00 6.00 4.00 5.00 2.00 4.00 4.00 1.00 4.00 2.00 4.00 2.00 7.00 V6 4.00 4.00 3.00 5.00 2.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 3.00 6.00 4.00 4.00 4.00 4.00 3.00 6.00 3.00 4.00 4.00 7.00 4.00 7.00 5.00 3.00 7.00 2.00

19-16

Correlation Matrix

Variables V1 V2 V3 V4 V5 V6

V1 1.000 -0.530 0.873 -0.086 -0.858 0.004

V2 1.000 -0.155 0.572 0.020 0.640

V3

V4

V5

V6

1.000 -0.248 -0.778 -0.018

1.000 -0.007 0.640

1.000 -0.136

1.000

19-17

Results of Principal Components Analysis

Communalities
Variables V1 V2 V3 V4 V5 V6 Initial 1.000 1.000 1.000 1.000 1.000 1.000 Extraction 0.926 0.723 0.894 0.739 0.878 0.790

Initial Eigen values

Factor 1 2 3 4 5 6 Eigen value 2.731 2.218 0.442 0.341 0.183 0.085 % of variance 45.520 36.969 7.360 5.688 3.044 1.420 Cumulat. % 45.520 82.488 89.848 95.536 98.580 100.000

19-18

Results of Principal Components Analysis

Extraction Sums of Squared Loadings
Factor 1 2 Eigen value 2.731 2.218 % of variance 45.520 36.969 Cumulat. % 45.520 82.488

Factor Matrix
Variables V1 V2 V3 V4 V5 V6 Factor 1 0.928 -0.301 0.936 -0.342 -0.869 -0.177 Factor 2 0.253 0.795 0.131 0.789 -0.351 0.871

Conducting Factor Analysis

Rotate Factors

19-19

Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix is transformed into a simpler one that is easier to interpret. In rotating the factors, we would like each factor to have nonzero, or significant, loadings or coefficients for only some of the variables. Likewise, we would like each variable to have nonzero or significant loadings with only a few factors, if possible with only one. The rotation is called orthogonal rotation if the axes are maintained at right angles.

Conducting Factor Analysis

Rotate Factors

19-20

The most commonly used method for rotation is the varimax procedure. This is an orthogonal method of rotation that minimizes the number of variables with high loadings on a factor, thereby enhancing the interpretability of the factors. Orthogonal rotation results in factors that are uncorrelated. The rotation is called oblique rotation when the axes are not maintained at right angles, and the factors are correlated. Sometimes, allowing for correlations among factors can simplify the factor pattern matrix. Oblique rotation should be used when factors in the population are likely to be strongly correlated.

19-21

Results of Principal Components Analysis

Rotation Sums of Squared Loadings
Factor Eigenvalue % of variance 1 2.688 44.802 2 2.261 37.687 Cumulat. % 44.802 82.488

Rotated Factor Matrix

Variables V1 V2 V3 V4 V5 V6 Factor 1 0.962 -0.057 0.934 -0.098 -0.933 0.083 Factor 2 -0.027 0.848 -0.146 0.845 -0.084 0.885

Conducting Factor Analysis

Interpret Factors

19-22

A factor can then be interpreted in terms of the variables that load high on it.

Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.

19-23

Factor Loading Plot

Rotated Component Matrix Variable V1 V6 1 0.962 factor 2

Factor Plot in Rotated Space Factor 1 1.0 0.5 0.0 -0.5 V4

-2.66E-02

V2

V2
V3 V4 V5 Factor 2

-5.72E-02
0.934 -9.83E-02 -0.933

0.848
-0.146 0.854 -8.40E-02

V5

V3
V1

V6

8.337E-02 0.885

-1.0 -1.0 -0.5 0.0 0.5 1.0

19-24

A few examples

We can now take few examples with hypothetical data and run

factor analysis using SPSS package.

19-25