Influence Statistics, Outliers, and Collinearity Diagnostics

Standardized Residuals Residuals divided by their estimated standard errors (like t-statistics). Observations with values larger than t
2n ,n p '

in absolute value are considered outliers.

ri =

ei s 1 vii

s = MS Re sidual

vii i thh diagonal element of P

Studentized Residuals Similar to standardized residuals, except s ( i ) = MS Re sidual ( i ) is computed on the regression fit on the remaining n-1 cases. Observations with values larger than t
2n , n p ' 1

in absolute value are

considered outliers ri* = ei s ( i ) 1 vii s = MS Re sidual ( i ) vii i thh diagonal element of P

These are labelled as RSTUDENT by SAS.

Leverage Values (Hat Diag) Measure of how far an observation is from the others in terms of the levels of the independent variables (not the dependent variable). Observations with values larger than 2p/n are considered to be potentially highly influential.

vii i th diagonal element of P DFFITS Measure of how much an observation has effected its fitted value from the regression model. p' Values larger than 2 in absolute value are considered highly influential. n ei e2 (n p'1) s (2i ) = (n p ' ) s 2 i s 1 v 1 vii s ( i ) vii ii (i ) Note that DFFITSi measures the number of standard errors that the fitted value for the ith case has shifted when it was not used in the regression fit. DFFITS i = Y i Y i (i ) = vii 1 vii

^ ^

DFBETAS Measure of how much an observation has effected the estimate of a regression coefficient (there 2 is one DFBETA for each regression coefficient, including the intercept). Values larger than in absolute n value are considered highly influential.

DFBETAS j ( i ) =

j (i )

s ( i ) c jj

c jj ( j + 1) st diagonal element of ( X ' X ) 1

Note that DFBETASj(i) measures the number of standard errors that the regression coefficient for the jth predictor variable has shifted when the ith case was not used in the regression fit. Cooks D Measure of aggregate impact of each observation on the group of regression coefficients, as well as the group of fitted values. Values larger than F.50, p ',n p , are considered highly influential.

Di =

( ( i ) )' ( X ' X )( ( i ) ) p' s 2

ri 2 vii p' 1 vii

^ ^ ^ ^ Y (i ) Y Y (i ) Y 2 p' s

'

Note that Di is like an F-statistic used for testing K ' = m where K=I. Can also be thought of as the shift in a 100(1-)100% confidence ellipse when Di = Fa , p ',n p ' when the ith case is not used to fit the regression model. COVRATIO Measure of the impact of each observation on the variances (and standard errors) of the 3 p' regression coefficients and their covariances. Values outside interval 1 considered highly influential. n det( s (2i ) ( X (' i ) X ( i ) ) 1 ) COVRATIOi = det( s 2 ( X ' X ) 1 ) Variance Inflation Factor (VIF) Measure of how highly correlated each independent variable is with the other predictors in the model. Values larger than 10 for a predictor imply large inflation of standard errors of regression coefficients due to this variable being in model.

1 2 2 where Rk is the coefficient of multiple determination when Xk is regressed on the p-1 remaining 1 Rk independent variables. VIFk =

Obtaining Influence Statistics and Studentized Residuals in SPSS

Choose ANALYZE, REGRESSION, LINEAR, and input the Dependent variable and set of Independent variables from your model of interest (possibly having been chosen via an automated model selection method). B. Under STATISTICS, select Collinearity Diagnostics, Casewise Diagnostics and All Cases and CONTINUE Under PLOTS, select Y:*SRESID and X:*ZPRED. Also choose HISTOGRAM. These give a plot of studentized residuals versus standardized predicted values, and a histogram of standardized residuals (residual/sqrt(MSE)). Select CONTINUE. C. Under SAVE, select Studentized Residuals, Cooks, Leverage Values, Covariance Ratio, Standardized DFBETAS, Standardized DFFITS. Select CONTINUE. The results will be added to your original data worksheet.
A.

Obtaining Influence Statistics and Studentized Residuals in SAS

PROC REG; MODEL Y = X1 X2 XP / R INFLUENCE VIF; RUN;

Obtaining Influence Statistics and Studentized Residuals in SAS

reg1.reg <- lm(y ~ x1 + + xp) reg1.rstudent <- rstudent(reg1.reg) reg1.inf <- influence.measures(reg1.reg)