Developing Statistical Job-Evaluation Models PDF

HTDevelStaJobEvalModBook_10.
qxp 9/23/05 7:19 AM Page 1
H O W - T O S E R I E S
fo r t h e H R P ro f e s s i o n a l
Developing Statistical
Job-
Evaluation
Models
N. Elizabeth Fried, Ph.D., CCP
John H. Davis, Ph.D., CCP
HTDevelStaJobEvalModBook_10.qxp 9/23/05 7:19 AM Page C1
H O W - T O S E R I E S
fo r t h e H R P ro f e s s i o n a l
Developing Statistical
Job-
Evaluation
Models
N. Elizabeth Fried, Ph.D., CCP
John H. Davis, Ph.D., CCP
HTDevelStaJobEvalModBook_10.qxp 3/16/2010 7:37 AM Page C2
About WorldatWork
WorldatWork is the worlds leading not-for-profit professional association
dedicated to knowledge leadership in compensation, benefits and total rewards.
Founded in 1955, WorldatWork focuses on human resources disciplines
associated with attracting, retaining and motivating employees. Besides serving
as the membership association of the professions, the WorldatWork family
of organizations provides education, certification (Certified Compensation
Professional CCP, Certified Benefits Professional CBP and Global
Remuneration Professional GRP), publications, knowledge resources,
surveys, conferences, research and networking. WorldatWork Society of
Certified Professionals and Alliance for Work-Life Progress (AWLP) are
part of the WorldatWork family.
Any laws, regulations or other legal requirements noted in this publication are, to the best of the
publishers knowledge, accurate and current as of this books publishing date. WorldatWork is
providing this information with the understanding that WorldatWork is not engaged, directly or by
implication, in rendering legal, accounting or other related professional services. You are urged to
consult with an attorney, accountant or other qualified professional concerning your own specific
situation and any questions that you may have related to that.
This book is published by WorldatWork. The interpretations, conclusions and recommendations
in this book are those of the author and do not necessarily represent those of WorldatWork.
2004, 1995 WorldatWork.
ISBN 1-57963-013-8 (Spiral bound)
978-1-57963-294-6 (E-book)
No portion of this publication may be reproduced in any form without express written permission
from WorldatWork.
WorldatWork
14040 N. Northsight Blvd., Scottsdale, AZ 85260
480/951-9191 Fax 480/483-8352
www.worldatwork.org
Publishing Manager: Dan Cafaro
Graphic Design: Mark Anthony Muoz
Production Manager: Rebecca Williams Ficker
Staff Contributor: Andrea Ozias
HTDevelStaJobEvalModBook_10.qxp 9/23/05 7:19 AM Page 1
Table of Contents
Introduction ....................................................................3
Chapter 1: Define Compensation Strategy
and Job-Evaluation Objectives........................................5
Choosing an Emphasis: Market vs. Job Content ................................6
Chapter 2: The Nuts and Bolts

of Broad-Based Stock Option Plans................................9
Chapter 3: Develop Structured Questionnaire ............13
Chapter 4: Administer Structured Questionnaire........17
Chapter 5: Gather Data for Dependent Variable ........21
Chapter 6: Develop and Test
Multiple-Regression Model on Benchmark Jobs ........25
Statistical Criteria................................................................................26
Nonstatistical Criteria ........................................................................30
The Model-Building Process ..............................................................31
Chapter 7: Revise Questionnaire

and Gather Data from Remaining Jobs........................35
Chapter 8: Apply Final Model
to Entire Organization ..................................................37
Chapter 9: Get Management Approval ........................39
Developing Statistical Job-Evaluation Models | 1

2 | Developing Statistical Job-Evaluation Models

Introduction
This booklet assumes the reader has a general understanding of compensation principles and a variety
of statistical techniques. It provides basic guidelines to develop and install statistically derived
job-evaluation plans, and it also addresses philosophical, organizational, statistical and practical issues
that require attention throughout the design process.
he power and flexibility of todays personal computers are opening
T new doors for compensation professionals. High-speed hardware and

sophisticated software are both available and affordable, allowing easy
access to programs that generate statistical models for job-evaluation systems.
With only a few mouse clicks or simple commands, this technology cranks
out a single job grade in seconds, and all job grades can be completed in a
few minutes.
User-friendly computer screens with fancy graphics may lure profession-
als onto the bandwagon. However, these programs are complex software tools
that must be placed in skillful hands. They are not for every organization or
for compensation professionals who are unfamiliar with the interpretation
and use of sophisticated statistical methods.
Nevertheless, demands for instant information continue to pound at the
door. Complex assignments stretch already thin staffs to their limits. Another
worry is comparable worth and its potential impact on current methods of
valuing jobs. Its toehold in the public sector could expand and threaten the
legality of traditional private sector job-evaluation systems. To cope with these
pressures, efficient, bias-free, comprehensive plans are sought. Is a multiple-
regression system the answer? A statistically based technique such as multiple
regression may be an alternative, but it is not a panacea.
Statistically derived systems are exciting, mathematically elegant plans. If
used in the proper environment, they can be efficient, cost-effective tools.

Define Compensation Strategy

1
and Job-Evaluation Objectives
he first step in valuing jobs is to assess and define the compensation
T strategy and job-evaluation objectives. This process helps answer the fol-
lowing question: Does the organization want to emphasize the external
marker or its internal values?
Statistically derived plans typically use multiple regression, which requires
the selection of one dependent variable and several independent variables. The
dependent variable is the outcome or predicted value generated by a regression
formula. This predicted outcome may be a market rate or some type of inter-
nal value, such as a desired job grade. Choosing a dependent variable is a fun-
damental design consideration, and it may have legal and philosophical impli-
cations for the outcome and its acceptability.
The compensation professionals objective is to develop a job-worth
hierarchy. Because the outcome is known in advance (e.g., to match the market
or produce a desired job grade), it is possible to seek independent variables, or
compensable factors, that consistently will predict the preferred outcome.
Several plausible factors may be tested and tried until a regression formula is
achieved that most closely produces the desired outcome.
Choosing an Emphasis: Market vs. Job Content

Although statistically derived systems typically are categorized as job-content
plans, this can be misleading. In fact, an emphasis on either the market or
content is valid. The choice of emphasis depends on the organizations strategy
and job-evaluation objectives. Figure 1 compares and contrasts the characteris-
tics of both approaches.
For example, if market-average salary is selected as the dependent
variable, then market price ultimately will drive the system. All compensable
factors selected and tested to develop the regression model only will mirror the
desired outcome they will not control the outcome. This is perfectly accept-
able if the strategy is to align jobs closely with the market. However, in some
U.S. state or city governments and Canadian provinces, pay-equity laws may
preclude the selection of market pay as the dependent variable.

Despite the market emphasis of this approach, job content does come into
play while assessing those jobs for which there is no available market data. The
regression formula generated allows for the performance of a comprehensive,
sophisticated slotting procedure for all other nonbenchmark jobs. It mathemati-
cally produces estimated or predicted market rates for nonbenchmark jobs that
are equivalent to benchmark jobs with similar job-content profiles.
If an internal value is selected as the dependent variable, then the system
will maintain a job-content emphasis. For example, assume that management
reviews benchmark jobs and assigns them a desired job grade based on a
preliminary whole-job ranking. In that case, the desired job grade assigned by
management becomes the dependent variable that will drive the system.
The primary challenge is to select compensable factors whose regression
formula will produce a job-worth hierarchy as consistent as possible with
managements judgment of relative worth. Once an acceptable hierarchy is
achieved, the regression formula can be applied to nonbenchmark jobs. This
formula generates job grades for nonbenchmark jobs that are equivalent to
benchmark jobs with similar job characteristics.
FIGURE 1: PLAN CHARACTERISTICS

OF MARKET- VERSUS CONTENT-DEPENDENT VARIABLES
DEPENDENT
VARIABLE PLAN CHARACTERISTICS
MARKET CONTENT
M Produces a job-worth hierarchy that primarily reflects the external market

C Produces a job-worth hierarchy that primarily reflects an organizations internal values
M Reflects changes in the market each time the model is updated
M Promotes executive-level buy-in due to market sensitivity
C Promotes executive involvement in strategic valuing of jobs
M Requires annual regression-model updates to reflect changes in the market
M Alters internal relationships of the jobs as the market changes
M Requires sufficient market data points for statistical credibility of results
C Requires sufficient internal benchmark data points for statistical credibility of results
M May produce a job-worth hierarchy that is inconsistent with the internal values of the organization
C May produce a job-worth hierarchy that is not competitive with the market
M C Reflects equal pay for equal work (equal job characteristics)
C Reflects equal pay for work of equal value (comparable worth)
M Builds in market inequities (e.g., sex and race) that could influence pay model
C Allows for identification and removal of potential sex and race bias factors
C Does not consider external market in the development of the initial job-worth hierarchy
C Requires the existence of an actual or desired internal hierarchy
M C Eliminates the need to slot nonbenchmark jobs


The Nuts and Bolts

2
of Broad-Based
Stock Option Plans
ompensable factors are those aspects of jobs that an organization
C values and for which it is willing to pay. They commonly fall into
four basic categories: skill, effort, responsibility and working
conditions. These broad categories contain a number of variations. For
example, skill factors often encompass job knowledge, education, experience,
analytical ability, creativity or dexterity. Effort may include factors relating to
physical and mental exertion. Responsibility may consist of job requirements
relating to the impact of decisions as well as control of finances, material or
human resources. Working conditions relate to the environment in which the
work is performed.
While selecting compensable factors, first consider the overall order of
worth of benchmark jobs as well as past and present pay relationships both
within the organization and within its competitive labor market. This initial
review should help ascertain key factors that will predict differences in worth
among the benchmark jobs. For example, higher-level jobs may tend to re-
quire higher levels of education.
While selecting compensable factors,

involve management and other key
stakeholders to help identify political
and cultural concerns.
Compensable factors reflect the values of an organization and will serve

as independent variables during the multiple-regression analysis. These
factors assess an attribute or job requirement that differentiates the worth of
one job from another. While selecting compensable factors, involve
management and other key stakeholders to help identify political and
cultural concerns. Also, apply the following guidelines:

Choose factors that apply to all jobs and that yield differences in job
worth.
Select factors that will be relatively independent to reduce potential
redundancy with other compensable factors. For example, avoid selecting
two factors that typically go hand in hand, such as level of supervision
and total number of direct and indirect reports.
Eliminate factors that exist in all jobs to the same or in similar degrees.
For example, dont measure exposure to hazards for administrative jobs,
which typically have limited or no exposure to hazards.
Choose only those factors that are essential to differentiate levels of job
worth.
Verify that each factor can be measured realistically and reliably.
Eliminate factors that generate bias. Ensure that factors are neutral with
respect to protected classes such as sex, race or age.
Create factors that are simple, specific, concrete and clearly defined.


Develop Structured
3
Questionnaire
nce the compensable factors have been identified, it is necessary to
O find an efficient method to collect the data. A structured

questionnaire is one such method. Consider the following questions
and details while developing a questionnaire:
What is the appropriate number of levels for each compensable factor?
This will vary for each factor. For example, one factor may have only four
clearly distinguishable levels; another may have eight levels.
Will data from the questionnaire be entered manually, and how will the
data be coded for easy computer entry? If data entry will be manual, it is
critical that the forms design specialist and programmer coordinate their
efforts in advance. This precaution will prevent input delays and potential
errors.
Will data be scanned with an optical character reader? If so, how will the
questionnaire be formatted for easy data importing? Again, this is an
issue for the forms design specialist and programmer to resolve in
advance.
What criteria will be used to select benchmark jobs for the initial
development of a regression model? For example, the criticality of the job
or the job family to the organization, the number of levels in the job
family and/or the number of job incumbents in the position may be
considered. Moreover, to enhance the representativeness of the sample
and its subsequent credibility, consider the diversity of functions and the
presence of race- and gender-dominated jobs as well as geographic
locales. Finally, if market is selected as the dependent variable, consider
availability of market data as an additional criterion.
After these details have been addressed, it is time to draft the question-
naire. Include clear, comprehensive instructions, list the questions in a logical
order and then run the questionnaire through a readability software program.
Continue to modify the questionnaire until the appropriate reading level is
achieved for the group of jobs it covers. Because of the skills required to
develop a highly structured questionnaire, outside expertise may be required.

Test the questionnaire on selected jobs to assess its clarity and compre-
hensiveness. Adjust the questionnaire and have it formatted professionally so
it is visually appealing and easy to complete. Be conservative with the number
printed. This is an iterative process, and the questionnaire probably will go
through additional changes by the time the project is completed. See Figure 2
for a sample questionnaire page.
FIGURE 2: SAMPLE QUESTIONNAIRE ITEMS
EDUCATION
Select the one statement that indicates the minimum educational level or formal training required to perform the
duties of your job successfully. Your education may be different, but you should select the minimum level
of education that the job requires.
Choose the one best statement

A Less than a high school diploma
B High school or vocational school diploma
C High school diploma and some additional job-related courses
D Completed technical school certificate program (six to 18 months of training)
E Associate degree
JOB-RELATED EXPERIENCE
Select the one statement that indicates the minimum amount of job-related experience required to successfully
perform the duties of your job. Include both previous experience in a lower-level position and the time required for
on-the-job training.

A As much as three months
B More than three months and as much as one year
C More than one year and as much as three years
D More than three years and as much as five years
E More than five years and as much as seven years
F More than seven years
IMPACT OF DECISIONS
Select the one statement that completes the sentence I most frequently make decisions that directly impact

A The way I do my job.
B My entire work group.
C Other work groups or departments.
D Vendors or other outside contacts.
E Customers.
LEADERSHIP OF OTHERS
Select the one statement that indicates the responsibility for leadership of others that your job requires.

A This job does not require leadership of other employees.
B This job requires leadership of less experienced employees on an as-needed basis.
C This job requires coordinating and monitoring work for an assigned group of employees on an ongoing basis.


Administer Structured
4
Questionnaire
hile planning administration procedures for the questionnaire,
W consider time, budget and statistical constraints along with any

other organizational or political issues. Planning communication,
distribution and verification strategies are critical steps to ensure successful
data collection. Here are some key factors to consider:
Sample versus population. If the questionnaire will be administered to a
sample of the employee population in benchmark jobs, take steps to
avoid potential sampling errors. Determine what percentage of
incumbents in each job should complete the questionnaire to provide
adequate coverage. This percentage will depend on a variety of factors
such as the number of incumbents in the job and the uniformity of job
content. If the organization has multiple locations, consider any
implications for geographic differences that may affect a job and thus
impact sampling procedures.
Identify criteria to select which incumbents will complete the
questionnaire. Generally, select experienced, competent incumbents.
Develop acceptable ranges for experience and performance levels, and
select the participants accordingly.
Communication. Do not underestimate the value of communication.
Advance, consistent and reinforcing communication is the key to a
successful data-gathering effort. Despite the pilot testing of a structured
questionnaire, employees will continue to have questions. Make sure
someone is available and responsive to employee inquiries. This will help
improve the accuracy of data collection as well as identify snags and
allow adjustments to be made along the way.
Review process. Determine if there will be supervisory review of
incumbent questionnaires before they are returned. This decision will
depend on the culture and style of the organization. Generally, superviso-
ry review helps ensure that data are relatively accurate.

Internal validity checks. Ensuring that the statistical software has a

variety of internal validity checks will facilitate accurate data entry and
help identify inconsistent responses. For example, software should check
automatically for missing or inconsistent data. This is probably less of an
issue if questionnaires are scanned with an optical character reader. How-
ever, if data are entered manually, the program can be set with data
parameters that limit the types of data allowed in specified fields. This
helps catch on-the-spot data entry errors.
Despite the pilot testing of a structured

questionnaire, employees
will continue to have questions.
Make sure someone is available
and responsive to employee inquiries.
Also, arrange for logical internal validity checks. In many cases, if the
respondent answers one item within a defined range, it would be illogical to
respond to another item outside a specified range. For example, assume an
incumbent indicated no to a question concerning responsibility for super-
vising people. If the incumbent responded to another question about the type
of work supervised, there would be an obvious error. These logical internal
validity flags help assess the reliability of the data so that cleanup can be
performed efficiently.
Unfortunately, data cleanup typically is one of the most time-consuming
aspects of the project. Although it probably is impossible to eliminate
problems altogether, it is prudent to work with computer experts to help avert
potential opportunities for errors.


Gather Data
5
for Dependent Variable
ake sure a sufficient number of benchmark jobs is selected to meet
M statistical constraints. Generally, the minimum number of data

points or cases required for credibility in regression analysis are 10
per variable, including both independent and dependent variables. If seven
independent variables and one dependent variable are to be examined, it will
be necessary to collect data on at least 80 benchmark jobs (i.e., 80 cases of
data) to develop a credible model.
Benchmark jobs should cover the full range of the hierarchy and
represent all major job families in the organization to which the statistically
based system will apply.
Use one of the nonquantitative

job-content techniques such as
whole-job ranking or paired-comparison
ranking for this process.
If the emphasis of the analysis is on the market, it will be necessary to

gather survey data on the benchmark jobs. Market data can be collected at
any time during the process. Follow standard survey procedures for collecting
and analyzing market data. First, identify the organizations competitive
position (e.g., market average, 50th percentile, 75th percentile). Next, gather
data from reliable survey sources.
However, if the focus of the analysis is on job content rather than the
market, a different approach is needed. For example, it is possible to use
existing grades as dependent variables. If that is the case, meet with
management and other key stakeholders to ensure they agree with the current
hierarchy of benchmark jobs. If management determines that the existing

hierarchy requires adjustment, recalibrate the grades to its satisfaction. When

consensus is reached on the modified grade structure, use the modified grades
as the dependent variables.
Suppose there is no formal existing hierarchy, such as a grading system,
or that a fresh look at the current hierarchy is needed. In these cases,
management and key stakeholder opinions should be solicited to develop a
desired grade structure from scratch. Use one of the nonquantitative job-
content techniques such as whole-job ranking or paired-comparison ranking
for this process. When management achieves a satisfactory preliminary
hierarchy, use the new desired levels as the dependent variables. These may be
simple rankings or grade classifications.


Develop and Test

6
Multiple-Regression Model
on Benchmark Jobs
fter gathering, entering and checking dependent- and independent-
A variable data, it is time to conduct a multiple-regression analysis and

build the model. This analysis is an iterative process. It requires
testing a variety of combinations of compensable factors before a suitable
model is achieved. It requires logic, inquiry and analysis. Despite the mathe-
matics, the process of model-building often is more of an art than a science.
The model is the outcome of the statistical analysis. It is a mathe-
matical equation that predicts or estimates the average grade (or market rate)
for given levels of the compensable factors used in the model. An example for
predicting job grades is shown in Figure 3.
The first consideration for model development involves the types of data
that are used. Figure 4 uses an actual hierarchy of grades for the benchmark
jobs as its dependent variable. It displays the values for the dependent vari-
able (actual grades) and independent variables (levels of compensable
factors) for some of the jobs. This display represents the data input for the
model-building process.
The other aspect of model development involves the general parameters
that should be satisfied by the job-evaluation model. These are called criteria
input, and they come in two forms: statistical and nonstatistical.
Statistical Criteria
Following are descriptions of the most common statistical criteria:
Standard error of estimate (SEE). The SEE reflects the prediction hit
rate. Its value indicates how close on average the predicted grades (or
predicted market rates) are to the actual grades (or actual market rates).
The smaller the value of the SEE, the closer the predicted values are to the
actual values, and the more credibility the model has. Most of the time,
the predicted values will be within two times the SEE of the actual values.
If salary grade is the dependent variable, SEE values in the range of 0.5
grade to 1.0 grade usually are acceptable. Figure 5 shows a value of 0.685

FIGURE 3: EXAMPLE OF GRADE PREDICTION USING A MODEL
Mathematical Model
y' = a + b x + b x + b x + b x + b x + b x + b x
1 1 2 2 3 3 4 4 5 5 6 6 7 7
y' = predicted grade (dependent variable)
a = constant = 4.761
b = coefficient for level of education = +1.394

1
x = level of education (independent variable)

1
b = coefficient for level of experience = +0.704

2
x = level of experience (independent variable)

2
b = coefficient for level of physical effort = +0.479

3
x = level of physical effort (independent variable)

3
b = coefficient for level of working conditions = +0.468

4
x = level of working conditions (independent variable)

4
b = coefficient for level of guidance received = +0.422

5
x = level of guidance received (independent variable)

5
b = coefficient for level of complexity = +0.385

6
x = level of complexity (independent variable)

6
b = coefficient for level of impact of decisions = +0.141

7
x = level of impact of decisions (independent variable)

7
Interpretation of Coefficients: The coefficient for a given factor is the unique impact
of that factor on grade while holding all other factors constant. For example, each level of
education (x1) is worth 1.394 grades, holding all other factors constant. The coefficients
as well as the constant are derived during the multiple regression analysis.
Using the Model: To predict the grade of a job titled Mechanical Technician 2, substitute
the levels of compensable factors (shown in Figure 4, on page 12) into the equation
and calculate the predicted grade. Note that this particular model uses only seven of the
nine factors shown in Figure 4.
y' = 4.761 + (1.394 x 4) + (0.704 x 3) + (0.479 x 1) + (0.468 x 2) +

(0.422 x 2) + (0.385 x 2) + (0.141 x 1) = 6.097
The predicted grade is rounded to 6.
(rounded) as the SEE. This means that most of the time the predicted
grade will be within 1.37 grades (e.g., two times the SEE) of the desired
grade.

FIGURE 4: EXAMPLE OF DATA INPUT
COMPLEXITY
EXPERIENCE
LEADERSHIP
CONDITIONS
EDUCATION
PERSONAL
CONTACTS
IMPACT OF
DECISIONS
GUIDANCE
RECEIVED
PHYSICAL
WORKING
ACTUAL
TITLE
EFFORT
GRADE
WORK
Assembler 1 1 1 1 2 2 1 1 1 1 1
Assembler 2 1 2 2 2 2 1 1 1 1 2
Parts Clerk 2 3 2 2 3 2 1 2 1 4
Clerk Typist 3 3 1 1 2 2 2 2 1 4
Secretary 3 4 1 1 2 2 2 2 1 6
Senior Secretary 4 5 1 1 3 2 3 2 1 8
Executive Secretary 4 6 1 1 4 3 4 3 2 9
Customer Services
Representative 4 4 2 3 3 3 5 3 1 10
Customer Services
Specialist 4 5 2 3 4 4 5 3 1 11
Mechanical Technician 1 4 2 1 2 2 2 1 1 1 5
Mechanical Technician 2 4 3 1 2 2 2 1 1 1 6
Equipment Operator 4 4 1 2 3 3 1 1 1 8
Heavy Equipment
Operator 4 5 1 2 4 3 1 1 1 9
If market rate is the dependent variable, SEE values ranging from 6

percent to 12 percent of the benchmark jobs average market rates are rea-
sonable. If SEE values can be achieved within or below these recommend-
ed ranges, consider the analysis to be on the money.
Coefficient of determination (R2). R2 is the proportion of variation of
the dependent variable that is explained by the particular combination of
independent variables in the model. For example, if grade is the depen-
dent variable, R2 is the proportion of the variation of grades that can be
attributed to the particular combination of compensable factors. If, for
example, R2 had a value of 0.94, as in Figure 5, it can be said that the
combination of compensable factors in the model explains 94 percent of
the variation in grade.
R2 will have a value ranging from 0 to 1.0. The closer R2 is to 1.0, the
better the model will be. Typically, and fortunately for model-building, as

FIGURE 5: EXAMPLE OF STATISTICAL OUTPUT IN MODEL-BUILDING
DEPENDENT VARIABLE: GRADE 86 VALID CASES
COEFF OF DETERM: 0.944429 ESTIMATED CONSTANT TERM 4.76090
STANDARD ERR OF ESTIMATE: 0.684550

F PROB
189.373 0.0000
VARIABLE REGRESSION STANDARDIZED T PROB

COEFFICIENT COEFFICIENT
EDUCATION 1.393831 0.454216 9.62949 0.0000
EXPERIENCE 0.704155 0.340951 4.81492 0.0000
PHYSEFF 0.479427 0.101961 2.08360 0.0416
WORKCOND 0.468080 0.148134 3.03765 0.0036
GUIDANCE 0.421802 0.116158 1.95114 0.0559
COMPLEXITY 0.385102 0.112525 1.83265 0.0720
IMPACT 0.141014 0.067626 1.75918 0.0838
R2 increases, SEE decreases. To be acceptable for most job-evaluation

models, R2 should be at least 0.90.
F probability. This is an indicator of the overall goodness of the model.
Assume the regression model was developed using a sample of
benchmark jobs from a given company. Would the same relationship
between the dependent variable and the independent variables be true for
other jobs in the company that are not part of the sample? The answer
can be found using the F, which is a statistical value reflecting the degree
of goodness of the model. In Figure 5, the value of F is 189.373.
The main area of focus for model-building, however, is the F probability.
Generally, when a regression analysis is run, the computer printout also
will include the probability values for F. More confidence can be had in

models with lower F probabilities. For acceptability in job-evaluation

models, the F probability should be less than 0.05. Usually, it will be zero
to three decimal places (0.000). In Figure 5, the F probability is zero to
four decimal places (0.0000), suggesting a slightly stronger-than-expected
confidence level.
t probability. While the F looks at all factors at once, the t examines one
independent variable at a time, and it indicates the goodness of each
independent variable in the model. In Figure 5, the t values range from
1.759 to 9.629. A high absolute value of t signifies that the same
relationship between the dependent variable and the independent
variable would be true for all other jobs in the company that are not part
of the sample.
As in the F, the focus in model-building is on the probability of t, not the
t value. Statistically speaking, the t probability value is the probability of
having a nonzero coefficient when it really would be zero if all jobs were
included. The lower the t probability, the better. For acceptability in job-
evaluation models, the t probability should be less than 0.10 for each
factor. In Figure 5, t probability values range from 0.00 to 0.08.
Marginal coefficient of determination. In developing the model, the
addition of another variable sometimes adds only slightly more to R2.
The added factor may even be statistically significant by the t probability,
but it doesnt add very much more explanatory power to the model.
Factors that add at least 0.05 to R2 usually are keepers. Those that add
less than 0.01 are not. Any factor between these values represents a
judgment call. If the decision is made to exclude a factor, be sure to rerun
the regression without that factor because the formula derived from the
remaining factors probably will change.
Nonstatistical Criteria
Although sound statistical criteria are important, a model also must meet
nonstatistical criteria for acceptance. Following are descriptions of common
nonstatistical criteria:
Number of compensable factors. In any statistical model-building, the
best bet is to keep things simple. Hence, the fewer the number of factors,
the better. However, statistical purity sometimes must give way to other
nonstatistical influences to achieve credibility. These influences may
require the inclusion of more factors than are needed statistically. For

example, although education may be valued highly in an organization, it

may add little to the predictive power of the model when combined with
other more influential factors. Despite the limited statistical contribution
of the education factor, it may be retained simply because it adds
acceptance leverage. Generally, three to seven compensable factors in
a job-evaluation model are sufficient to satisfy the statistical and most of
the nonstatistical criteria.
Values of the coefficients. In the model shown in Figure 3, the
coefficient for the independent variable level of impact of decisions has
a value of 0.141. This means that, while holding the other factors
constant, each level of that factor changes the predicted grade by only
0.141. During the model-development process, it may be decided that
this is not enough of an influence on grade level to keep the factor.
Compensation considerations. Suppose there is a factor that appears
superfluous under statistical criteria. It still is possible that it should be
included in the model for compensation purposes. For example, suppose
a model has factors encompassing skill, responsibility and working
conditions, and it satisfies statistical criteria such as SEE, R2, F probability
and t probability. However, management may want the model to include
a factor that describes some aspect of effort, such as physical demands.
Remember, just because a model satisfies all statistical criteria, it doesnt
necessarily meet all compensation concerns. If a factor is added, the
model should be rerun to obtain the final coefficients.
Employee relations/culture considerations. The culture of an organi-
zation may require inclusion of a specific factor for credibility purposes.
Also, if the questionnaire process gathered considerable information and
took a great deal of employee time, management may feel obligated to
recognize employee efforts and make the model reflect as much of the
data gathered as possible.
The Model-Building Process

Figure 5 includes extracts from a typical computer output from a statistical
software package. This printout displays the statistics addressed here. While
developing a job-evaluation model, it is necessary to review such computer
outputs repeatedly. Remember, this process typically takes several iterations
before a model is developed that successfully meets statistical and nonsta-
tistical criteria and also is credible and acceptable.

It is necessary to consider statistical and nonstatistical criteria in concert

while developing a model. Select an appropriate statistical software package and
proceed with a standard model-building process. During this process, examine if
the data make sense for the statistical application. Check for linearity by
reviewing various two-way plots between the dependent variable and each of the
compensable factors. Figure 6 is an example of a two-way linear plot.
If the relationships are not linear, the variable may be dropped or retained
depending on the severity of the nonlinearity. Another option is to obtain
expert advice about how to increase the variables linearity through modifi-
cation processes such as scaling. Also, check for factor redundancy by
comparing two-way plots and correlations between the factors themselves.
There are a number of ways to build a multiple-regression model. One
technique called forward stepwise regression uses statistical criteria that
automatically determine which factors should be in the model. It also is
possible to select the order of the factors personally. The decision to do this
may be driven by strong organizational values or some other compensation
rationale. Enter the factors in the order desired to discover what models they
produce. Then examine the models against statistical measures of goodness
to assess their credibility.
There are two other statistical considerations that commonly show up on
most computer printouts: multicollinearity and beta weights. Outside
assistance may be
FIGURE 6: PLOT OF GRADE VERSUS EXPERIENCE LEVEL
needed to help
14 analyze these consid-
erations in detail.
12
Multicollinearity
10 measures factor
redundancy among
8
independent variables
GRADE
6
and has implications
for the stability of the
4 model. In Figure 5,
multicollinearity was
2
not a problem, so the
software did not
0 1 2 3 4 5 6 7
provide a warning
EXPERIENCE LEVEL statement.

The other statistical consideration is the use of standardized coefficients,

which also are known as beta weights. They are listed in Figure 5 in the
standardized coefficient column and range from about 0.07 (rounded) to
0.45. Beta weights provide clues about the degree of influence each
compensable factor has on the total model.
Model-building is an art, and developing a final candidate model
requires a combination of compensation expertise and statistical techniques.
Figure 7 summarizes some statistical guidelines that tend to produce
workable, credible job-evaluation systems.
FIGURE 7: GUIDELINES FOR BUILDING A STATISTICAL MODEL
Elements to consider Guidelines and

for model acceptability parameters
Use 10 benchmark jobs for every

variable used in the regression
analysis. For example, if there are
Number of data points five independent variables and one
(i.e., benchmark jobs) dependent variable meaning a
total of six variables are being used
include 60 benchmark jobs.
Coefficient of determination (R2) R2 should equal or exceed 0.90.
Independent variables Retain three to seven

(compensable factors) compensable factors.
Keep the factor if R2 increases by

0.05 or greater.
Marginal coefficient of
determination (used as a basis Consider keeping the factor if R2
for retaining compensable factors increases by more than 0.01 but less
than 0.05.
in the model)
Drop the factor if R2 increases by
less than 0.01.
Up to 0.5 grades is excellent. Up to

one grade is acceptable.
Standard error of estimate
(prediction hit rate) Less than 6 percent of market pay is
excellent. Twelve percent of market
pay is acceptable.
F probability p < 0.05 is acceptable.
t probability p < 0.10 is acceptable.


Revise Questionnaire
7
and Gather Data
from Remaining Jobs
fter developing a job-evaluation model that meets statistical and
A nonstatistical criteria, the questionnaire can be revised. During

the model-building process, factors were eliminated that were
ineffective predictors of job grade or market rate. Now, the questionnaire
can be streamlined to reflect only those factors that are critical to
predicting grade or market rate. However, from a cosmetic, logical or
organizational culture standpoint, it may be necessary to keep more
questions than really are necessary for the analysis. Seek the advice of the
job-evaluation committee and other stakeholders before exercising
judgment about which items to retain.

Apply Final Model

8
to Entire Organization
fter revising the questionnaire, it is time to collect data on all
A nonbenchmark jobs. Gather all the data and run it through the
model. Next, examine the hierarchy for a reality check. If the model
has been developed properly with benchmark jobs, the predicted grades or
market rates for the nonbenchmark jobs should mesh nicely into the
hierarchy of benchmark jobs.
Unfortunately, there usually are some jobs that appear ill-fitted to the
system. For example, certain factors in the model such as impact on
finances typically are present in high-level managerial benchmark jobs.
However, this factor may be only marginally characteristic of select high-level
professional benchmark jobs. Because of the nature of the statistical modeling
process, the predicted value of these high-level professional jobs typically will
be lower in the hierarchy than is logical or acceptable to the organization.
This phenomenon does not indicate that the entire model should be
thrown out. If only a few jobs are affected, it is possible to rely on profes-
sional judgment. It may be acceptable to value these jobs strategically and
assign them grades apart from the system.

Get Management Approval

9
fter developing the total job-worth hierarchy, present the results to key
A management for approval. Assuming there has been a representative

group of key managers on the job-evaluation committee, this process
will be similar to the implementation of other job-evaluation systems.
Perhaps the greatest challenge, however, will be communicating how the final
hierarchy was developed. Typically, management interest focuses on the
bottom line. Managers will want a simple, concrete explanation of how the
questionnaire translates to a job grade or market rate.
Remember, statistically derived systems are sophisticated and, therefore,
complex to communicate. They can be perceived as a black box to the
statistically uninitiated. Consequently, an explanation of these systems
requires exceptional communication skills. Keep explanations on a practical
level. Point out that the statistical approach is merely a tool to ensure a fair
assessment of the relative value of all jobs to each other.
Most management groups are not composed of highly technical or scien-
tific types who are familiar with statistical jargon. So, avoid losing the
audience with a long discourse that describes the elegance of the statistical-
modeling process. Instead, focus on the outcome. Point out how the system
helps establish relative equity throughout the organization, division or job
family.
If the management group is highly technical, be prepared to respond to
pointed questions concerning the processes used. It also may be helpful to
have outside expertise or someone in the organization with credible statistical
know-how to provide support during the presentation. Prepare for the
audience, identify potential areas of concern and have thoughtful, ready
responses to satisfy those questions.

Although statistically based job-evaluation systems offer ease of adminis-

tration, cost-effectiveness and mechanisms to examine potential pay discrimi-
nation, they contain inherent weaknesses such as a high degree of complexity
and subsequent communication challenges. These weaknesses and other
considerations may make such programs undesirable for many organizations.
Statistically based systems also require extremely knowledgeable profes-
sionals to guide their development and implementation. It is important for
professionals to assess all aspects carefully to make a wise choice for their
organizations.


Developing Statistical Job-Evaluation Models

This booklet outlines steps for using statistically derived job-evaluation models,
which can be used to construct a job-worth hierarchy that is cost effective and easy
to administer.
N. Elizabeth Fried, Ph.D., CCP, is President of N.E. Fried and Associates,

Inc., a consulting firm in Dublin, Ohio, that focuses on performance
management, employee development and executive coaching. Fried is
nationally recognized for her research studies on secretarial grading prac-
tices, compensation arrangements to hold key people during acquisi-
tions, mergers and divestitures (stay bonuses); and on-call (beeper) pay.
Fried, who has written two books, regularly contributes articles to professional publi-
cations and speaks nationally to business and professional groups. Fried has been a
guest on more than 175 radio programs and a dozen television interviews addressing
compensation and employment issues. She holds her doctorate from Ohio State
University and established her practice in 1983 after a career in private industry.
John H. Davis, Ph.D., CCP, is President of Davis Consulting and is a

nationally recognized consultant and teacher of reward systems and
compensation strategy. He has over 25 years of compensation practi-
tioner and consulting experience.
Davis serves on the faculty of the WorldatWork certification program and
is a popular speaker and workshop leader for compensation and human
resources associations across the country. He teaches statistics for clients, compensation
associations, and human resources associations.
Davis is the author of Salary Surveys and Antitrust: An Overview for the HR Professional,
Sound Compensation Practices: A Theoretical Foundation, and Make Smart Decisions Using
Statistics (and Excel). He has written various articles in the WorldatWork Journal,
Compensation & Benefits Review, and The Comp Doctor (on the Web site of the North
Texas Compensation Association). He is co-author of Salary Survey Fundamentals in
the WorldatWork 2003-2004 Survey Handbook & Directory.
Davis holds a Ph.D. in statistics from Ohio State University. He was granted the first
honorary life membership in the North Texas Compensation Association for his con-
tributions to the organization and to the profession.
14040 N. Northsight Blvd.

Scottsdale, AZ 85260
www.worldatwork.org

Developing Statistical Job-Evaluation Models PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Developing Statistical Job-Evaluation Models PDF

Загружено:

Авторское право:

Доступные форматы

HTDevelStaJobEvalModBook_10.

qxp 9/23/05 7:19 AM Page 1

Chapter 2: The Nuts and Bolts

Chapter 7: Revise Questionnaire

Developing Statistical Job-Evaluation Models | 1

2 | Developing Statistical Job-Evaluation Models

he power and flexibility of todays personal computers are opening

T new doors for compensation professionals. High-speed hardware and

4 | Developing Statistical Job-Evaluation Models

Define Compensation Strategy

he first step in valuing jobs is to assess and define the compensation

Choosing an Emphasis: Market vs. Job Content

6 | Developing Statistical Job-Evaluation Models

FIGURE 1: PLAN CHARACTERISTICS

M Produces a job-worth hierarchy that primarily reflects the external market

Developing Statistical Job-Evaluation Models | 7

8 | Developing Statistical Job-Evaluation Models

The Nuts and Bolts

ompensable factors are those aspects of jobs that an organization

While selecting compensable factors,

Compensable factors reflect the values of an organization and will serve

10 | Developing Statistical Job-Evaluation Models

Developing Statistical Job-Evaluation Models | 11

12 | Developing Statistical Job-Evaluation Models

nce the compensable factors have been identified, it is necessary to

O find an efficient method to collect the data. A structured

14 | Developing Statistical Job-Evaluation Models

FIGURE 2: SAMPLE QUESTIONNAIRE ITEMS

Choose the one best statement

Choose the one best statement

Choose the one best statement

Choose the one best statement

Developing Statistical Job-Evaluation Models | 15

16 | Developing Statistical Job-Evaluation Models

hile planning administration procedures for the questionnaire,

W consider time, budget and statistical constraints along with any

18 | Developing Statistical Job-Evaluation Models

Internal validity checks. Ensuring that the statistical software has a

Despite the pilot testing of a structured

Developing Statistical Job-Evaluation Models | 19

20 | Developing Statistical Job-Evaluation Models

ake sure a sufficient number of benchmark jobs is selected to meet

M statistical constraints. Generally, the minimum number of data

Use one of the nonquantitative

If the emphasis of the analysis is on the market, it will be necessary to

22 | Developing Statistical Job-Evaluation Models

hierarchy requires adjustment, recalibrate the grades to its satisfaction. When

Developing Statistical Job-Evaluation Models | 23

24 | Developing Statistical Job-Evaluation Models

Develop and Test

fter gathering, entering and checking dependent- and independent-

A variable data, it is time to conduct a multiple-regression analysis and

26 | Developing Statistical Job-Evaluation Models

FIGURE 3: EXAMPLE OF GRADE PREDICTION USING A MODEL

y' = predicted grade (dependent variable)

b = coefficient for level of education = +1.394

x = level of education (independent variable)

b = coefficient for level of experience = +0.704

x = level of experience (independent variable)

b = coefficient for level of physical effort = +0.479

x = level of physical effort (independent variable)

b = coefficient for level of working conditions = +0.468

x = level of working conditions (independent variable)