Human Resource Management Review 16 (2006) 310 – 323

www.socscinet.com/bam/humres

Innovations in job analysis: Development and application

of metrics to analyze job data
Lauren E. McEntire, Lesley R. Dailey, Holly K. Osburn, Michael D. Mumford ⁎
Department of Psychology, University of Oklahoma, 455 W. Lindsey, Room 705, Norman, OK 73019, United States

Abstract

Job analysis is an integral part of any human resource function. Recent advancements in technology and changing worker
environments have drastically altered the means by which job analysis data are collected and stored. These changes have led to an
increase in the amount of data that is collected and the potential for the data to inform complex decision making. However, due to a
lack of tools available for configuring and analyzing data, human resource professionals are often unable to keep themselves
abreast of changes in their workforce, make complex decisions using job data, and facilitate communication across jobs, job
families or departments in their organization. As a result, advanced methods for analysis of job data are needed. Metrics are
quantitative algorithms applied to job data that aid in decision making in areas such as recruitment, selection, transferability,
promotion, training, and development. Metrics are a sophisticated, user-friendly approach to analyzing job data that have the
potential to meet the needs of human resource professionals in today's dynamic workplace. The development of metrics, their
application and benefit to human resource professionals, and their use of O⁎NET are discussed.
© 2006 Elsevier Inc. All rights reserved.

Keywords: Job analysis; Metrics; Job data

Job analysis involves the collection of various types of job data and worker requirements in an organization (Harvey,
1991; McCormick, 1976). The information obtained from a job analysis serves as a foundation for most, if not all,
human resource (HR) related activities and issues (Dessler, 2003; Jeanneret & Strong, 2003; Riggio, 1990). In fact, job
analysis is important to an organization because it serves as a strategic human resource management (HRM) practice
through which organizations gain a better understanding of their jobs and their workers (Schneider & Konz, 1989;
Siddique, 2004).
There are a number of ways that job analysis information can be applied to HRM functions (Harvey, 1991). In its
most generic form, job analysis data provides a point of reference to the user (i.e., human resource professional, job
incumbent or manager). However, the potential of job analysis data to aid in organizational human resource functions
extends far beyond the scope of its current use (Sanchez, 1994; Walker, 1990). There are a vast range of possibilities for
application and use in human resource decision making if the methods used for analyzing job analysis data can be
refined and updated.

⁎ Corresponding author. Tel.: +1 405 325 5583; fax: +1 405 325 4737.
E-mail address: mmumford@ou.edu (M.D. Mumford).

1053-4822/$ - see front matter © 2006 Elsevier Inc. All rights reserved.
doi:10.1016/j.hrmr.2006.05.004
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 311

Human resource decisions using job analysis data often rely on loose inferences or qualitative hunches rather than
using the data to its full potential (i.e., in dynamic and complex ways to draw objective conclusions) (Chermack, 2003;
Korte, 2003; Murphy & Zandvakili, 2000; Sanchez & Levine, 2000). This tendency to ineffectively apply and make
decisions using job analysis data are often due to the unavailability of simple means and tools for data configuration
and analysis (Harvey, 1991). The issues resulting from this deficiency in tools for human resource professionals are
three-fold. First, human resource professionals are not able to keep their job analysis outcomes current with respect to
the status of their human capital, changes in organizational needs, and advancements in technology and equipment. In
other words, decisions made using the data are not a reflection of the current state of their workforce because it is not
analyzed and applied quicker than job changes are occurring. Second, job analysis data are highly underutilized
especially in the context of complex decision making issues. Job analysis is often only used for simple, routine decision
making or administrative reports. Third, job analysis decisions are not portable across jobs, job families, departments or
organizations. Inferences and decisions are often only applicable to one job or a set of individuals and are not flexible
for further interpretation.
Therefore, the purpose of this article is to present a method for configuring and analyzing large amounts of job
analysis data that can address these issues and ultimately aid in human resource decision making. This method provides
a quantitative approach for deriving value from job analysis data that is not only less cumbersome than traditional
methods, but also holds the potential to reduce the monetary costs that are often associated with job analysis. In
addition and prior to the presentation of metrics, we would like to discuss the three aforementioned issues related to the
use of job analysis data in more detail as well as the utility of the Occupational Information Network (O⁎NET) when
collecting and analyzing job data.

Issue 1: Workforce and technological trends affecting job analysis

Today's professional and industrial work settings have evolved in recent years to become fast paced, dynamic
environments filled with technology and innovation (Cascio, 1995; Walker, 1990). Work environments and worker
requirements have transformed drastically in the last few decades to keep up with these trends and they are expected to
continue in this direction (McCann, 2004; Schneider & Konz, 1989; Walker, 1999). Some contributing factors to this work
evolution are movements toward broad and cross-functional job responsibilities (Cunningham, 1996), dissolving
manager–labor distinctions between professional workers, regular shifts in work responsibilities, global competition, free
trade, and prevalence of teamwork and self-managing teams (Sanchez, 1994). Additionally, organizations are using
automation in virtually every administrative and human resource activity to increase speed of performance, garner more
information, and reduce human error (Buckley, Minette, Joy, & Michaels, 2004; McDonald & Cornetto, 2005).
Recent technological advancements in the methods for collection and storage of job data have embraced these
changes in the nature of work. (Crespin & Austin, 2002). Computer applications and software are now prevailing
methods for collecting job analysis data. In fact, many organizations have completely eliminated the use of interviews
and paper–pencil methods (Patterson & Lindsey, 2003; Sanchez, 1994). Computerized methods for collecting job
analysis data are often transferred to an online, interactive format where all data are collected on the computer using the
internet. As a result of these technological advances, job analysis data can be collected, stored, maintained, and updated
with relative ease.
Though the dynamic changes in the workforce can be documented with the technology available for job data
collection, if the tools needed for human resource professionals to analyze the data are not present or utilized then the
meaningful results and decision making opportunities may be forfeited. It is possible that the job information acquired
using more time consuming, traditional methods of analysis will be obsolete because the job data are already in need of
an update.

Issue 2: Underutilization of job analysis data for decision making

While there are clear advantages to collecting job analysis data using computers and the internet, this method has
created new issues for human resource practitioners to address. One issue that was not applicable when paper-pencil
methods for collecting job data were in place is the sheer amount of data that can be collected (Patterson & Lindsey, 2003).
Online formats allow for a much larger dataset than has been possible in the past. Because of this, human resource
professionals must be quite specific in the ways they intend to use the data a priori. This specificity allows for the creation
312 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323

of interrelationships among the data during the collection process, thereby allowing it to be organized in meaningful ways
for the user. Otherwise, it is possible to obtain a dataset that is so large and unorganized that it is difficult to analyze.
Even with an organized data collection and linkage process, databases used to store job analysis data are typically
very large. These large databases prove difficult for users to efficiently apply the data to its greatest capability,
especially when users lack the necessary skills and procedural steps needed to conduct data analysis. Consequently,
analysis of job data has become increasingly time consuming, disorganized and ineffective. The significant amount of
data makes commonly used qualitative approaches difficult to apply effectively. It is conceivable that the decisions
made using qualitative analyses may lack the complexity and influence necessary to justify the resources spent on the
analysis. Though we do not discount the use of qualitative methods, other options for effectively analyzing workforce
data to make complex decisions using quantitative methods are needed.

Issue 3: Job analysis data portability and usefulness of O⁎NET

Human resource professionals often lack the resources in their job data and data analysis techniques to combine data
and make decisions at multiple levels. Job analysis data has the potential to be analyzed so that information can be
acquired about several jobs or job families and consequently multi-level decisions can be made. However, the data
must be structured to accommodate such an analysis. The job data must have the capability for aggregation. This is
especially true when a job analysis is needed across several jobs and job families as is the case in many large
organizations including government and military. Without a method to aggregate the data, decisions made using the
data across levels will not be valid (Harvey & Wilson, 2000).
For example, a human resource professional should not quantitatively compare the skills used on two different jobs if
the skills for each job are not from the same skill taxonomy. Occupational specific skills cannot be compared without a
structured link between the two skillsets; therefore inferences cannot be made about the two jobs in terms of skills (e.g.,
similarity, complexity). A solution to this problem of aggregation can be attained through the use of an online database
called O⁎NET. A significant part of the development and application of metrics uses standardized data from O⁎NET.
The Occupational Information Network replaced the Dictionary of Occupational Titles (U.S. Department of Labor,
1991) as a standardized, comprehensive, and online system available for performing worker and job oriented job analysis
and developing descriptions of jobs. This system can be used as a starting point to describe most any job in the
organizations of today and in the future (Mariani, 1999, 2001). The system is designed to present job information at both a
general and an occupation-specific level (Peterson, Mumford, Borman, Jeanneret, & Fleishman, 1999; Peterson et al.,
2001).
There are three key advantages to using O⁎NET based job analysis data in the development of data analysis techniques.
The first advantage is its use of multiple windows into the world of work. The “multiple windows” approach considers the
fact that jobs can be described and understood from a variety of different perspectives (Mumford & Peterson, 1999). That
is, one approach might be to describe a job in terms of the abilities necessary to do the work, while another approach might
be to describe the job in terms of the context in which the job occurs.
The second advantage of using O⁎NET as a framework for the development of data analysis techniques is its use of
cross-job descriptors which provide a common language to describe and compare jobs. In other words, these
descriptors portray the job in terms of broader variables that cut across occupations. They cut across occupations
because O⁎NET skills, abilities, Generalized Work Activities (GWAs), and knowledges are standardized taxonomies
that relate to all types of jobs. This particular feature makes the O⁎NET system highly efficient because new descriptive
systems do not have to be developed for each new job.
The third advantage of using O⁎NET is its use of a hierarchical organization of the variables used as descriptor
domains. In other words, the O⁎NET structure includes more general descriptors that can be seen as subsuming the
more specific descriptors (Mumford & Peterson, 1999). This hierarchical structure is central to the development of data
analysis techniques because it allows for the combination of data. Though each element (i.e., skills, abilities, GWAs,
knowledges) is unique, all elements can be aggregated for analysis because they come from standardized taxonomies
and are meaningful when related to each other. For example, users can combine broad descriptors, like GWAs with
more specific descriptors, like basic and cross-functional skills. Though these elements are different in the level of their
description, they are both still standardized elements of the job and hence can be compared with other related jobs (as
related jobs will have these same elements). This means of aggregation expands the capabilities previously available in
analyzing job data.
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 313

In sum, O⁎NET uses standardized work activities and skill descriptions to provide a common language to describe
and characterize various jobs and occupations (Borman, 1996). The framework provided by O⁎NET is highly
efficient because it provides a common point of reference when describing and comparing jobs, and thus eliminates
the need to develop new descriptive systems when describing new jobs (Peterson et al., 2001). In addition, this
standardized framework makes it possible to aggregate data not only within a job, but also between jobs in an
occupational family. By providing a way to aggregate data, the O⁎NET framework allows for the creation of data
management formulas. These formulas, often called metrics, can be used with job analysis data to answer questions
about the workforce.
The introduction of O⁎NET and various technological advances have led to the possibility for increased number of
applications for job analysis data. Subsequently, human resource professionals are in need of a means to analyze data
that is progressive enough to address dynamic and demanding work environments and advancements in technology as
well as allow portable decisions across levels, yet simple enough to be used every day to help make decisions with ease.
The remainder of this article will discuss a method for using data collected in the job analysis process for human
resource decision making. In particular, metrics are described in terms of their development, future validation, and their
benefit to numerous areas of HRM.

4. Metrics

Metrics use quantitative and qualitative job analysis data and apply step-by-step mathematical formulas, or algorithms,
to answer workforce analysis questions. These step-by-step formulas allow for the transformation of job analysis data into
more manageable and easily interpretable components, thus providing a more economical and effective way of using job
analysis data. The use of metrics can mesh data analysis and decision making to aid those making personnel related
decisions applicable to the workforce, job, or employee of interest.
For example, metrics can be employed that will help the organization determine the most qualified individuals for
promotion or to help determine the skills that an employee needs to be successful on the job. The nature of the
algorithms employed and the type of data used in metrics allow for more objectivity in answering workforce
questions, which ultimately leads to more reliable conclusions. To ensure reliability and proper aggregation of data,
metrics use O⁎NET and typical organizational job analysis information such as worker and job oriented data. O⁎NET
elements are aggregated together, weighted accordingly, and analyzed together to draw the most meaningful
conclusion.
The metrics discussed in this article are the result of a collaborative research effort in a contract with the Department
of Defense, SkillsNET® Corporation and the University of Oklahoma. The goal of this effort was to produce a user-
friendly, innovative and quantitative means to address a variety of HR issues in the Navy. Three specific problem areas
were identified as the primary focus points for the development of metrics. These areas included 1) metrics to organize
and analyze basic job data, 2) metrics to aid in recruitment, selection, and transferability related decisions and 3)
metrics to aid in training related decisions.
Several metrics were developed to address each area in as many meaningful ways as possible. Additional metrics
were developed that addressed areas such as using complex job data and military mission data. All elements of the most
recent Navy job analysis, O⁎NET, and some external data sources were used in the metric development and testing
process. All the metrics were developed, tested, and will be validated by trained Industrial/Organizational
psychologists and statisticians. No metric has received thorough validation at this stage in the development effort. The
next section describes the metric development process, followed by a detailed illustration. For a full list of developed
metrics see Appendix A.

4.1. Metric development process

The development of any metric involves a systematic process that consists of a series of key steps. In this section an
overview of the steps involved in metric development will be presented. This overview will provide a general view of
the issues one must consider and address when creating a metric. To provide more insight into the development process
two examples are provided in the next section. These examples are provided in a detailed manner to illustrate the
development of a metric from conception to current version. Additionally, Fig. 1 may be referenced as a means to
visually represent the metric development process prior to validation.
314 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323

Fig. 1. Metric development process.

The first step in metric development is to determine what workforce analysis issues are of interest to the
organization. Typically these questions are developed by human resource professionals in an organization through
discussion of the areas in which an analysis of job data would create the most benefit. Once the question(s) of interest
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 315

are generated, a rationale for the importance of each question to the organization is developed. This rationale helps to
clarify the workforce analysis question and to identify what specific, measurable benefit will arise from answering each
question. Components of the rationale include how the information will be used in an organizational setting, how it will
contribute to management strategy, and the added value this particular workforce information could bring to the
organization. This identification of workforce questions and the impact of the questions on the organization can initiate
a more in depth analysis of the area of interest. This analysis allows the organization to more effectively assess and
target the areas that could benefit from additional examination and can provide the foundation for the next step in the
process, the development of algorithm components to be used in the metric.
There are three steps involved in determining the components of the metric algorithm. First, a list of key elements is
developed. These key elements are broad in scope and are essentially the basic ingredients of the metric. The key
elements represent components that will contribute to answering the workforce analysis question. Second, the list of
key elements is shortened so that only the elements deemed most relevant to the workforce analysis question are
included. Third, a list of potential sources is identified. A source is a piece of information or set of data that can populate
a key element. For example, it may be determined that a key element is a list of abilities, and source options for
populating this key element are O⁎Net abilities, occupation specific abilities, or technical abilities. Sources are
identified by researching available data options relevant to the key element.
Once a list of potential sources has been developed, an examination of these sources is needed to establish which will be
kept and which will be eliminated. At this point, users ascertain what specific pieces of data are most appropriate to be used
as a source. This is typically obtained from job analysis information. For example, users would determine whether an
existing taxonomy such as O⁎NET is available that could supply the needed information, whether a new scale needs to be
created and collected from job experts or other relevant personnel, or whether information could be taken from other data
sources, such as the Department of Labor and/or Occupational Outlook Handbook. Upon determining what types of data
can be used for sources, it is necessary to determine the feasibility of obtaining this data. This is where the majority of
unused sources are eliminated. For example, some sources may be appropriate for populating the key element but are not
obtainable for use (i.e., limited access to information, copyrighted information, information that has yet to be collected in
job analysis). Therefore, the final decision as to what sources will populate the metric is based on a combination of
feasibility and necessity of including certain data.
Steps six and seven in the process create an algorithm that combines the identified pieces of data in a way that will
answer the target question. For the metrics described, statisticians developed various ways to combine metric data,
specifically taking into account issues such as standardization, weighting and cut scores. These approaches were then
evaluated, sometimes combined, and eventually configured into an initial formula. In many cases, the algorithms
consist of procedural steps and equations with varying levels of complexity.
After an algorithm is created, the eighth and last step in development prior to validation involves testing the
algorithm using sample or normative data. Industrial/organizational psychologists should test the metric to determine
whether the theoretical approach makes sense. During this phase of the process, flaws in theory may be discovered. For
instance, it might be discovered that a normal distribution cannot be assumed for data on a particular metric. This is
possible if the workforce analysis question targets a specialized group of workers whose technical expertise and ability
is very similar or not likely to vary. The data may be specific enough to a particular job or job family so as not to require
the assumption of normality. This is often the case with the use of job data that is unique to a particular job or
organization.
In addition, a review of the initial output by persons familiar with the job to determine whether the output makes
logical sense. It may be determined through this evaluation process that the metric formula needs to be modified to
better accommodate the data or more appropriately answer the workforce analysis question of interest. For example,
suppose an algorithm was created to determine the important tools required for a job and was applied using data from a
group of meteorologists. Examination of the output revealed that the most important tool was “weather balloon” while
the least important tool was “weather radar system”. After reviewing the output, job experts indicate that this is an
incorrect representation and in fact the tool identified as least important is much more important than the tool identified
as most important. This examination of the algorithm and output revealed that the algorithm applied needs to be
reevaluated and revised.
After the algorithm has been tested and revised, another test using job data is needed. This data can be related to the
set used to test the algorithm the first time or it can be from a different job. This process continues until the algorithm
consistently produces logical and reliable output in that job experts are able to verify the output of the metric with their
316 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323

expertise on the job. It may even be necessary to modify other steps in the metric development (e.g., key elements,
sources, or combination of elements) to obtain results that target the workforce analysis question of interest (see Fig. 1).
It is important to note that this process is not validation of the metric; it is only an initial check to determine if the metric
is producing information aligned with its purpose. Full validation of any metric must be completed before it can be
applied for decision making.

4.2. Metric development summary

In the previous section, a comprehensive explanation of the metric development process was presented. The
development of the metric Essential O⁎NET Skill Requirement for the Job is described in this section to further
illustrate the metric development process. This description can be supplemented using the examples in Fig. 1.

4.2.1. Metric Example: Essential O⁎NET skill requirements for the job
For every job there are necessary skills a worker must possess in order to perform effectively. Some of these skills
are more critical, or more essential, for the job than other skills. Typical job analysis data provides practitioners with
information about the skills used on the job, however, many times this set of skills is not bifurcated into those skills that
are the most important for employee to possess versus the skills that are not critical to the performance of the job. As
such, the main workforce analysis question that needs to be addressed (i.e., a metric needs to be developed) is “What
are the essential skills needed to perform the job?”
Identification of the essential skills for a job can be applied in areas such as selection and training/development. For
example, understanding those skills that are essential to performing the job can aid practitioners in developing selection
tools (e.g., structured interviews, simulations, skill proficiency tests) that ensure qualified individuals are selected. Using
the output from this metric, selection tools can be developed to focus on those skills that an employee must have in order
to be selected into the job. Those skills identified as essential in the output of this metric can provide a foundation upon
which employers develop interviews. In addition, employers can use the information from the output of this metric to aid
in determining which skills should be focused on in the interviewing stage of selection and which skills should be
assessed through other means, such as tests. This can be accomplished by identification of those skills from the metric
output that are best assessed through interviewing and those skills that are best assessed with other methods.
This metric can also be used in conjunction with performance data on existing employees to determine the areas
(skills) in which an individual needs training and/or development. Such performance data can be judged against those
skills identified as essential to the job. The information revealed can help to identify those skills in which the individual
needs development, thus allowing for tailored training programs to maximize the training effort.
In order to determine the ‘essential skills needed for the job’, a list of key elements was first identified. The first
element needed to answer this question is a list of skills. The next element needed is an importance level of the skill to
the job. The final element needed is the frequency with which each skill is performed on the job. These three elements
were selected because they were highly related to the workforce question of interest in that a combination of these
elements have the potential to facilitate the desired applications to selection, training, promotion and other human
resource areas described earlier. These elements were selected from a list that included other possible key elements
such as skill performance difficulty, skill learning difficulty, and skill quality/utility.
Having determined these key elements, a list of sources to populate them was established. Sources are options for
how the user/developer will operationalize the key element. There are multiple sources for every key element.
However, only a few will be feasible to obtain and appropriate for the metric under development. Source options
considered for the first key element include the O⁎NET skill taxonomy, Bloom's Skill Taxonomy (Bloom, Engelhart,
Furst, Hill, & Krathwohl, 1956), and occupational specific skills (i.e., skills that are not standardized across
occupations but that are unique to the occupation or job being considered). Any of these options could provide some of
the necessary information to populate the key element, a list of skills. However, the user should be primarily interested
in the source that provides the most coverage in answering the workforce analysis question of interest.
The O⁎NET skill taxonomy is potentially useful because it consists of 46 skills which can be divided into 2 lists: basic
skills and cross-functional skills. Basic skills are skills that lead to the acquisition of other skills. Examples of basic skills
include “writing” and “mathematics”. Cross-functional skills are performance relevant skills likely to be used on a
variety of jobs. Examples of cross-functional skills include “instructing” and “troubleshooting”. Another important
characteristic of using O⁎NET skills is that it is a standardized set of skills that can be used for comparisons across jobs.
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 317

The taxonomy of educational objectives, or Bloom's taxonomy, is another taxonomy considered as a source for this
key element. This taxonomy was developed primarily for classifying, identifying, and creating test items for
educational objectives (Krathwohl, 2002). In its revised form Bloom's taxonomy consists of 2 dimensions: cognitive
and knowledge. Examples of cognitive skills include “application” “evaluation” and “creation”. Examples of
knowledge skills include “factually-based skills” and “conceptually-based skills”. Though this taxonomy is not job-
centered or directly linked to occupational work, it still may be useful for identifying broader, non-technical skills.
Another classification of skills considered was occupational specific skills. Occupational specific skills are a
grouping of tasks that also include other job information like equipment and knowledges used on the job. This group of
tasks is given a representative label that identifies the information within it. For example, an occupational specific skill
for the job of meteorologist could be “Weather Radar Operations”. This specific skill would involve all tasks,
equipment and knowledges relevant to weather radar operations. Occupational specific skills are potentially useful
because they provide more job specific information about skills. However, they are often unstandardized and as such
there is no way to utilize them in comparisons of skills across jobs.
Given that Bloom's taxonomy is not specific enough to provide information to differentiate between jobs and that
occupational specific skills are not standardized, it was decided that the O⁎NET skill taxonomy would be retained for
use in this metric.
The next key element needed to identify the essential skills for a job is an indication of the importance level of the skill to
the job. There are various sources that could be used to reveal information about the importance of skills. Most of these
sources include information about the tasks linked to skills and are obtained through job analysis data. Options include:
information about task importance, consequence of task failure, dependency of task, demand of task, and difficulty of
tasks. Further examination of each of these sources led to the conclusion that task importance operationalized by
consequence of error during task performance was the most applicable piece of information for this metric.
More specifically, it was determined that task dependency may not be a sound representation of task importance
because a task can be important without being dependent on other tasks. Likewise, the level of task difficulty does not
necessarily relate to the task's importance in that easily learned tasks may still be critical to job performance. In regards
to demand for the skill, the importance of a skill cannot be determined by whether it is necessary upon entry into the
job. In fact, many times the important tasks of a job are not heavily applied until the incumbent has been on the job for
awhile. With these sources eliminated, the final two sources were ratings of task importance and task consequence of
error. While a task importance rating could be used to tell us the importance of skills linked to tasks, it does not
represent the positive or negative significance of not performing that task on the job. The consequence of error rating
will not only reveal information about the task's importance but also about relationship between the tasks and effective
job performance, (i.e., if these tasks are not performed properly, job performance severely suffers) thus telling us more
about the skills important to the job. Therefore, task consequence of error rated by job incumbents was selected as the
source for the second key element.
The final key element needed is an indicator of the frequency with which the skill is used on the job. The source
options for this key element include: a task frequency rating, a count of the number of tasks in the job by O⁎NET skill
(e.g., Reading Comprehension is linked to 5 total job tasks, Written Expression is linked to 2 total job tasks) and a
percentage of people in the job that use each O⁎NET skill. Two of the sources include information about the tasks
linked to skills and are obtained through job analysis data. Information of particular importance in choosing the key
element that will address the workforce analysis questions of interest here is the frequency of task application and the
breadth of task use. Option one concerns the frequency component but not the breadth. Option three taps a version of
breadth but is still not very informative. Option two does not address breadth in its current form, but can be expanded to
incorporate both frequency and breadth. Frequency of application can be examined by identifying the count of the
number of tasks in the job by O⁎NET skill. Breadth of use can be examined by modifying source option two into
identifying the number of occupational specific skills linked to each O⁎NET skill. Occupational specific skills have not
been included as a source up to this point in the metric. However, adding this component will add additional
information in describing the frequency of skill use on the job. For further illustration of the key elements and sources
identified for ‘essential skills needed for the job’, see Fig. 1.
Now that the sources for populating the key elements have been identified, metric algorithm develop can begin. The
purpose is to combine the three elements (i.e., O⁎NET skills, task consequence of error ratings, and task frequency
ratings) in a way that is meaningful in answering the workforce analysis question. By using these sources for data the
user is operationalizing “essential” as a combination of consequence of error and frequency of the tasks and how this
318 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323

information relates to the O⁎NET skills used on the job. One algorithm considered for application involved two steps.
First, all tasks in the job were rank ordered by consequence of error. This way the tasks that had the highest
consequence of error could be identified. Second, the skill linked to each task was listed. The output revealed that the
skills with the highest task consequence of error ratings and the most task linkages would be identified as most
essential. It is important to note that a cut score would need to be created to differentiate the essential from non-essential
skills. Though this algorithm would provide meaningful information about the skills on the job, it was not selected as
the final algorithm for essential skills because it did not include the task frequency rating that provides meaningful
information about how often a task, and therefore skill, is performed on the job.
Another option for the essential skills algorithm is to first identify all the skills out of the standard O⁎NET taxonomy
that are used on the job. This was conducted by extracting all skills for the job that had at least one O⁎NET skill linked
to them. Then under each of these skills the task consequence of error was averaged. Next, the number of times each
standard O⁎NET skill was linked to any of the occupational specific skills through tasks was identified. Therefore, the
output of this metric was the product of the average task consequence of error under each O⁎NET skill ⁎ the number of
tasks linked to the O⁎NET skill ⁎ the number of occupational specific skills linked to the O⁎NET skill. This algorithm
was repeated for every O⁎NET skill used on the job. The final values for each O⁎NET skill were ranked ordered to form
the output. This set of steps was recognized as the chosen algorithm for the essential skills metric because it provides
information about both the importance of tasks and frequency/breadth of use.
In order to assess whether the algorithm for essential skills produced interpretable output, the metric was applied
using actual job data. In addition, the output was examined by individuals familiar with the job as well as individuals
familiar with developing algorithms to analyze job data. The metric seemed to be differentiating essential from non-
essential skills on the job as indicated by experts. Though this metric received initial support, it is important to note that
this metric will be validated before application in real world settings.
The output from this metric includes a rank ordered list of skills required for the job, and can be broken down into
basic skills and cross functional skills. A cut off point should establish those skills that are essential for the job. This
cutoff can be made at two different points to identify 3 different “bands” of skills, one band that is essential, one that is
less essential, and one that is least essential to the job. This cutoff point will be established through validation. As
previously mentioned, this metric output can be used for the development of selection tools, as well as for the
development of training programs.

4.3. Metric validation

Validation is an essential step in the development of any measure or tool that will be used to make personnel related
decisions. Determining validity includes the establishment of a multitude of relationships to assess whether the item in
question measures what it is designed to measure (McDonald, 1999; Messick, 1989; Whetzel & Wheaton, 1997).
Establishing these relationships requires the accumulation of various types of information, both qualitative and
quantitative, such as job performance scores and scores on other tests related to and not related to the new measure.
When speaking of the validation of the metrics, the interest specifically lies in whether the output from the metric is an
actual indicator of what the metric was intended to measure.
Although there are a variety of approaches to validation, the approach for metric validation should concentrate
specifically on construct validity and content validity. The systematic procedures used in the development of the metrics,
such as the extensive use of existing literature related to key concepts used in the construction of each metric and the use of
the O⁎NET framework as building blocks, contribute to initial content validity. However, further evidence bearing on the
validity of these tools is needed prior to application in real world settings.

4.4. Implications

Using metrics as a job data analysis tool presents several implications for researchers and HR practitioners alike.
They can assist human resource professionals in addressing needs created by the lack of tools available to analyze job
data discussed earlier in this article. Ultimately, with further development, testing, and validation, metrics have the
potential to greatly impact the future of job data application and analysis.
One meaningful implication is that when using metrics, data analysis output can be assessed in its entirety, rather
than by sections. Data analysis output is often segmented because data are too massive and complex to assess as a
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 319

whole, especially with respect to qualitative techniques. Thus, when using job analysis output in workforce analysis
and planning efforts, potentially important information may not be taken into account. This missing information
could render any decisions using the job analysis data invalid, or at the very least biased. The use of O⁎NET and
metrics ameliorate this occurrence by allowing the user to include all relevant data in the analyses.
Another practical implication pertaining to the use of metrics is that they allow human resource professionals to
keep themselves current as to the status of their workforce. When job information is quantified, tagged, and
electronically stored, data are readily available for processing. As a result, job information is retrieved and processed at
a faster rate. When using qualitative or even some quantitative methods, it can take months or even years for job
analysis information to be processed and turned into meaningful information to be used in decision making (Urbanek,
1997). Part of this delay is due to the fact that the data are unique to the organization and the particular job or job family
being analyzed. Thus, data may be years old before it is used, increasing the chances that when analysis takes place, the
data are obsolete. Metrics make data analysis, and use of data, less cumbersome and more practical.
Metrics also provide a method for conducting complicated analyses of job information at a lower cost to the
customer. In industry, budget constraints often limit the type of analyses that can be conducted on job information. Job
analysis procedures often require experts who are able to analyze and interpret data. The time and expertise of these
individuals is costly. This cost is especially great when customized data interpretation is requested. Due to the high
costs associated with expert interpretation, basic reports devoid of highly applicable conclusions are often requested by
customers. Although these reports may provide supplementary information, they do not provide the kind of
information needed to answer workforce questions relevant to the organization. Metrics automate complicated types of
analysis and provide customization of interpretation without the use of analysts' time (and organizational/client
money). Thus, the use of metrics has the potential to significantly reduce the costs involved in workforce analysis and
planning. Additionally, the needs of human resource professionals are met in that they are able to use metrics to conduct
more complicated analysis with their data and therefore aid in addressing more complicated organizational and job
concerns.
Metrics allow for the circulation of job analysis information to a variety of sources in an organization. As stated,
human resource professionals often need their job analysis information to be portable across different parts of their
organization. This is often not feasible when using other job analytic methods. Since job information can be easily
misinterpreted by those that are less familiar with job analysis techniques, data interpretation is then left to those who
have expertise in occupational analytics such as Industrial/Organizational psychologists. In addition, since
interpretation requires certain specific knowledge, feedback from the data is often delivered only to the top tiers of
an organization. Other areas of the organization that could benefit from feedback are often left out, or at the very least,
only provided standard reports without the opportunity to customize reports to their specific needs. Metrics can aid in
increasing the dissemination of job analytic information by allowing multiple individuals and departments within an
organization to access and retrieve the information relevant to their concerns.
For example, companies often allow each of their departments to define what experience requirements are
necessary. These are typically referred to as skill sets or competencies. The problem with this practice is that the
information defined by each department can vary so drastically that it is difficult to consider employees across
departments for internal transfers or hires. Consider the difference between an accounting and marketing department.
An accounting department may require their employees to have basic mathematics skills, accounting and computer
knowledge, whereas a marketing department expresses the need for their employees to have customer service,
organization, and leadership skills. This lack of standardization poses potential problems for the organization by
making it difficult to know whether an employee with relevant background can transfer from accounting to
marketing or vice versa. This departmental distinction also makes promoting an employee to management
complicated because it requires comparing two employees from two different departments. It also makes it difficult
to find shared skills that can be trained or emphasized consistently company wide. Using metrics with standardized
data from sources like O⁎NET allows for the comparison of individuals within the organization on the same
elements. Therefore, decisions regarding transfer can be made after reviewing the qualifications of each individual
using the same point of reference. This increases fairness in selection and permits the transfer or promotion of the
most appropriate employee.
Furthermore, standardized information can help organizations analyze information across their industry. For
example, if every job was defined by the same set of skills, abilities, and work duties, a company might find
comparable occupations from which to recruit that were previously untapped. For instance, if an examination of an
320 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323

industry reveals that foodservice employees have the same general skillset as employees of customer service centers,
this may introduce a new avenue for recruiting in an organization.
Finally, as mentioned in the introduction, organizations exist in environments that are often unpredictable. The
utilization of O⁎NET in conjunction with metrics allows organizations to use job analytic information to respond to the
demands of increasingly dynamic organizational environments. Many times, job analysis information is poorly
collected, analyzed and interpreted. Often obsolete data is utilized in order to make current or strategic organizational
decisions. Not lost in this point is the fact that an organization's only alternative to using this deficient job data would
be to re-conduct job analyses for the workforce. Despite the importance of this job data to the functioning of
organizations, organizations are often unwilling to apply the financial resources required to complete such an effort.
Using O⁎NET, organizations can continually update job analytic information. This is plausible because O⁎NET uses
existing taxonomies (i.e., knowledges, skills, abilities) on which information is gathered and stored electronically. The
use of metrics and O⁎NET provides organizations with an advantage in that they are able to answer person, job or
workforce specific questions. Additionally, organizational decision makers can be more accurate when they are trying
to match organizational goals structure and production with that of their changing environment.

4.5. Future of metrics

Though the metrics that are currently developed address traditional workforce areas such as recruitment, selection,
promotion, and training, there is considerable room for expansion of metrics into other relevant workforce areas. Other
organizational areas that have the potential to benefit from the use of metrics include teamwork and leadership.
Additionally, the future of metrics involves implementing online, automated systems for use by HR professionals.
As mentioned, organizations are moving toward the use of cross-functional, highly independent teams.
Incumbents often are assembled into work teams to facilitate performance and increase knowledge of other's jobs. As
a result, team factors are emerging as an important area of performance for assessment. Researchers have identified
five individual level characteristics relevant to team performance. These include team orientation, mutual
performance monitoring, back-up behaviors, team leadership, and adaptability/flexibility (Cannon-Bowers & Salas,
1997). These and other team characteristics could be assessed at an employee level and analyzed to make decisions
about work team formation in an organization. Additionally, assessing performance of these characteristics using
metrics and providing feedback to employees could facilitate employee development and therefore increase team
performance.
Another area of future metric development is leadership. Leadership metrics could be used to determine individual
employee leadership performance or the capability of individuals to perform well in leadership roles. Researchers argue
that a major contributor to effective leadership is an individual's skill in solving problems (Mumford, Zaccaro,
Johnson, Diana, & Threadfall, 2000). This skill based approach to leadership gives rise to the possibility for metrics
that use an individual's proficiency in problem solving skills as an indicator of their leadership potential. Metrics such
as these could be used in conjunction with promotion or transferability metrics to ascertain which employees are best
suited for upper level managerial positions in the organization.
Metrics can be created or adapted to address most any area of HRM. Teamwork and leadership are just two of the
untapped areas that could benefit from the use of metrics. If relevant organization, job, or individual level data are
collected or exist currently, metrics can be used to effectively analyze it for decision making.
Currently, only the logic, formulas, and output interpretation of the metrics described here are available to users. As
stated, any metric should undergo a full content and construct validation prior to being used with job data to make
human resource decisions. All metrics described in this article and listed in Appendix A will be validated as a part of an
ongoing effort with SkillsNET® Corporation and the Department of Defense. In the future, metrics could be in an
online, automated format promoting ease of use and time management. Users would then be able to simply select the
job data and metric(s) they would like to use, click a button, and print the output. This online format will only further
advance the capability of metrics for use in organizations.

5. Conclusion

Job analysis information is a critical component in many HRM functions. The data obtained from job analysis
can aid in decision making for the areas of recruitment, selection, promotion, transfer, training, individual and
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 321

organizational development, and performance appraisal. These decisions made by human resource professionals are
becoming more critical to the successful functioning of the organization because of various changes in the
workforce. Worker requirements are transforming due to prevalence of cross-functional work teams. Distinctions
between supervisors and subordinates are disappearing as organizational structures become flatter and
communication occurs across many levels. These changes have led to the need for different types and larger
amounts of job analysis data than traditionally obtained.
The collection of job analysis data is now performed with relative ease due to the increased use of technology in
data collection efforts. With the use of technology, large amounts of data can be collected from multiple individuals
in shorter amounts of time. Thus, databases to house the data are typically very large and complex. Due to the
sheer size and various types of data stored in these databases, it is very difficult to effectively utilize the data.
Many times data are not used to full potential because of a lack of appropriate methods to do so. Furthermore,
without standardized elements in the job data used for analysis, decisions made using the data are not applicable
across jobs or departments within an organization.
Metrics offer an advanced yet user-friendly methodology that allows job data to be used to answer questions
related to HRM. In conjunction with O⁎NET and unique job data, metrics are innovative tools designed to aid
HRM professionals in making more objective decisions. Metrics introduce capabilities for job data that have
previously been unattainable when using qualitative and less advanced quantitative methods. Ultimately, metrics
allow organizations and HR professionals to embrace the changes resulting from the evolving world of work with
an equally advanced analysis tool.

Appendix A. Developed metrics

Name of metric Objective of metric

Job analysis metrics
Essential knowledge requirements for the job To determine the knowledges that are essential to performing the job
Essential O⁎NET ability requirements for the job To determine the abilities that are essential to performing the job
Essential O⁎NET skill requirements for the job To determine the O⁎NET that are essential to performing the job
Essential tool requirements for the job To determine the tools that are essential to the job
Estimated task importance To determine the importance of a task in the absence of a specific
rating of task importance
Job essential occupational specific skills To determine the occupational specific skills and tasks essential
to accomplishing a job
Task variety To determine the degree to which tasks performed on the job are performed
in a repetitive manner
O⁎NET skill variety To determine the degree to which a variety of skills are used on the job
Cognitive demands of job To determine the degree to which the job is cognitively demanding
Degree of technological skills To determine the degree to which a job requires technological skills
needed on the job for adequate performance

Complex job descriptors

Job complexity To determine the complexity involved in performing the tasks of the job
Physical demand To determine the degree to which the job is physically demanding
Psychosensory demands of a job To determine the degree to which the target job requires psychomotor
and sensory abilities
Job difficulty To determine the degree of difficulty related to performing the job
Stress demands on job To determine the degree to which the job is stressfully demanding
Temporal change To identify significant changes in job characteristics over time
Temporal job proficiency change To identify changes in the proficiency levels needed to
adequately perform a job
Job essential occupational To determine the complexity involved in performing the tasks
specific skill complexity associated with the Job Essential Occupational Specific Skills
Job delay tolerance To determine the degree to which an organization can function
in the absence of a particular job
Level of expertise required To determine the overall level of expertise necessary to properly perform the
skills associated with a job
(continued on next page)
322 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323
Appendix A (continued )
Level of proficiency required
Recruitment, selection, and transferability metrics
General difficulty of external recruiting for a job
General difficulty of internal recruiting for a job
Immediate proficiency needs
Key capabilities needed for the job
Eligibility for promotion
Personnel overage requirements
Job family metrics
Transferability of people across positions
Transferability of people across jobs
Transferability of jobs
Difficulty of replacement
Skill, ability, and GWA similarity
Skill set required for emerging job
Similarity of jobs based on content (non-O⁎NET jobs)
Training metrics
Cost of training
Entry level of transfer employee training time
Ease of training implementation
Material to be trained
On-the-job trainability of job essential
occupational specific skills
Overall on-the-job trainability
Estimated task learning difficulty
Degree of job technical learning difficulty
Job learning difficulty based on rating
Estimated job learning difficulty
Mission/military specific metrics
Criticality of job to the mission
Degree of required teamwork
Essential skills for the mission
Person fit for the mission
Likelihood of mission detrimental error
Number of people required for the mission
To determine the overall level of proficiency to properly perform
the skills associated with a job
To determine the level of difficulty associated with recruiting individuals
from outside the organization for a job
To determine the level of difficulty associated with recruiting individuals
from inside the organization for a job
To determine the degree to which a job requires complete skill and task
proficiency upon entry
To determine the key capabilities that the employee must have in order
to be able to perform the target job
To determine if a person fulfills the requirements to move to a higher position
To determine the degree to which additional personnel are required to be
available to cover the functions of a job under adverse conditions and high demand
To determine the degree to which a person in a current position
is able to fulfill the requirements of a target position
To determine the degree to which a person in a current job is able to fulfill
the requirements of a target job
To determine the degree to which job components/requirements can be
fulfilled by the components of another job
To determine the degree of difficulty involved with finding a person to fill a job
To determine the degree of similarity between two jobs that have
data available in the O⁎NET database
To determine the essential skills needed for a new job with no prior connected job data
To determine the degree of similarity between two jobs that do not have
data available in the O⁎NET database
To determine the cost involved in training the job
To determine the amount of time it takes to train an entry level or
transfer employee to a required level of proficiency in a job
To determine the relative difficulty of training program for a job
To determine what material within a job is essential to train in order
for the job to be successfully performed
To determine the degree to which the job essential occupational
specific skills can be trained using on-the-job training methods
To determine the degree to which the job essential tasks can be trained
using on-the-job training methods
To determine the learning difficulty of a task in the absence of a specific
rating of task learning difficulty
To determine the degree of difficulty required to learn the how to use the
tools associated with a job
To determine the amount of time it takes to learn the occupational specific
skills associated with a given job
To determine the amount of time it takes to learn the occupational specific
skills associated with a given job
To determine the criticality of the job to the mission
To determine the degree to which the tasks performed on the job require
coordination with other people
To determine the essential skills across all jobs in the mission
To determine the requirements needed for a mission and then determine
the best capable person(s) to contribute successfully to a mission
To determine the likelihood of making a serious mistake on the job that
may negatively impact the mission
To determine the number of people needed to successfully complete the mission
 L.E. McEntire et al. / Human Resource Management Review 16 (2006) 310–323 323

References

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of
educational goals. Handbook 1: Cognitive domain. New York: David McKay.
Borman, W. C. (1996). The occupational information network: An updated dictionary of occupational titles. Military Psychology, 83,
263−265.
Buckley, P., Minette, K., Joy, D., & Michaels, J. (2004). The use of an automated employment recruiting and screening system for temporary
professional employees: A case study. Human Resource Management, 43, 233−241.
Cannon-Bowers, J. A., & Salas, E. (1997). Teamwork competencies: The interaction of team member knowledge, skills, and attitudes. In H. F. O'Niel
(Ed.), Workforce readiness: Competencies and assessment (pp. 151−174). Mahweh, NJ: Lawrence Erlbaum Associates.
Cascio, W. F. (1995). Whither industrial and organizational psychology in a changing world of work? American Psychologist, 50, 928−939.
Chermack, T. J. (2003). Decision-making expertise at the core of human resource development. Advances in Developing Human Resources, 5,
365−377.
Crespin, T. R., & Austin, J. T. (2002). Computer technology applications in industrial and organizational psychology. CyberPsychology and
Behavior, 5, 279−303.
Cunningham, J. W. (1996). Generic job descriptors: A likely direction in occupational analysis. Military Psychology, 8, 247−262.
Dessler, G. (2003). Human resource management. Upper Saddle River, New Jersey: Pearson Education.
Harvey, R. J. (1991). Job analysis. In M. D. Dunnette b L.M. Hough (Eds.), Handbook of industrial and organizational psychology, vol. 2 (2nd ed.)
(pp. 71−163). Palo Alto, CA: Consulting Psychologists Press Inc.
Harvey, R. J., & Wilson, M. A. (2000). Yes Virginia, there is an objective reality in job analysis. Journal of Organizational Behavior, 21,
829−855.
Jeanneret, P. R., & Strong, M. H. (2003). Linking O⁎NET job analysis information to job requirement predictors: An O⁎NET application. Personnel
Psychology, 56, 423−429.
Korte, R. F. (2003). Biases in decision making and implications for human resource development. Advances in Developing Human Resources, 5,
440−457.
Krathwohl, D. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41, 212−218.
Mariani, M. (1999, Spring). Replace with a database: O⁎NET replaces the dictionary of occupational titles. Occupational Outlook Quarterly,
3−9.
Mariani, M. (2001, Fall). O⁎NET update. Occupational Outlook Quarterly, 26−27.
McCann, J. (2004). Organizational effectiveness: Changing concepts for changing environments. Human Resource Planning, 27, 42−50.
McCormick, E. J. (1976). Job and task analysis. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 651−696).
Chicago: Rand McNally.
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.
McDonald, E., & Cornetto, M. (2005). Putting HR data to work is easier than ever. Employee Benefit News, 19, 18−19.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13−103). New York, NY: Macmillan Publishing Co, Inc.
Mumford, M. D., & Peterson, N. G. (1999). The O⁎NET content model: Structural considerations in describing jobs. In N. G. Peterson, M. D.
Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development
of O⁎NET (pp. 21−30). Washington, DC: American Psychological Association.
Mumford, M. D., Zaccaro, S. J., Johnson, J. F., Diana, M., & Threadfall, K. V. (2000). Patterns of leader characteristics: Implications for performance
and development. Leadership Quarterly, 11, 115−133.
Murphy, T. E., & Zandvakili, S. (2000). Data and metrics driven approach to human resource practices: Using customers, employees and financial
metrics. Human Resource Management, 39, 93−105.
Patterson, B., & Lindsey, S. (2003, September). Mining the gold: Gain competitive advantage through HR data analysis. HR Magazine, 48,
131−136.
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., & Fleishman, E. A. (1999). An occupational information system for the 21st
century: The development of O⁎NET. Washington, DC: American Psychological Association.
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman, E. A., Levin, K. Y., et al. (2001). Understanding work using the
Occupational Information Network (O⁎NET). Personnel Psychology, 54, 451−492.
Riggio, R. E. (1990). Introduction to industrial/organizational psychology. Glenview, IL: Scott, Foresman and Co.
Sanchez, J. I. (1994). From documentation to innovation: Reshaping job analysis to meet emerging business needs. Human Resource Management
Review, 4, 51−74.
Sanchez, J. I., & Levine, E. L. (2000). Accuracy or consequential validity: Which is the better standard for job analysis data? Journal of
Organizational Behavior, 21, 209−220.
Schneider, B., & Konz, A. M. (1989). Strategic job analysis. Human Resource Management Review, 28, 51−63.
Siddique, C. M. (2004). Job analysis: A strategic human resource management practice. International Journal of Human Resource Management, 15,
219−244.
Urbanek, S. J. (1997). Job analysis: A local government's experience. Public Personnel Management, 26, 423−430.
U.S. Department of Labor (1991). Dictionary of occupational titles (4th ed.). Washington, DC: Author.
Walker, J. W. (1990). Human resource planning, 1990s style. Human Resource Planning, 13, 229−240.
Walker, J. W. (1999). Is HR ready for the 21st century? Human Resource Planning, 22, 5−7.
Whetzel, D. L., & Wheaton, G. R. (1997). Applied measurement methods in industrial psychology. Palo Alto, CA: Davies-Black Publishing.