English

Hello guest

Register
Top of Form

Login

Bottom of Form

Catalog Network Info

Top of Form

Search

UPLOAD YOUR PAPER

Bottom of Form

Kirkpatrick model: Training evaluation practices in the pharmaceutical industry.

Doctoral Thesis / Dissertation, 2008, 116 Pages Doctor of Philosophy (Ph.D.) Business economics - Marketing, Corporate Communication, CRM, Market Research Recommend

Follow Like Add comment Download PDF for only 49.99 USD

Top of Form

Description
Title:

Bottom of Form

Kirkpatrick model: Training evaluation practices in the pharmaceutical industry.

Institution / College: Capella University

Author: Sharon M Foreman

Advisors: Lisa M. S. Barrow, Suzann Kavli, Charle Huston

Year: 2008 EAN: 9780549538585 Pages: 116 Category: Doctoral Thesis / Dissertation Degree: Doctor of Philosophy (Ph.D.) Language: English

Abstract or Introduction
The purpose of this study was to explore the Kirkpatrick model's four levels of evaluations in the North America pharmaceutical industry. There was a lack of information on training evaluation practices for the pharmaceutical industry. This study selected members of the America Society of Training and Development who were professionals working in North America pharmaceutical companies' human resources and training departments. The criterion sampling approach confirmed the use of informed participants who were involved in evaluating organizational training programs. The four research questions were (a) How is the pharmaceutical industry currently using training evaluations? (b) To what extent are Kirkpatrick's four levels of evaluation valued to measure the effects of training programs in the pharmaceutical industry? (c) What influence do organizational structures or characteristics have on the evaluation of training programs in the pharmaceutical industry? (d) What are some of the barriers to implementing training evaluations in the pharmaceutical industry? Data analysis gathered information on the amount of evaluation, methods used, reasons for not evaluating, organizational training practices, respondents' perceptions about the value of evaluation, and demographics. Of the 275 in the targeted population, there were 93 survey interviews conducted, a response rate of 34%. The use of descriptive statistics, correlations, and one-way ANOVAs analyzed data from survey questions. This study found that the percentages of usage for Kirkpatrick's four levels were Level 1-67%, Level 2-57%, Level 3-45%, and Level 4-25%. There was no analysis of methods used pertaining to Level 1. The most commonly used methods at Level 2 were skills demonstrations and posttest with no pretest; at Level 3, observation, and performance appraisals; and at Level 4, regulation compliance. The results indicated that training departments have the knowledge and skills required to perform evaluations. However, organizations seldom required training departments to evaluate, and the time requirements for evaluations seemed to outweigh the benefits. The only significant correlation between perceptions for demonstrating value to management and the frequency of reporting evaluation outcomes to managers was at Level 1. There were no significant correlations between the four levels of evaluation and the influence of organizational characteristics

Excerpt (beta)

List of Figures

Figure 1. Evaluation model bins approach 13 x CHAPTER 1: INTRODUCTION Organizations invest large amounts of money, resources, and time in training programs (Bersin, 2003). The 2006 U.S. Training Industry Report shows organization budget increases of 7% over the last year. In 2006, U.S. organizations spent a total of $55.8 billion on training, with $15.8 billion on external learning products and services: 2005 figures presented $51.1 billion in total industry spending and $13.5 billion in spending on products and services (Bersin, 2006). The high cost of training has caused human resource development (HRD) and training practitioners to focus their attention on understanding the extent of how training programs contribute to the organizations success and performance measures (Bersin, 2003). In tandem with training budgets on the rise, evaluation practices are changing from measuring a particular training program to using data on a continual basis for monitoring the efficiency and effectiveness of the learning function across the enterprise (Davenport, 2006). There are various approaches for measuring the effectiveness of training programs in terms of business value and return on investment. According to Bassi, Cheney, and Van Buren (1997), Training departmentslike all business unitsare being forced to reexamine their roles and focus more on measurable results (p. 47). HRD and training practitioners demonstrate the value and return on investment to the organization by creating programs that clearly support business objectives, using evaluation tools to measure the effectiveness of training (Bersin, 2003). This study explored evaluation 1

practices using Kirkpatricks model to demonstrate the effectiveness, value, and contribution to achieving organizational objectives in the pharmaceutical industry. According to McFarlane (2006, p. 96), Kirkpatrick and Kirkpatrick have provided the most comprehensive, extensive and widely used approach to evaluate training programs in corporate, government, academic and other business and institutional areas. Background of the Study Survival of the pharmaceutical industry in the U.S. depends on the organizations ability to meet the demands of intense global competition, customer demands for world pricing, and rapidly escalating domestic production costs (Davenport, 2006). According to Dauch (2006), Global competition and evolving technology mandates that manufacturing practitioners continue to learn how to improve productivity throughout their careers. Hence, training curricula must be regularly reviewed and modernized (pp. 1819). The importance of workplace learning to a companys long-run viability created a need for significant organizational strategy: the bar was set high in terms of the skills required by professionals in workplace learning and standards for showing performance accountability on the job (Bassi et al., 1997). Effective training programs remain on managements continuum for producing organizational value and increasing people value (McFarlane, 2006; Rossett, 2007). Increasingly, organizations are demanding evaluation of a wide variety of organizational interventions (Chelimsky, 1997; Phillips, 1997). Training evaluations measure several things: effectiveness, potential success, resistance level, workability-applicability, progress, and quality along a continuum (Kirkpatrick & Kirkpatrick, 2006). Training 2 evaluations are tools used to provide feedback about the effectiveness of organizational initiatives: their purpose is to inform management decision-making and facilitate action (Weinstein, 2007). The success of HRD and training practitioners depend on the ability to communicate

accomplishments in business language that show value of training in terms familiar to decision makers (Torres, Preskill & Piontek, 1998). Zenger and Hargiss work on training evaluations showed that training focuses on improving knowledge, skills, and abilities of an individual, and it is functional and relevant only when evaluated (as cited in Gabr, 2000). Gabr advanced that to evaluate a training course is to evaluate its effectiveness by measuring the achievement of objectives. Training courses can have more than one objective; valuations can be effective in meeting one objective and ineffective in meeting another objective (Gabr). According to Goldstein (1993, p. 147), Training evaluation is defined as the systematic collection of descriptive and judgmental information necessary to make effective training decisions related to the selection, adoption, value, and modification of various instructional activities. Gabr stated that descriptive information describes what is happening or has happened, while judgment information reflects some opinion or belief about what has happened: both descriptive and judgment information is required for effective evaluation. The purpose of any training evaluation is to provide a guideline for future improvements. HRD and training practitioners commonly evaluate training programs using the Kirkpatrick four-level model (Kirkpatrick & Kirkpatrick, 2006). This model classifies evaluation into four levels: reaction, learning, behavior, and results. The simplicity of Levels 1 and 2 has contributed to the popularity of Kirkpatricks model 3 (Sutton, 2005). When organizations do not fully implement the model, it is often due to the perceived complexity of Levels 3 and 4 (Kirkpatrick & Kirkpatrick). Most organizations (94%) surveyed by the Association of Training and Development Society (ASTD) in 2006 measured participants reactions (Level 1) to training, 31% measured learning at Level 2, 13% measured behavior at Level 3, while only 3% measured business results at Level 4 (Rivera and Paradise, 2006). Result-based evaluation assesses the organizational benefit for performance improvements, cost/benefits analysis, and timely quality

outputs (Kaufman, Keller, & Watkins, 1995). Kirkpatricks evaluation at level four provides an effective model for demonstrating the effectiveness, value, and contribution to achieving organizational objectives in the pharmaceutical industry. Statement of the Problem Many organizations have chosen the Kirkpatrick evaluation model as an instrument for measuring the effectiveness of training programs (Bassi & Cheney, 1997; Chelimsky, 2001). Despite HRD and training practitioners advancements in developing effective training programs, there is insufficient knowledge of the Kirkpatricks model evaluation practices in the pharmaceutical industry (Kirkpatrick & Kirkpatrick, 2006; Phillips, 1997; Ruff-Eft & Praskill, 2005). The problem is that because there is insufficient knowledge regarding evaluation practices in the pharmaceutical industry, it is difficult to determine training effectiveness, value, and contribution to organizational objectives. Extensive studies of the Kirkpatrick model exist for hospitals, information technology, government, and financial industries; however, no study meeting this 4 criterion represented the pharmaceutical industry (Gomez, 2003; Hill, 1999; Twitchell, 1997). Purpose of the Study The purpose of this mixed method (qualitative and quantitative) study is to explore HRD and training practitioners evaluation practices that demonstrate the effectiveness and value of training programs. This study described evaluation practices in the pharmaceutical industry to demonstrate HRD and training practitioners contribution to achieving organizational objectives. The Kirkpatricks model is the dominant approach for evaluating training programs (Nickols, 2005). Kirkpatrick has been the industry standard for half a century, and it has a creditable record for evaluation practices (Rossett, 2007). Davenport (2006) contended, Many organizations rely on widely accepted levels framework for training evaluations that were developed by Donald Kirkpatrick and amplified by Jack Phillips (p. 50). The literature

presented Kirkpatricks four levels in different variations; however, when asking HRD and training practitioners about the levels of evaluation, the response remained: Level 1 is reaction, Level 2 is knowledge, Level 3 is behavior in the workplace, and Level 4 is results (Rossett, 2007). Rationale Evaluation is important to HRD and training practitioners because it demonstrates value to the organization (Davenport, 2006). Kirkpatrick and Kirkpatrick (2006) posited, During the past decade, workplace learning and performance professionals have faced corporate demands, economic uncertainty, and personal needs and dilemmas that have 5 significantly affected their tasks and roles (p. 48). Training practitioners ability to demonstrate value to the organization provides evidence of how training meets business needs (Davenport, 2006). Torres et al. (1998) asserted: The survival of training leaders and training programs now weighs on their ability to communicate tangible results to decision makers. This information needs to be presented in business language to communicate the value of training in terms familiar to decision makers. (p. 36) When training leaders cannot communicate tangible results, the current business trend is to downsize or eliminate departments and programs that do not effectively demonstrate value to the organization (Kirkpatrick & Kirkpatrick, 2006). Due to the need to demonstrate value, training leaders have begun to realize the benefits of evaluating training programs (Russ-Eft & Preskill, 2005). According to Kirkpatrick and Kirkpatrick, If top-level managers need to cut back, their impression of the need for a training department will determine whether they say, Thats one department we need to keep or Thats a department that we can eliminate or reduce without hurting us. And

their impression will be greatly influenced by trainers who evaluate at all levels and communicate the results to them. (pp. 19-20) When downsizing occurs, top leaders pursue people and departments to eliminate that will produce the smallest impact to the organization (Kirkpatrick & Kirkpatrick, 2006). Managers pursue ways to protect personal and departments credibility or standing, want to be accepted, and want the companys trust and respect. Most importantly, leaders want the view of being essential to the company (Davenport, 2006). Davenport went on to say that, the long-term benefit of evaluation is that trainers begin to assess programs in terms of employees ongoing relationships with the learning, with each other, and the company. Kirkpatrick and Kirkpatrick contended, 6 When you are accepted, trusted, respected, and needed, lots and lots of wonderful things happen. (1) Your budget requests are granted; (2) you keep your job (you might even get promoted); (3) your staff keep their jobs; (4) the quality of your work improves; (5) senior management listens to your advice, and; (6) you are given more control (p. 16). Research Questions This study seeks to gain a clearer understanding of the effectiveness of Kirkpatricks model by exploring the evaluation practices of HRD and training practitioners working in the pharmaceutical industry. This study explored four central questions: 1. How is the pharmaceutical industry currently using training evaluation? 2. To what extent are Kirkpatricks four levels of evaluation valued to measure the effects of training programs in the pharmaceutical industry? 3. What influence do various organization structures or characteristics have on evaluation of training programs in the pharmaceutical industry? 4. What are some of the barriers to implementing training evaluations in the pharmaceutical industry? Hypotheses

Using the Kirkpatrick (1979) four levels framework, the hypotheses challenged evaluation practices in the pharmaceutical industry to determine the effectiveness of training programs. Knowledge of the pharmaceuticals use of evaluation levels (reaction, learning, behavior, and results) examined whether training programs had an effect in meeting organizational objectives. Understanding the barriers in demonstrating training effectiveness to achieve organizational objectives revealed the value of training programs in the pharmaceutical industry. The four hypotheses explored in this study were: 7 H 1 : The majority of formal, employer-sponsored training evaluations in the pharmaceutical industry are at Level 1 (reaction) and Level 2 (learning). H 2 : When the pharmaceutical industry uses evaluation to improve training programs, the greatest value is at Level 3 (behavior). H 3 : There is no correlation between organizational characteristics, such as type or size and percentages of programs evaluated at various levels. H 4 : Difficulty in barriers is associated with the level of evaluation conducted in the pharmaceutical industry. Significance of the Study Alliger & Janek (1989) conducted a citation search on the Kirkpatrick levels from 1969 to 1989. Of the 55 articles reviewed, only eight dealt with actual evaluation of training programs, with no reported studies of correlations among all levels of the Kirkpatrick model. McFarlane (2006) contended, Training programs are the most effective, subtle and passive method of overcoming resistance to change, and represent the most viable approach to cultivating new philosophies and values in employees (p. 96). Nickols (2005) stated that the purpose of evaluating training programs is to determine their value. The training leaders task is to demonstrate their programs value to the organization by providing

evidence of accomplishing business objectives (Kirkpatrick & Kirkpatrick, 2006). According to Russ-Eft and Preskill (2005), Executives and employees want to know the extent to which and how these programs 8 and processes are contributing to the organizations success. Customers, as well, want to be assured of high quality products and services (p. 71). This research contributed to the literature by exploring the effectiveness of Kirkpatricks four levels of evaluation in the pharmaceutical industry. Findings of this study may assist HRD and training practitioners in their efforts to understand how the pharmaceutical industry allocates training resources to support business objectives. Definition of Terms In order to understand the context of this study, it is important to have clarity of specific key terms. The following descriptions are key terms often cited in the training industry. These terms appear throughout this study. ASTD. The American Society for Training and Development (ASTD), founded in 1944, is the worlds largest association dedicated to workplace learning and performance professionals. Members are from over 100 countries and connect locally in 140 U.S. chapters and 24 Global Networks. Members work in thousands of organizations of all sizes (ASTD Public Council, 2006, p. 14). HRD. Human research development has been around for some time under many different names. It is a broad field, encompassing many subject areas: training and development, career development, and organizational development. Gilley & Egglands (1989) definition of HRD is, organized learning activities arranged within an organization in order to improve performance and/or personal growth for the purpose of improving the job, the individual, and/or the organization

(p. 5). It relates to human resource management, consisting of human resources research and information systems, 9 union/labor relations, employee assistance, compensation and benefits, selection and staffing, performance management systems, HR planning, and organization/job design. Kirkpatricks model. This model is one of the leading measurement tools used for training evaluations. Developed by Donald L. Kirkpatrick in 1959, it provides a way to evaluate training programs. Phillips added a fifth level to the four levels of evaluation developed by Kirkpatrick (1997, p. 42). 1. Level 1Reaction: Measures reaction to training events, usually through a survey. 2. Level 2Learning: Measures mastery of appropriate skills or knowledge, usually through a test. 3. Level 3Behavior: Measures on-the-job behavior to see if training transfers into on-the-job skills, usually through a survey of the learner and supervisor, or actual observation on the job. 4. Level 4Results: Measures achievement of organizations goals as a result of the training event. 5. Level 5Return on Investment: Measures cost benefits of the training program. Training program. For the purpose of this study, the use of the term training program refers to technical training within the context of a work environment. It includes any training that address the skills and knowledge needed to perform job tasks. The scope of the term pertains only to technical training that occurs in the United States. Assumptions and Limitations An assumption in this study is that participants provided honest answers based on personal knowledge and experience of evaluation practices. Another assumption was all

10
Excerpt from 116 pages - scroll top
Top of Form

Add comment

Send Your comment is reviewed before being published

Bottom of Form

This text can be quoted and accessed from this url: