Evaluation in Nursing

EVALUATION IN NURSING & HEALTH CARE EDUCATION

Evaluation in Nursing & Health Care Education Dennis Tanner Saint Josephs College

NU 643 Final Paper

Evaluation in Nursing Evaluation in Nursing & Health Care Education Relevance of topic Evaluation, by definition, is a systematic process by which the value of something is judged. A well thought out and executed clinical evaluation is the means of validation that can justify an educational process. This process differs from an assessment process where data is gathered, summarized and interpreted for the purpose of determining a course of action. The purpose of a clinical evaluation is to validate the knowledge and skills of multidisciplinary healthcare workers. Taking the concept of evaluation a step further, competency evaluation allows an educator to assess learners technical skills, and the ability to critically think. Providing meaningful education to a multidisciplinary group of healthcare employees can present a challenge in that the collective skill mix has varying levels of education and job responsibilities. Using a competency checklist format, all class participants can be evaluated on their clinical skills related to behavioral responses, as well as monitoring and the documentation of patient care as it relates to a given skill level. Competency checklists can be devised to reflect psychomotor skills, interpersonal skills, and the ability to critically think. The same competency checklist can also include age specific care as well as cultural diversity. The reason evaluation by competency checklist is so important is that upon completion of any educational program such as nursing, social work, occupational therapy, etc., there is no further required learning in a formal, focused learning environment. However, healthcare professionals are expected to be competent in regards to new treatments, new equipment, and practice related regulatory updates. In order to meet this constant need, a competency based evaluation process can be used as the method to guide the educational approach. This is an

Evaluation in Nursing effective way of training new hires, and updating current employees, regardless of skill level, in a constantly changing healthcare environment. Literature review According to Klein (2006), competence is essential for health care professionals (p. 379). In order to practice in healthcare today, workers must have a theoretical understanding of health and diseases. In addition to a sound knowledge base, workers also have to demonstrate competence in technical skills, as well as the ability to critically think. In order to promote teamwork and good communication, a competency based assessment teaching method works well in terms of meeting needs of all members in the interdisciplinary team. Competency based assessment is an appropriate as well as a progressive approach to instruction and evaluation. Using competencies allowed for a practical, objective way of assessing learning. Klein used competency evaluation to as a tool improve student nurse pass rates on NCLEX-RN examinations. Morton (2002), investigated ways to prevent (and manage) patient violence by providing competency based training for all interdisciplinary staff. Competency evaluation included behavioral responses, prevention, and physical intervention techniques. After devising protocols and policies, and gaining support of administration, Morton was able to institute an effective violence prevention and management program. The comprehensive program was developed to in order to ensure that all personnel have an opportunity to attend training and gain the skills and knowledge required to respond and act appropriately in an emergent or potentially violent situation. A critical component of the program was the development and use of interdisciplinary competencies.

Evaluation in Nursing In the development of a survey targeted at assessing the educational needs of Obstetric and Pediatric nurses, Robinson, Flynn, Canavan, Cereta & Krivak (2006) note, nurses in staff development need to look beyond the course evaluation tools as the primary assessment of the education plan. Educators should use a valid educational assessment tool that addresses the staffs specialized needs on an annual basis (p. 68). The goal of the project was to gain feedback from bedside nurses. What was their perception related to current educational offerings? Were offerings adequate to meet the needs of new hires as well as existing staff? A survey was conducted in order to evaluate the how Pediatric and Obstetric nurses perceived the amount of educational offerings and the relevance of the offerings in meeting their specialized needs as care givers working with a unique population of patients. In an effort to devise meaningful and innovative education techniques for a multidisciplinary target population, Tryssenaar & Gray (2004) state, there is a growing need for professional development for healthcare providers in gerontology because of the complexity of health problems in the clientele. Clients groups and family members are better informed about their health concerns resulting in higher expectations (p. 1). This observation reinforces that competence in nursing is not only a professional responsibility, but the employer must also support professional growth and development. Facing the ever present realities of decreasing financial resources and increasing regulatory demands, devising strategies to provide learning opportunities is a necessity in todays healthcare environment. Ringerman, Flint & Hughes (2006), successfully instituted an innovative peer competency evaluation model. Prior to the institution of the competency evaluation model, hospital administration felt that the current approach to education was not meeting the needs of the staff. Prior competency evaluations only focused on the psychomotor component and had

Evaluation in Nursing many unsatisfactory characteristics. Given the climate of todays healthcare environment, a different approach was required. This decision led to the practice of selected clinical competencies and best practice competency evaluation. In addition to the new approach, a new method of assessing competence had to be developed. The new approach included bedside competencies that are evaluated by select peers who work closest with staff members. Competency assessment remains a vital part of annual performance reviews, and is specifically tailored to scope of practice. Application of topic Patient violence in health care is an unfortunate but frequent occurrence. Some institutions choose to allow the responsibility of responding to violent episodes fall to understaffed security departments or local law agencies. The problem with this type of approach is that violent acts hardly ever occur when security or law personnel are in the general vicinity. The solution to this problem is to identify the patient care areas where violent behavior has been a problem, and train the employees working in those areas on appropriate self-defense techniques in order to make them competent responders. Self defense means more than how to ward off a potential assailant. Trained, competent staff, can use de-escalation techniques or can recognize when it necessary to get all available staff together in order to intervene. The milieu on an acute care, adult inpatient psychiatric unit is such that all disciplines are able to navigate anywhere on the unit, and be in constant contact with the patients at almost any time. Because of this intense clinical exposure with patients who may have impaired thought processes, the potential for being assaulted is a common occupational hazard. Therefore, the importance of having competent front line staff is to provide a rapid response to the situation at hand. The philosophy is similar to that of training staff in CPR techniques. Time equals cardiac

Evaluation in Nursing muscle. In a violent situation, rapid response time equals less opportunity for staff or other patients to be exposed to potential injury. In the absence of staff training or other formal interventions, one can only expect to see increased episodes of aggression and violence, and an increased number of staff injuries. Self-defense techniques can include observing patient behavior, verbal and non-verbal responses, and actual hands on maneuvers. The education cannot be passive, and the training has to be multi-disciplinary. All employees have to engage and participate in the class. By the end of the educational session, participants cannot expect to be experts, but hey should be deemed at least minimally competent in the maneuvers. Competence is essential because employees will be expected to act on the behalf of themselves, co-workers, and patients. In this learning situation, competency is defined as a staff member being able to demonstrate the ability to independently perform self-protection and physical intervention techniques at an expected level. Rationale discussion Bastable (2003) notes, There is no guarantee that the learner will learn the information given, although there is more of an opportunity to learn if the educator assesses the determinants of learning (p. 76). Through timely assessment of the learner, the faculty can determine if there is a problem or deficit. The intent of a formative evaluation process is to determine if teaching was appropriate, if students learned something, and if the objectives of the course were met. There are various ways of determining learner progress towards meeting educational objectives. Primarily, evaluation takes place through classroom testing and direct clinical observation although ultimately, evaluation location is limited to ones imagination. Because learning takes place in many different settings, (classroom, simulation lab, on-line), objectives

Evaluation in Nursing have to be specific in regards to intended outcomes. A multiple question or true / false test format is basically an all or nothing proposition. A competency checklist evaluation can show gaps in learning and/or clinical competency. The same check list can also be used for remediation and to track improvement thus providing continuous feedback. Evaluation is essentially the assessment of what goes on in the learning environment. Are teaching strategies appropriate? Are the materials as well as the methods used effective? To a large degree, competency will indicate if learners are engaged, if the learning opportunities are stimulating and if the learners are adding to their knowledge base. As with the overall topic evaluation, the competency evaluation is an internally generated tool that asks specific questions about teaching effectiveness, as evidenced by learner behaviors. This internal tool allows for variables in the design. This process also presents the opportunity for receiving feedback from peers. A peer review process can improve consistency, which in turn improves quality. Regardless of the approach taken, there is a major time / energy / financial investment in the development of an educational program. Those invested in the educational process want the end result to be a positive outcome. How can the amount of actual learning be assessed? The answer is, through competency assessment. This quantitative assessment can occur through problem-based learning scenarios and skill application demonstrations. Klein (2006) states, competency-based assessment is an appropriate and progressive method of instruction and evaluation (p. 383). In order for any health care institution to adopt an effective competency-based educational program, the organization must support the process. This means providing policy level support and allowing time for multidisciplinary staff to attend educational sessions. This process includes new hires during orientation as well as veteran staff. All staff should be competent in the desired skill(s) as well as knowledgeable in institution

Evaluation in Nursing protocols. The goal of any competency-based program is to put staff in a position to be properly prepared to respond to a given situation. The competency evaluation is also useful for indicating if the curriculum design is appropriate. Information contained within the curriculum should be sequenced in such a manner that new information builds on prior information. A competency checklist does this. Learners represent the core group that can offer the most qualified feedback regarding the effectiveness of teaching strategies. Learners are in the best position to promote educational content changes based on what they put in an evaluation. Learner evaluation data can be used to make a determination if the evolution of the course content is moving in an acceptable direction. If students cannot demonstrate minimal competence within the course objectives, than the expected outcome has not been achieved and alternative methods of teaching and assessing performance may have to be employed. Learning In my role as a Clinical Nurse Educator for the multidisciplinary staff on a twenty-eight bed acute care psychiatric unit; the learning that occurred as a result of this process has made me realize that a complete overhaul of my current method of providing education is required. First, it will be vital to obtain management support for the method of hands on training that will be employed in the annual educational/regulatory update in 2007. The annual review consists of four-hour update sessions that are open to all multi-disciplinary inpatient psychiatric staff members, regardless of skill level. Although program evaluations were distributed at the end of the 2006 update sessions, new knowledge suggests that the evaluations were ineffective and do not do a good job of capturing the needs of staff as determined by the staff. Therefore, evaluations will also be revised to more closely resemble a survey. Feedback gained in the

Evaluation in Nursing January sessions will be used to make changes for the February and March educational sessions as opposed to waiting a full year before making changes. A formal competency assessment process will also become a bigger part of the sessions. In prior years, the practice was to provide passive education by rotating guest speakers through the class over the four-hour time block. For 2007, competency assessment will include increased hands on demonstration time and critique of non-offensive self-defense techniques. Because restraint and seclusion documentation are low frequency yet high liability topics on the inpatient unit, staff will also have an opportunity to do hands-on navigation in the electronic medical record. This will allow for all staff to gain exposure a complete picture of what is included in the proper documentation of a restraint or seclusion episode. In addition to more hands-on time with staff, mini topics will also be included. Staff will be given a simple pre-test, then receive the topic information via lecture, video, or CD-ROM, and a post-test will be given to determine learning. The pre-test and post-test results will be saved in each employee file to show topic content of the subject matter as well as age specific competence. If the pre/post results do not show progress, then the remediation process and subsequent addition of new knowledge can be included in the employee annual evaluation. The competency evaluation process will be included as part of their goals and objectives for the up coming year. In an effort to make the competency evaluation more meaningful, peer evaluators will also be included in the process. Not only do staff respond better to their peers, the workers in the trenches have a very good understanding of what to do, when to do it, and why to do it. This intimate knowledge application to real world situations helps to identify select co-workers as the go to people when questions arise.

Evaluation in Nursing Summary The evaluation process in education involves gathering, summarizing, interpreting, and using data in order to make a determination about how efficient, effective, and useful an educational activity actually is. This evaluation includes all the participants in the educational process, attendees, instructors, and peer evaluators. A good evaluation focuses on a specific purpose, with the goal of providing information that can be used to help others benefit from the evaluation results. One should keep in mind that regardless how detailed and complete an evaluation is, it is meaningless if the results are not used to as a resource to help in planning and executing interventions in future educational settings.

Evaluation in Nursing References

Bastable, S.S. (2003). Nurse as educator: Principles of teaching and learning (2nd ed.) Sudbury, MA: Jones and Bartlet Billings, D.M., & Halstead, J.A. (2005). Teaching in nursing: A guide for faculty. (2nd ed.). St. Louis, MO: Elsevier Saunders Oermann, M., & Gaberson, K. (2006). Evaluation and testing in nursing education. (2nd ed.). New York, NY: Springer Publishing Company, Inc. Klein, C., (2006). Linking competency-based assessment to successful clinical practice. Journal of Nursing Education. 45(9), 379-383. Morton, P., (2002). An evolution in interdisciplinary competencies to prevent and manage patient violence. Journal For Nurses In Staff Development. 18(1), 41-47. Ringerman, E., Lenora, F., Hughes, D., (2006). An innovative education program. Journal For Nurses In Staff Development. 22(3), 114-121. Robinson,J., Flynn, V., Canavan, K., Cerreta, S., & Krivak, L., (2006). Evaluating your educational plan. Journal For Nurses In Staff Development. 33(2), 65-69. Tryssenar, J., & Gray, H., (2004). Providing meaningful continuing education in a changing long-term care environment. Journal for Nurses In Staff Development. 20(1), 1-5.