Running head: ACTIVE LEARNING ASSESSMENT PLAN 1

Active Learning Assessment Plan

Jennifer Dulek

Trevecca Nazarene University

ACTIVE LEARNING ASSESSMENT PLAN 2

Active Learning Assessment Plan

Knowledge of learner characteristics, identification of instructional objectives, and

development of instructional strategies and tools are key steps in designing effective instruction

(Morrison, Ross, Kalman, & Kemp, 2013). However, if the results of the instruction are not

evaluated against the instructional objectives and other training goals, the effectiveness of the

instruction will remain in question. Such evaluation may occur during the instruction, at its

conclusion, and continuing into the future, and a combination of these three sources of data can

strengthen confidence in and ability to draw conclusions and make recommendations based on

the evaluation (Morrison et al., 2013). Therefore, I have developed a three-tiered assessment

plan to outline how my course aimed at developing the ability to use active learning strategies in

the classroom will meet the needs of its targeted audience, instructors who are hired to teach at

American Career College (ACC).

Formative Assessment

Formative assessment allows designers, instructors, and learners to assess learning as

instruction occurs, providing feedback regarding effectiveness of the materials as well as learner

progress toward identified learning objectives (Morrison et al., 2013). In order to effectively

identify such objectives as well as useful evaluation methods, a designer must first determine the

needs of the target audience. For the Active Learning training, the target audience consists of

newly-hired instructors who have worked in certain healthcare fields and have been hired to

teach within Associate-level programs at ACC. I determined the needs of this audience through

observation and experience.

During my four years working at ACC, I have made several relevant observations

regarding newly-hired instructors within the organization. Most instructors come directly to
ACTIVE LEARNING ASSESSMENT PLAN 3

teaching from their respective healthcare fields, and do not have significant teaching experience.

A majority hold Bachelor degrees, though level of education ranges from Associate degree to

Doctorate. Recent observation and discussion with our Campus Director have indicated that new

instructors are often not familiar with active learning strategies and when introduced, are hesitant

to try them in their classrooms. However, their 90-day and annual reviews require them to use at

least some active learning in their teaching, so this is a required skill that is currently under-

demonstrated across the campus.

When I began working at ACC four years ago, I fit the description of the new instructor

provided above, so I also based my needs assessment on my own experiences. I had very little

teaching experience, and believed that lecturing was required to be a good teacher. I was not

aware of the kinds of active learning strategies that I could use in the classroom, or how these

could help my students meet their learning goals. I also felt isolated and wished I could have

support and feedback as I tried new things in my teaching.

The learning objectives and formative assessments that I included in my instruction on

active learning address these needs of my target audience. At the beginning of the instruction,

learners are asked to do a focused listing of the benefits of active learning. This evaluation

method is aimed at the new instructors’ likely beliefs that active learning is not useful, and

assesses the affective goal of changing this belief. During the instruction, learners are directed to

review available resources for active learning, and to post to ACC’s Slack workspace under the

channels for ideas and resources; this is meant to help them generate strategies and identify

sources that they can use in their future teaching, and to assess their ability to do so. To help

learners distinguish active learning strategies and the rationale for their use, they are invited to

visit a more experience instructor’s classroom and complete an observation and reflection. This
ACTIVE LEARNING ASSESSMENT PLAN 4

formative assessment ensures that the new instructor sees what active learning looks like and

reflects on his or her readiness to teach a similar active lesson. Instruction also involves

development of lesson-planning skills using active learning techniques, and learners are required

to develop a lesson plan that integrates at least one interactive activity. Lastly, learners are to be

observed in the classroom by a peer, and then again by their Program Director or the Director or

Education, in order to assess the new instructor’s implementation and use of active learning

strategies.

One of the benefits of formative assessment is that it offers feedback that may identify the

need for remediation if a learner is not progressing as expected. Within the active learning

lessons, there are several opportunities for remediation to occur. If a new instructor struggles

with identifying the benefits of active learning, he or she may be directed to view additional

videos or even engage in discussion with an instructor who regularly uses active learning

strategies in his or her teaching. If a learner demonstrates difficulty with recognizing resources

or identifying the learning strategies observed during the classroom visit, an individualized

review of the available online materials may be necessary. Should the learner find application

and creation of a lesson or leading an active class difficult, he or she may need individualized

practice and feedback with a peer, Program Director, or Director of Education. This practice and

feedback is already built into the training and may be repeated as needed until the learner

demonstrates progress and expresses readiness to teach independently using active learning.

Through this formative assessment, important data will be collected and will be used to

collaboratively determine learners’ needs and guide remediation and instruction efforts. At each

point of formative assessment, learners will be provided with subjective and objective feedback

on their performance. They will be guided to engage in remediation if certain criteria are not met
ACTIVE LEARNING ASSESSMENT PLAN 5

at each evaluation point, and will collaborate with the instructor to identify how to improve.

Performance will be tracked to demonstrate learner progress toward overall learning objectives

and training goals.

Summative Assessment

Summative evaluation determines objectively how well learners have reached expected

levels of knowledge and performance (Morrison et al., 2013). This, in turn, can help designers

and administrators assess the effectiveness of a training program or course (Morrison et al.,

2013). To determine the effectiveness of the active learning training, a seven-step process may

be applied.

The first step of this process is “specifying program objectives” (Morrison et al., 2013, p.

330). This step was completed following the needs assessment described previously, and

resulted in four primary learning objectives (LO): 1) List the benefits of active learning for

students and instructors at American Career College, 2) Choose strategies and resources for

active learning relevant to topics being taught within assigned courses, 3) Distinguish active

learning strategies used in peer observation and justify their use, and 4) Develop and lead an

active learning session in an assigned course. These objectives lay the foundation for the

instruction and assessment plan, and provide sources of data and measurement.

The second step of the summative assessment process is to “determin[e] the evaluation

design for each objective” (Morrison et al., 2013, p. 330). Applied to the active learning training

objectives, this step allows for differentiated assessment and multiple sources of data. For

example, LO1 suggests a pretest-posttest approach, as learners are expected to move from being

unaware of the benefits of active learning to a point where they can list several benefits based on

the available evidence and personal experiences. LO2 suggests a one-group descriptive
ACTIVE LEARNING ASSESSMENT PLAN 6

approach, as learners begin to recognize available strategies and resources for active learning and

can locate and assess these for use in their teaching. LO3 and LO4 should be designed with an

experimental-control group, as they are aimed at measuring the ability to plan and implement a

lesson utilizing active learning strategies, and this can be compared to those who have not

engaged in training in order to demonstrate effective results.

Step three of determining program effectiveness involves “developing data collection

instruments and procedures for each objective” (Morrison et al, 2013, p. 330). This step requires

the designer to identify key sources of information regarding each identifying learning objective,

addressing behavior, cognition, or affect. In the case of the active learning training, LO1 assesses

attitude toward active learning as it focuses on recognition of its benefits. To gather these

responses, an open-ended question will be used, allowing learners to share their own knowledge

and observations as it relates to the use of active learning. This question could be posed in

written format on a questionnaire or via an interview. LO2 requires that learners identify and

select active learning strategies that they might use in their teaching. The formative assessment

of this LO involves posting to Slack, so attainment will be directly observed and assessed for

relevance. LO3 requires that the learner observe a peer’s teaching and assess for use of active

learning, then reflect on this experience. To complete this task, the learner will use the Active

Learning Classroom Observation Tool (ALCOT; Birdwell, Roman, Hammersmith, & Jerolimov,

2016). This tool includes a pre-observation checklist, a chronological note-taking instrument,

and a five-part, open-ended subjective questionnaire addressing items such as “In what ways did

the instructor engage students in active learning during this class?” and “What instructional

choices worked exceptionally well?” (Birdwell et al., 2016, p. 49-50). The ALCOT (Birdwell et

al., 2016) will guide the learner toward critical reflection and assessment, and will determine how
ACTIVE LEARNING ASSESSMENT PLAN 7

well he or she recognizes active learning strategies, allowing for analysis of a naturally occurring

event (Morrison et al, 2013). Lastly, LO4 will be assessed via direct observation, as a peer and

Program Director or Director of Education visits the learner’s classroom to observe active

learning strategies. This observation will also be completed using the ALCOT (Birdwell et al.,

2016), but will also include data from the Classroom Observation Protocol for Undergraduate

STEM (COPUS; Smith, Jones, Gilbert, & Wieman, 2013). The COPUS “allows observers…to

reliably characterize how faculty and students are spending their time in the classroom” (Smith et

al., p. 618), resulting in objective data demonstrating how active students are in their classroom

learning. This tool will provide a specific measurement of the instructor’s use of active learning

versus passive learning or lecture, which can be compared against others who have not been

trained in active learning strategies in order to demonstrate program effectiveness.

Step four of the program effectiveness measure requires evaluation to be carried out and

all data collected (Morrison et al., 2013). This step will involve first scheduling the active

learning training and then identifying key dates where each LO is expected to have been met.

Because the summative evaluation process includes the formative assessments, these dates will

be scattered throughout the instruction, with the final observation by the Program Director using

the ALCOT and COPUS occurring once all other objectives have been met and assessments

completed.

Continuing on to steps five and six in the summative evaluation, the designer must

analyze and interpret the results from each instrument (Morrison et al., 2013). For LO1, the

designer will gather the questionnaires or notes from each interview and identify the subjective

results; this will be in the form of lists of benefits, which the designer or subject matter expert

(SME) will need to identify as being an accurate and/or possible benefit. The designer will count
ACTIVE LEARNING ASSESSMENT PLAN 8

how many responses each learner was able to identify in order to determine effect on his or her

beliefs about active learning. When analyzing and interpreting the results for LO2, the designer

will need to view posts in the Slack workspace under the channels labeled #ideas and #resources.

The designer will need to ensure that each learner posted at least one link or idea in each channel,

and that the links fit the category and are unique. The results of the ALCOT will be examined to

analyze and interpret the results for LO3 and LO4, so the designer will need to be familiar with

this tool and its subjective reports. In particular, the designer/SME will need to review the

learner’s completed ALCOT on a peer observation to determine if active learning strategies were

accurately identified and relevant suggestions given, and will need to review the ALCOT

completed on the learner to ensure that he or she demonstrated active learning during observation

by the Program Director. In addition, the designer will need to complete the training on how to

interpret the results of the COPUS, and will then be able to code and graph the results of the

Program Director’s observation against those who have not been training in active learning

techniques.

Lastly, the summative evaluation steps involve disseminating the results and conclusions”

(Morrison et al., 2013, p. 330). This will first be done with each learner individually, then with

associated Program Directors and the Director of Education. The results will also be

summarized in an evaluation report that will be provided to College Administration for review;

this may allow for the training to be expanded to all instructors at ACC or to new instructors at

one of the institution’s other campuses.

ACTIVE LEARNING ASSESSMENT PLAN 9

Confirmative Assessment

Confirmative assessment ensures that the results of instruction have the intended long-

term results, relying on several different instruments from multiple data collection points

(Morrison et al., 2013). This allows the designer and other interested parties to confirm that

learners continue to perform well over time and that the instruction was effective (Morrison et

al., 2013). Because it is occurring outside of formal instruction, such assessment should occur in

realistic contexts and include learner reactions as a basis for improvement.

For new instructors who complete the active learning training, the primary confirmative

assessment used initially will be ongoing and regular observation with the ALCOT and COPUS

tools. When used together, these tools allow for subjective and objective data to be collected and

analyzed, and both have been shown to demonstrate sensitivity and reliability over time

(Birdwell et al., 2016; Smith et al., 2013). The ALCOT in particular allows for the learner to

self-identify current challenges via the pre-observation checklist (Birdwell et al., 2016); this will

give him or her the chance to gain relevant and meaningful feedback through this ongoing

observation, and to partner with the observer (Program Director or Director of Education) to

guide improvement. In contrast, the COPUS will provide concrete data regarding percent of

class time where active learning strategies are utilized, offering objective information on which

to base future goals and current interventions.

In addition to these measures, information gathered from students in courses taught by the

new instructor will also provide useful confirmative assessment data. This information may be

collected via course evaluations or interviews conducted within small focus groups. Although

not specifically geared toward active learning, course evaluations may offer a glimpse into

student satisfaction with teaching through ratings on items such as “My instructor teaches so that
ACTIVE LEARNING ASSESSMENT PLAN 10

I can learn,” and “My instructor makes my learning interesting and relevant.” There is also room

for open-ended student feedback on course evaluations, and review of these comments may

indicate instructor success or difficulty with active learning strategies. Focus group interviews

with students could also elicit specific observations and feedback regarding what is working and

what is challenging for new instructors in their teaching, as well as how the new instructor uses

classroom time and engages students with the course material.

Lastly, instructors may engage in a formal self-report of use of active learning strategies

as a part of confirmative assessment. This will engage them directly in assessing their progress

in using these strategies on a regular basis. This may be useful because student report may be

biased (based on grades, instructor likeability, etc.), and observation using the ALCOT and

COPUS could occur on a day when the instructor has an usual lesson planned that either under-

or over-utilizes active learning, and therefore does not represent everyday practice. Self-report

would allow for subjective observation over time, and offer the instructor another way to

determine his or her current needs with regard to teaching with active learning.

Conclusion

A thorough and well-developed assessment plan is required in order to improve student

learning and instructional design. By engaging in a three-tiered assessment process and

developing a comprehensive assessment plan, a designer and instructor can gain valuable insights

into learner and design needs. A plan such as the one detailed here offers critical information to

improve instruction and meet learner and institutional goals.

ACTIVE LEARNING ASSESSMENT PLAN 11

References

Birdwell, T., Roman, T. A., Hammersmith, L., & Jerolimov, D. (2016). Active learning

classroom observation tool: A practical tool for classroom observation and instructor

reflection in active learning classrooms. Journal on Centers for Teaching and Learning,

8, 28-50.

Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013) Designing effective

instruction (7th ed.). Hoboken, NJ: John Wiley & Sons.

Smith, M. K., Jones, F. H. M., Gilbert, S. L., and Wieman, C. E. (2013). The Classroom

Observation Protocol for Undergraduate STEM (COPUS): A new instrument to

characterize university STEM classroom practices. CBE-Life Sciences Education, 12(4),

618-627.