Why transform to a language of Why transform to a language of
assessment? assessment? Specific learning outcomes lead to: Specific learning outcomes lead to: More measurable outcomes More measurable outcomes More measurable outcomes More measurable outcomes Better assessment Better assessment Higher quality feedback Higher quality feedback Improved courses and programs Improved courses and programs Improved student learning and Improved student learning and achievement achievement Design Backward Alignment Within Courses Alignment Within Courses Intended Learning Outcomes of the Lesson Intended Learning Outcomes of the Unit Intended Learning Outcomes of the Course Deliver Forward Outcomes Based Assessment Outcomes Based Assessment:: A process A process by which you by which you 1. 1. determine the indicators determine the indicators of an effective of an effective program, program, 2. 2. use those indicators use those indicators as criteria for as criteria for The Language of Assessment The Language of Assessment 2. 2. use those indicators use those indicators as criteria for as criteria for assessing the program, and assessing the program, and 3. 3. apply the results apply the results of the assessment of the assessment toward the ongoing and continuous toward the ongoing and continuous improvement of the program. improvement of the program. Objectives vs. Outcomes Objectives vs. Outcomes 11 Program/course Program/course objectives objectives are general goals that are general goals that define what it means to be an effective program/course. define what it means to be an effective program/course. They are They are general, indefinite, and not intended to be general, indefinite, and not intended to be measured measured. They set the overall agenda for the . They set the overall agenda for the program/course. program/course. Student learning Student learning outcomes outcomes are are specific results specific results the the program/course seeks to achieve in order to attain the program/course seeks to achieve in order to attain the general goals defined in the objectives. Outcomes are general goals defined in the objectives. Outcomes are definite and intended to be measured definite and intended to be measured. They establish . They establish the particular means by which the agenda (as defined by the particular means by which the agenda (as defined by objectives) is achieved. The achievement of outcomes objectives) is achieved. The achievement of outcomes is evidence that our students are learning. is evidence that our students are learning. Direct assessment Direct assessment of learning: gathers of learning: gathers evidence evidence, based on , based on student performance student performance, , which demonstrates the learning itself. which demonstrates the learning itself. Examples: most classroom testing for grades Examples: most classroom testing for grades or evaluation of a research paper on specific or evaluation of a research paper on specific Direct vs. Indirect Assessment Direct vs. Indirect Assessment 11 or evaluation of a research paper on specific or evaluation of a research paper on specific criteria criteria Indirect assessment Indirect assessment of learning: gathers of learning: gathers reflection reflection about the learning or about the learning or secondary secondary evidence evidence of its existence. of its existence. Examples: student, alumni, employer surveys Examples: student, alumni, employer surveys Embedded assessment Embedded assessment:: a means of gathering a means of gathering information about student learning that is information about student learning that is built built into into, and is , and is a natural part of a natural part of the teaching the teaching-- learning process. learning process. Assessment is not always an add Assessment is not always an add--on on 11 Example: as part of a course, expecting each senior Example: as part of a course, expecting each senior to complete a research paper that is graded for to complete a research paper that is graded for content and style, but is also assessed for advanced content and style, but is also assessed for advanced ability to locate and evaluate Web ability to locate and evaluate Web--based information based information (as part of a program level, or a college (as part of a program level, or a college--wide outcome wide outcome to demonstrate information literacy). to demonstrate information literacy). Formative assessment Formative assessment: the gathering of : the gathering of information about student learning information about student learning -- during the during the progression progression of a course or program and usually of a course or program and usually repeatedly repeatedly -- to improve the learning to improve the learning of current of current students. students. Formative vs. Summative Formative vs. Summative Assessment Assessment 11 students. students. Summative assessment Summative assessment: the gathering of : the gathering of information information at the conclusion at the conclusion of a course, of a course, program, or undergraduate career to program, or undergraduate career to improve improve learning of the next cohort learning of the next cohort of students or to of students or to meet meet accountability demands accountability demands. . Assessment for accountability Assessment for accountability: assessment of : assessment of some unit (could be a department, program or some unit (could be a department, program or entire institution) to entire institution) to satisfy stakeholders satisfy stakeholders external external to the unit itself. Results are often compared to the unit itself. Results are often compared across units, compared to state and national across units, compared to state and national norms, and norms, and always summative always summative.. Accountability and Accountability and Improvement Improvement 11 norms, and norms, and always summative always summative.. Assessment for improvement: Assessment for improvement: assessment assessment that that feeds directly feeds directly, and often immediately, , and often immediately, back back into revising into revising the course, program or institution to the course, program or institution to improve student learning results. This can be improve student learning results. This can be formative or summative formative or summative.. Assessment of individuals Assessment of individuals: uses the individual student, : uses the individual student, and his/her learning, as the level of analysis. and his/her learning, as the level of analysis. Assessment of programs Assessment of programs: uses the department or : uses the department or program as the level of analysis. Ideally program goals program as the level of analysis. Ideally program goals and objectives would serve as the basis for the and objectives would serve as the basis for the Levels of Assessment Levels of Assessment 11 and objectives would serve as the basis for the and objectives would serve as the basis for the assessment. assessment. Assessment of institutions Assessment of institutions: uses the institution as the : uses the institution as the level of analysis. Ideally, institution level of analysis. Ideally, institution--wide goals and wide goals and objectives would serve as a basis for the assessment. objectives would serve as a basis for the assessment. At this level it is essential to examine institutional At this level it is essential to examine institutional documents such as mission and vision statements, as documents such as mission and vision statements, as well as strategic plans. well as strategic plans. Characteristics of Well Stated Characteristics of Well Stated Learning Outcomes Learning Outcomes 22 student student--focused rather than professor focused focused rather than professor focused focused on the learning resulting from an activity focused on the learning resulting from an activity rather than on the activity itself rather than on the activity itself focused on skills and abilities central to the focused on skills and abilities central to the discipline and based on professional standards of discipline and based on professional standards of discipline and based on professional standards of discipline and based on professional standards of excellence excellence general enough to capture important learning but general enough to capture important learning but clear and specific enough to be measurable clear and specific enough to be measurable focused on aspects of learning that will develop focused on aspects of learning that will develop and endure but that can be assessed in some and endure but that can be assessed in some form now form now Common Problems with Common Problems with Learning Outcomes Learning Outcomes Using vague terms, such as: Using vague terms, such as: Appreciate Appreciate Become aware of Become aware of Become familiar with Become familiar with Develop Develop Develop Develop Know Know Learn Learn Understand Understand Describing action taken by someone other than Describing action taken by someone other than the learner. the learner. The program will... or The program will... or The course will The course will A Comparison of Poorly and Well A Comparison of Poorly and Well Stated Outcomes Stated Outcomes Students will understand Students will understand Eriksons developmental Eriksons developmental stages. stages. Students will be familiar with Students will be familiar with the major sociological the major sociological perspectives and how they perspectives and how they Students will identify and Students will identify and summarize each of Eriksons summarize each of Eriksons stages of development. stages of development. Students will describe each of Students will describe each of the major sociological the major sociological perspectives and will illustrate perspectives and will illustrate perspectives and how they perspectives and how they relate to their daily lives. relate to their daily lives. Students will develop the skills Students will develop the skills necessary for conducting necessary for conducting research in the natural research in the natural sciences. sciences. perspectives and will illustrate perspectives and will illustrate how each perspective relates how each perspective relates to events in their daily lives. to events in their daily lives. Students will design, conduct, Students will design, conduct, and analyze a research project and analyze a research project using appropriate scientific using appropriate scientific theory and methodology theory and methodology Why Classify Learning Why Classify Learning Outcomes? Outcomes? All learning outcomes are not developed, All learning outcomes are not developed, delivered, or measured equally delivered, or measured equally Learning Domains Learning Domains Three primary domains for classifying Three primary domains for classifying educational goals: educational goals: Cognitive (knowledge) Cognitive (knowledge) Cognitive (knowledge) Cognitive (knowledge) Affective (attitudes) Affective (attitudes) Psychomotor (skills) Psychomotor (skills) Blooms Taxonomy of Blooms Taxonomy of Educational Objectives Educational Objectives 33 ((Cognitive domain) Cognitive domain) A now classic system that classifies educational A now classic system that classifies educational goals to facilitate the development and goals to facilitate the development and evaluation of college and university curricula. evaluation of college and university curricula. evaluation of college and university curricula. evaluation of college and university curricula. A hierarchical taxonomy of student behaviors A hierarchical taxonomy of student behaviors that reflect the development of increasingly that reflect the development of increasingly complex cognitive abilities and skills as a result complex cognitive abilities and skills as a result of instructional experiences. of instructional experiences. Blooms Taxonomy of Cognitive Blooms Taxonomy of Cognitive Levels Levels There are 6 categories, listed hierarchically from There are 6 categories, listed hierarchically from simplest to most complex simplest to most complex Knowledge Knowledge Comprehension Comprehension Comprehension Comprehension Application Application Analysis Analysis Synthesis Synthesis Evaluation Evaluation Blooms Taxonomy: Knowledge Blooms Taxonomy: Knowledge 44 Definition: Definition: ability to remember information from ability to remember information from simple (facts, terminology) to more simple (facts, terminology) to more complex/abstract (theories, principles) complex/abstract (theories, principles) Student Learning Verbs: Student Learning Verbs: List, name, identify, show, define, List, name, identify, show, define, recognize, recall, state, describe, label, recognize, recall, state, describe, label, match, outline, reproduce, select match, outline, reproduce, select Blooms Taxonomy: Knowledge Blooms Taxonomy: Knowledge 44 Examples: Examples: Knowledge of dates, Knowledge of dates, events, places, major ideas, and events, places, major ideas, and mastery of subject matter mastery of subject matter The student will The student will Define the 6 levels of Blooms Define the 6 levels of Blooms taxonomy of the cognitive domain taxonomy of the cognitive domain Blooms Taxonomy: Blooms Taxonomy: Comprehension Comprehension 44 Definition: Definition: ability to understand material at a level ability to understand material at a level sufficient for grasping its meaning and sufficient for grasping its meaning and inferring its implications inferring its implications Translating, comprehending, or interpreting Translating, comprehending, or interpreting Translating, comprehending, or interpreting Translating, comprehending, or interpreting information based on prior learning information based on prior learning Student Learning Verbs: Student Learning Verbs: Summarize, explain, interpret, describe, Summarize, explain, interpret, describe, compare, paraphrase, differentiate, compare, paraphrase, differentiate, demonstrate, restate, illustrate demonstrate, restate, illustrate Blooms Taxonomy: Comprehension Blooms Taxonomy: Comprehension 44 Examples: Examples: Translates knowledge into Translates knowledge into next context, interprets facts, compare and next context, interprets facts, compare and contrast, order, group, infer, predict contrast, order, group, infer, predict The student will The student will Explain the purpose of Blooms Explain the purpose of Blooms taxonomy of the cognitive domain taxonomy of the cognitive domain Blooms Taxonomy: Application Blooms Taxonomy: Application 44 Definition: Definition: ability to correctly and independently bring to bear ability to correctly and independently bring to bear abstractions (e.g., theories, principles, methods) in abstractions (e.g., theories, principles, methods) in solving concrete problems solving concrete problems The selection, transfer, and use of data and principles The selection, transfer, and use of data and principles The selection, transfer, and use of data and principles The selection, transfer, and use of data and principles to complete a task with a minimum of direction to complete a task with a minimum of direction Student Learning Verbs: Student Learning Verbs: Solve, illustrate, calculate, compute, use, interpret, Solve, illustrate, calculate, compute, use, interpret, relate, manipulate, apply, classify, modify, relate, manipulate, apply, classify, modify, demonstrate, construct, discover, predict demonstrate, construct, discover, predict Blooms Taxonomy: Application Blooms Taxonomy: Application 44 Examples: Examples: Use information, methods, concepts or Use information, methods, concepts or theories in new situations, solve problems using required theories in new situations, solve problems using required skills or knowledge skills or knowledge The student will The student will The student will The student will Write an instructional objective for each level of Write an instructional objective for each level of Blooms taxonomy of the cognitive domain Blooms taxonomy of the cognitive domain Blooms Taxonomy: Analysis Blooms Taxonomy: Analysis 44 Definition: Definition: ability to parse information into is constituent ability to parse information into is constituent elements and to identify the relationships between elements and to identify the relationships between those elements those elements Student distinguishes, classifies, and relates the Student distinguishes, classifies, and relates the assumptions hypotheses, evidence, or structure of a assumptions hypotheses, evidence, or structure of a assumptions hypotheses, evidence, or structure of a assumptions hypotheses, evidence, or structure of a statement or question statement or question Student Learning Verbs: Student Learning Verbs: Analyze, organize, categorize, deduce, choose, Analyze, organize, categorize, deduce, choose, contrast, compare, distinguish, separate, differentiate, contrast, compare, distinguish, separate, differentiate, discriminate discriminate Blooms Taxonomy: Analysis Blooms Taxonomy: Analysis 44 Examples: Examples: Seeing patterns, organization of parts, Seeing patterns, organization of parts, recognition of hidden meanings, identification of recognition of hidden meanings, identification of components components The student will The student will Compare and contrast the cognitive and affective Compare and contrast the cognitive and affective domains as specified by Bloom domains as specified by Bloom Blooms Taxonomy: Synthesis Blooms Taxonomy: Synthesis 44 Definition: Definition: ability to combine elements into new wholes (e.g., ability to combine elements into new wholes (e.g., ideas, plans of action, abstract relations) that are ideas, plans of action, abstract relations) that are more than the sums of their respective parts more than the sums of their respective parts Student originates, integrates, and combines ideas Student originates, integrates, and combines ideas into a product, plan, or proposal that is new to him or into a product, plan, or proposal that is new to him or her. her. into a product, plan, or proposal that is new to him or into a product, plan, or proposal that is new to him or her. her. Student Learning Verbs: Student Learning Verbs: Design, create, hypothesize, invent, develop, support, Design, create, hypothesize, invent, develop, support, schematize, write, report, discuss, plan, devise, schematize, write, report, discuss, plan, devise, compare, construct, compose, generate compare, construct, compose, generate Blooms Taxonomy: Synthesis Blooms Taxonomy: Synthesis 44 Examples: Examples: Use old ideas to create new ones, Use old ideas to create new ones, generalize from given facts, relate knowledge from generalize from given facts, relate knowledge from several areas several areas The student will The student will The student will The student will Design a classification scheme for writing educational Design a classification scheme for writing educational objectives that combines the cognitive, affective, and objectives that combines the cognitive, affective, and psychomotor domains. psychomotor domains. Blooms Taxonomy: Evaluation Blooms Taxonomy: Evaluation 44 Definition: Definition: Ability to offer quantitative and qualitative judgments Ability to offer quantitative and qualitative judgments about the value of ideas and methods about the value of ideas and methods Student appraises, assesses, or critiques on a basis Student appraises, assesses, or critiques on a basis of specific standards and criteria of specific standards and criteria of specific standards and criteria of specific standards and criteria Student Learning Verbs: Student Learning Verbs: Evaluate, choose, estimate, judge, defend, criticize, Evaluate, choose, estimate, judge, defend, criticize, justify, recommend, critique, interpret, support justify, recommend, critique, interpret, support Blooms Taxonomy: Evaluation Blooms Taxonomy: Evaluation 44 Examples: Examples: Compare and discriminate Compare and discriminate between ideas, assess value of theories or between ideas, assess value of theories or presentations, make choices based on reasoned presentations, make choices based on reasoned argument, verify value of evidence, recognize argument, verify value of evidence, recognize subjectivity subjectivity The student will The student will Judge the effectiveness of writing objectives Judge the effectiveness of writing objectives using Blooms taxonomy using Blooms taxonomy Learning Outcomes by Blooms Learning Outcomes by Blooms Taxonomy Taxonomy XX 1.a.1. 1.a.1. Define the three Define the three tenets of the Central tenets of the Central Limit Theorem Limit Theorem 1.a 1.a learn the learn the conceptual conceptual foundations of foundations of 1. 1. Introduce Introduce students to students to inferential inferential statistics statistics Eval Eval-- uation uation Syn Syn-- thesis thesis Anal Anal-- ysis ysis Appli Appli-- cation cation Com Com-- prehen prehen-- sion sion Know Know-- ledge ledge Bloom's Taxonomy of Cognitive Bloom's Taxonomy of Cognitive Categories Categories Student Learning Student Learning Outcomes Outcomes Unit/Lesson Unit/Lesson Learning Learning Outcomes Outcomes Course Course Goals/Objectives Goals/Objectives Note: While this worksheet accommodates 7 learning outcomes, your specific course will most likely have more than this singl Note: While this worksheet accommodates 7 learning outcomes, your specific course will most likely have more than this single w e worksheet can orksheet can accommodate. The purpose of the worksheet is to provide a framework and not set parameters. accommodate. The purpose of the worksheet is to provide a framework and not set parameters. 2.b.1. 2.b.1. Outcome Outcome 1.c.1. 1.c.1. Outcome Outcome 2.a 2.a Outcome Outcome 2. 2. Introduce Introduce students to students to descriptive descriptive statistics statistics 1.c 1.c Test for difference Test for difference between means between means XX 1.b.1. 1.b.1. Outcome Outcome 1.b 1.b Apply to confidence Apply to confidence intervals intervals XX 1.a.2. 1.a.2. Describe three Describe three key distributions key distributions foundations of foundations of inference inference statistics statistics 2.b 2.b Outcome Outcome 2.a.1. 2.a.1. Outcome Outcome 1.a.2. 1.a.2. Combine to explain Combine to explain the relationship between the relationship between the three distributions the three distributions Course-Level Learning Outcomes Alignment within a Given Course Teaching and Learning Activities Assessments of Student Learning The Next Step: The Next Step: Given your student learning outcomes, Given your student learning outcomes, what specific tasks or activities will you what specific tasks or activities will you have students complete to promote have students complete to promote learning? learning? learning? learning? Given these student learning outcomes, Given these student learning outcomes, how will you know when your students how will you know when your students have achieved the outcomes for that have achieved the outcomes for that lesson or course (what assessments will lesson or course (what assessments will you use)? you use)? Teaching & Learning Activities by Teaching & Learning Activities by Outcomes and Blooms Taxonomy Outcomes and Blooms Taxonomy Class lecture and Class lecture and students will read students will read assigned chapter. assigned chapter. 1.a.1 1.a.1 Define the Define the three tenets of the three tenets of the Central Limit Central Limit Theorem Theorem Evaluation Evaluation Synthesis Synthesis Analysis Analysis Application Application Comprehension Comprehension Knowledge Knowledge Bloom's Taxonomy of Cognitive Categories Bloom's Taxonomy of Cognitive Categories Student Student Learning Learning Outcomes Outcomes 2.b.1 2.b.1 Outcome Outcome 2.a.1 2.a.1 Outcome Outcome 1.c.1 1.c.1 Outcome Outcome In class, students will In class, students will calculate sample means calculate sample means and construct a and construct a sampling distribution. sampling distribution. Homework will reinforce Homework will reinforce lesson. lesson. 1.b.1 1.b.1 Outcome Outcome 1.a.3 1.a.3 Combine to Combine to explain the explain the relationship between relationship between the three the three distributions distributions 1.a.2. 1.a.2. Describe three Describe three key distributions key distributions Theorem Theorem Class lecture and Class lecture and students will read students will read assigned chapter. assigned chapter. Assessments by Outcomes and Assessments by Outcomes and Blooms Taxonomy Blooms Taxonomy Homework and Homework and examination. examination. 1.a.1 1.a.1 Define the Define the three tenets of the three tenets of the Central Limit Central Limit Theorem Theorem Evaluation Evaluation Synthesis Synthesis Analysis Analysis Application Application Comprehension Comprehension Knowledge Knowledge Bloom's Taxonomy of Cognitive Categories Bloom's Taxonomy of Cognitive Categories Student Student Learning Learning Outcomes Outcomes 2.b.1 2.b.1 Outcome Outcome 2.a.1 2.a.1 Outcome Outcome 1.c.1 1.c.1 Outcome Outcome Homework and Homework and examination. examination. 1.b.1 1.b.1 Outcome Outcome 1.a.3 1.a.3 Combine to Combine to explain the explain the relationship between relationship between the three the three distributions distributions 1.a.2. 1.a.2. Describe three Describe three key distributions key distributions Theorem Theorem Homework and Homework and examination. examination. Beyond Course Alignment Beyond Course Alignment Thinking Globally: Alignment of course level Thinking Globally: Alignment of course level learning outcomes: learning outcomes: Sequentially Sequentially How do my outcomes fit with those of more advanced How do my outcomes fit with those of more advanced courses? courses? What do I expect students to have learned by the time What do I expect students to have learned by the time they when they enter this class? they when they enter this class? Programmatically: Programmatically: How do all of the courses within a programs How do all of the courses within a programs curriculum fit together? curriculum fit together? How do the program curricula contribute to the How do the program curricula contribute to the institutional outcomes? institutional outcomes? Design Backward Alignment Between Course Outcomes Alignment Between Course Outcomes and Institutional Outcomes and Institutional Outcomes Intended Learning Outcomes of the Lesson Intended Learning Outcomes of the Unit Intended Learning Outcomes of the Course Intended Learning Outcomes of the Academic Program Intended Learning Outcomes of the Institution Deliver Forward