Академический Документы
Профессиональный Документы
Культура Документы
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
Abstract
Engineering students’ communication skills have a significant impact on placement and
higher education opportunities. With the paradigm shift towards competence-based
communication skills enhancement, the methods of assessment should also change.
This study examined communication skills assessment model based on Outcome
Based Education of NBA criteria, using rubrics to find the reliability for the final
engineering students. The author held a series of discussion with the facilitators and
prepared criteria for verbal, non-verbal, interpersonal strategies, graphical, visual, and
statistical and audio-visuals of communication skills. The assessment tool provided
tangible information and recommendations for the continuous improvement for personal
and programme development.
Introduction
Higher education students’ communication skills have a valuable impact on placement
opportunities and entry into higher education institutions. Communication skills include
the blend of verbal, non-verbal and interpersonal strategies needed to interact
confidently and effectively with a variety of audiences. A Competent communicator
draws on a number of different means like graphical, visual, statistical, audio-visual and
technologists to get the point across. The human resource professionals select the right
candidates during ‘On and Off Campus’ events. They conduct various rounds include
group discussion, interviews, and stage speaking skills to select the right candidate.
Twenty first century has brought bountiful opportunities for the millions, with more focus
on the students’ overall capability apart from academic proficiency. Many students,
predominantly those from non-English medium schools, find that they are not favoured
due to their inadequacy of communication skills, even though they are good at their
technical knowledge in countries like India. Keeping in view their industry and business
needs and professional requirements, the technical universities introduced the courses
exclusively for communication skills development.
1
2
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
The new challenges are integrated into the American ABET Accreditation Board for
Engineering and Technology) criteria and the European EUR-ACE (European
Accreditation Board for Engineering Education) criteria for accreditation of engineering
education. Both accreditation institutions have formulated requirements for skills that go
far beyond technical knowledge (UNESCO, 2010). Some of the ABET criteria are:
An ability to function on multi-disciplinary teams.
An ability to identify and solve applied science problems.
An understanding of professional and ethical responsibility.
An ability to communicate effectively.
The broad education necessary to understand the impact of solutions in a global
and societal context.
Recognition of the need for, and an ability to engage in life-long learning.
Knowledge of contemporary issues.
An ability to use the techniques, skills, and modern scientific and technical tools
necessary for professional practice.
Graduate Attributes:
The POs formulated for each programme must be consistent with the Graduate
Attributes mentioned in Washington Accord. Graduates Attributes (GAs) form a set
of individually assessable outcomes that are the components indicative of the
graduate’s potential to acquire competence to practice at the appropriate level. The
GAs is exemplars of the attributes expected of a graduate of an accredited
programme. The Graduate Attributes of the NBA are as following:
2
3
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
Outcomes Assessment:
NBA guidelines clearly state that “The development of different rubrics and the
achievement of the outcomes need to be clearly stated in the Self Assessment Report
(SAR). The results of assessment of each PO should be indicated, since they play a vital
role in implementing the Continuous Improvement process of the programme. The
institute shall provide the ways and means of how the results of assessment of the POs
help to refine processes of revising/redefining the POs”
ABET Definitions released in 2011 define assessment as one or more processes that
identify, collect and prepare data to evaluate the attainment of student outcomes.
Effective assessment uses relevant direct, indirect, quantitative and qualitative
measures as appropriate to the outcome being measured. Reliable assessment of
outcomes is important not only for recording student performance in a subject but also
for monitoring the overall teaching-learning process of a programme. Based on the
results of the assessment, stakeholders can take corrective action to ensure that
outcomes are attained to the satisfaction of all.
In this article, author discusses key features of a quality rubric; presents models of
rubrics for assessing communication skills courses of higher education institutions. The
author describes the models for developing and institutionalizing the assessment of
components of communication skills using rubrics. Consensus was built regarding
competencies for communication skills, rubrics by working with higher education English
teachers.
This study finds how rubrics enhance the learning and whether students enhanced
learning rubrics and whether the teachers can improve the performance of students as
they progress before and after introduction of rubrics. To establish and assess on
communication skill of engineering students and to find suitable level of mastery of the
skills necessary to be successful, this study will be helpful. At the Jayam College of
Engineering and Technology, Dharmapuri Tamil Nadu, India which has been accredited
by Anna University, Tamil Nadu, India, a committee was formed by the Faculty of
Engineering to investigate, develop, and implement a common set of rubrics for the
purpose of consistently assessing graduate attributes across all engineering disciplines.
3
4
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
measured and a rating scale for each dimension. Dimensions are generally referred to
as criteria, the rating scales as levels, and definitions as descriptors.
Rubrics help to make implicit assumptions and expectations more explicit. Rubrics offer
a clear insight into the elements, assess, find reason and motivate students towards
developing the competencies. Rubrics have been shown to provide high-level feedback
(Nordrum et al. 2013). Catherine Hack (2013) found that the rubrics very helpful in
clarifying performance and promoting self-assessment, whilst the tutors felt that it was a
time efficient and informative method of providing feedback. William et al (2012)
assessed the graduate attributes of problem analysis, design, individual and team work,
communication skills, and economics and project management using rubrics.
Whenever students are asked to demonstrate their ability to carry out a particular skill or
to create a certain product, faculty have expectations of quality they use to assess that
work. In some cases, these expectations are shared with the students, but at times,
students are not clear on exactly what is expected of them. Rubrics make explicit to
students and faculty the criteria on which products and performances will be judged.
Rubrics can provide a framework for faculty to evaluate the communication skills and
performances integral to the communication skills course. Rubrics are criterion-
referenced tools and they are used to evaluate student performance against a standard
of mastery, not just to compare students. When the criteria are accurately delineated, a
rubric has the capability of providing a fair, reliable, and valid method of measurement.
Mechanism of a Rubric
A rubric entails four components:
1. Task Description
o Involve a ‘performance’ by the student
o The task can take the form of a specific assignment; e.g., Oral presentation,
group discussion, seminar presentation, mock interview, etc.
o The task can take the form of overall behaviour; e.g., Articulation, innovative
ideas, language activities and behavioural outlook in the classroom
2. Scale
o Describes how well or inadequately any given task has been performed
o In positive terms which may be used: Emerging, Progressing, Excellent, Amazing
and Excellent
o Language style: Beginning level, Middle level and High level
o Commonly used labels:
4
5
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
3. Dimensions
o Lay out the parts of the task merely and completely
o Should actually represent the type of component skills students must combine in
a successful intellectual work
o Breaking up the assignment into its distinct dimensions leads to a kind of task
analysis with the components of the task clearly identified
Among other advantages, rubrics are valuable when grading student performance that
relies on higher order thinking or other complex matters. Whatever criteria the faculty
chooses, they should be ones students must combine for their training activities to be
successful. By defining the criteria carefully, the faculty will provide each student with
detailed feedback on how well or poorly he or she did on specific parts of the whole as
well as with an overview of the student’s strengths and weaknesses for the assignment.
Rubrics are an assessment tool which can be used to evaluate student performance
over a range of criteria. A rubric normally comprises main features (Reddy and
Andrade, 2011):
5
6
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
Evaluating communication skills can be subjective which can lead students to feel that
they are being assessed disproportionately. To make the process more objective and
enable the academic to impartially compare communication skills, rubrics can be used.
The holistic and analytical are main rubrics. A holistic rubric Table 1 provides a guide to
scoring a product or performance on the basis of the overall impression it creates and
the effectiveness of the work as a whole. An analytical rubric (Table 2) breaks down a
product or performance into its essential features and then describes levels of
performance relative to each of those features or criteria. The power of the analytical
rubric is that it provides detailed information to students, teachers and parents on
individual student strengths and areas for improvement.
6
7
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
7
8
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
On every occasion students are asked to demonstrate their ability to carry out a
particular skill or to create a certain product, teachers have expectations of quality they
use to assess that work. In some cases, these expectations are shared with the
students, but at times, students are not clear on exactly what is expected of them.
Rubrics make plain to students, teachers and parent the criteria on which products and
performances will be judged. Rubrics can provide a framework for teachers to evaluate
the complex skills, products, and performances integral to the core curriculum in
technical institutions.
Rubrics are criterion-referenced tools. That is, they are used to evaluate student
performance against a standard of mastery, not just to compare students. When the
criteria are accurately delineated, a rubric has the capability of providing a fair, reliable,
and valid method of measurement. Of course, the reliability and validity of the tool
depends very much on how well the rubric developer was able to identify the important
criteria and then differentiate between the levels of performance that could be expected
from a particular group of students along those criteria (Table 3).
8
9
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
Rubrics are a way of explicitly stating ‘what is the expected student performance in a
9
10
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
chosen area of learning’. They may lead to a grade or a score but they are necessarily
more specific and more detailed than a grade or a mark-sheet. Rubrics imagine a student
performance to be comprised of many levels and provide descriptors for the performance
at each level. These descriptors help students know what they need to work on in future
so as to improve their performance. They tend to increase overall reliability of the
assessment process especially when more than one evaluator is present for assessing
the communication skills. Moreover, they provide a common language of interaction
between teacher, evaluator, and the students. Table 4 describes overall reliability of the
assessment process of non verbal communication skills.
Table 4 Non Verbal Communication Skills Rubric
Proficient More Proficient Most Proficient
Eye Speaker displays Speaker holds the Speaker holds the attention
contact minimal eye attention of the entire of the entire audience with
contact with audience with the use the use of direct eye
audience. Student of direct eye contact contact. Students rely on
sometimes reads most of the time. notes for prompting only.
from notes. Student relies on notes
for prompting only.
Poise Speaker Speaker stands up Speaker stands up straight
sometimes stands straight. Makes minor and displays a relaxed,
up straight. mistakes, but quickly self-confident nature with
Displays mild recovers from them; no mistakes.
tension; have displays little or no The movements seem
trouble recovering tension. The fluid, enhance the
from mistakes. movements seem fluid audience’s understanding,
Very little and help the audience and help the audience
movement or visualize. visualize.
descriptive
gestures.
Volume Speaker’s voice is Speaker’s voice is loud Speaker’s voice is loud
low. Audience enough to be heard by enough to be heard by all
members have most audience audience members
some difficulty members throughout throughout the whole
hearing the whole presentation. presentation.
presentation.
Clarity Speaker mumbles Speaker uses a clear Speaker uses a clear voice
at times and/or voice and pronounces and practice pronunciation
incorrectly most words correctly. of words.
pronounces some
words.
Rate Speaker speaks at Speaker speaks at a Speaker speaks at a
an inconsistent steady pace for most of steady pace; changes in
rate throughout the presentation but at rates are purposeful.
most of the times is too fast or too
presentation. slow.
Tone Speaker’s delivery Speaker’s delivery Speaker’s delivery
10
11
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
rarely matches the matches the emotion of matches the emotion of the
emotion of the the topic most of the topic. The presenter
topic and /or rarely time. The presenter establishes the appropriate
establishes the establishes the atmosphere for the
appropriate appropriate atmosphere audience toward the topic.
atmosphere for the for the audience toward
audience toward the topic most of the
the topic. time.
This also shows accuracy of the assessment tool. If deviation in peer group rating is
objectionably large, descriptors or dimensions or both, can be modified by talking to the
group and removing any element of subjectivity in the given rubric. Student is
encouraged to work on his stage presentation till he comes up to at least level 3 in his
performance. Since the rules of the competition are known and the assessment fair,
students enthusiastically participate in ‘level improvement competition’. Round number
in the assessment signifies number of the attempts that have been put in for the
improvement.
The faculties enlightened on speaking skills and how one can enrich and enhance.
‘While learning a language, being able to articulate what one wants to say can be one of
the most rewarding achievements of all language skills. Speaking is one of the essential
skills of real life in the present day job market. This skill raises the self esteem of a
student and helps in getting proper employment. Therefore, the focus has transferred to
acquiring speaking skill for qualification at under graduate level of professional
education’.
The faculties insisted that stage speaking skills are imperative because they play a vital
role in students' learning in all areas of the curriculum and industry expects candidates
to address the small gathering at certain issues at later stages. The students need to
listen keenly and speak in order to understand and explore ideas and concepts, identify
and solve problems, organize their experience and knowledge, express and clarify their
thoughts, feelings, and opinions, and convey information. Listening and speaking skills
are also essential for cooperative learning activities and for social interaction at home,
at college, and in the society (Table 5). The scores were recorded without and with
rubrics for the speaking skills and presented for the study.
11
12
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
12
13
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
6. Body Most competent (5): Maintained eye contact properly; the facial
Language expression is quite natural and positive; used positive gestures and
avoided fingers; used appropriate body movements; appropriate
postures.
More competent (3): Maintained eye contact; the facial expression
is natural; used appropriate body movements.
Competent (1): Maintained somewhat eye contact; used
appropriate body movements.
7. Conclusion Most competent (5): Concluded with appropriate quotations;
summarized with key points; reinforced the central idea; ended with
dramatic statements; concluded unexpectedly.
More competent (3): Concluded with quotations; summarized with
points; reinforced the central idea.
Competent (1): Concluded with some points; ended with
statements.
Implementation Issues:
Teaching faculty is at the heart of assessment process. Faculties must be involved in
rubrics development where they can offer insights based on their collective and
cumulative assessment experience. They may start with holistic rubrics first and then
refine them gradually into meaningful and workable analytic rubrics. Faculty consensus
on rubric is crucial to the fair and reliable outcome assessment.
Development and use of rubrics is a time consuming process and due regard should be
given to this fact while deciding a teacher’s load. If possible, teaching assistants should
be employed to facilitate the assessment work.
Outcome definition should be precise and clear before a rubric is developed for its
assessment.
It should be noted that rubrics are not needed for all the outcomes being assessed.
They are needed only where the assessment is likely to become more subjective.
While evaluating the data obtained from assessment process, it should be noted that
not all students are expected to attain all the outcomes. We are expected to develop our
own and realistic benchmarks for outcome attainment level.
Conclusions
Rubrics are very constructive for teacher assessment or second language student of
self-assessment, especially in higher education institutions. They evaluate a student’s
performance based on the sum of a full range of criteria rather than a single numerical
score. The criteria are logically linked to the outcomes intended for a learning activity.
Teachers can increase the quality of their direct transaction by providing focus,
emphasis, and attention to particular details to direct student learning. Students have
open guidelines regarding teacher expectations. Students can use rubrics as a tool to
develop their communication skills and abilities. Teachers can reuse or slightly modify
an established rubric for language developmental activities.
It is suggested that the teachers of English of higher education colleges need to use the
outcome of this study to identify students’ strengths and weaknesses to make changes
in communication skills course content and teaching methods that will enhance
students’ communication skills as they enter the 21st century.
There was feedback from students indicated that they found the rubrics very helpful in
clarifying performance and promoting self-assessment, whilst the faculties felt that it
was a time efficient and informative method of providing feedback. While implementing
rubrics, faculty’s consensus and active involvement is a must. Since rubrics
14
15
NBA’s Outcome Based Education: Communication Skills Assessment Model Using Rubrics
development and use is a time consuming process, due regard has to be given to it
when deciding a teacher’s load. Initial work inputs and entry level barriers involved in
outcome based education in general, and rubric development exercises in particular,
are understandably high. It is hoped that Indian teaching fraternity will not only adapt
itself to outcome based education system but will also see its own creativity unleashed
in the process.
References
Andrade, H. (2000). Using rubrics to promote thinking and learning. Educational
Leadership 57 (5), 13-18.
Ashley Kranov, Michael Milligan, Joe Sussman, (2010) ‘Dispelling Myths: Common
Misconception about ABET and Accreditation’, ABET Annual Conference, October
2011
Bloom, B.S. (1977). Taxonomy of Educational Objectives: The Classification of
Educational Goals. Handbook I: Cognitive Domain. New York: Longman
Catherine Hack (2013), Using rubrics to improve marking reliability and to clarify good
performance, The Higher Education Academy.
Hancock, A.B & Brundage, S.B (2010). Formative feedback, rubrics, and assessment of
professional competency through a speech-language pathology graduate program,
Journal of Allied Health 39, 110-9
Jamison, P. (1999). Student evaluation a staff development handbook. Regina, SK:
Saskatchewan Education and Saskatchewan Professional Development Unit.
15