Академический Документы
Профессиональный Документы
Культура Документы
Luay Askar
Language Institute (DLI) to the principles of the Diagnostic Assessment (DA). The targeted
audience is newly joined and seasoned instructors, aiming at familiarizing the former to the basic
principles and the process of conducting the DA, and as refresher training for the later. This
familiarization training is not a replacement for the DA specialist certification training, but it will
help the teachers to conduct informal DA for their students in order to support them during the
learning process. Formal DA reports are written by certified DA specialists only according to the
current regulations.
The learning process is a collaborative work of both the instructor and the learner.
Implementing the student-centered approach in the classroom has changed the role of the teacher
from Sage on Stage to Guide on the Side (King, 1993). The teacher needs to facilitate, to
prepare materials, and to assess the students. There are different types of assessments using
different tools, all aim at gathering data and the teachers require being familiar with the
diagnostic tool they will be using for the assessment. Assessment is the process of gathering
data. More specifically, assessment is the ways instructors gather data about their teaching and
The mission of the DLI is to provide culturally based foreign language education, training,
evaluation and sustainment to enhance the security of the nation. At the DLI, the time allocated
to each language program differs depending on the category of the language being taught.
Recently, the DLI set up a new standard for the students to pass the final exams with the goal of
achieving an Interagency Language Roundtable (ILR) level of 2+/2+/2, in reading, listening and
speaking skill modalities, consecutively. This goal is a challenge for both the students and the
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 3
teachers. For the students, acquiring a new language is not a personal goal but also a job-related
goal, thus requiring more effort and dedication. As for the teachers, their standards surpassed the
traditional teaching standards to include and not limited to; implementing more advanced
instruction in their instructional model. Traditional teaching following the rule one size fits all
is no longer acceptable, if we are aiming to achieve the DLI goals of a proficiency level of
requires knowledge of the diagnostic tools and thus the DA. Alderson (2004) came up with a list
of features that can help to clarify the benefits of the DA, of these:
Therefore, this familiarization training is to introduce the teachers to the basics of the DA,
what and how to prepare material for the DA, how to construct the DA questions, when to
conduct it, which student requires the DA, and how to conduct it.
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 4
Literature Review
Many of the studies conducted about assessment approach the topic from the learners
perspective, while few articles approach the topic from the teachers perspective. In addition,
studies are mainly directed towards the different types of assessments and specifically towards
Diagnostic Assessment (DA). Review of literature was done to support the proposed DA
familiarization training.
An article by Blood (2011) and another by Dunn & Mulvenon (2009), both talking about
assess, evaluate and give feedback to learners. Diagnosis, according to Blood (2011), is a term
familiar in the medical field but in second language acquisition, broadly speaking, diagnosis is
any form of assessment whether formative or summative that yields feedback. According to
Dunn & Mulvenon (2009), Although an assessment may be designed and packaged as a
formative or summative assessment, it is the actual methodology, data analysis, and use of the
results that determine whether an assessment is formative or summative (p.2). Blood also
The two authors emphasized the importance of the content of the assessment in revealing
more accurate data; Blood questioned whether the tasks used by the teacher to construct
assessment material are appropriate for a formative or a summative assessment, while Dunn &
Mulvenon questioned if the assessment will still be called formative if the teachers use it for
grading.
summative), should not be according to their nomenclature, the main points of differentiation
one should keep in mind are; where is the assessment used, when is it used and how is it used.
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 5
Mentioning that the most important question teachers should keep in their mind is; what they are
going to do with the collected data? Answering this question will give the appropriate
teachers perception to and the use of assessment in the classroom and there is a tendency to use
the study of Tsagari (2016) about the assessment orientation of teachers in two Mediterranean
countries, where teachers concentrate on the traditional use of assessment with contents limited
to assessing vocabulary and grammar and lack creativity and higher order thinking. In both
articles, the assessment used by the teachers tends to be more summative than formative. In
addition, both articles recommend that teachers need more training in language assessment
A study carried out by Mertler (2003), comparing the in-service teachers with the
preservice teachers (recent graduates) regarding assessment literacy. This article confirms
what Mendoza (2009) and Tsagari (2016) recommend in their studies; that teachers need more
training in regard to assessment. In addition, Mertler mentioned, Previous studies have reported
that teachers feel and actually are unprepared to adequately assess their students(p.22). Also, the
teachers believe that they did not have sufficient training to build up the skills in assessing the
learners (p.22). He compared in this study between the preservice and the in-service teachers in
regard many standards including learnersassessment and found that; in-service teachers scored
higher than preservice. Mertler questioned whether on-the-job (in the classroom) assessment
training is preferred than training during the undergraduate study. This study highlights the
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 6
importance of understanding the concept of assessment, training and experience that will be
Analysis
Needs Assessment
Identifying the problem. The DLI is aiming towards teachers adhering to student-
centered teaching methodology, with the teachers being facilitators that guide the students
through their learning process. Also, the teachers are required to prepare teaching materials and
that need improvement for each learner. This should be followed by an interventional
instructional plan to overcome the difficulty. Being a trainer for the DA specialist certification
course, I have noticed that most of the participants lack the basic knowledge about the DA. This
problem has a direct negative influence on the outcome of the teaching-learning process, and
Learner Analysis
Identifying the target audience. Data collected from learners analysis was provided
from four sources; from the direct supervisors through their class observations and from follow-
up checks of teachers performance. The other source are direct interviews (sensing sessions)
conducted by the academic specialist and the dean with the teachers. The third source is the
questionnaire survey given to the teachers by the end of the DA certification course
(retrospective data collection). The last source is a questionnaire I conducted recently through
Survey Monkey (see Appendix A for the questionnaire). The responses collected from the survey
include suggestions from the participants that, if they had been enrolled in familiarization
training, it would have made their certification course more productive. Another issue is the
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 7
diversity of knowledge about the basic principles of the DA among the participants attending the
certification course. This varies from a novice with lack of knowledge to experts requiring
honing of their experience. The results of the survey conducted using Survey Monkey revealed
that around 30% of the responders have little information about DA, and 39% less often conduct
DA and another 39% moderate frequency. Also, around 50% less often been exposed to DA
report. These figures confirm that the audience varies between novice with lack of knowledge to
experts with the limited practice of DA. Collectively, all these analyses support the proposal of
conducting familiarization training that will cover the issues of DA, as well as set the solid
ground for the DA certification course, which is a long training process. In addition, such
Context Analysis
Logistic information. This training is faced with a logistical problem of time constraints.
The time that will be allotted to the training, how it will affect the teaching hours of the teacher,
and if there are any drawbacks on the learning process should be taken into consideration. The
training for the diagnostic teaching is already being conducted at the DLI by faculty development
specialists over four hours, usually one morning or afternoon. The proposed DA familiarization
training is not a substitute for the DA specialist certification training. As the DA specialist
practicum and the submission of a final portfolio. This will increase the out-of-classroom time
for the instructors and thus their teaching hours which is part of the teachers standards.
During the proposed familiarization training, the trainees will not conduct DA for a
student as in the DA certification training. Therefore, it will start as blended with instructor-led
for the first iteration so as to pinpoint areas that require modification or adjustment. Then the
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 8
proceed in the training will be given as a Portable Document Format (PDF). In addition, the
designed model will include videotaped DA conducted with a volunteer. The time intended for
the blended course is 1-2 days and for the e-learning around 4-6 hours, but the latter will depend
This training will save time for the organization and for the instructor and will not require
a classroom or a conference room to conduct the training. The teachers can undertake the
training at their workplace and they can pause the training in order to attend a class or to perform
any other task. In addition, the availability of MacBook and IPads for each teacher in addition to
the high-speed internet will facilitate the use of the self-instruction and asynchronous training.
Content Analysis
The DA certification model consists of five modules. Two main modules can be used for
the familiarization training that contains the practical information required for the DA interview
process.
The first module will contain information related to selecting passages to be used
during the DA interview. The level of this material is according to the ILR
o The audience will be provided with listening and reading sample passages at
different levels. The audience will assign a level of the passages and check their
o After selecting the level appropriate passage, the audience will learn how to
attachment.
The second module will cover the DA interview for the four skills (speaking, reading,
and listening).
o The audience will be provided with the electronic forms of speaking, reading,
o Filled forms will be provided for reference and to have an idea how to complete
o In addition, there will be videos of a real interview the audience can watch.
Design
Learning Objectives
Diagnostic teaching is based on using diagnostic tools during the teaching-learning process in the
classroom, while the DA is mainly conducted outside the classroom. The benefit of the former is
the ability to identify the learners areas that require improvement within the teaching hour, and
that could be related to that specific hours contents or contexts. Also, it gives the teachers an
idea of their lesson plan, activities, teaching strategies, and what sort of modification can he
implement. As for the DA, the desired outcome is broader and gives an idea of what major issue
is taking place in the learning process of that particular learner. This DA is neither related to
specific teaching hour, nor to a certain topic. It gives an overall reflection of the learners
The goals of this DA familiarization training include and are not limited to:
areas of strength and areas that need improvement at the current time for that
specific learner/student.
Explain to the teachers the importance of ILR levels and their implementation in
Demonstrate to the instructors how to conduct DA interview for the three skills;
Illustrate to the teachers how to fill and complete DA report forms for the three
Identify and select level-appropriate reading and listening material following the
Create and construct DA questions, guided by the sample questions given, to use
them during the interview that will cover the global function, vocabulary,
taken into consideration before, during, and after the interview with 100%
accuracy.
Test Instruments
During the training, two types of test items will be used; declarative and procedural.
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 11
From memory, the audience will be given a matching quiz. They will match the
Given the IRL descriptors, the audience will read or listen to a passage and will
From memory and from the handout, the audience will organize the steps of a DA
Watching a real video of a DA interview, the audience will fill the form for that
Participants will revive feedback in different ways; the matching activity will give an
immediate feedback upon failure to match correctly. Feedback for assigning a level for passages
(reading and listening) will be given when the participant selects an incorrect level accompanied
by a justification. For the organizing activity, being procedural, the participants will receive
Following the analysis, a suitable delivery mode for the participants is the blended both
instructor-led and self-instruction. Thus, the former will be synchronous and the latter will be
asynchronous. The instructor-led may be required in the beginning as a delivery mode, later on,
the delivery mode for the training will be self-instructed. This mode will enhance knowledge
acquisition, knowledge retention and problem-solving that are required for this training. In
addition, self-instruction mode will reduce the out-of-class hours, and is more appropriate
because the contents are relatively stable, and the training will be repeated more often. A good
example of the online use of DA is the Online Diagnostic Assessment(ODA). The purpose of this
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 12
ODA is to assess students reading and listening skills. The idea of the OAD training supports
the applicability of performing an online training for the teachers on how to perform the DA.
Participants will receive instructions on the same platform the training will be delivered.
There will be step-by-step instructions on the opening page in addition to a PDF file attached to
download and print if required. All the material will be available for the participants to access
through their computers, as each participant is possessing a MacBook that contains the Mac side
and virtual window side, in case the participants are not familiar with Mac. I will consider using
Adobe Captivate in designing this training.At the DLI, the Universal Curriculum and Assessment
Tool (UCAT) is becoming popular as a Learning Management System (LMS). The UCAT can be
accessed online or it can be used as standalone, where the participants can download the training
and access it from anywhere. There will be certain material that will be provided using a
This training module follows the basics of the ADDIE model, the analysis is performed
for both the organization and learners(teachers). The design of the training module consists of
different sections; one section presents the ILR descriptors and activities related to them like
matching, assigning level for passages and constructing questions for the passages. The other
section of the module includes an introduction to DA interview protocol, the interview report
forms, step by step instruction on how to complete these forms. The last section will be a mock
develop this training is adapted from previous ILR training and from training designed for DA
The training material will include material previously developed in addition to newly
The proposal for the training depends upon needs assessment collected from four sources.
The Survey Monkey questionnaire was created and delivered to the faculty after taking all the
official permissions. The deliverable will include readings for the ILR and also readings about
the Modes for text typology. In addition, using google doc, I will create multiple choice
questions to check the trainees knowledge about ILR level descriptors. Also, I will prepare
level-appropriate texts for the trainees to rate. Then I will create a text-rating matching activity
for both reading and listening skills. Finally, I will upload electronic form so that the trainees
will be able to download and use in the future. Moving from design to development, I should
consider the applicability of the activities in the delivery moodle and the issue of navigation
between the pages. Also, I should keep in mind how to the feedback will be accessed in the
development phase.
Development Plan
This is a requirement for the master's degree in the MIST program, so the time constraint
is the timeline given to us by the program and I will try to complete it on time. The only cost will
be the required programs needed for the development of this model. Developing this module will
require more knowledge that I am expecting to gain during the coming terms both theoretical and
Implementation Plan
be appropriate to conduct Beta Test where the facilitators and the SMEs take the training in
order to test the contents and how they are connected to each other. In addition, they can give a
recommendation on applicability and timing, also, any technical issue will be addressed. This is
followed by Pilot Test where a number of targeted audience will take the training in a
classroom or conference room under the supervision of the trainer/facilitator. The facilitators,
preferably certified DA specialist and those undertook the training during the Beta Test, with
knowledge in the field in order to answer any question that the trainees may raise related to DA
Evaluation
Formative Evaluation
During the training, the trainees will have different forms of activities that they require to
submit in order to get feedback. These activities are constructed on Google forms, and thus when
submitting the form, results will appear to the trainer or facilitator and will give an idea about the
completeness of the training and how accurate was the performance of the trainee. In order to
have the best out of this, the designer will share Google forms with the trainees facilitators and
supervisor.
The design of the training is to be used by different trainees and at different occasions, so
to evaluate the usability of the training module, the trainee will answer evaluation questionnaire
(see Appendix B for the questionnaire). Also, looking at the training records will give an idea
about the compliance, that is whether the training material has been implemented as designed or
not.
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 15
Summative Evaluation
The trainee will answer a Kirkpatrick Level 1 (reaction ) evaluation questionnaire (see
Appendix C for the questionnaire). This evaluation will give the opportunity to have a feedback
on certain areas that require revision in the future. Level 2 (learning) evaluation will include test
instrument like matching ILR activities, assigning a rate to a reading or listening passage, and
filling a DA report after watching a mock DA interview. The facilitator will follow up on the
learners performance and whether the knowledge has been transformed and retained by the
learner. In Level 3 (application) evaluation, the trainees will apply the information they gained
in the classroom and in their job. This evaluation can be recognized by their direct supervisors
through reflection at their counseling reports and on students performance. As for evaluation
References
Continuum.
University academic commons. Working Papers in TESOL & Applied Linguistics, 11(1),
http://www.dliflc.edu/about/mission-vision/
Dunn, K. and Mulvenon, S. (2009). A critical review of research on formative assessment: the
http://pareonline.net/getvn.asp?v=14&n=7
Hanna, G. S., & Dettmer, P. A. (2004). Assessment for effective teaching: Using context-
King, A. (1993). From Sage on the Stage to Guide on the Side. College Teaching, 41(1),
30-35.
Mendoza, A. & Arandia, R. (2009). Language testing in Colombia: a call for more teacher
http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S1657-
07902009000200005&lng=en&tlng=en.
Mertler, C. (2003). Preservice versus inservice teachers' assessment literacy: Does classroom
Piskurich, G., M. (2015).Rapid instructional design: Learning ID fast and right, Hoboken, NJ:
Mediterranean Countries. Center for Educational Policy Studies Journal / CEPS Journal,
Appendix A
10. How familiar are you with using different software to record and edit
1 3 5
audios?
https://www.surveymonkey.com/r/CBB9SR2
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 19
Appendix B
DIAGNOSTIC ASSESSMENT FAMILIARIZATION 20
Appendix C
Level 1 Evaluation