Quality assurance

Defining “quality”
• It is subjective and dynamic.
• It has different meaning for different stakeholders.
• Relative terms .
Different definitions of quality in education are

1. Quality as value for money

2. Quality which meets the standard expectation of the

consumer

3. Quality as deemed fit for research and teaching in

education
An education can be compared to a
service industry.

The production side - the creation of the next

generation of appropriate medical graduates.

The perceptual side, - maintenance of the values and

principles of the medical educational institutions.
 Each stakeholder has its own expectation

COGNITIVE,PSYCHOMOTOR & AFFECTIVE

PASS/FAIL,DROUPOUTS,NO.OF STDS.
Individual development and position in society

Assess the quality of input, quality of process & quality of output.

Determine its quality

We can measure sugar %,Tannin level & sediment

Taste & smell ?
Quality assurance can be defined as the planned and systematic
activities put in place to ensure that quality requirements of an
educational program are fulfilled.

Example knee jerk

Formative
Summative
Extend this definition to medical Institution.

Indian
Medical
Graduate
Advantages of quality assurance
• Ensures that the product is relevant to the current
demands.

• The curriculum, the teaching learning methods, and

the assessment methods are reviewed.

• Revised on a continuous basis.

World Federation for Medical Education
(WFME)
“The motivation to maintain quality may come from external agencies
such as regulatory bodies, which prescribe minimum requirements or
from internal impetus.”
.
In the background of an intense
competition and globalization of the
medical profession, it is imperative
that our graduates are ready to face
the challenge to take up professional
positions anywhere in the world.
Feedback
• The current students ,
• Alumni on the courses offered,
• Structured and objective peer evaluation,

The use of wide range of assessment methods will improve

the quality of doctors produced.
Cognitive, Psychomotor and Affective evaluated are
evaluated
with well-defined learning outcomes
Aligning the teaching and learning to assessments.

A reliable and valid assessment adds to the quality

assurance system.
 Objectivising the evaluation.
Blue printing
Item analysis and
Question banking.
Blue printing
• Detailed plan of action

• Its template that guide the examiner

• It’s a Map and specification for an assessment .

What is the use of blue printing ?
• It makes assessment clear and transparent.
• Makes assessment fair to students
• Makes sampling content , competencies and
tools for assessment in a rational and balanced
manner.
• Maintain content validity.
Steps in preparing blue printing
1.Define purpose and scope of blue printing.

Formative /summative.

Use Miller’s Pyramid.

2.Give weightage to content , competency and
assessment method.

Importence (1) Score Frequency (F) Score

Not urgent 1 Rarely seen 1

Urgent 2 Relatively common 2

Emergency 3 Very common 3

3.Deciding on total weightage and number of items to be selected.

• Use formula W = I × F.
Topic Importance Frequency I×F W=I × F/T No.of
Items
N=W ×
100.
GIT 1 2 2 0.08 8(9)
Resp 2 2 4 0.17 17
CVS 3 3 9 0.37 37
CNS 3 3 9 0.37 37
TOTAL 24 1.00 100
In case of Basic science subjects,
weightage is decided by clinical application of topic

Topic with no clinical application W= 1

Topic with moderate clinical application W = 2

Topic with high clinical application w = 3.

 4.Prepare test specification table

Distribution of content area to the domains to be

assessed.
Content MCQ (1) SAQ(3) SEQ(5) LONG TOTAL
ESSAY
GIT 3 2×3=6 - - 9
Resp 1 2×3=6 2 × 5 =10 - 17
CVS 9 1×3=3 3 × 5 = 15 1 × 10 =10 37
CNS 12 - 3 × 5 = 15 1 × 10 = 10 37
25 15 40 20 100
 5.Prepare individual Questions.

Select from question bank.

For MCQS select after item analysis.

• MCQS are popular class performance.

• Item analysis is used for performence of MCQS.

Item analysis

•Difficulty index
•Discriminatory index
•Distractor effectiveness
•Reliability of the test
Procedure of item analysis. Facility value(P).
1. MCQS are given to group of students.
2. Divide the group into two based on
performances
I,e.-top 50% to bottom 50%.
( if > 100 stds take top 30% to bottom 30%)
3.Prepare a table for each item
Option No .selecting the option Against.

High achivers Low achievers

A 5 10
B 5 10
C 30 10
D 10 10
E nil 2
NO Nil 8
RESPONSE
50 50
4. Count the number of students who have responded correctly to
each question.

5. Add the number of students who have responded correctly to each

question in the two groups.

6. Express this as a percentage of the total number of students (who

responded) in the two groups.
CALCULATION OF ‘p’ = H+L X 100
T

Where, H= No. of correct responses in the upper group

L = No. of correct responses in the lower group

Acceptable 30-60%,
Easy Question >70%
Difficult. Question,<30%
DISCRIMINATION INDEX(D)
D = H-L/T
Acceptable 0.25-0.35, < 0.2 are poor.

• The ability of a question to distinguish between more able and less

able students.

• Higher the index better is the discrimination.

INTERPRETATION OF ‘D’

Reject R evise G ood E xcellent

0 0.05 0.15 0.25 0.35
Distracter effectiveness
• GOOD DISTRACTOR
Lower ability students pick it as the correct answer.
Ideal 20 – 30%
Satisfactory 10 – 20%

POOR DISTRACTOR
Picked by higher ability students.
< 5% for each distractor.
CALCULATION OF DISTRACTOR EFFECTIVENESS
A B C D Blank (NR)
4 2 1 13 0

•Distractor A = 4 X 100 / 20 = 20%

• Distractor B = 2 X 100 / 20 = 10%
• Distractor C = 1 X 100 / 20 = 5%
D is the key
 ITEM LEVELS

ITEM CLASS p D
Level I (Best) 30 – 60% + 0.20 or higher
Level II (Very easy) 60 – 91% + 0.15 to + 0.20
Level III (Very 30 – 44% + 0.10 to + 0.15
difficult)
Level IV (Too easy or < 30 or > 91 Any discrimination
too difficult)
IDEAL TEST (Question paper)

•Level – I questions – 80%

•Level – II questions – 10%
•Level – III questions – 10%
•Never include Level – IV questions
ITEM CARD
• Subject: Community Medicine
• Type of question: One best response
• Reference number: 1
• Topic: Man and Medicine
• Time of answering: 1 minute
• Mark: 1
• Prepared by: __________________________
• Question: Provision of free medical care to the people
at Government expense is known as
• Alternatives: A) Socialized Medicine
B) Social Therapy
C) Social Medicine
D) Social Insurance
• Key: A
 ITEM BANKING
Ref. Date Group Size of Chosen answers p D Level
no. group
A B C D
1 13/12/10 115 H (31) 25 2 2 2 57.38 0.49 I
L (31) 10 3 2 16

2 11/11/10 115 H (31) 2 22 6 0 50.00 0.63 I

NR (1)
L (31) 1 5 13 5
NR (7)
Uses of item analysis
• Creation of item banks and prepare item card
• Easy questions should be in the beginning to improve
morale.
• Use of class room teaching

• Easy items - skipped over or treated lightly

• Difficult items - explained more fully
• Defective items - pointed out
 Example 1
• The basic abnormality in case thalassemia is Key is C.
A. Abnormality of red cell membrane
B. Enzyme deficiency
C. Deficient synthesis of normal haemoglobin
D. Synthesis of abnormal haemoglobin
E. Antigen antibody reaction

Batch 1 P = 27.5% d= 0.45

Batch 2 P = 27.5%. d= 0.15
Batch 3. P = 22.5%. d = 0.35
Batch 4 P = 20. % d=0.10
Example 2
• Calcitonin is produced by Key. D.

A Cuboidal cells of the thyroid

B . Chief cells of parathyroid
C . Oxyphil cells of parathyroid
D . Parafolicular cells of the thyroid
E . None of the above

Batch 1 p = 77.5% d = 0.05

Batch 2 p = 77.5% d = 0.45
Batch 3. p = 62.5% d = 0.65
Batch 4 p = 60. %. d = 0.50
Example 3
• Fissure in Ano is believed to be due to weakness of

A. Subcutaneous part of the external sphincter

B. Superficial part of anal sphincter.
C. Deep part of external anal sphincter
D internal anal sphincter
E Puborectalis sling.

Low facility value

Negative discriminative value
New items

• we actually use a test, that have the data related to each item available.
• Calculate various indices related to new questions is to give them a trial run
as dummies.
• Thus, in an actual test situation, the first 20 questions out of 100 can be
new questions.
• The students answer them, they are marked on them but scores obtained
on these 20 questions are not used for computing the results.
• They are used only for calculating the FV and DI and only when a question
has been found to have a satisfactory level of FV and DI,
Advantages of question banking
• Increase the skills of teachers
• Right kind of question requires a thorough understanding of the
subject matter
• Creating question banks will bring transparency into the evaluation
process and make inter college/ university comparisons easier.
• This will help to build the faith of the society in examination systems
and maintain uniform standards of teaching.
References
• “Quality assurance in medical education, Medha A. Joshi,Quality Assurance
Unit, Health Sciences, Gokula Education Foundation (Medical), M.S.
Ramaiah Campus, BEL Road, Bangalore, India. E-mail: medhaj@gmail.com”
• Excerpt From: “Quality assurance in medical education”. Apple Books.
• Tariq, M., Ali, S. A. (2014). Quality assurance and its application in medical
education.. Journal of the College of Physicians and Surgeons--Pakistan,
24(3), 151-152.
• COURSE MANUAL,CONTENTS,Workshop on Basic Medical Education
Technologies ,JJM Medical College, Davangere.
• Singh,T.Singh D.Gupta P.Indian academy of peadiatrics, Principles of
Medical education,fourth edition. Jaypee brothers medical publishers.