RESEARCH, EVALUATION

RESEARCH EVALUATION,
MONITORING AND SUPERVISION

Sarva Siksha Abhiyan

by
by
Neelam Shami Rao

Department of school Education and Literacy, 1

MHRD, Govt. of India
 P
Presentation
t ti Contents
C t t

™ SSA Goals
™ Monitoring Principles Under SSA
™ Monitoring
g Systems
y Under SSA
™ Monitoring Institute
™ DISE
™ QMT
™ Researches
™ Results Framework
Department of school Education and Literacy, 2
MHRD, Govt. of India
 SSA GOALS

™ All children in school, Education Guarantee

Centre Alternate School
Centre, School, ‘Back-to-School’
Back-to-School
camp by 200
™ Bridging gender and social gaps in:
• Primary : 2007
• Elementary : 2010
™ Universal retention by 2010
™ Focus on satisfactory quality with emphasis
on education for life
Department of school Education and Literacy, 3
MHRD, Govt. of India
Monitoring Principles Under SSA

SSA Envisages
™ Community based Monitoring system

™ EMIS to Incorporate Correlation of

school level data with community
b d information
based i f i from
f microplanning
i l i
y
and surveys
™ PMIS for periodic reporting system

Department of school Education and Literacy, 4

MHRD, Govt. of India
 Contd/…(Monitoring Principles Under SSA)

Basic Principles in Monitoring

™ Community Ownership
™ Periodic quality checks by external
teams
™ Independent feedback on programme
implementation
™ Provision
P i i off upto t RRs 1500/-
1500/- per school
h l
per year for REMS
Department of school Education and Literacy, 5
MHRD, Govt. of India
Monitoring Systems Under
SSA
™ District Level Committee comprising local
public representatives for monitoring
™ Progress against key monthly indicators and
quarterly
q y p
progress
g report
p from States to
Government of India
India..
™ Educational MIS (DISE) - annual school school--
b
based d data
d t .
data.
™ Educational Development Index (EDI) To
assess state progress towards UEE UEE..
™ School Report Cards developed at
www..nuepa
www nuepa..org
org..
Department of school Education and Literacy, 6
MHRD, Govt. of India
 M it i Systems
Monitoring S t Under
U d SSA

™ 41 Independent
p National Social Science
Institutions to make independent &
regular field visits to monitor
performance..
performance
™ An independent Joint Review Mission
reviews the progress of SSA twice a year
™ Pupil achievement level sample surveys
are conducted everyy three yyears byy
NCERT to check increase in learning
levels
Department of school Education and Literacy, 7
MHRD, Govt. of India
 Contd/…(Monitoring Systems Under SSA)

™ Several independent
assessments/studies carried out for
independent feedback on implementation of
SSA
¾ Out of School Study, 2005.
¾ Student & Teacher attendance rates 2007
¾ study on Para-
Para-Teachers 2007.
™ Independent concurrent financial reviews
commissioned
i i dbby th
the Mi
Ministry.
i t 32 StStates/
t /
UTs covered so far & review in respect of 3
States/UTs under progress.
progress
Department of school Education and Literacy, 8
MHRD, Govt. of India
 Contd/…(Monitoring Systems Under SSA)

™ A set of quality monitoring tools, developed in

collaboration with NCERT,, to provide
p quarterly
q y and
annual information on several quality related indices of
SSA has been rolled out, with 29 States reports
received,, covering:
g -
¾ Student enrolment and actual attendance
¾ Pupil achievement levels
¾ Teacher availability and teacher training
™ Classroom Practices
™ A d i supervisions
Academic i i off schools
h l b by Cl
Cluster
t
and Block Resource Centres.
™ Community perceptions of school functioning.
functioning
™ Regular reviewsDepartment
at secretary level and field visits
of school Education and Literacy,
MHRD, Govt. of India
9
 Monitoring Institutes to give
Independent Feedback – triangulation of
i f
information
ti received
i d from
f States
St t through
th h
PMIS/ DISE/QMT
™ To assess the progress of implementation of approved plans at District and
State level.
To sample
p check progress
p g in achievement of some keyy out come indicators.
To verify process and procedures undertaken for implementation of SSA.
The monitoring institutions engaged for a 2 year period.
5% of Elementary Schools/EGS/AIE centre will be visited
during period of two years.
Si monthly
Six thl reports
t covering
i 25% off the
th number
b off districts
di t i t
allotted to the Monitoring Institutes.
„ SPDs are required
q to share the findings
g of MIs with the District level officer
in their Monthly meetings for remedial action
action..
Department of school Education and Literacy, 10
„ 41 Institutes provide feedback till now more
MHRD, Govt. than 400 districts covered
of India
 Sample of Observations from MIs
Textbooks
™ All categories of children in the districts received
text books within one month of the beginning of
the academic year 2006-
2006-07 (Gauwahati
University)
™ 100 per cent children received textbooks (IDS,
Jaipur)
™ Text books not been supplied within one month
of the opening of the schools in any sampled
schools
h l (A(A..N. Sinha
Si h Institute,
I tit t Patna) P t )
™ Full sets of textbooks have not been received
and distributed to the Upper primary students
even up to Department
September
MHRD, Govt. of India
2007(Kurukshetra
2007(Kurukshetra
of school Education and Literacy, 11

University, Haryana)
District Information System
of education (DISE)
™ DISE started in 1990s
™ Scaled up in all districts since 2004
™ Provide a time series data
™ Base for evidence based p planning
g
™ School based information system
¾ Physical
y infrastructure and facilities
¾ Availability of teachers
¾ Enrollment and academic performance
p
¾ Training of teachers

Department of school Education and Literacy, 12

MHRD, Govt. of India
District Information System
of education (DISE)
™ 30th September as a record date
™ Computerised indigenous and flexible
™ Software responds to field requirement
™ Data Utilised for monitoring as well as planning
™ Data Triangulation
™ 5% sample check by Independent Agency for data
veracity
™ Consistency checks inbuilt in the system

Department of school Education and Literacy, 13

MHRD, Govt. of India
 Educational Development Index (EDI)

• Educational Development Index

developed based on District
Information System of Education
(DISE) in 2006-07
• Ranking on 4 parameters – Access,
Infrastructure, Teachers, Outcome
((GER, GPI etc))
• No. of States on low EDI Value
reducing
EDI No. of States
Value > 0.75 0.625-
0.625- <0.62
0.75 5
2007--08 3
2007 16 16
2006--07 0
2006 13 22
• Ranking of districts on similar basis
• More SSA funds for low EDI areas.
Department of school Education and Literacy, 14
1-9 19-27 10-18 28-35 MHRD, Govt. of India
Tracking Quality Outcomes
„ National Sample Surveys for Pupil Achievement
every
y3yyears
„ Base line studies for Class III, V, VII and VIII
completed
p
„ Fund set up to enhance NCERT capacities for
modernizing
g testing
g methodologies
g
„ Second round of achievement surveys
surveys - Class V
completed,, Class III & VII/VIII under way.
completed way
y.
„ National Curriculum Framework 2005 rolling out
„ Measurable outcomes for Class III revised

Department of school Education and Literacy, 15

MHRD, Govt. of India
MONITORING FOR QUALITY DIMENSIONS
UNDER SSA: Quality Monitoring Tools

STATE

DISTRICT

BLOCK

CLUSTER

SCHOOL/ COMMUNITY

™Qualityoutcome:
¾ Learners’ Achievement in scholastic and co-
co-
scholastic Department of school Education and Literacy, 16
MHRD, Govt. of India
¾Quality of outcome, input, processes,
processes,
Objectives of Quality
Monitoring tools
„ The Objective of monitoring has been to:
„ perform
f a ffacilitative
ili i andd supportivei role
l rather
h
„ than supervisory role in strengthening the
existing
i i systems at variousi l l.
levels
levels.
„ focus on processes rather than on physical
targets
„ provide feed back to States for taking corrective
measures andd evolving
l i strategies
i for
f managing
i
Quality
Department of school Education and Literacy, 17
MHRD, Govt. of India
Dimensions
i i f Quality
for Q li Elementary
l Education
d i
(Components of Formats)

• Basic Infrastructure and other Facilities

• Management
M and
dCCommunity
i Support
S
• School and Classroom Environment
• Curriculum
CurriculumandTeachingLearningMaterial
and Teaching Learning Material
Quality
Dimensions • Teacher and Teacher Preparation
of • Opportunity Time (Teaching-Learning
(Teaching LearningTime)
Time)
Elementary
• Classroom Practices and Processes
Education
• Learners
Learners' Assessment,
Assessment MonitoringandSupervision
Monitoring and Supervision

Department of school Education and Literacy, 18

MHRD, Govt. of India
Two way flow
fl off Information
f i
N ational level

State level

SP O + SCER T

D istrict
i t i t level
l l

D P O + D IET

B RC level

CR C level

School Level / C omm unity Level

Department of school Education and Literacy, 19

MHRD, Govt. of India
 Pre--requisites for Effective Monitoring
Pre
„ Structures (BRCs, CRCs) are in place.
„ Personnel are in position at all places (Teachers,
CRCCs,BRCCs, DEO, DIET faculty, etc.)
„ J b profiles
Job fil are delineated
d li t d properly. l
„ Capacities are built-
built-up among personnel at different levels.
„ Systems are geared-
geared-up to enable the personnel to
discharge their responsibilities.
„ Need identification exercise in respect of capacity building of
personnel is undertaken regularly.
„ Organizational climate is conducive.
„ Data are gathered timely on Quality Monitoring Formats and
analyzed/ shared at each level.

Department of school Education and Literacy, 20

MHRD, Govt. of India
 Results Framework

Against all four Goals

• Outcome Indicator
• Baseline
• Target Value for the next three year
• Frequency
q y & Report
p
• Data Collection Instruments
• Responsibility for
f Data Collection
C

Department of school Education and Literacy, 21

MHRD, Govt. of India
Researche: Triangulating
Information
„ Rs 1,500/-
1,500/- per school – Rs 200 at the National level
„ Committee for approval of research projects (CARP)
under chairmanship of Secreatry SE&L at National
level to decide on research agenda
„ CARP State level
„ Important information for triangulation
– Out of school children sample
p studyy 2005
– Study on Student and teacher attendance
– Study on para teachers etc
„ Many studies in pipeline
„ Research including action researches at the district
and subdistrict level
Department of school Education and Literacy, 22
MHRD, Govt. of India
 Tracking Quality: Triangulating Information
• Tracking teacher and student attendance
• States commitment to revamp teacher accountability mechanisms

Students Attendance 68.5% Teacher Attendance 81.7%

Primary

< 70%
70 – 80%
80 – 90%
Department of school Education and Literacy, 23
MHRD, Govt. of India
> 90%
Goal I : All children in school / EGS centres /
Alternative and Innovative Education centres

Outcome Indicator:
Indicator Number of children aged 6-14
years not enrolled in School / EGS Centres / AIE
Centres

Baseline: 13.4
13 4 million children were out of school
(Independent Study 2005)

Target: Reduce OoSC to Zero

Frequency & Report:

¾ Annual PMIS Report
¾ Independent Sample Survey on OoSC in 2008
2008-09
09
Department of school Education and Literacy, 24
MHRD, Govt. of India
 Goal 4 :Contd/….

Outcome Indicator: Process indicators on quality -

Attendance Rates of Students and Teachers
Baseline:
Student Attendance level is 70% at primary and 75% at upper
primary
Teacher Attendance level is 81% at primary and 80% at
upper primary
(2006 : Independent Study 17 States)
Target Value:
Increase student and teacher attendance to more than 90%
at all levels
Frequency & Report:
Annual PMIS/ QMT Reports
p disaggregated
gg g by
y States
Independent Sample Department
Study onEducation
of school studentand Literacy, and teacher
MHRD, Govt. of India
25

attendance to be repeated in 2009-10

 Goal 4 :Contd/….
Outcome Indicator: National student achievement level
outcomes
Baseline:
Learning levels for Class III -58 % in Maths & 63 % Language
Learning levels for Class V 46% in Maths, 58 % in Language,
50%
0% iin EVS
Learning levels for Class VII/VIII - 30% / 38% in Maths, 52% /
52% in Language,
Language 36% / 40 in Science and 33% / 40% in
Social Science
( NCERT National Assessment Sample Survey -1 1st round)
Target: Improve Learning levels in each round.
Frequency & Report:
2nd round outcomes for Class III and Class VII in 2008 - 09
3rd round for Class V in 2009-10
2009 10
3rd round for Class III Department
and VII/. of school VIII
Educationin 2010 - 2011
and Literacy, 26
MHRD, Govt. of India
 Tracking Quality Outcomes Cntd/...
State Rankings for overall MAS Class V
Ranks States • Improvement in learning
achievement in all the subjects in
1 to 5 West Bengal, Karnataka, class V.
Gujarat, Jharkhand, Tripura
Classes Maths Lang
Lang. EVS
6 to 10 Uttar Pradesh, Maharastra,
Tamil Nadu, Delhi, Madhya Class V 46.51 58.57 50.30
Pradesh (1st
11 to 15 Oi
Orissa, Himachal
Hi h l Pradesh,
P d h round))
round
Mizoram, Manipur, Rajasthan Class V 48.46 60.31 52.19
(2nd
16 to 20 Kerala, Punjab, Arunachal
Pradesh, Nagaland, Haryana
round))
round

21 to 25 Andhra Pradesh, D & N H, • Goa, HP, Gujarat, J& K, UP,

Daman & Diu, Chandigarh,
Jammu & Kashmir Karnataka Kerala,
Karnataka, Kerala improved by
more than 5 percent points
26 to 30 Meghalaya, Assam,
Uttarakhand, Sikkim, Goa
31 to 33 Chhattisgarh, Puducherry, A
& N Island, Department of school Education and Literacy, 27
MHRD, Govt. of India
Department of school Education and Literacy, 28
MHRD, Govt. of India