TIMSS & PISA

15 Mei 2013

Objectives for this discussion

1. Present findings from roundtable with universities and school

students.
2. Present analysis of questionnaires administered during TIMSS 2011
- Factors influencing students performance in Science
3. Present findings of benchmarking with other countries
2

4. Present findings of item analysis

5. Decision on format of Mocktest

6. Present work done on the use of TIMSS & PISA questions in teaching
and learning to enhance students acquisition of HOTS.

| 1

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 2

We use the heuristic of Problem Solving and the Risk Management

Strategy of FADE (Focus, Analyse, Develop, Executive) to tackle the
issue of TIMSS dan PISA
EXECUTE
Suggested solutions

We use scientific method to identify the problem

FOCUS

Define and
Identify the
Problem
Malaysia perform badly
in TIMSS and PISA,
bottom 1/3 in TIMSS
2011

ANALYSE

Analyse the
Problem
Various factors* influencing
performance. 70% TIMSS
and PISA questions
HOTS(application &
reasoning)

Assess possible
solutions from time
to time based on
new findings

Develop Action
Plan

*Students & teachers

unfamiliar with TIMSS &

i
c
Identifying
Possible
Solutions

PISA questions

Solutions
suggested for
each factor

DEVELOP

d
Evaluating &
Selecting the
Best solutions

ii

administrative matter
Ensuring advantage through
sampling and participate in
decision making

iii

*Gap in Curriculum
Alligning KSSM with TIMSS
and PISA content

Using scientific method to identify solutions to the problem

Briefing to all secondary

school principles and
teachers.
Using TIMSS & PISA
questions in teaching and
learning
Mock Test
Item
analysis and
*Sampling

*Teachers lack of content

Roundtable to obtain feedback and suggestions from stakeholders.
Analyse existing studies to identify problem and solutions, conduct research
on identified areas
Benchmark with high performing countries on the method they adopted in
preparing their students to answer TIMSS and PISA questions
Short term measures/Quick win

Long term solution

iv and pedagogical skills

Upskilling S & M teachers in
HOTS

*Lack of teaching
resources in HOTS

Develop online teaching

resources
| 3

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 4

Alligning TIMSS and PISA initiative with HOTS

Long term
Short term

There is a need to allign suggested programs within the three

initiatives

Preparing
Students
with TIMSS
PISA type
questions

Comprehensive in
house training
Use TIMSS PISA
type questions in
T&L

Teacher
Upskilling
Program

Mock Test

HOTS
TIMSS PISA

Provide
comprehencive
on line resource
alligned with VLE
requirement

Teaching &
Learning
Resource

Curriculum
what to teach

Include topics in TIMSS

and PISA in KSSM

| 5

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 6

We have started in Nov 2012. Taking stock of what we have done

Task already carried out
Raising
I awareness of
TIMSS and PISA
Dedicated time
II for TIMSS PISA
questions

III
Mock test
IV
Sampling and
administration
V
Gap in
Curriculum
VI Teacher
upskilling
program
VII
Teaching
resources
VIII
Research

Task to be carried out

1 science and 1 mathematics Form 1 teachers from

all school attended briefing conducted by BPK and
subsequently did in-house briefing. Briefing to
school principals
TIMSS and PISA released items categorized and
uploaded, teachers instructed to let students
answer these questions after each topic.
Letter to inform state to instruct schools to use
TIMSS PISA released items
Guide to use TIMSS PISA questions being prepared
and piloted.
Preparing HOTS item similar to TIMSS and
Mock test in June and September to be taken by all
Form 1 and Form 2 students
Ensuring advantage through sampling and
participate in TIMSS decision making meetings

Alligning subtopics in TIMSS and PISA with KSSM

Initial discussion with RECSAM and other experts

on course content
Overall plan on teacher upskilling

Execute the teacher upskilling program

RFQ prepared and disemminate to interested

company with excellent track record who have
worked with successful TIMSS PISA countries
Overll plan on teaching resources

Execute the teaching resources program

Item analysis of Malaysias achievement

Compile research findings

Conduct research to identify factors influencing

TIMSS PISA performance

| 7

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 8

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 9

Perbincangan Meja Bulat Berkaitan TIMSS dan PISA Siri 2/2013

bersama Pemimpin Pelajar pada 8 April 2013
Antara Panel
jemputan dalam
kalangan Pelajar
IPTA / IPTS /
Sekolah:

Persatuan
Mahasiswa
UKM

mengenal pasti 4 perkara untuk dipertimbangkan dalam inisiatif TIMSS / PISA

1) Wujudkan ekosistem pembelajaran yang saling berinteraksi.

Mewujudkan rangsangan rasa-ingin-tahu dan rasa-ingin-buat.

Mewujudkan sekolah aplikasi untuk memenuhi rasa-ingin-buat.
Memerlukan tempat utk berbincang & masa utk belajar yang mencukupi di sekolah.
Hadkan bilangan murid dalam kelas untuk lebih tumpuan.

2) Memberi fokus kpd pembinaan sikap positif dalam pembelajaran.

MPP UNITEN
Sekolah
Sultan Alam
Shah
Sekolah Seri
Puteri

SMA (P)
Kajang

Menggalakkan minat membaca pelbagai bahan bacaan.

3) Suasana pdp yang selari dengan zaman.
Menggunakan bahan sumber dalam persekitaran murid secara optimum
termasuk aplikasi teknologi trend terkini (iPad, Tablet, FB, twitter, YouTube, web,
dll) dalam pdp.
Pembelajaran perlu menyeronokkan dan mencabar.
Memperkasa aritmetik mental dan amali sains.
Merapatkan jurang antara murid dengan mengadakan program bimbingan rakan
sebaya.

4) Soalan pentaksiran yang bukan-rutin.

SM (C) Kuen
Cheng

Memerlukan soalan yang menguji pemikiran kritikal dan bukan-rutin.

Tugasan yang diberi berkaitan dgn masalah dalam situasi harian atau kajian kes.
| 10

Meja Bulat TIMSS/PISA yang pertama telah dilaksanakan bersama dengan

ahli akademik dan guru cemerlang pada 6 Feb 2013
Meja bulat

mengenalpasti 7 fokus untuk dirangkumi dalam pelan kerja inisiatif TIMSS/PISA terkini

dengan ahli akademik

terkemuka dan guru
pakar berikut
dikendalikan

Prof. Tan Sri Dato

Dzulkifli Abdul
Razak (AIU)

Prof. Dato Dr.

Mohd. Yusof Hj.
Othman (UKM)

Dr. Ho Chee
Cheong (ASM)

Prof. Dr. Ewe

Hong Tat (UTAR)

Mempertingkatkan usaha penyelidikan ke atas pencapaian Malaysia dalam TIMSS/PISA

Pendekatan pdp yang kreatif sesuai dengan mindset generasi remaja terkini

Penglibatan yang lebih aktif dalam urusan pentadbiran TIMSS/PISA antarabangasa

6
Guru pakar Sains
dan Guru pakar
Matematik

Menjalankan bimbingan berasaskan klinikal/on-site yang lebih bermakna kepada guru berbanding dengan latihan
jurulatih secara cascade, kurangkan latihan yang bersifat teoretikal

Prof. Dr. Rosnani

Hashim (IIUM)

Menyediakan laman sesawang khusus sebagai pusat bahan sumber untuk guru merangkumi ciri seperti item bank
soalan KBAT, video klip pdp dan contoh-contoh tugasan yang mencabar

Mengubah kaedah latihan dalam perkhidmatan guru kepada yang lebih bersifat klinikal

Dr Soon Ting Kueh

(BTP)

Mengambil kira cara remaja kini berfikir, merekabentuk program berdasarkan perspektif remaja dan bukan sahaja
dari sudut pandangan orang dewasa
Menjadikan pdp lebih menarik, menyediakan tugasan mencabar yang dapat meningkatkan minat murid serta
sikap yang positif terhadap Sains dan Matematik

Menyediakan satu pusat bahan sumber digital untuk guru Sains dan Matematik

Membentuk kumpulan penyelidik untuk kajian menyeluruh pelbagai tahap (multilevel analysis) berkaitan pencapaian
Malaysia dalam TIMSS/PISA melibatkan KPM, universiti, dan agensi lain

Memainkan peranan proaktif dalam pentadbiran TIMSS/PISA, tingkatkan penglibatkan Malaysia dalam membuat
keputusan pembangunan item dan urusan pemarkahan.
Pengesanan pencapaian KPI TIMSS dan PISA secara berterusan

Menjalankan Ujian Saringan mirip TIMSS/PISA untuk proses pengesanan dan membiasakan murid dengan
bentuk soalan TIMSS/PISA
Menyediakan insentif dan motivasi kepada guru yang mengambil bahagian dalam TIMSS/PISA
Mengubah bentuk soalan dalam peperiksaan awam

Soalan peperiksaan tidak berbentuk stereotaip, skema pemarkahan tidak boleh terlampau
spesifik dan rigid.

| 11

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 12

Common policy reactions to PISA scores

Increasing National Assessments & Oversight

Switzerland required all schools to add testing in 4th and

8th grades to ensure compliance with new nationwide
standards
Slovakia implemented new national measurements of
reading and mathematics modeled after PISA
Sweden introduced national tests in biology, chemistry and
physics for 12- and 15-year-olds, and developed diagnostic
material in science for the earlier years
The Canadian national assessment was redesigned in
2007 to better complement the PISA schedule, target
population and subject areas covered
South Korean test developers and subject area experts
are trying build assessments items and assessment
frameworks, based on the PISA approach and content
Singapore began making changes in the late 1990s to the
item types in national exams to test for higher-order
thinking, using PISA items as models

Addressing Educational Inequality

Strengthening Standards & Curricula

Improving Teaching Quality

Spain, Greece and Slovakia have all implemented

Finnish approaches to selecting and training high quality
teachers after receiving lower PISA scores
New focus on recruiting high performers
New focus on rigorous pre-service training

South Korea took steps to lessen gender and

income achievement gaps new policies included
free tutoring resources
Denmark implemented strategies to target socioeconomically disadvantaged and immigrant students
Germany enacted policies making it easier for more
students to attend competitive schools

Switzerland enacted legislation that obligated all 26

cantonments to operate with the same education
attainment goals and standards
Japan shifted from a low pressure curriculum and
added more material to textbooks and exams
Greece added more PISA elements to the national
curriculum
In the United States, PISA was used to help build
the Common Core State Standards
Norway deepened their focus on basic skills in
reading and mathematics when redesigning subject
curricula
Luxemburg redesigned curricula and assessments
to student use of knowledge in real-life situations

SOURCE: European Journal of Education The OECD PISA Study as a Soft Power in Education? Lessons from Switzerland and the US (link);
OECD The Policy Impact of PISA; press search

| 13

Japan case study

Background

Selected Approaches

The proportion of top

performers in PISA reading
has increased from ~10% to
above 13% since 2000
Compared with 2000, more
Japanese students now read
for enjoyment and are
motivated to read
Compared with 2000
data, more students now
like talking about books
and cite reading as a
favorite hobby
Fewer students report
that reading is a waste of
time for, or find it hard to
finish books
Science and math TIMSS
scores have dropped slightly,
but Japanese scores have
remained high overall
PISA results show that socioeconomic disadvantage does
not have as strong an impact
on student performance in
Japan as it does in other
countries

In 2005 the Japanese Ministry of Education, Culture, Sports, Science and

Technology (MEXT) launched a project to improve PISA reading proficiency,
using three main objectives
Develop critical reading skills
Improve writing skills on students own thoughts
Provide support so that students read a wider variety of texts
The National Course of Study has undergone several shifts intended to
increase PISA and TIMSS scores
Literacy class instruction hours were increased in 2008 along with a new
emphasis on the importance of students ability to their thoughts
2003 revisions encouraged a focus on advanced learning, instead of
setting the bar at basic proficiency
2008 reforms increased instruction hours in mathematics and science in
order to ensure the comprehension of specific concepts
Increased class hours provided students with the opportunity to apply
math and science knowledge in practical lab settings
In 2010 the Education Ministry began enlarging textbooks and raising
expectations as part of a back-to-basics drive to boost PISA scores
Science and math textbooks received 60 percent more pages
1-2 hours of school were added to each week, depending on the grade
Tutoring and extra help has also contributed to high Japanese scores
The 1998 Course of Study encouraged supplemental learning through
after school or small-group guidance
1998 reforms encouraged class assignments based upon each students
mastery and proficiency in each subject
Japan is one of 16 OECD countries in which disadvantaged schools have
more favorable student-teacher ratios than socio-economically advantaged
schools

SOURCE: Strong Performers and Successful Reformers In Education: Lessons from PISA for Japan (link), Associated Press Back to basics
or cram school? To reverse sliding rank, Japan adds 1,200 pages to textbooks (link)

| 14

South Korea case study (1/2)

Selected approaches which impact PISA and TIMSS scores
Improvements specifically impactful for PISA Reading Score
The new national curriculum put more emphasis on critical and creative thinking skills through reading and writing
Reading assessment more focused on thinking ability
The university entrance system changed
Essay test that assesses both writing skills and logical thinking abilities introduced
Curriculum Alignment & Teaching Strategies
Students in South Korea begin learning algebra, geometry, physics and chemistry between ages 12 and 14
Math concepts are taught in a cohesive manner instead of in a disparate, sequential order, leading to deeper learning
Tutoring & Hogwans
South Korea has the largest system of private tutoring in the world
In 2011, approximately 72 percent of Korean students received tutoring, such as private lessons or for-profit cram"
schools, for an average of 7 hours per week
Families spend an average of 10 percent of their annual incomes on extra educational services like tutoring
In order to ensure that children at all socioeconomic levels have access to tutors, the South Korean government is
supporting efforts to increase free public access to supplemental academic lessons
These efforts include a cyber learning program that provides free customized, online tutoring programs
High Stakes Exam Culture
High school entrance exams and the college entrance exams are very high stakes
Most forms opportunity, from marriage prospects to job prospects, depends on which high school you got into and
which college you went to
As a result, students are highly incentivized to learn study techniques and exam taking-techniques, and schools and
parents prioritize these skills
The number of daily and weekly study hours per student is higher than in any other OECD country
National Exams shape educational policy more than PISA and TIMSS scores
South Koreas PISA science has fallen from the top of the league tables in the 2000 assessments to sixth place in the
2009 rankings
This is most likely due instructional time for science was reduced, science subjects became optional in school, the
university entrance examinations no longer require students to take exams in science and the science and
technology professions have become less attractive to Korean students
SOURCE: EdWeek Out-Of-School Classes Provide Edge (link);Center on International Education Benchmarking: South Korea Overview (link)

| 15

South Korea case study (2/2)

The South Korean Government Has Taken An Active Role in the Tutoring Space
The South Korean Government fears that low-income students may not be able to compete
In 2011 families in the highest income quintile spent six times more on private tutoring than those in the lowest
The gap between the household expenditure on private tutoring of high- and low-income families continues to widen
The Government has implemented a variety of measures to equalize the tutoring space
Regulating operating hours of private tutoring institutions
As of 2010 evening tutoring schools have been required to close by 10 p.m. in Seoul and other metro areas
Providing price subsidies to low income households
Providing free online tutoring options
In 2004, EBS, a government-run educational TV network, opened a Web site that offered free tutorials on the national
exam
EBS has ~3 million members
Imposing taxes on private tutoring and using resulting tax revenues to provide vouchers for private tutoring to children from
low income households
Vouchers have been provided to children of low income families since 2006
The vouchers can be used for after-school programs that are held in schools after the official school day is over.

SOURCE: Press search; University of Pennsylvania Private Tutoring and Educational Inequality (link)

| 16

Germany case study

Background

The German public and policy

makers assumed that the German
education system was one of the
best in the world
It was a system thought to be
fair, effective and rigorous
The first warning came with poor
performance on TIMSS scores in
1995, but the German public did not
notice
The PISA-2000 results were a
shock and disappointing to the
public
No one expected that one quarter
of German 15-year-olds could not
read fluently. And worse yet, the
PISA results showed that German
at-risk students performance was
among the worst in the world.
PISA data and analyses showed
that German standards were low
and highly variable
The German States started many
programs and projects at the
regional/local level as well as at the
federal level to approach the
problems that PISA had revealed

Selected Approaches

Revision of curriculum standards, often to include and emphasize PISAlike competencies

Additional standards were added to secondary for mathematics,
German, French, English, biology, chemistry and physics
PISA has led to the growing importance of principles such as
outcome control, competence orientation and external assessment
Oversight commissions and protocols were established to ensure the
new standards were adopted and implemented across the country
Changes to the school structure to reduce inequality
German states redesigned the enrollment system to enable students
to more easily attend the more rigorous Gymnasien-style schools
A new balance of accountability and autonomy
Germany introduced a testing system which provides school leaders
with more autonomy in how they design and run schools
But the system requires they take part in accountability protocols
Addressed the high correlation between childrens command of German
on entering elementary school and academic performance
The federal government introduced programs, supervised by the
states and run by municipalities or charities, to significantly increase
the level of organized, high quality, affordable language training for
children whose families did not speak fluent German at home
This language training is offered to Kindergarten-level students so
that they are as fluent as native Germans by elementary school
Increased access to high quality preschool and early childhood
education
Germany created a right to a place in Kindergarten for every child
from the age of three until they begin elementary school
Germany moved to longer school days

SOURCE: OCED The Impact of PISA 2006 on the Education Policy Debate; OECD The Policy Impact of PISA; Edutopia Germany Takes On
Education Reform (Education Everywhere Series)

| 17

Denmark case study

Background

Denmark had historically had many

educational strengths
A tradition of democracy
National commitment to education
Adequate investment in education
Ample school choice
Involved parents and local communities
Low 2003 scores and evidence of an equity
gap in Denmark sparked public debate
Danish school system had traditionally
placed a large amount of trust in
municipalities and schools
The Ministry of Education only provided
some oversight. Ministry actions included
Setting targets subject, which were
mandatory-but not detailed
Publishing curriculum guidelines, but
they were not mandatory
Public scrutiny also highlighted several
system weaknesses
Serious underachievement in terms
of both quality and equity
Lack of a culture of evaluation
Absence of school self-evaluation
Inadequate sharing of good practice
Lack of attention to early reading
problems
Inadequate teacher preparation

Selected Approaches
The PISA shifted the focus of the Danish system and led to
many changes
New national objectives for subjects - attainment targets
New culture of evaluation
New mandatory national assessment system
Establishment of Municipality monitoring units
New mandatory leaving examination system
New agency for evaluation and quality development
Stronger pre-service and in-service teacher training
Annual trainings
Yearly self-evaluations
Pre-service training specific to student age groups, in
order to ensure stronger early grade teachers
Design of collective agreement documents concerning
school roles and responsibilities
Stronger mentorship of new school leaders
A dissemination of bilingual teaching experts and a reevaluation of bilingual teachers and their use of mother
tongues

SOURCE: Danish Ministry of Education The changes in evaluation of the educational system in Denmark after PISA (link)

| 18

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 19

Teachers factor
Correlation between Teachers Instructional Practice and Students Performance
Best practice
generally
Malaysia Spore
accepted as
effective

Correlation
Thailand UK

US

- 0.01

0.01

0.01

- 0.07 - 0.01

Collaborate to 0.01
Improve
Teaching

0.01

0.06

- 0.08 0.05

Instruction to
Engage
Students in
Learning

0.02

0.00

- 0.02 0.06

Teachers Relate
Lessons to
Students Daily
Lives and Bring
Interesting
Materials to Class

- 0.06

0.02

Japan

Hong Korea Australia Finland

Kong

- 0.11 0.02

0.01

- 0.05

- 0.01

0.00

0.03

- 0.03

0.02

0.02

0.11

- 0.02

- 0.02

0.04

Negative or low correlation between teachers

instructional practice with students performance

| 20

Collaborate to Improve Teaching (TIMSS 2011)

Country

Very Collaborative

Collaborative*

% of
Students

Average
Achment

% of
Students

Malaysia

32

424

64

Singapore

22

585

66

595

Chinese
Taipei

15

568

58

Finland

15

557

England

27

United States
International
Ave
Teachers
questionnaire
*(interacting 2-3 times
a month in all areas

Somewhat Collaborative

Average
% of
Achment Student
s
430
4

Average
Achment

Average Scale
Score

378

10.5

11

573

9.9

563

28

563

9.0

59

552

26

551

9.2

521

57

536

16

535

9.9

38

528

47

529

16

516

10.2

29

476

58

479

13

472

How often do you have the following types of interactions with other teachers:
Discuss how to teach a particular topic;
Collaborate in planning and preparing instructional materials;
Share what I have learned about my teaching experiences;
Visit another classroom to learn more about teaching; and
Work together to try out new ideas.
| 21

Instruction to engage students

Country

Most Lessons
% of
Students

About Half of the

Lessons
% of
Average
Students Achieveme
nt
19
418

Some Lessons

Malaysia

77

Average
Achieveme
nt
427

Singapore

63

593

30

585

586

9.1

Chinese
Taipei
Finland

61

562

26

569

13

563

8.8

48
93

555
532

45
7

549
533

7
1

549

8.4
10.8

88

532

10

514

80

478

17

474

England
United
States
International
Average

% of
Students

Average
Average
Achieveme Scale Score
nt
425
9.7

10.5
509

The Instruction to Engage Students in Learning(IESL) scale encompassed the following items:
Summarizing the lessons learning goals;
Relating the lesson to students daily lives;
Questioning to elicit reasons and explanations;
Encouraging students to show improvement;
Praising students for good effort
| 22

Relate lessons to students daily lives and bring interesting materials to class

Country

Malaysia
Singapore
Chinese
Taipei
Finland
England
United
States
Internationa
l Ave

Relate Lessons to Students Daily Lives

Every Lesson or
About Half the
Almost Every
Lessons or Less
Lesson
% of
Average
% of
Average
Students Achment Students Achment

Bring Interesting Materials to

Every Lesson or
About Half the
Almost Every
Lessons or Less
Lesson
% of
Average
% of
Average
Students Achment Students Achment

62
46
55

428
590
558

38
54
45

424
591
571

16
14
15

431
600
555

84
86
85

242
589
565

53
42
64

555
530
531

47
58
36

548
534
527

17
23
38

560
530
536

83
77
62

550
533
526

63

478

37

476

32

480

68

476

| 23

Teacher emphasis on science investigation (TESI)

Country

About Half of the Lessons

% of Students
Average
Achievement

% of Students

Some Lessons
Average
Achievement

Average Scale
Score

Malaysia
Singapore
Chinese Taipei

53
29
21

433
595
552

47
71
79

417
588
567

10.2
9.1
8.6

Finland
England
United States
International
Average

32
37
47
48

558
544
537
479

68
63
53
52

549
525
524
474

9.3
9.4
9.7

TESI involves the following items:

Observe natural phenomena and describe what they see
Watch me demonstrate an experiment or investigation
Design or plan experiments or investigations
Conduct experiments or investigations
Use scientific formulas and laws to solve routine problems
Give explanations about something they are studying
Relate what they are learning in science to their daily lives
| 24

Students factor

Correlation between Students Attitude and Behaviour and their

performance in TIMSS 2011(all reported by students)
Correlation

Enjoyed Learning
Science
Interested in What
Teachers Say
Students engaged in
science (SESL)
Students
Confidence in
Science
Students Value
Science
Language Used by
Students

Malaysia Singapore Thailand UK

US

Japan Hong
Kong

Korea Australia

- 0.23

- 0.24

- 0.04

- 0.21

- 0.23

- 0.31

- 0.23

- 0.40

- 0.30

- 0.08

- 0.02

0.00

- 0.13

- 0.05

- 0.22

- 0.15

- 0.22

- 0.15

0.15

0.09

0.07

0.12

0.16

0.30

0.16

0.35

0.23

0.22

0.26

0.12

0.32

0.35

0.38

0.24

0.48

0.39

0.34

0.26

0.19

0.17

0.20

0.34

0.23

0.38

0.32

0.34

0.26

0.19

0.17

0.20

0.34

0.23

0.38

0.32

More positive correlation between students attitude

and behaviour with students performance
| 25

Students Confidence in Science

Country

Confident

Somewhat Confident

Not Confident

Average
Scale
Score

% of
Students

Average
Achievement

% of
Students

Average
Achievement

% of
Average
Students Achievement

Malaysia

511

45

437

51

411

9.1

Singapore

14

630

48

600

37

562

9.6

Chinese
Taipei

648

27

599

67

543

8.3

England

23

579

52

529

25

503

10.2

United
States
Internationa
l average

26

565

47

524

27

492

10.3

20

536

49

482

31

450

Finland

How much do you agree with these statements about science?

I usually do well in science.
Science is more difficult for me than for many of my classmates
Science is not one of my strengths
I learn things quickly in science
Science makes me confused and nervous.
| 26

Students Value Science

Country

Value

% of
Students
Malaysia
Singapore
Chinese
Taipei
Finland
England
United
States
Internationa
l average

Somewhat Value

Do Not Value

Average
Scale
Score

% of
Students

Average
Achievement

% of
Students

Average
Achievement

49
41
12

Average
Achievemen
t
453
616
612

34
43
30

419
583
586

17
17
58

370
546
543

10.3
10.2
8.5

41
36

547
544

37
34

530
525

22
29

516
506

10.1
9.7

41

502

33

477

26

457

How much do you agree with these statements about science

I think learning science will help me in my daily life;
I need science to learn other school subjects;
I need to do well in science to get into the university of my choice;
I need to do well in science to get the job I want;
I would like a job that involves using science; and
It is important to do well in science.
| 27

Students Like Learning Science

Country

Malaysia
Singapore
Chinese
Taipei
Finland
England
United
States
Internation
al average

Like Learning Science

Somewhat Like
Learning Science

Do not Like Learning

Science

Average
Scale
Score

% of
Students
42
38
17

Average
Achievement
457
617
618

% of
Students
44
46
43

Average
Achievement
418
584
571

% of
Students
13
16
40

Average
Achievement
364
542
534

10.4
10.2
9.0

32
29

562
555

45
43

532
523

23
28

500
500

9.9
9.6

35

515

44

472

21

450

How much do you agree with these statements about learning science?
I enjoy learning science
I wish I did not have to study science.
Science is boring
I learn many interesting things in science.
I like science.
| 28

Students Engaged in Science

Country

Like Learning Science

% of
Students

Average
Achievement

Somewhat Like
Learning Science
% of
Students

Average
Achievement

Do not Like Learning

Science
% of
Students

Average
Scale
Score

Average
Achievement

Malaysia
Singapore
Chinese
Taipei
Finland
England
United
States
Internation
al average

How much do you agree with these statements about learning science?
.

| 29

Findings

Low to negative correlation between instructional practices with

students performance

Higher correlation between students attitude and behavior with students

performance.

Students interested or enjoyed science is negatively correlated to

achievement. It doesnt matter if they enjoy or interested in science or
not.

What the teacher thinks is engaging students might not really engaging
the students
Suggested action

Besides enhancing teachers pedagogy in teaching science, there is a

need to look at ways to enhance students attitude and behaviour, e.g.
increase students confidence in science, raising awareness of
importance of science to students

Teachers need to prepare and assess effectiveness of his or her

teaching by looking at students perspectives.

| 30

Something to ponder
What Science Teaching looks like: An International perspective (based on The TIMSS Video Study of Science 1999)
Published in December 2006/January 2007 | Volume 64 | Number 4 Science in the Spotlight Pages 16-23
Comparison made between US, Czech Republic, Japan, Australia, and the Netherlands
by Kathleen Roth and Helen Garnier

Findings:
Two major differences between the United States and the other countries:
All of the higher-achieving countries had strategies for engaging students with
core science ideasthat is, their science lessons focused on content. In U.S.
lessons, content played a less central role, and sometimes no role at all;
instead, lessons were usually built around engaging students in a variety of
activities.
All the higher-achieving countries in the video study shared high content
standards and a content-focused instructional pattern. In Australia and
Japan, for example, teachers carefully and thoughtfully developed just one or
two science ideas across a lesson, presenting the ideas in a logical, coherent
sequence. They expected students to understand science concepts and be
able to support those concepts with specific evidence.
Suggestion of improvement:
Develop a Clear, Coherent Science Content Storyline
Link all activities to science ideas
Strengthen teachers content knowledge
Rethinking the role of content in science teaching
| 31

Item Analysis: Science and Mathematics

| 32

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 33

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 34

BAHAN SUMBER
PENINGKATAN
KEMAHIRAN ARAS
TINGGI (KBAT)
SAINS
Panduan Penggunaan Soalan TIMSS
dalam Pengajaran dan
Pembelajaran
BPK
Mei/Jun 2013

ORGANISASI BAHAN
Penerangan
ringkas tentang;

Petunjuk tentang
tugasan murid;

Apakah TIMSS?

Mengapa
TIMSS?

Mengapa
Panduan Ini?
Bagaimana
Menggunakan
Bahan Yang
Disediakan?
Bilakah Bahan
Ini Perlu
Digunakan?
1. PENGENALAN

Nombor
Tugasan

Panduan guru
membimbing murid
melaksanakan tugasan;

Tahun soalan ini

dikeluarkan
dalam TIMSS

Tema yang
berkaitan dalam
KBSM

Panduan
membimbing murid
dengan EMPAT
LANGKAH GENERIK
Kenal pasti kehendak
soalan
Kenal pasti fakta yang
diberi dalam soalan
Kemukakan soalan
yang merangsang
pemikiran untuk
menyelesaikan
masalah
Selesaikan masalah

Skima jawapan

Domain Kognitif
TIMSS

Halaman
berkaitan

2. TUGASAN

Soalan TIMSS dalam

BM dan BI

3. PANDUAN
GURU

Lembaran kerja
untuk murid
melaksanakan
tugasan;

Soalan TIMSS
dalam BM

Panduan
untuk murid
menyelesaikan masalah
dengan empat
langkah
mudah yang
sama seperti
dalam
Panduan
4. Guru
LEMBARAN
MURID

Bahan Sumber Peningkatan Kemahiran

Berfikir Aras Tinggi

PANDUAN PENGGUNAAN SOALAN

PROGRAMME FOR INTERNATIONAL
STUDENT ASSESSMENT (PISA)

MATEMATIK

Bahagian Pembangunan Kurikulum

Kementerian Pelajaran Malaysia

Susun Atur Bahan

19
set tugasan

Dwibahasa

Susun Atur Bahan

Panduan Guru
Cadangan
prosedur
pelaksanaan
Tugasan
Cadangan
strategi dan
pendekatan
Panduan
Skor

Susun Atur Bahan

Panduan Guru

Terdapat tiga jenis petunjuk yang

maksudnya dilambangkan oleh ikon yang
berikut:

Cadangan
prosedur
pelaksanaan

Memberi arahan untuk guru

menjalankan aktiviti dengan
murid.

Tugasan
Cadangan
strategi dan
pendekatan

Cadangan soalan untuk

mencungkil pemikiran murid
atau tugasan yang guru
boleh gunakan semasa PdP

Panduan
Skor

Cadangan penyelesaian,
nota PdP, kaedah PdP dll.

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 43

Garis masa utama

Garis masa utama
Taklimat
kepada
Pengetua
dan Ketua
Bidang
Sains dan
Matematik

Mac

April

Pelancaran
Program
Peningkatan
Kemahiran
Guru dan
Bahan
Sumber

Mei

Bahan
Sumber
Guru siap
untuk
digunakan

Merintis Bahan
Sumber Guru
Pentaksiran
Kompetensi
& Literasi
pertama

Jun

Julai

2013
Maklumat berkaitan TIMSS dan PISA diusahakan
secara berterusan daripada kajian, perbincangan
meja bulat dan kolokium.

Pentaksiran
Kompetensi &
Literasi Kedua

Sep

Feb

Apr

Pentaksiran
Kompetensi &
Literasi Ketiga

Taklimat
pentaksiran
TIMSS 2014

Jun

Sep Okt

2014

Pentaksiran
Kompetensi
& Literasi
Keempat

Apr Mei Sep Okt

2015
TIMSS 2014
PISA 2015

Pelaksanaan di sekolah
| 44

Content

Conceptual Framework of the Malaysian

TIMSS PISA Enhancement Plan
Alignment of TIMSS PISA Initiative with
HOTS Initiative
Taking stock of what we have done
Findings from roundtable
Findings from benchmarking with other
countries
Share analysis of data :
- Factors influencing students performance(data from questionnaires)
- Item analysis (BPPDP)

Format for Mock-test (LP)

Use of questions in teaching and learning
Timeline
Next steps
| 45

Potential studies to be conducted

Ascertain possible relationship between
teaching and learning methodologies and
students achievement in TIMSS and PISA

Pick three best achievement schools

and three worst achievement schools
Observe teaching and learning to
compare between practice in both
categories of schools
Observe school ambience to compare
between practice in both categories of
schools
Investigate the development of
science concept through science and
Maths lessons