Академический Документы
Профессиональный Документы
Культура Документы
Consortium: Australian Council for Educational Research (ACER, Australia) cApStAn Linguistic Quality Control (Belgium) Deutsches Institut fr Internationale Pdagogische Forschung (DIPF, Germany) Educational Testing Service (ETS, USA) Institutt for Lrerutdanning og Skoleutvikling (ILS, Norway) Leibniz - Institute for Science and Mathematics Education (IPN, Germany) National Institute for Educational Policy Research (NIER, Japan) The Tao Initiative: CRP - Henri Tudor and Universit de Luxembourg - EMACS (Luxembourg) Unit d'analyse des systmes et des pratiques d'enseignement (aSPe, Belgium) Westat (USA)
TABLE OF CONTENTS
INTRODUCTION .........................................................................................................................4 What is PISA? .................................................................................................................................... 4 The PISA assessment ........................................................................................................................ 4 STAGE I: PREPARATION ..............................................................................................................6 1. Nominate dates for the assessment ............................................................................................. 6 2. Prepare List of Eligible Students ................................................................................................... 9 3. Complete School s Computer System Diagnostic Report ............................................................ 10 STAGE II: BEFORE THE ASSESSMENT ....................................................................................... 111 4. Receive and check materials sent by EPRD .............................................................................. 111 5. Notify teachers, students and parents of the assessment ....................................................... 111 6. Review and update Student Tracking Form .............................................................................. 111 7. Give School Questionnaire to appropriate person at your school ............................................. 17 8. Organise and confirm assessment plans with school staff and with Test Administrator........... 17 9. Collect completed School Questionnaire ................................................................................. 177 STAGE III: DURING/AFTER THE ASSESSMENT ........................................................................... 188 10. Provide Test Administrator with Student Tracking Form and School Questionnaire............... 188 11. Be present throughout the assessment to assist Test Administrator (if possible) ................. 188 12. Arrange for follow-up session (if needed) .............................................................................. 188 13. Arrange assistance for Test Administrator for Computer-Based Assessment session ........... 188 Appendix 1: PISA 2012 Main Survey - Assessment Dates Form ....................................................... 200 Appendix 2: PISA 2012 Main Survey - School's Computer System Diagnostic Report ..................... 211 Appendix 3: PISA 2012 Main Survey - List of Eligible Students........................................................... 32
Introduction
1. Thank you for agreeing to be a School Co-ordinator for PISA. The purpose of this manual is to help you step by step to understand how you can assist with the successful implementation of this large scale international assessment programme.
What is PISA?
2. PISA stands for the Programme for International Student Assessment, sponsored by the Organisation for Economic Co-operation and Development (OECD). PISA 2012 is the fifth PISA study that has been conducted since 2000 and involves more than 60 participating countries. PISA has the following characteristics: y y y y 4. 5. It is the world s largest international survey in education; It surveys students aged about 15 years; It assesses students preparedness for adult life; and It measures students Mathematical, Scientific, Problem Solving and Reading Literacy.
3.
Typically, between 4,500 and 10,000 students in each country take part in this programme. Undertaking PISA is important because the results of the assessment can be used: y y y to indicate how well prepared students in Malaysia are for learning once they leave school; by schools, education systems and governments to identify areas for improvement over time; and to allow a comparison of student performance and the learning environment between different countries.
6.
For more details about PISA, please refer to the information materials that Educational Planning and Research Division (EPRD) has provided or the OECD website at: http://www.pisa.oecd.org/. Note 1: All data collected in PISA are kept strictly confidential. No individual or school is identified in any report released from the assessment programme.
8. 9.
Formatted: Font: No
17 18 18 18
18
If you have any questions, please contact us at: EPRD, MOE. Tel: 03-88846548/6595, Fax:03-88846579, email:pisa@moe.gov.my
Stage I: Preparation
1. Nominate dates for the assessment
10. PISA will take place between 09.04.2012 and 11.05.2012. Please provide details of your preferred assessment dates by completing the attached Assessment Dates Form (see Appendix 1Appendix 1). Then, e-mail or fax this form to pisa@moe.gov.my or 03-88846579 before 20.02.2012. Please take into account the issues listed on the Assessment Dates Form when nominating assessment dates. You will receive details of your assigned assessment date and time by 19.03.2012. Please inform the Principal and school staff of the details. The Paper-Based PISA will take approximately 3 hours and 15 minutes to administer as outlined in Table 2Table 2, and the Computer-Based Assessment (including preparation and administration time) will take about 1 hour 30 minutes as outlined in Table 3Table 3. Table 2: Timing of Paper-Based PISA Administration Activity Distributing the materials and reading the General Directions The Test Booklet Short break The Test Booklet Break The Student Questionnaire Collecting the materials and ending the session Total Time 10-15 minutes (approximately) 1 hour (exactly) No more than 5 minutes 1 hour (exactly) 15 minutes 35 minutes (approximately) 3-5 minutes (approximately) 3 hours and 15 minutes (approximately)
Formatted: Font: 11
11.
Formatted: Font: No
Formatted: Font: No
Table 3: Timing of Computer-Based Assessment Administration Activity Computer room setup Student log-ons, passwords and introducing the test Computer-Based Assessment practice session Completing the Computer-Based Assessment Ending the assessment session, collection and transfer of data Total 12. Time 15 minutes (approximately) 5 minutes (approximately) 15 minutes (approximately) 40 minutes (exactly) 15 minutes (approximately) 1 hour 30 minutes (approximately)
Please note that the Computer-Based Assessment session should take place after the Paper-Based PISA, and preferably on the same day. Because some students will participate in both the Paper-Based and Computer-Based sessions, it is best to schedule the start of the Paper-Based PISA at the beginning of the school day. It is recommended that the Computer-Based Assessment be scheduled for the afternoon. Please allow an adequate interval of at least 1 hour after the Paper-Based PISA has finished before beginning the Computer-Based Assessment session. The Test Administrator will need to prepare the Computer-Based Assessment area and run checks and preparations for each computer. Also, students participating in the Computer-Based Assessment will need a break between assessments.
13.
DIRECTIONS: Please list ALL students in your school in Form 1 and higher born between 01.01.1996 and 31.12.1996.
(B) Form
Ronny Liew M. Muthusamy Salina Ahmad P. Letchumy Ahmad Amir Khaw Meng M. Mahendran Mariah Muda Florence Teoh Lai Kwang Tammy Lim V. Shahrukhkhan Sharifah Azman Kamal Hassan
4 3 4 4 4 4 4 3 4 4 4 4 3 3
(D) Birth Date (MM-YYYY) 12 11 12 05 04 09 07 04 07 10 12 08 09 07 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996
(F) SEN*
3 3 1
PISA needs accurate information about all students at your school who meet the above criteria for student eligibility. A small number of these students may be unable to complete the PISA assessment. For now, any such students must be added to your list of students. Later, you may need to identify students who cannot participate in PISA. This will be covered in the next chapter.
Step 2: List each of the eligible students using the form in Appendix 3Appendix 3, along with their demographic information (see Figure 1Figure 1 for an example)
y y y y Student name Column A; Grade Column B; Gender Column C: Indicate 1 (female) or 2 (male); and Birth Date Column D: Indicate as MM/YYYY. For example, if an eligible student was born in September 1996, write 09/1996.
Formatted: Font: 10
Step 3: Write a Study Programme code for each eligible student Column E
16. Table 4Table 4 below shows a code for each study programme. Please choose the appropriate study code for each eligible student and fill in column E. Table 4: Codes for Study Programme Study Programme Science Upper Secondary Arts Upper Secondary Technical/Vocational Secondary Religious Secondary Lower Secondary Code 1 2 3 4 5
Formatted: Font: No
Formatted: Font: No
Cognitive, behavioral or emotional disability in the opinion of qualified staff, student has a cognitive, behavioral or emotional disability Limited assessment language experience student is not a native speaker of any of the languages of the assessment in the country and has limited proficiency in these languages
Formatted: Font: No
y y y y
20.
21.
23.
10
Please check whether the package includes these materials. If you find any materials missing or damaged, please contact EPRD, MOE as soon as possible. Please note also that EPRD, MOE will send all assessment materials to the Test Administrator who has been assigned to your school.
28.
Formatted: Font: No
11
12
Codes to enter into SEN column: 1 Functional disability 2 Cognitive, behavioral or emotional disability 3 Limited assessment language experience
Figure 2: PISA 2012 Main Survey - Student Tracking Form (example after completion) School Name: SMK Putrajaya 2 School Coordinator: Maria Adam Completed by EPRD, MOE To be checked by School Co-ordinator Test Administrator: Rashid Ishak
To be completed by School Coordinator Completed by EPRD, MOE, identifies CBA students
Codes to enter into N/P columns: 2 Parent refusal 3 Excluded on SEN basis 4 Now enrolled at another school 5 Not enrolled in this school, enrolment unknown n Does not meet PISA criteria for eligibility
Region
Stratum ID
School ID
Student ID
Student Name
Form
Birth Date
Study Prog.
SEN*
N/P PISA
CBA
COMMENTS
00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
01 01 01 01 01 01 01 01 01 01 01 01 01 01
010 010 010 010 010 010 010 010 010 010 010 010 010 010
00004 00009 00005 00002 00014 00008 00011 00013 00001 00007 00012 00010 00003 00006
00003 00008 00012 00016 00020 00024 00028 00033 00037 00041 00045 00049 00053 00057
Ronny Liew M. Muthusamy Salina Ahmad P. Letchumy Ahmad Amir Khaw Meng M. Mahendran Mariah Muda Florence Teoh Lai Kwang Tammy Lim V. Shahrukhkhan Sharifah Azman Kamal Hassan
4 3 4 4 4 4 4 3 4 4 4 4 3 Y6 3
2 2 1 1 2 2 2 1 1 2 1 1 1 2
12 11 12 05 04 09 07 04 07 10 12 08 09 07
1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1996 1995 1996 1996 1996
1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 n 3 n 3 3 1 3 3 4
3 9 5
7 10 6 2
Corrected her grade, which is lower than the eligibility criterion Corrected her birth year, which is outside the eligibility criterion
13
Formatted: Font: No
15
Codes 4 and 5 - Student was transferred to another school or left the school
y y If a student has transferred to another school, record a code 4 for this student in column N/P PISA . If a student has left school, but it is not known which school the student has transferred to, record a code 5 for this student in column N/P PISA .
Formatted: Font: No
Formatted: Font: No
y y
16
V. Shahrukhkhan has been professionally assessed as cognitively delayed and is coded as 2 in column SEN . She does not participate in other testing programs. Therefore, she will not participate in the assessment and is coded as 3 in column N/P PISA . Note 3: Column N/P PISA of Student Tracking Form The purpose of setting a column N/P PISA is to record the reasons of non-participation for those students who have no possibility to participate in PISA under any circumstances even if a follow-up session is provided after the assessment. Therefore, you should NOT use a code 2 if a student is unable to attend the PISA assessment, for example, due to a competing activity in/outside the school or illness, as he/she might be able to attend a follow-up session.
8. Organise and confirm assessment plans with school staff and with Test Administrator
38. During the two weeks preceding the assessment, you will need to confirm these plans: y y y y arranging for a technical support person to prepare computers for the Computer-Based Assessment; asking participating students to bring a: (a) pencil; (b) eraser; (c) ruler; and (d) calculator on the day of the assessment or preparing the above stationery at the school; obtaining a few spare items of the stationery listed above and some books or magazines for students who finish the assessment ahead of schedule; and informing those concerned of any change in the date or time of the assessment. In particular, it is important to advise EPRD, MOE, so that the Test Administrators and Quality Monitors can be advised.
39.
The Test Administrator assigned to your school should contact you approximately one week before the assessment date to confirm the updates of the Student Tracking Form and details of the assessment session.
17
42.
11. Be present throughout the assessment to assist Test Administrator (if possible)
43. If it is not possible for you to be present throughout the assessment, you should be present at least at the beginning of the assessment session. This will help set the tone of the session and demonstrate the importance of the study. Note 4: y Because PISA materials may be used in the future, their security is very important; y You and other school staff members are allowed to look through unused booklets only after the assessment, and in the presence of the Test Administrator; and y DO NOT photocopy or copy the test booklet and Computer-Based Assessment materials in any way under any circumstances.
45.
13. Arrange assistance for Test Administrator for Computer-Based Assessment session
46. If the Paper-Based PISA and Computer-Based Assessment sessions will be conducted by different Test Administrators, please make sure that the Test Administrator for the Computer-Based Assessment will obtain from the other Test Administrator: y The Student Tracking Form to check the consistency; and y A copy of the Session Attendance Form used for the Paper-Based PISA. This will enable her/him to check that only students who were present for the Paper-Based PISA are assessed in the Computer-Based Assessment.
18
Your help with PISA is very important to its success. EPRD, MOE and the International Project Team appreciate your time and effort. Thank you!
19
Please nominate three alternative dates for the PISA assessment to take place at your school:
Preferences First Second Third Date Time of commencement
Please specify any dates which will NOT be suitable for testing:
Please e-mail, fax or post back this form to EPRD, MOE by 20.02.2012. Thank you!
20
Appendix 2: PISA 2012 Main Survey - School's Computer System Diagnostic Report
Introduction
1. Fourteen (14) students in your school will participate in the Computer Based Assessment (CBA) as part of the PISA assessment. It is important to establish whether the computers students will use for the test have the required capabilities to run the CBA test successfully. 2. This manual provides instructions for the completion of a Systems Diagnostic (SD) which will test computer capabilities for running the CBA test. The information gained from this diagnostic will then enable the completion of the CBA Systems Diagnostic test results forms so that the number of suitable computers available in one room can be determined. This information will need to be provided to EPRD, MOE.
10. Click on the Start Diagnosis button and the following screen will appear:
21
11. The top section of the screen About the System provides general information about the computer being tested. "OS" refers to the operating system. "Service Pack" is the latest installed service pack, if there is no service pack installed -- N/A -- is displayed. "User" is the user currently logged into the computer. "USB key size" displays the amount of free memory and total capacity of the USB drive. 12. The buttons on the right of the screen are the three different diagnostic tests that can be implemented: the basic check, the virus scan, the advanced check and a button for exiting the diagnostic. Please note that in the home edition of Windows XP the Advanced Check button will be disabled.
Basic check
13. Click on the Basic check button on the right of the screen. This check may take a few seconds to run. Below is an example of what will appear when the basic check is complete.
22
23
14. The following table indicates each type of basic check, what it is checked, the associated requirement for the CBA test and the possible messages that will appear in the centre section of the screen above:
Label CPU What it tests The speed of the CPU installed in MHz The operating system Requirement for CBA Must be at least 1500Mhz Possible message OK : CPU speed is "n" MHz OR Failed : CPU speed is "n" MHz Must be Windows XP, Windows Vista or Windows 7 OK : Operating system is "one of Win XP, Win Vista, Win 7" OR Failed : Operating system is "system other than Win XP, Win Vista, Win 7" System memory The memory installed on the computer in MB Required memory for Windows XP = 512 MB Required memory for Windows VISTA = 1024 MB Required memory for Windows 7 = 1024 MB Screen resolution The current screen resolution (width x height) Whether the current user is logged in as an administrator If Skype is running on the computer Minimal screen width = 1024 px Minimal screen height = 768 px Must be logged in as an administrator OK : System memory OR Failed : System memory (required "n" MB) OK : Screen : Width x Height OR Failed : Screen : Width x Height OK : Account with admin rights OR Failed : Account with admin rights required The CBA assessment cannot run with Skype running OK : Skype is not running OR Failed : Skype should not be running System s DPI setting The computer s current DPI setting The USB key data transfer rate in MB/s DPI setting must be 96 DPI (100%) OK : DPI setting is 96 DPI OR Failed : DPI setting is "n" DPI Required USB data transfer rate = 1 MB/s OK : Speed "n" MB/s OR Failed : Speed "n" MB/s OR Can't run the test - in cases where the program could not detect the transfer rate
OS
Skype running?
15. If all basic checks are highlighted in green and labelled OK, the computer is suitable for running the CBA assessment. If any appear in red, the computer may not be suitable.
24
16. For each computer tested, please complete the test results form on page 29. Please refer to the section on Reporting Results for more information.
Virus scan
17. The virus scan can be used to check any drive on your computer. The scan must be completed on the USB drive and will take approximately 30 seconds. 18. It is recommended (but not mandatory) to also run the scan on the local hard drive. Please note that this may take between two and five hours for each computer. 19. To run the virus scan, click on the Virus Scan button on the bottom right of the Systems Diagnostic check screen. All local and network drives will appear as available for scanning. The following screen shot is an example of what will appear when 8 drives (including USB) are available for scanning.
20. To scan the USB drive, ensure that the correct drive is selected, then click on the Scan button at the bottom of the screen. The following screen will appear and may take approximately 30 seconds to run on the USB (or up to 5 hours for the local hard drive scan).
25
21. Please note that while the virus scan is running the computers local antivirus software may detect the virus scan software being used for PISA (ClamWin Free Antivirus) as a virus. This is normal and should be ignored. Below is an example of the screen that appears when Trend Micro OfficeScan is the local antivirus software. Dialog screens and options will vary depending on the type of antivirus software installed locally.
22. In this example, the local virus scanner (Trend Micro Office Scan) has detected the Systems Diagnostic virus scanner (ClamWin Free Antivirus) as a virus. If this occurs click OK and the virus scan will continue. 23. The following screen is an example of what will appear when the virus scan is complete.
24. In this example the scan was successful and the computer is suitable for the CBA. It indicates no infected files were found.
26
25. If, however, the scan indicates that there are infected files, you will need to clean the USB (or local hard drive) with professional antivirus software or return the USB to EPRD, MOE. Until the virus scan reveals that there are no infected files on the USB drive, the USB should not be used to run the CBA. 26. Click on the Close button to exit the virus scan.
Test input
28. Click on the Exit button and return to the control panel which has the Test input button as indicated in the screen shot below.
29. Click on the Test input button. An automatic virus scan of the memory will commence. This scan will take two or more minutes depending on the size of the computers memory. 30. If the software indicates that a virus is detected a popup window will appear informing you of this and asking whether you would like to proceed with the CBA. You may choose to proceed but it is recommended to clean the computer with professional antivirus software first. This reduces the likelihood that a virus will be transferred to the USB. 31. Once the virus scan is complete, enter the password: 123456789
27
32. If this page does not appear, press ALT TAB to return to the PISA 2012 Control Panel and click on the Exit button. Eject the USB from the computer and start the process of running the Test input check again. If the error continues to appear, re-start the computer before trying again. 33. This mock item is designed to test if the functionality of the CBA will work on the computer. Please complete the following: 1. 2. Drag and drop the box as indicated and record whether this function works or not. Enter text into the text box and record whether this function works or not.
34. Once complete press ALT TAB to access the Pisa 2012 Control Panel, click on the Exit button and remove the USB from the computer.
28
Reporting results
35. For each computer tested, please complete the test results table below. Make photocopies as required. 36. Please then collate the information from all test results and determine how many computers are suitable in a single room for CBA testing. Then pass this information onto the <NC>.
Not Applied
FAILED
YES
NO
29
Use the information in the summary section of the test results to establish if the computer is suitable for the CBA test: a. b. c. If all checks are OK (or Not Applied) then the computer is suitable for the CBA test. If CPU, OS or System memory failed the test, the computer is not suitable for the CBA test. If the screen resolution test fails, you may attempt to alter the screen resolution. To do this, go to Settings and then Control Panel from the Start menu. Open Display and go to the Settings tab. Ensure that the resolution is set to at least 1024 x 768 pixels. If this level of resolution is not available the computer is not suitable for running the CBA test. If this level of resolution is available, apply the new settings and run the basic check again. If the screen resolution test is ok, the computer is suitable with this setting; if it fails the computer is not suitable for the CBA test. If the account with admin rights test fails it may be possible to change the administration settings of the computer. To do this, go to Settings and then Control Panel from the Start menu. Open User Accounts and change the user group membership to Administrator. Note that this setting should be altered using the main administrator account for the computer. After adjusting the administrator settings on the computer, run the basic check again. If the account with admin rights test is ok, the computer is suitable with this setting, if it fails the CBA test can not be launched. For more detailed instructions on adjusting the administration rights on the computer, visit one of the following websites depending on your operating system.
Creating user accounts
http://www.microsoft.com/windowsxp/ using/setup/winxp/accounts.mspx http://windows.microsoft.com/enUS/windows-vista/Create-a-user-account http://windows.microsoft.com/enus/Windows7/Create-a-user-account
d.
Windows Vista
Windows 7
e.
If the Skype test fails, ensure that Skype is shut down on the local computer and run the basic check again. If the Skype test still fails the computer is not suitable for running the CBA test.
30
f.
If the Systems DPI setting fails, it is possible to adjust these settings. The systems DPI must be set to 96 DPI (100%). For instructions on adjusting the systems DPI setting visit one of the following websites depending on your operating system.
Website for instructions for altering the system s DPI settings http://www.microsoft.com/windowsxp/using/setup/personalize/look.mspx http://windows.microoft.com/en-US/windows-vista/Make-the-text-on-yourscreen-larger-or-smaller http://windows.microsoft.com/en-us/windows7/Make-the-text-on-yourscreen-larger-or-smaller
After making this adjustment, run the basic check again. If the Systems DPI setting is ok, the computer is suitable however if the test still fails, the computer is not suitable for the CBA test. g. If the USB Key data transfer rate test fails, the USB is not suitable for running the test and you will need to contact your <NC>. A USB that has a data transfer rate of 1 MB/s must be used.
Exit
37. Once the basic check, virus scan, advanced check and mock item check have been completed, please exit the SD by clicking on the Exit button on the main screen and remove the USB from the computer.
Return of materials
38. Once all computers have been tested using the Systems Diagnostic, please return the USBs to the national centre.
31
DIRECTIONS: Please list ALL students in your school in Form 1 or higher born between 01.01.1996 and 31.12.1996. (A) Student Name (B) Grade (C) Gender (Female=1; Male=2) (D) Birth Date (MM-YYYY) (E) Study Programme (F) * SEN
32