Quality Verification of Testing Instruments

Pearson Vue BASI

A. Test Rationale Appendix C. Item A.1. Response: The Basic Achievement Skills Inventory (BASI) is a multilevel, norm-referenced achievement test that measures language and math skills in adults and children. The BASI has many potential uses. Babbage Net School utilizes the test as an accurate way to measure math and language skills in order to identify students academic strengths and weaknesses. The results help Babbage identify appropriate educational programming and instructional plans to improve learning. Appendix C. Item A.2. Response: The BASI consists of subtests at multiple grade-specific levels. For each of the test levels there are six subtests: Vocabulary, Spelling, Language Mechanics, Reading Comprehension, Math Computation, and Math Application. Each subtest is organized into three composite areas to provide us with total scores of Reading, Written Language, and Math. Appendix C. Item A.3. Response: The BASI consists of different grade-specific assessments: Kindergarten Level Assessment, Grade 1 Level Assessment, Grade 2 Level Assessment, Level 1 Assessment for grades 3 and 4, Level 2 Assessment for grades 5 and 6, Level 3 Assessment for grades 7 and 8, and Survey Level BASI for grades 9-12. Each level is tested on the subjects listed above (Appendix C. Item A.2). B. Technical Qualities of the Test Appendix C. Item B.1a(1). Response: The BASI was standardized was standardized during the 2002-03 school year. During this time, validity and reliability studies were conducted. The BASI content validity was developed and refined in consultation with content experts using the Model Curriculum and Assessment Database (MCAD), consisting of national educational standards. Test items were first written by teachers with experience in BASI subject matters. The teachers were trained by Pearson Education in item writing and were supervised and coordinated by a professional with a background in the respective content areas. Once an item was offered for test consideration, a team of four to five Pearson experts reviewed the item and then made submissions of potential BASI test items. The next level of item review was conducted by the Pearson Assessments test development team. Pearson also utilized a team of teachers who served as consultants to review the test items and content coverage areas. At each step in the process, items were reviewed for content coverage and item appropriateness. In addition, all items were checked for possible race, gender, and regional bias. Appendix C. Item B.1a(2). Response: Pearson conducted a pilot study in the fall of 2001 to test a large group of potential test items. Test forms were administered to a total of 1,874 students in various grades with no time limits. Each item was reviewed for its psychometric properties. From the results of this pilot study, items were selected for the final standardization version of the test. In the spring of 2002, an additional pilot study was conducted on a smaller group of students with test items from the original study and new items created and/or modified from the results of the original study. Following both studies, the final version of the BASI was created. Pearson also presents evidence of construct validity. For the BASI, Pearson showed relationships with scores on other tests of academic achievement such as the Iowa Test of Basic Skills (ITBS). Pearson compared 285 students scores on the ITBS and BASI. Significant correlations were observed between subtests and total scores measuring similar constructs. Complete data and correlation tables are available at: http://www.pearsonassessments.com/pai/ca/RelatedInfo/BASISeriesandCorrelation.htm

Appendix C. Item B.1b(1). Response: Pearson conducted a number of studies to test the reliability of the BASI including Cronbachs alpha coefficients to calculate internal consistency, a test-retest study, and a test to investigate alternate-forms reliability. Because of page limitations, only the test-retest study will be discussed here. Test-retest reliability coefficients indicate stability of scores obtained when the same people are tested more than once over a period of time. The BASI was examined using samples of students who were administered the test twice over a period of two weeks. Please see below for test results (Appendix C Item B.1b(2)). Appendix C. Item B.1b(2). Response: The results of the test-retest reliability study were very positive. All test-retest coefficients were statistically significant. Of the 27 test-retest coefficients, 24 were .70 or greater and 20 were .80 or greater. These reliability values serve as an indication of stability of BASI scores over time. A trend towards small gains in mean scores did appear. However, this was expected due to practice effects. Please see Table 5.7 (below) for a sample chart of BASI Level 2 test data.

Appendix C. Item B.1c(1). Response: A total of 2,439 students in grades 3-12 were administered the BASI during test standardization. An additional 2,130 students participated later that year. Testing areas were chosen based on the 2000 US Census in order to provide a fair representation of demographic variables. Students who were found to be members of the various subgroups were given weighted scores exactly proportionate to the US population. Weights were calculated by dividing the percentage of the individuals subgroup in the population by the percentage for that subgroup in the BASI standardization sample. As a result, each student in the sample contributed to the final sample proportionally. During standardization, each test administration was conducted with the same/similar testing environment and procedures. All answer sheets were carefully inspected and scored to ensure accuracy. During the standardization of the BASI, the six subtests were administered in the same order as they appear in the test booklet. The order is: Vocabulary, Spelling, Language Mechanics, Reading Comprehension, Math Computation, Math Application. This was the procedure for standardization at every administration for each grade level. Raw scores are given to students who take the BASI for the number of questions answered correctly. The raw score is then converted into the standard score. Each BASI subtest is normed at grade level. Appendix C. Item B.1c(2). Response: The BASI is designed to represent the general population of students at the same grade level of the examinee.

Results are either hand scored or scored with a computer to produce a students raw score. The raw score is transformed into a standard score with a mean of 100 and a standard deviation of 15. This is a common metric for many achievement and intelligence tests. Appendix C. Item B.1c(3). Response: Score transformations for the grade level and age equivalent were determined using an anchor test design. The anchor test used was the BASI survey, where groups of students at each grade level were administered the BASI survey and comprehensive versions of the test. Age equivalent and grade level scores were based on these data. C. Assessing Students Requiring Special Accommodations

The BASI recognizes that students may need to take the assessment with accommodations in order to allow their test results to accurately reflect their level of achievement. BASI does not have standard procedures for testing accommodations. However, the test can easily be administered according to each organizations testing accommodation policies. Babbage Net School hires tutors and field managers who are bilingual. These individuals are always on hand to proctor the BASI for students who are English Language learners. During testing administration, the proctors will assist students with basic language comprehension if necessary.