JOURNAL OF RESEARCH IN SCIENCE TEACHING

VOL. 44, NO. 9, PP. 13481369 (2007)

Impact of the Collaboratives for Excellence in Teacher Preparation Program

Frances Lawrenz,1 Douglas Huffman,2 Amy Gravely1

1

University of Minnesota
2

University of Kansas

Received 30 May 2006; Accepted 23 February 2007

Abstract: This study investigated the national impact of the Collaboratives for Excellence in Teacher Preparation program (CETP). Impact of the program was examined in two different settings: institutions of higher education, and K12 science and mathematics classrooms. The focus of this study was to determine the impact of the CETP program on the institutional culture and collaborations among faculty, and changes in instructional techniques used by higher education faculty and K12 teachers. Data were gathered over a 3-year period from 12 different CETP projects. At the higher education level faculty reported more collaboration and a slight increase in the use of standard-based teaching. At the K12 level, students of teachers who were prepared by the CETP program viewed classroom instruction as slightly more standards-based than comparable students of non-CETP prepared teachers. Additionally, external observers rated classes taught by teachers educated in CETP projects as more standards-based than classes taught by non-CETP teachers educated in other programs. Implications of the results for national largescale reform of science and mathematics education are discussed. 2007 Wiley Periodicals, Inc. J Res Sci Teach 44: 13481369, 2007 Keywords: general science; science teacher education; college/university; assessment and evaluation

Numerous reports have raised concerns about the performance of U.S. students in mathematics and science especially when compared to other countries (U.S. Department of Education, 2003). To help students achieve in science and mathematics we need well-qualied teachers (National Research Council, 2001). However, it is extremely challenging to produce science and mathematics teachers who have the advanced knowledge and ability needed to fully prepare our students in the elds of science and mathematics (Guarino, Santibanez, Daley & Brewer, 2004). One way to help improve the knowledge and skills of science and mathematics teachers is by improving teacher preparation. Improving the preparation of teachers in postsecondary institutions is central to sustaining and deepening quality education for all students (National Commission on Teaching and Americas Future, 1996). However, how to best

Contract grant sponsor: National Science Foundation; Contract grant number: DUE 9908902. Correspondence to: F. Lawrenz; E-mail: Lawrenz@umn.edu DOI 10.1002/tea.20207 Published online 6 August 2007 in Wiley InterScience (www.interscience.wiley.com). 2007 Wiley Periodicals, Inc.

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1349

prepare teachers has been the subject of continuing debate (Darling-Hammond, Chung, & Frelow, 2002). On one side of the debate are those who advocate for emphasis on teachers knowledge of science and mathematics; on the other side of the debate are those who advocate for emphasis on the pedagogical skills necessary to teach science and mathematics using more standards-based techniques. The National Science Foundations (NSF) Collaboratives for Excellence in Teacher Preparation (CETP) program has focused its effort on both teacher knowledge and pedagogical skills as a means to improve science and mathematics teachers. The CETP program is designed to better prepare science and mathematics teachers; however, as a eld, we do not yet know to what extent a large-scale national program such as CETP can actually improve teacher preparation. The purpose of this study is to better understand the extent to which a large-scale national program can impact the instructional practices of teachers and the institutional characteristics of the universitylevel courses taken by preservice teachers. The NSF is particularly concerned about helping to prepare high-quality mathematics and science teachers because of its mandate to improve the eld of science, mathematics, engineering, and technology. Although the NSF funds several programs focusing on improving the quality of instruction in science and mathematics, the CETP program was one of the few programs targeted directly at teacher preparation. According to the NSF, the CETP program promotes comprehensive change in the undergraduate education of future teachers by supporting cooperative, multiyear efforts to increase substantially the quality and number of teachers well prepared in science and mathematics, especially members of traditionally underrepresented groups. Furthermore, the program was unique in that these goals were to be reached through the vertical integration of inquiry teaching and institutional cultural change. The types of teaching that future teachers were exposed to in their preparation programs was designed to match the type of teaching they would be expected to use once they became teachers. This teaching was to be supported by changes in the cultures of educational institutions. The goal of cultural change was to provide for long-term, sustainable improvements in science and mathematics teacher preparation. Not only was this approach to changing teacher preparation unique, but also the many individual projects funded by the program provided many different pathways to achieving these goals, thereby providing the opportunity to test the ideas themselves in multiple iterations not just in one application. If this national program were shown to be successful, it would indicate that the concept represents a viable approach to improving teacher preparation. As would be expected, the causal chain in the CETP program is quite complex, involving several different types of institutions and participants. The model behind the CETP program was based on improving the culture at teacher education institutions. Improving the culture was to be achieved through increasing collaboration between institutions such as community colleges and K12 schools, and increasing collaboration within institutions such as between faculty in the sciences and education. Theoretically, the enhanced culture would be much more inquiry oriented, collaborative, and able to capitalize on the strengths of all the different partners. In addition, the introductory and capstone science, technology, engineering, and mathematics (STEM) courses, along with the educational methods courses and eld experiences for preservice teachers would be improved. These improved courses would, in turn, lead to teachers who were better prepared in their content areas and better prepared to teach using standards-based techniques as described in the National Research Council (NRC, 1996, 2001) and National Council of Teachers of Mathematics (NCTM, 1989, 1991, 2000). Standards-based instructional techniques were dened according to the NRC and NCTM, and include a focus on a more investigative and studentcentered instructional methods such as collecting and analyzing data, studying real-world problems and scenarios, using technology including computers, using a variety of assessment
Journal of Research in Science Teaching. DOI 10.1002/tea

1350

LAWRENZ, HUFFMAN, AND GRAVELY

methods, and helping students develop problem solving and critical thinking skills. One of the goals of the CETP program was to create such standards-based experiences in the courses at the university level to provide preservice teachers with the type of learning experiences the teachers would be expected to use with their own students when they began their teacher careers. The learning environments these teachers provided for their students would also be more conducive to learning, and therefore, student understanding of science and mathematics would improve. The purpose of this study was to examine the impact of the CETP program using data collected from the CETP projects across the country. The sample was not a nationally representative sample of teachers or students, but rather, the data in this paper are from CETPs across the nation. The authors were funded separately by the NSF to externally conduct this evaluative study, and were not part of the design or implementation of any individual CETP project. The different types of impact investigated follow the steps in the CETP model described previously and include data from two different settings: institutions of higher education, and K12 science and mathematics classrooms. The focus of this study in institutions of higher education was to examine the impact of the CETP program on the institutional culture and collaborations among and between faculty, and changes in instructional techniques used by higher education faculty. At the K12 level, the focus was on teaching practices exhibited by teachers participating in CETP projects compared to teachers participating in other programs. Teaching practices were examined from the perspective of the teachers, their students, and external observers. The CETP Projects Nationally, 19 CETP projects were funded by NSF from 1993 through 2000. Projects were funded for 5 years with the option for an additional 3 years. These were large comprehensive projects of several millions of dollars each. Although each project was unique, they all pursued the overriding goal of the CETP program stated previously. There were many characteristics common to all projects; they all engaged faculty from the sciences, mathematics, and education. All included several institutions of higher education (including community colleges) in a particular geographic area (e.g., a state). All included mechanisms for improving undergraduate education in the sciences and mathematics. All had relationships with K12 schools in the geographic region. All offered scholarships to students from underrepresented groups. There was a great deal of variation within these common characteristics, however. For example, although all CETPs included several institutions, the numbers ranged from 3 to 13 or more. Additionally, all had unique elements like liaisons with Tribal Colleges or industrial/science internship placements, to name a few. All of the projects focused on course reform, increasing standards-based literacy of K12 teachers, and recruitment and retention of underrepresented groups. Figure 1 includes descriptions of the CETP projects that participated in this evaluation study. These descriptions are shortened versions of program descriptions provided by the participating CETPs for presentation at one of evaluation meetings. As can be seen, all of them focused on course reform, increasing standards based literacy of K12 teachers, and recruitment and retention of underrepresented groups. Efforts listed on the gure were considered to be unique to the CETP project. Previous studies of the impact of specic CETP projects on teachers and students have been published by several individual CETPs. No published studies were found examining data across multiple CETPs. Sawada, Piburn, Judson, Turley, Falconer, Benford, and Bloom (2002) developed an observation instrument used to measure the impact of the Arizona Collaborative for Excellence in Teacher Preparation (ACEPT). They found that reformed teaching practices used by the ACEPT teachers were signicantly related to higher student achievement in science and mathematics. In a different study of the ACETP working with in-service teachers, it was
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1351

Funding CETP Yr 1993 Louisiana Collaborative for Excellence in the Preparation of Teachers (LaCEPT) 1993 Maryland Collaborative for Teacher Preparation (MCTP)

Description Statewide; 12 4-yr institutions and associated K-12 school districts Created faculty internships Last three years of evaluation efforts based at Towson University after the first five years at the University of Maryland; five 4-yr and four 2-yr HE institutions in partnership with area K-12 school districts Resident internships & integrated math and science courses Statewide initiative based at Montana State University ? Bozeman. Involved five state colleges & universities, seven tribal colleges, and both public & private K-12 school districts Recruitment of teacher candidates from underrepresented groups Temple University, Community College of Philadelphia, & Philadelphia K-12 school district Interdisciplinary courses, revised articulation, increased the number of Professional Development schools Involved three 4-yr & three 2-yr HE institutions and area K-12 school districts Course reform Phoenix/Tempe region; based at ASU & included two 2-yr colleges and area K-12 school districts Established district-based post bac program Los Angeles region; involved five 4-yr and four 2-yr HE institutions El Paso, TX, based at UTEP; involved UTEP & El Paso Community College as well as three local K-12 school districts Recruitment and retention Located in New York City and involves six HE institutions and area K-12 school districts New teacher education curriculum & materials California Bay area, based at San Jose University; two 4-yr & 4 2-yr HE institutions with area K-12 school districts

1993

Systemic Teacher Excellence Preparation (STEP)

1994

1994

1995

1995

1995

Collaborative for Excellence in Teacher Preparation of Philadelphia County (Philadelphia/Temple) Rocky Mountain Teacher Excellence Collaborative (RMTEC) Arizona Collaborative for Excellence in Teacher Preparation (ACEPT) LA Collaborative for Teacher Excellence (LACTE) Partnership for Excellence in Teacher Education (PETE) New York Collaborative for Excellence in Teacher Preparation (NYCETP) Mathematics and Science Teacher Education Partnership (MASTEP) Oklahoma Teacher Education Collaborative (O-TEC) Virginia Collaborative for Excellence in the Preparation of Teachers (VCEPT)

1995

1996

1996

1996

Involved eight 4-yr & one 2-yr HE institutions, and area K-12 school districts Recruitment and retention Based at Virginia Commonwealth University. Involved three 4-yr and three 2-yr HE institutions as well as the Richmond K-12 school districts

Figure 1.

List of CETPs.

Journal of Research in Science Teaching. DOI 10.1002/tea

1352

LAWRENZ, HUFFMAN, AND GRAVELY

1997

1998

2000

2000

2000

Science, Technology, Engineering, & Mathematics Teacher Education Collaborative (STEMTEC) Collaborative for Excellence in Teacher Preparation of Puerto Rico (PR-CETP) Collaborative for Excellence in Teacher Preparation in Pennsylvania (CETPPA) Maine Mathematics and Science Teaching Excellence Collaborative (MMSTEC) Texas Collaborative for Excellence in Teacher Preparation (TxCETP)

Involves eight 4-yr HE institutions in Massachusetts including the Five-College Consortium, three community colleges, and several K-12 school districts Recruitment of teacher candidates from underrepresented groups Involves seven 4-yr HE institutions and area K-12 school districts Recruitment and retention Statewide; includes 12 4-yr & 12 2-yr HE institutions in partnership with area K-12 school districts Recruited teacher candidates from underrepresented groups; paired content & ed supervisors Three HE institutions & the Maine Math and Science Alliance in partnership with area K-12 school districts Recruitment of math and science teachers Involves ten 4-yr HE institutions (including 8 TAMU), ten 2-yr, and area K-12 school districts Capstone courses

Figure 1.

(Continued)

reported that in-service teachers who took the reformed undergraduate courses demonstrated signicantly higher scores on a measure of reformed instruction, and the students of these teachers demonstrated signicantly higher achievement (Adamson, Banks, Burtch, Cox, Judson, Turley, Benford, and Lawson, 2003). A study on the Maryland Collaborative for Teacher Preparation (MCTP) found that CETP teacher candidates had attitudes and beliefs more in line with reform goals than non-CETP teachers (McGinnis, Kramer, Shama, Graeber, Parker, & Watanabe, 2002). Other studies on teachers prepared by the Maryland Collaborative for Teacher Preparation reported that perceptions of school culture was an important factor in the implementation of reform-based instruction (McGinnis, Parker, & Graeber, 2004), and that teachers prepared by the MCTP were more likely to have reform-oriented perspectives (McGinnis & Marbach-Ad, in press; McGinnis, Roth McDufe, & Graeber, 2006; Marbach-Ad, & McGinnis, in press). In a study of the Massachusetts CETP it was reported that institutional and sociocultural difculties made it challenging to create a collaborative reform effort (Davis, Feldman, Irwin, Pedevillano, Capobianco, Weiss, and Bray, 2003). Overall, the studies of individual CEPT projects suggest that collaborative reform efforts have had a positive impact on students and teachers. These studies also highlight the importance of better understanding the larger national impact of such reform. The contributions to the literature by individual CEPT projects are valuable, but what is missing in the literature is the analysis of data across multiple CETPs, which this present study provides. Methodology Purpose The purpose of this study was to examine the national impact of the CETP program in two different settings: institutions of higher education, and K12 science and mathematics classrooms. The focus of this study was to examine the impact of the CETP program on the institutional culture and collaborations among and between faculty, and changes in instructional techniques used by higher education faculty and K12 teachers. Specically, this study was guided by the following questions:
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1353

1. To what extent did the CETP program impact the collaboration and focus of university faculty on instructional issues? 2. To what extent did the CETP program impact the instructional techniques used by university faculty? 3. Do K12 teachers who participated in CETP projects view their preparation programs differently from teachers who participated in other preparation programs? 4. Are the instructional practices exhibited by K12 teachers who participated in CETP projects different from the instructional practices exhibited by teachers who participated in other preparation programs?

Instrumentation Several different sources of data were identied for this study. At the higher education level data were gathered from faculty and principal investigators. At the K12 level data were gathered from principals, teachers, students, and evaluators who observed science and mathematics classes. Instruments included Web-based surveys for the higher education faculty members and the principle investigators (PIs); paper and pencil surveys for K12 principals, teachers, and students; and classroom observation protocols. All surveys were developed in collaboration with representatives from the participating CETP projects. Instruments were developed through a series of revisions using face-to-face and distance discussions, pilot sample data, opinions of expert consultants, and eld testing. The instruments were designed to assess issues surrounding the CETP model presented previously. Individual items were obtained from existing evaluation instruments developed by CETP projects or other sources such as the surveys used by Horizon Research Inc. in their national curriculum surveys of their evaluation of the Local Systemic Change Initiative. Initial draft instruments were considered and revised by teams of three experts from the various CETP projects and then by the full group of evaluators from the CETPs both via the Web and face to face. Pilot data were collected from small groups of each type of respondent and talk alouds with individuals. These efforts showed that the survey items were being interpreted as expected, and that the terms used, such as making models, were likely to be clear to the respondents. An external survey expert also reviewed the items and the pilot data. Instruments were revised based on feedback and were again reviewed by the representatives from the participating CETP projects. These versions were then used in a eld test. Field test data were obtained from 15 PIs, 84 faculty, 26 principals, 146 teachers, and 893 students. There were also 117 classroom observations. These eld test data were used to nalize the instruments. The nal instruments were determined by all of the participating CETPs to be valid for examining the impact of the program. Each item on the survey instruments was included to address a concept deemed to be important to assessing impact. Therefore, calculating internal consistency reliability measures was inappropriate because the items were not viewed as parts of a larger overall construct as would be the case in a test or scale. This belief was substantiated by exploratory factor analyses conducted on the survey data. These analyses did not reveal any underlying structure within the items, suggesting that each item measured a unique concept. The instruments are available at the following Web site: http:/ /education.umn.edu/CAREI/CETP/ default.html. Higher Education Instruments. Surveys were designed to measure the level of collaboration within and among institutions of higher education and the amount of use of various instructional techniques before and after the CETP project. The faculty survey contained 36 main items and some of the main items contained subsets of related questions. The PIs of each CETP were asked 11 questions about the conduct and outcomes of their projects, including the extent to which their
Journal of Research in Science Teaching. DOI 10.1002/tea

1354

LAWRENZ, HUFFMAN, AND GRAVELY

CETP improved interactions among and between STEM faculty, education faculty, and K12 schools. K12 Instruments. For K12 settings the teacher survey was used as the model for the other surveys. The teacher survey contained 35 main items, some of which included subsets of questions. Principals and students were asked similar, although fewer questions than teachers on the surveys. The principal survey had 13 main items while the student survey had 21 items (no student items had subsets). The teacher and student surveys focused on measuring how often teachers used various standards-based instructional methods as dened by the NCTM and NRC in their classroom. Teachers were also asked about various characteristics of their teacher preparation programs. Principals were asked questions about teachers instructional practices and general questions related to their schools characteristics. The observation protocol was developed to gather actual classroom data on a sample of CETP prepared and non-CETP prepared teachers. The protocol included aspects of the Horizon Research observation protocol developed for use in the Local Systemic Change program funded by NSF (Banilower, Boyd, Pasley, & Weiss, 2006) and development procedures for the CETP observation protocol followed those outlined above for the surveys with pilot and eld testing. Appeldoorn (2004) provided detailed information on the development and characteristics of the protocol. She concluded that the protocol was highly internally consistent with a coefcient alpha of 0.9. Mean ratings for a sample of observers indicated that all sections were clear and understandable. Reports from CETP projects about their observers and from the training sessions conducted by the central CETP program evaluation team indicated that intra- and interrater consistency could be obtained. The individual CETP projects were responsible for hiring and training specialists to conduct the observations using the protocol. To ensure the ratings across CETP projects were consistent, each project was provided a video-training guide that described and provided actual classroom video examples for all of the rating categories. The video-training guide also included several scored, video classroom segments for practice in obtaining interrater reliability. In addition, three national in-person training sessions were held for observers who desired more personalized training. At these national workshops and at training sessions conducted at the individual CETP projects, observers watched sample video clips of science and mathematics classes and rated each aspect of the class. Ratings were discussed to gain consensus on the denition of each rating. The process was repeated multiple times until observers could consistently rate classes. The classroom observation protocol contains 12 evaluative ratings of classroom activity as well as an overall capsule rating. The overall capsule rating was identical to the scale developed by Horizon Research for use on the Local Systemic Change Evaluation (Banilower et al., 2006). That scale has ve levels, with the middle level of three having three scoring options: low, solid, and high. Therefore, the scale essentially has seven categories and we used scores of 17 to represent the levels. A score of 1 is dened as ineffective instruction. A score of 2 is elements of effective instruction. A 3, 4, or 5 is indicative of three different levels of beginning stages of effective instruction. A score of six is accomplished effective instruction. A score of 7 is exemplary instruction.

Sample Although the NSF encouraged all 19 CETP projects funded from 1993 to 2000 to participate in this national impact evaluation, participation in the evaluation was voluntary, and in the end, 12
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1355

of the 19 CETPs actually provided data for this study. Different CETPs also provided different types of data so the numbers of projects providing any one type of data varied. In addition to returning different surveys, not all respondents answered all of the items on their surveys. There appeared to be no pattern in the missing data so for most reporting all of the people responding to a particular item were included. To use multivariate analysis techniques, the whole set of respondents had to have responded to all of the included items so the Ns for those analysis tend to be somewhat smaller. Higher education data were collected from faculty at all institutions involved in a CETP project. The faculty asked to respond were those who had been involved in the CETP program in some way. Additionally, as is always the case, responding to the survey was voluntary. Response rates from contacted faculty were quite high however, because requests for responses came from the individual CETP project personnel and follow-up requests were sent to all nonrespondents. PIs were also asked to respond to surveys. Data were obtained from 37 PIs, 232 faculty members (183 STEM and 49 education) from 132 institutions. The STEM academic areas of faculty included astronomy, biochemistry, biological sciences, chemistry, engineering, geology, mathematics, physics, and general science. The STEM and education faculty came from all ranks, with about 25% full professors, 26.3% associate, 23.3% assistant professors, and 22.8% instructors/adjunct faculty. Fifty-one percent of the faculty members were female. Approximately 58.6% of the students in the classes of these faculty said they intended to become licensed as teachers indicating that these classes were indeed part of the teacher preparation programs. The K12 sample included new teachers (15 years of experience) who had participated in a CETP project, and matched new teachers who had not participated in a CETP project but who had the same number of years of experience, were teaching at the same grade level or same subject, and at the same school or a nearby school with similar geographic and demographic characteristics. To obtain the sample, the CETP projects located their recent graduates who were teaching and asked them to complete surveys. The projects then located matched teachers in similar teaching situations. Principals of the schools where CETP and non-CETP teachers were teaching and students of CETP and non-CETP teachers were also asked to complete surveys. Using these procedures, K12 data were obtained from 441 teachers (262 CETP and 179 nonCETP), 256 principals, and 7,289 students 3,702 females and 3,331 males (with the remaining students not specifying their gender). Only 288 (169 CETP and 119 non-CETP) of the teachers provided information to categorize their classes. Of these, there were 83 elementary level classes, 98 middle or junior high level classes, and 108 high school level classes. Of the total responding teachers, 386 reported having a specialty in science or mathematics (generally middle school or high school teachers) or elementary or other, 127 said science was their specialty, 84 said mathematics, 59 said elementary, and 59 said other. Table 1 includes the K12 sample teacher characteristics broken down by CETP and non-CETP. As can be seen, the percents in each category are quite similar except for a slightly larger percentage of non-CETP teachers at the elementary school level and slightly more CETP teachers reporting a science specialty. Therefore, it appears that the matching procedures did result in samples of CETP and non-CETP teachers with similar characteristics. The schools were mostly urban, with 44.8% of principals reporting that their students came from urban environments, 18.3% reporting suburban, 15.9% reporting town or small city, and 21% reporting rural. School enrollments were quite varied, with numbers of students varying from 28 to 2,900, with a mean of 797 students and a standard deviation of 536. The schools had somewhat supportive science environments, with 36.7% of principals reporting less than adequate science laboratory facilities but only 8.2% reporting the facilities as meeting all needs. In terms of money
Journal of Research in Science Teaching. DOI 10.1002/tea

1356

LAWRENZ, HUFFMAN, AND GRAVELY

Table 1 Percentage of K12 CETP and non-CETP teachers reporting various characteristics CETP n 262 Elementary Middle school High school Specialty Science Math Elementary Other Years of experience 1 2 3 4 5 23.0 35.4 41.7 35.6 21.8 27.8 14.9 21.8 32.5 25.4 12.3 7.8 Non-CETP n 179 36.9 31.6 31.6 28.6 21.4 34.2 15.8 17.9 34.5 25.4 15.3 6.9

CETP, Collaboratives for Excellence in Teacher Preparation program.

available to provide supplies for science and math instruction, 56% of the principals reported less than adequate money and only 4.8% reported the amount of money as meeting all needs. There were 101 classes taught by teachers prepared by CETP projects and 110 classes taught by non-CETP prepared teachers observed. These classes were supposed to have been selected randomly from the teachers who completed surveys, but because the teachers had to agree to be observed, the sample was more opportunistic. Observers generally knew if the class were being taught by a CETP or non-CETP teacher. The classes ranged from kindergarten through grade 12 with modes of 12 classes for fourth grade, 11 for fth grade, and 13 for sixth grade for CETP prepared teachers and modes of 13 classes for third grade, 14 for fth grade, and 15 for sixth grade for the non-CETP prepared teachers. More of the observations of CETP prepared teachers were conducted while teachers were conducting mathematics lessons than while conducting science lessons (73%/23%), while the distribution of science and mathematics lessons for the non-CETP prepared teachers was 50/50. Seventy-nine percent of the CETP prepared teachers were female compared to 64% of the non-CETP prepared teachers. Fourteen percent of the classrooms of the CETP prepared teachers were considered by the observers as sparsely equipped compared to 20% of the classrooms of the non-CETP prepared teachers. In terms of the arrangement of the classrooms to support interaction among students 16% of both the classrooms of CETP and non-CETP prepared teachers were viewed as inhibiting interactions among students. Data Analyses Several different types of data analyses were conducted. Demographic information and individual item data were considered straightforwardly, and are presented in tables as percentages of the different responding groups. For analyses where several items were involved, such as the teacher and student reports of classroom activities, multivariate analysis of variances (MANOVAs) were conducted to compensate for the large numbers of analyses and then if the MANOVAs were signicant, the individual ANOVA results were considered. The Wilks Lambda statistic was used for determining the signicance of all the MANOVA analyses. Additionally, the analysis procedures were conducted with list-wise deletions so the numbers of subjects in each of the analyses varied from the total numbers of subjects who provided at least some data.
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1357

The K12 data were nested with students within classrooms and teachers within CETP projects. There were also unequal numbers of teachers and students at each CETP project. Some CETP projects were large, with lots of students and teachers; others were smaller, with fewer students and teachers. Therefore, the data from the CETP projects were weighted so that each project had an equal number of teachers and students. This allowed each CETP to be equally represented in the results. Weighting the data ensured that the larger CETPs with more teachers and students did not skew the results. In other words, weighting the data provided the most representative national view of the impact of the CETP program. For the K12 teachers weighting the data reduced the number of projects included in the analyses from 12 to 11 because one CETP had too few teachers to be weighted. For the student survey data, there were 9 CETPs in the nal analyses instead of the original 11. All the analyses that compared CETP prepared teachers versus non-CETP prepared teachers were conducted with weighting. Results Higher Education Results PIs were asked to respond to six questions about how much they believed their CETP had improved different conditions. These conditions and responses are included in Table 2. Overall, the 37 PIs felt that the CETP had improved the conditions at least somewhat. Most improvement was believed to have been in terms of interaction within institutions. The least improvement was believed to have occurred in relationships with 2-year institutions and in changing the considerations surrounding faculty merit.
Table 2 Percent of PIs reporting the amount of improvement from their CETPs Question Item: To What Extent Do You Believe the CETP Improved the way STEM faculty interact about improving instruction? Improved the way STEM and STEM education faculty interact about improving instruction in higher education? Improved the way STEM and STEM education faculty interact with the faculty at 2-year institutions of higher education? Improved the way STEM, STEM education faculty and K12 teachers interact about improving instruction? Improved the way higher education and K12 schools interact? Improved the way institutions of higher education consider teaching performance and/or instructional improvement in promotion/tenure and merit decisions? Amount of Improvement (n 37) No Change 2.7 2.7 A Little 2.7 5.4 Some 16.2 16.2 Substantial 67.6 62.2 Exceptional 10.8 13.5

12.9

22.6

32.3

22.6

9.7

5.4 13.9 16.2

8.1 13.9 24.3

40.5 19.4 37.8

32.4 41.7 13.5

13.5 11.1 8.1

CETP, Collaboratives for Excellence in Teacher Preparation program; STEM, science, technology, engineering and mathematics. Journal of Research in Science Teaching. DOI 10.1002/tea

1358

LAWRENZ, HUFFMAN, AND GRAVELY

Faculty were asked several questions about changes in their perceptions related to the goals of the CETP program: whether or not they interact more with faculty in other institutions, whether or not they interact more with faculty in their own institution, whether or not they changed their teaching style, whether their course inuenced changes in other courses. These results are presented in Table 3. The responses to these items were considered by gender and by three types of professor status (i.e., adjunct, assistant, and associate/full). Faculty from all levels felt that there were changes with well over half the faculty reporting that change had occurred in the various ways covered by the items. The comparisons within the categories of gender and type of appointment showed quite a bit of variation; however, the female faculty were consistently more likely to report that changes had occurred. It also appeared that
Table 3 Percentage of faculty responding positively to questions about change and instruction in their institution over the past 5 years by position and gender Adjunct Professor Question Item In the past few years have you made substantial changes in your teaching style? Have your courses inuenced changes in other courses in your institution?1 In the past few years have you observed any colleagues teaching and then discussed your observations with them (or visa versa)?1 In the past 5 years has there been any change in the way you and your colleagues interact with the faculty in other institutions?1 In the past 5 years has there been any change in the way you and your colleagues interact with the faculty in other area(s) at your institution?1 What percent of your professional time do you expend on teaching and/or curriculum reform?2 What percent of your faculty colleagues are actively involved in improving their teaching and/or in reforming curriculum?3 Male n 16 64.7 18.8 58.8 Female n 31 82.4 37.1 51.4 Assistant Professor Male n 22 81.8 26.1 65.2 Female n 29 77.4 51.7 54.8 Associate or Ful Professor Male n 69 70.6 47.0 60.3 Female n 48 75.0 67.4 77.1

56.3

67.7

72.7

65.5

63.8

77.1

50.0

62.1

95.0

65.5

71.6

71.7

81.3

74.3

47.8

67.7

48.5

53.2

46.7

31.2

13.6

40.0

19.4

29.2

Note: 1 yes/no answers. 2 the percent of faculty saying that 70% or more of professional time is expended on teaching and/or curriculum reform.3 the percent of faculty saying that 70% or more of faculty colleagues are actively involved in improving their teaching and/or in reforming curriculum. Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1359

associate and full professors were more likely to report changes. This might be a factor of people in those groups having been at the institutions for longer periods of time, and therefore more able to recognize change. The lowest percentage was in amount of inuence changes in courses had on other courses. Faculty were asked 17 questions about how they taught before the CETP program began and how they presently teach. Statistical comparisons were conducted on these items using a repeated-measures MANOVA to determine if the change from before the CETP to after were signicant. As shown in Table 4, all but one of the 17 items showed a signicant pre- to postdifference. The item showing no difference was how often do students complete assessments/assignments that include multiple choice/short answer items? The largest pre- to postdifferences were on items related to the use of technology especially for communication, how often students were asked to write descriptions of their reasoning, how often students were asked to design and make presentations, and how often students were asked to make connections to other elds. The difference scores for each item were also examined by gender, by level of appointment (adjunct, assistant, and associate or full professor) and for any interactions using MANOVA. There were no signicant main effects, that is, no differences for gender or for type of position, but there was a signicant interaction. The individual analysis of variance (ANOVA) tests showed that only one individual interaction was signicant. The signicant interaction was found for giving assessments that involve full length papers or reports (F 4.39, p .02). Female assistant professors became less likely to assign full-length papers, while other female professors became more likely to do so. Male assistant professors were more likely to require papers than male adjunct or tenured professors. K12 Results To examine the impact of the CETPs it was important to determine which aspects of the teacher preparation program were different. Teachers were asked eight different questions about what was included in their teacher preparation programs (i.e., computer-assisted instruction, science methods courses, mathematics methods courses, science or mathematics capstone courses, eld experiences in education in addition to student teaching, and eld experiences in science and mathematics) and how they would rate it overall. Teachers were also asked if their preparation program included information about the mathematics or science standards. Table 5 includes the teacher responses to these questions broken down by elementary, science, and mathematics teachers and by CETP and non-CETP. There were some similarities between the CETP and non-CETP teachers in terms of what they perceived was covered in their teacher preparation programs. As would be expected, it was uncommon for science teacher preparation programs to include mathematics methods courses and conversely for mathematics teacher preparation programs to include science methods courses. However, elementary teachers reported that their preparation programs included both science and mathematics methods. Capstone courses were also uncommon, especially for students preparing to be elementary school teachers, but these types of courses were slightly more common in non-CETP programs. There were also differences between CETP and non-CETP teacher preparation programs and among science, mathematics, and elementary teacher education programs. In general, there were fewer differences between CETP and non-CETP elementary teachers than between CETP and non-CETP science and mathematics teachers. More of the CETP science and mathematics teachers reported that their programs included information on computer use while more of the non-CETP elementary teachers reported computer use. Substantially fewer
Journal of Research in Science Teaching. DOI 10.1002/tea

1360

LAWRENZ, HUFFMAN, AND GRAVELY

Table 4 Mean scores and standard deviations for faculty pre- and postratings of frequency of use of various instructional techniques Faculty Pre- and Postratings n 139 Preratings Question Item How often do students work with other students where the whole group gets one grade* How often do students participate in whole-class discussions during which the teacher talks less than the students* How often do students use or make models, for example, physical, conceptual, or mathematical models* How often do students write descriptions of their reasoning* How often do students work on problems related to real-world or practical issues* How often do students perform investigative activities that include data collection and analysis* How often do students make connections to other elds science, technology, engineering, and mathematics STEM and non-STEM* How often do students design and make presentations that help them learn class concepts* How often do students evaluate the extent of their own learning* How often do students complete assessments/assignments that include problems with complex solutions* How often do students complete assessments/assignments that include portfolios* How often do students complete assessments/assignments that include multiple choice/short answer items How often do students complete assessments/assignments that include full-length papers/reports* How often do students use technology, for example, computers or calculators to understand or explore concepts taught in class in more depth* How often do students use technology, for example, computers or calculators as a tool in investigations to gather and analyze scientic or mathematical data* How often do students use technology, for example, computers or calculators as a tool for assessment* How often do students use technology, for example, computers or calculators as a tool to communicate with you or with other students*
Note: 1 never; 2 seldom; 3 occasionally; 4 regularly. STEM, science, technology, engineering and mathematics. *p < .01.

Postratings Mean 2.50 2.83 3.01 2.94 3.14 2.95 2.99 2.90 2.65 2.99 2.26 2.80 2.66 3.14 2.92 2.48 3.13 SD 1.02 .88 .88 .92 .75 .92 .75 .89 .92 .81 1.13 1.02 .97 .80 .93 1.03 .79

Mean 1.96 2.22 2.49 2.20 2.58 2.45 2.33 2.24 2.04 2.59 1.93 2.72 2.30 2.36 2.27 1.96 2.19

SD 1.07 1.01 1.03 1.02 .91 1.07 .94 1.05 .95 .96 1.19 1.07 1.04 1.05 1.05 1.09 1.08

of the non-CETP mathematics teachers compared to all others reported having extra eld experiences in education. Surprisingly, eld experiences in science or mathematics were most common for students in elementary teacher preparation programs. Receiving information about the national standards was least common for science preparation programs and most common for elementary teacher and CETP mathematics teacher preparation programs. Interestingly, although
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1361

Table 5 Percentage of CETP and non-CETP math and science and elementary teachers reporting different characteristics of their teacher preparation programs Math Teachers Question Item Did your teacher preparation program require courses teaching computer-assisted instruction?1 Did your teacher preparation program require science methods courses?1 Did your teacher preparation program require mathematics methods courses?1 Did your teacher preparation program require Capstone or Jr./Sr. level science or mathematics (not education) courses, for example, culminating, integrating experiences?1 Did your teacher preparation program require eld experiences in education (in addition to student teaching)?1 Did your teacher preparation program require eld experiences in mathematics/science?1 Did you receive information about national, state, and/or professional Science, Technology, Engineering, and Mathematics (STEM) standards, for example, National research Council (NRC) science standards, National Council of Teachers of Mathematics (NCTM) mathematics standards, from your teacher preparation program or institution?1 How would you rate the level of quality of your teacher preparation program?2 Science Teachers Elementary

CETP Non-CETP CETP Non-CETP CETP Non-CETP n 61 n 30 n 84 n 48 n 50 n 48 54.1 9.8 91.8 31.1 50.0 6.7 63.3 33.3 58.3 85.7 6.0 20.2 43.8 72.9 4.2 29.2 64 86.0 90.0 12.0 68.8 79.2 85.4 12.5

67.2 49.2 71.2

36.7 30.0 58.6

67.9 36.9 51.2

66.7 45.8 60.9

86.0 62.0 73.9

81.3 58.3 80.4

46.5

51.7

55.5

44.5

62.0

68.8

Note. 1 percent of yes answers; 2 percent of 3 or 4 answers (1 less than adequate; 2 adequate; 3 more than adequate; 4 exceptional).

more CETP teachers generally report the listed activities as occurring in their preparation programs, fewer CETP mathematics and elementary teachers than non-CETP mathematics and elementary teachers reported that the quality of their preparation program was more than adequate or exceptional. ANOVAs conducted on these differences in quality ratings between teachers participating in CETP and non-CETP programs, however, were not signicant. As discussed previously, the numbers of K12 teacher and student responses were adjusted for the analyses so that the responses from teachers and students from each CETP were weighted equally. The 21 items related to frequency of instructional activities on both the teacher and student surveys were examined simultaneously through the use of MANOVA to compare CETP
Journal of Research in Science Teaching. DOI 10.1002/tea

1362

LAWRENZ, HUFFMAN, AND GRAVELY

and non-CETP teachers. There was no signicant difference on the set of items for the teachers, but there was a signicant difference for the students. Therefore, the individual item comparisons were not conducted for the teachers but were conducted for the students. The mean scores for both the teachers and students are presented in Table 6 along with the results of the individual item signicance testing for the students. The individual ANOVAs for the students showed differences on 12 of the 20 items, with all favoring the CETPs. Overall, teachers and students in both groups reported somewhat low levels of use with most means approximately 3 on a four-point scale, where 3 represents occasionally. A MANOVA on the observation items on the observation protocol showed a signicant difference; therefore, the individual ANOVA results were considered. As shown in Table 7, the comparison of the observers ratings of CETP and non-CETP classrooms showed signicant differences on 9 of the 12 evaluative items as well as on the nal capsule rating. All of the differences favored the CETP classrooms. Overall, capsule ratings suggest that the CETP classes were, on average, on the higher end of beginning stages of effective instruction, while the nonCETP classes were rated on the lower end of beginning stages of effective instruction.

Conclusions The results provide the following answers to the questions that guided this study.
1. The CETP program was believed by the PIs and participating faculty to have increased the collaboration of university STEM and STEM education faculty with each other within their institutions and with STEM and STEM education faculty from other institutions. Additionally, the responding faculty reported changing their teaching styles, observing colleagues teach and spending substantial amounts of time on their teaching. 2. Faculty involved in the CETP program reported signicant changes in the use of standards-based instructional techniques. 3. Teachers who participated in the CETP preparation programs viewed their programs differently than teachers who participated in other preparation programs. In general, more CETP teachers reported their preparation programs as having more computer assisted instruction, more specic topic methods classes, more eld experiences, and more information about national standards, while more non-CETP teachers reported having capstone courses. 4. The instructional practices of teachers prepared by CETP programs were viewed by students and observers as signicantly different from the instructional practices of teachers prepared in other programs.

This evaluative study of the CETP program also provided an opportunity to test the plausibility of the model underlying the program. The program espoused the notion of the vertical integration of standards-based teaching through institutional cultural change. The study also ts into the research agenda for teacher preparation suggested by the American Educational Research Associations Studying Teacher Education report (Cochran-Smith & Zeichner, 2005). CochranSmith and Zeichner recommend uncoupling the individual characteristics of teacher education from the approaches to teacher education, providing multi-institutional studies, and developing common outcome measures. The model for the CETP program began with changing the culture in the teacher preparation institutions. One of the main goals of the CETP program was to increase collaboration within and between institutions. It appears there was some increase in the amount of collaboration within
Journal of Research in Science Teaching. DOI 10.1002/tea

Table 6 Mean scores and standard deviations for CETP and non-CETP teachers and students for frequency of use of various instructional techniques Teachers CETP n 145 Mean 2.76 3.48 3.50 2.61 2.82 3.25 2.96 3.34 3.18 3.04 3.08 3.17 3.10 2.08 .78 .77 1.06 2.64 .64 3.13 2.70 3.09 3.03 2.00 .75 3.00 .78 .80 .88 .78 .77 .97 .80 .69 2.86 3.37 .80 .67 .66 2.99 .82 2.37 2.37 2.76 2.99 2.37 2.42 2.96 3.03 1.95 .77 2.74 .79 2.28 .73 .77 3.19 2.57 1.00 .98 3.33 2.48 .87 .95 .95 .97 1.06 .99 .97 .98 1.01 .90 .89 1.01 .64 3.42 .68 2.92 .88 .60 2.78 .68 2.57 1.01 2.50 2.81 3.24 2.36 2.23 2.38 2.36 2.72 2.90 2.40 2.27 2.93 2.88 1.83 SD Mean SD Mean SD Mean Non-CETP n 120 CETP n 1,864 Non-CETP n 1,088 SD 1.01 .97 .91 .98 .96 1.01 1.07 1.00 .95 .99 1.03 .91 .96 1.00
1363 IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

Students

Question Item

How often do students have a voice in decisions about class activities How often is new information based on what students already know about the topic** How often do students have enough time to learn what is required** How often do students work with other students where the whole group gets the same grade** How often do students participate in whole-class discussions during which the teacher talks less than the students How often do students use or make models, for example, physical, conceptual, or mathematical How often do students write descriptions of their reasoning How often do students work on problems related to real world or practical issues How often do students do activities that include data collection and analysis** How often do students make connections to other elds: science, technology, engineering, and mathematics, STEM and non-STEM How often do students design and make presentations that help them learn class concepts** How often do students determine how much they know about something How often do students complete assessment/assignments that include problems with complex solutions** How often do students complete assessment/assignments that include portfolios**

Journal of Research in Science Teaching. DOI 10.1002/tea

1364

3.30 2.41 3.05 2.86 2.65 2.27 1.05 2.16 1.09 1.97 1.07 .96 2.58 .97 2.69 1.11 .85 2.70 .98 2.76 1.00 2.59 2.59 1.98 .80 2.78 .90 2.74 1.04 2.59 1.06 1.05 1.13 1.05 .86 2.27 .91 2.30 1.06 2.11 1.06

How often do students complete assessment/assignments that include multiple choice/short answer items** How often do students complete assessment/assignments that include full-length papers/reports** How often do students use technology, for example, computer or calculators to understand or explore in more depth concepts already taught in class** How often do students use technology, for example, computers or calculators as a tool in investigations to gather and organize information** How often do students use technology, for example, computers or calculators as a tool for checking understanding testing* How often do students use technology, for example, computers or calculators as a tool to communicate with you or other students

.74

3.18

.85

3.04

.89

2.92

.99

Journal of Research in Science Teaching. DOI 10.1002/tea LAWRENZ, HUFFMAN, AND GRAVELY

Note: 1 never; 2 seldom; 3 occasionally; 4 regularly. *p < .05. **p < .01. for student comparisons.

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1365

Table 7 Mean scores and standard deviations for classroom observation protocol rating items for CETP and non-CETP teachers CETP n 76 Question Item This lesson encouraged students to seek and value alternative modes of investigation or of problem solving** Elements of abstraction (i.e., symbolic representations, theory building) were encouraged when it was important to do so Students were reective about their learning** The instructional strategies and activities respected students prior knowledge and the preconceptions inherent therein* Interactions reected collaborative working relationships among students (e.g., students worked together, talked with each other about the lesson), and between teacher/ instructor and students The lesson promoted strongly coherent conceptual understanding** Students were encouraged to generate conjectures, alternative solution strategies and ways of interpreting evidence** The teacher/instructor displayed an understanding of mathematics/science (e.g., in her/his dialogue with students)* Appropriate connections were made to other areas of mathematics/science, to other disciplines, and/or to real-world contexts, social issues, and global concerns Students understanding of mathematics/science as a dynamic body of knowledge generated and enriched by investigation** Students understanding of important mathematics/science concepts** Students capacity to carry out their own inquiries** Capsule description of the quality of the lessona,** Mean 2.97 3.14 3.01 3.58 3.46 SD 1.18 1.20 1.18 .91 1.41 Non-CETP n 85 Mean 2.51 3.01 2.48 3.18 3.08 SD 1.2 1.17 1.08 1.18 1.50

3.49 3.17 4.25 2.57 3.04 3.29 2.93 4.70

.92 1.27 .97 1.13 1.08 .85 1.09 1.60

2.79 2.49 3.94 2.55 2.35 2.56 2.41 3.32

1.24 1.35 .97 1.21 1.32 1.12 1.24 2.03

Note: 1 not at all; 5 to a great extent. 1 not standards based instruction; 7 exemplary standards based instruction. a The capsule rating was scored on a seven-point scale. *p < .05. **p < .01.

institutions with the STEM faculty interacting with each other more and also interacting more with the STEM education faculty. There was less progress in terms of interactions with faculty from 2year institutions. This may have resulted in less impact on recruitment of diverse students and less attention to dealing with diverse students in introductory classes. The results from survey items suggest that this might be true. The faculty members indicated they were already dealing with diversity through other projects and grants, so CETP was seen to have had little impact in that area. One major aspect of the improved culture was altering university level courses in science, technology, engineering, and mathematics. Meeting the needs of preservice teachers requires that the introductory courses model excellent instructional practices (Saunders, Keith, Yank, and Gustafson, 2003), particularly because teachers often teach as they have been taught (Thomas & Pedersen, 2003). Faculty members reported that they changed their teaching practices, suggesting the culture was improved. Despite the faculty at CETP institutions reporting more
Journal of Research in Science Teaching. DOI 10.1002/tea

1366

LAWRENZ, HUFFMAN, AND GRAVELY

standards-based teaching techniques, the most common teaching techniques were still only reported as being used between seldom and occasionally. The most commonly used techniques were working on problems related to real-world or practical issues, performing investigative activities that include data collection and analysis, making connections to other elds, completing multiple choice assessments, and using computers as a tool to communicate. This moderate level of standards based instruction also ts with the ndings of Smith and Gess Newsome (2004). Another step in the CETP program model was the improvement of capstone courses or eld experiences in science or mathematics. These types of experiences have been shown to be benecial to future teachers (Melear, Goodlaxson, Warne, & Hickok, 2000). Therefore, the low percentage of students reporting that these occurred in their preparation programs is an area where improvement is possible. It is disappointing that these seemed somewhat less common for the CETP-trained teachers. Although the use of eld experiences in education was higher than in STEM, it is not always clear in what ways eld experiences in education might be most useful (Hancock & Gallard, 2004). Some of the largest changes in faculty teaching practices were in the use of technology. The CETPs appeared to provide better training in the use of technology in teacher education programs, as recommended by Ludlow (2001). Negishi, Elder, Hamil, and Mzoughi (2003) reported that preservice teachers tend to integrate a higher degree of technology in their classes when they reported high general technology prociency so it is important that technology be used in teacher preparation courses. However, studies show that the use of technology is much lower than desirable (Ludlow, 2001; Pederson & Yerrick, 2000). In the CETP study, it is particularly surprising that there was so little use of technology in STEM classes to collect, display, and analyze data. There are many programs available that would enhance learning in university level science and mathematics courses (Lessie, 2001; Witham, Krockover, Ridgway, & Zinsmeister, 2003). Using technology for communication also increased in CETP institutions, although some of the increase was no doubt due to the national expansion of email during this time frame. McGinnis (1996) showed that this increased communication capacity can help foster the development of community within future teachers. Following through with the model, the improved culture and courses in CETP teacher preparation programs was designed to improve the environments in K12 classrooms as the CETP prepared teachers entered the classroom. The results of this study indicate that the preparation provided by the CETP institutions appears to have helped teachers use more standards-based techniques in their classrooms. However, it is important to point out that the use of standards-based techniques was not as high as one might hope, especially for a reform-based program focused on helping teachers use such techniques in their classroom. The external observers rated the CETP prepared teachers more positively than the non-CETP prepared teachers. The ratings, on average, indicate that instruction for both groups of teachers was purposeful and characterized by quite a few elements of effective practice. However, once again, neither group received ratings on the upper end of the scale (i.e., accomplished effective instruction). The K12 teachers themselves reported using the various instructional techniques in the occasionally range, while their students tended to rate the frequency of activities slightly lower. It is quite interesting to note that although the students tended to rate the frequency techniques lower than the teachers, there were still more student items showing signicant differences between CETP and non-CETP teachers classrooms than the teacher items. This is due partially to the higher power of the student contrasts, but it may also be true that the students were more sensitive to the use of standards-based techniques. The largest differences found in this study were in terms of the standards based teaching. Based on experience working with the CETPs (Lawrenz, Michlin, Appeldoorn, Goebel, &
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1367

Schultz, 2004), one of the pervasive components of the CETP program was its emphasis on achieving consensus about what constituted standards-based teaching. There were extensive discussions about standards-based teaching both locally and at national meetings. Furthermore, because of the participatory nature of the CETP evaluation, the instruments reected this shared understanding. A potential explanation that ts with all of the ndings is that the CETP institutions developed a stronger and more consistent belief about what constituted standards-based teaching. This consistency then may have contributed to the outcome of more standards-based teaching in the graduates. This notion is in keeping with the ndings of the AERA report on Studying Teacher Education (Cochran-Smith & Zeichner, 2005), which stated that program components, such as clear and consistent vision of teaching and learning are related to teacher quality and student achievement. In summary, it appears that the National Science Foundation CETP program had a positive impact on both the CETP institutions and on the science and mathematics teaching of teachers prepared by the CETP program, although the results are limited by the lack of qualitative data and the fact that it is a predominately self-report survey design. Although this study provides support for the CETP model, there are rival hypotheses that could explain these results as well. This study provides support for the CETP model of using institutional cultural change and vertical integration of standards-based teaching into university-level courses as a means of helping to better prepare science and mathematics teachers. Underlying this model, it appears that the most substantial driver may have been the development of consensus of opinion about what constitutes standardsbased science teaching. In that regard, this study provides support for wide dissemination and discussion of teaching standards. This study provides more support for the idea that having a clear and consistent message throughout different components of teacher preparation programs results in teachers more likely to implement the desired types of instruction. Such conversations can be an important means of helping teachers employ more standards-based techniques in the classrooms, and hopefully improving the achievement of all students in science and mathematics.
The views expressed in this paper are those of the authors, and do not necessarily reect the views of The National Science Foundation.

References Adamson, S.L., Banks, D., Burtch, M., Cox, F., Judson, E., Turley, J.B., Benford, R., & Lawson, A.E. (2003). Reformed undergraduate instruction and its subsequent impact on secondary school teaching practice and student achievement. Journal of Research in Science Teaching, 40, 939957. Appeldoorn, K. (2004). Developing and validating the Collaboratives for Excellence in Teacher Preparation (CETP) core evaluation classroom observation protocol (COP). PhD dissertation, University of Minnesota, Minneapolis, MN. Banilower, E., Boyd, S., Pasley, J., & Weiss, I. (2006). The LSC Capstone Report: Lessons from a decade of mathematics and science reform. Chapel Hill, NC: Horizon Research Inc. Cochran-Smith, M., & Zeichner, K. (2005). Studying teacher education: The report of the AERA panel on research and teacher education. Mahwah, NJ: Erlbaum. Davis, K.S., Feldman, A., Irwin, C., Pedevillano, E.D., Capobianco, B., Weiss, T., & Bray, P.M. (2003). Wearing the letter jacket: Legitimate participation in a collaborative science, mathematics, engineering, and technology education reform project. School Science and Mathematics, 103, 121134.
Journal of Research in Science Teaching. DOI 10.1002/tea

1368

LAWRENZ, HUFFMAN, AND GRAVELY

Darling-Hammond, L., Chung, R., & Frelow, F. (2002). Variation in teacher preparation: How well do different pathways prepare teachers to teach? Journal of Teacher Education, 53, 286302. Guarino, C., Santibanez, L., Daley, G., & Brewer, D. (2004). A review of the research literature on teacher recruitment and retention, RAND Technical Report, TR-164-EDU. Santa Monica, CA: RAND Corp. Hancock, E., & Gallard, A. (2004). Preservice Science teachers beliefs about teaching and learning: The inuence of K-12 eld experiences. Journal of Science Teacher Education, 15, 281 291. Lawrenz, F., Michlin, M., Appeldoorn, K., Goebel, A., & Schultz, M. (2004). CETP Core Evaluation: 20022003 Results. Minneapolis, MN: University of MinnesotaMinneapolis. Center for Applied Research and Educational Improvement. Lessie, D. (2001). Video capture and analysis: Seizing on computer technology to teach the physical sciences. Journal of College Science Teaching, 30, 247251. Ludlow, B.L. (2001). Technology and teacher education in special education: disaster or deliverance? Teacher Education and Special Education, 24, 143163. Marbach-Ad, G., & McGinnis, J.R. (in press). A measurement of beginning science and mathematics teachers beliefs of subject matter and instructional actions over time. Journal of Science Teacher Education. McGinnis, R. (1996). Promoting an electronic community with the use of communication technology in a graduate elementary science methods class. Journal of College Science Teaching, 8, 3963. McGinnis, J.R., Kramer, S., Shama, G., Graeber, A.O., Parker, C.A., & Watanabe, T. (2002). Undergraduates attitudes and beliefs about subject matter and pedagogy measured periodically in a reform-based mathematics and science teacher preparation program. Journal of Research in Science Teaching, 39, 713737. McGinnis, J.R., & Marbach-Ad, G. (in press). What beliefs and intended actions do reformprepared mathematics and science teachers convey to the workplace? The Journal of Mathematics and Science: Collaborative Exploration. McGinnis, J.R., Parker, A., & Graeber, A. (2004). A cultural perspective of the induction of ve reform-minded new specialist teachers of mathematics and science. Journal of Research in Science Teaching, 41, 720747. McGinnis, J.R., Roth McDufe, A., & Graeber, A. (2006). Perceptions of making connections between science and mathematics in a science methods course. Journal of Elementary Science Education, 18, 1329. Melear, C., Goodlaxson, J., Warne, T., & Hickok, L. (2000). Teaching preservice science teachers how to do science: Responses to the research experiences. Journal of Science Teacher Education, 11, 7090. National Commission on Teaching and Americas Future. (1996). What matters most: Teaching for Americas future. Reston, VA: Author. National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author. National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston, VA: Author. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author. National Research Council. (1996). National science education standards. Washington, DC: Author.
Journal of Research in Science Teaching. DOI 10.1002/tea

IMPACT OF THE COLLABORATIVES FOR EXCELLENCE

1369

National Research Council. (2001). Educating teachers of science, mathematics, and technology: New practices for the new millenium. Committee on Science and Mathematics Teacher Preparation. Washington, DC: Author. Negishi, M., Elder, A.D., Hamil, J.B., & Mzoughi, T. (2003). Predicting elementary education candidates technology integration during their eld placement instruction. Paper presented at the annual meeting of the Mid-South Educational Research Association, Biloxi, MI. Pederson, J., & Yerrick, R. (2000). Technology in science teacher education: Survey of current uses and desired knowledge among science educators. Journal of Science Teacher Education, 11, 131153. Saunders, D.K., Keith, R.L., Yank, E.G., & Gustafson, P.E. (2003). Redesigning research: The value of interdisciplinary research in undergraduate education. Journal of College Science Teaching, 32, 377381. Sawada, D., Piburn, M.D., Judson, E., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School Science and Mathematics, 102, 245253. Smith, L., & Gess Newsome, J. (2004). Elementary science methods courses and the national science education standards: Are we adequately preparing teachers? Journal of Science Teacher Education, 15, 91110. Thomas, J.A., & Pederson, J.E. (2003). Reforming elementary science teacher preparation: What about extant teaching beliefs? School Science and Mathematics, 103, 319330. U.S. Department of Education. (2003). Highlights from the trends in international mathematics and science study (TIMSS). Washington, DC: GPO. Witham, S., Krockover, G., Ridgway, K., & Zinsmeister, W. (2003). Lessons online: Educational technology for the undergraduate geology classroom. Journal of College Science Teaching, 32, 264269.

Journal of Research in Science Teaching. DOI 10.1002/tea