Description: Tags: Transcript-07262007

1 1
1 UNITED STATES OF AMERICA
2
3 + + + + +
4
5 DEPARTMENT OF EDUCATION
6
7 + + + + +
8
9 OFFICE OF THE DEPUTY SECRETARY
10
11 + + + + +
12
13 PUBLIC MEETING
14
15 + + + + +
16
17 A FRAMEWORK FOR DEVELOPING HIGHQUALITY ENGLISH
18 LANGUAGE PROFICIENCY STANDARDS AND ASSESSMENTS
19
20 + + + + +
21
22 THURSDAY, JULY 26, 2007
23
24 + + + + +
25
26
27 The meeting was convened at 1:00 p.m. in
28 the South American Room of the Capital Hilton,
29 1001 16th Street, N.W., Washington, D.C. Hanna
30 Skandera, Senior Policy Advisor, presiding.
31
32 PRESENT:
33
34 ROBERT LINQUANTI, Moderator of the Expert Panel
35 DAVID ABRAMS, Expert Panelist
36 SUPREET ANAND, Expert Panelist
37 DIANA AUGUST, Expert Panelist
38 ALISON BAILEY, Expert Panelist
39 TIM BOALS, Expert Panelist
40 MARGARITA CALDERON, Expert Panelist
41 RICHARD DURAN, Expert Panelist
42 REBECCA KOPRIVA, Expert Panelist
43 SARI LUOMA, Expert Panelist
44 CHARLENE RIVERA, Expert Panelist
45 HANNA SKANDERA, Department of Education
46 KATHRYN DOHERTY, Department of Education
47 KATHLEEN LEOS, Department of Education
48
2 NEAL R. GROSS
3 COURT REPORTERS AND TRANSCRIBERS
4 1323 RHODE ISLAND AVE., N.W.
5 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com
6 2
1 TABLE OF CONTENTS
2
3 PAGE
4
5 I. Opening Remarks and Introductions 3
6
7 II. Expert Roundtable Discussion 12
8
9 III. Public Comment Period 90
7 NEAL R. GROSS
11 3
1 PROCEEDINGS
2 (1:04 p.m.)
3 MS. SKANDERA: I just want to welcome you.
4 This is our third and final public meeting in regards
5 to building a framework for highquality standards and
6 assessments for ELP.
7 We're thrilled you're here. Our goal was
8 to piggyback on another meeting that we felt like
9 folks would already be attending to make sure that we
10 got a lot of input, and also didn't inconvenience you
11 in your travels, etcetera, knowing that you probably
12 are all going back to big jobs back at home, wherever
13 that may be.
14 Just a quick introduction of myself. My
15 name is Hanna Skandera, and I've just joined the U.S.
16 Department of Education in the last four months as a
17 Senior Policy Advisor. But prior to that, I was the
18 Deputy Commissioner in Florida, and the Undersecretary
19 for Education in California prior to that.
20 And so for those of you from states, I
21 just want to say I know this is hopefully a long
22 awaited document and framework that will be produced
23 in short order and serve you well. That's certainly
24 our goal.
25 I'd like to go over a few kind of
12 NEAL R. GROSS
16 4
1 logistics and introductions, and then we'll kick off.
2 I apologize for having my back to you all for a minute
3 here.
4 First of all, from the U.S. Department of
5 Education, we have Kathy Doherty, Wendy Tada, and Tyler
6 Hester, and also Kathleen Leos as the Assistant Deputy
7 Secretary for Title III. We also have, from the
8 Office of Intergovernmental and Interagency Affairs,
9 Gloria Frederick and Janet Mills, who have been
10 outside. You've probably met them registering,
11 etcetera.
12 We also have interpreters here, Stacy
13 Tingle, and I believe Amanda is interpreting right
14 now. I'll just pause for a minute. Does anyone need
15 interpretive services today? So we certainly have
16 them available, but, if not, maybe we'll have them
17 available still but not necessary for the entire time.
18 So thank you. And if that changes, let us know and
19 we'll go from there.
20 We also have our Court Reporter, Eric,
21 with us. So thank you.
22 Once again, I just wanted to say that the
23 purpose of today it's our third public meeting
24 is to receive input from you, whether it's verbally
25 which we'll have the opportunity to do in about an
17 NEAL R. GROSS
21 5
1 hour and a half, or written, to contribute to the
2 development of a framework for highquality ELP
3 standards and assessments.
4 Our hope and our goal is at the
5 partnership meeting that we anticipate this fall to
6 provide you with a draft of that, and then have an
7 opportunity to partner with you and states to really
8 refine and fine tune that document so that it really
9 does serve you well.
10 Looking ahead as far as just details
11 regarding this afternoon actually, prior to details,
12 I just want to mention that the LEP partnership, as
13 you know, was formed last year to really begin to look
14 at, what are those things that we at the Department
15 can do to serve you and provide you with guidance and
16 technical assistance in the areas that are most needed
17 at this time.
18 And as a result, we have several partners
19 in the LEP partnership. One is Ellen Forte
20 representing CCSSO. So she is here today. Peter
21 Zamora representing MALDEF. I don't believe he's
22 here, but MALDEF is a partner in the LEP partnership.
23 Melissa Lazaine with LaRaza, Jack Levy with the
24 National Center on English Language Acquisition, and
25 then we've got Robert Linquanti here today with us
22 NEAL R. GROSS
26 6
1 from the Comprehensive Center on Assessments and
2 Accountability.
3 Four kind of procedural moving forward
4 today, I just want to review the agenda. I think I
5 mentioned we will have 90 minutes, and we have an
6 expert panel, quite a large expert panel here today.
7 I just want to say thank you for being with us, and we
8 really anticipate your thoughts and feedback on the
9 four primary questions that were published in the
10 Federal Register, and also over here on this board
11 that you can review. And I think Robert will also, as
12 our facilitator, kind of kick off with which question
13 we're responding to.
14 After the 90 minutes, we'll take a short
15 break. I'll come back up and just talk about the
16 process for public comments. Just so you have it in
17 your back pocket, if you have not signed up already
18 and would like to make a public comment, you'll need
19 to go out here at the break and sign up, you'll
20 receive a number. And I'll review this process, but
21 if that is something you came today to do make sure
22 that on the break you sign up and are prepared.
23 So after the 90 minutes, we'll take a
24 break and then we'll come back and we'll have a full
25 two hours of opportunity for public comments.
27 NEAL R. GROSS
31 7
1 With that, I will turn it over to Robert
2 and our expert panel, and we'll get going.
3 MR. LINQUANTI: Thank you very much,
4 Hanna. Can everyone hear me?
5 ALL: Yes.
6 MR. LINQUANTI: Okay. Great. As Hanna
7 just mentioned, this is the last of three scheduled
8 hearings, and for the purposes of consistency and
9 continuity with those other hearings I'll be reading
10 some of my opening remarks.
11 I'm Robert Linquanti, Project Director and
12 Senior Researcher at WestED representing the
13 Assessment and Accountability Content Center, which is
14 one of the federally funded centers. The AACC has
15 worked very closely with the Department of Education
16 and many states in supporting the LEP partnership
17 initiative. So on behalf of the AACC and its
18 director, Stanley Rabinowitz, I'm pleased to continue
19 the support during these important hearings.
20 We recognize that building the LEP
21 framework there are many complex and technical issues
22 that need to be addressed. So in addition to the
23 input that we're going to be receiving from the public
24 today, we have convened a group of experts to share
25 their knowledge and experience as part of a roundtable
32 NEAL R. GROSS
36 8
1 discussion, and that's what will take place first.
2 The group of experts we have convened has
3 experience in the research and implementation of
4 English language learner standards and assessments as
5 well as instructional and accountability issues. They
6 work across state agencies, universities, and research
7 and development agencies, as well as private testing
8 companies, and they have generously volunteered their
9 time to inform these hearings.
10 So for each question listed in the Federal
11 Register there are four of them, as Hanna
12 indicated, and they are over here on my left four
13 questions. The AACC staff, in consultation with the
14 Department of Ed, have prepared to facilitate a
15 discussion that will help frame what successful
16 implementation might look like at the state level and
17 focus discussion on key technical issues and
18 strategies for addressing these issues.
19 So over the next 90 minutes I will be
20 facilitating the discussion with this expert panel.
21 And as Hanna indicated, this is actually the largest
22 of the three panels that we've held. It's about 60
23 percent larger than the last one, so that will raise
24 some logistical issues, which is of course my
25 challenge.
37 NEAL R. GROSS
41 9
1 The panel will address each of the four
2 questions that were listed in the Federal Register.
3 They received these questions in advance with a number
4 of subquestions to help stimulate their thinking.
5 Now, in order to ensure that each expert,
6 or at least most of the experts, have an opportunity
7 to speak on most of the questions, I will begin asking
8 each expert to in turn respond to a question. We'll
9 begin more formally, and then as we get the process
10 underway we can break the order in which you all
11 respond. However, I will ask that all requests to
12 speak be addressed to me, okay, just to keep it
13 structured.
14 Before we begin addressing the questions,
15 though, I'd like for the panel to briefly introduce
16 themselves, so your name, organizational affiliation,
17 your area or areas of expertise for example,
18 measurement, psychometrics, linguistics, policy, and
19 so on. So I'll begin.
20 My name is Robert Linquanti. I'm a
21 Project Director and Senior Researcher at WestED, and
22 I specialize in the areas of assessment, evaluation,
23 and accountability regarding English learner students.
24 MS. ANAND: I'm Supreet Anand.
25 MR. LINQUANTI: And please use the
42 NEAL R. GROSS
46 10
1 microphone. I'm sorry.
2 MS. ANAND: Okay. My name is Supreet
3 Anand. I'm with the Maryland State Department of
4 Education, and I guess my expertise is in implementing
5 all of these wonderful policies and practices that we
6 have to put in place.
7 MS. KOPRIVA: Rebecca Kopriva, University
8 of Wisconsin researcher, measurement.
9 MS. LUOMA: My name is Sari Luoma. I work
10 for Ballard and Tye Publishers, and I'm my
11 expertise is in language test development.
12 MR. BOALS: Tim Boals, University of
13 Wisconsin and the WIDA Consortium, and my expertise is
14 in policy and best practices for English language
15 learners.
16 MS. AUGUST: Diana August, Senior Research
17 Scientist, the Center for Applied Linguistics. My
18 expertise is in the development of literacy and second
19 language learners.
20 MS. RIVERA: Charlene Rivera, and I'm the
21 Director of the Center for Equity and Excellence in
22 Education at George Washington University. And my
23 expertise is in the area of policy, state assessments,
24 and accommodations for English language learners.
25 MS. BAILEY: I'm Alison Bailey, University
47 NEAL R. GROSS
51 11
1 of CaliforniaLos Angeles, and faculty associate
2 researcher at the Center for the Study of Evaluation,
3 and my area is in language acquisition.
4 MR. DURAN: Hi. I'm Richard Duran from
5 the Gevirtz Graduate School of Education, U.C.Santa
6 Barbara. And my expertise is in the area of
7 educational policy, instruction of English language
8 learners, and assessment.
9 MR. ABRAMS: Good afternoon. I'm David
10 Abrams. I'm the Assistant Commissioner for Standards
11 Assessment and Reporting for the New York State
12 Education Department. I oversee the design,
13 implementation, certification, and validation of its
14 entire testing on accountability and attendant
15 reporting processes. I would argue that my level of
16 my background expertise is literary theory.
17 (Laughter.)
18 MS. CALDERON: Margarita Calderon, Johns
19 Hopkins University, College of Education. And my area
20 of expertise is adolescent literacy, ELL, dual
21 language.
22 MR. LINQUANTI: Okay. Thank you. I think
23 we are going to need to add an extra five minutes just
24 for the maneuvering of microphones.
25 (Laughter.)
52 NEAL R. GROSS
56 12
1 So we'll calculate that accordingly.
2 Let's begin. I'll ask the first question,
3 and then pick the first victim, I suppose.
4 (Laughter.)
5 The first question and, again, it's
6 written up here, and I want to alert the panelists
7 that the question is slightly different than what you
8 may have received. Some folks already caught this
9 minor error. So I'll read it slowly; it's just one
10 phrase that was missing from your mailed handout.
11 First question: "What are the critical
12 elements that states should examine to ensure that
13 their ELP standards" and here is the phrase that
14 was missing "promote effective intervention"
15 sorry, "promote effective instruction to raise the
16 level of English proficiency in LEP students?"
17 Okay. And I'm going to ask Alison to
18 begin.
19 MS. BAILEY: All right. Do I need to
20 press anything on here?
21 MR. LINQUANTI: It's on.
22 MS. BAILEY: It's on, okay.
23 MR. LINQUANTI: Move it a little closer to
24 you.
25 MS. BAILEY: Okay.
57 NEAL R. GROSS
61 13
1 MR. LINQUANTI: I would also ask that when
2 you're not speaking make sure that the mic your mic
3 is turned off.
4 MS. BAILEY: Okay. Well, I understand why
5 Robert has asked me to begin, because before we can go
6 anywhere with this assessment we have to define what
7 English language development actually is as a content
8 area. It's not English language arts, although
9 sometimes the lines get blurred. And it's at the
10 same time, it's intimately mixed with the content that
11 students are learning in the content areas the
12 language of science, math, social studies, and so on.
13 So it's a very difficult endeavor to even
14 begin defining the construct of English language
15 development. Rather than go over all of the research
16 and there has been a lot of research over the
17 years, quite a lot of research done prior to No Child
18 Left Behind, and then a lot of research done since No
19 Child Left Behind. But rather than go over the
20 specifics of that research, I thought it might be
21 useful to try and synthesize what the implications of
22 a lot of the research has what implications it has.
23 And, if necessary, I can go back and talk
24 about the specifics, things like language functions,
25 the kinds of academic language that we think students
62 NEAL R. GROSS
66 14
1 need, whether vocabulary can be conceived of at the
2 general level, or the specifics of the different
3 content areas. We can revisit that if that's the
4 level of detail that people want.
5 But let me just give you some features
6 that have occurred to me that over the years of
7 working in this area I have been able to glean from
8 that research literature. And so I just want to talk
9 about these distinguishing features of the ELD
10 construct. And I've pulled apart about seven of these
11 distinguishing features.
12 The first thing to note is that the ELD
13 construct involves social language, so the language
14 that kids encounter outside the school context, as
15 well as the language that they're going to need to
16 access and engage with the curriculum.
17 And that language that they need for
18 accessing the curriculum can be broken down into at
19 least two areas something that we could call school
20 navigational language, the language that students will
21 need in school to make sense of the school
22 environment, generally speaking, and another area of
23 academic language which is content to access the
24 content of the curricula.
25 In this regard, then, these three areas
67 NEAL R. GROSS
71 15
1 social language, school navigational language, and
2 curriculum content language they differ by purpose.
3 Those should be obvious. I can go back and give the
4 different purposes, if necessary, but, first, the
5 distinguishing feature would be the purpose.
6 Second distinguishing feature is the
7 degree of formality, so that social language is often
8 informal. The school environment requires a more
9 formal use of language.
10 Context of use obviously, social
11 language, home, peer group, out of school activities.
12 The school navigational language actually is going to
13 be a mix of both the social and the more formal in
14 terms of that context actually will pull for language
15 that will require a different degree of formality.
16 And then, the context, the instructional time that
17 students have, obviously will be much more focused on
18 content learning.
19 Next feature that I was able to
20 distinguish in the literature is the context of
21 acquisition, and by that I'm talking about the degree
22 of explicit instruction that students might need to
23 acquire English language. And obviously in the social
24 language context that instruction is very implicit.
25 And in the school navigational context
72 NEAL R. GROSS
76 16
1 that language would be more explicit, although I think
2 in most cases teachers take it for granted that kids
3 will acquire language for how to negotiate the
4 classroom, the management classroom management
5 language very implicitly. We might not actually pay
6 attention to how much explicit instruction students
7 might need for just that social and instructional
8 navigational language.
9 And then, of course, the curriculum
10 content language would be the most explicit context of
11 acquisition.
12 Then, there will be differences in
13 modality across those different types of language of
14 ELD construct. And so modality being listening,
15 speaking, reading, and writing. The social language
16 context would be much more of an oral language
17 context, and school navigational would be
18 predominantly oral. And then, by the time we get to
19 the curricula content language, we'll be using
20 listening, speaking, reading, and writing.
21 And then, two further distinguishing
22 features teacher expectations. Those ELD construct
23 differences will I think lead teachers to different
24 expectations for the amount of English language
25 development that students acquire.
77 NEAL R. GROSS
81 17
1 And so, for instance, in the social
2 language domain, teachers will be expecting students
3 to come to school already proficient, perhaps with
4 proficient in English in terms of the social language
5 domain, with perhaps the exception of their most
6 recent ELL student arrivals.
7 And so in the school navigational language
8 domain, teacher expectations will be more along the
9 lines of students coming to school readily able to
10 learn the kinds of language that go on for classroom
11 management, and so on.
12 And, finally, in the curricula content
13 language domain, teacher expectations may be that all
14 students, not just English learners, but all students
15 will be acquiring that kind of specific language in
16 the context of school, predominantly in school.
17 And, finally, grade level expectations,
18 and I think we have different we should have clear
19 differences across the grades in terms of what we
20 expect of English language development and those three
21 different domains social, school navigational,
22 language, and curriculum content language will fall
23 out differently across those across the school
24 grades.
25 So I think basically, just to summarize,
82 NEAL R. GROSS
86 18
1 this makes assessing English language development very
2 difficult. The implications of these dimensions,
3 these distinguishing features I've just gone over,
4 make the English language development construct very
5 dynamic and very interactive. Each student will have
6 a separate exposure history, a separate acquisitional
7 history, and it makes testing the construct and
8 operationalizing it sufficiently for largescale
9 assessment very difficult as a result of that dynamic
10 interactive element.
11 And I think as a result, then, assessment
12 one way we have to work with ELD and the ideas
13 around assessment would be to start at the classroom
14 level. So how is English language development
15 monitored and used that information used by
16 teachers in a formative assessment way for their
17 instruction? We have to think of it, then, at the
18 next level, at benchmarks for periodic sort of sensing
19 where kids are in their ELD growth, and then at the
20 largescale level for states to be
21 measuring/monitoring yearly growth in ELD.
22 And I think for it to work there has got
23 to be continuity, then, across those three types of
24 assessments from formative, benchmark, through to
25 summative. And if there isn't consistency in what the
87 NEAL R. GROSS
91 19
1 construct is at a systemic level, it won't be a very
2 meaningful process for us to be assessing English
3 language development.
4 And I think one way we can do this is
5 through states working with their teachers to really
6 put together what we people are calling learning
7 progressions. We don't have very good learning
8 progressions of ELD in terms of how it is developing,
9 and how it is impacted by these different
10 distinguishing features that I just mentioned.
11 So that's where I would say we need to go
12 is to actually put together learning progressions for
13 English language development by grade and taking into
14 account all of these contextual and childlevel
15 variables.
16 MR. LINQUANTI: Okay. Thank you. I think
17 we've just seen how interrelated all these questions
18 are, and so as further panelists go, I know it's hard
19 but try to stay with the particular question and we'll
20 circle back around. There's a lot to talk about.
21 Charlene?
22 MS. RIVERA: Okay. The particular
23 question is the you know, to identify the critical
24 elements that states should examine to ensure that ELP
25 standards promote effective instruction. I guess in
92 NEAL R. GROSS
96 20
1 that first phrase I am seeing an assumption there that
2 standards are including what Alison is talking about
3 to some extent, and that's that it is that the
4 standards have addressed the issue of academic
5 language.
6 And I'm not sure that all standards have,
7 but I think it's a very important point and states
8 need to look at their standards currently and to make
9 sure that they are addressing academic language. And
10 by that I mean that states should be looking at what
11 is required in classrooms for in different content
12 areas, and there should be ways of examining them.
13 And I know Alison's work, and Fran
14 Butler's work, etcetera, has led us and helped us to
15 understand what some of the academic language is. But
16 it really is imperative that I think that we take
17 another look at what is required of students, what
18 kind of language functions, what kind of structures,
19 etcetera, are required of students to be able to
20 participate and to be able to actually actively
21 participate in a content area classroom, not okay,
22 so that's an assumption, and I think that we need to
23 just put it out there and to say that if we're looking
24 at what states need to do, they need to examine their
25 standards and make sure that they include that
97 NEAL R. GROSS
101 21
1 reference to academic language, and that it's in a
2 meaningful and explicit way.
3 Then, I would say that the next thing that
4 would need to be done is to states need to develop
5 curriculum or to at least look at what they've got in
6 terms of curriculum. Is it is this academic
7 language included in the curriculum, and in what ways
8 is it explicit?
9 That leads to teacher training, to
10 actually having explicitly making sure that ESL
11 teachers and content teachers understand what the
12 standards are, both the ELD standards and the content
13 standards, and then have that relationship and both
14 sets of teachers need to be trained, need to have some
15 and they need to build a communication link. So I
16 would say that that would be a main kind of building
17 block that needs to be put in place.
18 MS. AUGUST: I think that Alison has done
19 a really good job of describing the features of
20 English language proficiency, but I actually have a
21 question I'd like to address to the panel. And that
22 is: what actually is the difference between English
23 language proficiency and English language arts?
24 Because this is critical, especially in the areas of
25 reading and writing. And I'm talking about English
102 NEAL R. GROSS
106 22
1 language arts broadly construed to include the kind of
2 English that you need to be able to function in those
3 content area assessments.
4 I mean, to me, this is a really critical
5 question, and I don't think anyone that I know of has
6 done a really good job of differentiating this. It
7 has a lot of implications for assessment, in that if
8 we're talking about a continuous variable you would
9 develop one kind of an English language proficiency
10 assessment, and if we're talking about very different
11 kinds of animals here we would measure it very
12 differently.
13 So there are a lot of experts on the
14 panel, and I'd be really interested in, you know,
15 their thoughts on this. We develop I mean, a lot
16 of people believe, as I do, that a lot of English
17 language proficiency is developed through exposure to
18 a second language through interactions with native
19 speakers of that language. And so if that's the case,
20 it's different than what we think about in English
21 language arts or in content knowledge. Those are
22 things that are actually instructed.
23 So I think we're talking about different
24 things here, I mean, from my point of view. So I
25 would just be really interested in what other people
107 NEAL R. GROSS
111 23
1 here, their thoughts I mean, there are a lot of
2 experts here, their thoughts on this.
3 MR. BOALS: Okay. Well, the point I think
4 I'd like to build on, because I think Charlene and
5 Alison and Diana have done a good job of kind of
6 laying out what we know and maybe even what we don't
7 know about some of this, but looking at it now from
8 the classroom teacher's point of view, how are we
9 going to help classroom teachers understand what this
10 thing called academic language is.
11 In particular, as we've moved to the
12 concept of the language of mathematics, the language
13 of science, very, very different than we were talking
14 10 years ago when we used terms like BICS and CALPS,
15 or BICS and CALP.
16 So I think we need as we build these
17 English language proficiency standards, we need to be
18 mindful of the way in which we're building them, that
19 these documents are very much living documents that
20 speak to teachers and provide the kind of framework
21 that we need for professional development with
22 teachers so that these standards don't just collect
23 dust on the shelves. They get out there and we use
24 them in a very dynamic way to change practice, because
25 I think that is the end game here that we need to keep
112 NEAL R. GROSS
116 24
1 in mind.
2 MS. CALDERON: I'd like to build on this
3 as well, because I think it's critically important,
4 especially for secondary students. My concern is with
5 SIFE students, students with interrupted formal
6 education, and how the standards, the benchmarks, the
7 classroom assessments, the teacher training has to
8 back up and pick up the students who are reading at
9 first grade level, not only in English but also in
10 their primary language, the students that come with
11 interrupted formal education, but also students that
12 have been in elementary school since kindergarten and
13 are lifelong ELL.
14 Something happened there, so how can we
15 make sure that the gaps in their learning, in their
16 literacy development, in their vocabulary are met?
17 Particularly since the students have so little time
18 before graduation, how can we bring them up to par?
19 So these are some of the things that I think we need
20 to integrate, you know, at the classroom level. That
21 is definitely where it starts.
22 And if teachers don't understand this
23 phenomena and the differences in between SIFE and
24 ELS what about the benchmarks? Are there going to
25 be provisions so that the students have a different
117 NEAL R. GROSS
121 25
1 set of benchmarks?
2 And then, of course, at the state level,
3 what accommodations, what kinds of things are going to
4 be offered at that particular point in time?
5 MR. ABRAMS: Thank you. I'd like to pose
6 a question, too, but in a way sort of answer it from
7 my thinking. I'm glad that we had such a beautiful
8 introduction to the concept and that it worked its way
9 to essentially reading and instruction, because once
10 when you're inside of a state testing program,
11 they're in effect reading tests. They all are, except
12 if you put a speaking test somewhere. Outside of the
13 ELP assessments, most state testing programs at scale
14 do not do that.
15 And so when I think about academic
16 language and we begin to ask this question about
17 vocabulary and concepts, I would kind of pose it that
18 it seems to me that it's less of an issue of
19 vocabulary and concepts and more of an issue of modes
20 of discourse or the underlying rhetoric that forms the
21 epistemological foundation for how you generate
22 knowledge in a discipline.
23 And so now kids come in and they're
24 learning language two based on what they have from
25 language one, and are intersecting with relatively
122 NEAL R. GROSS
126 26
1 complex forms of state standards in relationship to
2 the social studies, economics, and government and
3 geography, obviously mathematics and the different
4 forms of sciences.
5 And I think what I see instructionally is
6 teachers are grasping at the notion of teaching a term
7 without really a good underlying understanding of the
8 rhetoric of the discipline, which is kind of the
9 foundation for where we're building the exams and also
10 the foundation I would argue for the type of critical
11 thinking they are going to do inside of a discipline.
12 MR. DURAN: Thank you. I will follow up
13 on that and give it a little bit of a controversial
14 turn. Academic language proficiency cannot be
15 represented solely by competence in discrete language
16 skills and even language functions. We need to think
17 more deeply about what communication and language use
18 is in activity.
19 And to the kinds of issues that Alison
20 raised about the academic progression and development
21 of students, language proficiency is in relation to
22 their academic competence, which comes back to what
23 David was raising about understanding how students are
24 extracting meaning from their lessons, and how that is
25 built into resources and tools for learning and
127 NEAL R. GROSS
131 27
1 advancement.
2 Now, that stated, the reality is that we
3 have state standards for English language development,
4 and we have standards for English language proficiency
5 assessments. A challenge I will oppose will be that
6 it will be impossible for psychometrically viable
7 assessments to cover all the bases.
8 It will be very valuable to think through
9 carefully how the inference is drawn from our
10 assessments, and the extent to which they align with
11 English language proficiency standards can be made
12 useful to the inferences that are being tied to the
13 English language development of students, which then
14 poses some interesting challenges with regard to
15 figuring out what you can do well and what you can't
16 do well and what you might be able to do middling well
17 if you relax some of the psychometric qualities that
18 you'd like for assessment, such as introducing
19 observational protocols which have known reliability
20 problems but which have high utility for teachers.
21 MS. LUOMA: To build on to what the other
22 people were saying before, so far we've seen about
23 half the educational world, concepts in the
24 educational world of the school introduced into this
25 one question that regards
132 NEAL R. GROSS
136 28
1 (Laughter.)
2 what are the critical elements that
3 states should examine to ensure that their ELP
4 standards promote effective instruction? And the
5 thing that I'd like to bring us all to focus back on
6 is: which parts of this whole world are the ELP
7 standards responsible for? And which other parts of
8 the school environment and the frameworks that guide
9 work in school environment could take over, so that we
10 could really focus the ELP standards on the ELP
11 construct?
12 And to that effect, I'd like to emphasize
13 what several people have said, but I'd like to bring
14 it to a closer emphasis that the ELP standards, it
15 is our responsibility to focus the ELP standards on
16 English language proficiency. And we need to remember
17 the language in that.
18 And I don't mean let's just teach them
19 grammar and vocabulary, but I do mean let's not forget
20 the grammar and vocabulary. And I don't mean let's
21 teach it in isolation. I do mean let's teach it in
22 and define it in the standards in the context in
23 which the language is used at school. But let's not
24 let the standards go to the level of functions and
25 activities only and forget about the language.
137 NEAL R. GROSS
141 29
1 And so the difference, as far as I can
2 see, between ELP standards and ELA standards is that
3 ELA standards define the skills, the discourse skills,
4 that all students at school are learning. And the ELP
5 standards the highest level of proficiency of ELP
6 standards helps the students join that developmental
7 continuum at the end of the proficiency development.
8 Now, the focus of the ELP standard should
9 be on the continuum of the ELP development and not
10 only at that top level.
11 And then, what else within the school
12 environment should take care of all of this all of
13 these concepts that have been introduced?
14 Well, there should be some kind of
15 guidelines for goals in school and some entity taking
16 responsibility for teaching academic English as a
17 register to all students, not just the ELP students
18 but also the students whose literacy is below their
19 grade level, whose schooling is not who have
20 interrupted schooling background.
21 But this is not a problem for ELP students
22 only. That's something that maybe it would be helpful
23 to have other standards to guide, so that we could
24 focus the ELP standards on the needs of the ELP
25 students and on the concrete ways in which teachers
142 NEAL R. GROSS
146 30
1 can help students achieve.
2 MS. KOPRIVA: Well, hmm.
3 (Laughter.)
4 Ditto.
5 (Laughter.)
6 And I guess the one other thing is I am
7 not an ELP person. I am a measurement person. But
8 I'll tell you, I know when I'm not measuring
9 something, so and there is a heck of a lot of that
10 going on. So I guess what I'm mainly concerned about,
11 I'm very concerned about disentangling the language of
12 content and the content of language.
13 And that certainly is between English
14 language proficiency and English language arts. It is
15 also between English language proficiency in science,
16 English language proficiency in mathematics, and so
17 on. When are we teaching when are we testing
18 mathematics, and when are we testing linguistic skills
19 associated with?
20 To me, that's not a simple question.
21 That's part of this whole disentangling business that
22 I think, unfortunately, we are trying to research while
23 we're flying the plane. But I think it's imperative
24 that we continue to think about that.
25 Actually, maybe I think it's, first,
147 NEAL R. GROSS
151 31
1 imperative that we understand that we don't know that,
2 and that we are struggling to figure that out. And so
3 I'd say that for anybody who is doing testing, if we
4 think we have a lock on it, that's a problem. I think
5 we that's one of the reasons why it's this line
6 in the middle that we're messing with and that we're
7 trying to understand and we're trying to articulate,
8 which I think is what I'm hearing here as well.
9 Let's see. I guess my other point is, to
10 be able to distinguish, from a measurement
11 perspective, when we are measuring language and when
12 we are measuring content, I think it needs to be done
13 at an itembyitem level. It can't be done at a test
14 level. It has to be done at an itembyitem level,
15 very, very precisely, disentangling content, the
16 targeted content of that item, which goes way deeper
17 than a general standard.
18 The linguistic complexity and the
19 cognitive complexity if we are focusing on
20 measuring content, we should be minimizing linguistic
21 complexity. If we are focusing on measuring language,
22 we should be minimizing the content we are the
23 academic content we are using to get at the
24 measurement of the linguistic complexity. So I think
25 that we need to disentangle those at the item level.
152 NEAL R. GROSS
156 32
1 I'd say, if I was going to say one other
2 thing as far as measurement, that I think the other
3 place that I'd say states need to kind of focus on is
4 empirically empirically that we can do all the best
5 work we can at the item level in development. I
6 actually think we need to we need to see if we
7 actually did it. I mean, it isn't enough just to say
8 we think we did it, but did we?
9 And the purpose there is to what that
10 means is not only looking at general validity indices
11 for the total population, but looking at the validity
12 indices for subpopulations, to ensure that we are in
13 fact measuring the same thing. Okay? And that has
14 lots of implications.
15 I guess the last thing I'd like to say is
16 that we did do an extensive study not so long ago in
17 PG County when Supreet was there, and we did all the
18 best stuff. I mean, we gave everything to those kids,
19 did wonderful things, and we looked at the in the
20 test itself, gave them the right accommodations, did
21 stateoftheart plain languaging. We did the whole
22 ball of wax.
23 And then, we looked and we looked at
24 the validity of the scores using a variety of things.
25 Essentially, I can tell you that the state of the art
157 NEAL R. GROSS
161 33
1 is not good enough, especially for low literacy kids.
2 So we need to keep moving. We need to know that we
3 are continuing to struggle with this, and I think
4 that's important.
5 MR. LINQUANTI: Rebecca, just to clarify,
6 your last reference was to an academic content
7 assessment?
8 MS. KOPRIVA: It was an academic content
9 assessment, yes, sorry.
10 MR. LINQUANTI: Supreet?
11 MS. ANAND: I guess at this point all I
12 can say the experts have spoken.
13 (Laughter.)
14 But what I wanted to share is how we did
15 look at the principles of language acquisition when we
16 were designing our state standards. And in going back
17 to what Tim mentioned, that operationalizing these
18 standards is something that we do have to keep in mind
19 at the very onset of development, because if these
20 standards are not filtering down to the teacher and
21 student level, then there is just a nice little thing
22 that we've done to meet federal requirements.
23 And so at the very onset we made sure that
24 this was a collaborative effort of educators who were
25 actually teaching and working with our ESL students.
162 NEAL R. GROSS
166 34
1 We had large input from them, and that's why we have a
2 lot of buyin from the local district, so that they
3 are happy to operationalize them.
4 Our basic aim and what we sought to do
5 was identify and describe the language skills that
6 reflected the stages of language acquisition, and we
7 felt that were necessary for our ELL students to
8 participate effectively in the content classes. We
9 did look at all of our state content we call it our
10 voluntary state curriculum.
11 It's really not voluntary, but it's our
12 state curriculum, on each of the content areas. And
13 we looked at the language implications in those
14 content areas and tried to backmat those into our
15 content standards.
16 And I think Diana brought up a very good
17 point about reading language arts. That's something
18 we struggled with, and we kept asking ourselves at
19 each objective in our standards: is this reading
20 language arts? Is it language? Is it and so that
21 was something we did struggle with. I'm not sure we
22 had the answers, but we did do what Sari referred to
23 is look at that as the endpoint.
24 Okay. So if the student needs to is
25 going to be taught the skill prediction in reading
167 NEAL R. GROSS
171 35
1 language arts, predicting what is going to happen next
2 in the story, what language features is he going to
3 need? And so we backmat that. Oh, well, he is going
4 to need future tense, he is going to need transitional
5 words, and that's what we kind of looked at in our
6 standard is making sure that the child can get to that
7 negotiating that kind of language in reading
8 language arts.
9 So I'm not sure we have the answers, but
10 we try to find some kind of a solution to match the
11 two.
12 And the other thing I did want to also
13 point out is, in addition to the academic language in
14 BICS and CALP and all of that, the other standard or
15 item that we added to our standards was cultural
16 integration. And we're not talking about culture in
17 the surface sense, but we're talking about the
18 academic culture.
19 Many of our ELL students and being an
20 ELL person myself, I felt that was one thing that I
21 when I came into the academic culture here, I was in
22 a culture shock, so that was the other piece that we
23 don't want to forget in all of this is that our
24 children do need that acculturation into this.
25 MR. LINQUANTI: For those who may have a
172 NEAL R. GROSS
176 36
1 strong comment I think Diana felt a need to
2 comment.
3 MS. AUGUST: Yes, I just want to raise
4 this again, because I think until we figure this out
5 we are going to have a hard time developing decent
6 English language proficiency standards. And the issue
7 is: are we talking about qualitatively different
8 constructs? Are we talking about sort of different or
9 lower levels of the same construct? That's the first
10 thing I raise.
11 And then, the second I agree with my
12 colleagues that in developing standards it's important
13 to develop them in a way that will guide instruction.
14 But if we think about what we know about second
15 language acquisition, a lot of what children acquire
16 is through sort of a natural acquisition process.
17 It's not instructed, and it's through meaningful
18 interaction with fluent speakers.
19 So if that's the case, what are we talking
20 about besides sort of access to age and grade level
21 appropriate content instruction? So I just raise
22 those questions. I think they're important questions.
23 MR. LINQUANTI: Unless any of the other
24 panelists feel the need yes, Tim. Tim, and then
25 Charlene.
177 NEAL R. GROSS
181 37
1 MR. BOALS: Well, just following up on
2 what Diana is saying, I think it might not be the
3 worst thing that we're flying this plane while we're
4 figuring it out, because what's going on across the
5 country is that different states and different
6 consortia are taking different approaches to looking
7 at the building of English proficiency standards.
8 And I think that's positive, because with
9 some data that is going to come out of that, as time
10 goes on we are going to begin to see a clearer picture
11 of these things.
12 MS. RIVERA: I was just going to say that
13 I think that if they're the same construct ELA and
14 ELP, then we really we don't have a lot of work to
15 do. But I'm going to say that that's not the case. I
16 do believe that there is a difference and that for, you
17 know, what I think just our experience has shown, that
18 if we immerse kids in a regular classroom, and we
19 teach them ELA at grade level standards, what grade
20 level, you know, there is a gap.
21 Something happens and the kids don't catch
22 on, so there is something about having ELP standards
23 or ELD standards that specifically speak to helping
24 the students make a transition from you know, from
25 understanding the language, the formalities of the
182 NEAL R. GROSS
186 38
1 language, all of the language structures that are
2 required to be able to process what goes on in the ELA
3 assessment.
4 And that's really our challenge. And I
5 know it's murky, but I think it is our challenge, and
6 I think it is something that we can do and that has
7 been started, and we really need to keep on moving
8 forward with it.
9 MR. LINQUANTI: Sari, the last word, and
10 then we'll move to the next question.
11 MS. LUOMA: And just to build on to I
12 would have said exactly the same thing, that I do
13 believe that there is a difference, and there is
14 evidence that there is a difference. And states have
15 been building these standards to four different
16 purposes.
17 Also, even though some informal
18 acquisition is happening when some proportion of the
19 ELP population interacts with native speakers, others
20 don't. Others don't interact in English except in the
21 classroom. And even though informal acquisition is
22 happening, that doesn't mean that that defines our
23 construct.
24 MR. LINQUANTI: Well, I'd say we're off to
25 a roaring start.
187 NEAL R. GROSS
191 39
1 (Laughter.)
2 And, obviously, these questions are
3 interrelated, so we will be touching on many of the
4 same things for the next question, which is, again, up
5 on the board over here.
6 So question 2, "What are the critical
7 elements that states should examine to ensure that
8 their ELP assessments provide a valid and reliable
9 assessment of English language proficiency?" All
10 right. Supreet, since you had to wait so long last
11 time, start off this time.
12 MS. ANAND: I think that when we talk
13 about validity of a test, we have to talk about it in
14 context of how we are using the test. And my question
15 is, for the English language proficiency test the
16 way that we are implementing it at the state level
17 what are we using it for? Is it only for calculating
18 EMLs? Is it to inform instruction? Where are we
19 going with the test? And those are questions that we
20 have to answer first, and I'm not sure we have done
21 that effectively right now.
22 And I also want to point out that, you
23 know, compared to Title I, Title IIA, Title III get a
24 very small amount of money. And so we have to
25 prioritize in the funds whether we want to use that
192 NEAL R. GROSS
196 40
1 for assessment/testing, or do we want to use it to
2 supplement instruction? And those are tough choices
3 we have to make.
4 And the other part I want to point out is
5 that our ELL students are already tested twice. They
6 are tested on their grade level test for Title I
7 purposes, and then they are tested for Title III ELP
8 tests. That's a lot of time that is spent testing and
9 away from instruction.
10 And when we look at we want to test the
11 depth and the breadth of all of the language
12 acquisition domains, we have to balance that with the
13 time factor. And I'm not sure you know, there are
14 people here who are experts in testing who can
15 probably address that, but that is a concern that we
16 have and when we are implementing the test.
17 And so I am just putting this out there
18 for the experts to think about is, let's think outside
19 the box and see, are there any other ways that we can
20 develop these tests? Can we in Title III test
21 speaking and listening, and for EML purposes
22 extrapolate reading and writing from the grade level
23 tests? Can we look at perhaps looking at formative
24 assessments throughout the year, English language
25 proficiency formative assessments, and at the end of
197 NEAL R. GROSS
201 41
1 year aggregate that data as an EML measure?
2 I mean, I'm just putting this out there,
3 that I am very concerned about the time we are
4 spending testing and not instruction for our students
5 who need more time on task.
6 MR. LINQUANTI: Rebecca?
7 MS. KOPRIVA: Okay. Well, I think the
8 first thing is which I think is happening is that
9 academic language proficiency needed to be added to
10 the traditional English language proficiency test.
11 And so I think the full range of as these wonderful
12 folks are defining academic language proficiency, that
13 that full range has to be sampled in any kind of an
14 assessment.
15 That said, I think I also think that
16 the academic language proficiency that is sampled, I
17 think the questions will occur within content
18 academic content stimuli, but and that's where it
19 gets tricky. The trick is to measure English language
20 proficiency, not math, or not science.
21 And so to me, in terms of the provide a
22 valid and reliable assessment, reliability we know how
23 to do. We just do it. I mean, we know how to do that
24 kind of stuff, and I think the assessment people can
25 do that probably with their toes crossed behind their
202 NEAL R. GROSS
206 42
1 back.
2 I think in terms of validity, I think
3 validity is huge, and I think that I think
4 particularly from what I said in my last comments, I
5 think it is differential validity. It is really
6 focusing on how the students are doing, and everything
7 from and in getting that information, everything
8 from qualitative to thinkalouds, how are they
9 approaching these particular items that have the
10 stimuli of academic content, to disentangle the
11 linguistic side from the science.
12 The other is that I think we need to have
13 larger differential studies to be able to and
14 empirically based, so on quantitative, to be able to
15 differentiate and make sure that we are defining that
16 target on that side of the continuum fairly closely.
17 Okay.
18 MS. LUOMA: All right. I will just
19 address some of the stuff that Supreet raised briefly,
20 in addition to seconding Rebecca's remarks that we
21 actually know how to assess the measurement quality of
22 measurements. But what we need to focus on is what
23 Supreet began with validity for a purpose.
24 If we do use English language proficiency
25 tests to inform the instruction of the ELP students,
207 NEAL R. GROSS
211 43
1 and to then at the top end mainstream them and make
2 sure that they can cope with the linguistic demands of
3 the Englishonly classroom, we need assessments that
4 actually speak to the validity of those decisions.
5 If you know that in your state you do not
6 have differentiated ELP instruction, then the focus of
7 your ELP assessment should be at the mainstreaming
8 level. If you know that you do have differentiated
9 instruction, then you need to decide how many levels
10 of proficiency you need to distinguish within the ELP
11 standards, and agree with all of the ELP folks in the
12 state that this this is the program focus,
13 therefore, this is going to be the assessment focus.
14 And, lastly, I think it might be dangerous
15 to use a very blunt reading/writing instrument, which
16 is if it has been developed for ELA assessments,
17 which we just tried to say were two different
18 constructs, if we try to use that blunt instrument to
19 make fine distinctions.
20 If you are focused on the proficient
21 distinction only, then maybe you can use your ELA
22 assessment using different criteria, assessment
23 criteria from the same performance. But if you are
24 trying to make continuum distinctions, I think that
25 would tend to indicate that you are using an
212 NEAL R. GROSS
216 44
1 assessment invalidly.
2 MR. BOALS: Okay. I think what I would
3 like to emphasize about this new generation of
4 assessments that are coming to us now is how what a
5 great difference they really represent from what we
6 were doing a few years ago when you had teachers
7 pulling down the box kit, and you had 15 different
8 types of box kits being used oftentimes in the same
9 state.
10 So I think what this presents to us now is
11 a tremendous opportunity to create English language
12 proficiency standards that are used across a state,
13 and in some cases across multiple states, assessments
14 that are of much higher quality that are adhering to
15 the industry standards for academic assessments in
16 many ways, which the old generation of assessments
17 didn't have to. That wasn't their purpose.
18 Now, we have raised the stakes, and we
19 really do have to create assessments that are built
20 off of quality English proficiency standards, but we
21 have also got to make sure that they have all of the
22 psychometric properties and the bias and content
23 properties that we need to see in assessments that are
24 being asked to do what these assessments are now being
25 asked to do.
217 NEAL R. GROSS
221 45
1 MR. LINQUANTI: You are allowed to pass.
2 (Laughter.)
3 MS. RIVERA: Okay. So we're answering the
4 question: what are the critical elements that states
5 should examine to ensure their ELP assessments provide
6 a valid and reliable assessment? I think that I
7 mean, I what has been said at the end of the table
8 is very good, but I also think maybe we should just go
9 back and talk about the standards a little bit and
10 make sure that the standards that the items are
11 mapped to the standards.
12 I mean, we are making the assumption we
13 have good standards. It's really important to have
14 good standards and that they explicitly include all of
15 these language functions that are necessary for
16 students to have academic success. So items should be
17 mapped to the language items should be mapped to
18 the standards. Every item should be written with a
19 standard in mind.
20 And, I mean, sometimes tests test things
21 that are off not really necessarily they are
22 sort of linked to the standards, but not fully linked
23 to the standards. So it's really important to make
24 sure that that is done.
25 It is also necessary to look at the
222 NEAL R. GROSS
226 46
1 cognitive complexity and at the depth of knowledge,
2 and write items that reflect the depth of knowledge
3 and complexity of the standards.
4 So I will just leave it at that, but I
5 think we just have to make sure that we are addressing
6 those issues.
7 MS. BAILEY: I had a few comments to make
8 in response to everything that has just been said
9 about standards and assessment. Charlene said
10 something that reminded me, having been involved in an
11 alignment study at the state level, that we can't take
12 it at face value, that even within the different
13 modalities or domains that items that claim to be
14 tapping oral language development aren't also, in
15 fact, requiring students to use reading and writing
16 skills, so that the constructs actually within ELD
17 itself get conflated at the test item level.
18 So they don't map cleanly back to the
19 standards, so I think that was a very important
20 comment that Charlene brought up about the standards
21 themselves or the underpinnings, and then the items
22 written to them sometimes not always being clear as to
23 which modality they are even going after.
24 But in response to Rebecca's comments
25 about needing more validity studies, I just wanted to
227 NEAL R. GROSS
231 47
1 put on the table that part of that validity work
2 should include by states some notion of opportunity to
3 learn.
4 So as we figure out whether or not a
5 student is reaching competency in the English language
6 proficiency, do we know what kind of opportunities
7 they've had for acquiring some of the content
8 knowledge that they are going to be exposed to as they
9 move into mainstream Englishonly classrooms? And,
10 therefore, what role did that play in their
11 assessment?
12 And a part of the opportunity to learn
13 would not just be the content area, but their exposure
14 to academic English. That should I know some
15 states are looking at the opportunity to learn, and
16 that may be missed off those kinds of surveys of your
17 schools and your and what your teachers are doing
18 in the classroom. So exposure to academic English
19 would be important.
20 And just one more comment, to Sari,
21 actually. This notion of it's almost as if we're
22 in danger of creating tests that are too precise. And
23 you pointed out, would you need that level of
24 precision for making programming decisions?
25 And I think and so, you know, that's a
232 NEAL R. GROSS
236 48
1 very important point that maybe we have an
2 instrument that is more powerful than we actually have
3 in terms of we get the information, but then, what do
4 we do with it in terms of putting students into
5 suitable English language programming?
6 But the other would be, along the same
7 lines, the standards that states have themselves often
8 aren't as delineated as much as their state tests. So
9 my understanding, looking across several different
10 states, that standards may be at just three or four
11 proficiency levels, but the state ELD assessment is at
12 a greater number of proficiency levels.
13 So even having these ELD standards or ELP
14 standards aligned to the number of different
15 proficiency levels on the state test would be, you
16 know, a starting point for to making alignment
17 tighter.
18 MR. ABRAMS: Thank you. Yes, that was a
19 great point. And what I was thinking as I heard some
20 of my colleagues speak here was, from my chair in New
21 York, the very first thing that is happening of course
22 with our Title III ELP instrument is that it is
23 designed to declassify students.
24 And so the very first thing I have is my
25 DBRAK issues, opportunity to learn, and do you have
237 NEAL R. GROSS
241 49
1 an assessment built on standards that when you say to
2 a student, "You are no longer part of a special
3 population and entitled to a series of opportunities
4 and rights and issues in relationship to earning a
5 diploma," can you go out and function effectively?
6 And I think that's critical, and I was
7 thinking as the Federal Government is thinking about
8 next steps how I really wanted to say firstly and
9 then I'll get back to my issue on validity how
10 important it is to respect the DBRAK construct here,
11 because of the stakes associated with ELP exams. And
12 I view that when I talk with my Commissioner the same
13 way when we're doing our gatekeepers that we do at the
14 high school levels.
15 Kids have to have that time, the states
16 have to have the time to build the instrument,
17 permeate the standards through. And, obviously, states
18 have standards now, and in the case of my state we are
19 going into standard review and revision on all of our
20 standards. So we'll put these back under the gun.
21 I really think that the validity issue,
22 then, the use of the test and the inferences that we
23 want to draw up, we want so many things from large
24 scale assessment that states give now, because my
25 colleague Tim raised the great point prior in your
242 NEAL R. GROSS
246 50
1 answer in the previous question, and then at the
2 start of this one.
3 We have so many different assessment
4 practices and standardized assessments being
5 administered around a state, even within a known
6 school system, that it was very hard to get
7 comparability and even generalizability about
8 achievement, what does it mean, and where are kids
9 going? And so it has been difficult to build and
10 implement instruments on the fly, especially with some
11 of the accountability constructs, but we are certainly
12 in a much better place.
13 So my feeling is that I have begun to view
14 sort of the design of our validity arguments is that
15 it's an ongoing tapestry of research. And I agree
16 with my colleagues at the end of the table Rebecca
17 and Supreet about I think the measurement community
18 knows how to build and certainly certify the
19 instruments. And I have a long list of research
20 studies we are doing on our types of instruments, how
21 to get better.
22 I'm very interested now with looking at
23 the calibration issues between the ELP and the ELA
24 assessments across the board. I'm exceptionally
25 interested in people who are going to start looking at
247 NEAL R. GROSS
251 51
1 reading research, because I still as much as
2 everyone is talking about academic language, I'm still
3 hung up on reading and reading instruction. For me,
4 that's still what academic language boils down to and
5 what the kids have to be able to do. And so I think
6 that there's a lot of work there for us to do.
7 I'm going to have one slight friendly
8 disagreement with my colleague from Maryland on the
9 loss of instructional time, only because while I
10 believe that we are certainly testing and certainly
11 testing more than what people thought, though not
12 always as much because a lot of, you know, instruments
13 were in place, I do agree that the double testing gets
14 a lot of people upset, but we have no way to get good
15 data anymore.
16 And there's a lot of instructional time,
17 quite frankly, that is wasted in the classroom, and
18 there is a lot of time in the school day that gets
19 wasted, too. And this becomes something that devolves
20 right down to the local level, to superintendents,
21 assistant superintendents, and building principals,
22 about how they're watching their buildings, how
23 they're governing their systems, and what is happening
24 at the most important level the teacher in the
25 classroom.
252 NEAL R. GROSS
256 52
1 So I'm not always convinced that the loss
2 of instructional time argument should take us off the
3 track of building assessments that ultimately can
4 better improve what happens in the instructional time.
5 And then, maybe at some point we won't have to test as
6 much anymore.
7 I am not so much I said this earlier to
8 someone, that I'm amazed at the exquisite complexity
9 of the instruments that are at use around the country
10 right now. There is convergence on what we're
11 assessing for off of the standards with the banding.
12 I'm seeing multiple types of psychometric
13 architectures, from straight vertical scaling to
14 vertically moderated standards, you know, to a flat
15 scale. I'm seeing different constructs. I mean, I'm
16 seeing the modalities evaluated, but different
17 weighting when you go in and you start looking at
18 people's tech manuals.
19 So right off the top, how we weight New
20 York and ultimately give a proficiency score on how my
21 colleagues from California, et al., very, very
22 different. So even still I think it's hard from a
23 statetostate perspective, even the standardsetting
24 methodologies when I was reading reviewing
25 Jamal's chapter recently where he you know, he
257 NEAL R. GROSS
261 53
1 inventoried everyone, I've seen body of work, modified
2 body of work, modified Angoff, which is a pretty gutsy
3 approach I would think, but really, really interesting
4 when you start to think about this population, though.
5 So I'm going to actually call a couple of
6 the states that have done a modified Angoff standard
7 setting to talk with them. And then, of course,
8 everyone's favorite item mapping. And then, I'm
9 seeing multiple cuts three and fourcut systems,
10 and it goes back to Alison's point.
11 So you always get this convergence and
12 divergence in largescale assessment. You sort of get
13 an agreedupon testing format, psychometric
14 architecture and approach. And then, you get so many
15 different ways in which what the states want to build
16 is conducive to their standards or their instructional
17 approach or their population.
18 You know, a state like mine has a
19 significantly lower population of nativeborn and a
20 higher population of newlyarrived versus some of the
21 other states, you know, that we've seen. So some of
22 those nuances I think are reflected in people's
23 testing architecture.
24 I go back to the notion now that the thing
25 I'm most interested in for New York that we now
262 NEAL R. GROSS
266 54
1 that we are in a doubletesting phase is beginning to
2 calibrate the ELP instrument better, but to get the
3 ELP instrument to serve one foundational master ELP.
4 I'm very concerned when you start to mush
5 too much stuff into a test that's very technical
6 the technical language there. I don't want to have
7 to define those terms. But when, you know, you start
8 to have a test serve too many different masters, then
9 it gets so when I was listening to Charlene talk
10 about the mapping and the precision, that's something
11 I'm interested in, because then I think where Supreet
12 and I do agree because I agree with the majority of
13 her points other than instructional time is that
14 we'll wake up with a better instrument to make: a)
15 policy determination, and then, b) how that will
16 reflect accountability policy.
17 MS. CALDERON: Okay. Well, I really want
18 to emphasize something that you said, David, and that
19 is the issue of looking at reading. We've talked
20 about language, we've talked about content, but if the
21 students can't really read the test item, can't
22 decode, can't comprehend, that's a whole different
23 issue.
24 And I don't think we've looked at reading
25 enough in the field of ELLs. And so there has got to
267 NEAL R. GROSS
271 55
1 be a way of integrating that all that research into
2 the subject areas and also, of course, the language.
3 MR. LINQUANTI: I saw an indication for
4 that Richard did actually want to say something. Is
5 that correct, Richard? And then, Diana and
6 MR. DURAN: Yes. I think that some of
7 these issues are dovetailing with area 3, so I
8 withheld my comments to make them more pointed.
9 MR. LINQUANTI: So we'll actually begin
10 area 3 with you, Richard.
11 (Laughter.)
12 MR. DURAN: Other comments on are there
13 other comments from
14 MR. LINQUANTI: Diana has
15 MS. AUGUST: Yes, I just have a couple of
16 comments. First of all, not to be contentious, but
17 I'd like to side with my colleague from Maryland that
18 I think there is a huge issue in the amount of time
19 that we're spending testing kids, because it really
20 does take away from instruction, which is really what
21 we need to be doing.
22 That's not to say that assessment is not
23 important. I think it really is important. But I
24 think that there is too much standardized testing
25 going on right now, because it is getting in the way
272 NEAL R. GROSS
276 56
1 of instruction. That's the first thing.
2 The second thing is just a comment on the
3 amount of funding that is being used to develop,
4 validate, and administer assessments, compared with
5 overall spending on K12 education. So there was
6 about $750 million out of a total K12 spending of
7 more than $500 billion spent on assessment. There is
8 just not enough.
9 So to do a decent job of developing
10 assessments and validating them, and administering
11 them, which would include providing technical
12 assistance to those administering the assessment, we
13 really need to probably increase the amount of funding
14 available for the development of these assessments.
15 A third comment has to do with and this
16 has been raised already, but developing assessments
17 that meet the needs of students from diverse language
18 backgrounds. I don't know if there is anyone from
19 Montana here, but there is a real issue with, for
20 example, in Montana at least the data I have
21 indicate that there 81 percent of the LEP students
22 are American Indians in that state.
23 So what kinds of assessment issues, for
24 example, do we need to think about when we think about
25 assessing English language proficiency in different
277 NEAL R. GROSS
281 57
1 populations of students? And then, finally, the last
2 comment has to do with developing language proficiency
3 assessments that really tell us when we can provide
4 content area instruction to second language learners.
5 This is really critical.
6 So if we're giving these kids language
7 proficiency tests, whatever they may look like, based
8 on whatever theory they are based on, we need to be
9 able to figure out, given this assessment, when is it
10 reasonable to put children into all English
11 instruction without additional support.
13 MS. KOPRIVA: A couple of points. The
14 first one and this is not the next question, so I'm
15 in good shape is I do agree with Charlene that we
16 have to we have to make sure that our ELP standards
17 to the test are precisely followed.
18 I think the other thing, though, is we
19 need at the standards level we have to we have
20 to build standards that differentiate clearly. Just
21 like I was just talking about with the test, they have
22 to differentiate clearly between English language
23 proficiency standards and the content standards. And
24 that includes in science and math as well as in
25 reading.
282 NEAL R. GROSS
286 58
1 And I think that's an important
2 distinction, because I think for my colleagues in
3 mathematics and science and social studies I think
4 that there is a language component, and that component
5 has got to get has got to get teased out and decide
6 which side of the fence it is going to be sitting on.
7 All right. The second thing is that at
8 the beginning of David's point
9 (Laughter.)
10 there you are at the beginning of
11 David's point, he had said something about developing
12 validity arguments. I think it is very important that
13 states begin to get in the habit of developing
14 validity arguments for any of their tests their
15 tests that are new, but also their tests that are old.
16 Validity arguments are that you,
17 essentially, oh my God, think about which validity
18 studies you want to do. And it means that you
19 prioritize, and it means that you actually put some
20 kind of timelines on them, and, oh, you do them,
21 because if we so the validity argument, if it's
22 followed through and carried through, it begins to
23 what it will do with the data is strengthen the
24 inferences that you can make.
25 But right now, the way state testing and
287 NEAL R. GROSS
291 59
1 all of our testing is done my testing, everybody's
2 testing is that we do validity studies in a rather
3 haphazard manner, and so this this validity
4 argument point really needs to occur, even if we're
5 have had the test for 10 years or five years.
6 The last point is I think when we are
7 doing validity studies, particularly as we are moving
8 in as we have these new areas of English language
9 proficiency for instance, I think it's important that
10 we use independent indices of validity of the
11 constructs, that in other words it's not enough to
12 just do correlational, and so on, but that we actually
13 need to get some other kind of independent information
14 that we are using to measure the success of our
15 instruments and what it is we're really measuring.
16 A lot of times validity can be a hmm,
17 what's a good word for that? Essentially, it follows
18 it just kind of runs around like a dog and a tail
19 that, you know, essentially, you know, you got a bad
20 test, and if it correlates great with a bad test it
21 must be a good test.
22 (Laughter.)
23 Makes no sense, so
24 MR. LINQUANTI: Well, I'm just going to
25 make note that we're right on time to finish at about
292 NEAL R. GROSS
296 60
1 4:00. However, we only have until 2:30.
2 (Laughter.)
3 We started a bit late, and we have more
4 folks than we've had in any of these other hearings,
5 so I think we'll go a bit over. But we do need to
6 respect the public comment period as well.
7 So with that, I'm just going to transition
8 us rather forcefully and arbitrarily to question 3,
9 trusting that if you had a point to make you were
10 going to weave it in anyway, so
11 (Laughter.)
12 Richard, why don't you head up?
13 MR. DURAN: Okay. Moving right along, as
14 you see, we are blending across areas quite
15 effectively. So the issue of aligning language ELP
16 assessments and standards, this we have a history
17 of language proficiency testing which is influencing
18 the way we think about things.
19 If you read folks like Bernard Spolski
20 going way back, the separation of assessments into
21 modalities, the four modalities, is part of our
22 legacy, but it has also created a contradiction for
23 us. I know one state, when they were trying to
24 develop their English language development standards,
25 who their initial draft of their English language
297 NEAL R. GROSS
301 61
1 proficiency development standards were entirely
2 integrative. Okay?
3 They were about how to combine speaking,
4 reading, writing, and listening, because that's the
5 way language happens. We've inherited a kind of
6 contradiction by going for assessments in separate
7 modalities, so we want it all. We want to say that
8 we're measuring different skills, yet language
9 practices occur in a blended way.
10 We want an issue that states face,
11 then, is in terms of validating their standards. Do
12 they conceptualize that English language proficiency
13 is one construct that uses four skills, or are there
14 four constructs that contribute to one overall
15 language construct of language proficiency?
16 The psychometricians can be very helpful.
17 (Laughter.)
18 They can fit anything to anything. How
19 to what degree is the issue.
20 (Laughter.)
21 But you can model anything. But the
22 answer to that question is really, what state policies
23 for English language learners and English language
24 development will test scores have an effect on, have
25 an influence on? And that will affect the design of
302 NEAL R. GROSS
306 62
1 instruments.
2 The test developers for English language
3 proficiency can create unidimensional English
4 language proficiency tests that sample across the four
5 modalities. And they can extract out reading and
6 listening to create a comprehension measure. That's
7 kind of the default way it's going.
8 If they want to have a single construct,
9 you don't have to have that. You can create separate
10 constructs for the four modalities that will allow you
11 to separate them, and then you have to come back and
12 figure out how to blend them. So those are our
13 technical arguments.
14 But there is more here that's challenging.
15 One very important issue is whether the construct of
16 English language proficiency, however you conceive of
17 it, changes across grades and the academic demands
18 across grades. This is a very challenging problem,
19 given the heterogeneity of English language learners
20 and when they enter the schools, and what English
21 language background they have and what academic
22 background they have, because there is going to be
23 differences that might emerge among populations, and
24 we need to understand that better.
25 So I think the answer is that states are
307 NEAL R. GROSS
311 63
1 going to come up, and consortia are going to come up,
2 with different strategies. The bottom line is: how
3 will they argue that their construct of English
4 language proficiency is mapped onto their English
5 language proficiency assessment? What can't be mapped
6 onto the English language proficiency assessment, and
7 how states then hold themselves accountable in terms
8 of the inferences that they are drawing from the test
9 scores or other kinds of evidence?
10 MR. LINQUANTI: So Richard is raising that
11 issue of accessibility of the standards for sure.
12 Feel strongly, David? Okay.
13 MR. ABRAMS: I'm just happy to ride
14 Richard's coattails. For the sake of disclosure,
15 Richard is on the New York State Technical Advisory
16 Committee. He is on our TAC, and he has been very
17 helpful for the work that he has done for our state,
18 and I just want to thank him publicly for that at this
19 hearing.
20 The TACs I think are very important to
21 states, and a lot of people seem to miss that in the
22 larger discussions about policy and testing and
23 accountability. And I really grappled or struggled
24 with this question only because of the experiences we
25 had with the Title I peer review and running alignment
312 NEAL R. GROSS
316 64
1 studies and independent alignment studies, and seeing
2 what the research is, the methodologies that are out
3 there. And as Richard said, the ways that ultimately
4 they do or don't work for you.
5 And we've been grappling in New York with
6 the results of our first you know, our alignment
7 study, which were solid, but, still, there are things
8 we are not assessing. There are still things that are
9 kind of manufactured by the methodology of the study,
10 so I am not 100 percent comfortable with the "hard
11 science" of it.
12 In many ways, to go back to Rebecca's
13 point I can see from distances, so I don't feel bad
14 that I didn't put my glasses on.
15 (Laughter.)
16 Because, as we were grappling with one
17 of the things the Commissioner said to me at the end
18 of some of the peer reviewed things with Title I was,
19 you know, there were some hard lessons to be learned,
20 but they helped us all in the various states to
21 sharpen and get better.
22 And one of the things that grew out of it
23 for our state was a very the ability to define and
24 prioritize research against our capacity for what is
25 really important. What are you doing with these
317 NEAL R. GROSS
321 65
1 tests? What are the stakes associated with them?
2 What do you want to know? How do you want to tool it
3 back into your standards design?
4 So I'm still kind of with all of that
5 work going on, I'm kind of hoping that as the panel,
6 you know, goes through, you'll answer the question for
7 me about the best issues with alignment, but I don't
8 think it's there, because I'm still not seeing it in
9 the research with, you know, sort of the methodologies
10 that are there.
11 So I would say that I want to echo the one
12 the one point about the change in ability across
13 grades, and what all I'm sure all states have to
14 see this, that as kids come in in later grades their
15 ability to graduate begins to plummet, because of the
16 academic demands. Supreet talked about some of the
17 cultural issues that are certainly in play.
18 But it's the expectations of what a
19 credential signifies. So we still don't have, I
20 think, policy that takes into account late arrivals,
21 SIFE, and some of the things that happen when a kid
22 comes into your system at the middle level on, and
23 especially at the high school level, which is why I
24 have so many questions about the dynamics of language
25 acquisition and reading simultaneously, and how you
322 NEAL R. GROSS
326 66
1 are going to deal with the two.
2 We seem to be not so much skirting around
3 this question, but hitting it head on, that we really
4 have to sort of solidify this discipline of English
5 acquisition, language proficiency acquisition, and
6 then begin to take a look at, when do you tool it out
7 into the content, the academic content areas for the
8 rhetoric that kids need to be able to manipulate? And
9 how do you make that grade level appropriate?
10 The kids come in at so many different
11 levels of achievement. You know, I'm sure many of you
12 in your states see especially with any type of
13 special population assessment, everyone does the
14 sweeping generalization, "The kids can't read; don't
15 ask them to do it. The kid has been here a year and a
16 half; don't test them."
17 And what you begin to see with that type
18 of mindset is you are losing how to tease out the
19 differences in the subpopulations of this larger
20 special population to find those specific groups and
21 then get your policies and your standards aligned to
22 their needs.
23 And I think there are some differences
24 the point that Diana raised about the western states
25 with lots of concentrations of Native Americans, the
327 NEAL R. GROSS
331 67
1 language issues there versus, say, in an eastern
2 state.
3 So I think that that is a very important
4 thing for us to look at as well as maybe getting some
5 convergence in the measurement community on the best
6 practice for alignment study, or, as Rich said and
7 Rebecca said and I'll echo, we'll all figure out a way
8 to run an alignment study and make the tests look
9 aligned. And they will be, but that doesn't
10 necessarily mean we caught everything.
11 Should I pass it to Margarita?
12 MS. CALDERON: I'll just say that Richard
13 said everything I wanted to say, except nicer.
14 (Laughter.)
15 And, of course, the changes across the
16 grade levels, that is so important, especially for the
17 older students.
18 MR. LINQUANTI: Colleagues on the panel
19 that feel particularly strongly about this question?
20 Because we're really trying to split the time between
21 this and the last, so if you everybody feels that
22 the most important things have been said? Okay,
23 great.
24 Then, on to the last question. You are
25 surprising me. I'm getting disoriented.
332 NEAL R. GROSS
336 68
1 (Laughter.)
2 So, then, the last question, "What are the
3 strategies that states can use to ensure that their
4 ELP standards" and this is a mouthful "are
5 aligned with the achievement of challenging state
6 academic content standards and student academic
7 achievement standards?"
8 Let's try that again. "What are the
9 strategies that states can use to ensure that their
10 ELP standards are aligned with the achievement of
11 challenging state academic content standards and
12 student academic achievement standards?"
13 MR. BOALS: I'd like to say that as we
14 look at all of these parts, we are getting a very
15 complex system of interactions going on here between,
16 as Charlene said earlier, well the standards have to
17 be good, but we know that the standards are going to
18 get better over time, and that the assessments are
19 going to have to be revised and they are going to have
20 to be aligned.
21 And I think the difficult piece for states
22 is they are not used to this, especially in the world
23 of English language learners. Probably more used to
24 it in the world of kids at large, so hopefully we can
25 build on the precedents that have been set out there.
337 NEAL R. GROSS
341 69
1 But this is not an inexpensive venture
2 to create and recreate standards, to then align and
3 improve, based on those alignments, the assessments.
4 And, you know, if you look at those prior generation
5 of assessments, a lot of them were static for 20
6 years.
7 And now we're in an era where these
8 assessments are going to have to change almost
9 annually to keep up with the changes in the standards,
10 what we learn about alignment methodologies that allow
11 us to improve that as we go, so that's the other thing
12 about flying the airplane as you go is that you've got
13 to keep on maintaining it.
14 And this is going to be costly, frankly.
15 And we also have to bring it back down to the level
16 of, you know, once the states have a handle on this,
17 they've got to take it all and translate it into
18 meaningful professional development tasks for teachers
19 that help them actualize this within the classroom.
20 MS. ANAND: I'm going to answer this
21 question from a practitioner's perspective, because,
22 like many of my other colleagues at the state level,
23 we are still a little confused on how to put into
24 practice this alignment with our ELP standards and the
25 content standards.
342 NEAL R. GROSS
346 70
1 We have looked at it from two perspective,
2 and I want to share those with you. One way we've
3 looked at it is we've taken the content objective, and
4 we have differentiated across proficiency levels by
5 adding the language implications for each of those
6 proficiency levels.
7 And then, our confusion is in that, is
8 that ELP alignment, or are we just supporting content?
9 We can't dissociate the model of instruction when we
10 are talking about these kinds of issues, because that
11 works well perhaps in a SIOP model, and you have
12 language implications, you've got content
13 implications, you can mesh the two. But is that ESL
14 instruction, then? Is that in lieu of ESL
15 instruction? Is that enough? Should we still have
16 additional ESL support? Where do we go from that?
17 And we're not clear on all of that.
18 We are also looking at you know, we
19 talk about collaboration, and TESOL has put out a
20 wonderful book on collaboration for ESL teachers and
21 content teachers. And recently we did a workshop with
22 our content and ESL teachers combined to talk about
23 collaboration, and one issue that kept coming up is I
24 am an ESL itinerant ESL teacher.
25 I teach at five schools. I interact with
347 NEAL R. GROSS
351 71
1 50 teachers. Who do I collaborate with? How do I
2 make this work for me? And so what I put this out to
3 just say is that we have to look at all of those
4 practical issues, if we truly want this to be
5 implementable, if that's such a word.
6 The other thing we looked at is perhaps at
7 the concept level. Perhaps we could say that the role
8 of the English language proficiency standards or the
9 English language proficiency teacher is to teach the
10 language concept like, let's say, words have multiple
11 meanings, so that the student with the ESL teacher
12 learns that concept through different examples. And
13 then, when he goes to the math class, he carries that
14 concept with him, so he understands that the table
15 means more than just this.
16 Is that a way to approach it? You know,
17 what is the best way? Is there a best way? Are there
18 multiple ways? We, at the practitioner's level, are
19 not clear on that, and we really need guidance on,
20 what should that look like? What's the best way? And
21 how to implement don't leave that part out.
22 MR. LINQUANTI: Yes, Richard.
23 MR. DURAN: One issue here is: what are
24 states likely to do in terms of trying to show a
25 relationship between their ELP assessments and their
352 NEAL R. GROSS
356 72
1 content standards and achievement areas? And one
2 thing we are going to see, no doubt, will be a
3 sampling of items that come from achievement tests in
4 the content areas, and some interpretation that they
5 have a language load.
6 I don't happen to agree fully with Rebecca
7 in terms of how, let's say, the goal of separating
8 English language proficiency academic English
9 language proficiency from content, and it's a slippery
10 slope. It's not that I disagree entirely. It's a
11 slippery slope, you know, so more is a term that is
12 used in regular English, but it has a quantitative
13 meaning.
14 And so, but I think it is going to be very
15 challenging to try to create these relationships, but
16 it is going to be incumbent on states to somehow have
17 a sampling design of some kind to say how they get
18 academic content into their English language
19 proficiency tests.
20 One thing that is very unlikely would be
21 that there would be any possibility of getting any
22 subscore on an English language proficiency
23 representing competence or proficiency in a math
24 register. You're not going to have enough items,
25 you're not going to be able to sample adequately, and
357 NEAL R. GROSS
361 73
1 it's a great burden to go across grades to create a
2 system of that kind.
3 So you're going to have some content that
4 will be represented in the English language
5 proficiency. That's likely.
6 One thing that is going to happen that I
7 think is going to be very interesting to monitor from
8 a measurement perspective is that it's likely to
9 increase the relationship/association/correlation
10 between English language proficiency test scores and
11 content scores.
12 Now, what also is happening here is this
13 very mysterious murky relationship between what some
14 call a G factor, ability to learn and be able to learn
15 content and language that is going to blend across.
16 And that is going to have some interesting influences
17 in that it will show that there's more of a
18 relationship than otherwise might be, and that needs
19 to be thought through.
20 But I think it's going to require,
21 obviously, states to make an argument on how they
22 sample from the content areas and how to interpret the
23 test scores with regard to knowledge of English as
24 required in those content domains, with the
25 understanding that you can't have a discrete event on
362 NEAL R. GROSS
366 74
1 a test represent learning in a math lesson in the same
2 way that you can when it's a real activity.
4 MS. KOPRIVA: I would agree it's not easy,
5 and I would agree it's a slippery slope. I think that
6 I think the correlations are there. I think they
7 will be there in terms of the relationships between
8 content and English language proficiency.
9 The issue here really is that there
10 each needs to be defensible, and they need to be
11 defensible on different grounds overall on the test.
12 So, in other words, the and I, again, am not an
13 English language proficiency expert at all, so I'm
14 only going to say the wrong words using the wrong
15 stuff, so sorry.
16 But anyway, essentially, I mean, I think
17 what we're focusing on here is that in the academic
18 language proficiency portions I think that you have
19 essentially, you have to provide opportunities for
20 students to use structural and discourse elements that
21 and that those I guess you were using the term
22 "register." Those registers change over content areas.
23 So in a science classroom, it looks
24 different. In a physics classroom, it looks different
25 than it does in a history classroom, or whatever. And
367 NEAL R. GROSS
371 75
1 that those for English language proficiency
2 purposes, to measure the structural elements of a
3 formal language in those two kind of content areas,
4 that's part of the English language proficiency test.
5 If it's the stimuli the stimuli may be history,
6 but, really, we're looking at those structural
7 elements, those okay, and the discourse elements.
8 In history, we're focusing on history.
9 And, you know, I think that how do we disentangle
10 some of these? I think I hate to say it, but I
11 think it gets back to people talking to each other,
12 and, really I'm serious people that have are
13 evolved in the English language proficiency
14 understanding, especially academic language
15 proficiency understanding, talking with the content
16 people and sorting out these issues of discourse, and
17 so on, and then also and when that's content and
18 when that's the other, and then, the English language
19 proficiency.
20 And also, this issue of vocabulary and
21 technical phrases and all of that. I think this is
22 something I do think these are workable. I don't
23 think it's black and white, but I do think that it is
24 something that in a sense, it's just like we always
25 do in assessment. There are some decisions that have
372 NEAL R. GROSS
376 76
1 to be made, but I think if we stick our heads in the
2 sands and don't make them, we won't we are going to
3 continue to confuse the issues.
4 MR. LINQUANTI: Sari, and then David.
5 MS. LUOMA: Just briefly to build on these
6 two previous comments, you have a requirement to
7 relate the state ELP standards to the state's academic
8 content standards now. So one simple way to do that
9 is what Supreet began with, but didn't actually
10 emphasize that they did, is go back to the content
11 standards and define the language requirements,
12 because that's where you're going to find the
13 alignment, and only that should be where you find the
14 alignment, because the ELP standards are not supposed
15 to be content standards, and the content standards are
16 not supposed to be ELP standards. But the content
17 standards do imply language requirements.
18 So before this alignment is conducted,
19 someone needs to go through the content standards and
20 try to work out what the language implications of
21 those content standards are. And it's not going to be
22 easy, because when the content standards were written
23 they weren't written from the perspective of language
24 requirements very often.
25 So you will find, because we did in our
377 NEAL R. GROSS
381 77
1 test development we went through a number of
2 content standards and we found the different verbs
3 that are used for activities that are going to be
4 happening in the classroom. And it is very difficult
5 to distinguish which skill is going to be required,
6 what does "interpret" mean, what does "interpret"
7 require, what does but basically, if you don't go
8 through some of that work, you won't be able to show
9 alignment to anything except content, which is going
10 to be the individual tick here, tick here, tick
11 here, five percent alignment, thank you.
12 MR. LINQUANTI: David, and then Alison.
13 MR. ABRAMS: Thank you. I wanted to
14 briefly follow up. Those are good points, and when I
15 read this, I was sort of looking at that question
16 about the empirical studies, one of the bullets down,
17 and I was thinking firstly, from my chair, I have
18 always sort of viewed exams as multilingual, because I
19 tend to look at rebuilding our testing programs from
20 our language our verbal language and our
21 mathematical language.
22 So someone brought up math before and I
23 always have argued to math people when they complain
24 about English, it's a bilingual exam, you know, and
25 that the goal is to make sure that the English doesn't
382 NEAL R. GROSS
386 78
1 confound the math. But I so now, with the ELP
2 kids, you are introducing multiple languages, because
3 for one thing they could be reading the test in the
4 language it's presented, and then begin to process the
5 test and you can't see how they are processing the
6 test in multiple languages mathematic, language 1,
7 language 2, 3, 4, etcetera.
8 So that's a real issue that sort of
9 intrigues me and fascinates me simultaneously, which
10 is why when I heard at the first LEP or second LEP
11 partnership meeting, the discussion of the language
12 threshold measure, it sent me back to the books to
13 start doing a lot of research with the linguists.
14 And it's something that I talked to my own
15 Commissioner in our state for standards review, that
16 as we put all of the standards into review that,
17 between subject area experts, cognitive psychologists,
18 that linguists are sitting in the room as well for a
19 lot of this issue these issues, to help us.
20 My concern is when we get pushed into
21 consequential validity studies because of the politics
22 or the dynamics of saying a student is proficient to
23 go out, and there are a group of people who say,
24 "Great, they are going to go out and pass everything,"
25 and then they don't, and then they don't pass the test
387 NEAL R. GROSS
391 79
1 and people say, "See, there is something wrong,
2 there's something wrong with your test, because there
3 is always the" that's why I talked about these
4 calibrations between what's our ELP exam telling us,
5 what's our ELA exam telling us, and it's very possible
6 that a kid, you know, can be proficient and still fail
7 a test.
8 Monolingual kids are failing the ELA exam
9 to a tune of about 20 percent a year in my state, so I
10 don't understand why that is so hard to figure out
11 that that probability theory exists. And if an
12 English major can see the probability, I would think
13 everyone else can, too.
14 So I would say that this is an area where
15 I hope, in conjunction with the Federal Government and
16 the measurement experts, we can get some consensus
17 about with the measurement experts on how to go
18 about it, so we don't wake up in some respects, you
19 can't dictate to all the states, "Build your test like
20 this, build your standards like that."
21 But if we can agree upon some
22 methodologies, so that we can begin to get a better
23 sense of respecting states' rights by still generating
24 data, then we can all begin to pool and do some of
25 these consequential validity studies and alignment
392 NEAL R. GROSS
396 80
1 studies, I think we'll get to a better place for
2 understanding the discipline, but then begin to do a
3 better chance of the instruction, because Tim said
4 something and I agree with him that it still all
5 goes back to the quality of instruction.
6 And so my last comment on this is not to
7 be contentious, but only to say to my colleagues, who
8 I still disagree with, of loss of instructional time,
9 our teachers are testing kids all the time. So they
10 are testing kids with instruments they are building on
11 their own, they are testing kids with instruments that
12 they buy through curricular programming, and there is
13 very little good jurying of those instruments in
14 schools, because there is not a lot of good assessment
15 practice or knowledge, because in your teacher prep
16 programs and your administrator prep programs it's not
17 there. And a methods class is not a measurement
18 class.
19 So this is one of the reasons why it's
20 hard to have to live in the universe of always giving
21 standardized tests. But as I say to my own folks in
22 my own state, and in the Department, it is the one
23 thing that can help protect the right, and it does
24 protect the right of the individual student. So it's
25 incumbent upon us to do them correctly.
397 NEAL R. GROSS
401 81
1 MS. BAILEY: I mean, the thing is but,
2 really, I mean, if we're going to get the answers we
3 need for validity, for what it means to be aligning
4 tests to standards, and when this piece of Title III
5 came out about aligning content standards to academic
6 language sorry, just language standards, ELP
7 standards, I remember reading that and thinking, what
8 the heck is this? I couldn't even begin to consider.
9 I thought it was a typo, that it was
10 you know, anyway, and then we spent the last four or
11 five years in California or Richard will know this
12 from our TAC in California trying to interpret
13 this, and then have a methodology to operationalize
14 that requirement.
15 But the thing is, if we're going to play
16 with those things, you know, to get states to feel
17 comfortable, to do the experimental work that needs to
18 be done, maybe we do need to ask for certain special
19 requirements in time to actually work with it, and not
20 have it be high stakes for those students and schools
21 who are involved. So I would just say that.
22 MR. LINQUANTI: Charlene, do you
23 MS. RIVERA: All this reminds me is that
24 we really need to keep in mind what the purpose of the
25 ELP assessment is, and I think it has a limited
402 NEAL R. GROSS
406 82
1 lifespan in a sense in a student's life. And so I
2 just would like to recommend that we don't get
3 overzealous in putting all of our efforts or our
4 resources into developing perfect ELP assessments when
5 we really have the challenge of creating content
6 assessments that are accessible to English language
7 learners that are at lower levels of English language
8 proficiency.
9 And I think that's I just would like to
10 say that I think that is where we need to put our
11 resources in the future.
12 MR. BOALS: Well, I will just say that I
13 wholeheartedly agree with that, and, Rebecca, I'm sure
14 you would as well that we need to develop content
15 assessments that truly measure what students,
16 particularly at the lower end of the continuum I
17 think that's where we're losing the data. I think the
18 accommodations can work to a lesser or greater extent.
19 We're still struggling with how well they
20 work, but we know that when they do work it's probably
21 the upper end of the continuum of English language
22 development. And for those kids that are really in
23 the first three years, that have arrived here and
24 speak very little English, I mean, what we're doing
25 largely around the country is we're placing we're
407 NEAL R. GROSS
411 83
1 handing them number two pencils and we're putting them
2 in the regular test.
3 And at the gut level, we know that's
4 wrong, and we know that teachers at the local level
5 are screaming, "This is not right." Yet we have to
6 have accountability, so we're not denying the need for
7 that accountability, but we've really got to hurry up
8 here and create assessments that tell us something
9 meaningful for those kids.
10 MR. LINQUANTI: Diana?
11 MS. AUGUST: Yes. I mean, I would agree
12 with that. One recent study indicated that the only
13 valid and effective accommodation was the use of
14 glossaries, for example, in English, and possibly
15 using native language assessments for children who had
16 been instructed in their native language.
17 I'd just refer you to a report by David
18 Francis from published by the Center on
19 Instruction. So, I mean, for me the and I can't
20 agree more with Charlene. The issue really is how we
21 can assess the content area knowledge of children with
22 not just different levels of English proficiency
23 across the grades, but also with different levels of
24 content knowledge, which is a separate issue, right?
25 If you're giving a child a grade
412 NEAL R. GROSS
416 84
1 appropriate assessment, say at the seventh grade
2 level, and that child has had no prior schooling, it's
3 not an issue just of language proficiency. It's an
4 issue of creating enough items at the lower end of the
5 scale to measure what that child might and might not
6 know.
7 And so we're talking about assessments
8 that cover a wider range can validly assess different
9 levels of content knowledge as well as language
10 proficiency. And if I were to put my money anywhere,
11 I mean, that's where it would be.
13 MS. KOPRIVA: Because I have been doing
14 this for too long, I do think that the research over
15 the many years has produced assessments that are
16 that at least are helping kids who are more literate.
17 So students who are still English language learners,
18 but so it is but it is the ones who are more
19 literate. So I think we actually have made progress.
20 Those of you that were around 10 and 15
21 years ago, the assessments are changing. The content
22 assessments are changing, and the accommodations are
23 getting better. And we do have more empirical backup
24 for the ones that work and the ones that are
25 ridiculous.
417 NEAL R. GROSS
421 85
1 So I think that I think mainly who it's
2 working for are your advanced, your more your
3 higher levels of English language proficient students.
4 So it's good. I mean, those have been valuable, and
5 they have been useful, and I'm glad that they are
6 helping. And I think it's making a big difference.
7 I also think it has made a big difference
8 in the general test that I think we've got much I
9 think we have better editing techniques, that we have
10 forced better editing techniques. You can call it
11 universal design, though I'm always amazed at what's
12 called universal design. But anyway, better editing
13 techniques, and that has forced that I think for great
14 for all students, not just for your English
15 language learners.
16 So I but I do, of course, agree with my
17 colleagues that I think low literacy is where it's at.
18 It's a whole set of reasons why that's true.
19 MR. LINQUANTI: And when you mention
20 editing techniques, I imagine you're talking about
21 removing unnecessary linguistic complexity that's
22 unrelated to
23 MS. KOPRIVA: For what?
24 MR. LINQUANTI: construct? In the
25 content area assessment.
422 NEAL R. GROSS
426 86
1 MS. KOPRIVA: In content area assessment,
2 yes.
3 MR. LINQUANTI: Yes.
4 MS. KOPRIVA: Well, we're measuring the
5 same it's extremely important to measure the same
6 content, and the same cognitive complexity of that
7 content. You never, never change that. What you do
8 is you take when you're reducing some of the
9 others, we seem to be fairly pretty effective at
10 knowing how to do that now for your upper range of
11 English language learners.
12 MR. LINQUANTI: Richard? Final word?
13 MR. DURAN: Yes. Well, a quick comment on
14 the longterm English language learners and who go
15 into the middle and high school years who are don't
16 exit ELL status and who are performing low in reading
17 and language arts tests and in other areas.
18 I think right now this is
19 underconceptualized in terms of federal legislation.
20 That may not be so bad if states can take seize the
21 handle on that and figure out how their state policies
22 can deal with heterogeneity of English language
23 learners and how lowachieving adolescent middle
24 school and high school students, ELL students, what is
25 going on with them.
427 NEAL R. GROSS
431 87
1 There is room here for other ways of
2 conceiving of the acquisition of academic and language
3 competence that deal really with the socialization of
4 these students and their opportunity to learn that's
5 accumulated across years. Right now, this is a kind
6 of hole that's there, but it also represents
7 potentially an opportunity for states to step up and
8 figure out how to take their existing assessments and
9 how to be able to deal with this policy issue.
10 MR. LINQUANTI: I think we will give the
11 final word to Margarita.
12 MS. CALDERON: Okay. Just to follow up on
13 something that Supreet said, and that is the whole
14 notion also of empirical studies for helping math,
15 science, social studies teachers also take more of an
16 ownership of ELL content areas, but that implies a lot
17 of teaching, teaching the reading teachers, special ed
18 teachers, ELL teachers, content teachers, and so, as
19 Diana mentioned, there should be more funding for
20 assessment, there should definitely be more funding
21 for empirical studies on professional development and
22 teacher development.
23 MR. LINQUANTI: Well, we've gone 20
24 minutes over, but I think it was well worth it. My
25 apologies to the folks who are waiting to give public
432 NEAL R. GROSS
436 88
1 comment.
2 I'd like to thank the panel. This is a
3 tough task in the given timeframe, and I think they
4 really shed a lot of light on these issues today. And
5 I exercised great restraint in not contributing,
6 because it's very tempting, and I'm sure you all did,
7 too. So I want to thank you all, and thank the
8 panelists for coming.
9 (Applause.)
10 MS. SKANDERA: Thank you very, very much,
11 and we will take a 15minute break. And I'll just
12 exhort you once again, if you want to make comments,
13 to sign up outside these two doors, and you'll get a
14 number, so you can know where you are in the queue.
15 Oh, and there's refreshments outdoors. And we'll look
16 forward to seeing you in 15 minutes.
17 Thank you.
18 (Whereupon, the proceedings in the foregoing matter
19 went off the record at 2:52 p.m. and went
20 back on the record at 3:20 p.m.)
21 MS. SKANDERA: As I mentioned before, this
22 is the opportunity for public comments on the and
23 input for the framework for developing highquality
24 ELP standards and assessments. And there was an
25 opportunity over the break to sign up, get a number,
437 NEAL R. GROSS
441 89
1 so you're in the queue, to share. There is also an
2 opportunity until August 1st to actually go online and
3 provide comments. If public speaking is not your
4 favorite, there is still an opportunity for you, or
5 you can speak today and also provide public comment
6 via online until August 1st, until 5:00 p.m.
7 I will say that we have had two other
8 sessions just a refresher not unlike this one,
9 with different experts, so there has been a great deal
10 of high just really good input and dialogue around
11 this issue. And I would encourage you we are
12 publicly posting all of the comments made by experts
13 and the public comment period or written comments, so
14 you can take a look at that online as well.
15 It is my understanding that as of about
16 three minutes ago we actually did not have anybody in
17 the queue for public comment, but I want to exhort you
18 because this is your moment. This is it.
19 So for example, Robert, while you were
20 facilitating, you were, you know, curbing your
21 comment, but now is the opportunity. So unless
22 someone has recently signed up and received a number
23 oh, we've got one. Oh, that's cheating. But you
24 know what? We're going to let you kick it off.
25 (Laughter.)
442 NEAL R. GROSS
446 90
1 And just a real quick as far as
2 procedures, five minutes, although it looks like we
3 could have plenty of time for public comments.
4 (Laughter.)
5 And Tyler will provide you with kind of
6 your time constraints. Please identify yourself and
7 where you're from and who you're with as far as
8 organizations, and then
9 MS. BAILEY: Okay. My name is Alison
10 Bailey, and I'm at the University of CaliforniaLos
11 Angeles. And I just want the comment I want to
12 make, really, is that none of our discussion has been
13 a member of that expert panel dealt with the
14 comparison of English learners and Englishonly
15 students, and I just want it to go on the public
16 record that the tests we are developing are should
17 one form of validation would be to test those new
18 assessments with Englishonly students, because we do
19 not want to create a system that sets the bar higher
20 for English learners than for the Englishonly or
21 Englishproficient population. That's all I wanted to
22 say.
23 MS. SKANDERA: Do we have a public
24 commenter number two? And if not, I just want to open
25 up the mic, obviously, for public comments across the
447 NEAL R. GROSS
451 91
1 board. So feel free if you are inspired, even if you
2 are on the panel, to provide additional comments.
3 MR. LINQUANTI: Even if you are the
4 moderator?
5 MS. SKANDERA: Or the moderator. Didn't
6 take much.
7 (Laughter.)
8 The pressure was great.
9 MR. LINQUANTI: I'm Robert Linquanti from
10 WestED, and just one observation which actually came
11 in a debrief of the last panel. A question was posed:
12 could a student, an EL student, perform at the
13 proficient level on the ELP assessment and still be at
14 a fairly low level on the ELA assessment? And this
15 was related to the issues of alignment and linkage of
16 the standards.
17 And the question came back from the
18 experts I was talking to, yes, of course, because they
19 are measuring somewhat different things. And so a
20 student could easily be English proficient but yet not
21 meeting all of the standards to perform well enough to
22 be considered proficient in the ELA assessment.
23 Then, the next question I posed was:
24 could a student be at the English could a student
25 be proficient on the English language arts assessment
452 NEAL R. GROSS
456 92
1 and yet not measure proficient on the English language
2 proficiency assessment? And the answer came back,
3 yes, but it shouldn't be too common. It shouldn't be
4 too likely.
5 And I think it's important, because when
6 we're talking about issues of alignment and linkage,
7 to recognize where the constructs somewhat overlap,
8 where they diverge, and what is it that we're
9 measuring with each of these assessments. And so we
10 can think of scenarios of students that perform that
11 way beyond just sort of simple test error or bad test
12 day.
13 And I think that will be important for us
14 to remember, particularly when we're thinking of
15 predictive validity, that the ELP assessment will
16 necessarily predict how well the student will do on
17 the ELA assessment. If they don't, then there's
18 something wrong with the ELP assessment.
19 I think David was alluding to that as well
20 as others on the panel today, but I just thought that
21 one example, those two questions, would be good to
22 explicitly bring that out.
23 MS. SKANDERA: That's all?
24 (Laughter.)
25 Come on up to the mic if you will, and
457 NEAL R. GROSS
461 93
1 still introduce yourself and the affiliation you have.
2 MS. ANAND: I'm Supreet Anand from the
3 Maryland State Department of Ed, and I just wanted to
4 go back to what Robert just brought up. We actually
5 saw that in our data when we looked at our student
6 performance on the English language proficiency test,
7 and on our state reading language arts test. We had
8 both scenarios.
9 In the first scenario, I can't quite
10 explain that yet, but what we did was in our exit
11 criteria we made sure that we were looking at multiple
12 measures and not just English language proficiency as
13 an exit criteria. And on the second issue of where we
14 had students who did not score proficient on the
15 English language proficiency test, but scored
16 proficient on the reading language arts test grade
17 level, when we looked at that indepth we found
18 accommodations were playing a factor in that.
19 Certain accommodations allowed the
20 students to score proficient on the grade level tests,
21 even though they were not really proficient in English
22 yet. So I just wanted to share that.
23 MS. SKANDERA: Yet another panelist.
24 (Laughter.)
25 MR. DURAN: Hello. Richard Duran from
462 NEAL R. GROSS
466 94
1 U.C.Santa Barbara. And following this line of
2 discussion, I wanted to ask whether I could get a
3 clarification from the Department on ways in which
4 state redesignation criteria operate in relationship
5 to testbased criteria for exit from the status of
6 being an English language learner.
7 So as we know, states tend to use four
8 different criteria of course, performance on the
9 ELP test. They also use performance on the English
10 language arts test. They use teacher input and
11 judgment on students' ability to benefit from
12 instruction in English, and they also use parent
13 input.
14 Further, within states, the weighting of
15 these criteria can vary by school district, depending
16 on the education code of a state. And what I wanted
17 to understand better was how the Department
18 perspective on classifying students as English
19 language proficient based on test scores is treated in
20 comparison to criteria by states that then add in
21 these other markers in making their classification
22 decision on students, because I see that there is a
23 potential discrepancy here where state education code
24 and practices may not be aligned well with well
25 enough with criteria under NCLB.
467 NEAL R. GROSS
471 95
1 Thank you.
2 MS. SKANDERA: Unfortunately, we're not
3 able to answer questions on the
4 (Laughter.)
5 Federal Register. There's a chuckle.
6 (Laughter.)
7 But we appreciate it.
8 MR. DURAN: Let me come back and answer
9 it.
10 (Laughter.)
11 I'll answer the question.
12 No, I understand. I'm posing this as the
13 kind of dilemma that states face, and the bottom line
14 issue here for me in making this remark is that I
15 think we have to consider that state education policy
16 and its impact on the utility of information that is
17 provided back on the performance of ELL students is
18 extremely important.
19 And states may have different answers, and
20 we need to consider what the strengths and limitations
21 are of different strategies and issues that arise
22 within states because of the ways in which they can
23 vary within states in terms of interpreting their own
24 reclassification criteria.
25 Thank you.
472 NEAL R. GROSS
476 96
1 MS. SKANDERA: If you have a comment, feel
2 free to come on down.
3 MS. RASMUSSEN: We're not doing the
4 numbers thing?
5 MS. SKANDERA: We've gone through the
6 numbers.
7 (Laughter.)
8 We are now come freely. There's a
9 little button down at the bottom of the mic that
10 you'll want to
11 MS. RASMUSSEN: Okay. I'm Mari Rasmussen,
12 Title III, State Education Agency, and Director in
13 North Dakota. You may be surprised why there's not a
14 flurry of Title III directors coming up to comment,
15 and maybe there will be after this. Maybe I'm
16 breaking ground. I don't know, I can only speak for
17 me.
18 I'm just so excited about this, and it was
19 just wonderful to sit and listen to the panel, and I
20 am excited about the ELL framework of what can come
21 out. I think we will just I don't want to say
22 we'll be happy with everything anything that will
23 come up, because, of course, we have our ideas. But
24 we definitely do want guidance. We want the best for
25 the English language learners, and we need help in
477 NEAL R. GROSS
481 97
1 figuring that out.
2 We maybe have ideas; we don't know how to
3 put them into action. We do know we want
4 accountability. We also want flexibility. Obviously,
5 you know North Dakota is a low incident state, though
6 our numbers are growing more than you may think they
7 are growing, but we are low incident, so we want
8 flexibility.
9 But more important, we do want
10 accountability. We do want answers. We realize there
11 aren't easy answers for all of this. I learned that
12 even among all these experts there is differences of
13 opinion, and maybe some of that could be taken to this
14 ELL framework in that we would have the flexibility
15 and accountability maybe in terms of our own technical
16 advisory committees that are composed of experts to
17 assure that some of these things happen, but yet it
18 isn't specifically spelled out, yet we would be
19 required to bring experts together.
20 Maybe this will initiate some discussion
21 from some of the rest of you. Anyway, I commend you
22 for doing this. I am excited for what will come out,
23 and I think it will benefit us all.
24 MS. SKANDERA: Thank you.
25 MS. WARING: I will follow Mari. My name
482 NEAL R. GROSS
486 98
1 is Sarah Waring, and I'm from South Dakota. And I
2 would just like to also comment on the flexibility,
3 and especially looking at Section 3122 of the Redbook.
4 And especially looking at the section, I have asked on
5 a number of occasions regarding the achievement
6 objectives and some of the waivers that are available.
7 One of the districts that we have in South
8 Dakota is largely impacted by children from refugee
9 and wartorn countries, and I heard the panelists
10 speaking about students that are impacted by SIFE, but
11 we also have some impact of some students who are
12 secondary students who are illiterate in their native
13 language, and have come from these refugee areas and
14 these wartorn countries.
15 And I just think we need to look at the
16 flexibility and the trauma that testing students, even
17 in a oneyear time period, to get that baseline, what
18 is the value of that? But looking at that flexibility
19 as well.
20 MS. SKANDERA: Thank you. And for
21 clarification, the Redbook is Title III.
22 (Laughter.)
23 MS. HOUSTON: So I guess she was the
24 groundbreaker. I'm Lorena Houston from CTB, McGraw
25 Hill, a test publisher. And we have been working on
487 NEAL R. GROSS
491 99
1 language proficiency assessments for quite a while,
2 analyzing and studying indepth all of the language
3 proficiency or language development standards.
4 And as you probably know, some states have
5 more flexibility than others in terms of rewriting
6 their standards if, after studying them, looking at
7 the results of the alignment studies and all of that,
8 of changing them or not.
9 But one of the interesting things that
10 perhaps you can help states encourage in
11 encouraging them is in looking at the ways to to
12 analyze those ELD standards in a manner that they can
13 identify what can be accessible in a standardized
14 test, as well as identifying what in those standards
15 are not really standards, but are the strategies for
16 teaching English language learners.
17 We know that standards documents are very
18 important, and, as some of our panelists were saying,
19 they need to be like documents that can help at the
20 classroom level. That's what we want. We all want to
21 provide tools for teachers to be able to help our
22 English language learners. But the standards of such,
23 or documents, are a mixture of standards, the
24 strategies for learning or teaching, and all of that.
25 So teasing those things out is going to be
492 NEAL R. GROSS
496 100
1 very important for states to not only to provide
2 you with what they states individually believe is a
3 reasonable alignment to their state's tests, but also
4 to identify what are the true standards versus
5 teaching strategies that can help in the classroom.
6 DR. MEDINA: Good afternoon. I'm Barbara
7 Medina from Colorado, and I want to concur with my
8 colleague Supreet from Maryland that we're very
9 confused.
10 (Laughter.)
11 I'd really like to address my comments to
12 question number 4. As you know, Colorado is a
13 destination state, which means we have had an
14 incredible increase over the last 10 years of 241
15 percent of English language learners. Our language
16 represented in the state are now 277.
17 One of the things that is problematic for
18 us is that I oversee a district that may have two
19 students who are ELL and another district that may
20 have 20,000 students who are ELL. There is great
21 variability and complexity in these issues, and I
22 would implore that we consider the issue of
23 flexibility.
24 I also receive 40 districts that prior to
25 last year had no English language learnings. So at
497 NEAL R. GROSS
501 101
1 the state and at the practical level, our experience
2 has been that the issue of particularly the issue
3 of accommodations, and I think the panelists and I
4 want to thank the panelists for sharing their
5 expertise the whole issue around accommodation and
6 how much language proficiency a student needs to have
7 in order for those accommodations to really be
8 effective.
9 And then, the corollary issue is we simply
10 don't have enough resources to build the capacity and
11 the expertise in the area of measurement. It is just
12 not there. As you could tell from the comments today
13 and the panelists, you have measurement experts, and
14 then you have ELL experts.
15 And between the two of them we have begun
16 to build somewhat of I think some conversation,
17 meaningful conversations I guess would be the word,
18 but we don't have the capacity in professional
19 development to provide teachers with the training to
20 be more effective at the use of accommodations.
21 And for us to be accountable in a state
22 where we've had the kind of growth that we have over
23 the time that we've had to implement them, it just
24 I would just implore for that flexibility of the
25 states that are making good faith effort to meet
502 NEAL R. GROSS
506 102
1 requirements but don't have the technical resources to
2 get there.
3 MR. DURAN: I made a promise to someone
4 that I need to fulfill that is a very important one.
5 I think we have
6 MS. SKANDERA: Could you reintroduce
7 yourself?
8 MR. DURAN: My name is Richard Duran from
9 U.C.Santa Barbara. I made a promise to deliver a
10 comment that is of special interest to a number of
11 states, and that is the possibility that the
12 Department might support the development of non
13 English language academic achievement tests where
14 states might be able to form consortia in the
15 development of tests, working with test publishers and
16 among themselves to be able to create better grounded
17 achievement assessments tied to their standards, with
18 the dilemma that states have different statements of
19 their achievement standards within a content area, but
20 they also have limited resources to be able to invest
21 in the development of nonEnglish language achievement
22 tests pursuant to both Title I and Title III in terms
23 of the objectives of meeting AYP goals.
24 So in I think that there are some
25 possibilities here that need to be thought through.
507 NEAL R. GROSS
511 103
1 It may be I think it would be prudent for the
2 Department to invest some resources in convening
3 people who do want to form these consortia, because
4 states are ready to explore these alternatives, but
5 because of the burden that is involved in the
6 development of these achievement tests it creates a
7 lot on them. So thank you.
8 MS. SKANDERA: So, obviously, we continue
9 to welcome comments, and, if you're hesitating, don't.
10 But I also want to mention that we will be here until
11 5:00.
12 (Laughter.)
13 That is our commitment. You all have had
14 a long day, and certainly should feel free and the
15 liberty to engage or have another conversation outside
16 those doors.
17 (Laughter.)
18 MS. SAAVEDRA: Lisa Saavedra, Florida
19 Department of Education. Three points, possibly some,
20 if not all, have been brought up not only concerns or
21 issues by the panelists, but, as we proceed forward in
22 terms of this framework, things that we would like
23 considered or addressed.
24 Guidance or establishment on the quality
25 standards that need to be in other words, we need
512 NEAL R. GROSS
516 104
1 to be established for reliability and validity
2 processes, because obviously in light of the
3 assessments that have been developed we can't use the
4 same reliability and validity processes that were used
5 for the English language arts assessments.
6 The next point would be looking and I
7 think a couple of the panelists mentioned this
8 Florida similar to Midwestern states, Florida has
9 well over 286 different languages. So an examination
10 of the reliability and validity standards, when you
11 have single language versus multiple language groups.
12 And the last is looking at definitions of
13 instructional or curricular validity as it relates to
14 ELP assessments. How does that alignment work for ELP
15 assessments that are aligned to ELP standards that are
16 aligned to the content standards? We don't feel that
17 there is good definition or good models out there, so
18 guidance on the definitions of instructional or
19 curricular validity.
20 MS. SKANDERA: Once again, I will just
21 encourage you, if you're hesitating, come on down.
22 Otherwise, you're not a hostage here.
23 (Laughter.)
24 You can feel free to participate or not
25 from here forward.
517 NEAL R. GROSS
521 105
1 (No response.)
2 (Whereupon, at 3:44 p.m., the proceedings in the
3 foregoing matter were concluded.)
10
11
12
13
14
15
16
17
18
19
20
21
22
23
522 NEAL R. GROSS

Description: Tags: Transcript-07262007

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Description: Tags: Transcript-07262007

Загружено:

Авторское право:

Доступные форматы

1 1

8 to piggyback on another meeting that we felt like

11 in your travels, etcetera, knowing that you probably

20 And so for those of you from states, I

21 just want to say I know this is hopefully a long­

25 I'd like to go over a few kind of

7 Secretary for Title III. We also have, from the

8 Office of Intergovernmental and Interagency Affairs,

9 Gloria Frederick and Janet Mills, who have been

10 outside. You've probably met them registering,

12 We also have interpreters here, Stacy

13 Tingle, and I believe Amanda is interpreting right

15 interpretive services today? So we certainly have

16 them available, but, if not, maybe we'll have them

20 We also have our Court Reporter, Eric,

23 purpose of today ­­ it's our third public meeting ­­

25 which we'll have the opportunity to do in about an

1 hour and a half, or written, to contribute to the

2 development of a framework for high­quality ELP

4 Our hope and our goal is at the

5 partnership meeting that we anticipate this fall to

6 provide you with a draft of that, and then have an

10 Looking ahead as far as just details

12 I just want to mention that the LEP partnership, as

14 at, what are those things that we at the Department

19 in the LEP partnership. One is Ellen Forte

20 representing CCSSO. So she is here today. Peter

21 Zamora representing MALDEF. I don't believe he's

23 Melissa Lazaine with LaRaza, Jack Levy with the

24 National Center on English Language Acquisition, and

25 then we've got Robert Linquanti here today with us

1 from the Comprehensive Center on Assessments and

3 Four kind of procedural ­­ moving forward

5 mentioned we will have 90 minutes, and we have an

8 really anticipate your thoughts and feedback on the

9 four primary questions that were published in the

10 Federal Register, and also over here on this board

15 break. I'll come back up and just talk about the

17 your back pocket, if you have not signed up already

19 to go out here at the break and sign up, you'll

21 if that is something you came today to do make sure

23 So after the 90 minutes, we'll take a

3 MR. LINQUANTI: Thank you very much,

6 MR. LINQUANTI: Okay. Great. As Hanna

7 just mentioned, this is the last of three scheduled

8 hearings, and for the purposes of consistency and

12 Senior Researcher at WestED representing the

14 one of the federally funded centers. The AACC has

15 worked very closely with the Department of Education

16 and many states in supporting the LEP partnership

17 initiative. So on behalf of the AACC and its

20 We recognize that building the LEP

22 that need to be addressed. So in addition to the

24 today, we have convened a group of experts to share

3 experience in the research and implementation of

7 and development agencies, as well as private testing

11 Register ­­ there are four of them, as Hanna

12 indicated, and they are over here on my left ­­ four

14 Department of Ed, have prepared to facilitate a

15 discussion that will help frame what successful

17 focus discussion on key technical issues and

19 So over the next 90 minutes I will be

21 just want to say I know this is hopefully a long

23 purpose of today it's our third public meeting

2 development of a framework for highquality ELP

3 Four kind of procedural moving forward

11 Register there are four of them, as Hanna

12 indicated, and they are over here on my left four

17 your area or areas of expertise for example,

10 for Ballard and Tye Publishers, and I'm my

1 of CaliforniaLos Angeles, and faculty associate

5 the Gevirtz Graduate School of Education, U.C.Santa

5 The first question and, again, it's

13 their ELP standards" and here is the phrase that

14 was missing "promote effective intervention"

9 sometimes the lines get blurred. And it's at the

11 students are learning in the content areas the

16 and there has been a lot of research over the

23 academic language which is content to access the