Академический Документы
Профессиональный Документы
Культура Документы
qxd
26/2/08
4:27 PM
Page 273
Book reviews
Two new textbooks for introductory courses on language testing differentially clarify basic principles of language assessment for relatively
similar audiences. Both H. Douglas Browns Language Assessment:
Principles and Classroom Practices and James Dean Browns Testing
in Language Programs: A Comprehensive Guide to English Language
Assessment give excellent and concise summaries of the processes
involved in test development, but they differ in that H. D. Browns
book views testing from the standpoint of the teacher in the language
classroom, providing, in his own words, a clear and reader-friendly
(p. ix) overview of the foundations of second language (L2) testing
and specifically avoiding instruction on statistics related to testing
which could be viewed as allowing the course instructor more freedom to teach test statistics in the manner and with supplements he or
she chooses. In contrast, J. D. Browns book engages testing primarily from the viewpoint of the program administrator, giving a stepby-step overview of statistical applications and procedures for
testing that, in his own words, provide[s] a balance between the
technical and practical aspects of language testing that is neither too
complex nor too simplistic (p. x).
The contrast between the two books highlights the fact that many
introductory, graduate-level language testing classes often include two
different types of students, those from MA or undergraduate TESOL
programs who plan to teach and/or be program administrators and
2008 SAGE Publications (Los Angeles, London, New Delhi and Singapore)
DOI: 10.1177/0265532207086784
086784_LTJ_271-286.qxd
274
26/2/08
4:27 PM
Page 274
Book reviews
086784_LTJ_271-286.qxd
26/2/08
4:27 PM
Page 275
Book reviews
275
086784_LTJ_271-286.qxd
276
26/2/08
4:27 PM
Page 276
Book reviews
086784_LTJ_271-286.qxd
26/2/08
4:27 PM
Page 277
Book reviews
277
craft of designing items. These chapters can be seen as the heart of the
book, and instructors will want to spend considerable time reviewing
the types of items presented and guiding students through item writing practice sessions. Readers will find the example items quite helpful, especially the multiple picture-description and picture-cue tasks,
which can be used as guides in finding or designing other picturebased prompts for oral or written test tasks at multiple levels, and
which are very beneficial as a testing task for more visual learners.
Chapter 10 is an example of H. D. Browns commitment to treating standardized testing issues and alternatives in assessment equally
and objectively. Here he explains the history of alternative assessment. This history involves a discussion on how standardized tests
have shortcomings and that other options (portfolios, journals, observations, self-assessments, peer-assessments) hold more ethical
potential. The chapter goes on to explain the pros and cons of various types of alternatives in assessment with helpful comparisons
between large-scale standardized tests and portfolios, journals, and
conferences. Another useful topic in this chapter is a discussion of
performance-based assessment. H. D. Brown continues with benefits
and guidelines for using portfolios, journals, conferences/interviews,
observations, and self-/peer-assessments. There is also a nice comparison table (Table 10.1, p. 278) of all the above assessment forms
at the end of the chapter that rates each form of assessment according to the principles of practicality, reliability, face validity, content
validity, washback, and authenticity.
Chapter 11 is enlightening for its frank discussion of the impact
grades have on students lives and for its treatment and attention to
the topic of assigning grades, which is often a central issue of concern
for new teachers. There are sections referring to guidelines for selecting grading criteria, a discussion of absolute versus relative grading,
teachers perceptions of appropriate grade distributions, institutional
expectations and constraints, the question of difficulty from a crosscultural perspective, and the eternal question: what do letter grades
mean? This chapter is a useful reference for new teachers as they
design their first syllabus and form their grading philosophies.
Another excellent feature of chapter 11 is its reference to the
issues covered by the National Center for Fair and Open Testing, also
known as Fairtest, a non-profit watchdog of large-scale, standardized
tests in the USA. Knowledge of the goals of this organization and
knowing how to interpret and use the information stemming from it
will be of particular benefit to US-based MA TESOL students long
after they graduate. On the other hand, since Fairtest is a US-based
086784_LTJ_271-286.qxd
278
26/2/08
4:27 PM
Page 278
Book reviews
organization, an international audience may find this and other references US-centric rather than international in scope. For example,
the Test of English as a Foreign Language (TOEFL) is described and
discussed in detail, but the International English Language Testing
System (IELTS) is merely mentioned in passing.
J. D. Browns book Testing in Language Programs, which is a
revised edition of his 1996 volume, views testing from two perspectives: for making program-level decisions (norm-referenced testing),
and for making classroom-level decisions (criterion-referenced testing). The main goal of the book is to prepare teachers and administrators with the tools they need to critically evaluate L2 tests
reliability, validity, and dependability from several theoretical and
statistical vantage points. Part of this goal is met by instructing readers to conduct robust statistical procedures using Microsoft Excel.
The book spends less time on discussing test construction and test
items and more time on how to evaluate tests and test items that
already exist. The book also focuses on how to use test data to investigate the soundness of a test. There are 11 chapters in this book, and
they can be divided into four areas, as described below.
The first two chapters provide an overview of testing from practical and theoretical perspectives. Furthermore, J. D. Brown delves
into the differences between discrete-point and integrative testing,
prompting the reader to consider which type of testing he or she
prefers. In contrast, H. D. Brown presents these two types of tests
from a theoretical standpoint, emphasizing the arguments for and
against discrete-point and integrative testing in light of the unitary
trait hypothesis, which argues that discrete points of language can
not be separated from each other during language production.
The next two chapters review item development issues from qualitative and quantitative perspectives. Chapter 3 answers the question
of what makes a good item (from a qualitative perspective) and
chapter 4 answers the question of how to analyze items (from a
quantitative perspective). Chapter 3 is highly effective in that it presents principles of sound and valid item construction through a series
of questions; for example J. D. Brown asks, when item writing, have
race, gender, and nationality bias been avoided? (p. 46). He then
explains what this means and how we can avoid such bias in items.
Scoring procedures are also discussed, including how to design and
use rating scales. Chapter 4 begins what is an in-depth yet user-friendly
approach to teaching and learning test statistics. J. D. Brown should
be commended for his attention to detail in this section and those that
follow. A person unversed in mathematics beyond a high-school
086784_LTJ_271-286.qxd
26/2/08
4:27 PM
Page 279
Book reviews
279
level, yet armed with a spreadsheet program such as Excel, can work
through J. D. Browns book and teach him or herself the basics in
test statistics, such as (in chapter 4) calculating item facility (IF) and
item discrimination (ID), calculating an items difference index
(with pre- and posttesting) (DI) or calculating an items B-index.
J. D. Brown takes pains to provide screen shots of an Excel spreadsheet
showing how data can be organized for these statistical analyses, and
describes in detail each and every step, from how to open Excel and
how to enter data, to how to find and use formulas in Excel to automatically calculate IF and ID, so that readers cannot get lost and, at
the end of the chapter, can calculate the statistics explained and
understand what they mean.
Chapters 5 and 6 discuss test results and explain descriptive statistics associated with the interpretation of test results. In addition,
J. D. Brown refers to large-scale tests, such as the SAT, GRE and
TOEFL to help explain standardized scores, thus giving theoretical
points a very concrete reference point.
Chapters 7, 8, 9, and 10 respectively outline the books strongest
point in a review of statistical procedures for demonstrating a tests
(a) correlation with other variables, (b) reliability, (c) dependability,
and (d) validity. These four chapters are the core of the book in that each
chapter explains not only important test theories and their constructs,
but also ways to quantitatively measure these important constructs.
J. D. Brown spends considerable time addressing the different test statistics not just from a mathematical perspective (all formulas are
presented in nicely contained boxes), but also from theoretical and
practical perspectives. Again, as in chapter 4, the statistics themselves
are taught to the reader through clear examples, screen shots of actual
data sheets, and step-by-step instructions on how to do the calculations
using formulas in Excel. J. D. Brown also teaches readers how to use
Excels Chart Wizard, and shows which charts best present data from
which types of statistical analyses. For example, in chapter 7, J. D.
Brown (a) explains what a Pearson productmoment correlation coefficient theoretically can do, (b) explains what to do with missing data,
(c) provides the formula for calculating the Pearson productmoment
correlation coefficient with a key to all mathematical symbols used in
the formula, and (d) provides a screen shot of what data should look
like in Excel prior to calculation. Assumptions of the coefficient are
discussed, diverse scatterplots showing positive and negative correlation coefficients are explained, and finally, following J. D. Browns
detailed directions, readers use sample data in Excel to calculate the
Pearson correlation coefficient and create a scatterplot.
086784_LTJ_271-286.qxd
280
26/2/08
4:27 PM
Page 280
Book reviews
086784_LTJ_271-286.qxd
26/2/08
4:27 PM
Page 281
Book reviews
281
References
ACTFL. (2006). Standards for foreign language learning in the 21st century
(3rd ed.). Lawrence, KS: Allen Press.
Bachman, L. F. (2000). Modern language testing at the turn of the century:
Assuring that what we count counts. Language Testing, 17(1), 142.
Davies, A. (1997). Demands of being professional in language testing.
Language Testing, 14, 328339.
086784_LTJ_271-286.qxd
282
26/2/08
4:27 PM
Page 282
Book reviews