See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/6682500

Validity and Reliability of the Classroom Participation Questionnaire With

Deaf and Hard of Hearing Students in Public Schools

Article  in  Journal of Deaf Studies and Deaf Education · February 2007

DOI: 10.1093/deafed/enl028 · Source: PubMed

CITATIONS READS
38 1,256

3 authors, including:

Shirin Antia Darrell L. Sabers

The University of Arizona The University of Arizona
41 PUBLICATIONS   977 CITATIONS    69 PUBLICATIONS   788 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Center on Literacy and Deafness (CLAD). View project

All content following this page was uploaded by Shirin Antia on 20 May 2014.

The user has requested enhancement of the downloaded file.

 Journal of Deaf Studies and Deaf Education Advance Access published November 17, 2006
Validity and Reliability of the Classroom Participation
Questionnaire With Deaf and Hard of Hearing
Students in Public Schools
Shirin D. Antia
Darrell L. Sabers
University of Arizona
Michael S. Stinson
Rochester Institute of Technology
The Classroom Participation Questionnaire (CPQ) was ad-
ministered to 136 deaf or hard of hearing (D/HH) students
attending general education classrooms in Grades 4–10. The
CPQ is a student-rated measure that yields scores for Un-
derstanding Teachers, Understanding Students, Positive
Affect, and Negative Affect. Validity and reliability of a long
(28-item) and a short (16-item) form are reported. We pro-
vide evidence of (a) internal structure validity through an
examination of the relationships between the subscales and
an analysis of interitem reliability within each scale, (b) re-
liability over time by examining the scores of students over
a 3-year period, and (c) external structure validity through
an examination of the relationships of the CPQ with meas-
ures of teacher-rated academic competence and Stanford
achievement scores. The results suggest that both the long
and short form of the CPQ can be used to assess participa-
tion of D/HH students in general education classrooms.
Today, the majority of students who are deaf or hard of
hearing (D/HH) and receiving special education serv-
ices are educated in public schools; of these, a large
number attend general education classrooms for some
or all of their academic subjects. Data from the Annual
Survey of D/HH Children and Youth (Karchmer &
Mitchell, 2003) show that during the 2000–2001
school year, 75% of all D/HH students were reported
to attend public schools and 44% spent some propor-
This research was supported by the U.S. Department of Education,
Office of Special Education Programs, Field Initiated Research
(H324C010142). Correspondence should be sent to Shirin D. Antia,
Department of Special Education, Rehabilitation and School Psychology,
College of Education, University of Arizona, Tucson, AZ 85721 (e-mail:
santia@u.arizona.edu).

The Author 2006. Published by Oxford University Press. All rights reserved.
For Permissions, please email: journals.permissions@oxfordjournals.org
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
tion of the school day in general education classrooms.
These numbers are likely to be underestimates of the
true numbers of D/HH students in general education
classrooms because school districts that serve only
a few D/HH students may not respond to the annual
survey. A major concern expressed by researchers re-
garding D/HH students in general education class-
rooms is their ability to actively and fully participate
in classroom instruction and discussion because of
their communication difficulties (Garrison, Long, &
Stinson, 1994; Saur, Layne, Hurley, & Opton, 1986).
For all students, the ability to communicate with
teachers and peers can be a major component of aca-
demic success as teacher–student communication and
student–student communication are the primary means
of learning in classrooms. Students who have difficulty
communicating may choose not to participate in class-
room activities; nonparticipation can adversely affect
their learning and eventual academic success (Long,
Stinson, & Braeges, 1991). Conversely, students who
think they understand the communication of teachers
and classroom peers are likely to be academically en-
gaged because they have a sense of control over the
learning outcome; they are also likely to believe that
they have a good chance of succeeding academically.
Several researchers have documented the specific
communication difficulties of D/HH students in
classrooms with hearing students. Saur et al. (1986)
observed D/HH students in college classrooms over
three academic quarters. They reported that a major
doi:10.1093/deafed/enl028
2 Journal of Deaf Studies and Deaf Education
barrier to classroom participation was the rate of class-
room presentation and discussion. Rapid rates resulted
in a time lag in sign language interpreting; conse-
quently, D/HH students were not able to respond to
instructor questions in a timely manner or responded
inappropriately (Saur et al., 1986). Similar results
were obtained by Stinson, Liu, Saur, and Long (1996)
who interviewed 50 D/HH college students at
the National Technical Institute for the Deaf. They
divided students into two groups by their preferred
mode of communication (oral only or oral and/or
sign). They reported that, regardless of the preferred
mode, all students perceived classroom communica-
tion as a challenge. For the oral students, difficulties
with teachers included their lack of clarity of speech;
students who depended in part or wholly on sign
language interpreters expressed frustration keeping
up with the flow of classroom communication. Both
groups also indicated that communication with hearing
students was difficult, especially during class discus-
sions, because of rapid turn taking, lack of topic coher-
ence, and frequency of interruptions.
Classrooms in which D/HH students find it diffi-
cult to participate can lead to ‘‘an island of deafness . . .
where hearing-impaired students appear passive and
unresponsive’’ (Saur et al., 1986, p. 327). Such pas-
sivity can negatively impact academic achievement
(Braeges, Stinson, & Long, 1993; Long et al., 1991).
Long et al. (1991) examined the relationship between
communication ease, academic engagement, and aca-
demic achievement of D/HH high school students at
a school for the deaf. These researchers found signif-
icant positive correlations between student self-rated
communication ease, student-rated academic engage-
ment, and Stanford achievement scores in language,
mathematics, and science. Furthermore, student-rated
communication ease was a significant predictor of ac-
ademic achievement. In a related study, Braeges et al.
(1993) found that, for high school D/HH students,
teacher-rated communication ease and teacher-rated aca-
demic engagement were significantly related. Teacher-
rated academic engagement was also a significant predic-
tor of academic achievement (as measured by the
Stanford Achievement Test).
The previous studies were conducted with college-
age students in mainstream college classes or high
school students at schools for the deaf. Some limited
and indirect data are available about participation
difficulties of school-age D/HH students in general
education classrooms. Clearly, classroom participation
for students who use sign language is dependent on an
interpreter. Sign language interpreters in public schools
may not be highly skilled, especially in rural school dis-
tricts (Antia & Kreimeyer, 2001). Schick, Williams, and
Bolster (1999) report that 56% of interpreters in Colo-
rado had less than the minimal skills needed for
interpreting in classrooms. A more recent study that
included a national sample of 2,091 educational inter-
preters (Schick, Williams, & Kupermintz, 2006)
showed that only 38% met the minimum standards set
by the state. Thus, many interpreters may not be able to
adequately interpret teacher talk. Moreover, any sign
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
language interpreter will have difficulty interpreting
all the communication in the classroom and may have
difficulty deciding what classroom communication to
interpret (Schick et al., 1999). Consequently, inter-
preters may omit peer responses to teacher questions,
peer questions, and other communication that would
contribute to the student’s understanding of classroom
content (Shaw & Jamieson, 1997). Schick et al. (2006)
found that areas of particular difficulty for interpreters
included discourse mapping, accurately representing
key vocabulary, and accurately indicating the speaker.
These aspects of interpreting are crucial for students
not only to understand teacher lectures but also to be
able to understand the content of, and participate ap-
propriately in, classroom discussion. Another area of
interpreting that might affect the student’s ability to
contribute to classroom discussion is sign-to-voice in-
terpreting, as the student’s comments that are inaccu-
rately interpreted may be perceived by peers and
teachers as irrelevant or off the point. Schick et al.
(2006) found that Registry of Interpreters for the Deaf
certified interpreters were quite variable in their
sign-to-voice interpreting scores on the Educational
Interpreters Performance Assessment and that inter-
preters who took the elementary version of the test
scored significantly less well in sign-to-voice interpret-
ing than those who took the secondary version of the
examination. Thus, it is possible that some students
in general education classrooms may have interpreters
who have difficulty interpreting their comments.

As is true for college students, school-age students
who use oral communication may find it easier to join
classroom discussions than students who use sign
communication (Stinson et al., 1996). However, unlike
their hearing peers, they may have to visually locate
the speaker during a classroom discussion and, conse-
quently, miss much of what the speaker says. Average
classroom noise levels may make it impossible for
D/HH students to understand the communication of
peers, and they may also have difficulty comprehend-
ing other students when several are talking at the same
time (Preisler, Tvingstedt, & Ahlstrom, 2005). The
degree of hearing loss, classroom acoustics, and the
use of group and individual amplification all may affect
these students’ access to classroom communication and
classroom participation.
Because many D/HH students spend a proportion
of their day in general education classrooms, it is im-
portant to obtain some measure of their classroom
participation. One method is to obtain the student’s
self-perceptions of participation. Such information can
assist teachers of D/HH students to work appro-
priately with general education teachers and hearing
students, as well as help the D/HH students develop
strategies to enhance their own participation.
Measuring Classroom Communication
To measure classroom communication, Long and
colleagues at the National Technical Institute for the
Deaf developed a rating scale named the Perceived
Communication Ease Questionnaire (Long et al.,
1991) and a subsequent version, the Classroom Com-
munication Ease Scale (Garrison et al., 1994). These
authors conceptualize communication ease as having
both a cognitive and an affective component. The
cognitive component includes the student’s self-
perception of the amount and quality of information
received and expressed in the classroom. The affective
component includes individual subjective responses
to the communicative situation. These subjective re-
sponses may be positive feelings, such as feeling good,
relaxed, or comfortable, or negative feelings, such as
being frustrated, nervous, or upset. The questionnaire
statements are written so students can respond to
them regardless of the mode of communication. The
Communication Participation 3
questionnaire used by Long et al. (1991) consisted of
28 statements rated on a four-point scale and yielded
ratings for four subscales. The two cognitive subscales
are Understanding Teachers (UT) and Understanding
Students (US); the two affective subscales are Positive
Affect (PA) and Negative Affect (NA). This version of
the scale was validated with students in Grades 7–10 at
a school for the deaf (Long et al., 1991). An adapted,
revised, but unpublished, version of the scale was used
with deaf college students (Stinson et al., 1996).
Given the large numbers of D/HH students in
public elementary and middle schools, it is important
to know whether the scale can be used with students
in elementary and middle school as well as with high
school students. Consequently, we modified the admin-
istration of the most recent scale so that it could also
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
be used with younger students. The modified scale
is renamed the Classroom Participation Questionnaire
(CPQ). The purpose of this article is to present reliabil-
ity and validity data on the CPQ when given to school-
age students in public schools and to provide a short
16-item version that would be of considerable practical
use to teachers. We envision that the CPQ can be used
for two purposes: one is to obtain information about an
individual student’s participation in the general edu-
cation classroom and the second would be to measure
group performance on a pre- and post-evaluation.
Determining Validity of Educational Measurements
Anastasi and Urbina (1997) suggest that ‘‘almost any
information gathered in the process of developing
or using a test is relevant to its validity’’ (p. 138). For
purposes of analysis, we use the categories of relevant
validity information suggested in Nitko (2001) that
pertain to the CPQ. Nitko, like Anastasi and Urbina,
treats interitem reliability as a separate aspect of
the test, although it is a precursor of validity. Because
validity information (as well as reliability) must be
interpreted as being relevant to the sample of students
from which the data are obtained (Sechrest, 2005), we
do not generalize any of this information to any other
population of students and certainly not to the CPQ
as an instrument. The following is a brief summary of
the relevant categories of validity evidence for the
CPQ derived from Nitko.
4 Journal of Deaf Studies and Deaf Education
• Content representativeness and relevance is the
evaluation of whether the tasks are representative of
the domain being assessed. In the case of the CPQ , an
examination of the items can show whether communi-
cation difficulties in the classroom are being addressed.
• Internal structure evidence is the relationship
among the parts of the assessment. In the case of the
CPQ , an examination of the relationships between the
different scales provides this evidence. Internal struc-
ture evidence is also provided by evidence of interitem
reliability.
• Reliability over time provides information about
the stability of the measured construct. We are able to
present data about the relationships of CPQ scores
obtained for the same students over a 3-year period.
• External structure evidence is the evaluation of
whether the assessment procedure predicts current or
future performance on external criteria. We expect the
students’ perception of their classroom participation
to be influenced by their degree of hearing loss and
their use of interpreters. Because the CPQ is meant to
measure classroom participation, which in turn is
expected to influence academic achievement, the rela-
tionship of the CPQ scores to external measures of
academic achievement provides additional evidence
of external structure validity. Consequently, we exam-
ine the relationships between classroom participation
as measured by the CPQ , degree of hearing loss, in-
terpreter use, and students’ academic outcomes as
measured by the Academic Competence Subscale of
the Social Skills Rating System (SSRS; Gresham &
Elliott, 1990) and the reading, language, and mathe-
matics subtests from the Stanford Achievement Test
(9th edition).
• Practicality evidence is the evaluation of the cost
and efficiency of obtaining the assessment information.
We are able to provide some data on how many stu-
dents were unable to take the CPQ.
Methods
The data for this article were collected as part of a
5-year longitudinal study of academic and social prog-
ress of D/HH students in public schools. Students
were eligible to participate if they (a) had an identified
bilateral or unilateral hearing loss, (b) did not have
additional severe cognitive disabilities, (c) received
direct or consultative services from teachers of D/HH
or had an Individual Education Plan, (d) attended gen-
eral education classrooms in public schools for 2 or
more hours each day, and (e) were in Grades 2–8 at
the beginning of the study. Students were recruited
from Arizona and Colorado through state agencies
and school districts. Requests to allow students to par-
ticipate in the study were sent to parents of all eligible
students. Initially, permission for participation was ob-
tained for 187 students. Data were obtained annually for
all participating students on a battery of teacher and
student questionnaires that assessed academic perfor-
mance and social behavior. One of the instruments used
was the CPQ. The CPQ and academic data presented
here were collected from the students in Year 3 (2003–
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
2004) of the 5-year study. Students responded to the
CPQ in the fall semester while the academic data for this
report were obtained in the spring of the same academic
year. To examine reliability over time, we additionally
used the CPQ data obtained with the same students in
Years 1 and 2 of the study.
Participants
Data on the CPQ were obtained on 136 students active
in the study during Year 3. The students not partici-
pating that year were missing for various reasons,
primarily because they had moved out of the school
districts and we were unable to locate them. Data on
student characteristics shown in Table 1 were obtained
from each student’s teacher of D/HH who completed
a demographic form on the student annually. Unfor-
tunately, we were not always able to obtain complete
data on each student despite repeated contacts with
teachers. Data that were unavailable to (or from) teach-
ers are recorded as missing. Students who received
only monitoring or consultation services from the
teacher of D/HH were most likely to have missing data.
Table 1 shows that the sample of students is quite
evenly distributed across grades except for the youn-
gest and oldest students. Almost equal numbers of
males and females are included. Hearing loss is pre-
sented as pure tone average (PTA) in the better ear.
A majority of these students had mild and moderate
hearing loss as would be expected in public school
 Communication Participation 5

Table 1 Student characteristics in school; most of these students used a sign language
Number Percent interpreter in the classroom.
Total number of children 136
Grade Instruments
4 5 4
5 26 19 Classroom Participation Questionnaire. The CPQ
6 22 16 consists of 28 statements that a student rates on
7 22 16 a four-point scale (1, almost never; 2, seldom; 3, often;
8 18 13
4, almost always). The questionnaire yields four
9 29 21
10 13 10 subscale scores: Understanding Teachers (UT) (eight
Missing information 1 1 statements), Understanding Students (US) (five
Gender statements), Positive Affect (PA) (six statements), and
Male 70 52 Negative Affect (NA) (nine statements). To respond to
Female 66 49 the teachers who suggested that the CPQ was too long
Average level of hearing loss (unaided)a for annual administration, we developed a 16-item
Unilateral loss 18 13 questionnaire with four items in each subscale. The

Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013

0–20 dB (high frequency) 4 3 28-items with the selected 16-items for the short version
21–40 dB (mild) 37 27
are provided in Appendix A. As mentioned earlier, UT
41–60 dB (moderate) 26 19
61–90 dB (severe) 26 19 and US were envisioned as cognitive components,
91–1001 dB (profound) 16 12 whereas PA and NA were envisioned as affective
Missing information 9 7 components.
Student’s home language To develop the 16-item scale as part of this study, we
English 107 79 requested M. Stinson, one of the authors of the original
Spanish 14 10 Perceived Communication Ease Questionnaire and the
ASL 4 3
Classroom Communication Ease Scale, to select four
Other 4 3
Missing information 7 5 items for each subscale. The intent was to best represent
the constructs envisioned by the original authors. In this
Mode of communication in school
Spoken 105 77
paper we present the validity evidence for the 28-item
Spoken and signed 18 13 (long) and 16-item (short) forms.
Sign only 10 7 The modifications made to the CPQ to adapt its use
Missing information 3 2 for younger students were primarily in presentation and
Interpreter use administration. The test was formatted to be in reason-
No 107 79 ably large print, and care was taken that items were not
Yes 26 19 crowded together on a page. Students took the test in-
Missing information 3 2
dividually (i.e., not in a group situation as with the
Hours/day in regular classroom
original test). They could take the test in more than
1–,3 13 10
one sitting as necessary. They could read the items in-
3–5 36 26
51 78 57 dependently or have the items read or signed to them by
Missing information 9 7 their itinerant or resource teacher of D/HH. These
ASL, American Sign Language. teachers could give explanations of words that were
a
Average of 500, 1,000, and 2,000 Hz in better ear. not understood by the student but were instructed to
cease administering the questionnaire if the student
needed explanation of more than two or three words.
classrooms (Karchmer & Mitchell, 2003), and a sub-
stantial percentage of students had a unilateral hearing Academic Competence Scale of the SSRS. The Aca-
loss. Only about 20% of students used sign language demic Competence Scale is one of three scales of the
6 Journal of Deaf Studies and Deaf Education
SSRS (Gresham & Elliott, 1990). The other two scales
yield information on the student’s social skills and
problem behaviors. The Academic Competence Scale
is a nine-item scale completed by teachers. Teachers
rate students on a five-point scale, placing each
student in the lowest 10%, the next lowest 20%,
middle 40%, next highest 20%, or highest 10% on
reading and mathematics compared to classmates and
grade expectations. They also rate the students on
motivation, intellectual functioning, classroom behav-
ior, and parental encouragement. The Academic Com-
petence Scale yields a standard score with a mean
of 100 and an SD of 15. The SSRS was normed on
a national sample of 2,109 students in Grades 3–12.
Seventeen percent of the students in that sample were
identified as having a disability. Coefficient alpha re-
liability for the Academic Competence Scale was .95
and test–retest reliability over a 4-week period was .93
in the standardization sample.
Stanford Achievement Test—9th edition. The Stan-
ford Achievement Test (Harcourt Brace Educa-
tional Measurement, 1997) is a standardized, norm-
referenced assessment including reading, mathematics,
and language that was administered to Arizona students
in Grades 2–9 each spring as required for the state of
Arizona accountability purposes. Colorado students did
not take the Stanford and are therefore not included in
analyses that included this test. The Stanford reading
subtest measures reading vocabulary and comprehen-
sion; the mathematics subtest measures problem solving
and mathematics procedures; and the language subtest
includes prewriting, composing, and editing. The Stan-
ford Achievement Test was normed on approximately
250,000 students from 1,000 school districts across the
United States. Students with disabilities or limited
English proficiency who received instruction within a
regular education classroom composed 5.5% of the
norming sample. The Gallaudet Research Institute con-
ducts a norming study and publishes normative data on
each version of the Stanford with D/HH students.
However, because the sample for this study included
primarily students in general education classrooms,
we have used the norms for the general population,
not the D/HH population. The norms for D/HH stu-
dents do not provide a good comparison for our sample
because they only provide percentile ranks and include
large numbers of children who have taken out-of-level
tests. Therefore, these norms do not necessarily provide
comparisons to children at the same age or grade level.
These norms also do not provide comparisons for the
children with minimal hearing loss who are in our sam-
ple. The scores reported on the Stanford are normal
curve equivalents; these are standard scores with a mean
of 50 and an SD of 21.06.
Procedure
Each student’s itinerant or resource teacher of D/HH
administered the CPQ in the fall semester during the
regular time to meet with the student. Teachers gave
the CPQ using the modifications described. A space
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
was provided for teacher notes, so that teachers could
inform us if the student’s responses seemed invalid for
any reason.
All students completed the ratings in reference to
their general education classrooms. Middle and high
school students were asked to think of their language
arts or social studies classes. If none of these were
appropriate, they were asked to choose another class
including both lecture and class discussion. Students
who used interpreters were instructed to answer the
questions about communication with other students
and with teachers taking into account communication
through the interpreter.
The SSRS Academic Competence Scale was com-
pleted by each student’s general education teacher in
the spring semester. For elementary students the class-
room teacher completed the scale, whereas for second-
ary students the scale was completed by the language
arts teacher with input from other teachers as needed.
Data were obtained on 130 of the 136 students for
both the SSRS Academic Competence Scale and the
CPQ.
The Stanford achievement scores on reading,
mathematics, and language were obtained from stu-
dent records. These tests were administered once a year
(during the spring) only for students in Arizona
(the students in Colorado did not take the Stanford
Achievement Test). There were no makeup days;
therefore, scores were not available for students who
were absent on the testing days. Data were available
 Communication Participation 7

for 69 students for the reading subtest, 66 for mathe- Table 2 Intercorrelations between subscales with
matics, and 70 for language. 28-item form correlations above the diagonal and 16-item
form correlations below the diagonal (N ¼ 136)
Subscale 1 2 3 4
Results 1. UT __ .68 .57 .28
2. US .64 __ .72 .44
The following section provides the data for each cat- 3. PA .57 .70 __ .54
egory of validity: content representativeness, internal 4. NAa .31 .44 .45 __
a
structure, external structure, reliability over time, and Reverse scoring.

practicality. p , .001 for all correlations.

Content Representativeness variables correlated (see Nunnally & Bernstein, 1994,

The best evidence of content representativeness is the for the formula and the explanation of the approach).
list of the items in Appendix A. The potential user of The correlation between two perfectly reliable meas-
the CPQ should examine this set of items to determine ures would best describe the relationship between the
two constructs being compared. Table 3 presents the

Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013

whether the items are appropriate for measurement of
the construct of interest. A further examination of the corrected-for-attenuation correlations between pairs of
content is included later when we discuss the deter- subscales for the 28-item and the 16-item forms.
mination of total scores. The data in Tables 2 and 3 address the issue of how
the tasks help define the construct of classroom par-
ticipation. Because of some differences in the reliabil-
Internal Structure Evidence: Results and
ity coefficients described above, the data in Table 3 are
Interpretation
the better indicators of task intercorrelations of the
We present first the relationships between the four parts of the construct. As shown in Table 3, the cor-
subscales of the CPQ (UT, US, PA, and NA). These relations among the UT, US, and PA subscales are
relationships help determine whether the components substantially higher than the correlations involving
of the CPQ work together so that each component NA. The highest relationship occurs between US
contributes positively toward assessing the relevant and PA, although US was intended to measure a cog-
construct. We then present data for interitem consis- nitive component and PA was intended to measure an
tency. For the NA scale, the scoring is reversed so affective component of communication. Thus, for our
that for all scales high scores are desirable. data, the cognitive and affective aspects of the CPQ
are not well differentiated.
Subscale correlations. Table 2 presents the correla- The type of reasoning used in the previous para-
tions between pairs of subscales. Because the correla- graph involves an examination of what are termed
tion obtained between two subscales depends on the
reliability of each subscale, it is easier to examine the
correlations if they are corrected for attenuation Table 3 Corrected-for-attenuation intercorrelations
between subscales with 28-item form correlations above
(Nunnally & Bernstein, 1994). The correction for at-
the diagonal and 16-item form correlations below the
tenuation is intended to estimate the correlation that diagonal (N ¼ 136)
would be obtained if each subscale were perfectly Subscale 1 2 3 4
reliable, and thus, the corrected correlations are an
1. UT __ .80 .64 .32
attempt to directly infer the degree to which the sub- 2. US .80 __ .64 .52
scales measure the same construct. The correction for 3. PA .71 .89 __ .61
attenuation used to adjust the correlations in Table 3 is 4. NAa .38 .55 .56 __
applied by dividing an obtained correlation by the a
Reverse scoring.
square root of the product of the reliabilities of the two p , .001 for all correlations.
8 Journal of Deaf Studies and Deaf Education

convergent and discriminant aspects of validity. The Table 4 Coefficient alpha reliability estimates, means,
examination of convergence asks whether the task cor- and SDs for subscales and UT/US/PA composite
(N ¼ 136)
relates highly with those traits that it should resemble;
the examination of discriminant validity asks whether Long form Short form
a
the task correlates lower with traits that it should not K Alpha Mean (SD) K Alpha Mean (SD)
resemble. The NA subscale was originally intended to UT 8 .89 3.3 (0.56) 4 .82 3.3 (0.60)
have more convergent validity with PA and more dis- US 5 .81 3.1 (0.61) 4 .78 3.1 (0.64)
PA 6 .88 3.2 (0.65) 4 .79 3.2 (0.64)
criminant validity with respect to UT and US. In this
NA 9 .88 3.2 (0.57) 4 .82 3.5 (0.60)
respect, NA is not correlated high enough with PA UT/US/PA 19 .93 3.2 (0.53) 16 .90 3.2 (0.54)
but does have the desired lower correlations with UT composite
and US. UT and US correlate well enough with each a
K ¼ number of statements.
other, but the US with PA correlation is problematic
because discriminant validity was expected for that rela- (1991) who reported interitem consistency reliabilities
tionship by the original authors (Garrison et al., 1994). of .77 for UT, .73 for US, .63 for PA, and .80 for NA.
A possible resolution to the lack of convergent/ Means and SDs for the subscales for the 28-item
discriminant validity of the subscales is to avoid a cog- and 16-item forms are similar. Mean subscale scores

Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013

nitive versus affective interpretation of the data. Thus,
it is not appropriate to report averaged scores for cog-
nitive (UT, US) and affective (PA, NA) communica-
tion or to report a total score. However, each subscale
can be considered a separate construct. Given the high
correlations between UT, US, and PA, an additional
solution is to have a summative score for the three
convergent subscales (UT, US, and PA) and use the
NA subscale as an additional measure. Therefore, the
following tables will include reliability and validity
data on the composite, UT/US/PA, as well as the
individual subscales.
Interitem consistency. Interitem consistency (Anastasi &
Urbina, 1997) measures the degree to which the items
in a set measure the same construct, and we used
coefficient alpha (Cronbach, 2004) for this assess-
ment. Interitem consistency reliability was examined
separately for the UT/US/PA composite and each
subscale of the 28-item and 16-item forms. Table 4
presents these data obtained from the total sample of
136 students. Reliability for each subscale of the 28-
item scale is higher than the reliability for the corre-
sponding subscale of the 16-item scale, as is expected
due to the increased number of statements. The re-
liability coefficients in Table 4 for the 28-item test
range from .81 to .89 and on the 16-item test from
.78 to .82. The reliabilities for the subscales for both
the 28-item and 16-item scales of the CPQ from this
sample are higher than those reported by Long et al.
are calculated by summing individual item ratings and
dividing by the number of items in the subscale; so the
number of statements does not change the magnitude
of the scores. Note that for the NA scale the scoring is
reversed to make scoring comparable for all scales.
Reliability Over Time
For measuring reliability over time (stability), we used
the test–retest method over 1- and 2-year intervals for
those students who had data available for those years.
These correlations are presented in Table 5. The sta-
bility estimates presented in Table 5 are, as expected,
considerably lower than the intercorrelations between
subscales reported in Tables 2 and 3, probably because
of the changing situations occurring as the students
moved into different classrooms with different class-
mates and teachers each year. We were unable to give
the CPQ to students more than once a year to get
a better measure of test–retest reliability under more
consistent conditions.
External Structure Evidence
Relationships of the CPQ to other meaningful varia-
bles provide external structure evidence of validity.
This category of evidence allows the reader to examine
whether the tasks exhibit meaningful relationships
with variables that might be used to better understand
the construct measured by the CPQ. To examine the
 Communication Participation 9

Table 5 Stability estimates (test–retest correlations) for Table 6 shows the correlations of the CPQ sub-
subscales across changing conditions scales and the UT/US/PA composite with the SSRS
Years 2–3 Years 1–2 Years 1–3 Academic Competence Scale and the Stanford read-
Student N 111 121 116 ing, mathematics, and language scores; better ear PTA;
Long form subscales and interpreter use. Student ratings of classroom par-
UT .43* .49* .36* ticipation and teacher ratings of academic competence
US .49* .39* .35*
and Stanford scores are positively related (p , .01),
PA .49* .36* .31**
NA .45* .34* .29** with the exception of the Stanford mathematics on the
UT/US/PA composite .52* .46* .37* 16-item form. A less severe hearing loss, as measured
Short form subscales by better ear PTA and not having an interpreter is
UT .37* .42* .35* associated with UT and US but not with PA ratings.
US .44* .35* .32* Degree of hearing loss and not having an interpreter is
PA .49* .36* .31** associated with NA ratings only on the 16-item form.
NA .40* .22** .20**
The composite UT/US/PA is significantly related to
UT/US/PA composite .52* .45* .38*
all the external academic measures.
*p , .001.
To determine whether the CPQ can explain vari-

Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013

**p , .01.
external structure evidence, we correlated each of
the four subscales and the composite of UT/US/PA
with teacher ratings of the SSRS Academic Com-
petence Scale and with the Stanford test scores.
We also correlated the CPQ scores with two demo-
graphic variables that we expected to be related to
the student’s perceptions of classroom participation:
better ear PTA and whether the student used an
interpreter.
ance in academic achievement beyond explanations
based on demographic variables such as degree of
hearing loss and use of interpreters, we analyzed the
data using sequential multiple regression. Two analy-
ses were conducted: In the first analysis, interpreter
use was entered as an independent variable in Step 1,
better ear PTA and interpreter use were entered as
independent variables in Step 2, and finally the CPQ
composite score (UT/US/PA) and NA were entered
additionally in Step 3. In the second analysis, PTA
was entered as an independent variable in Step 1;

Table 6 Correlations of 28-item and 16-item CPQ with academic achievement and student characteristics
UT US PA NAa UT/US/PA
28-item
Academic competence (N ¼ 132) .28** .32** .21** .27** .30**
Stanford reading (N ¼ 69) .31** .48** .36** .17 .44**
Stanford mathematics (N ¼ 66) .22* .30** .26* .18 .29**
Stanford language (N ¼ 70) .30** .43** .41** .15 .43**
Better ear PTA (N ¼ 127)
.23**
.22** .00
.08
.16*
Use of interpreter (N ¼ 133) .21** .15* .08 .10 .16*
16-item
Academic competence .26** .31** .23** .32** .31**
Stanford reading .22* .46** .33** .23* .39**
Stanford mathematics .15 .30** .24* .22* .27*
Stanford language .20* .43** .40** .19 .40**
Better ear PTA
.19*
.20* .01
.20*
.14
Use of interpreter .17* .12 .06 .19* .14
a
Reverse scoring.
*p , .05 (one tailed).
**p , .01 (one tailed).
10 Journal of Deaf Studies and Deaf Education

Table 7 Predictions of academic achievement by hearing loss, interpreter use, and CPQ scores for 28-item and
16-item scales
Total R2 (Step 1 Total R2
Dependent variable interpreter use) Incremental R2 (Step 1 PTA) Incremental R2
Academic competence (N ¼ 119) Step 1a .05* .01
Step 2b .06* .01 .06* .05*
28-item Step 3c .13** .07** .13** .07**
16-item Step 3c .15** .09** .15** .09**
Stanford reading (N ¼ 60) Step 1 .16** .06
Step 2 .16** .00 .16** .10**
28-item Step 3 .32** .15** .32** .15**
16-item Step 3 .30** .13** .30** .13**
Stanford mathematics (N ¼ 56) Step 1 .09* .00
Step 2 .09 .00 .09 .09
28-item Step 3 .21* .12* .21* .12*
16-item Step 3 .20* .11* .20* .11*
Stanford language (N ¼ 60) Step 1 .12** .02
Step 2 .13* .01 .13* .11**

Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013

28-item Step 3 .30** .18** .30** .18**
16-item Step 3 .28** .16** .28** .16**
a
Step 1 includes interpreter use in columns 1 and 2; better ear PTA in columns 3 and 4.
b
Step 2 includes interpreter use and better ear PTA.
c
Step 3 includes interpreter use, better ear PTA, and CPQ scores for either the 28-item or 16-item scale as indicated.
*p , .05.
**p , .01.

the remaining analysis was the same. The results for
the 28-item and 16-item scales are shown separately in
Table 7. The first two columns show the results when
interpreter use is entered first; the last two columns
show the results when PTA is entered first. In both
pairs of columns, CPQ scores are entered as a third
step. These data show that PTA alone does not appear
to predict much variance in academic outcomes for
this sample of students or to add to the prediction
based on interpreter use. Interpreter use and PTA
together significantly predict academic outcomes,
while the prediction is improved further when the
CPQ scores are added. The complete model (i.e., bet-
ter ear PTA, interpreter use, and 28-item CPQ scores)
explains 13% of variance for Academic Competence
scores, 32% of the variance for reading, 21% of the
variance for mathematics, and 30% of the variance for
language scores. The CPQ scores explained the most
additional variance (18%) for language scores. The
16-item scale explains less variance than the 28-item
scale for each outcome except Academic Competence
scores.
Practicality
The CPQ is fairly easy for most students to complete.
Only two students between Grades 4 and 10 were
reported not to be able to take the test because of their
inability to comprehend the items when either read
or signed to them. Teacher comments that the 28-item
scale was too long provided the rationale for develop-
ing the 16-item scale.
Discussion
The purpose of this article is to provide evidence for
using the CPQ to assess the participation of D/HH
students in general education classrooms. The CPQ
has previously been used only with high school and
college students; the data presented here provide evi-
dence that the instrument can be used with students in
elementary and middle school classrooms as well. Ad-
ditionally, the CPQ had previously been used with
high school students at a center school; our data in-
dicate that elementary and middle school students can

reliably rate their own participation in general educa-
tion classrooms. Our sample also provides evidence
that the CPQ can be used with hard of hearing stu-
dents, as our sample, unlike previous samples, includes
a high percentage of students with unilateral, mild,
and moderate hearing losses. Because classroom par-
ticipation is a concern for D/HH students in these
classrooms and can impact student learning, teachers
can use the results of the CPQ in addition to other
assessments to assist in decision making about place-
ment, interpreter use, and other issues. Our findings
suggest that both the long (28-item) and short (16-
item) forms can be useful to teachers or researchers,
although they may be used for different purposes. We
recommend using the long form when individual
decisions are to be made (e.g., changes in classroom
placement for a student). However, teachers might
be able to use the short form if time is a factor, for
example, when teachers have to read the scale to the
students. The short form would also be of interest to
researchers who are interested in examining primarily
group data. Some possible uses for researchers might
be to measure change after specific interventions to
increase communication participation, to examine stu-
dent’s perceptions of differences in communication in
different educational settings, or to compare student’s
self-perception with teacher’s perceptions of student
communication.
Our data indicate that items on the CPQ do not
measure separately the cognitive and affective compo-
nents of communication in classrooms as envisioned
by the authors of the original scale. The results show
a high correlation between UT, US, and PA, whereas
NA does not correlate highly with any of the other
subscales, perhaps because responding to the NA neg-
atively criterial statements is different than responding
to positively criterial statements in the other three
subscales. We therefore recommend using a composite
score (UT/US/PA) and a separate score for NA. One
of the reasons for the high correlations between US
and PA may be because both subscales include several
items about group discussion.
In this study, NA does not correlate significantly
with any of the Stanford scores in the 28-item scale
but it does correlate significantly with reading and
mathematics in the 16-item scale. The correlations
Communication Participation 11
reported here are lower than those reported by Long
et al. (1991) who reported significant correlations of
.42 for mathematics and .29 for language. Differences
in the characteristics and school experiences of the two
sets of participants may have contributed to the dif-
ferent results. The students in the Long et al. study
were enrolled in a school for the deaf, in contrast to
this study where the students were in public schools.
Despite our results, we do not at this time recommend
deleting the NA scale. Instead, we recommend that
teachers use it diagnostically and consider interven-
tions for students who complete ratings of often or
almost always on the NA statements.
The CPQ scores are significantly, though modestly,
correlated with academic achievement. The correla-
tion between communication participation and student
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
achievement lends support to the idea that students
who participate and who feel positively about their
participation are more likely to do well academically,
although, of course, a cause-and-effect relationship
cannot be established. It is interesting that each of
the CPQ subscale scores is positively and significantly
correlated to teacher-rated academic competence.
Braeges et al. (1993) also reported significant positive
correlations between teacher-rated academic engage-
ment and student-rated communication ease for stu-
dents in Grades 7–10 attending a school for the deaf.
Their data are not exactly comparable with ours as
they used a total score of all four subscales and re-
ported correlations of .48 for mathematics and .32 for
language; our data (on the 28-item scale) show a cor-
relation of .29 for mathematics and .43 for language
with the composite score of UT/US/PA. Of course,
the samples in the two studies are quite different; their
sample consisted of deaf high school students attend-
ing a center school; our sample can be characterized as
mostly hard of hearing students in general education
classes. Also, our sample of students are likely to have
a much wider range of academic achievement than
those in their study. Our data, therefore, extend their
findings and show a relationship between teacher-
rated academic competence and student-rated class-
room communication for younger students in general
education classrooms. It should be remembered that
the teachers who rated academic competence were
the students’ general education teachers, whereas the
12 Journal of Deaf Studies and Deaf Education
teachers of D/HH administered the CPQ to the stu-
dent. The general education teachers were not aware
of the students’ CPQ ratings. Thus, the general edu-
cation teachers’ ratings were independent of the stu-
dents’ ratings.
We had expected that degree of hearing loss and
interpreter use would be significantly correlated with
CPQ scores. Our data show that they are moderately
(though significantly) correlated with UT and US but
not PA or NA (on the 28-item scale). Our data also
show that CPQ scores are reasonably good predictors
of academic achievement when added to demographic
predictors such as degree of hearing loss and inter-
preter use. Degree of hearing loss alone added very
little to the prediction of academic achievement. In-
terpreter use predicted achievement in reading, mathe-
matics, and language, as well as teacher-rated academic
competence. Whereas hearing loss and interpreter use
together accounted for between 6% and 16% of vari-
ance on academic outcomes, adding the CPQ scores
allowed us to account for between 13% and 32% of
variance on academic outcomes. Although these predic-
tions are significant, they are still modest, and clearly
several other factors, not measured here, affect out-
comes. However, our data make clear that an assessment
of students’ perceptions of classroom participation is
valuable and that assumptions about the ability or ease
of student participation in the classroom should not be
based solely on student demographic variables.
An examination of Table 6 shows that US con-
sistently has the highest correlations with all the
academic outcome measures. The relationship between
US and academic achievement points to the impor-
tance of understanding peers as part of classroom
learning. As noted by Stinson et al. (1996), college-
age D/HH students indicated that communication
with peers was a challenge. Clearly, the same challenge
might exist for school-age D/HH students.
Although the interitem reliabilities for both the
28- and 16-item scales are respectable, they are prob-
ably not high enough to make important decisions
about individuals, but important decisions should
not be made based on a single test. The coefficients
are high enough for most group decisions. Reynolds,
Livingston, and Willson (2005) suggest that judg-
ments about reliability should be made in the context
of decisions being made based on test results, and
reliabilities of .80 are acceptable in many testing
situations. The interitem correlations for the 16-item
form provide evidence that the items chosen are more
reliable than a random sample of items would be.
Although the correlations between years are sig-
nificant, they are quite modest. We expected lower
correlations across years because students encounter
different teachers and must adjust to a new environment
each year. The higher correlations across years for the
composite (UT/US/PA) compared to the individual
subscales are expected because of the increased number
of items when three subscales are combined. However,
the expectation that additional items result in higher
correlations was not evident in the comparison of the
stability of the 16-item subscales or composite com-
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
pared with the 28-item subscales. Again, the similarity
of the stability estimates for these two forms is evidence
that the items chosen for the short form are more stable
than a random sample of items would be.
Teachers can use the individual scales of the
CPQ diagnostically to alert them to the student’s prob-
lems in classroom communication that may result in
academic difficulties. The results of the CPQ might
help them to determine when and how to intervene.
For example, if students rate themselves low in
understanding their peers, the teacher of D/HH
could observe the classroom to identify specific group
discussion practices that might be modified. The
teacher of D/HH might also provide information to
classroom teachers and hearing classmates to increase
their sensitivity about classroom communication
difficulties that the D/HH student might have. The
teacher could use specific CPQ responses to engage
with the student in developing self-advocacy strategies
such as requesting clarification during group discus-
sions or requesting a quiet space for group projects.
The modifications made to the questionnaire
administration allowed students who were not able to
read fluently to complete the self-rating. Teachers
did not report that students had difficulty under-
standing the questionnaire items. However, we did
not obtain data on how many students read the items
independently. Future research on the use of the CPQ
should examine carefully the effects of test-taking
modifications.

In conclusion, the CPQ may be a useful tool to
examine participation in general education classrooms
for elementary and middle school students as well
as high school students. The instrument could be
used diagnostically by teachers to intervene in difficult
classroom communication situations where necessary.
However, we do not advocate using the CPQ alone to
make decisions about educational placement.
Appendix A
CPQ Items Arranged by Subscale
Understanding Teachers
My teacher understands me
*I understand my teacher
I have enough time to answer the teachers’ questions
I understand the homework assignments my teacher
gives me
I understand when my teacher tells me what to study
for a test
*I understand my teacher when she gives me home-
work assignments
*I understand my teacher when she answers other
students’ questions
*I understand my teacher when she tells me what to
study for a test
Understanding Students
The other students in class understand me
*I understand the other students in class
*I join in class discussions
*I understand other students during group discus-
sions
*I understand other students when they answer my
teacher’s questions
Positive Affect
*I feel good about how I communicate in class
I feel relaxed when I talk to other students
*I feel relaxed when I talk to my teacher
I feel relaxed in group discussions
*I feel happy in group discussions in class
*I feel good in group discussions in class
Communication Participation 13
Negative Affect
I feel lonely because I cannot understand other stu-
dents
*I feel frustrated because it is difficult for me to com-
municate with other students
*I get upset because other students cannot understand
me
*I get upset because my teacher cannot understand me
I feel nervous when I talk to other students
I feel nervous when I talk to my teacher
I feel nervous in group discussions in class
I feel frustrated in group discussions in class
*I feel unhappy in group discussions in class
*Items included in the 16-item short scale.
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013
References
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.).
Upper Saddle River, NJ: Prentice-Hall.
Antia, S. D., & Kreimeyer, K. H. (2001). The role of inter-
preters in inclusive classrooms. American Annals of the Deaf,
146, 355–365.
Braeges, J., Stinson, M. S., & Long, G. (1993). Teachers’ and
deaf students’ perceptions of communication ease and en-
gagement. Rehabilitation Psychology, 38, 235–247.
Cronbach, L. J. (2004). My current thoughts on coefficient
alpha and successor procedures. Educational and Psycholog-
ical Measurement, 64, 391–418.
Garrison, W., Long, G., & Stinson, M. S. (1994). The classroom
communication ease scale. American Annals of the Deaf, 139,
132–140.
Gresham, F. M., & Elliott, S. N. (1990). Social skills rating
system. Circle Pines, MN: American Guidance Service.
Harcourt Brace Educational Measurement. (1997). Stanford
achievement test series, ninth edition: Spring multilevel norms
book. San Antonio, TX: Author.
Karchmer, M., & Mitchell, R. E. (2003). Demographic and
achievement characteristics of deaf and hard-of-hearing
students. In M. Marschark & P. E. Spencer (Eds.), Oxford
handbook of deaf studies, language and education (pp. 21–37).
New York: Oxford University Press.
Long, G., Stinson, M. S., & Braeges, J. (1991). Students’ per-
ception of communication ease and engagement: How they
relate to academic success. American Annals of the Deaf, 136,
414–421.
Nitko, A. J. (2001). Educational assessment of students (3rd ed.).
Upper Saddle River, NJ: Merrill Prentice Hall.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory
(3rd ed.). New York: McGraw-Hill.
Preisler, G., Tvingstedt, A., & Ahlstrom, M. (2005). Interviews
with deaf children about their experiences using cochlear
implants. American Annals of the Deaf, 150, 260–267.
 14 Journal of Deaf Studies and Deaf Education
Reynolds, C. R., Livingston, R. B., & Willson, V. (2005). Mea-
surement and assessment in education. Boston: Pearson.
Saur, R. E., Layne, C. A., Hurley, E. A., & Opton, K. (1986).
Dimensions of mainstreaming. American Annals of the Deaf,
131, 325–330.
Schick, B., Williams, K., & Bolster, L. (1999). Skill levels of
educational interpreters working in public schools. Journal
of Deaf Studies and Deaf Education, 4, 144–155.
Schick, B., Williams, K., & Kupermintz, H. (2006). Look whose
being left behind: Educational interpreters and access to
education for deaf and hard of hearing students. Journal
of Deaf Studies and Deaf Education, 11, 3–20.
View publication stats
Sechrest, L. (2005). Validity of measures is no simple matter.
Health Services Research, 40, 1584–1604.
Shaw, J., & Jamieson, J. (1997). Patterns of classroom discourse
in an integrated, interpreted elementary school setting.
American Annals of the Deaf, 142, 40–47.
Stinson, M. S., Liu, Y., Saur, R. E., & Long, G. (1996). Deaf
college students’ perceptions of communication in main-
stream classes. Journal of Deaf Studies and Deaf Education,
1, 40–51.
Received January 20, 2006; revisions received September 11,
2006; accepted October 24, 2006.
Downloaded from http://jdsde.oxfordjournals.org/ by guest on June 3, 2013