Learning and Instruction 19 (2009) 322e334

www.elsevier.com/locate/learninstruc

Instructional effects on critical thinking: Performance on ill-defined issues

Charoula Angeli*, Nicos Valanides
Department of Education, University of Cyprus, 11-13 Dramas street, Nicosia CY-1678, Cyprus
Received 13 January 2008; revised 30 May 2008; accepted 16 June 2008

Abstract
Undergraduate students in dyads (N 72) were randomly and equally assigned to four groups, namely three teaching groups (General,
Infusion, and Immersion) and the control group. Students were initially administered the California Critical Thinking Skills Test (CCTST). After
instruction, each dyads critical-thinking performance on an ill-defined problem was tested. A one-way ANCOVA, with the mean CCTST score
of each dyad as covariate, indicated that the covariate and the teaching method were significant. Post hoc comparisons showed that the Infusion
and the Immersion groups outperformed only the control group. Other quantitative and qualitative analyses revealed that students assigned to the
different teaching groups exhibited diverse understandings of critical thinking.
2008 Elsevier Ltd. All rights reserved.
Keywords: Critical thinking; Ill-defined problems; Problem solving; Teaching approaches

1. Introduction

1.1. The concept of critical thinking

Critical-thinking skills are necessary for active citizenship in

any pluralistic and democratic society, where citizens are daily
confronted with tremendous amounts of information and illdefined problems with real uncertainty as to how they can be
best solved. Consequently, critical thinking should not be just
another educational option, but rather an indispensable part of
general education (Education Commission of the States, 2007;
Ennis, 1989, 1992; Paul, 1995). Creating instructional environments for critical thinking raises a number of issues. For
example, is it more appropriate to invest a significant portion of
educational resources on teaching general critical-thinking
skills in isolation of subject matter or to contextualize instruction through a knowledge base?
The aim of the present study was the examination of the
extent to which the method of teaching general criticalthinking skills differentially affects (a) learners criticalthinking performance on an ill-defined issue, and (b) learners
self-reported understandings of critical thinking.

Critical thinking has mainly been conceptualized in terms of

dispositions and skills (Ennis, 1989; McPeck, 1990a; Paul,
1995). Nonetheless, researchers in the literature have also
acknowledged the existence of other perspectives that conceptualize critical thinking beyond the dispositions and skills
paradigm (Alston, 2001; Bailin, 2002; Papastephanou &
Angeli, 2007; Peters, 2005; Ten Dam & Volman, 2004).
Critical-thinking dispositions include truth-seeking, openmindedness, systematicity, analycity, maturity, inquisitiveness,
and self-confidence (Facione & Facione, 1994). Most
researchers also agree that dispositions toward critical thinking
play an important role in critical-thinking performance
(Facione, Facione, & Sanchez, 1994; McBride, Xiang, & Wittenburg, 2002; Perkins, Tishman, Ritchhart, Donis, & Andrade,
2000), but research attention has mostly focused on the concept
of critical thinking as a set of thinking skills and how to develop
appropriate instructional processes for promoting them (Bailin,
Case, Coombs, & Daniels, 1999).
Critical-thinking skills include taking into consideration
multiple perspectives, examining implications and consequences, resolving disagreements with reason and evidence,
and re-evaluating a point of view in light of new information

* Corresponding author. Tel.: 35 799478030; fax: 35 722377950.

E-mail address: cangeli@ucy.ac.cy (C. Angeli).
0959-4752/$ - see front matter 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.learninstruc.2008.06.010

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

(Paul, 1995). The present study focuses on the conceptualization and teaching of critical thinking as a set of thinking
skills. According to Ennis (1989), critical thinking starts as
a problem-solving process in a context of interacting with the
world and other people. Then, it continues as a reasoning
process informed by background knowledge and previously
acceptable conclusions, and it results in drawing a number of
inferences through induction, deduction, and value judging.
Lastly, according to Ennis (1989), the critical-thinking process
ends in a decision about what to believe or do.
Similarly, Paul (1995) defined critical thinking as a set of
integrated macro-logical skills, and McPeck (1990a, 1990b)
characterized critical thinking as the ability to suspend judgment, or temporarily suspend judgment, until sufficient
evidence is accumulated to establish the validity of a proposition or action. The Delphi Report on critical thinking (Facione,
1990a), which was a product of a Delphi research project
involving 46 experts in thinking, described critical thinking as
a purposeful thinking process. During this process, a person
forms a judgment of what to believe or do in a given context,
and, in doing so, he or she uses a set of cognitive skills, such
as, analysis, interpretation, inference, explanation, evaluation,
and self-regulation (Facione, 1990a).
In recent years, researchers have continued to mostly direct
their efforts toward thinking skills and how to find effective
ways for teaching them (Anderson, Howe, Soden, Halliday, &
Low, 2001; Frijters, Ten Dam, & Rijlaarsdam, 2008; Halpern,
2003; Kalman, 2002; Kuhn, 1999; Pithers & Soden, 2000;
Tsui, 1999). The focal point of the most well-known debate
relating to the concept of critical thinking concentrates on
whether critical thinking should be viewed as generic across
different disciplines (Ennis, 1989; Paul, 1995) or disciplinespecific (McPeck, 1981). Ennis (1989, 1992) argued that
knowing a field of study, understanding what constitutes
a good reason in that field, and having a grasp of the fields
semantic concepts do not sufficiently produce critical thinking.
Instead, he concluded that general strategies are applicable to
many subjects, and argued that critical thinking is best taught
with a content-free approach. Along the same line of
reasoning, Paul (1995: 372) also insisted that, the logics we
use, and which we are daily constructing and reconstructing,
are far more mutable, less discrete, more general, more opentextured and multi-textured, more social, more dialectical, and
even more personaldand hence far less susceptible to
domain-specific skills and concepts. We need to base our
model of the critical thinker, not on the domain-bound individual with subject-specific skills, but on the disciplined
generalist.
On the contrary, McPeck (1981: 19) argued against the
notion of critical thinking as general skills bound and stated
that, thinking, critical thinking, is always about some particular thing or subject (let us call this thing X), and that it
therefore makes little or no sense to say, I teach thinking simpliciter, or I teach in general, but not about anything in
particular. According to McPeck (1981), teaching general
skills in order to enhance someones critical thinking is useless,
since there is no such thing as a general critical-thinking ability.

323

Therefore, if one is an effective critical thinker in one area, it

does not mean that he or she will be an effective thinker in all
other areas, because the skills and knowledge required for one
area are different from the skills and knowledge required for
another.
During the last years, this debate has faded away, since
most researchers commonly accept the notion that some
general critical-thinking skills, which are widely applicable in
various contexts, exist, and that familiarity with a knowledge
base also plays an important role in thinking (Tsui, 1999; Voss,
Perkins, & Segal, 1991). Thus, research focuses primarily on
how to develop effective instructional designs for teaching
general critical-thinking skills.
1.2. Critical-thinking skills and instructional approaches
Most studies on the teaching of critical thinking fall in one
of two categories, that is, the General approach with a focus on
teaching general critical-thinking skills separately from
subject matter, and the Infusion approach that calls for the
teaching of general critical-thinking skills embedded in
subject matter. An example of the General approach to
thinking skills is the work of Whimbey and Lochhead (1986).
According to this approach, thinking is a skill of know-how
that invokes general-purpose heuristics. Consequently, what
one needs to become an effective thinker is a set of general
heuristics that are likely to be effective in a variety of problem
situations, along with the meta-knowledge about situations in
which specific heuristics are more appropriate. The Whimbey
and Lochhead (1986) approach advocates for students to be
working in pairs and thinking aloud, while they are trying to
solve problems. This way, students may learn something by
listening to what other people say and by exposing ones own
thought processes.
In the 1990s, the method of teaching critical thinking has
shifted from the General approach to the Infusion approach.
This shift was based on the assumption that, through infusion,
transfer to other domains can become possible (Prawat, 1991).
For example, proponents of the Infusion approach (Brown,
1997; Halpern, 1997; Kennedy, Fisher, & Ennis, 1991) argued
that critical thinking must be taught using authentic ill-defined
problems for which critical thinking is indeed needed. Other
researchers have examined the effects of specific instructional
tactics on critical thinking, such as, the role of cognitive
dissonance (Browne & Freeman, 2000; Tsui, 1999), studentled seminars and role play (Dennick & Exley, 1998), dialogue
(Frijters et al., 2008), academic controversy and cooperative
learning (Johnson & Johnson, 1993; Kaplan & Kies, 1994;
Pithers & Soden, 2000). Garside (1996) also examined the
extent to which group discussion can facilitate the development of critical-thinking skills more than traditional forms of
instruction, such as lectures, and found no significant differences between the two methods. In addition, Tynjalas (1998)
research findings show that learning in a constructivist environment with an emphasis on examining different perspectives, free expression of ideas, and analysis from the
perspective of students own experiences can enhance critical

324

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

thinking more easily than a traditional learning environment

with an emphasis on grades. Recently, Frijters et al. (2008)
found that within the domain of biology, a dialogic lesson
series, as compared to a non-dialogic, resulted in higher scores
on critical-thinking skills that were not gained at the expense
of subject-matter knowledge of biology.
In the 1990s, researchers also proposed a third approach to
teaching critical thinking, namely, the Immersion approach
(Prawat, 1991). In the Immersion approach, the most important
resource or tool in promoting thought is the role of ideasdnot
thinking skills. Concomitantly, critical-thinking skills are not
made explicit during teaching with the Immersion approach. In
the Immersion approach, students are involved in dialogue
where they are prompted to consider, analyze, and evaluate
different points of view.
Obviously, there are many arguments both for and against
each one of the three approaches. For example, an advantage
of the General approach is that it does not penalize students
who do not have prior subject-matter knowledge. At the same
time, the approach does not guarantee that the generic skills of
critical thinking will necessarily be transferred to different
content areas or out-of-school tasks (Resnick, 1987). The
Infusion approach integrates the teaching of explicitly stated
general critical-thinking skills with the teaching of a specific
content domain and maintains an equal balance between the
two. Advocates of the Infusion approach (Collins, Brown, &
Newman, 1989) have different views about the most appropriate starting point for teaching critical-thinking skills. Some
argue that skills should be taught first, followed by a direct
application of the skills in a content domain, while others
argue that skills should be taught when the need arises (Prawat, 1991). Those who were skeptical about the Infusion
approach expressed a concern about diverting attention from
important curricular issues, and this concern about the
importance assigned to skills versus subject matter gave rise to
the Immersion approach. Ennis (1992), however, expressed
concern about the Immersion approach arguing that if general
critical-thinking skills are not made explicit to students, then
students will not be able to draw generalizations, and thus
critical-thinking skills might not transfer from one domain to
another.
1.3. The present study
No research to our knowledge has set out to empirically
compare the effects of the General, the Infusion, and the
Immersion approaches on teaching critical-thinking skills. As
Ennis (1992) suggested more than a decade ago, and as Ten
Dam and Volman (2004) recently stated, the literature on
critical thinking constitutes a rich theoretical body of knowledge about the conceptualization of critical thinking, but the
field is still missing a systematic body of research on the
effects of complete and detailed instructional designs about
the teaching of critical-thinking skills.
Most efforts related to the teaching of general cognitive
skills have been implemented in the form of courses isolated
from subject matter (Nickerson, Perkins, & Smith, 1985), with

the exception of a few studies that successfully managed to

teach general thinking skills in contextualized ways (e.g.,
Perkins, 1986; Valanides, 1990). It is believed that this latter
direction about the teaching of general thinking skills in
contextualized ways is promising and might be the way to
follow to adequately prepare critical thinkers. Accordingly,
contextualized critical-thinking skills instruction may be
potentially more effective than instruction with a focus on
teaching general critical-thinking skills in isolation from
context-specific knowledge.
1.3.1. Research questionsdHypotheses
The present study set out to empirically investigate the
effects of the General, the Infusion, and the Immersion
teaching approaches on learners critical-thinking performance, as compared to a control condition in which students
received no instruction. In particular, the study was designed
to seek answers to the following research questions: (a) Does
the method of teaching general critical-thinking skills, that is,
particular versions of the tactics that are used in the General,
the Infusion, and the Immersion teaching approaches, and the
control group, differentially affect learners critical-thinking
performance on an ill-defined issue? (b) Does the method of
teaching general critical-thinking skills differentially affect
learners self-reported understandings of critical thinking?
It was hypothesized that students in the Infusion and
Immersion groups will outperform students in the General and
the control groups in terms of their critical-thinking performance about an ill-defined problem. Also, students in the
Infusion group will outperform those in the Immersion group,
and students in the General group will outperform those in the
control group (Hypothesis 1).
Furthermore, it was hypothesized that the Infusion
approach will be more effective than the General and the
Immersion approaches as well as the control condition in
inducing correct understandings of critical thinking. Similarly,
it was hypothesized that the General and the Immersion
approaches will also be more effective than the control
condition in inducing correct understandings of critical
thinking. Additionally, the Immersion approach will be more
effective than the General approach and less effective than the
Infusion approach in inducing correct understandings of critical thinking (Hypothesis 2).
2. Method
2.1. Participants
A total of 160 undergraduates volunteered to participate in
the study. Of the 160 students, data from 16 of them were used
to pilot-test the research materials in order to clarify the
meaning of certain expressions as well as to determine the
duration of the research procedures. The remaining 144
students participated in the study. Their mean age was 19.81
years (SD 1.94). Of the 144 participants, 36 (25%) of them
were freshmen, 79 (54.86%) were sophomores, 19 (13.19%)
were juniors, and 10 (6.94%) were seniors. Also, 100

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

(69.44%) of the participants were females and 44 (30.56%)

were males. Lastly, 97 (67.36%) majored in education, and the
remaining 47 (32.64%) in other areas, such as psychology,
computer science, business, and political science. None of the
participants had any previous experience with the teaching of
critical-thinking skills.
2.2. Design
Each student participated in three sessions, namely, Sessions I,
II, and III. The procedures for the three sessions are summarized
in Table 1. As shown in Table 1, the procedures for Sessions I and
III, but not for Session II, were the same for all participants.
Table 1
Design of the study and time (min) devoted to sessions (total 225 min).
Time
Session I (85 min)
1. Introduction to Session I
2. Demographics Questionnaire
3. California Critical Thinking Skills Test
4. Participants individually read and summarized the issue Are
American values shaped by the mass media?
Session II (80 min)
1. Introduction to Session II
2. Each student in each dyad reviewed his/her summary of the
issue Are American values shaped by the mass media?
3. Teaching method (each one 65 min)
General
Lecture
Each dyad discussed the issue and prepared a joint
outline for a paper on the issue
Infusion
Each dyad discussed the issue and started preparing a
joint outline for a paper on the issue
Students in each dyad reflected on their thinking
Short lecture
Dialogue with the researcher
Each dyad completed a joint outline for a paper on the issue
Immersion
Each dyad discussed the issue and began to prepare
a joint outline for a paper on the issue
Students in each dyad reflected on their thinking
Socratic questioning with the researcher
Each dyad completed a joint outline for a paper on the issue
Control
Each dyad discussed the issue and prepared a joint
outline for a paper on the issue
4. All students
Students were given the issue Should drugs be legalized?
to read and summarize at home
Session III (60 min)
1. Introduction to Session III
2. Each student in each dyad reviewed his/her summary of
the issue Should drugs be legalized?
3. Each dyad discussed the issue Should drugs be legalized?
and prepared a joint outline for a paper on the issue
4. Student evaluations

5
15
45
20

5
10

35
30

15
10
10
15
15

15
10
25
15

65

5
10
30
15

325

2.2.1. Session I
For Session I, participants were divided into six groups of
24 students each. The first author introduced students to the
purpose of the study. Students in all four groups were told:
Our interest focuses on your thinking about some issues.
Specifically, we are interested in critical thinking; that is, in
the nature of critical thinking and how we can promote it
effectively. We feel that critical thinking is very important
in our life. We have seen so much thinking that is biased,
distorted, unquestioned, and even down-right prejudiced. Yet
the quality of our life, and the growth of our society, depends
precisely on the quality of our thinking. Shoddy thinking is
costly, both in terms of money and quality of life.
After that, participants individually filled out a questionnaire for demographic information and took the California
Critical Thinking Skills Test (CCTST; Facione, 1990b).
Students were subsequently given a one-page text about the
debate surrounding the issue Are American values shaped by
the mass media? The text presented the opposing views of
two experts on the specific issue. The first expert argued
succinctly in 300 words that American values are shaped by
the mass media and accordingly he presented three arguments
to support his position. Similarly, the second expert argued,
also in 300 words, that American values are not shaped by the
mass media and he also presented three arguments to support
his position. Students were asked to individually read and
summarize the text, and turn in their summaries. The procedures for Session I lasted 85 min.
Following Session I, students were randomly assigned into
72 dyads. The dyads were then randomly divided into three
teaching groups and a control group of 18 dyads each. Each
group was randomly assigned to a different teaching method
(versions of the General, the Infusion, and the Immersion
approaches) or the control group. The decision to group
students in dyads was based on the rationale that thinking with
others, as opposed to thinking alone, about a controversial illdefined issue would trigger more discussion and thus richer
information (data) for analysis.
2.2.2. Session II
For each of the 72 dyads, meetings for Sessions II and III
were scheduled on 2 consecutive days and at times convenient
for the participants. Thus, data collection for Sessions II and
III was done in parallel and was completed in 4 months.
Sessions II and III were facilitated by the same researcher
(first author) with only one dyad at a time. During Session II,
the researcher reminded the students of the issue Are
American values shaped by the mass media? they read and
summarized in Session I, and each student was given back his
or her summary from Session I to review. Students summaries
were not graded by the researchers and no specific feedback
about them was provided. Students were then informed that,
during Session II, they would work together in their dyads to
discuss the same issue and produce an outline for a paper
representing their joint position on the issue. Subsequently,
each dyad, except those in the control group, was taught about

326

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

general critical-thinking skills with one of the three teaching

methods, namely, versions of the General, the Infusion, or the
Immersion approach.
As shown in Table 1, the duration of each teaching intervention was 65 min. Students across the four groups worked in
dyads and were given the same instructions of how to tackle
the same ill-defined issue. The instructions were given to them
in writing so that students by themselves could refer to them,
when they were feeling unsure about their task, or the facilitator could ask the students to read the instructions again,
when they were not following them duly. Indicatively, the
written instructions in Session II stated:
I would like the two of you to work together and develop
a common point of view about the impact of the media on
societys values. I want you to develop a two-page outline
for a paper that presents your joint position. Remember, we
are interested in seeing your best thinking about these
issues. It is very important for you to do your best in
analyzing these issues and coming to a position. Please be
sure to raise and generate reasons and arguments that you
think are important, and be sure to try to understand,
evaluate, and resolve differences in points of view. Do
allow yourself to enjoy the discussion and be open to the
prospect of learning from one another.
The three teaching methods differed only in the approach
they adopted for teaching five general critical-thinking skills,
namely: (a) analyze the problem, (b) generate solutions, (c)
develop the reasoning for each solution, (d) decide which is the
best solution, and (e) use criteria to evaluate ones thinking.
These five general critical-thinking skills were chosen, because
they reflect the characterization of critical thinking, as it is
described in the Delphi definition (Facione, 1990a).
In the General group, a content-free teaching strategy was
used, because it attempted to teach critical-thinking skills
separately from the presentation of any content. A 35-min
lecture in videotaped format was used to explicitly deliver via
lecture mode the five general critical-thinking skills. During
the 35-min videotaped lecture, tactics, such as presentation,
examples, practice of the first three critical-thinking skills
before teaching the last two, and review, were used to teach
participants the skills of a general critical-thinking process.
There was no interaction between the researcher and the
students regarding the content of the videotaped lesson, and no
feedback was given to the students about their critical-thinking
performance. At the end of the 35-min lecture, the researcher
instructed the students in each dyad to work together for
30 min and develop a common point of view on the issue Are
American values shaped by the mass media?
In the Infusion group, each dyad was initially given 15 min
to start developing a joint position on the issue Are American
values shaped by the mass media? Then, the researcher
interrupted the students, and asked them to reflect on their
joint thinking process for about 10 min and to write down on
a piece of paper their rules for good critical thinking. Next, the
researcher presented each dyad with a 10-min videotaped
lecture about the five general critical-thinking skills. The

instructor on the videotape for the Infusion group was the

same instructor as the one on the video for the General group.
The 10-min videotaped lecture was a shorter version of the 35min lecture on the five general critical-thinking skills. The
main difference between the two lectures was that only in the
35-min lecture students were asked to practice the first three
critical-thinking skills with different practice examples
provided by the instructor on the videotape, before learning
about the last two critical-thinking skills. After the 10-min
lecture, the researcher engaged each dyad in a 15-min critical
dialogue of comparing and contrasting the skills presented to
them with those they had earlier written down. The purpose of
this interaction was to involve students in a process of
reflection and evaluation of their thinking. Finally, each dyad
was allowed an additional 15 min to complete the outline.
Thus, in the Infusion group, participants were encouraged to
think about critical-thinking skills within the presentation of
a controversial real-world issue. In contrast to the General
group, the researcher in the Infusion group guided student
learning by first asking students to write down their own
critical-thinking skills, and then to compare and contrast their
critical-thinking skills with those presented to them.
In the Immersion group, each dyad was also given 15 min
at the beginning to think about the same issue Are American
values shaped by the mass media? and start developing an
outline for a paper based on their joint position on the issue.
Then, the researcher asked them to reflect on their joint
thinking process for about 10 min and to write down on
a piece of paper their rules for good critical thinking. Thereupon, through questioning that lasted for about 25 min, the
researcher challenged participants to re-think their rules about
good critical thinking, and re-evaluate their reasoning and the
way they developed their point of view. The goal of the
discussion was to have students clarify, analyze, and evaluate
thoughts and perspectives, compare and contrast different
points of view, and stimulate their thinking. At the end, each
dyad took an additional 15 min to complete the outline. Thus,
the five general critical-thinking skills were never made
explicit to students, as it was the case with the other two
teaching methods, but students were left to infer them by
responding to relevant questions such as: Have you analyzed
the problem in depth?, What are your reasons for supporting this point of view?, Are there different perspectives on
the issue? Hence, the Immersion teaching method was
a variation of the Infusion method, because participants were
again guided to think about critical-thinking skills within the
presentation of a controversial real-world issue, but the five
critical-thinking skills were not explicitly stated.
Students in the control group were first given the same
written instructions, as those given to the students in the
General, Infusion, and Immersion groups, about how to tackle
the ill-defined issue Are American values shaped by the mass
media?, and were then engaged in problem solving without
receiving any instruction or feedback from the researcher.
They were only told that they had 65 min to prepare an outline
for a paper that presented the dyads joint position on the issue
Are American values shaped by the mass media?

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

At the end of Session II, each participant from the three

teaching groups and the control group was given a new text to
read and summarize at home before coming to Session III.
Students were specifically instructed to read and summarize
the text alone and to refrain from discussing the issue with
others. The new one-page text presented the opposing
perspectives of two experts on the issue Should drugs be
legalized? As it was the case with the first issue Are
American values shaped by the mass media?, the first expert
argued succinctly in 300 words that drugs should be legalized
and based his position on three arguments that he presented
and discussed. Accordingly, the second expert argued also in
300 words that drugs should not be legalized, and also presented and discussed three arguments to support his position.
2.2.3. Session III
In Session III, students in each dyad were instructed by the
researcher to work together, discuss the new issue Should
drugs be legalized?, and then produce an outline for a paper
that presented their joint position on the issue. Subsequently,
students answered two questions relating to their participation
in the study, namely, (a) What is your understanding of
critical thinking after participating in Sessions II and III?
and, (b) What were the difficulties that you encountered?
Session III procedures were completed in 60 min.
A month after the intervention, four dyads from each of the
three teaching groups and the control group were randomly
selected (32 participants from 16 different dyads) and participated in a debriefing session for the purpose of sharing their
thoughts and reflections about the intervention with other
participants and the researcher. The session was videotaped
and later transcribed.
2.3. Instruments
2.3.1. California critical thinking skills test (CCTST)
The CCTST is a 34-item multiple choice test which targets
general critical-thinking skills essential in college education
(Facione, 1990b; Facione & Facione, 1994). No disciplinespecific college level content knowledge is presumed on the
CCTST. The items are given without any technical vocabulary or
critical-thinking jargon, and they are set in contexts that are
familiar to college students. The CCTST is available in two
equivalent forms: Form A, which was used in this study, and
Form B. The CCTST reports six scores: an overall score on ones
critical-thinking skills, and five scores on subscales which are (a)
Analysis; (b) Evaluation; (c) Inference; (d) Deductive Reasoning;
and (e) Inductive Reasoning. The internal consistency reliability
(KudereRichardson 20) of the CCTST for Form A is KR20 0.70.1 In terms of concurrent validity, the CCTST correlated with the scores of Grade Point Average (GPA; r 0.20),
Scholastic Aptitude Test-verbal (SAT-verbal; r 0.55), and
Scholastic Aptitude Test-math (SAT-math; r 0.44) (Facione
1
According to Facione and Facione (1994), for an instrument of this kind,
a KR-20 between 0.65 and 0.75 is sufficient.

327

et al., 1994). But, given the concerns about variations in theoretical constructs, such correlations should be cautiously interpreted. In the present study, KR-20 was found to be 0.78.
2.3.2. Critical-thinking performance
Students critical-thinking performance was measured with
a rubric that was constructed to assess the 72 outlines prepared
by the dyads in Session III for the issue Should drugs be
legalized? The rubric was constructed inductively using the
constant comparative analysis method, which constitutes the
core of qualitative analysis in the grounded-theory approach
developed by Glaser and Strauss (1967) (Strauss & Corbin,
1990). The constant comparative method was carried out in
three phases: (a) comparison within a single outline, (b)
comparison among outlines within the same group (i.e.,
General, Infusion, Immersion, control), and (c) comparison of
outlines from different groups. The analysis led to four criteria
for discriminating between the different levels of the scoring
rubric. The four criteria used in the scoring regarded whether
students in each dyad (a) reasoned clearly within a point of
view, (b) discussed the issue from different perspectives, (c)
identified pros and cons for each perspective, and (d)
explained with reasons and evidence which perspective they
considered as the best. Consequently, the scoring rubric was
formed as shown in Appendix A.
The critical-thinking scoring rubric had five mutually exclusive levels, and participants scores ranged from 1 (low performance) to 5 (high performance). An independent rater was
trained on the scoring rubric and assessed 13 (72.22%) outlines
from each group, which were randomly selected by the
researchers, in order to determine the interrater reliability. Pearsons correlation between the rating of the independent rater and
that of the researchers was calculated and was found satisfactory,
r 0.85. The rater and researchers also discussed the observed
disagreements and easily resolved the existing differences.
2.3.3. Understanding of critical thinking
Two independent raters performed a qualitative analysis
(Patton, 2002) of students responses to the two questions of
the evaluation questionnaire, which was completed at the end
of Session III. The qualitative analysis revealed four categories
of student understandings. In particular, participants exhibited
precise understandings of critical thinking, such as A critical
thinker evaluates different perspectives and A critical
thinker is systematically engaged in the thinking process.
They also developed erroneous or imprecise understandings,
such as A critical thinker compromises to reach a decision
and A critical thinker collaborates and listens to others. The
interrater agreement was found to be 0.89.
3. Results
3.1. Effects of teaching methods on students
critical-thinking performance
Table 2 shows descriptive statistics (means and
standard deviations) of students CCTST scores and their

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

328

Table 2
Means and SD of CCTST scores and critical-thinking performance of
students dyads per teaching method.
Teaching method CCTST scores

General
Infusion
Immersion
Control
Total

students in the Immersion group, as compared to those in the

Control group, was Cohens d 0.99. This effect size,
although smaller than the previous one, was also very high
indicating that the critical-thinking performance of the average
dyad in the Immersion group was at 0.99 standard deviations
above the mean critical-thinking performance in the control
group. Effect size statistics should be however interpreted
cautiously taking into consideration possible departures from
normality, especially when sample sizes are small (Feingold,
1992; Wilcox, 1995), as it was the case in the present study.

Critical-thinking performance

SD

Dyads (N) M

SD

Dyads (N )

15.89
15.89
15.94
14.75
15.62

5.56
4.65
4.45
4.60
4.76

18
18
18
18
72

1.45
1.46
1.42
1.38
1.53

18
18
18
18
72

3.11
4.00
3.83
2.44
3.35

critical-thinking performance regarding the ill-defined issue

(Should drugs be legalized?) for the three teaching groups
and the control group. The CCTST score of each dyad is the
average value of the individual scores in each dyad.
A one-way analysis of covariance, with teaching method as
the independent variable, the average value of the individual
scores in each dyad on the CCTST as the covariate, and critical-thinking performance as the dependent variable, was
performed. The covariate was found to be significant, F(1,
67) 7.56, P < 0.01, partial h2 0.10, indicating that dyads
with better CCTST scores had better critical-thinking performance than dyads with lower CCTST scores. The results also
indicated that, after partialling out the effects of the covariate
on the dependent variable, the differences in critical-thinking
performance among the four groups were statistically significant, F(3, 67) 4.30, P < 0.01, partial h2 0.16.
Post hoc comparisons, using the Bonferroni procedure,
were subsequently performed and revealed that students
assigned to the Infusion and Immersion groups significantly
outperformed those assigned to the control group (P < 0.05).
There was no other significant difference between the
performances of students assigned to the Infusion and
Immersion groups or between the performances of students
assigned to any other two groups.
The magnitude of the superior critical-thinking performance of students in the Infusion group in comparison with
students performance in the control group was estimated
using Cohens d effect size, which is the degree of mean
difference between the first (Infusion) and second group
(control) of students relative to, or divided by, the pooled
standard deviation (SD 1.42) (Cohen, 1988). The effect size
(Cohens d 1.10) was very high indicating that the criticalthinking performance of the average dyad in the Infusion
group was at 1.10 standard deviations above the mean criticalthinking performance in the control group. Similarly, the
magnitude of the superior critical-thinking performance of

3.2. Effects of teaching methods on students

critical-thinking understanding
As shown in Table 3, the four categories of understanding
critical thinking were expressed in different frequencies across
the three teaching groups and the control group. In the three
teaching groups, it was explicitly stated or strongly implied
that a critical thinker takes into consideration different points
of view and systematically evaluates diverse ideas. From this
perspective, a critical thinker often has to collaborate and
listen to what others have to say for the purpose of becoming
better informed. However, collaborating and listening to others
alone is not enough and it should not be understood as an act
for reaching a compromise; but, it should be understood as an
act for becoming better informed, before thinking further and
deciding about something.
To determine the significance of the differences in the four
categories of understanding critical thinking across the three
teaching groups and the control group c2 statistics were used.
In all cases they were significant, for A critical thinker
evaluates different perspectives, c2(3, N 77) 27.10,
P < 0.01; for A critical thinker is systematically engaged in
the thinking process, c2(3, N 50) 21.20, P < 0.01; for A
critical thinker compromises to reach a decision, c2(3,
N 23) 9.68, P < 0.01, and for A critical thinker collaborates and listens to others, c2(3, N 46) 17.76, P < 0.01.
Subsequently, pairwise comparisons were carried out to
determine significant between-group differences in the four
categories of student understandings of critical thinking. The
results of these comparisons showed that students assigned to
the three teaching groups were more inclined to evaluate
different perspectives than students assigned to the control
group. Significant differences between the control group and
the General, Infusion, and Immersion groups were found,
c2(1, N 25) 7.41, P < 0.01, c2(1, N 33) 20.20,
P < 0.001, and c2(1, N 33) 20.20, P < 0.001, respectively.

Table 3
Frequencies of students understandings of critical thinking.
Teaching method

A
A
A
A

critical
critical
critical
critical

thinker
thinker
thinker
thinker

evaluates different perspectives

is systematically engaged in the thinking process
compromises to reach a decision
collaborates and listens to others

General

Infusion

Immersion

Control

Total

18
22
9
9

26
15
0
11

26
8
7
5

7
5
7
21

77
50
23
46

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

Clearly, students in the General, the Infusion, and the

Immersion groups correctly understood that a critical thinker
evaluates different perspectives; however, this was not the case
with the students of the control group.
Significant differences between the control group and the
three teaching groups were also observed for the category A
critical thinker collaborates and listens to others, c2(1,
N 30) 8.23, P < 0.01, c2(1, N 32) 5.63, P < 0.05, and
c2(1, N 26) 15.41, P < 0.001, for the General, Infusion,
and Immersion groups, respectively. These significant differences indicate that students in the control group were more
likely to think that a critical thinker collaborates and listens to
others than students assigned to the other three teaching
groups. Students in the control group, who received neither
instruction nor guidance from the researcher, thought that the
goal of the intervention was to learn to collaborate and listen
to others. In contrast, students assigned to the other three
teaching groups did not exhibit any significant differences
among them in terms of this false impression, and they were
less inclined to think about collaborating and listening to
others as a critical-thinking skill.
The results also showed significant differences between the
control group and the General or the Infusion group for the
category A critical thinker is systematically engaged in the
thinking process, c2(1, N 27) 17.13, P < 0.001, and
c2(1, N 20) 6.92, P < 0.01, for the General and Infusion
groups, respectively. Similarly, a significant difference was
found between the General group and the Immersion group for
the same category as above, c2(1, N 30) 11.20, P < 0.01.
This finding is justified by the fact that only in the General and
the Infusion groups it was explicated that critical thinking is
a systematic thinking process. Moreover, none of the students
assigned to the Infusion approach developed the misunderstanding that a critical thinker should compromise to reach
a decision, while several students from the other three groups
did so.
3.3. Difficulties encountered
Regarding the difficulties students encountered during
Sessions II and III, three main themes emerged, namely: (a)
lack of evidence, knowledge, and information to back up
a point of view; (b) difficulty in putting beliefs and biases
aside, and examining the other side of the issue; and (c)
difficulty, at times, in communicating ideas and resolving
differences. Interrater agreement was found to be 0.92. These
difficulties were expressed with different frequencies across
the four groups as shown in Table 4. A c2 analysis, however,

329

indicated that the existing differences amongst the three

teaching groups and the control group were nonsignificant and
thus no further analyses were conducted.
3.4. Students reflections
As it was previously mentioned, 1 month after the intervention the researcher hosted a debriefing session in which
four dyads from each of the four groups participated (32
research participants from 16 different dyads), and expressed
their thoughts and reflections about the intervention.
Students from the Immersion group strongly stated that
during the intervention they felt frustrated at times, because
the researcher was not giving them the correct answers or
answers to questions they were asking. As a student clearly
stated:
The biggest frustration, I felt during the study, was not
knowing if what we were doing was right. We had to come
up with the answers on our own (S15).
This initial puzzlement turned out to be however the biggest
satisfaction not only for student S15, but for almost every
student, as indicated by the excerpts from the reflections of
three of them. In these cases, students stated that the intervention was satisfying and proved to be useful for improving
their ability to write better papers and get higher grades in
their courses:
Even though this was my biggest frustration, it led to my
biggest satisfaction. It was satisfying to know at the end
that my partner and I came up with the answers on our
own.Even though, I didnt have complete mastery of
critical thinking; I thought I had vastly improved from what
I used to know (S15).
The most valuable thing that I learned was that it is
important to toughly think things through from all perspectives.I believe that teaching by letting people figure
things out on their own is very good.You asked us simple
questions that triggered explosive thinking about the topic
area (S34).
The biggest satisfaction is that I am able to do papers a lot
better now. The intervention helped me out a lot. To be
honest, I have gotten better grades on my papers since the
intervention. My grades have increased tremendously.
I think I owe it all to the intervention (S52).
Students from the Infusion group compared the quality of
the outlines they prepared in Session II with those they

Table 4
Observed frequencies of learners perceived difficulties.
Teaching method

Lack of evidence, knowledge, and information to back up a point of view

Difficulty in putting beliefs and biases aside, and examining the other side of the issue
Difficulty, at times, in communicating ideas and resolving differences

General

Infusion

Immersion

Control

Total

8
6
7

3
7
11

7
5
16

7
5
10

25
23
44

330

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

prepared in Session III. They stated that after they watched the
videotape with the critical-thinking skills in Session II, they
realized that their thinking exhibited in the outline about the
first ill-defined issue reflected mostly biased thinking, since
they never really examined the opposite point of view. They
candidly admitted that they gained insights about the
dynamics involved in the critical-thinking process. They also
suggested that the specific experience would be helpful to
them throughout their lives, and that they would use the same
tactics to teach their own students how to think critically.
Some representative examples of these ideas are illustrated in
the following excerpts:
I was surprised to realize that I had not really been thinking
clearly about the critical-thinking process.I feel that I will
be using the critical-thinking process a lot more now that
I understand the dynamics involved. I also think that I may
use this type of process as an example of how my future
students should really think when writing an argumentative
or persuasive paper. I learned more about myself from
being a participant in this study (S77).
In this case, however, I learned many things that I believe
will be helpful throughout my life. This intervention
brought me to the realization that I had a very vulnerable
flaw in my critical-thinking skills. What I took away from
the study was a valuable way to interpret two separate sides
on an issue and to be able to use both in the evaluation of
my final thought process (S121).
Furthermore, there were also students in the Infusion group
who compared and contrasted their experience from participating in this study with their experiences from other interventions. Student S98 eloquently stated:
I wanted to touch upon how this experience was valuable for
me. I volunteer for an organization called the Victim Offender
Reconciliation Program. This program is an alternative to the
penal system that brings the victim and offender of a crime
together so they can make things right between them again. As
a mediator, I must meet with both individuals separately and
listen to each of their stories. This involves a great deal of
open-mindedness which is a critical-thinking skill. Talk about
a lesson in critical thinking! .This was an important study. I
say this because I was able to take something away from it. I
have participated in many psychology studies but have never
really gotten anything from them. They just had me take
a series of battery and IQ tests and tried to predict my
personality type or something. Such interventions really dont
help the participant very much.
The same student went on to explain why the study was
different from other studies she participated in, and explained
why she considered this study to be beneficial for her and
other participants as well:
But this study was different - it taught you something about
yourself. It made you realize things you may not have
noticed before about the way you think. This, in my mind,
makes for a valuable and worthwhile intervention. The

researcher will be able to use these results to actually help

people. Perhaps the data she collects could be used to
implement different strategies in the classroom to help
students improve their critical-thinking skills (S98).
While students in the Immersion and Infusion groups were
very enthusiastic and very talkative about their participation in
the study, students in the General group were very skeptical
and expressed disappointment about their overall performance.
They also attributed their poor performance to their previous
educational experiences. This was clearly stated in the excerpt
from student S36:
I am really disappointed about my performance, but if I
could somehow explain why I performed so poorly I would
say this. One of the problems is that often we are taught at
an early age never to address the other side of our arguments in papers. We are told that this only weakens our
stand and makes us look as if we are not sure of ourselves
or that we are undecided. I dont think that a paper should
simply be a thesis and then three facts supporting this
thesis. Yet unfortunately, too often, all teachers want is
a simple thesis and three supporting facts.
The majority of the students indirectly stated that the
General approach did not provide enough help and scaffolding
for critically examining whether drugs should be legalized,
and that only after the debriefing session they realized what
they needed to do. Indicative examples of these ideas are the
excerpts from students S57 and S88:
I guessed I did not realize that I needed to examine all
possible points of view. But, I see now how important that
was. I wished I realized that earlier, so I could practice the
skill in Session III. I also wished that the researcher would
say something when I was not really thinking critically about
the issue, but I do understand that she could not do that (S57).
Now, that I know how badly I performed in the intervention, I feel more disappointed about it. However, I am glad
that we are having this discussion today, since I am now
realizing what I needed to do. Better late than never (S88).
Finally, the students from the control group stated that they
thought the goal of the intervention was to learn to collaborate
and listen to others. They were in most cases disappointed
with their performance and expressed complaints relating to
their educational experiences, and the lack of opportunities
throughout high school and college to develop criticalthinking skills that are lost amidst all the commotion in the
life of a typical student according to student S42:
Now that I know what the intervention was all about, I believe
that if I had had a brief overview of what critical thinking
meant, I might have been more successful in my task. I think
that much of the education throughout high school and college
is lost due to the constant hustle and necessity to focus on the
specific tasks at hand. Critical-thinking skills are easy to lose
amidst all the commotion in the life of a typical student. I, for
one, feel that I have lost the skill to think critically.

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

Students also consistently expressed disappointment

regarding their nave understandings of critical thinking, but they
also requested from the researcher copies of the critical-thinking
teaching materials that were used in the study, so they could also
have an opportunity to benefit from them. The researcher
informed the students that the teaching materials were available
and could be picked up during office hours. Two students (S3 and
S131) verbatim statements are indicative of these ideas:
I am very frustrated that I was so over-confident in something that I no longer practice as much as I should. I am so
disappointed that I assumed so much. I had assumed that
everyone shared the same point of view as I did, even
though I had read the opinion of a professor, which stated
quite the opposite. I assumed that I was an effective critical
thinker, even though I am considerable out of practice. I am
just very surprised at all the assumptions I had made and
without even realizing what I was doing (S3).
I would really like to get a copy of the materials that you
used to teach the skills of critical thinking. I heard from
somebody that you will make them available so that all
participants can have an opportunity to benefit from them. I
am hoping that I can get a copy of them (S131).

4. Discussion
The findings of the study clearly indicated that, after
adjusting for possible initial differences concerning their
general thinking skills, students critical-thinking performance
was differentiated depending upon the kinds of learning
experiences in which they were engaged. The findings
partially supported Hypothesis 1, while they supported
Hypothesis 2 for the most part. Regarding Hypothesis 1,
students assigned to the Infusion and Immersion groups had
significantly better critical-thinking performance on the illdefined issue Should drugs be legalized? than students
assigned to the control group. There were no other significant
differences in terms of critical-thinking performance between
students assigned to any other groups. Regarding Hypothesis
2, students assigned to the three teaching groups reported
significantly better understandings of critical thinking than
students in the control group. Additionally, students in the
Infusion group reported better understandings of critical
thinking than students in the Immersion group and in all other
groups. However, students in the Immersion group did not
always report better understandings of critical thinking than
students in the General group.
Thus, students were not automatically disposed to think
critically even for controversial issues that lend themselves
naturally to critical examination of different perspectives. On
the contrary, the evidence from the present study clearly
suggests that the kind of teaching method is important for
someone to construct correct understandings of critical
thinking and learn how to think critically. Also the findings
show that we cannot rest assured that students will eventually
become critical thinkers without any deliberate instruction

331

targeting the teaching of critical-thinking skills. Moreover,

students in the General, Infusion, and Immersion groups did
not report more difficulties than students in the control group,
and students in the Infusion and Immersion groups seemed
more satisfied from their performance and learning than
students in the other two groups.
In explaining these findings, the differences among the
teaching methods should be taken into careful consideration.
The same directions were given to all dyads about what they
were supposed to do and what task they had to complete.
Students in the dyads from all groups, including the control
group, also had to work together to prepare an outline for the
issue Are American values shaped by the mass media? in
Session II, and an outline for the issue Should drugs be
legalized? in Session III. Students in the control group did
not have any kind of training, were not engaged in any kind of
dialogue with the researcher, and were simply engaged in
problem solving without any instruction, guidance or feedback. Students assigned to the General group watched the
videotaped lecture and followed the researchers instructions
regarding the task they needed to complete, but they were also
not engaged in any kind of dialogue with the researcher and
did not receive any kind of feedback from the researcher about
their critical-thinking performance. Students in the Infusion
and Immersion groups did not have the same training, mainly
because the skills were not stated explicitly in the Immersion
group. But, in both cases, students were engaged in dialogue
with the researcher, and were guided to construct their own
critical-thinking skills and subsequently to confront their
thinking through reflection for the purpose of evaluating it.
The dialogues in the Infusion and Immersion groups were
particularly important for inducing critical thinking. They
started out as inquiry aiming toward the answering of
a specific question, such as, What is your point of view? or
Why do you think that? Then, the researcher moved to
dialogues as guided instruction aiming toward directing the
students to decide about something. For instance, the
researcher asked: Someone could argue that a critical thinker
is one who evaluates different perspectives before deciding
about an issue. In your outline though, you have not done this.
Can you explain to me why? This was the question that
stimulated sharp thinking and spurred tremendous interest
from the part of the students. Initially, most students said that
they did not need to explain a perspective that someone else
might have, because everyone is entitled to his or her point of
view, and people should respect that. At this point, the dialogue took the form of a debate between the researcher and the
students trying to decide about the question Should we
examine different points of view before deciding about an
issue or not? While many students showed skepticism and
a willingness to reexamine their beliefs, some others stayed
firm in their beliefs and showed no willingness to alter the way
they were thinking. This seems to have happened in both, the
Immersion group where the five critical-thinking skills were
not explicitly spelled out, and the Infusion group where the
five critical-thinking skills were explicitly and clearly spelled
out.

332

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

Based on these results, students critical thinking seems to

improve most in teaching environments where learning is
mediated by someone who confronts students beliefs and
alternative conceptions, encourages students to reflect on their
own thinking, creates cognitive dissonance or puzzlement, and
challenges and guides students thinking when they are
actively involved in problem solving. These tactics seem to
increase students metacognitive awareness enabling them to
think and talk about their own thinking processes (Hartman,
2001). Rogoff (1990) termed this type of social interaction
guided participation. By guided participation, she means
active involvement by learners in structured activities with the
guidance, support, and challenge of companions. Accordingly,
asking students to constantly draw distinctions not in the
abstract but concretely, allowing them to doubt things as well
as to be engaged in debates, and making them aware of the
fact that there are multiple ways of carving up the same
domain are key strategies for the teaching of critical thinking.
Most importantly, creating social contexts for instruction does
not mean throwing students in an activity and letting them
learn alone. It means guiding students learning by providing
a support system tailored to their needs and gradually
prompting them to take control of their own learning.
The results also indicate that learning, which is situated in
a meaningful context of challenging problem solving, scaffolded by a more experienced other, is significantly more
effective than unguided exploration. Interestingly, in the
present study, it was the control group that was engaged in
unguided exploration. In the literature, this type of instructional approach is referred to as discovery learning (Klahr &
Nigam, 2004), or inquiry learning (Windshitl, 2004). Despite
the fact that these approaches have been very popular, the
results of this study indicate that free exploration with no
guidance from the teacher does not work. These findings
support research by Mayer (2004), Klahr and Nigam (2004),
and Kirschner, Sweller, and Clark (2006) who concluded that
teaching guidance is more effective than discovery and
unstructured exploration.
The results also showed that students in the General group,
who were taught about general critical-thinking skills in a decontextualized way and with no support or feedback from the
researcher, performed relatively lower than those assigned to
the Infusion and the Immersion groups, but the differences
among them were not significant in this study. Despite the fact
that these findings do not confirm Hypothesis 1, which stated
that the Infusion and Immersion groups would outperform the
General group, future research investigations with larger
sample sizes may show significant differences between the
General and the Infusion and Immersion teaching approaches.
In addition, the findings relating to the nonsignificant differences between the General and the control groups corroborate
Garsides (1996) results, which showed that lectures alone was
not a more effective strategy for the development of criticalthinking skills than mere group discussions. Obviously, the
lack of feedback in both the General and control groups
influenced in a negative way the quality of students criticalthinking performance in those groups.

The results of the present study also corroborate other

research findings (Moreno, 2004), indicating that in instructional environments where students learn with minimal guidance, not only do they become lost and frustrated, but they
also develop misunderstandings, or alternative understandings,
that diverge from the widely accepted ones. Students in the
control group were, for example, more inclined to believe that
a critical thinker collaborates and listens to others than
students in the three teaching groups. Also, students in the
General group were more inclined to believe that a critical
thinker is systematically engaged in the thinking process than
students in the Immersion group. Similarly, only the students
assigned to the Infusion group did not express the idea that A
critical thinker compromises to reach a decision. The latter
qualitative findings show that students in the Infusion group
were more likely to construct correct understandings of critical
thinking than students in the Immersion group, as well as in
the other groups, despite the fact that no significant differences
were found between students critical-thinking performance in
the Infusion and Immersion groups. However, more research is
needed to further investigate the differences between the
Infusion and Immersion teaching approaches. Regarding
students self-reported difficulties, no significant differences
were found across the three teaching groups and the control
group.
An issue that needs to be followed up in future studies
concerns the specific sequence of the instructional tactics to be
presented to learners. In the present study, the sequence of
tactics was pre-specified and applied as such for both Infusion
and Immersion. However, future studies may examine the
effects of different instructional sequences taking into consideration students prior knowledge, so that the question
regarding the most appropriate starting point in instruction for
students to correctly understand the concept of critical thinking,
as well as for learning how to think critically can be answered.
A second issue for future research relates to the possible
transfer and retention effects of the Infusion and Immersion
approaches. In the present study, no significant differences
were found between the Infusion and Immersion groups,
despite the fact that the five general critical-thinking skills
were stated explicitly only in the Infusion group and not in the
Immersion group. This difference between the two instructional approaches is important and it is worthy of further
research, because it might be the case that if general criticalthinking skills are not made explicit to students, then students
critical-thinking skills may transfer from one domain to
another in the short-run, but not in the long-run.
It is also important to interpret the evidence from the
present study taking into consideration both its limitations in
terms of sample size and the short duration of the interventions. First, although there were 144 students who participated
in the study, the unit of analysis was the dyad and not the
individual participant. As a result, the sample size that was
used for detecting differences in instructional effects included
data from the 72 dyads that were randomly formed and
randomly assigned to the three teaching groups and the control
group. Second, the duration of the instructional intervention

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

was limited. These limitations can always have implications

on the effects of the different instructional treatments and can
mask to a certain extent any existing differences among them.
Other concerns also seem pertinent. For example, it cannot be
inferred at this point in time whether the differences in criticalthinking performance, as they were reported in the present study,
were enduring or temporary. Based on students comments
during the debriefing session, learning with both the Infusion and
Immersion approaches enabled them to use their critical-thinking
skills in contexts beyond the immediate instructional context of
this study. However, despite of this promising evidence, more
systematic experimental studies are needed to further examine
the issue of transfer across different contents and contexts, and
whether the identified gains in critical-thinking performance
persist and become enduring habits of mind.
Lastly, the dyads in this study were randomly formed and
also randomly assigned to each approach, but no information
was collected concerning the interactions and the dynamics
involved in each dyad. Recent developments relating to the
socially shared distributed cognition emphasize the importance of examining the learner in interaction with others
(Angeli, 2008; Salomon, 1992, 1993; Valanides & Angeli,
2008). This type of analysis was not the focus of the present
investigation, but it should be considered in future studies
about the teaching of critical thinking, because it is commonly
accepted that much of human cognition is so varied and so

Appendix A
Critical-thinking performance scoring rubric.
Criteria

Scoring

1.
2.
3.
4.

Reason clearly within a point of view.

Discuss the issue from different perspectives.
Identify pros and cons for each perspective.
Explain with reasons and evidence which
perspective they think is best.

1.
2.
3.
4.

Reason clearly within a point of view.

Discuss the issue from different perspectives.
Identify pros and cons for each perspective.
Do not explain with reasons and evidence
which perspective they think is best.

1.
2.
3.
4.

Reason clearly within a point of view.

Discuss the issue from different perspectives.
Do not identify pros and cons for each perspective.
Do not explain with reasons and evidence which
perspective they think is best.

1.
2.
3.
4.

Reason clearly within a point of view.

Do not discuss the issue from different perspectives.
Do not identify pros and cons for each perspective.
Do not explain with reasons and evidence which
perspective they think is best.

1.
2.
3.
4.

Do not reason clearly within a point of view.

Do not discuss the issue from different perspectives.
Do not identify pros and cons for each perspective.
Do not explain with reasons and evidence which
perspective they think is best.

333

sensitive to cultural context that we must also seek mechanisms by which people actively shape each others knowledge
and reasoning process (Resnick, 2004: 2).
References
Alston, K. (2001). Re/thinking critical thinking: the seductions of everyday
life. Studies in Philosophy and Eduction, 20, 27e40.
Anderson, T., Howe, C., Soden, R., Halliday, J., & Low, J. (2001). Peer
interaction and the learning of critical-thinking skills in further education
students. Instructional Science, 29, 1e32.
Angeli, C. (2008). Distributed cognition: a framework for understanding the
role of technology in teaching and learning. Journal of Research on
Technology in Education, 40(3), 271e279.
Bailin, S. (2002). Critical thinking and science education. Science & Education, 11(4), 361e375.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Conceptualizing
critical thinking. Journal of Curriculum Studies, 31, 285e302.
Brown, A. (1997). Transforming schools into communities of thinking and
learning about serious matters. American Psychologist, 52, 399e413.
Browne, M. N., & Freeman, K. (2000). Distinguishing features of criticalthinking classrooms. Teaching in Higher Education, 5, 301e309.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd
ed).. Hillsdale, NJ: Erlbaum.
Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship:
teaching the crafts of reading, writing, and mathematics. In L. B. Resnick
(Ed.), Knowing, learning, and instruction: Essays in honor of Robert
Glaser (pp. 453e494). Hillsdale, NJ: Erlbaum.
Dennick, R. G., & Exley, K. (1998). Teaching and learning in groups and
teams. Biochemical Education, 26, 111e115.
Education Commission of the States. (2007). Framework for reauthorizing the
No child Left Behind (NCLB) Act. Recommendations to improve and
strengthen the law. Denver, CO: Author.
Ennis, R. H. (1989). Critical thinking and subject specificity: clarification and
needed research. Educational Researcher, 18, 4e10.
Ennis, R. H. (1992). The degree to which critical thinking is subject specific:
clarification and needed research. In S. P. Norris (Ed.), The generalizability
of critical thinking: Multiple perspectives on an educational ideal (pp. 21e
37). New York: Teachers College.
Facione, N., Facione, P., & Sanchez, M. (1994). Critical-thinking disposition
as a measure of competent clinical judgment: the development of the
California Critical Thinking Disposition Inventory. The Journal of Nursing
Education, 33, 345e350.
Facione, P. A. (1990a). Critical thinking: A statement of expert consensus for
purposes of educational assessment and instruction. Millbrae, CA:
Academic.
Facione, P. A. (1990b). The California critical thinking skills test (Reports No.
1e4). East Lancing, MI: National Center for Research on Teacher
Learning (ERIC Document Reproduction Service No. ED 327 549, ED
327 550, ED 327 584, and ED 327 566).
Facione, P. A., & Facione, N. C. (1994). The California critical thinking skills
test: Test manual. Millbrae, CA: Academic.
Feingold, A. (1992). Sex differences in variability in intellectual abilities: a new
look at an old controversy. Review of Educational Research, 62, 61e84.
Frijters, S., Ten Dam, G., & Rijlaarsdam, G. (2008). Effects of dialogic learning
on value-loaded critical thinking. Learning and Instruction, 18, 66e82.
Garside, C. (1996). Look whos talking: a comparison of lecture and group
discussion teaching strategies in developing critical-thinking skills.
Communication Education, 45, 212e227.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory:
Strategies for qualitative research. Chicago: Aldine.
Halpern, D. F. (1997). Critical thinking across the curriculum. Mahwah, NJ:
Erlbaum.
Halpern, D. F. (2003). Thought and knowledge (4th ed.). Mahwah, NJ:
Erlbaum.
Hartman, H. J. (2001). Metacognition in learning and instruction: Theory,
research and practice. Dordrecht, The Netherlands: Kluwer.

334

C. Angeli, N. Valanides / Learning and Instruction 19 (2009) 322e334

Johnson, D. W., & Johnson, R. T. (1993). Creative and critical thinking.

Through academic controversy. American Behavioral Scientist, 37, 40e53.
Kalman, C. S. (2002). Developing critical thinking in undergraduate courses:
a philosophical approach. Science and Education, 11, 83e94.
Kaplan, E. J., & Kies, D. A. (1994). Strategies to increase critical thinking in
the undergraduate college classroom. College Student Journal, 28, 24e31.
Kennedy, M., Fisher, M. B., & Ennis, R. H. (1991). Critical thinking: literature
review and needed research. In L. Idol, & B. Fly Jones (Eds.), Educational
values and cognitive instruction: Implications for reform (pp. 11e40).
Hillsdale, NJ: Erlbaum.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance
during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching.
Educational Psychologist, 41(2), 75e86.
Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early
science instruction. Psychological Science, 15(10), 661e667.
Kuhn, D. (1999). A developmental model of critical thinking. Educational
Researcher, 28, 16e25.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure
discovery learning? The case for guided methods of instruction. American
Psychologist, 59, 14e19.
McBride, R. E., Xiang, P., & Wittenburg, D. (2002). Dispositions toward
critical thinking: the preservice teachers perspective. Teachers and
Teaching, 8, 29e40.
McPeck, H. (1981). Critical thinking and education. New York: St. Martins
Press.
McPeck, H. (1990a). Critical thinking and subject specificity: a reply to Ennis.
Educational Researcher, 19, 10e12.
McPeck, H. (1990b). Teaching critical thinking: Dialogue and dialectic. New
York: Routledge.
Moreno, R. (2004). Decreasing cognitive load in novice students: effects of
explanatory versus corrective feedback in discovery-based multimedia.
Instructional Science, 32, 99e113.
Nickerson, R., Perkins, D. N., & Smith, E. (1985). The teaching of thinking.
Hillsdale, NJ: Erlbaum.
Papastephanou, M., & Angeli, C. (2007). Critical thinking beyond skill.
Educational Philosophy and Theory, 39(6), 604e621.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.).
Thousand Oaks, CA: Sage.
Paul, R. (1995). Critical thinking: How to prepare students for a rapidly
changing world. Santa Rosa, CA: Foundation for Critical Thinking.
Perkins, D. N. (1986). Knowledge as design. Hillsdale, NJ: Erlbaum.
Perkins, D., Tishman, S., Ritchhart, R., Donis, K., & Andrade, A. (2000).
Intelligence in the wild: a dispositional view of intellectual traits. Educational Psychology Review, 12, 269e293.

Peters, M. A. (2005). The new Prudentialism in education: actuarial rationality

and the entrepreneurial self. Educational Theory, 55, 122e137.
Pithers, R. T., & Soden, R. (2000). Critical thinking in education: a review.
Educational Research, 42, 237e249.
Prawat, R. S. (1991). The value of ideas: the Immersion approach to the
development of thinking. Educational Researcher, 20, 3e10.
Resnick, L. (1987). Education and learning to think. Washington, DC:
National Academy Press.
Resnick, L. (2004). Shared cognition: thinking as a social practice. In
L. Resnick, J. Levine, & S. Teasley (Eds.), Perspectives on socially shared
cognition (pp. 2e20). Washington, DC: American Psychological
Association..
Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social
contexts. New York: Oxford University Press.
Salomon, G. (1992). New challenges for educational research: studying the
individual within learning environments. Scandinavian Journal of Educational Research, 36(3), 167e182.
Salomon, G. (1993). Distributed cognitions: Psychological and educational
considerations. New York: Cambridge University Press.
Strauss, A. L., & Corbin, J. (1990). Basics of qualitative research: Grounded
theory procedures and techniques. Newbury Park, CA: Sage.
Ten Dam, G., & Volman, M. (2004). Critical thinking as a citizenship
competence: teaching strategies. Learning and Instruction, 14, 359e379.
Tsui, L. (1999). Courses and instruction affecting critical thinking. Research in
Higher Education, 40, 185e200.
Tynjala, P. (1998). Traditional studying for examination versus constructivist
learning tasks: do learning outcomes differ? Studies in Higher Education,
23, 173e189.
Valanides, N. (1990). Pragmatic versus syntactic approaches to deductive
reasoning. Doctoral Dissertation. University at Albany, State University of
New York. 1175A. (University Microfilms No. 90-24947).
Valanides, N., & Angeli, C. (2008). Distributed cognition in a sixth-grade
classroom: an attempt to overcome alternative conceptions about light
and color. Journal of Research on Technology in Education, 40(3),
309e336.
Voss, J. F., Perkins, D. N., & Segal, J. W. (1991). Informal reasoning in
education. Mahwah, NJ: Erlbaum.
Whimbey, A., & Lochhead, L. J. (1986). Problem solving and comprehension.
Hillsdale, NJ: Erlbaum.
Wilcox, R. R. (1995). ANOVA: a paradigm for low power and misleading
measures of effect sizes? Review of Educational Research, 65, 51e77.
Windshitl, M. (2004). Caught in the cycle of reproducing folk theories of
Inquiry: how pre-service teachers continue the discourse and practices
of an atheoretical scientific method. Journal of Research in Science
Teaching, 45(5), 481e512.