Running Head: Visible Thinking

1

The Use of Thinking Routines in Constructing
Open-Ended Responses in Thematic Studies

Stephanie J. Franklin

A Research Study Submitted to Gannon University
In Fulfillment Of
Standards One, Two, Three
Portfolio Requirements for the Course
Educational Research Methods
GEDU 601
Dr. Eugene M. Dangelo
May 19, 2012


Running Head: Visible Thinking

2
Abstract
The following research study was an attempt to determine whether a correlation
between the use of thinking routines during thematic studies and student responses to
open-ended questions during thematic studies could be drawn. The study was conducted
in 2012 in a Charter School in Pittsburgh, Pennsylvania. Nine and ten year old students
from a fourth grade class were selected to participate in the study if they had three
baseline open-ended responses available for assessment. The students engaged in two to
four thinking routines each week during thematic studies, over the course of a twelve-
week time period. At the end of the study, the students open-ended responses were
assessed based on the same rubric as had been used to assess the original responses. The
study indicated that students open-ended responses improved, however, a correlation
between the thinking routines and the writing could not be drawn. Additionally,
improvement was seen in students abilities to verbalize their thinking in an open
discussion forum.






Running Head: Visible Thinking

3
Table of Contents
I. Introduction 4
II. Statement of the Problem 5
A. Research Questions 5
B. Delimitations 5
C. Rationale 5
III. Review of Related Research 6
IV. Design of the Study 12
A. Selection of the Thinking Routines 12
B. Characteristics of the Thinking Routines 13
C. Construction of Evaluation Methods 14
D. Selection of Group 16
E. Schedule of Implementation 17
V. Results of the Study 17
A. Findings 17
B. Conclusions 20
C. Recommendations 20
References 23


Running Head: Visible Thinking

4
I. Introduction
In classrooms across America daily, teachers ask students to answer questions,
solve problems, and explain their thinking. When students are unsure of what to do, they
are often told to think about it. In a country filled with high-stakes testing, the field of
education focuses on the mastery of specific skills in content areas at certain ages. With
this increased emphasis on skill-attainment, and the idea that all students must be
proficient, less emphasis is put on teaching students how to think. If surveyed, many
teachers would tell you that they want their students to be higher-level thinkers or
critical thinkers, though many would not be able to tell you what it means to be a
higher-level thinker or what they do to support higher-level thinking. (Barahal, 2008).
The process of making thinking visible can help teachers encourage students thinking
processes, and actually teach kids how to think. This process, referred to as visible
thinking can be done in a variety of ways; one of these ways is through the use of
thinking routines. Thinking routines are defined as repeated patterns that support and
scaffold specific thinking moves or actions (Ritchhart, Palmer, Church, Tishman, 2006,
p.1).
Additionally, high-stakes testing requires students to be able to demonstrate their
thinking in writing in response to an open-ended question. Open-ended response
questions can be understood as questions that led themselves to a variety of answers,
generally do not have one correct answer, and require explanation of the thinking that
brought a student to respond with that answer. Therefore, students must be taught how to
Running Head: Visible Thinking

5
think, but also how to express their thinking in both verbal and written language.
Thinking routines provide one avenue for students to learn to do this.
II. Statement of the Problem
Research Questions
This study is intended to address the following research question: Does the use of
visible thinking routines during thematic studies improve students ability to demonstrate
their thinking when answering open-ended questions?
Delimitations
This study is delimited to fourth grade students in a single classroom in a Charter
School in Pittsburgh, Pennsylvania.
Rationale
Research has been conducted in the use of thinking routines, which suggests that
continuous use of thinking routines gives students an avenue for thinking about complex
issues, and for exploring new issues. Thinking routines provide access to the material, as
students verbalize, write, or draw what they are thinking about a certain topic. In the
reality of high-stakes testing, students are often prompted to explain their answers or to
verbalize their thinking in paragraph form, and many have trouble putting pen to paper
when explaining what made them answer the question in a particular way. Allowing
students to use thinking routines regularly, may provide a means for them to think about
their answers to open-ended questions. The access to complex thinking that thinking
Running Head: Visible Thinking

6
routines provide, combined with the need for students to explain their thinking on high-
stakes testing indicates a need for a study like this at the fourth grade level.
III. Review of Related Research
While thinking is something that has been going on for millennia, the idea of
making thinking visible has more recently come on the education scene as a way to help
students learn how to think. As mentioned previously, the term visible thinking refers to
the process of making thinking visible by using multiple methods, like thinking routines.
Thinking routines are repeated patterns that support and scaffold specific thinking
moves or actions (Ritchhart, Palmer, Church, Tishman, 2006, p.1). In the present
section, different thinking theories are examined and the use of thinking routines is
investigated to determine how thinking routines impact student thinking and students
ability to demonstrate their thinking through responses to open-ended questions.
Howard Gardners traditional research resulted in his theory of multiple
intelligences, which suggests that the brain learns different things in different ways and
that people have strengths in learning in those different ways. His theory suggests the
following intelligences exist: linguistic intelligence, musical intelligence, logical-
mathematical intelligence, spatial intelligence, bodily-kinesthetic intelligence, and the
personal intelligences (Gardner, 1993). The use of visible thinking routines focuses
mainly on the use of spatial intelligence. Spatial intelligence refers mostly to the ability
to be able to picture a problem or solve a problem relying on visual or spatial movement
or change. In his book, Frames of Mind, Gardner (1993) said it would be appropriate to
propose the descriptor visual because, in normal human beings, spatial intelligence is
Running Head: Visible Thinking

7
closely tied to, and grows most directly out of, ones observation of the visual world (p.
174). Gardners ideas on spatial thinking relate mostly to being able to problem solve
using spatial theories. Gardner goes on to discuss the importance of the spatial
intelligence in society. While being a visual-spatial thinker is linked to being able to use
visible thinking, this is not the only type of learning that visible thinking targets. While
visible thinking uses the visual-spatial modality, problems that involve all intelligences
can be considered with the use of visible thinking.
Similarly, Robert McKim wrote Thinking Visually as a manual for problem-
solving. In the book, he suggested that by learning to think visually people will have
much greater success at problem-solving. He considered thinking an activity being done
most of the time, and that most thinking cannot, in fact, be considered good thinking
(McKim, 1980). He noted that, a fascinating way to learn about thinking is to watch
your own thinking in action (McKim, 1980, p. 3). This statement, while McKim may
not have known it at the time, paved the way for visible thinking, which exists on the
premise that the best way to teach students how to think is to allow them to watch
themselves think by making it visible. This means putting thinking on chart paper,
talking through it or making a diagram of it. The use of thinking routines is one of the
ways educators can help make students thinking visible.
Next, its important to consider Blooms Taxonomy of intellectual behavior,
which explores the multiple levels of learning that can occur in the educational setting.
Refer to Figure A for a visual of Blooms Taxonomy. The levels at the bottom of the
triangle refer to less complex levels of understanding like recall and comprehension,
Running Head: Visible Thinking

8
moving up the triangle moves toward more complex types of understanding like
application, analysis, synthesis, and evaluation (Old Dominion University, n.d.). These
types of learning show that students understand the knowledge more deeply because they
can not only repeat information that they have learned but they can apply it to other
settings, analyze the information, evaluate its effectiveness, and synthesize new
understandings based on what they have learned.

Figure 1.


Old Dominion University, n.d.
An educator can use thinking routines to encourage higher level types of learning,
like application, analysis, synthesis, and evaluation by combining Gardners
understanding of spatial intelligence with Blooms Taxonomy. Making thinking visible
with thinking routines gives students an outlet to demonstrate differing types of higher
level thinking, as demonstrated later in the present paper.
Running Head: Visible Thinking

9
Next, it is critical to consider the thinking research that has already been done
using thinking routines in the classroom. Ritchhart, Palmer, Church, and Tishman (2006)
reported the results of three action research studies with the Innovating with Intelligence
project from 2000-2005 in schools in Sweden, The Netherlands, and Belgium, the Artful
Thinking project in Traverse City, Michigan Public Schools, and the Cultures of Thinking
project is Melbourne, Australia. The shared use of thinking routines as core practice
across school sites, allowed for a qualitative study on how adding these thinking routines
transformed the classroom culture as well as students thinking. The purpose of the
action-research project was three-fold: to serve the professional needs of the local school
site, to inform the practice of teaching how to think while creating a culture of thinking,
and to contribute to the development of effective thinking routines.
This synthesis of results highlights the important aspects of thinking routines
including these six characteristics: individual and group practices, useful across a variety
of contexts, used over and over again, instrumental in nature, explicit, and having only a
few steps (Ritchhart, et. al, 2006, p. 13). The study acknowledges that valuing students
abilities to think requires a shift in classroom culture and will convey several messages
about thinking to students. The messages conveyed to students when using these
thinking routines as follows: learning is doing, learning starts with the learners own
ideas, learning involves getting personally involved, questions are engines and outcomes,
learning involves uncovering complexity, learning can be a group process and a group
outcome (Ritchart, et. al., 2006, p. 16).
Running Head: Visible Thinking

10
This qualitative studys results highlight what teachers noticed about the
development and growth of student thinking abilities as they engaged in using thinking
routines. They found that after introducing the thinking routine in the classroom,
modeling the expected use of the thinking routine, and giving students the opportunity to
use the thinking routine frequently, student responses became much more authentic, and
showed deeper thinking.
Next, we look to an article by Angela Salmon, examining two action research
projects in two preschools where early childhood educators used thinking routines to help
students learn to think. In this article, Salmon agreed with Ritchhart et. al. noting the
difference between housekeeping routines and learning routines, stating that
housekeeping routines involved managing the movement and physical materials within
the classroom (Salmon, 2010, p. 1) while learning routines focus childrens attention
on the specific topic being studied (Salmon, 2010, p. 1). Thinking routines are a subset
of these mentioned learning routines. The article suggests, much like the Ritchhart
article, that thinking routines help children construct knowledge from prior experiences
and are meaningful and functional (Salmon, 2006, p. 2). The research project consisted
of six preschool teachers, two university faculty members, a doctoral candidate, and 60
children. Teachers met weekly to review students work and documented thinking
routines as they uncovered students thinking about thinking. Teachers found that in
order to give students access to thinking about thinking, they needed to scaffold these
thoughts with thinking routines. The study also found that by allowing student thought to
guide the discussions and use of routines, the class was able to have more authentic
conversations centered around thinking, guiding students learning and cognitive abilities.
Running Head: Visible Thinking

11
The unique part of this study is that most of the students in these classes spoke Spanish as
a primary language. Using drawing with the thinking routines helped them express their
thinking in their first language, while giving the teachers access to their ideas. (Salmon,
2006).
Next we look at a study by Susan A. Barahal (2008), in which pre-service
teachers worked through an action research project looking at the use of thinking routines
for Artful Thinking. Artful Thinking is similar to visible thinking in that it explores six
different thinking dispositions: reasoning, questioning and investigating, observing and
describing, comparing and connecting, finding complexity, and exploring viewpoints
(Barahal, 2008). In this particular study, the pre-service teachers were the subjects of the
study, to see how their practice changed with the use of the thinking routines.
This qualitative study found several themes over the course of the year. First, that
few teachers actually think about thinking. Before using the thinking routines, none of
the pre-service teachers had considered how they could help students to make
connections to big ideas in the curriculum (Barahal, 2008). The second finding was
that teaching to think requires a big picture perspective (Barahal, 2008, p. 300),
meaning that teachers could use these thinking routines to identify and unify themes and
connections across subject areas. Like Salmon and Ritchhart, they also found that
teaching to think requires teachers to focus on students and their thinking by using
student work as a window into their ideas. Additionally, they found that teachers must
be both facilitators and mentors (Barahal, 2008, p. 302) through the use of modeling
with thinking routines and allowing students ideas to guide instruction with an inquiry-
Running Head: Visible Thinking

12
based approach. Finally, they state that quality thinking can be cultivated as long as
teachers allow time and repeated us of thinking routines to deepen thinking. Even
though these teachers were not art teachers, they were able to use thinking routines to
incorporate artful thinking.
Several themes emerge when considering the research about the use of thinking
routines in the classroom. First, each study demonstrated that the use of thinking routines
improved the quality of student thinking. Additionally, the studies show that the use of
thinking routines focused both teacher and student attention on thinking. Last, each study
demonstrated that the successful use of thinking routines required a shift in the classroom
culture, to one which values alternate opinions and considering multiple perspectives.
The current research on thinking routines does not, however, study a connection between
the use of thinking routines and the quality of written responses to open-ended questions.
This is a valuable connection to study as student achievement is often assessed using
standardized tests where students are required to answer open-ended questions with a
written response. Should the use of thinking routines help improve student open-ended
responses, this could be an invaluable tools for many teachers trying to help students
learn to express their thinking in written form, thus improving scores on many high-
stakes standardized tests.
IV. Design of the Study
Selection of the Thinking Routines
The thinking routines being used in the classroom were selected from a list of
thinking routines being tried out within the school building. Those thinking routines
Running Head: Visible Thinking

13
were selected after identifying the seven kinds of thinking the school would like to foster
among students. After identifying the seven kinds of thinking, thinking routines that
fostered those kinds of thinking were identified for use within the school building to
determine their effectiveness. See figure two below.
Figure 2.

The researcher then used this list of thinking routines to determine which types of
thinking she would like to foster within the classroom and determined which thinking
routines would be most beneficial for her particular students. The researcher decided
upon the following thinking routines: See, Think, Wonder; Claim, Support, Question;
Connect, Extend, Challenge; I used to think, but now I think; Think, Puzzle, Explore;
and Broadcaster.
Characteristics of the Thinking Routines
Throughout the research study students will be engaged in the use of Thinking
Routines during the Integrated Studies period of the day. Each thinking routine is
7 kinds of thinking Thinking Routines
Observing and Describing See, Think, Wonder; Look 10x2; I used to
think, but now I think
Questioning and Predicting Look 10x2; Think, Puzzle, Explore
Comparing and Connecting Connect, Extend, Challenge; I used to think,
but now I think
Reasoning and Drawing Conclusions Think, Puzzle, Explore; See, Think, Wonder;
Claim, Support, Question; I used to think, but
now I think
Exploring Viewpoints Perceive, Know, Care About; I used to think,
but now I think
Finding Complexity: problem has
multiple parts
DIG; I used to think, but now I think
Communicating Ideas Broadcaster; I used to think, but now I
think
Running Head: Visible Thinking

14
designed to elicit certain types of thinking. Identification of specific thinking routines
and the type of thinking they target can be found in figure two below.
Figure 3.
THINKING ROUTINES TYPE OF THINKING
See, Think, Wonder Observing and describing, Reasoning and
drawing conclusions
Claim, Support, Question Reasoning and drawing conclusions
Connect, Extend, Challenge Comparing and connecting
I used to think, but now I think Observing and describing, comparing and
connecting, reasoning and drawing conclusions,
exploring viewpoints, finding complexity
Think, Puzzle, Explore Reasoning and drawing conclusions
Broadcaster Communicating ideas

Construction of Evaluation Methods
Because a rubric designed to assess open-ended questions related to the Integrated
Studies content does not yet exist, the researcher will develop a rubric that evaluates
students open response answers. The rubric will provide an objective measure to these
subjective open-ended response questions by assessing the students understanding of the
topic, the students ability to explain relevant ideas, and the degree to which the student
describes his or her reasoning (see figure 3). This rubric will be used to evaluate student
responses once at the beginning, twice throughout the research study and once at the end.






Running Head: Visible Thinking

15
Figure 4.
Thinking Rubric
4 Advanced The journal entry demonstrates a full understanding of the topic. The
student identifies relevant ideas and explains their reasoning in detail
with support.
3 Proficient The journal entry demonstrates moderate understanding of the topic.
The student identifies a relevant idea and explains their reasoning with
moderate support.
2 Basic The journal entry demonstrates minimal understanding of the topic. The
student identifies an idea, partially relevant to the topic. The response
lacks support or explanation.
1 Below Basic The journal entry does not demonstrate an understanding of the topic.
The student gives an irrelevant response with or without support.

The students will also be evaluated based on the Harvard Project-Zero continuum,
measuring students oral communication abilities (see figure 4). Students will be
evaluated on this rubric during a whole-group discussion once at the beginning of the
study, and again at the end of the study.











Running Head: Visible Thinking

16
Figure 5.



Selection of Group
The population of this study will be a fourth grade class of twenty-three students
ages 9-10. The sample will be selected by looking for students that had at least three
responses to weekly open-response questions on kidblog.org to use as pre-assessment
data before the implementation of thinking routines. Students who have other extraneous
Running Head: Visible Thinking

17
factors that might affect their change in responses will be removed from the sample. A
student currently adjusting ADHD medication levels will likely show a change from
medication management, and thus will be removed from the study. The sample size is
fifteen students, it consists of seven females and eight males. The sample size has
66.67% white students, 33.33% African American students; all students are between the
ages of nine and ten.

Schedule of Implementation
The study will take place over a twelve week time period. Students will have the
opportunity to use the thinking routines between two and four times each week. Open-
ended responses will be assessed at the beginning of the study and twice again at the end
of the study. The quality of student discussion responses will be assessed once at the
beginning of the study and once at the end of the study.
V. Results of the Study
Findings
This study was an attempt to research whether the use of thinking routines during
the inquiry block would lead students to writing open-ended responses of significantly
higher quality. The sample used in the study consisted of fifteen fourth grade students,
all between the ages of nine and ten. The sample was obtained through pre-assessments
of student responses. Those students who had three previous response assessments were
included in the sample.
Running Head: Visible Thinking

18
Each open-ended response was graded on a 1-4 point scale. See Figure 4, for the
thinking rubric. In the pre-assessment, each subjects three scores were averaged, the
sample exhibited mean scores ranging from 1-3. The mean score of the entire sample
was 2.483, the median 2.5. The standard deviation of the sample on the pre-assessment
was 0.506 and the variance was 0.256. The administration of the pre-assessment open-
ended questions took place during the weeks of December 5, 2011, January 9, 2012, and
January 17, 2012.
After the collection of pre-test data, administration of thinking routines began
based on the proposed schedule during the inquiry block. The post-assessment was
administered to students and assessed during the week of April 16, 2012. In the post-
assessment response, the sample scores ranged from 2 to 4 points, with a mean score of
3.205 and a median score of 3. The standard deviation was 0.799 and the variance was
0.638. Table 1 summarizes the data for both assessments given.
Table 1.
Mean Median Standard-
Deviation
Variance
Pre-Assessment 2.438 2.5 0.506 0.256
Post-Assessment 3.205 3 0.799 0.638

Additionally, student ability to demonstrate thinking verbally was assessed before
the treatment and again after the treatment using the Thinking Rubric in Figure 5.
Students were assessed based on six thinking continua: depth of thinking, focus, detail,
Running Head: Visible Thinking

19
levels of dimension, generative vs. restrictive thinking, and recognizing themes, in a
discussion forum on January 17, 2012. These thinking continua were assessed based on
Thinking Rubric, Figure 5, which is also assessed from a possible 1 to 4 points. The
mean results of the pre-assessment are as follows: depth of thinking: 1.933, focus: 2.533,
detail: 1.933, levels of dimension: 1.333, generative/restrictive: 1.4, recognizing themes:
1.733. The mean results of this pre-assessment are depicted in Table 2.
The students ability to demonstrate their thinking verbally was evaluated twelve
school weeks later on April 23, 2012 in a discussion forum similar to the pre-assessment
discussion, the only difference being the content discussed. The post-assessment means
of the thinking continua were as follows: depth of thinking: 3, focus: 3.333, detail: 3,
levels of dimension: 2.867, generative/restrictive: 2.867, recognizing themes: 3.467. The
mean results of this post-assessment are depicted in Table 2.
Table 2.
Depth of
Thinking
Focus Detail Levels of
Dimension
Generative/
Restrictive
Recognizing
Themes
Pre-
Assessment
Mean
1.933 2.533 1.933 1.333 1.4 1.733
Post-
Assessment
Mean
3 3.333 3 2.867 2.867 3.467



Running Head: Visible Thinking

20
Conclusions
The results of this study allow for the following conclusions to be drawn. First
the mean of students open-response scores improved by about twelve percent from the
beginning of the study to the end. A direct correlation between the use of thinking
routines and the improvement in scores cannot however be deduced. One unforeseen
factor, which may have also contributed to the improvement of scores, is the additional
training students received in structuring open responses. This training encouraged the
use of transition words and had students consistently support their responses with three
examples.
Second, the students ability to verbalize their thoughts has improved
significantly. The amount ranged from 20% improvement to 37.5% improvement.
Students were given the opportunity to express their ideas and opinions using thinking
routines between two and four times per week throughout the course of the study. No
other factors which might affect how students express their thinking were altered during
the study. Therefore, it can be deduced that the use of thinking routines contributed to
students ability to express their thinking verbally.
Recommendations
The following are recommendations made by the experimenter for others in the
field to carry out in their action research projects.
1. This study should be replicated in a similar fashion, using a control group and a
test group, one, which is exposed to thinking routines, the other, which is not. This study
Running Head: Visible Thinking

21
might need to take place across multiple classrooms to be done effectively. By using a
control group, it would be easier to see which group has higher open-response scores at
the end of the study.
2. The study should also be replicated at a time in the school year, in an environment
where students are not simultaneously being taught to construct open-responses and
practicing writing them in other subjects. It was impossible for the experimenter to
determine which factor contributed to the increase in open-response scores. In the next
study, it is imperative that thinking routines are the only factor contributing to
improvement in students open-responses.
3. Develop a thinking routine that easily translates into creating an open-response
would be useful in developing a relationship between thinking routines and open-
response questions. Most of the thinking routines currently being used are applicable for
verbal responses and writing, but not in formal situations. Developing and using a
thinking routine which follows the structure of a paragraph would allow students to make
a direct connection between thinking through a problem and writing an open-response
paragraph.
4. Finally, the development of a classroom model, which regularly uses thinking
routines to foster student thinking and learning, is essential for the successful use of
thinking routines. The primary use of thinking routines is not to make students give the
correct answer, but to help students access their thinking, regardless of the correctness of
their ideas. A classroom teacher must be prepared to allow students to come up with
incorrect ideas, and think or problem-solve their way through them. To have success
implementing these thinking routines, a classroom must embrace student thinking and
Running Head: Visible Thinking

22
opinions in all subject areas. All curricula used should foster this same type of thinking.
Certain subjects should not be thinking subjects while others are memorization-based.
Therefore, other researchers in the field need to determine a classroom model, which
supports student thinking and exploration across all content areas, through the use of
thinking routines and other structures within the curricular day. This model will help
students embrace thinking routines and benefit most from using them on a daily basis.

































Running Head: Visible Thinking

23


References

Barahal, S. L. (2008). Thinking about thinking: Preservice teachers strengthen their
thinking artfully. Phi Delta Kappan, 90(4), 298-302. Retrieved May 4, 2012,
from
http://pzweb.harvard.edu/vt/VisibleThinking_html_files/06_AdditionalResources
/SusanBarahal2008.pdf
Bloom's Taxonomy. (n.d.). Old Dominion University. Retrieved May 4, 2012, from
http://www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm
Gardner, H. (1993). Frames of the mind: the theory of multiple intelligences (2. ed.). New
York: Basic Books.
McKim, R. H. (1980). Thinking visually: a strategy manual for problem solving.
Belmont, Calif.: Lifetime Learning Publications.
Ritchhart, R., Palmer, P., Church, M., Tishman, S. (2006, April). Thinking routines:
Establishing patterns of thinking in the classroom. Paper presented at the
meeting of the AERA, San Francisco, CA. Retrieved from
http://www.pz.harvard.edu/Research/AERA06ThinkingRoutines.pdf
Salmon, A. (2010). Engaging young children in thinking routines. Association for
Childhood Education International , 86(3). Retrieved May 4, 2012, from
http://www.freepatentsonline.com/article/Childhood-Education/220467971.html