Summary of Land and Water Assessment Data

Fall 2014

IslandWood Homewaters
Haley Rutherford

IslandWood Homewaters Program offers three programs to Seattle Public School

teachers that fit into and reinforce certain science units (STC kits); namely
Ecosystems for 4th grade and Microworlds and Land and Water for 5th. For the
purpose of this summary, the Land and Water program assessments will be
evaluated. This is a one-day three and a half-hour program that includes four
stations concerning water quality, water velocity, salmon habitat surveying, and
salmon habitat mapping. The focus of the science unit is erosion and deposition,
with students manipulating a stream table to study influence to- and effects ofwater on land. Homewaters program enhances this by taking students to a stream
in their neighborhood and using salmon as a framework for investigation of the
health of the stream. Since our main assessment goal is to track change over time,
assessments were only evaluated if they had a matching pre- and post-assessment
from each student. Therefore, the numbers represented in this report are
percentage increases between specific pre- and post-assessments, rather than as a
whole. Based on the evaluation of these assessments, over 90% of the students
showed some sort of increase in knowledge or behavior change in terms of
AmeriCorps standards.

Every teacher that brings a class to the Land and Water Field Study is provided with
assessments to be completed by students before and after participation in the
program. The assessments are designed to assess Homewaters goals of:
1) improving students science learning by reinforcing and expanding concepts
they are taught in the classroom, and
2) increasing students environmental awareness and stewardship by increasing
their awareness of their connections to their local ecosystem and helping
them discover ways that people affect ecosystems.
Specifically, the assessments are designed to engage and evaluate students
knowledge of erosion, deposition, scientific processes, and local environmental
awareness and stewardship. While some questions are evaluated on a standard
point system, others are coded for types of answers offered. The questions are
posed and scored as follows:
1) Pre: What are you excited to learn about during the Land and Water Field
Study?
Post: What did you learn during the Land and Water Field Study?
Coded for answers, see Table 1.
2) What is the difference between erosion and deposition?
2 points possible- one for characteristics of erosion (fast water,
weathering, water moving/pushing, channel formed, etc.); one for

3)

4)

5)

6)

7)

characteristics of deposition (slow water, dropping off of

land/dirt/sediment, mouth/delta, etc.)
Fill in the blank: If I were to visit a creek, I would want to_____ because
_____.
Coded: 1 for playing/no stream, 2 for an observation-related answer
(touch, learn, look for something scientific, etc.), 3 for a scientific
answer (test, investigate, etc.), and 4 for a stewardship related
answer (pick up litter, help the stream, etc.)
If I was a salmon, I would want to live in a stream in my neighborhood
Check yes/no/maybe and Why?
Coded: 1 for a random answer (no stream, it is cold, close to family,
etc.), 2 for an answer related specifically to stream or park (too much
litter, lots of plants, etc.), and 3 for an answer related directly to
topics covered in program stations (evidence, salmon needs, chemicals,
turbid water, etc.)
Add to the list of things people can do to help a stream
4 points possible- one for each added to existing examples (picking up
litter and scooping dog poop) of things that would help the stream
and/or salmon, or changes to the stream that could help
What are the steps scientists take to answer a field study question?
5 points possible- one for each (possible answers include observe,
predict, procedure, test, investigate, analyze data, conclude, make a
model, etc.) or for examples of questions or form of procedure
Read the three statements below and circle one answer per statement
a. I believe that I or my family could do things in our daily lives to help
streams
b. I want to know more about my local streams
c. I think about the thing I do that might help or hurt streams
Coded: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 =
agree, 5 = strongly agree

Since assessments are administered by teachers, Homewaters staff has little control
over when or how the assessments are administered, which may influence the
reliability of the results. Possible influences include but are not limited to: proximity
in time preceding or following the field study, how far along the class is in the Land
and Water STC kit, encouragement, and assessment environment and guidelines
(individual work, time allowed, etc.) In an attempt to encourage completion and
return rates, teachers are given a pre-paid envelope for post-assessments if preassessments are brought to the field study, and a $25 Target gift card is supplied
when both have been returned to Homewaters. Because of these methods, we
found a 39% increase in return rate since last year, from 6% to 44%. While this is
significantly higher than the fall 2013 season, it has decreased since fall 2012 which
saw a 60% return rate. Out of 849 students taught in the Land and Water Program,

376 completed and returned a pre- and post-assessment. While this can certainly be
improved, it is a large enough pool for us to appropriately evaluate our influence on
Seattle students science learning and stewardship behavior.

Question One: What are you excited to learn about/did you learn during
the Land and Water Field Study?
For question one in the pre-assessment, we saw a large variety of types of answers.
It was clear that while some students were far along in the unit or had a teacher
who was prepared and perhaps attended our program before, others were
unprepared for the field study. The answers were coded and majorities were
grouped together as shown in Table 1, with percentages of students whose answers
related to that topic. These answers are not correlated but simply show the most
common responses and could influence the development of this program and its
pre- and post-lessons. It is worth noting that our salmon lens dominated the
reported learning, though it isnt a surprise since many students have yet to learn
about salmon in school. In addition, it is interesting to find that all of the station
topics showed up as significant learning except the human influence focus we are
currently working on revitalizing.
Table 1. Percentages of most common answers for question one (What are you
excited to learn about/did you learn during the Land and Water Field Study?)
grouped together. Pre- and post- answers are not correlated. Percentages are
rounded.
Pre-Assessment
Post-Assessment
16% erosion and/or deposition
23% salmon habitat needs
16% salmon
20% water quality station
12% land and/or water
12% water speed/velocity
8% stream
8% salmon
7% real life
6% erosion and/or deposition
Answers relating to the stream. Examples include how streams work, where they go, etc.
Answers include learning outside, relations to stream table, etc.
In the water quality station, students test the temperature, turbidity, and pH or the water.

Question Two: What is the difference between erosion and deposition?

Based on reflections from the 2013 Fall program assessments summary, question
two was adjusted from What causes erosion and deposition in a stream? with one

point allotted for an answer addressing each process. Thirty-one percent of students
improved their scores on this question with 6% of the students increasing by one
point and 25% increasing by 2 points. This is a significant amount seeing as many
students were unable to gain additional points because they were awarded the full
amount (two) on the pre- and post-assessment. While this re-wording addressed the
problem of students only answering for one process, it proved to be difficult to
evaluate answers that indicated the relationship between erosion and deposition
(i.e. different speeds). For those students who were unable to earn more points, the
reason was more often than not that they started out with 0 in their preassessment and showed no learning in their post-assessment (rather than starting
with full points).

Question Three: Fill in the blank: If I were to visit a creek, I would want
to ___ because ___
The third question was coded and evaluated for behaviors students would
demonstrate ranging from playing to observing to participating in something
scientific to acting as a steward. We were looking for changes from activities
focused on playing or observing to the Homewaters-specific goal of making science
accessible or IslandWoods overarching goal of stewardship behaviors. From 1
(playing) or 2 (observing), there was a 12% change to scientific actions and a 22%
change to stewardship (see Table 2.). Once again, some disparity can be attributed
to students who answered a 3 (scientific) or 4 (stewardship) in the pre- and
post-assessment. It is encouraging to highlight the significant increase in students
who in the post-assessment reported wanting to participate in a stewardship-related
activity, seeing that stewardship is a high priority for IslandWood as a whole.
Looking forward, it may be interesting to see if students would answer the question
differently if they were asked to imagine themselves visiting the creek as a scientist.
This may engage them to think more about investigations and observations, rather
than resorting to answers they may personally enjoy or find silly (i.e. go swimming
because it is fun).
Table 2. Percentages (out of 374- total amount of students assessed) of increase in
score for student answers from pre-assessment to post-assessment. Pre- Totals
shows the total percentage of students to indicate that answer in the preassessment; Post- Totals shows the total percentage of students to indicate that
answer in the post-assessment. Percentages are rounded.

Preassessme
nt
Playing
Observing

Post-assessment scores
Playing
Stewardship
8%
4%

Observing
5%
19%

4%
8%

Scientific
5%
14%

PreTotals

PostTotals

22%
45%

13%
27%

Scientific
Stewardship

0%
1%

2%
1%

3%
2%

4%
8%

9%
12%

17%
41%

Question Four: If I was a salmon, I would want to live in a stream in my

neighborhood
For question four, the change in points was recorded as well as coding for preassessment answers. It also incited a large range of answers, from never having
been to a stream to the neighborhood being dangerous to being familiar with their
surroundings. The largest reason stated for not wanting to live in their local stream
at 21% was dirty water/pollution/litter. Most other answers were random, majority
including something about being eaten or being in familiar surroundings/close to
family. In the future, re-wording is needed to adequately clarify to students that it
isnt asking if they want to be a salmon, but where they would choose to live if they
were already a salmon.

Question Five: Add to the list of things people can do to help a stream
Question five is a fairly open-ended assessment of students awareness of the
positive influences humans have on a stream. We recorded a 7% increase of threefour points in relation to 0-1 points in the pre-assessment (see Table 3.) While some
of the discrepancy can be accredited to students given three to four points in the
pre-assessment, other examples include an absence of discussion within the
students field study group, lack of understanding of the question, or an addition of
one to two answers to the list. Since 7% is not a significant number (26 students out
of 376) and human influences relate directly to one of Homewaters goals, in the
future a lesson could be dedicated solely to human influences. There were some
students that offered less answers on the post-assessment, but it seems likely that
they simply did not want to write them out a second time (rather than losing
knowledge). This data reinforces our desire to include more information for students
in the field study about how they impact the stream. While this material is currently
in the curriculum, it is being lost due to time limitations.
Table 3. Percentage increase in points from the pre- to the post-assessment, not
taking into account original pre-assessment answer. Answer Increase equates to
the increase in amount of items reported on the post-assessment list; Percentage
Increase shows the percentage of students with said change in points awarded.
Percentages are rounded.
Answer Increase
Percentage Increase
No change
56%

1
2
3
4

25%
12%
5%
2%

Question Six: What are the steps scientists take to answer a field study
question?
Understanding of the scientific process is something Homewaters strives to
encourage and reinforce in students as a way to indicate scientific learning. There
was a 9% increase of three points for students, a 4% increase of four points, and a
1% increase of five points. More of an intentional emphasis on the steps should be
included in the future. Looking at the increase in points awarded, it seems that the
scientific process should either be more present in the lessons or eliminated
completely for a different assessment of science learning. Given that, there was an
improvement shown from about half of the students (counting the one- and twopoint increases). This is adequate but can clearly be improved; it is another focus of
the curriculum that is currently present but could be more explicit. Additionally, in
the future we should assess the starting and ending knowledge reported for this
question to more accurately obtain information.

Question Seven: Read the three statements below and circle one
answer per statement.
a. I believe that I or my family could do things in our daily lives to help
streams
b. I want to know more about my local streams
c. I think about the thing I do that might help or hurt streams
After assessing the answers provided for question seven, a suggestion would be for
Homewaters to re-evaluate the value of them. While they provide interesting
knowledge, many students scores actually decreased (for example from agree to
disagree) and it is unknown whether it was because they simply did not
remember their previous answer or if in fact they actually were less interested in
those topics. Rather, there could be a simple question of engagement included in
its place assessing students desire to visit the stream or park. This would fit better
into our goals as an organization, as well.

Notes and suggestions

For fall 2014, the pre- and post-assessments were adjusted and adapted to better
capture our goals and results that Homewaters is interested in. This is the main
reason for a lack of evaluative information; seeing as we were coding the answers
as we went. Something to discuss for future assessments is the necessity of each
question. The assessments are lengthy and some of the questions did not turn out
as useful as we expected them to. In addition, when scoring the assessments more
care could be given to recording the pre- and post- answer rather than the
difference between the two. Although this isnt captured in numbers, the fact is that
most students started with no points scored in the pre-assessment rather than full
points, so an increase in efforts to educate students on topics such as
erosion/deposition and what people can do to help a creek would be beneficial. It
might also be advantageous to go forward assuming the students know very little
about these topics, rather than assuming they do (since it is a part of the land and
water science unit).

While a substantial amount of students (96%) showed some form of learning or

behavior change, the change was not as significant in detail (most was in the form
of one additional point or step). Looking ahead, adjustments to the program lessons
and teaching techniques could be considered to improve student learning on a more
weighty scale. Efforts can be put towards emphasis on human influences and the
scientific process. Perhaps this can be done in a more financially sustainable manner
through the use of pre- and post-lessons administered by a Homewaters Educator in
the classroom. This would reduce teachers responsibility, which is presumably one
of the factors influencing the likelihood of receiving assessments. However, the
learning revealed by this assessment analysis proves our goals, as well as the
teachers and school districts goals, are being met and in some cases surpassed.