MyWriting Lab Evaluation Plan

MSD Wayne Township Language Arts Task Force

26 September 2013
Kristina Miller
Introduction

Pearson is a British multinational publishing and education company. Headquartered in London, it
is by far the largest education company and book publisher in the world. Their goal is to engage
students effectively via immersive content, resources, tools, and experiences. They have very high
standards for instructor and student learning and realize that not every student learns the same
way or at the same rate. Understanding the high expectations and demands Pearson has designed a
program with various products that will enable learning on a more personal level for the learner
that can be used as a tool to build mastery or the basic foundations of reading and writing. In fact,
with this adaptive learning focus, continuous evaluation of student performance and activity takes
place. This will improve the ability of the instructor to provide a blended learning environment that
focuses on targeting each students strengths and weaknesses (Pearson, 2013).
Due to the high failure rate and lack of attentiveness in most classrooms, the MSD Wayne Township
Language Arts Task Force wants to provide students with additional resources that are progressive
and differentiated to increase their performance levels on the common formative assessments and
essays required to write in the classroom. This will improve student performance and
understanding through personalized study plans and make them aware of their true potential and
the strategies they can use to be successful.
The Language Arts Task Force identified the MyWriting Lab as a highly effective and efficient tool
to use in the Language Arts and English classrooms when writing essays during composition
sections. This tool is located at the following link: http://pearsonmylab.com. Before implementing
this product in all Language Arts and English classrooms throughout the district, the members
would like to verify the reliability and quality of this tool to meet the objectives of the Language
Arts/English classroom, and well as the dual credit Reading Techniques and Composition
classrooms as well. If this tool can meet some or the entire gambit of objectives to improving
grades, engaging them and creating a personalized study plan, and the students reach a true
understanding within their courses with reliable solutions, it will be integrated into the program of
study in order to improve student performance rates on standardized testing, to meet the
objectives of each courses objectives, and to increase the performance level while understanding
learning gaps in order to make great strides in the entire Language Arts/English program within
MSD Wayne Township. If it does not prove to be effective, the curriculum will be designed without
the use of this particular tool.
This is real time, utilizes technology, and allows for tracking data while personalizing content for
each student. The individual learning path reinforces the content and concepts to produce better,
more proficient results by the student. With the need to adapt, demographically and dynamically
for student population, instructors must meet student needs and show gains in learning.
MyWriting Lab offers more control over the content and course itself while providing
management features such as a gradebook to track progress and see deficits or gains. Since this is
personalized for the student and online, students are given a plethora of practice and immediate
feedback on their individual performances. Improving writing skills via skill remediation in
grammar and punctuation, paragraph development, essay development, and research and improves
students overall writing. There are four modules within the course product, and all exercises are
graded through automatic scoring by Pearsons proven Intelligent Essay Assessor (IEA). The four
modules consist of: Sentence to Paragraph, Paragraph to Essay, Essay, and Combined-Sentence to
Essay.
Audience & Context

The audience of this learning tool will be students in the Reading Techniques and Composition
(RT&C) dual credit course at the Ben Davis Ninth Grade Center. This sample group has been taken
from all of the Language Arts/English classes because it will reflect the diversity of MSD Wayne
Townships student population and the varying degrees of writing abilities within it. There is an
even combination of males to females as well.
All of the students were accepted into the University high school track program for their freshman
year. This means they will be expected to write at a college level for their next level of
coursework, providing they earn a seventy percent or higher in their RT& C course. Some of the
students have a basic understanding of the writing process and varying types of essays, but many
have not been exposed to writing on a daily basis with high expectations. As a result, they may not
be aware of how to write coherent essays that include all of the criteria asked for.
A sample group of students have already been made aware that they will be participating in a
sample group study and will be sent the link and student code to the MyWriting Lab product via e-
mail. As indicated previously, this sample group represents the diversity of MSD Wayne
Townships current demographic. The e-mail will contain the instructions on expectations and
objectives and they will be required to complete two modules within a four-week timeframe:
Module 2 Paragraph to Essay and Module 3-Essay. These modules are to be completed within four
week of receiving the link via e-mail at their own pace. Setting aside a chunk of time where they
will not be disturbed is highly recommended, or providing the students with headphones or ear
buds to listen and tune others out within the classroom while completing the modules is
recommended.
Evaluation Instruments and Procedures

Reaction Measurement
A link to an online survey created with Survey Monkey. The sample group will be instructed to
follow the link to the survey again after completing the two modules. Collecting, analyzing, and
culminating the data online will make it easier and a truly inexpensive endeavor. The students will
not be asked to provide their names, the survey will be anonymous, and the reasoning behind this is
that if the students do not feel as if they will be targeted and answer the questions freely in hopes to
provide honest and critical feedback on the MyWriting Lab learning tool, as well as provide a
snapshot of the reactions of diverse student population so that we can record more accurate
measurements of specific reactions and troubleshoot prior to implementing the lab into the
Language Arts/English curriculum. The survey was created as to not pressure the students toward
a particular response, but an honest one.
In order to elicit honest and accurate responses, the questions focus is on the tool only. Topics vary
from the ease of navigation, the content itself, how engaging the activities are, did it meet what they
felt were their deficits, and after the two modules did the student feel better prepared to write
more proficiently and meet the expectations for writing. Did they feel the tool increased content
knowledge and allowed them to explore the writing at their own pace, and also whether the student
would actually recommend MyWriting Lab tool to other students? Learner feedback will provide
the Language Arts Task force with the data needed to determine the issues with the product from a
student perspective, which will provide insight as to how to modify the implementation to meet
individual needs of the students and teachers. Determining the affects of the tool is invaluable in
determining measurable gains in student learning outcomes and learning styles as well.
There will be a total of eight questions using a 5-point Likert rating system and open-ended
response questions. With only eight questions, and the implementation of the 5 -point Likert rating
system, the range of responses on the quality of the tools user interface will be determined, and the
open-ended questions, will provide an appropriate range of responses on the quality of the tools
content and activities. It is anticipated that there will be a 100% response by only using eight
questions, which will also ensure that the sample group will not just give random responses due to
the small number of questions, and the answers will be more thoughtful because of this. An
opportunity to make comments and suggestions as to how students would like to see this
implemented will be given at the end of the survey. What students feel are strengths and
weaknesses of the MyWriting Lab tool will allow the Language Arts Task Force to make informed,
student driven, and results driven decisions about implementation of the tool for all Language Arts
and English classroom throughout the district.
For the Likert scale questions, a standard of 5.0 will be used to determine whether the tool meets
the needs of the RT&C student sample groups needs satisfactorily. If responses fall below an
average 4.0 rating, the Language Arts Task Force will need to reflect on the development of a
training program for Language Arts and English teachers that integrates the tool and addresses and
comes to conclusions about how to resolve any issues the sample group provided insight on.
Student responses that fall below an average of 3.0 will need to be analyzed and remedied or
reconsidered if the product is implemented as an instrument within the district.
See Appendix A for the survey example.


Quantifying Learning
The product offers a comprehensive assessment of student writing skills in grammar (diagnostic testing).
Each student is provided with an individual learning path based upon the results of this assessment given as a
pretest. The data from this assessment not only creates a learning path, but it also identifies the modules in
areas where the student most needs assistance.
The program is a progressive learning environment. Students will complete recall (literal
comprehension), application (critical comprehension), and writing exercises (demonstrate
concepts) within each module. This flow or progression of critical thinking enables students to
master the skills and content vital to becoming improved and successful writers. Students monitor
their progression via the online gradebook available online that provides detailed reports on scores
for all of the exercises within each module of the product course. Teachers can also access reports,
class summaries, and then track the progress of their entire class as well as individual student
details, as well as views and alerts.
To measure improvement in learning, the objectives the Language Arts Task Force identified in the
questionnaire will be included in the design. It will contain ten questions with five per module that
will be anonymous like the previous. Understanding the fact that the students will be taking the
survey without an evaluator present, some of the students may just rush through without basing
their answers off of what they really learned. The e-mail sent to each student will explain that the
surveys purpose is to collect data on MyWriting Lab, not the individual student, and it will be
anonymous.
The survey will be given to a control group. This control group has the same number of students as
the sample group and represents the districts diversity and individual student characteristics. E-
mails will with a link to the survey to be completed without even participating in or viewing the
MyWrting" Lab instrument the sample group is participating in. This is done to collect data that
will be used as a foundation or baseline metric, allowing for a comparison to be completed for the
pretest (diagnostic test) results for the sample group to determine whether the sample group
already had significant knowledge on this topic. Comparisons to the posttest results of the sample
group will determine what gains the MyWriting Lab tool actually made.
The RT&C sample student group will be instructed to follow a link to this survey prior to signing
into MyWriting Lab tool. No answers will be provided to them that would indicate correctness in
responses. It will only serve as the foundation or baseline to compare in contrast to the results
following completion of the two modules. This will illustrate the learning gains as a result of
employing the tool. When the student has completed both modules, the RT&C sample student
group will be asked to go to a link to the exact survey to measure the results.
Posttest results will be compared to the RT&C student sample groups pretest results to determine
gains in learning were made via completion of the two modules selected. Another data comparison
of the pretest results will be done in contrast to the control groups results as to determine whether
or not there is a significant difference in the two groups that may be distorting or interfering with
the outcomes. Posttest results to demonstrate the difference in results shown by comparing with
the control groups outcomes.
If the results of the posttest are lower than an average of 8 correct answers, the task force will need
to develop a training program to integrate the MyWriting Lab tool and figure out a way to
overcome some of its oversights prohibiting student gains. On the other hand, if the results of the
posttest are lower than an average of 7.0, the task force will need to decide whether the instrument
should be used in any capacity and design a plan to modify curriculum accordingly.
See Appendix B for the survey example.
Reporting Data

The reaction data will be reported on the RT&C student sample groups average ratings for each
individual question. To prevent further review of the tool, the baseline or minimum standard set is
an average of 4.0 for each question. Average responses that fall below 4.0, will require the task
force review the product prior to integrating the tool, and if any question falls below an average of
3.0, will require an analysis of the expectations and objectives to decide whether the tool should be
used at all. The data will be displayed via a bar graph for each exercise within the modules, as well
as each students results of their performance.
By comparing the average number of correct responses from the posttest results to the pretest
results, the learning data will be reported. It will compare the control groups test results to the
RT&C student sample groups pretest and posttest outcomes. And, like it is stated above, if the
posttests average results are below an average correct response rate of 8.0, the task force will need
to develop and design a curriculum that integrates the tool, but if the results are below an average
of 7.0, the a review should be conducted and a determination of whether the tool should be utilized
in any capacity. These results will also be presented via a bar graph that will display the average
number of correct responses between the control groups outcome data, pre and posttest data.
Determining the effectiveness of MyWriting Lab learning tool, will be proven through
comparisons of Kirkpatricks level one reaction and level two learning data, and it will be used to
verify the effectiveness of the MyWriting Lab learning tool in meeting the objectives of MSD
Wayne Townships Language Arts Task force primary goal to improve student writing knowledge
and skills as related to essay writing. Dependent upon the results of the evaluation, MyWriting
Lab by Pearson will either be implemented into the Language Arts curriculum as a supplemental
tool for teaching writing, integrated into a training program created by the task force for Language
Arts or English teachers, or not used at all for the writing curriculum initiative.




Appendix A
Rate your writing ability
o 5-Excellent- Exemplary
o 4-Good- Proficient
o 3-Average
o 2-Okay
o 1-Poor
How easy was it to navigate the MyWriting Lab site to access the modules and the content?
o 5-Very Easy
o 4-Easy
o 3-Neutral
o 2-Difficult
o 1-Very Difficult
How engaging were the activities and the content of the modules?
o 5-Very Engaging
o 4-Engaging
o 3-Neutral
o 2-Bored
o 1-Very Bored
How proficient were you at developing an essay on your own prior to both modules?
o 5-Exemplary
o 4-Proficient
o 3-Neutral
o 2-Needed some help
o 1-Inadequate
How prepared do you feel in starting and finishing a proficient or higher essay after the modules? \
o 5-Very prepared
o 4-Prepared
o 3-Neutral
o 2-Unprepared
o 1-Very unprepared
How much do you feel you learned from the two modules?
o 5-A great deal
o 4-A good amount
o 3-Some
o 2-Not too much
o 1-Nothing at all

What new information did you specifically learn in both modules?
How did the diagnostic test help you come to a better understanding with your skill level?
When you are struggling during the lab, what tools within MyWriting Lab was helpful?
How likely are you to recommend MyWriting Lab to other students?
5-Very likely
4-Likely
3-Somewhat likely
2-Not really, but I might
1-Not at all



Appendix B
These questions are online within MyWriting Lab




Appendix B continued
The following is an example of the survey questions for knowledge and skills.
Choose the best thesis statement for each essay prompt.
1. What are the benefits of participating in a sport?
o A. Some benefits of participating in a sport include good health, working on a team, and a
competition.
o B. Some benefits of participating in a sport include having good health, the ability to work on a team,
and a competitive spirit.
o C. Some benefits of participating in sports include having good healthy, the ability to working on a
team, and a competitive spirited.

2. Which is better: classroom-based learning or online learning?
o A. Some benefits of online learning are ease of scheduling, lower costs than classroom-based
learning.
o B. Compared to classroom based learning, online learning is easier to schedule, costs less, and has
more flexible studying options.
o C. Some benefits of online learning are its scheduling, cost, and flexibility.

3. Does technology help students learn more efficiently?
o A. New technologies such as iPad applications allows students to learn more better, and more depth.
o B. New technologies such as iPad applications allow students to learn better, faster and more in
depth.
o C. Having students use new technologies such as iPads applications enable they to learn better, faster
and more in depth.

Choose the BEST writing process term for each of the examples.
4. The paragraph in an essay which may restate the thesis statement, give an opinion, predict the future, offer
suggestions, and connect the previous paragraphs in a coherent way.
o A. Introduction
o B. Conclusion
o C. Exposition
o D. Body

5. An essay written before the final draft is completed.
o A. First draft
o B. Revisions
o C. Brainstorm
o D. Editing


6. In a 5-paragraph essay, 3 key ideas are existent, one for each body paragraph.
o A. Paraphrase
o B. Concluding sentence
o C. Thesis statement
o D. Supporting details

7. The paragraph in an essay that leads the reader as to what the essay is will be discussing and how it will be
organized is called
o A. the introduction
o B. the conclusion
o C. the body
o D. the thesis statement

8. Which of the following is not a typical of a paragraph?

o A. It is a series of sentences about one central idea.
o B. It is comprised of three to five sentences.
o C. The topic sentence introduces the subject of the paragraph and shows the writers attitude
towards the subject.
o D. The body contains details that support the topic sentence.

9. What is wrong with the following thesis statement? In this essay, I will explain why we should recycle.

o A. It makes an invalid point.
o B. It is too narrow.
o C. It announces the topic.
o D. It is an incomplete thought.

10. Which of the following is not something you would do to organize your essay ideas?

o A. Choose three or four of your supporting details that are most gripping and most clearly
support your statement.
o B. Organize your ideas in a logical manner: chronologically, spatially, or empathetically.
o C. Group supplementary ideas together.
o D. Organize your supporting details in the order that seems most logical to you.









Appendix C
The following are examples of the two e-mails that the control group and sample group members
will receive.
Control Group E-mail
Dear Student,
We would like to thank you for agreeing to participate in this survey. You have been selected to
participate in a survey to help the Language Arts Task Force of MSD Wayne Township make some
critical decisions concerning the Language Arts/English curriculums for the 2014-2015. At the end
of this e-mail, there is a link to a ten-question survey that we would like you to complete. It should
only take you 5-10 minutes to complete it. Please make sure we get it back by the end of the week.
We would like to assure you that the survey is completely anonymous and will not be used in any
way except to help make decisions about future program implementation into the Language Arts
curriculum. MSD Wayne Township and, in particular, the Language Arts Task Force appreciate
your contribution and efforts.
http://www.surveymonkey.com/s/VBSJLZRRegards
Best,
The MSD Wayne Township Language Arts Task Force












Appendix C continued
Sample Group Email
Dear Student,
We cant thank you enough for agreeing to participate in this survey. All surveys are anonymous
and the information gathered will be useful and appreciated greatly. Furthermore, the information
gathered will be sued to assist the MSD Wayne Township Language Arts Task Force with decisions
concerning the 2014-2015 Language Arts/English curriculum. Before you access the MyWriting
Lab website, you need to complete the initial survey. Once you have completed the initial survey,
please begin to complete the modules selected from MyWriting Lab. After completion of the
modules, please complete the final survey.
Please, follow the directions below:
1. When you first enter the MyWriting Lab website (http://pearsonwritinglab.com) and
enter your passcode, you will be directed to complete a diagnostic assessment. Your results
will allow the task force to compare your results after completing the modules.
2. After you complete that survey, you can access the two modules.
Module 2 Paragraphs to Essays
Module 3- Essays
3. After completing the two modules, take the two following surveys. Again, these are
completely anonymous.
First Survey: http://www.surveymonkey.com/s/VBSJLZRRegards
Thank you for taking the time and effort to complete this process. It will provide valuable
information to the district.
Best,
MSD Wayne Township Language Arts Task Force