Академический Документы
Профессиональный Документы
Культура Документы
Royins Solano
Dr. Bude Su
TABLE OF CONTENTS
Introduction ..................................................................................................................................... 3
Methodology ................................................................................................................................... 3
Prototype ..................................................................................................................................... 3
Learners ....................................................................................................................................... 5
Instruction.................................................................................................................................... 9
Outcomes..................................................................................................................................... 9
Summary ....................................................................................................................................... 13
Appendices .................................................................................................................................... 20
.
IRRIGATION CONTROLLER TRAINING 3
Section I
Introduction
The Irrigation Controller training module was created with the objective of training
target audience have experience working on landscaping but not on irrigation. Approximately 28
employees fit this description. Currently the business does not have a process for training
employees on irrigation. Employees must learn the basic concepts of irrigation in order to
program a controller. Five learners participated in the evaluation of the training module.
For this tryout, the first module on How to program an irrigation controller was used.
The tryout measures the usability of the module, learning gains resulting from the instruction,
and users reactions through observations, pre- and post-tests, and a questionnaire. The tryout
will also help the designer understand areas needing improvement in terms of design, content,
Section II
Methodology
Prototype
The prototype used for this evaluation is the first part of my capstone project in How to
program an irrigation controller. It is focused on basic concepts and process. This training
module consists of four topics: Irrigation Controller Benefits, Types, Features and Basic Program
IRRIGATION CONTROLLER TRAINING 4
Sequence. The prototype module for this evaluation teaches how to program a popular residential
The prototype is accessible through computer and mobile devices laptop, tablet, and
smartphone. The prototype was built using Adobe Captivate and the learners accessed the
training through an online link. The duration of the training was approximately ten to fifteen
minutes. The module has audio instructions and provides the learners the option to read or to
listen to audio about each topic. There is opportunity for optional practice to help learners engage
with the course. At the end of the training, the learners complete a 5 question quiz. They receive
immediate feedback based on the chosen answer and a final pass or fail score. If they pass, they
are presented with a certificate of completion, otherwise they have infinite attempts to try again.
The prototype was designed with the intention that learners could get a first approach to
irrigation controllers without the fear of breaking something. Some pictures show actual jobs
completed by the company. See examples of the prototype slides in tables 2.1 and 2.2.
IRRIGATION CONTROLLER TRAINING 5
Table 2.1
IRRIGATION CONTROLLER TRAINING 6
Table 2.2
Learners
The learners were meant to represent the wide range of demographics for this
landscaping business. Employees share gender, language and occupation. Their age goes from 21
to 51 years old. They are not provided training in irrigation. Some of them claim having seen
work done on irrigation but have no experience of their own. The average age for the learners
was thirty-five.
Forty percent of the learners had some type of knowledge about irrigation. Helping to
replace a broken sprinkler was the most common interaction. Sixty percent of the learners had no
Experience
Table 2.3
High school is the highest education level achieved by the learners. They can read and
write but some senior employees have trouble with the computer interaction.
Tryout Conditions
The tryout conditions were designed to replicate the training environment of the
employees. Learners took the training independently on a windows computer with Internet
connection in the main business office. The designer stood up by their side to take notes for the
usability test and help them with any inquiry. He guided the learners through the correct order to
take the survey, tests, content and questionnaire. The learners did not repeat any part of the
Tryout Process
The tryout process was comprised of six parts: survey, pre-test, training, post-test,
questionnaire, and observation. The survey and questionnaires were created as a word document.
IRRIGATION CONTROLLER TRAINING 8
Then administered verbally in person. All information was confidential and anonymous.
Learners provided their names in the learning project but it was not recorded for reporting in
order to maintain confidentiality. Personal information only recorded demographics. The five
1. Survey
The learners responded to the survey first. The survey captured the demographic
information and any prior experience with irrigation. Learners were asked to provide their
occupation, gender, ethnicity, and age. They were asked if they had previous experience in
irrigation. The ones who said they had experience, explained they are rarely sent to help with
irrigation problems and clarified they have helped to replace a sprinkler but not yet to program a
2. Pre-Test
The pre-test was an integral part of the evaluation process since this captured the learners
knowledge and skill set prior to taking the training. The questions were designed to assess the
most basic concepts of irrigation and the sequence to program a controller. The questions
specifically addressed aspects that were taught in the training. For example, the benefits and
features of irrigation controllers. These questions were the same in the training modules pre-test
3. Training
The learners accessed the training through the link provided. They participated in the
training immediately after taking the pre-test. The web-based training was approximately ten
minutes in duration. You may access the training link in appendix 5.3.
IRRIGATION CONTROLLER TRAINING 9
4. Post-Test
Once the learners finished the training, they were instructed to continue taking the post-
test. The post-test questions were identical to the pre-test. Since the evaluation was assessing
knowledge before and after the training intervention, identical pre- and post-tests would be
appropriate for this evaluation. The post-test was also an essential part of the tryout process. In
order to measure any change in knowledge, I compared the scores of the pre-test with the post-
5. Questionnaire
The questionnaire was designed to capture the comments, reactions, and perceptions of
the learners. Most of the answers to the questions included a Likert scale. For example, one
question read: How confident do you feel in programming an irrigation controller? The range
of answers included: very confident, confident, undecided, not confident, and not confident at all.
Questions addressed the level of difficulty/ease in following the module instructions, the level of
difficulty/ease of the quizzes, and any level of frustration the learners may have experienced.
There were two open-ended questions that asked the learners to record anything they liked or
disliked about the training. See an example of the questionnaire in Appendix 5.4.
6. Observation
Five learners were observed individually in-person. The learners were contacted and
scheduled for observation during June. I approached the learners the day before the scheduled
observation to remind them of our session. The survey was in person and I provided access to the
tests and training. All learners accessed the tests and training from an office computer. I stood
IRRIGATION CONTROLLER TRAINING 10
near them and observed them taking the training on the computer. I created a checklist that I used
Section III
Results
Entry Conditions
The intended and observed entry conditions were not the same. Not all learners had
enough computer experience to feel comfortable with the mouse. I expected none of them to
have experience in irrigation because they are dedicated to landscaping, yet forty percent of the
learners said they had some type of knowledge about irrigation. Unfortunately, they did not pass
the pre-test. One of the learners faced a challenge using the mouse, maybe he is not a regular
computer user and felt cumbersome. He was among the senior employees so maybe age played a
factor here.
Instruction
The observed instruction was consistent with the intended instruction. Learners who
experienced trouble with the instruction were timely assisted. The learners followed the module
instruction, interacted with the practice, and completed the final evaluation. The most common
feedback from the learners was that the instruction provided very detailed information they could
not easily find such as the programming sequence. Most learners felt confident in responding to
the required tasks. The training navigation was intuitive for all but one of the observed learners,
Outcomes
IRRIGATION CONTROLLER TRAINING 11
Before I conducted the tests, I hypothesized that the training would improve the learners
knowledge in irrigation as reflected through higher test scores. The null hypothesis was that there
would be no statically significant difference in the pre and post test scores. I ran the paired two
sample t-test for dependent samples. The degree of freedom was 4. I used one-tail results for a
directional hypothesis. The t-test results are below in table 3.11. Since the test statistic: 13.88
was higher than the one-tailed critical value of 2.13, I reject the null hypothesis, and accept the
research hypothesis that there is in fact statistical difference. The training was a significant factor
Pre-Test Post-Test
Mean 1.2 4.6
Variance 0.7 0.3
Observations 5 5
Pearson Correlation 0.763762616
Hypothesized Mean Difference 0
df 4
t Stat -13.88044188
P(T<=t) one-tail 7.80959E-05
t Critical one-tail 2.131846786
P(T<=t) two-tail 0.000156192
t Critical two-tail 2.776445105
Table 3.11
Pre and post-test results (See Summary) show that learners achieved significant learning
gains as a result of completing the module.
While the learners had varying degrees of experience on irrigation some said they know
irrigation and irrigation controllers. Overall, learners experienced the most improvement in
Recommendations
1) Provide clearer instruction to navigate the course using the buttons. I assumed all
employees would be fine using a computer and a mouse but some of them had trouble
navigating the course. I will add more detailed instructions near the first slides of the
presentation.
2) Include more practice items. The current module only has one practice item in addition to
the lesson interactions of hovering the mouse over controller buttons to display the
description of each feature. It is recommended to have two practice items per topic to
allow the learners to get familiar with the content before getting to the quiz.
3) Motivate the learners. Not all employees seem excited about learning. Some of them
were avoiding the opportunity with excuses. For my capstone evaluation, I will try to
motivate them mentioning some other extra benefits of having this knowledge and make
it more appealing.
4) Include more test items. It would be good to include more test items so that learners can
have better chance to pass. With this version, the matching questions test for more than
one concept at the time. It could be good to include another question for each one of
those, so that participants can get partial credit if they dont get the first one right.
Section IV
IRRIGATION CONTROLLER TRAINING 13
Summary
The data was obtained through observation. I entered the scores from the pre- and post-test into
an Excel spreadsheet. The chart below demonstrates the scores for the pre- and post-tests. There
were five questions in total for the pre- and post- tests. All five of the learners scored higher
during the post-test. The average score for the pre-test was 1.2 or 24 percent. The average score
for the post-test was 4.6 or 92 percent. See table 4.1 for the distribution of the test scores. The
chart in table 4.2 demonstrates the difference between the pre- and post-test scores.
Pre-test Post-test
Scores Scores
2 5
1 4
2 5
0 4
1 5
Table 4.1
Learning Effectiveness
6
0
1 2 3 4 5
Table 4.2
Learners experienced the greatest improvement in their scores with questions 2, 4 and 5.
0
Pre-test Scores Post-test Scores
Table 4.3
During the pre-test, only two learners answered correctly. In the post-test, all learners responded
correctly.
Question 2: Match the controller feature with its corresponding description. See table 4.4.
IRRIGATION CONTROLLER TRAINING 15
Table 4.4
During the pre-test, only one learner answered correctly. In the post-test, four learners responded
correctly.
0
1 2
IRRIGATION CONTROLLER TRAINING 16
During the pre-test, only two learners answered correctly. In the post-test, all learners responded
correctly.
During the pre-test, no learner answered correctly. In the post-test, four learners responded
correctly.
0
1 2
During the pre-test, one learner answered correctly. In the post-test, all learners responded
correctly.
Usability Questionnaire
The learners attitudes and opinions of the training experience overall were quite positive.
Forty percent of the learners found that the instructions were very easy to follow. One learner felt
that the instructions were not clear and no one had a neutral response. See table 4.8.
Table 4.8
Eighty percent of the learners felt that the training very significantly contributed to
increment their knowledge in irrigation while twenty percent felt it significantly contributed. See
Table 4.9
The level of confidence that the learner felt to program a controller. Eighty percent felt
very confident, while twenty percent felt confident. See table 4.10
IRRIGATION CONTROLLER TRAINING 19
Table 4.10
The learners felt quite positive regarding their learning experience. See table 4.11. Sixty
percent felt absolutely no frustration, twenty percent felt hardly any frustration, and twenty
Frustration experienced
Table 4.11
IRRIGATION CONTROLLER TRAINING 20
Sixty percent of the learners felt that the quiz was easy, twenty percent felt the quiz were
more or less difficult and twenty percent felt that the quizzes were very easy. See table 4.12
below.
Table 4.12
Something learners mentioned they liked about the training, was the detailed information
provided. Something they cannot easily find, like the programming sequence.
Something they did not like was the audio getting stuck at random times and navigation buttons
Section V
Appendices
Have you ever worked with an irrigation controller? If yes, in what scenario? how often?
Manual-One Station
Permite activar un solo riego para una sola vlvula
(Una Estacin)
Water Days
Permite seleccionar los das o intervalos de riego
(Das de Riego)
IRRIGATION CONTROLLER TRAINING 22
Sensor ByPass
Desactiva el Sensor de Clima
(Ignorar Sensor)
3. Multiple Choice. Which of the following options are benefits of having an irrigation
controller?
c. Saves money
d. Provides convenience
http://itcdland.csumb.edu/~rsolanorodriguez/IST622/index.html
Questionnaire 5.4
1. Were the instructions easy to follow? Please choose your response from the drop down
menu. Circle only one option. Very easy. Easy. Neutral. Difficult. Very difficult.
2. How significant did the training contribute to your knowledge about irrigation
Circle only one option. Very confident. Confident. Undecided. Unconfident. Very
unconfident.
4. Did you experience any frustration while learning with the module? Circle only one
option. Lots of frustration. Some frustration. Undecided. Hardly any frustration. Absolutely
no frustration.
Circle only one option. Very easy. Easy. Undecided. Difficult. Very difficult.
6. What did you like about the instruction? Please type your answer.
7. What did you dislike about the instruction? Please type your answer.
Does the learner follow the 4 1 One learner was not familiar with
Does the learner have any 3 2 Some learners were not sure what
difficulty? buttons.
Does the learner encounter any 1 4 Yes, one learner noticed the audio
Does the learner encounter any 1 4 One learner was not familiar with