Академический Документы
Профессиональный Документы
Культура Документы
Evaluation Report
The main goal of the Center for Teaching and Learning (CTL) at the University of
Georgia is to develop support materials and provide valuable assistance for faculty,
graduate teaching assistants and students on the ever-changing online instructional
technology. The CTLs help webpage, http://www.ctl.uga.edu/elc provides eLearning
Commons (eLC) learning materials designed to assist users in the form of videos,
printables, and other various step-by-step tutorials.
The evaluators have been asked to survey faculty and make a formal
recommendation on how CTL can rebuild, organize and streamline its help webpage in
order to make it more user friendly and easier to navigate.
2. Evaluation Purpose
3. Stakeholders
Georgia is also a stakeholder in this evaluation since the final changes suggested will
directly affect the usefulness and accessibility of the help webpage.
The faculty and Primary and The usefulness A survey will be sent out
teaching assistants Secondary and accessibility to various faculty
of the University of of the eLC members to collect data
Georgia learning materials on the usefulness and
accessibility of the eLC
learning materials. The
targeted audience will
be the faculty of the
University of Georgia.
4. Logic Model
Context
The center of teaching and learning at UGA wants to seize the opportunity to
update their faculty support page for eLC prior to the release of a major eLC update that
will change the look and feel of the system.
Resource/Inputs
The center for teaching and learning has staff working on resolving this issue.
The staff has computers along with the software and hardware necessary to complete
the task. The staff also have access to faculty, evaluators, and web designers to assist
with the task.
Activities
First, common eLC questions will need to be compiled. Resources and guides
will be developed to aid with these common questions. These guides will be made
accessible through the CTL eLC webpage. Finally, a system for faculty to ask questions
will be installed.
Outputs
A common question list will be produced as well as resources and guides to
match each question. The CTL eLC website will be updated and a feedback and
question service will be made available to faculty.
Short-Term Outcomes
The faculty will become more self sufficient in answering their questions using
the resources and guides provided. The availability of 24/7 help will make more faculty
confident that they can effectively utilize the system and will encourage additional
faculty to begin using eLC.
Long-Term Outcomes
Faculty will be able to apply their acquired knowledge to their future courses.
Being more knowledgeable in the eLC system may allow faculty to increase their quality
of teaching and in turn increase the quality of their students learning.
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
5. Evaluation Questions
We expect that the questions to be investigated will give us insight into the
effectiveness of the eLC support site. This evaluation will address the following
concerns of stakeholders and attempts to meet the need of updating the Center for
Teaching and Learning eLearning website.
1) Is there any high frequency content missing from the eLC page?
a) Are users able to find the information they are looking for?
b) Do users submit tickets for similar topics?
i) Would a categorizing/logging method for ticket sorting be useful?
2) What is the overall satisfaction of UGA faculty with the eLC page?
a) Are users able to quickly and easily find the appropriate information?
b) If users have to submit a Help Center ticket, is a response received in an
optimal time frame?
3) What user feedback has already been given to managers about the eLC page?
a) Is there evidence that user feedback given during the 2016-2017 school
year has already been implemented into the eLC Help website?
b) Have any tickets submitted during the 2016-2017 school year already
been resolved?
Data gathered in response to these questions will provide the necessary information to
the client and the team to make decisions that positively impact the usability and
effectiveness of the eLearning website.
The team has broken down the tasks displayed in Table 5 between the team
members. Lana will be leading evaluation question 1, Davina will be leading 2 and 4,
and Jalonna will be leading number 3. The team will be in regular communication
throughout the process via online meetings and text messaging. The documents
created throughout the evaluation process will be a group effort. Upon approval of the
evaluation plan first with Dr. ChanMin Kim and subsequently with Sherry Clouser
implementation will begin. After the evaluation period is complete an Evaluation Report
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
will be submitted to Dr. ChanMin Kim for revisions and upon completion of a final
version it will be turned into the client, Sherry Clouser. Please see the spreadsheet
below for the project timeline and estimated time allotted to each task.
7. Limitations
This evaluation was specific to the University of Georgias Center for Teaching
and Learning (CTL). For the most part we received very positive responses from our
surveys and the data we collected however, there were a few limitations that occurred
during the implementation of our plan. The first limitation was the time we were given to
complete our evaluation. We believe that given more time, we could have opened our
survey to a larger percentage of the Faculty, Teaching Assistants and Student body and
included more responses. A second unforeseen limitation was the lack of response from
the Faculty and Teaching Assistants that we invited to inview. As we developed our
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
Evaluation Plan we also became aware that it is not plausible to include a link on the
eLC help page that will provide an answer for every question that might be proposed by
a user of eLC.
We utilized diverse data collection methods and sources in order to gain the most
complete insight into the needs of the stakeholder. We utilized an online survey,
eLearning support website statistics, and faculty-to-faculty e-mail correspondence.
Online Survey
We did not have direct access to evaluation participants such as UGA faculty
members so we gathered data via an online survey produced in Google Forms. Our
interview with Dr. Clouser suggested that we should utilize her access to the UGA
Teach listserv in order to increase the number of possible participants in the survey. We
allowed the participants two weeks to complete the survey. Participants used their own
devices to complete the survey within the allotted time. The survey consisted of 6 the
Likert-type scale questions and 2 open-ended questions:
Figure 1: Center for Teaching and Learning (CTL) eLC Support PAge Online Survey
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
E-mail Correspondence
Seasoned UGA faculty have a level of comfortability with Dr. Clouser and may utilize
standard e-mail to communicate any issues they are experiencing with eLC. We
analyzed specific emails directly relating to issues that may impact the effectiveness of
the eLC support site and used the results of our findings to help support any trends
discovered while investigating each Evaluation Question.
Is there any high DHC Tickets DAT-L Weekly Download and sort
frequency content Support Logs through excel
missing from the eLC (existing) spreadsheets
page?
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
9. Results
Over the duration of 1/1/17 to 3/24/17, 31 DHC (D2L Help Center) tickets were
submitted. Ten of these were submitted by faculty, 20 by students, and one by a CTL
staff member. Seventeen of these 31 (54.8%) tickets collected were common questions
that the CTL page has the potential to resolve. Seven out of the 31 (22.5%) could have
been resolved by current material on the CTL page. Four topics have been identified as
potential items to be added to the CTL page: I am having trouble logging in. Who can
help?, How to import quizzes, How to grade quizzes, How to release grade items &
date restrictions.
Over the duration 6/14/2016 to 4/03/2017 378 Central IT/CTL Tickets regarding
the eLC were submitted. The Graph Below shows how these tickets are broken down
into more generalized categories. The Login/Access Issues is the largest category,
followed by the Miscellaneous category. The Miscellaneous category was made up of
the tickets that were individualistic and required case by case assistance, for example:
a) submitting application materials online for the Art Department, b) a cross listed
course no longer showing up in the eLC and c) password issues.
The pie chart below displays the survey results analyzed to reflect the level of
overall satisfaction of the users. The percentages displayed in Graph 4 show the
mathematical equivalence to the number of points scored out of 30 in the survey.
10. Discussion
The collection of DHC tickets revealed that four topics could potentially be added
to the CTL eLC help page. These topics include: I am having trouble logging in. Who
can help?, How to import quizzes, How to grade quizzes, How to release grade
items & date restrictions. It was found that many of DHC tickets were specific questions
that would be best answered by the DHC and not the CTL eLC page. For example,
many questions were related to inquiries about if the eLC had specific capabilities or
questions related to an individual class scenario. Thus, the majority of inquiries are
being directed appropriately. It was found that only 22.5% of inquiries have the potential
to be answered by the current CTL eLC page. If the suggested additions are in fact
made, that percentage would rise to 54.8%. These additions would ensure that
EDIT7350E eLearning Evaluation & Assessment | Dr. ChanMin Kim | Spring 2017
potentially twice as many inquiries from eLC users could get the assistance they need
instantly from the CTL eLC page without having to contact the D2L help center where it
could potentially take days to get their questions answered.
The collection of the Central IT/CTL Tickets revealed that the most common
questions fell into the following categories: 1) Designing eLC Classes 13.8%, 2)
Uploading Content 9.5%, 3) Quizzes 10%, Login/Access Issues 27.2%, &
Miscellaneous 25.5%. The Login/Access Issues is the largest category, followed by the
Miscellaneous category. The Miscellaneous category was made up of the tickets that
were individualistic and required case by case assistance, for example: a) submitting
application materials online for the Art Department, b) a cross listed course no longer
showing up in the eLC and c) eLC password issues. As mentioned above, quite a large
number of the Central IT/CTL tickets were specific questions that would be best
answered by the DHC and not the CTL eLC page, for example: a) clicker issues and b)
b) a projector screen not responding to the touch panel, but door-side buttons work.
The information collected with the online survey revealed that many users enjoy
the functional categories topics are broken down into. The step-by-step tutorials are the
most helpful for first-time Desire 2 Learn (D2L) users. The level of instruction is
beneficial to novice users. Unfortunately, users who are more experienced with d2L find
the current instructions lack the necessary detail to help with their issues. The survey
participants reported difficulty in being able to find detailed guidance or information of
specific topics, such as how to track the output of students taking a quiz created outside
of eLC. This lack of resolution led to their submission of DHC tickets or Central IT/CTL
tickets. Although just over the majority of users do find the page extremely helpful, some
overall changes in the display and navigation of the page would be appealing to page
visitors. The consensus of the survey participants was that the current page seems
clunky and overwhelming.
11. Recommendations
Our group would like to recommend the following suggestions to help rebuild,
organize and streamline the CTLs help webpage in order to improve its accessibility
and usefulness.
5) Creating an initial page inquiring about the users level of experience will allow for
filtering of the results received during searches.