Вы находитесь на странице: 1из 13

Número de


Medición de
Mooc en 4

Medición de
Mooc en 5

Medición de
calidad y 4Y 5
Estrategía de Búsqueda

( ( TITLE-ABS-KEY ( mooc* ) OR TITLE-ABS- ( ( TS=( mooc* ) OR TS=( massive AND open AND
KEY ( massive AND open AND online online AND course ) ) AND ( TS=( measure ) OR
AND course ) ) AND ( TITLE-ABS- TS=( assessment ) OR TS=( evaluation ) OR
KEY ( measure ) OR TITLE-ABS- TS=( appraisal ) ) AND TS=( quality ) ) OR
KEY ( assessment ) OR TITLE-ABS- ( ( TI=( mooc* ) OR TI=( massive AND open AND
KEY ( evaluation ) OR TITLE-ABS- online AND course ) ) AND ( TI=( measure ) OR
KEY ( appraisal ) ) AND TITLE-ABS- TI=( assessment ) OR TI=( evaluation ) OR
KEY ( quality ) ) AND PUBYEAR > 2011 AND TI=( appraisal ) ) AND TI=( quality ) )
PUBYEAR < 2017 AND ( LIMIT- Período de tiempo: Todos los años. Índices: SCI-

( ( TS=( mooc* ) OR TS=( massive AND open AND

online AND course ) ) AND ( TS=( measure ) OR
( TITLE-ABS-KEY ( mooc ) OR TITLE-ABS-KEY TS=( assessment ) OR TS=( evaluation ) OR
( massive AND open AND online AND course TS=( appraisal ) ) AND (TS=( effectiveness) OR
) AND TITLE-ABS-KEY ( measure ) OR TITLE- TS=(achievement)) OR ( ( TI=( mooc* ) OR
ABS-KEY ( assessment ) OR TITLE-ABS-KEY TI=( massive AND open AND online AND course ) )
( evaluation ) OR TITLE-ABS-KEY ( appraisal ) AND ( TI=( measure ) OR TI=( assessment ) OR
AND TITLE-ABS-KEY ( effectiveness ) OR TI=( evaluation ) OR TI=( appraisal ) ) AND
TITLE-ABS-KEY ( achievement ) OR TITLE-ABS- (TI=( effectiveness ) OR TS=(achievement) AND
KEY ( success ) ) AND PUBYEAR > 2011 AND (TI=( success ) OR TS=(success))))
PUBYEAR < 2017 AND ( LIMIT-TO Refinado por: Años de publicación: ( 2015 OR 2016
( DOCTYPE , "ar" ) ) OR 2014 ) AND Tipos de documento: ( ARTICLE )
Período de tiempo: 2001-2017. Índices: SCI-
Número de artículos Quitando coincidentes

20 31 6 17

32 20 19 7

4 5 0 1
Número final de artículos
ando coincidentes Exluidos analizados

14 37 27 10

13 39 34 5

4 5 2 3

Chapman S.A., Goodman S., Jawitz J., Deacon A.

Vila R.R., Mengual-Andrés S., Guerrero C.S.

Sinclair, J; Boyatt, R; Foss, J; Rocks, C

Yousef A.M.F., Chatti M.A., Schroeder U., Wosnitza M.

Castano, C; Maiz, I; Garay, U

Loftis, E; Wormser, JM

Ramirez-Fernandez, MB; Silvera, JLS

Zhu, TJ
de la Garza, LYA; Sancho-Vinuesa, T; Zermeno, MGG

MacKay J.R.D., Langford F., Waran N.

Adam, M; Young-Wolff, KC; Konar, E; Winkleby, M

Ramirez-Fernandez, MB

Ramirez-Fernandez, MB

Yepes-Baldo, M; Romeo, M; Martin, C; Garcia, MA; Monzo, G; Besoli, A

Gomez Hernandez, Patricia; Monge Lopez, Carlos; Sebastian Heredero, Eladio

Masanet E., Chang Y., Yao Y., Briam R., Huang R.

Mengual-Andres, S; Lloret Catala, C; Vila, RR


A strategy for monitoring and evaluating massive open online courses

Assessment the pedagogical quality of the MOOC [Evaluación de la calidad

pedagógica de los MOOC]

A study of user participation across different delivery modes of a massive open

online course

A usability evaluation of a blended MOOC environment: An experimental case


Design, Motivation and Performance in a Cooperative MOOC Course

Developing Online Information Literacy Instruction for the Undergraduate Art

Student: A Collaborative Approach in the Context of the Framework for
Information Literacy

Edutool®: A tool for evaluating and accrediting the quality of moocs

Empirical Research on the Effectiveness of MOOCs in Developing Design

Students on Sino-foreign Cooperation University Programs
Indicators of pedagogical quality for the design of a Massive Open Online
Course for teacher training

Massive open online courses as a tool for global animal welfare education

Massive open online nutrition and cooking course for improved eating behaviors
and meal composition

MOOC appraisal: A quality perspective

Proposal quality certification of educational Spanish MOOC courses offer

conducted by the National Institute of Educational Technologies and Teacher

Quality indicators: developing "MOOCs" in the European Higher Education Area



Reflections on a massive open online life cycle assessment course

Validation of the Questionnaire of Quality Assessment of Online Courses

adapted to MOOC
Palabras Clave

Developmental evaluation; Evaluation; Monitoring;

MOOC; Online course; Performance monitoring

Assessment; Courses; E-learning; MOOC; Pedagogy;

Quality; Virtual Platforms

massive open online course; MOOC; course

evaluation; online learning; pedagogy

Blended MOOC; BMOOCs; Effectiveness; Massive

open online courses; MOOC design; MOOCs; Quality
assurance; Usability

Higher education; learning; motivation; personal

learning environments; cooperative learning
environments; virtual learning; educational
innovation; educational research


MOOC; Training; Educational Quality; Educational


Design Students; Effectiveness of MOOCs; Empirical

Research; Sino-foreign Cooperation University
Massive Open Online Courses; MOOC; evaluation;
pedagogical quality; teacher training

Animal welfare; Animal welfare education; Online

learning; Veterinary education

Nutrition; Cooking; Online education; Eating

behaviors; Meal composition

MOOC; UNE norms; virtual training; quality;

assessment; innovation

MOOC; Training; Educational Quality; Educational

assessment; INTEF

Massive Open On-line Courses (MOOCs); standards;

educational assessment; quality assurance

Assessment; Inclusion; MOOC; Quality

MOOC; pedagogical practice; online education;

review of distance education research; evaluation
We argue that the complex, innovative and adaptive nature of Massive Open Online Course (MOOC)
initiatives poses particular challenges to monitoring and evaluation, in that any evaluation strategy
will need to follow a systems approach. This article aims to guide organizations implementing MOOCs
through a series of steps to assist them in developing a strategy to monitor, improve, and judge the
merit of their initiatives. We describe how we operationalise our strategy by first defining the
different layers of interacting agents in Resumen
a given MOOC del system.
artículo We then tailor our approach to these
different layers. Specifically, a two-pronged approach was developed, where we suggest that
individual projects be assessed through performance monitoring assessment criteria for which would
be defined at the outset to include coverage, participation, quality and student achievement. In
contrast, the success of an overall initiative should be considered within a more adaptive, emergent
evaluation inquiry framework. We present the inquiry framework we developed for MOOC initiatives,
and show how this framework might be used to develop evaluation questions and an assessment
MOOC have burst We also define
into the the more
sphere fixed indicators
of education and measures
in an accelerated for project
way. Firstly, performance
the main US universities,
monitoring. Our strategy is described as it was developed
and later many others-among them, some Spanish ones - have been developing courses to inform the evaluation of a MOOC
under this
initiative at the University
format through of Capeon
various platforms Townthe(UCT),
Since Africa.
MOOC © 2016are still Elsevier Ltd. stage of
at an early
development, few studies have been published on their assessment so far. That is why the present
study has carried out 129 pedagogical assessments on 52 MOOC offered by 10 platforms. Their
pedagogical characteristics have been analyzed after the Cuestionario de evaluación de la calidad de
cursos virtuales (Arias, 2007). Attention is similarly paid to the hypothesis about the existence of
Massive open
statistically online courses
significant differences(MOOCs) are offered
regarding course by manydepending
quality universities, onwith hundredsthat
the platform of thousands
hosts the
of people
MOOC, worldwide
analyzing the having
impactregistered for one or
that the language ormore of the many
the presence availableguide
of a didactic courses.can Despite
have onthe the
quality level thatofhas been claimed
courses. The results for obtained
these courses in thetopresent
transform study education,
do not reveal in practice the majority are
deeply conservative
differences betweenin maintaining
courses, the educational
but statistically status
significant quo. Lacking
differences innovative
do exist in 5 ofpedagogic
the 36 pedagogical
foundation and with
quality indicators the need for approaches that scale, many courses rely heavily on very traditional
methods such as mini-lectures and quizzes. In particular, learner support is proving to be insufficient
for many participants. This paper reports results and experience from developing and presenting a
In which
the past fewprovides
years, thereboth has'traditional'
been an and supported
increasing modes.
interest Users can
in Massive Open optOnline
to study the course
Courses (MOOCs)in
as newfamiliar
form ofwithin most MOOCs (with
Technology-Enhanced peer support
Learning (TEL), inandhigherlimited tutor input)
education or to receive
and beyond. a high
level of experienced
the limitations tutor support.
of standalone MOOCs, Having
modes (bMOOCs)
run in parallel thatallows
aim atdirectbringingcomparison
in-class (i.e.
between the experiences and achievements of the two groups.
face-to-face) interactions and online learning components together have emerged as an alternative We present the motivation and
MOOC modelfor the course, discuss
of teaching and learningresultsinobtained and reflectcontext.
a higher education on lessons learned
In this paper, in we
design, implementation, and evaluation details of a bMOOC course on "Teaching Methodologies" at
Fayoum are seen asEgypt
University, the latest evolution inwith
in cooperation online
RWTH learning
Aachen and, since their
University, launch inprovided
Germany, 2008, they have
using the
bMOOC platform L2P-bMOOC. In order to gauge the usability and effectiveness of the course, we the
an integral part of university course curricula. Despite the social success of these courses,
employeddesign and efficacy
an evaluation of theirbased
approach results onhave been12
Conole's questioned.
dimensionsMost current
rubrics, research
ISONORM has focused
9241/110-S as
more on discussing
a general their potential
usability evaluation, and to offer quality,
a custom large-scale
effectiveness education worldwide
questionnaire reflecting the rather than
MOOC stakeholderlearning perspectives.
outcomes. This paper shows the results of a research study that focused on the
pedagogical design of a cooperative MOOC and its influence on motivation and academic results. A
Delphi study was used to validate the design, and the motivation variable was controlled using the
Instructional Materials Motivation Survey (IMMS). Academic performance was assessed through
Perceiving a lacklearning.of integratedThe paper arguesliteracy
information that design, which at
curriculum is our
defined by the
colleges, thestudents' intensiveof
library directors
the Association of Independent Colleges of Art and Design (AICAD) consortium resolved to createhas
of social networks and the activities they carry out in their Personal Learning Environments, a
an influence
series of online on tutorials
performance, aimedand the variable thatinmediates
at undergraduates the fieldsinofthat art relationship
and design. The is thecollaborative
level of
This research
project needed with
athe perception
partner thetoregulatory
produceof thethe
design. ofThe
tutorials MOOCsacademic
and (Massive
offer theresults Open
project and
in the students'
through the
an environment
accessible to support
of the Spanish
about Patent
thisin partnership,
and Trademark
Lynda.com, Officeone (3,087,298, at the
of the Internet's
present time). It arises from a line of research at the University
leading online learning companies, agreed to work with the AICAD libraries to create an online Pablo de Olavide (Seville) and
the Laboratory
course. The for Computational
course launchedIntelligence
in July 2015, (LCI).
and(R) theEduTool
developedan under the
assessment of the
of theUNE 66181quality
course standard,and while at theone
use nearly LCI,year
the later
significance of thelibrarians
by surveying sub-factors and forfaculty.
each of
article is a study(recognition
of how of that training,
was conceivedmethodology
and how and levels offrom
it evolved accessibility)
a Massive ofOpen
standard for Quality Management of Virtual Training
Online Course (MOOC) into its current online tutorial version on Lynda.com. for MOOC have been analyzed using fuzzy logic.
In this sense, aggregation of the linguistic variables of 10 experts was performed using the
probabilistic OR function and they were defuzzyfied using the centroid method to calculate these
weights. To the implementation
measure the qualityof ofSino-foreign
the courses,cooperative
we proceded education
to find and program
select has become
relevant MOOC an
platforms:way to internationalize
Udacity, EDX, MiriadaXChinese higher education.
and Coursera. Subsequently, The the largemostnumber
relevant of MOOC
MOOCs online
related to
and computer available, and categories
science their innovative teaching by
were selected modes has provided
intentional a new andsampling.
non-probability novel
channel to promote such programs. In order to systematically illustrate
For this data collection method, two engineers accessed these MOOC platforms to code and assessing and reveal the effectiveness of
the qualityMOOC in these
of these projects,
courses usingwhichEduTool sponsored by Undergraduate
(R). In general, the analytical cooperative
and graphic education
results showproject
of the averagedesign MOOCs between
analyzed Anhui Polytechnic
in each platformUniversity
is located and above University
the scoreofthat Bridgeport,
researchers carried out a one-year targeted and systematic tracking,
the required minimum (50 %). In this sense, a higher average quality was appreciated in Coursera investigation, comparative
and EDX platform summary (62.62 and%),the real-time assessment.
intermediate in the Udacity Experimental
platform (54.92%) data andand detailed
lower in the results show platform
MiriadaX that the MOOC (50.40%). intervention mechanism achieved a positive effect in promoting
and supplementing students' knowledge and understanding of their major. It not only strengthened
students' skills and literacy, it was also found to significantly promote traditional classroom teaching
at Anhui Polytechnic University. The project proved successful with design students by combining in
class and online teaching. It is recommended that such projects be further promoted and applied in
the future.
therefore, their pedagogical quality is often questioned. The objective of this study is to identify
indicators Behaviour
related to and Welfare wasfunctional,
pedagogical, a MassivetechnologicalOpen Online and Course time (MOOC)
factorshosted in orderontoCoursera
assess the as a
free introductory animal welfare course. Through interrogating
quality of the MOOC entitled "Liderazgo en gestion educativa estrategica a traves del uso de la Coursera data and pre-/post-course
tecnologia" experience
(Leadership surveys, we investigated
in strategic educational student retention,through
management studentthe experience, changes inoffered
use of technology),
as a teacher andtraining
changes in knowledge.
program throughThe coursetoran
Coursera for 5 participants.
10.161 weeks, and 33,501 Via thestudents
Delphi method,signed up, a of
group of 16.4% (n = 5,501)
55 experts agreed received
that time a Certificate
is a key factor of Achievement,
to be considered indicating
in the they had of
design completed
learning all
activities. It was within the course.
concluded that Thiswithout retention
measuring rate isresults,
above the the industry
success of standard
a MOOCofcould 10%; not however,
the value of retention rate as a metric to judge MOOC
evaluated; thus, institutions and consortia must establish evaluation indicators to focus their success is questionable. Instead, we focus on
on the enhancement with Coursera
of pedagogicaldata estimating
quality. Bythat 41% of relevant
providing learners information,
came from Europe, the learning35% from potential
North America,resources
of educational 11% frombased Asia, 6% from Oceania,principles
on connectivism 5% from South can beAmerica,
evaluated, and and2%sofrom can Africa.
the quality Mostof
learners had completed an undergraduate degree. Despite
MOOCs. The goal is to contribute to a vision of a future in which everyone has access to a world-class this wide range of backgrounds, 57.2% of
education. respondents (n = 2,399) strongly agreed that the information presented was at the right
level and 64.9% Behavioral
stronglyshifts agreed in that
eating, thefavoring
course was the increased
After completion, of highly more processed
(w2[4] healthier,
= 132.40, home-cooked
p < .001) understood alternatives, thathave animal ledwelfare
to widespread
was based healthon the problems.
results This study reports
of scientific study,
andthe effectiveness
significantly fewer of students
a massive(w2[4] open online = 361.32, course (MOOC),
p < .001) offeringwas
felt health integrated
the mostnutritionimportant andpart of
cooking instruction,
animal welfare. Overall,for improving
learners agreed eatingthe behaviors
course and was meal composition
enjoyable and informative,among course and 97.9% participants.
felt the
course wasThe course, consisting
a valuable use of theiroftime. 47 short (4-6 min.)that
We conclude videos,
MOOCs was areoffered through Coursera,
an appropriate vehicle an foropen,
welfare available
learning to toaindividuals
wide audience, worldwide who have
but require access tolevel
a significant the Internet.
of investment. ©
2016 AAVMC. in January 2014, participants viewed course videos, completed quizzes and participated in
optional cooking assignments, over a 5-week period. Participants were invited to complete optional
pre- and post-course surveys assessing their eating behaviors, typical meal composition and perceived
The researchto home cooking.
analyzes the McNemar-Bowker
normative quality of tests
fiveofMOOCsymmetry courses andofwithin
the EDX subject
platform t-tests were
conducted to evaluate pre-post survey changes in the
EduTool (R) instrument, brand registered in the Spanish Patent and Trademark Office (3,087,298, primary variables measured. Results: 7,422in
participants from more than 80 countries completed both
effect). This tool was the result of a Doctoral Thesis in the research line of work "2.0 ICT teaching pre-and post-course surveys, while 19,374
innovation the European the pre-survey
Higher Education only. Class Area" participants
of the Universitywere primarily
Pablo dewomen Olavideinofthe child-
(Spain).ages (20-49was
This work years of age). There
developed under were significant
the auspices positive
of the UNE changes
66181: 2012 in eating
and the weightsand
meal composition over time, including an increase in
of the subfactors of each of the dimensions of this standard for Quality Management of Virtual the percentage of participants who reported
This dinner
article at home
for discusses
MOOC weretheusing
possiblemostly using fresh
application fuzzy ingredients
of quality 5-7
In that times in
standards to the
MOOC previous
measure (Massivetheweek quality
Open (63.4 % to 71.4
of that
%), and who
Courses) were
at the felt that yesterday's
National by intentional
Institute dinner was very/extremely
of Educational sampling
Technologies healthy
andthe (39.3
most %
Teacher to 56.4of
Training %)the
coursesand enjoyable
Ministry were
(55.2 % toin66.7
Education, the %)
Culture (all
second valuesof<of
quarter .0001).
Spain Conclusions:
through thisEduTool
procedureIntegrated nutrition
for collecting
(R), registered and cooking
trademark in thewas courses,
Spanish Office
of via open
the platform
and twoonline
Trademark encoders learning
(3,087,298 platforms,
(a computerin force). offer
This aline
engineer free
and ofand
awork flexible
teacher comes andvenue afor
from linereaching
of adults
research of theof
Pablo and have
which University
de Olavide the potential
valued theofquality to
courses powerful
it wasbydeveloped behavioral
that instrument. shifts that align
The analytical Intelligence
in the Computational well with
results show efforts
that,to in
improve the
Laboratory eating
under behaviors
quality the theand
ofauspices averagemeal
of the ofcomposition.
UNEMOOC 66181.analyzedIn the tool in the
we platform
have analyzed is above thethe scoreof
weights that
describes the
subfactors minimum
of each of therequired
dimensions by the tool (50%).ofThus,
(recognition the average
training, learningqualitymethodology of the platform
and levels was of
62.62%. Particularly,
accessibility) all courses
of this standard forbeyond
Qualitythe minimum value
Management of the
of Virtual first dimension
Training for MOOCwith a 6.66%.
by fuzzy logic. AsInfor
the second
this sense, Itdimension,
arises the quality
possibility of 32.77%
to applywas theobserved. And finally,ofinthat
quality certification the tool
the educational is evident
that objective
MOOC all offer
courses ofofINTEF
this research
tothe acceptable
alleviate is tothe design quality
high an integrated
dropout with rate23.19%.system
of users of in evaluation
the MOOC of theplan
pilot quality of the
for teacher
education Open Online
occurred inCourses
2014, within (MOOC) the taking
lines ofinto account
action of the the features
project "New of this
forms type of of courses.and
training", Theit
criteria includedininthe
can be avoided theoffered
evaluation of the
training qualitythis
activities of e-learning
year 2015.training in a broad sense are
considered, and specifically those developed for the MOOCs. Using a quantitative, descriptive and
sectional design, a team of experts have valued those indicators proposed by the scientific literature.
To do this, eight courses, focusing on the field of entrepreneurship and innovation, were selected.
Most of the analysed courses meet designated quality criteria by having a clear and organized
academic structure. In addition, each course states clearlydata the from
timinga recent
and hours of dedication by the
This article summarizes student performance and survey massive open online
In the XXIHowever,
century, only
the a
needfew courses
of training do that
in, for onand the open
with Internet
Information andand on their
Communication own academic
course (MOOC) on life Nonecycle assessment (LCA). Its purpose is to shed light on studentThe learning
seems significant. websites. In this of the MOOCs
contextual framework, establishes
there the
are admission
two highlighted requirements.
elements: the main
outcomes, challenges, and success factors, as well as on improvement opportunities for the MOOC
Open Online of the
Courses, current
and research
the inclusive is that
school. it givesThus,MOOC the developers
main aim of some
this quality
study is toindicators,
evaluate arelated
and the role of online courses in LCA education in general.
to course
MOOC from design and platform,
Iberoamerican to better
context that plan emphasizestheir applied design and
the attention implementation,
to diversity from an approach which has a
based onpositive
inclusive impact
education. on theFor educational
this reason, quality
the case of such
study e-learning
was usedplatforms.
as research method, standing
out the application of several assessment instruments in relation to quality criteria of MOOCs. In
general, the results
Student survey datahighlight
and course a positive
performance evaluation data around almost allanalyzed,
were compiled, analyzedand dimensions
interpreted andfor 1257
subdimensions, particularly around pedagogical aspects.
students who completed a pre-course survey and 262 students who completed a post-course In conclusion, there is a MOOC in thesurvey.
Iberoamerican context that discusses the inclusive education,
Both surveys were designed to assess student learning outcomes, topical areas of difficulty, changing which has been valued very positively,
although it must
perceptions on the continue
nature to of improve.
LCA, and future plans after completing the MOOC.
Massive Online Open Courses (MOOC) constitute a virtual training modality which is already present
at today's
Results and educational
discussion scenario, and which reflects a specific conception of e-learning. Universities are
integrating this type of training as part of their formative offer or, at least, are debating whether to
integrate it or not.
Results suggest thatInonline
the light of thecan
courses above,attract it becomes
and motivate necessary
a large tonumber
gauge what the characteristics
of students and equip of
these courses
them with basic should be soskills
analytical that to theymove canon configure
to moreaadvanced high-quality LCAvirtual
studies. training
However, format. These
results alsoare
considered to be necessary instruments when it comes
highlight how MOOCs are not without structural limitations, especially related to mostly “locked to guaranteeing the quality of MOOC from in”
different perspectives, the pedagogical one in this case-since
content and the impracticality of directly supporting individual students, which can create challenges the success and consolidation of this e-
for teaching format will depend
difficult topics and thereon.
conveying The aim soughtlimitations
important with the present of LCA study was therefore to validate
in practice.
a questionnaire for the pedagogical evaluation of MOOC through an adaptation of the cuestionario de
Conclusionsde cursos virtuales [questionnaire for the evaluation of virtual courses] (Arias, 2007). The
validity and reliability of the said test was checked using a principal components factor analysis with
Varimax rotation.
Online courses, andThe conclusion
MOOCs drawn ismay
in particular, thatpresent
it wouldan suffice to introduce
opportunity for the adaptations
LCA community on thetolist of
efficiently recruitproposed in thisitsquestionnaire
and train next generations that of takeLCAinto account
analysts and,theinspecificities
particular, those of MOOC, studentssincewho the
latter cannot be exclusively analyzed from the general
might not otherwise have an opportunity to take an LCA course. More surveys should be conducted perspective of e-learning - as it was prior to
MOOC. Thus, the dimensions
by LCA instructors and researchers whichmoving serve as the basis
forward for thatscientific
to enable questionnaire developmentare: 1) quality
and sharingof of
best practice teaching and multimedia
methods and elementsmaterials. in online massive courses; 2) curricular consistency in such
courses as well as their degree of adaptation to the user; and 3) quality of their didactic planning.