Академический Документы
Профессиональный Документы
Культура Документы
ESCUELA DE INGENIERIA
EN EL DOMINIO DE LENGUAJE Y
PRIMARIA
Profesor Supervisor:
MIGUEL NUSSBAUM
ESCUELA DE INGENIERIA
tierras, a Nicole.
ii
AGRADECIMIENTOS
Quisiera agradecer primero a la Pontificia Universidad Católica de Chile por abrir sus
propicio para resolver los problemas más urgentes de nuestro país. Además, quisiera
quisiera agradecer al profesor Pablo Chiuminatto, por su fe, su amistad y por invitarme a
imaginar nuevos terrenos para el desarrollo intelectual. También a Miguel Nussbaum, por
No puedo dejar de agradecer a mis compañeros Damián Gelerstein y Martín Cáceres, con
Quisiera agradecer también a mis padres por su apoyo incondicional en mis derivas
nuevos desafíos.
Finalmente, a Nicole; por su amor, por su paciencia, por enaltecer lo mejor en mí. Bajo su
mirada atenta se gestó todo lo que hay de virtud en este trabajo. Los vicios, por supuesto,
iii
INDICE GENERAL
ABSTRACT ......................................................................................................................... ix
PRÓLOGO ............................................................................................................................ 1
BIBLIOGRAFIA................................................................................................................. 50
ANEXOS............................................................................................................................. 68
Anexo A. Tablas ........................................................................................................................... 69
Anexo B. Paper en ingles .............................................................................................................. 74
Anexo C. Recepción del Paper en Revista Académica............................................................... 107
v
INDICE DE TABLAS
vi
INDICE DE FIGURAS
vii
RESUMEN
Palabras Claves: pensamiento crítico, modelos de evaluación, habilidades del siglo XXI,
cultura, narrativa gráfica y textual
viii
ABSTRACT
The current trend in educational research recognizes the necessity to integrate the so
called “21st century skills” into the school curriculum. One of the most important skills,
in this sense, is critical thinking, which is recognized in the national curriculum. The
consequence to change the curricular orientation to this new paradigm implies a
transformation of the assessment of these competences. However, there are not available
instruments that fit this purpose for primary education. This research is directed towards
the cosntruction of an assessment model and the design of a test of critical thinking for
students in 3rd and 4th grade. The test was specifically in the domain of language and
communication by means of a graphic novel. The validity of the test was determined by
analyzing the construct, content and criterion validity, as well as its reliability and
performing an item analysis. The instrument shows an adequate design according to a
multidimensional model of the assessed skill, which delivers a more detailed and
complete picture of the student’s learning. Once it was validated, the collected data
allowed us to examine differences in the level of critical thinking according to the
student’s socioeconomic status. Significant differences were found between the groups.
These differences should be considered in future research due to possible influences of
the socioeconomic fator over the assessment of skills in primary education contexts.
This thesis had the support of CONICYT-FONDECYT 1150045, and of the VRI Interdisciplina
Research Project N° 13/2014.
Keywords: Critical Thinking, Assessment models, 21st Century Skills, Culture, Graphic
and Textual Narrative
ix
1
PRÓLOGO
escolar. Una de las habilidades centrales, en este sentido, es el pensamiento crítico, que es
información provocada por los nuevos medios de comunicación digital (Mineduc, OCDE
Una instrucción adecuada de esta habilidad requiere que los profesores y los
estudiantes (Ennis, 1993). Sin embargo, los actuales modelos de evaluación entregan una
imagen parcial o distorsionada del aprendizaje de los estudiantes, ya que utilizan métodos
En este sentido, el objetivo de este trabajo es diseñar un test que evalúe pensamiento
crítico para lograr a través de datos una imagen más acabada del aprendizaje de los
reales.
2
XXI”, al que se ciñe este trabajo. Luego, se especifican los conceptos que guiarán la
test de pensamiento crítico basado en una novela gráfica para estudiantes de tercer y cuarto
grado. Además, se analiza la posible influencia del estrato socioeconómico sobre el nivel
El tercer capítulo revisa las conclusiones generales del trabajo realizado en la construcción
Finalmente, en los anexos, se encuentra la versión en inglés del paper enviado junto a la
PENSAMIENTO CRÍTICO
Los últimos años han visto levantarse una enérgica demanda de la ciudadanía
máxima expresión es una reforma en curso por parte del gobierno de Michelle
estudiantes.
their own wants and concerns and can advocate for their goals and aspirations’’
(Sayed, 2013). Este llamado responde a un reclamo general por una enseñanza
alrededor del movimiento llamado “habilidades del siglo XXI”, cuyo impacto
Se puede notar que si bien el discurso de las habilidades del siglo XXI nace de
indica el reporte, “All too often (...) students in many countries are not attaining
para este tipo especial de habilidades entre países de la OECD de alto ingreso
en contraposición con aquellos en vías de desarrollo (WEF & BCG, 2015). Esta
diferencia ni siquiera pudo ser establecida respecto a los países de bajo ingreso,
puesto que muy pocos participan en tests que permitan la comparación como la
esfuerzos por reducir la brecha educacional entre los países ricos y los pobres,
Ante este escenario conviene preguntarse dónde vale la pena dedicar trabajo
por las distintas comunidades. Explica Dede (2010) que una de las mayores
que tanto los educadores y los estudiantes, así como los diseñadores de
en una prueba deben tener una única pregunta. Este tipo de resistencia no solo
evaluación.
Según Dochy (2001) existen dos paradigmas sobre el que se construyen las
políticas de evaluación.
contenidos aislados, sino una red de conceptos y principios desde los que
de las maneras que las habilidades a enseñar lo exijan. Así, muchas veces se
aportan muy poca información, por ejemplo, tests de opción múltiple, ante la
presión por mostrar resultados ante el foro público (Conley, 2015). De esta
esperar. Por ejemplo, en los Estados Unidos ha existido una baja sostenida
déficits (Ramírez, 2011). Similar al caso de los Estados Unidos, Chile ha sido
chilenos están entre los puntajes más bajos entre los países de la OCDE
(Moreno, 2014).
que sirvan para adaptar nuestro sistema educativo a las necesidades de la época
contemporánea
Muchas son las vías que la evaluación de las habilidades del siglo XXI ofrecen
2013), quien promulgaba el ideal de que todos los valores debían ser
examinados por cada ciudadano por su propio juicio racional. Con diferentes
ilustración” (1994/1784).
coherente ha sido establecida por Martha Nussbaum, quien en su libro Not for
filósofa, las sociedades actuales están compuestas por una gran diversidad de
“about a wide range of cultures, groups, and nations in the context of a grasp of
the global economy and of the history of many national and group interactions
is crucial in order to enable democracies to deal responsibly with the problems
we currently face as members of an interdependent world” (2012, p.10).
11
opinión de cada una debe ser valorada por los mismos estándares de
(Grossman, 2008).
aprendido a nuevos contextos. Por otra parte, Heyman (2008) declara que el
niveles del currículum de enseñanza escolar. Por sobre todo, la inclusión del
realidad.
de los estudiantes.
siglo 21. La tendencia histórica ha sido fijarse más en la confiabilidad del test
una serie de tests que miden partes discretas de las habilidades en cuestión en
Ennis,1991; Pithers & Soden, 2000; Lai, 2011; Dwyer, 2014). En este sentido,
habilidades que lo componen. Van Gelder (2005) arguye que la idea que el
operativamente en salas de clases (Butler, et al., 2012; Niu, et al., 2013; Reid &
Anderson, 2012).
“the monitoring and control of thought” (Martínez, 2006). Kuhn (1999) explica
pensamiento crítico no es tanto una habilidad general, sino una habilidad que
decir, rúbricas. Brown et al. (2014) sugieren que estas rúbricas que no estén
construidas desde adjetivos que expresen sus apreciaciones subjetivas, sino que
1.4 Cultura
cultura de una comunidad puede ser definida como los valores, reglas
Estudiar los factores culturales permite observar las maneras en las que las
1998; Schein, 1993). Sin embargo, aunque hay una gran variedad de estudios
sugiere que la cultura da forma a las maneras en que los estudiantes realizan su
aprendizaje cognitivo. Bajo esta premisa, Gruenert & Whitaker (2015) sugieren
en la sala de clases.
(Hofstede, 2001).
programación ocurre durante toda nuestra vida (Hofstede, 2001). Las personas
van adquiriendo poco a poco los valores que determinarán, así como, la manera
dimensiones medibles: (1) alta o baja distancia hacia el poder, (2) alta o baja
Este marco teórico, así como el test derivado de dicho marco, sirve de
culturales por diversas razones. En primer lugar, este modelo ha sido probado
en más de 50 países, por lo que está validado a través de una gran variedad de
culturas (Hofstede, 2001). En segundo lugar, tanto el modelo como los test
los años con una colección de datos coherentes (Jones, 2007). En cuarto lugar,
desde este marco, puesto que según explica Hofstede hay una división que
necesaria del ambiente” (Hofstede & Minkov, 2010, p. 8). En este período, el
cómo ésta puede causar, entre otros problemas, grandes desigualdades entre los
1986).
1987; Tsui, 1998; Tsui, 2000). Sin embargo, se le ha dado poco énfasis a temas
Cole (1971) explica que el desarrollo cognitivo está influenciado por las
una sociedad, por lo tanto, de su cultura (Hofstede, 1986). Así, estudios han
determinadas por el propio entorno (Sckade et al., 1978; Bennett, 1997). Por
para disminuir esta brecha cultural sino aumentar la conciencia sobre estas
valores que se nutren de dicho entorno — tiene una influencia en el uso del
un individuo y lo que tenemos que hacer frente a las diferencias culturales (Hill
& Shadija, 2010). Sin embargo, si se sostiene que la cultura del alumno afecta
prejuicios culturales (Bailin, 1995). Urdan & Giancarlo (2000) sugieren que
(2001) y Bates & Poole (2003) quienes describen la manera en que provenir de
La idea de que la cultura media los procesos cognitivos de nivel superior está
utilizando los signos y herramientas creados por su sociedad. Sin embargo, tal
como la comunidad que los crea, estos mediadores cambian con el tiempo
superiores, pasa necesariamente por las relaciones sociales que la persona tiene
pensamiento crítico.
la influencia de una tercera variable sobre la relación entre las dos variables
desarrollado por Baron & Kenny (1986). Este método consta de cuatro pasos,
Primero, se realiza una regresión (τ) para saber si la variable independiente (X)
Tercero, se realiza una regresión (β) para saber si la variable mediadora (M)
Cuarto, se realiza una regresión (τ’) para saber si la variable independiente (X)
mediadora (M).
parcial.
27
indicados. En lo que sigue se explicará cuál fue el proceso de construcción del test de
de esta tesis.
social, laboral y ético de los ciudadanos (Griffin, McGaw & Care, 2012;
los individuos con las herramientas necesarias para tomar decisiones por sí
En este trabajo se utilizará una definición que haga explícito este vínculo con la
sea un objetivo en todos los niveles del currículum K-12. En este sentido, la
et al. 1999). Es importante que la inclusión del pensamiento crítico sea a una
edad temprana, puesto que se han revelado diferencias significativas entre los
niños en la sala de clase, tal como explica Whitcomb & Merrell (2013), la
deriva del peso que los distintos dominios dan a diferentes evidencias o
Ennis (1989) recomienda que el mejor nivel para enseñar crítico es durante los
para dominios específicos sino que solo miden el pensamiento crítico como
una habilidad general (Ennis & Millman, 1985; Watson & Glaser, 1980;
Facione, 1990; Ennis & Weir, 1985; Halpern, 2003). Por tanto, al enfrentar este
educacionales (Breen & Goldthorpe, 1997); asi en tercer grado se ha visto que
2.1.2 Metodología
2.1.2.1 Definición del test.
En una primera etapa se realizó un estudio bibliográfico en el que se determinó
para su exitosa realización por niños en los últimos niveles de escuela primaria
(Halpern, 2003).
explicación, puesto que las estrategias cognitivas que los estudiantes utilizan al
Magliano, 2009).
argumentación, pues según muestran Berland & Mcneill (2012) los procesos de
crítico, puesto que es una de las herramientas más efectivas para discriminar
actividades que impliquen este tipo de habilidades con igual éxito que los
perspectiva, se estimó que la edad más temprana posible para ser efectivo el
2015).
33
lectura crítica y creativa. Por esta razón, se utilizó la novela clásica de Julio
Verne La vuelta al mundo en 80 días, adaptada para niños de tercer grado. Esta
pensamiento crítico.
En una tercera etapa, se estimó necesario utilizar una narrativa gráfica y textual
privilegiado que tienen los formatos visuales en nuestra cultura (Emmison &
Smith, 2000; Sturken & Cartwright, 2001), cuya consecuencia es que los
34
2006). Otro argumento por el que se consideró este formato se debe a que es el
Lazo & Smith, 2014). Hassett & Schleble (2007) sugieren que al leer narrativas
imagen y el texto.
tanto, la narrativa tiene que estar estructurada en torno a dichos criterios. Para
que preguntar al final de la historia (Van Den Broek, et al., 2011). Por ejemplo,
visuales en una narración, ya que el estudiante tiene que ser capaz de entender
cuál es la función del objeto como parte del disfraz y no una característica
en el ejemplo, el estudiante no solo necesita decidir por una opción u otra sino
(Nitko 1996).
explicaría porque mientras que en tercer grado los estudiantes pueden realizar
Okey, 1980; Nitko. 1996). Además, se agregaron para la validez del contenido
(Ericsson & Simon, 1993; Flaherty, 1974), tal como se recomienda en Facione
(1990).
38
al., 2011) se utilizó el Pearson product moment correlation entre los resultados
del test y las notas en lenguaje con el fin de examinar la criterion validity. Se
2013).
de Cronbach, índice que mide la consistencia interna del test (Cronbach, 1971).
eliminar o modificar ítems (Nitko, 1996). Estos índices por ítems fueron
observar qué área fue más y menos difícil para los estudiantes.
lado, para estudiar la validez de constructo se trabajó con dos grados, 3º y 4º.
distintos grupos.
Tercer grado
Masculino 55 13 17
Femenino 47 17 14
Total 102 30 31
Fourth Grade
Masculino 40 14 18
Femenino 42 14 10
Total 82 28 28
2.1.3 Resultados
2.1.3.1 Validez del constructo
El test de Shapiro-Wilk indicó que no había normalidad en la distribución de
1.613e-07 para cuarto). Por su parte, el test de Barlett indicó que las varianzas
El test para identificar la validez del constructo del instrumento indicó que hay
p<0.001.
de los estudiantes fueron 0.805 para el tercer grado y 0.732 para el cuarto
grado, que Dancey & Reidy (2004) clasifican como una correlación fuerte
2.1.3.4 Confiabilidad
Se encontró un Alpha de Cronbach de 0.917 para el test antes del análisis de
valor de 0.909, lo que muestra que el test es confiable (George & Mallery,
la sección siguiente.
tal como lo muestra la tabla 2. Para este estudio se seleccionó un rango entre
0.4 y 0.6, ya que, siguiendo a Womer (1968), este rango determina que una
medición. Además, Nitko (1996) explica que aquellos ítems con índice de
dificultad menor a 0.2 y mayor a 0.8 deben ser rechazados por ser muy fáciles
se decidió que los ítems con índice de dificultad entre 0.2 y 0.4, y aquellos con
índice de dificultad entre 0.6 y 0.8 se integrarían siempre que aportaran con la
discriminación era aceptable para un ítem si es mayor a 0.3, por tanto, aquellos
ítems sobre 0.3 se consideraron buenos y aquellos con una discriminación bajo
0.3 se consideraron malos (Adkins, 1974; Hinkle, et al, 2003; Nitko, 1996). Un
Por otro lado, la dimensión con menor dificultad fue Interpretación con un
de los ítems, aquellos que, en primera instancia, formarían parte del test (ítems
2a, 3, 4, 6a, 10, 14b, 22, 23b, 27a). Por otro lado, siguiendo los criterios de
Nitko (1996), se determinó que se incluirían tambien los ítems (1a, 2b, 7, 8, 11,
12, 13, 14a, 16, 17a, 17,b, 18, 19a, 19b, 21, 26, 28a, 28b, 29a, 29b), con el fin
test. Por último, se eliminó un ítem(6b) que tenía correlación 1 con otro que
que las diferencias eran significativas at p >0.05 entre todos los grupos entre sí.
alto, los puntajes son considerablemente más altos (22.15 para cuartos y 19.3
para terceros) que los colegios de estrato medio (14.75 para 4 y 10.73 para
terceros) y bajo (10.61 para cuartos y 6.97 para terceros). Por otro lado, uno de
cuarto básico un 28% (de 4.55 a 3.27 puntos), en el colegio de nivel medio, la
2.1.4 Conclusiones
El objetivo de este artículo fue responder las siguientes dos preguntas de
investigación.
Es interesante notar que no existen otros tests que hayan utilizado el formato de
ciencias y matemáticas,
desventaja respecto a otro grupo por la estructura del test y no por una
necesario como trabajo futuro estudiar cómo afecta la cultura local este marco
Segundo, una vez validado el test, se resolvió indagar la pregunta sobre ¿Qué
educacional. Esto sugiere que podría haber una relación entre el estrato
instrumento.
primaria cuenten con un instrumento que les permita conocer el estado del
desarrollo del pensamiento crítico en sus estudiantes. Este test es una mejora en
medición a modelos que entreguen una imagen más detallada y múltiple del
CONCLUSIONES FINALES
presenta una imagen más completa del nivel de pensamiento crítico de los
estudiantes.
El uso de este instrumento puede beneficiar dos áreas del sector educacional a
los que afecta directamente. Por una parte, el instrumento puede integrarse a
profesores adquieren una guía sobre el impacto que sus clases tienen en el
diseño de este test, así como el instrumento mismo, puede utilizarse para la
chilenos. Explorar este vínculo con mayor profundidad podría explicar cómo el
modelo propuesto podría aplicarse íntegramente. Una vez validado este test, se
podrá obtener información relevante sobre el impacto que tienen los factores
competencia lectora.
BIBLIOGRAFIA
research,58(4), 375-404.
Ary, D., Jacobs, L., Sorensen, C., & Walker, D. (2013). Introduction to research in
Bailin, S. (2002). Critical thinking and science education. Science & Education,
11(4), 361–375.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Conceptualizing critical
Bates, A. W., & Poole, G. (2003). Effective Teaching with Technology in Higher
critical thinking, and active learning in the classroom. San Francisco: John Wiley &
Sons.
Berland, L. K., & McNeill, K. L. (2012). For whom is argument and explanation a
Bourdieu, P. (1993). The field of cultural production: Essays on art and literature.
Bourdieu, P. (2011). The forms of capital. (1986). Cultural theory: An anthology, 81-
93.
Brown, J. D. (2000). What is construct validity? JALT Testing and Evaluation SIG
Butler, H. A., Dwyer, C. P., Hogan, M. J., Franco, A., Rivas, S. F., Saiz, C., &
Chapman, J. (2014). Critical Thinking and TOEIC: Advanced Skills for Teachers
Chase, M., Son, E. H., & Steiner, S. (2014). Sequencing and Graphic Novels With
Cheung, C. K., Rudowicz, E., Lang, G., Yue, X. D., & Kwan, A. S. (2001). Critical
thinking among university students: does the family background matter? College
Childs, R. (1990). Gender bias and fairness. Practical Assessment, Research &
Evaluation, 2(3).
Routledge.
Dancey C., Reidy J. (2004). Statistics Without Maths for Psychology: Using SPSS for
college and career readiness: Developing a new paradigm. education policy analysis
archives, 22(86), 1.
Deal, T. E., & Peterson, K. D. (1999). Shaping school culture: The heart of
Dede, C. (2010). Comparing frameworks for 21st century skills. 21st century skills:
Dillashaw, F.G. & Okey, J. R. (1980). Test of Integrated Science Process Skills for
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking
framework for the 21st century. Thinking Skills and Creativity, 12, 43-52.
Emmison, M., & Smith, P. (2000). Researching the Visual: Images, Objects,
Ennis, R. H. (1993). Critical thinking assessment. Theory into practice, 32(3), 179-
186.
Philosophy,21(1), 15-33.
Ennis, R.H. & Millman, J. (1985). Cornell Critical Thinking Test-Level X. Pacific
Ennis, R.H. & Weir, E. (1985). The Ennis-Weir essay test: An instrument for testing
Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis: Verbal reports on data.
Recommendations. http://assessment.aas.duke.edu/documents/Delphi_Report.pdf,
Filmer, D., Hasan, A., & Pritchett, L. (2006). A millennium learning goal: Measuring
real progress in education. Center for Global Development Working Paper, (97).
Frey, N., & Fisher, D. (2004). Using graphic novels, anime, and the Internet in an
Gelman, S. A., & Markman, E. M. (1987). Young children's inductions from natural
George, D and Mallery, P. (2003). SPSS for Windows step by step: a simple guide
2010, from,
http://www.21stcenturyskills.org/documents/p21_framework_definitions_052909.pd
f.
Griffin, P., McGaw, B., & Care, E. (2012). Assessment and Teaching of 21st Century
Gruenert, S., & Whitaker, T. (2015). School Culture Rewired: How to Define,
Knowledge and Skills. Theory and Practice in Language Studies, 3(3), 503-507.
Hapgood, S., Magnusson, S. J., & Sullivan-Palincsar, A. (2004). Teacher, text, and
Hassett, D. D., & Schleble, M. B. (2007). Finding Space and Time for the Visual in
Hill, R., & Shadija, D. (2010). Culture, Critical Thinking and Computing.
Malaysia, 21-26.
Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics for the
http://fuhu.dk/filer/FBE/Arrangementer/Denmark%20Unlimited%20080508/FBE_ge
ert_hofstede_teaching_learning.pdf
57
Hofstede, G., & Minkov, M. (2010). Cultures and Organizations: Software of the
Hong-Nam, K., Leavell, A. G., & Maher, S. (2014). The Relationships among
Howe, K. (1995). Validity, Bias and Justice in Educational Testing: The Limits of
Hudgins, B. B., et al. (1989). Children's critical thinking: A model for its analysis
and
Jennings, K. A., Rule, A. C., & Zanden, S. M. V. (2014). Fifth Graders’ Enjoyment,
257.
http://ro.uow.edu.au/commpapers/370
Kennedy, M., Fisher, M. B., & Ennis, R. H. (1991). Critical thinking: Literature
Kim, K., Sharma, P., Land, S. M., & Furlong, K. P. (2013). Effects of active learning
Ku, K. Y., & Ho, I. T. (2010). Metacognitive strategies that enhance critical thinking.
Ku, K. Y., Ho, I. T., Hau, K. T., & Lai, E. C. (2014). Integrating direct and inquiry-
Reports, 6, 40-41.
Lazo, V. G., & Smith, J. (2014). Developing thinking skills through the visual: An
116.
59
Levy, F., & Murnane, R. J. (2012). The new division of labor: How computers are
Lipka, J., & Ilutsik, E. (2014). Transforming the Culture of Schools: Yupi’k Eskimo.
Lyga, A. A. (2006). Graphic Novels for (Really) Young Readers: Owly, Buzzboy,
Pinky and Stinky. Who Are These Guys? And Why Aren't They Ever on The
Manalo, E., Kusumi, T., Koyasu, M., Michita, Y., & Tanaka, Y. (2013). To what
Marques, J.F., (2012), Moving from trance to think: why we need to polish our
2012, 87-95
May, D. B., Hammer, D., & Roy, P. (2006). Children's analogical reasoning in a
http://www.curriculumenlineamineduc.cl/605/articles-18978_programa.pdf
18962_programa.pdf
Mineduc, OCDE & SIMCE. PISA Evaluación de las competencias lectoras para el
http://www.agenciaeducacion.cl/wp-
content/files_mf/pisa_evaluacion_de_las_competencias_lectoras_para_el_siglo_xxi_
final.pdf
Moore, M.& Zigo, D. (2005). Chicken Soup for the Science Fiction Soul: Breaking
the Genre Lock in the High School Literacy Experience. Journal of Curriculum.
21(3), 29-45.
Moreno, G. (2014, Abril 1). Prueba PISA: escolares chilenos están entre los más
Abril, 2015.
61
Mayhew, C., & MacKinnon, M. (2015). Critical Thinking about Fables: Examining
Prentice-Hall.
Nussbaum, M. C. (2012). Not for profit: Why democracy needs the humanities. NJ:
41.
academic performances of primary school children. Stud Home Comm Sci,3(2), 143-
147.
62
Oaks: Sage.
Paul, R. (1992). Critical thinking: What, why, and how. New Directions for
Educational Association.
Reid, J., & Anderson, P. (2012). Critical thinking: The foundation of education in a
Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003).
Salili, F., Chiu, C. Y., & Lai, S. (2001). The influence of culture and context on
Santelices, M. V., Ugarte, J. J., Flotts, P., Radovic, D., & Kyllonen, P. (2011).
Development Agenda.
Schein, E. H. (1993). How can organizations learn faster? The challenge of entering
Schkade, L. L., Roman S. & Uzawa, M. (1978). Human Information Processing and
58-64.
(Ed.)
64
Silva, E. (2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 630-
634.
893-912.
Who Avoid Reading or Find Reading Difficult. Children's Literature in the Reading
Trompenaars, F., & Hampden-Turner, C. (1998). Riding the waves of culture. New
York: McGraw-Hill.
annual meeting of the Association for the Study of Higher Education, Miami, FL
Tsui, L. (2000). Effects of campus culture on students' critical thinking. The Review
UNESCO. (2000). Dakar Framework for Action. Education for All: Meeting Our
http://unesdoc.unesco.org/images/0012/001211/121147e.pdf
Van Den Broek, P., Kendeou, P., Lousberg, S., & Visser, G. (2011). Preparing for
Van Gelder, T. (2005). Teaching critical thinking: Some lessons from cognitive
the changing role of parents, teachers, schools and educational policies. European
Veenman, M. V. J., Wilhelm, P., & Beishuizen, J. J. (2004). The relation between
WEF & BCG (2015). New Vision for Education Unlocking the Potential of
Williams, R. L., Oliver, R., Allin, J. L., Winn, B., & Booher, C. S. (2003).
Yeh, S. S. (2001). Tests worth teaching to: Constructing state-mandated tests that
ANEXOS
69
Anexo A. Tablas
Tabla 4: : Resumen puntajes y dimensiones
Indicadores
de
Habilidad
Definición
Sub
dimensiones
logro
Dificultad
Discriminación
Comprender
y
expresar
el
significado
o
importancia
de
una
gran
variedad
de
Clasificar
elementos
experiencias,
situaciones,
Interpretación
Categorización
según
ciertas
reglas,
datos
eventos,
juicios,
criterios
o
procedimientos
convenciones,
creencias,
reglas,
procedimientos
o
criterios.
0,7
0,48
Comparar
conceptos
o
afirmaciones
Examinar
ideas
determinando
similitudes
y
diferencias.
Identificar
las
relaciones
0,72
0,51
inferenciales
convencionales
y
reales
entre
declaraciones,
Identificar
si
un
preguntas,
conceptos,
argumento
está
a
favor
o
Detectar
argumentos
Análisis
descripciones
u
otras
formas
en
contra
de
una
de
representación
para
afirmación
dada.
0,21
0,33
expresar
creencias,
juicios,
experiencias,
razones,
Identificar
el
punto
información
u
opiniones
principal
de
un
argumento.
(*)
0,35
0,42
Analizar
argumentos
Identificar
las
razones
que
apoyan
un
argumento.
(*)
0,35
0,42
0,69 0,52
Comunicar
las
conclusiones
de
un
procedimiento.
(*)
school
Abstract
theoretical and practical work. Differences have been demonstrated between students
who receive education in this area from an early age and those who do not. However,
not enough work has been done to measure these skills in a classroom setting. Given
that the best time to teach critical thinking is during the first years of primary education,
we designed a test to determine the level of critical thinking among 3rd and 4th grade
students in Language Arts using a graphic novel. We showed the legitimacy of the
correlation, Reliability and Item Analysis. Using the instrument we studied how critical
thinking skills differ among 3rd grade students, according to their socioeconomic status
(SES), studying schools with low, middle and high SES. We found significant
differences between the schools, which suggest that there may be a relationship
between socioeconomic status and the development of critical thinking skills. The test
unidimensional models with models that provide a more detailed and multidimensional
1. - Introduction
Teaching critical thinking in schools is one of the main topics in the discussion
regarding so-called 21st Century skills (Greenhill, 2009). Critical thinking has been
is also considered fundamental if citizens are to perform their social, professional and
ethical duties (Griffin, McGaw & Care, 2012; Greenhill, 2009). Critical thinking skills
allow individuals to make autonomous decisions and to question beliefs when these are
There is a wide range of definitions of critical thinking. Some of these definitions are
2011); others are more focused on how critical thinking is received and developed in
education (Facione, 1990; Halpern, 2003; Bailin, 2002). Despite these divergent
approaches, one essential element of critical thinking is that it is a metacognitive
process. Critical thinking allows us to think not just about the world around us (first-
order skills), but also about the thought process itself (second-order skills) (Kuhn,
relating directly to how the context is perceived and understood (Halpern, 2003). For
example, it has been established that metacognitive processes linked to critical thinking
are fundamental to the transfer of acquired knowledge (Nickerson, 1988). This study
will use a definition of critical thinking that explicitly states its link to metacognition. It
(e.g. analysis, evaluation and inference) that, when used appropriately, increases the
theoretical and practical work (Lai, 2011). The inclusion of critical thinking in school
curriculums has been widely reported since at least the first half of the 20th Century,
influenced by John Dewey (Bean, 2011). Facione (1990) suggests that developing
critical thinking skills should be an objective for every grade level in the K-12
curriculum. In this sense, teaching critical thinking in schools should be integrated into
regular classroom activities (Bailin et al., 1999). It is also important to teach critical
thinking from early childhood. This is because significant differences have been
demonstrated between students who receive education in this area from an early age
defended repeatedly, not enough work has been done to measure these skills in a
thinking skills is essential for improving the quality of education. If tests are understood
to shape both the curriculum and teaching, then an efficient way to improve the quality
The literature reports that specific knowledge of the subject in which critical thinking
skills are being taught is needed if these skills are to be measured properly (Facione,
1990; Ennis 1989; McPeck 1981; Bailin, 2002; Willingham, 2008). In order to assess
children’s critical thinking skills in the classroom, the assessment must therefore be
situated and take into account the specific subject in which these skills are being taught
(Merrel, 2003). This is because of the influence that different subjects can have on the
thinking skills, their use depends on their connection to a specific subject (McPeck,
1981).
Ennis (1989) suggests that the best time to teach critical thinking is during the first
years of primary education. This suggestion is coherent with other studies, which
conclude that young children benefit from being taught and assessed on critical
thinking (Kennedy et al., 1991). However, the critical thinking tests that are available
are not designed for young children and do not focus on a specific subject. Instead, they
only measure critical thinking as a general skill (Ennis & Millman, 1985; Watson &
Glaser, 1980; Facione, 1990; Ennis & Weir, 1985; Halpern, 2003). Faced with this
problem, our first research question asks: “How can we design and validate a test that
determines the level of critical thinking among primary school children in a specific
subject?”
The relationship between socioeconomic status and the acquirement of skills is best
expressed by Bourdieu (1986). In his study, the author shows that the accumulation of
educational skills and abilities (Bourdieu, 1986). It has been revealed that a student’s
Goldthorpe, 1997). It has also been shown that levels of poverty and reading skills
among third grade students have an impact on their performance in higher levels of
status could also plausibly influence their level of critical thinking (Cheung et al.,
2001). Our second research question therefore asks: “How do critical thinking skills
2.- Methodology
A literature review was conducted during the first stage of this study in order to define
a model of critical thinking. The aim of this model is to provide suitable scaffolding of
teach upper-elementary school students higher-order skills (Songer & Gotwals, 2012).
setting (Halpern, 2003) and helps the teacher monitor how the skills are learned . This
The proposed model was based on the Delphi Report (Facione, 1990), in which a group
of experts defined the principal skills of a critical thinker. These skills include
Furthermore, the proposed model also incorporates the concepts of transferability and
The model proposed in this study is based on sequential scaffolding. During this
sequence, the students address how their thoughts are formulated (interpretation),
information (analysis and inference), and, finally, explain the results of their mental
Questions requiring reasoning were included in the model so as to track the explanatory
process. According to Berland & Mcneill (2012), the explanatory and reasoning
processes occur simultaneously. Using reasoning to put critical thinking into operation
makes measuring this skill much simpler (Yeh, 2001). Reasoning is particularly
important when it comes to measuring critical thinking as it is one of the most effective
tools for discriminating between ideas based on evidence and those that are merely
In a second phase of the design process, the subject matter for the test was defined.
The potential for integrating critical thinking skills from an early age is demonstrated
by studies which show that children can carry out activities involving such skills as
successfully as adults (Gelman & Markman, 1986; Willinghman, 2008). In this sense, it
was estimated that the earliest possible age for the test to be effective is 3rd grade. This
is because between 3rd and 5th grade the majority of students are capable of decoding
Critical thinking has a significant impact on the learning skills that are associated with
language. For example, Nippold et al. (2015) show that tasks based on critical thinking
require sentences with more complex syntax than tasks based on conversations.
Chapman (2014) suggests that learning critical thinking skills improves student
Furthermore, the literature reports that the use of critical thinking strategies is one of
the most important factors in the development of reading skills (Hong-Nam et al., 2014.
Additionally, O’Flahavan & Tierney (2013) suggest that reading and writing are both
highly productive tools for promoting critical thinking. Given the above, Language Arts
In order to do so, an adaptation of the plot of a well-known story was developed. The
selection criteria for choosing the story took into account that, between the ages of 8
and 10, students’ thinking becomes more logical, their memory and intellectual skills
improve, and they appreciate humour (Das, 2013). In this sense, the recommendation
by Moore & Zigo (2005) to use science fiction was followed. This is because, as a
genre, science fiction encourages the use of critical and creative reading. For this
reason, the classic novel by Jules Verne, Around the World in 80 Days, was selected
and adapted for children from 3rd grade. This adapted novel was used as a means to
In a third phase of the design process, it was felt that a graphic novel, i.e., a narrative
based on both text and images, was needed in order to help deliver the test. This
decision was based on the privileged position enjoyed by visual media in our culture
(Emmison & Smith, 2000; Sturken & Cartwright, 2009). The consequence of this is
that students basically learn from visual resources, especially during their first years of
school (Downey, 2009). Frey & Fisher (2004) acknowledge that the use of graphic
novels can stimulate interest in reading among students who are unmotivated by more
elements of the Language Arts curriculum, such as literacy (Schwarz, 2006). Another
argument in favour of choosing this format is that it allows for the inclusion of
elements of visual literacy that are associated with critical thinking assessment
standards (Facione, 1990; Lazo & Smith, 2014). Hassett & Schleble (2007) suggest that
when reading graphic novels students do not only improve their basic reading skills;
they also discover new ways to think about the interaction between the image and text.
Following this, a series of performance criteria were defined for each of the skills
these criteria. Questions were developed for each of the performance criterion so as to
cover all of the skills (interpretation, analysis, inference, evaluation and explanation).
explanations. This decision is justified by the fact that the cognitive strategies used by
(McNamara & Magliano, 2009). This link is particularly important when developing
not only based on the contents of the question; their explanation may also come from
metacognitive considerations, such as analysing the type of task at hand and different
strategies for completing the task (Flavell, 1979). Such emphasis is placed on
metacognition because it is this process that articulates critical thinking (Dwyer, 2014).
Given the above, it was decided that using short written activities based on questions
would be most beneficial. This is a particularly effective teaching strategy for assessing
argument (Bean, 2011). It was also decided to have the questions appear throughout the
story, as this has been proven to be a more effective strategy than asking the questions
at the end of the story (Van Den Broek et al., 2011). An example of this can be seen in
Figure 1, which shows one of the scenes from the graphic novel, together with a
question for the student. In this scene one of the characters (the Detective) tries to fool
by determining the role of visual images in a narrative. Here, the student has to be
capable of understanding the role of the object as part of the disguise and not as an
innate characteristic of the character. The second factor that is measured is evaluation,
by weighing up the pros and cons of a given alternative. In this case, the student
assesses whether the moustache is a good or bad disguise with which to fool the other
character. The third factor is explanation, as the students not only have to decide
whether or not it is a good disguise, they also have to give their reasons (Facione,
1990). The criteria used to evaluate their written responses are also made explicit. The
markers must therefore decide whether or not the student’s response meets these
criteria. If it does they are awarded 1 point; if it does not they are awarded 0 points. The
marking guidelines were developed with the guidance of two experts in education and
Language Arts, with the aim of ensuring that the exercises were in line with the skill
Construct validity, content validity and criterion validity were all analysed in order to
validate the test (Brown, 2000; Wolming, 1998). Cronbach’s alpha was also used for
this purpose (Cronbach, 1971). These analyses were supplemented by an item analysis.
The aim of this analysis was to gather data in order to highlight the need to review,
select and remove any questions that were very difficult, very easy, or that failed to
thinking skills between students in 3rd and 4th grade (Brown, 2000). The presence of
this difference validates the construct as it is coherent with the literature, which shows
that the level of critical thinking linked to metacognition increases with age (Veenman,
Wilhelm, & Beishuizen, 2004). This is explained by the fact that in 3rd grade students
can give causal explanations, make simple predictions (Hapgood et al., 2004) and
partially correct analogies (May et al., 2006). From 4th grade on, they are capable of
developing problem solving strategies, defining tasks and monitoring their thoughts
A panel of experts was used to demonstrate content validity (Rubio et al., 2003). The
arts. The panel’s opinions were then compared with the test developers’ opinions in
order to judge the level of consistency between the two (Dillashaw & Okey, 1980;
Nitko, 1996). Four think aloud sessions were also included in the study to analyse
content validity (Ericsson & Simon, 1993; Flaherty, 1974). These sessions were
conducted using students of a similar age to those used in the study, as suggested by
Facione (1990).
Correlations have been detected between critical thinking test scores and school grades
among Chilean high school students (Santelices et al., 2011). Pearson’s product
moment correlation was therefore used to detect correlations between the test scores
and Language Arts grades so as to analyse criterion validity. This sample strategy was
used as it was not possible to access the full report cards for all of the students (Ary et
al., 2013).
Furthermore, Cronbach’s alpha was also used to analyse the test’s reliability. This
index is used to measure the internal consistency of the test (Cronbach, 1971). An item
analysis was then conducted using the test results. The aim of this was to use objective
data, i.e. difficulty and discrimination indices for each question, to identify the need to
review, remove or modify any of the questions (Nitko, 1996). These indices were
then to observe which areas the students found the most and least difficult.
stratified sample was selected based on the three types of schools that are found in the
respectively (Gallego, 2002). Furthermore, to analyse construct validity, the study was
conducted with students from two grade levels: 3rd and 4th.
The participants in the sample, Table 1, included 301 students from 3rd and 4th grade
from the three types of schools. It was decided to use an ANOVA in order to prove the
3rd
Grade
Gender
/ Socioeconomic status
High
Middle
Low
Male
55
13
17
Female
47
17
14
Total
102
30
31
th
4
Grade
Gender/
Socioeconomic status
High
Middle
Low
Male
40
14
18
Female
42
14
10
Total
82
28
28
Table 1. Participants in the study by socioeconomic status and gender.
3.- Results
The results of the Shapiro-Wilk test suggest that the students from 3rd and 4th grade
were not normally distributed (3.246e-06 for 3rd grade and 1.613e-07 for 4th grade).
Furthermore, the Bartlett test shows that variances within the study groups were
homogeneous (a value of 0.3007). Although the normality assumption was not met,
Sheng (2008) suggests that ANOVA is a robust test. It is therefore possible to use this
analysis, even though some of the assumptions were violated. In order to do so, the
The test used to analyse the construct validity of the test suggests that there are
objectives that were established for measuring critical thinking. This panel was more
than 70% in agreement with the test that had been developed, which is an acceptable
level. The think aloud sessions also allowed questions that were misunderstood or
The Pearson product-moment correlation between the test scores and the students’
grades was 0.805 for 3rd grade and 0.723 for 4th grade. Dancey & Reidy (2004) classify
strong correlation as being greater than 0.7. The results of this analysis therefore
3.4- Reliability
Before the item analysis, Cronbach’s alpha for the test was 0.917. Following the item
analysis, Cronbach’s alpha was 0.909, which shows that the test is reliable (George &
Mallery, 2003). The latter value is the result that was obtained having analysed the
following section.
The results from the item analysis were organized according to the area of critical
thinking that was measured, and measured the level of difficulty and discrimination of
each item (Table 2). A range of between 0.4 and 0.6 was chosen for this study as
Womer (1968) suggests that this range can determine whether or not a question should
be included on a test. Nitko (1996) also suggests that items with a difficulty index of
less than 0.2 or 0.8 should also be rejected as they are too easy or too difficult. This
recommendation was taken into account for the test used in this study. Finally, it was
decided that questions with a difficulty index between 0.2 and 0.4 or between 0.6 and
0.8 would also be included, so long as they added to the coherence of the narrative and
provided further information. Furthermore, a discrimination index greater than 0.3 was
considered acceptable for any item on the test. Therefore, items with an index greater
than 0.3 were considered good and items with an index less than 0.3 were considered
bad (Adkins, 1974; Hinkle et al., 2003; Nitko, 1996). A summary of the distribution for
all of the questions according to their level of difficulty and discrimination can be
found in Table 4 in the Annex. This table also shows the skill and sub-skill to which
The results in Table 5 reveal that Analysis was the area with the highest level of
difficulty (0.36) and lowest level of discrimination (0.38). The area with the lowest
selection was made of the items that would initially be included on the test (items 2a, 3,
4, 6a, 10, 14b, 22, 23b and 27a). Nitko’s (1996) criteria were then used to select
additional items for the test (1a, 2b, 7, 8, 11, 12, 13, 14a, 16, 17a, 17,b, 18, 19a, 19b,
21, 26, 28a, 28b, 29a and 29b). The aim of this was to obtain more information, while
preserving the overall coherence of the test. Finally, one item (6b) was removed from
the test as it had a correlation of 1 with another item and both items targeted the same
performance indicator.
By doing so, the difficulty and discrimination levels for all of the items fell within the
Annex).
A comparison was made between the groups from the different types of schools (state,
state-subsidized and private). The distribution for one of the groups did not meet the
normality assumption for the Shapiro-Wilk test (high: 3.208e-05, middle: 0.07288, low:
0.0207). The Bartlett test also revealed that the variances within the groups were not
robust test and allows assumptions to be violated (Sheng, 2008). The Kruskal-Wallis
test was therefore used as a non-parametric alternative to the ANOVA test (Osborne,
at p<0.001. Tukey’s test then revealed that the differences between the groups were
significant at p >0.05.
To analyse these differences in more detail, Table 3 shows the average test score for
each school, as well as the standard deviation. The results clearly show that test scores
in high SES schools (22.15 for 4th grade and 19.3 for 3rd grade) are considerably higher
than in middle SES schools (14.75 for 4th grade and 10.73 for 3rd grade) and low SES
schools (10.61 for 4th grade and 6.97 for 3rd grade). It is also important to highlight that
the standard deviation between test scores in the high SES school decreases
considerably (28%) from 3rd grade to 4th grade (from 4.55 to 3.27 points). In the middle
SES school, the standard deviation remains practically the same (6.95 in 4th grade and
6.97 in 3rd grade), while in the low SES school there is an increase in the standard
deviation (from 6.06 to 6.2). This suggests that differences in student performance are
reduced over time in the high SES school, which is not the case in the other schools.
School SES Grade Average Standard
Score Deviation
4.- Conclusions
The aim of this study was to answer the two research questions stated above.
The first question asked “How can we design and validate a test that determines the level of
critical thinking among primary school children in a specific subject?” To answer this
question, a critical thinking test was designed and validated within the context of Language
Arts, using a graphic novel. The challenge highlighted in the research question was overcome
as the test was validated for both 3rd and 4th grade students.
It is interesting to note that no other test has used a graphic novel to measure the level of
critical thinking among 3rd and 4th grade students. There are several benefits to designing a
test using a graphic novel. These include the impact it has on student motivation and interest
(Chase et al., 2014), the opportunity it provides for developing higher-order skills (Pantaleo,
2014), their effectiveness for teaching literacy (Jennings et al., 2014) and the support they
provide to readers who have difficulty visualizing stories (Lyga, 2006). It remains as future
work to validate tests in other areas of the curriculum, such as science and mathematics.
The main limitation of the test that was used is the cultural context in which it was designed.
Cultural factors are particularly important when developing critical thinking skills in
Language Arts classes (Guo, 2013). Examples of this include the significant effect of culture
on critical thinking among university students when writing blogs (Shahsavar, 2014) and the
influence of institutional culture on critical thinking among students (Tsui, 2000). It is also
important to highlight that, as the theoretical framework was developed within a specific
cultural context, students may be at a disadvantage because of the structure of the test and not
because of their level of critical thinking (Howe, 1995; Childs, 1990). Further study is
required to analyse how local culture affects the theoretical framework for critical thinking
described in this study. It is also necessary to study how local culture affects the performance
of the test, in terms of both how the images are understood, as well as the structure of the
narrative. Quantification of cultural factors has been studied among adults (Hofstede, 2001),
but is only incipient among children (Hofstede, 1986). Developing a tool in this sense would
help with the robustness of a critical thinking test for children from different cultural
backgrounds.
Once the test had been validated, it was decided to answer the second research question,
which asked “How do critical thinking skills differ among 3rd grade students, according to
their socioeconomic status?” In this sense, significant differences were found between the
schools with different socioeconomic statuses. This suggests that there may be a relationship
between socioeconomic status and the development of critical thinking skills. This is
particularly evident when observing that the standard deviation among students in the high
SES school decreases over time, increasing homogeneity among students, while the standard
deviation remains the same in the other schools. Although further findings are needed in this
area, it would be worth asking whether this difference can be reduced by teaching the critical
This study is a first step towards giving primary schools access to a test that allows them to
measure the development of their students’ critical thinking skills. This test is an
that provide a more detailed and multidimensional picture of student learning (Conley, 2015).
The limitations of this study include the range of grade levels that were covered, the cultural
context in which it took place, and the area of the curriculum for which the test was
Acknowledgements
This research was funded by CONICYT-FONDECYT grant 1150045, and by the VRI
5.- References
Ary, D., Jacobs, L., Sorensen, C., & Walker, D. (2013). Introduction to research in
Bailin, S. (2002). Critical thinking and science education. Science & Education, 11(4), 361–
375.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Conceptualizing critical
Bean, J. C. (2011). Engaging ideas: The professor's guide to integrating writing, critical
thinking, and active learning in the classroom. San Francisco: John Wiley & Sons.
Berland, L. K., & McNeill, K. L. (2012). For whom is argument and explanation a necessary
Breen, R., & Goldthorpe, J. H. (1997). Explaining educational differentials towards a formal
Brown, J. D. (2000). What is construct validity. JALT Testing and Evaluation SIG
Chase, M., Son, E. H., & Steiner, S. (2014). Sequencing and Graphic Novels With Primary‐
Chapman, J. (2014). Critical Thinking and TOEIC: Advanced Skills for Teachers and Test-
Cheung, C. K., Rudowicz, E., Lang, G., Yue, X. D., & Kwan, A. S. (2001). Critical thinking
among university students: does the family background matter?. College Student
Childs, R. (1990). Gender bias and fairness. Practical Assessment, Research & Evaluation,
2(3).
Conley, D. (2015). A New Era for Educational Assessment. Education Policy Analysis
Archives, 23, 8.
Dancey C., Reidy J. (2004). Statistics Without Maths for Psychology: Using SPSS for
Das, A. (2013). The Impact of Children’s Literature in the Teaching of English to Young
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework
for the 21st century. Thinking Skills and Creativity, 12, 43-52.
Emmison, M., & Smith, P. (2000). Researching the Visual: Images, Objects, Contexts and
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed
Ennis, R.H. & Millman, J. (1985). Cornell Critical Thinking Test-Level X. Pacific Grove,
Ennis, R.H. & Weir, E. (1985). The Ennis-Weir essay test: An instrument for testing and
Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis: Verbal reports on data.
Flaherty, E. G. (1974). The thinking aloud technique and problem-solving ability. Journal of
Gelman, S. A., & Markman, E. M. (1987). Young children's inductions from natural kinds:
George, D and Mallery, P. 2003. SPSS for Windows step by step: a simple guide and
Greenhill, V. (2009). P21 framework definitions document, Retrieved December 15, 2010,
from,
http://www.21stcenturyskills.org/documents/p21_framework_definitions_052909.pdf.
Griffin, P., McGaw, B., & Care, E. (2012). Assessment and Teaching of 21st Century
Halpern, D. (2003). Thought and knowledge: An introduction to critical thinking (4th ed.).
Hapgood, S., Magnusson, S. J., & Sullivan-Palincsar, A. (2004). Teacher, text, and
experience: A case of young children's scientific inquiry. Journal of the Learning Sciences,
13(A), 455-505.
Hassett, D. D., & Schleble, M. B. (2007). Finding Space and Time for the Visual in K-12
Hernandez, D. J. (2011). Double Jeopardy: How Third-Grade Reading Skills and Poverty
organizations across nations. Geert Hofstede (Ed.). Thousand Oaks, California: Sage.
Hong-Nam, K., Leavell, A. G., & Maher, S. (2014). The Relationships among Reported
Howe, K. (1995). Validity, Bias and Justice in Educational Testing: The Limits of
Hudgins, B. B., et al. (1989). Children's critical thinking: A model for its analysis and
Jennings, K. A., Rule, A. C., & Zanden, S. M. V. (2014). Fifth Graders’ Enjoyment, Interest,
Springer Netherlands.
Kennedy, M., Fisher, M. B., & Ennis, R. H. (1991). Critical thinking: Literature review and
needed research. Educational values and cognitive instruction: Implications for reform, 2,
11-40.
Kim, K., Sharma, P., Land, S. M., & Furlong, K. P. (2013). Effects of active learning on
16–26.
Lai, E. R. (2011). Critical thinking: A literature review. Pearson's Research Reports, 6, 40-
41.
Lyga, A. A. (2006). Graphic Novels for (Really) Young Readers: Owly, Buzzboy, Pinky and
Stinky. Who Are These Guys? And Why Aren't They Ever on The Shelf?. School Library
Marques, J.F., (2012), Moving from trance to think: why we need to polish our critical
thinking skills, International Journal of Leadership Studies, Vol. 7 Iss. 1, 2012, 87-95
May, D. B., Hammer, D., & Roy, P. (2006). Children's analogical reasoning in a third grade
Moore, M.& Zigo, D. (2005). Chicken Soup for the Science Fiction Soul: Breaking the Genre
Lock in the High School Literacy Experience. Journal of Curriculum. 21(3), 29-45.
Mulnix, J. W. (2012). Thinking critically about critical thinking. Educational Philosophy and
MacKinnon, M. (2015). Critical Thinking about Fables: Examining Language Production and
Nitko, A. J. (1996). Educational Assessment of Students. 2nd ed. New Jersey: Prentice-Hall.
O’Flahavan, J. & Tierney, R. (2013). Reading, Writing, and Critical Thinking. Educational
Osakwe, R. N. (2009). The effect of early childhood education experience on the academic
Osborne, J. W. (Ed.). (2008). Best practices in quantitative methods. Thousands Oaks: Sage.
Pantaleo, S. (2014). Reading images in graphic novels: Taking students to a'greater thinking
Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying
content validity: Conducting a content validity study in social work research. Social Work
Santelices, M. V., Ugarte, J. J., Flotts, P., Radovic, D., & Kyllonen, P. (2011). Measurement
of New Attributes for Chile's Admissions System to Higher Education. ETS Research Report
Schwarz, G. (2006). Expanding literacies through graphic novels. English Journal, 58-64.
Shahsavar, Z. (2014). The Impact Of Culture On Using Critical Thinking Skills Through The
Sheng, Y. (2008). Testing the assumptions of analysis of variance, In Osborne, J.W. (Ed.)
entry points of learning progressions. Journal of Research in Science Teaching, 49(2), 141-
165.
Sturken, M., Cartwright, L., & Sturken, M. (2001). Practices of looking: An introduction to
Swaggerty, E. A. (2015). Selecting Engaging Texts for Upper Elementary Students Who
Avoid Reading or Find Reading Difficult. Children's Literature in the Reading Program:
Tsui, L. (2000). Effects of campus culture on students' critical thinking. The Review of
UNESCO. (2000). Dakar Framework for Action. Education for All: Meeting Our Collective
Commitments. Adopted by the World Education Forum, Dakar, Senegal, April 26–28, 1999.
Van Den Broek, P., Kendeou, P., Lousberg, S., & Visser, G. (2011). Preparing for reading
comprehension: Fostering text comprehension skills in preschool and early elementary school
Veenman, M. V. J., Wilhelm, P., & Beishuizen, J. J. (2004). The relation between
Watson, G. B., & Glaser, E. M. (1980). WGCTA Watson-Glaser critical thinking appraisal
Yeh, S. S. (2001). Tests worth teaching to: Constructing state-mandated tests that emphasize
Performance
Skill
Definition1
Sub-‐Skill
Indicators
Difficulty
Discrimination
Classifying
elements
Categorize
according
to
certain
rules,
criteria
or
procedures.
0.7
0.48
Identifying
which
elements
are
important
for
solving
a
problem.
Understanding
and
0.65
0.34
expressing
the
meaning
or
Decode
significance
importance
of
a
wide
variety
Interpretation
of
experiences,
situations,
Determining
the
role
of
data,
events,
judgements,
images
in
a
narrative.
(**)
conventions,
beliefs,
rules,
0.8
0.35
procedures
or
criteria.
Representing
a
result
or
operation
in
different
ways.
Clarify
meaning
0.48
0.49
Clarifying
the
use
of
a
convention.
0.64
0.38
Clarifying
an
idea
by
0.39
0.37
1
The definitions of the skills are taken from the Delphi Report (Facione, 1990)
giving
an
example.
0.69 0.52
Communicating
the
conclusions
reached
by
following
a
procedure.
(**)
Stating
and
justifying
reasoning
in
terms
of
the
0.67
0.68
evidential,
conceptual,
methodological,
Graphically
criteriological,
and
communicating
the
Reasoning
/
Explanation
State
results
contextual
considerations
relationship
between
upon
which
one's
results
concepts
and
ideas.
were
based;
also
presenting
0.46
0.45
one's
reasoning
in
the
form
of
cogent
arguments.
Developing
a
narrative
which
examines
the
changes
in
a
topic
over
time.
0.82
0.35
Stating
the
standards
used
to
assess
a
literary
work.
0.75
0.64
Justify
procedures
Describing
the
strategy
used
to
make
a
reasonable
decision.
0.18
0.41
Arguing
in
favour
of
or
against
a
point
of
view.
0.63
0.65
Explaining
the
comprehension
of
a
Present
arguments
concept.
(**)
0.35
0.14
Identifying
and
expressing
evidence
as
part
of
reasoning
for
and
against
other
people’s
and
one’s
own
way
of
thinking.
0.72
0.66
Table 5. Average scores for Difficulty and Discrimination, before and after the Item
Analysis.
If you did not co-author this submission, please contact the Corresponding Author of this
submission atdamian.gelerstein@gmail.com;dgelerst@uc.cl; do not follow the link below.
An Open Researcher and Contributor ID (ORCID) is a unique digital identifier to which you
can link your published articles and other professional activities, providing a single record of
all your research.
We would like to invite you to link your ORCID ID to this submission. If the submission is
accepted, your ORCID ID will be linked to the final published article and transferred to
CrossRef. Your ORCID account will also be updated.
To do this, visit our dedicated page in EES. There you can link to an existing ORCID ID or
register for one and link the submission to it:
http://ees.elsevier.com/tsc/l.asp?i=7055&l=UUAIE5G1
Like other Publishers, Elsevier supports ORCID - an open, non-profit, community based
effort - and has adapted its submission system to enable authors and co-authors to connect
their submissions to their unique ORCID IDs.
Thank you,