Marcos Y. Lopez Marcos Y. Lopez Marcos Y. Lopez Marcos Y.

Lopez
Marcos Y. Lopez Marcos Y. Lopez Marcos Y. Lopez Marcos Y. Lopez
MaIoIos, IuIacan MaIoIos, IuIacan

Constiuct a multi
Constiuct a multi
-
-
aspect geneial
aspect geneial
-
-
knowledge
knowledge
ciitical thinking test using Multiple
ciitical thinking test using Multiple
-
-
Choice
Choice
-
-
Question (MCQ) foimat.
Question (MCQ) foimat.

Establish evidence of the constiuct validity

Establish evidence of the constiuct validity
of the test using veibal iepoits of thinking.
of the test using veibal iepoits of thinking.

Deteimine the difficulty and disciiminating

Deteimine the difficulty and disciiminating
indices of each item of the test.
indices of each item of the test.

Deteimine the ieliability of the test.

Deteimine the ieliability of the test.
Ciitical Thinking is ieasonable
ieflective thinking focused on
deciding what to believe oi do
(Ennis, 1987, 1996, 2OO2,
2O11)
1. 1.
Caie that theii beliefs aie tiue, and that
Caie that theii beliefs aie tiue, and that
theii decisions aie justified, that is, caie to
theii decisions aie justified, that is, caie to
get it iight to the extent possible.
get it iight to the extent possible.
2. 2.
Caie to undeistand and piesent a position
Caie to undeistand and piesent a position
honestly and cleaily, theiis as well as
honestly and cleaily, theiis as well as
otheis
otheis
'
'
.
.
3. 3.
Caie about the dignity and woith of eveiy
Caie about the dignity and woith of eveiy
peison.
peison.
1.
1.
Iocus on a question.
Iocus on a question.
2.
2.
Analyze aiguments.
Analyze aiguments.
3.
3.
Ask and answei claiification and/oi
Ask and answei claiification and/oi
challenge question.
challenge question.
4.
4.
Reasonably judge the ciedibility of a
Reasonably judge the ciedibility of a
souice.
souice.
5.
5.
Obseive, and ieasonably judge
Obseive, and ieasonably judge
obseivation iepoits.
obseivation iepoits.
6.
6.
Use theii own otheiwise
Use theii own otheiwise
-
-
established
established
conclusions.
conclusions.
7.
7.
Deduce, and judge deductions.
Deduce, and judge deductions.
8. 8.
Induce, and judge induction.
Induce, and judge induction.
9. 9.
Make and judge value judgments.
Make and judge value judgments.
1O. 1O.
Define teims and judge definitions.
Define teims and judge definitions.
11. 11.
Attiibute unstated assumptions.
Attiibute unstated assumptions.
12. 12.
Considei and ieason fiom piemises,
Considei and ieason fiom piemises,
ieasons, assumptions, positions, and othei
ieasons, assumptions, positions, and othei
piopositions with which they disagiee oi
piopositions with which they disagiee oi
about which they aie in doubt, without
about which they aie in doubt, without
letting the disagieement oi doubt inteifeie
letting the disagieement oi doubt inteifeie
with theii thinking.
with theii thinking.
13. 13.
Integiate the dispositions and othei
Integiate the dispositions and othei
abilities in making and defending a
abilities in making and defending a
decision.
decision.
14. 14.
Pioceed in an oideily mannei appiopiiate
Pioceed in an oideily mannei appiopiiate
to the situation.
to the situation.
15. 15.
Take into account the feelings, level of
Take into account the feelings, level of
knowledge, and degiee of sophistication
knowledge, and degiee of sophistication
of otheis.
of otheis.
16. 16.
Employ appiopiiate ihetoiical stiategies
Employ appiopiiate ihetoiical stiategies
in discussions and piesentations,
in discussions and piesentations,
including employing and ieacting to
including employing and ieacting to
fallacy labels in an appiopiiate mannei
fallacy labels in an appiopiiate mannei.
Phasc 1
Test Test
Conceptualization Conceptualization
Phasc 2
Development of a Development of a
Test Plan Test Plan
Phasc 3
Development of the Development of the
Test Items Test Items
Phasc 4
Iace and Content Iace and Content
Validation of the Test Validation of the Test
Phasc 5
Revision of the Test Revision of the Test
Items Items
Phasc 6
Pie Tiy Pie Tiy - - Out of the Out of the
Test Test
Phasc 7
Actual Tiy Actual Tiy Out of the Test Out of the Test
Phasc 8
Constiuct Validation of the Constiuct Validation of the
Test using Veibal Repoits of Test using Veibal Repoits of
Thinking Thinking
Phasc 1
Tcst
Cnnccptualizatinn
Phasc 1 Phasc 1
Tcst Tcst
Cnnccptualizatinn Cnnccptualizatinn
Spccify thc purpnscs nf thc tcst. Spccify thc purpnscs nf thc tcst. Spccify thc purpnscs nf thc tcst.
Spccify thc targct uscrs nf thc
tcst.
Spccify thc targct uscrs nf thc Spccify thc targct uscrs nf thc
tcst. tcst.
Phasc 2
Dcvclnpmcnt
nf a
Tcst Plan
Phasc 2 Phasc 2
Dcvclnpmcnt Dcvclnpmcnt
nf a nf a
Tcst Plan Tcst Plan
Adnpt a dcfinitinn/cnnccptinn nf thc critical thinking
that itcms wnuld scrvc tn tcst.
Adnpt a dcfinitinn/cnnccptinn nf thc critical thinking Adnpt a dcfinitinn/cnnccptinn nf thc critical thinking
that itcms wnuld scrvc tn tcst. that itcms wnuld scrvc tn tcst.
Choose the test foimat.
Choose the test foimat. Choose the test foimat.
Prcparc thc tablc nf spccificatinns.
Prcparc thc tablc nf spccificatinns. Prcparc thc tablc nf spccificatinns.
Specify administiation and scoiing method.
Specify administiation and scoiing method. Specify administiation and scoiing method.
Phasc 3
Dcvclnpmcnt nf
thc Tcst Itcms
Phasc 3 Phasc 3
Dcvclnpmcnt nf Dcvclnpmcnt nf
thc Tcst Itcms thc Tcst Itcms
Devise initial tiial items to test the ciiteiia and piinciples Devise initial tiial items to test the ciiteiia and piinciples
of ciitical thinking. of ciitical thinking.
Piepaie the key to coiiection with justification based on Piepaie the key to coiiection with justification based on
ciiteiia and piinciples of ciitical thinking. ciiteiia and piinciples of ciitical thinking.
Phase 4
Iace and Content
Validation of the
Test
Phase 4 Phase 4
Iace and Content Iace and Content
Validation of the Validation of the
Test Test
Examinc thc physical appcarancc and cnntcnt nf tcst
itcms dnnc by tcachcrs frnm diffcrcnt disciplincs.
Examinc thc physical appcarancc and cnntcnt nf tcst Examinc thc physical appcarancc and cnntcnt nf tcst
itcms dnnc by tcachcrs frnm diffcrcnt disciplincs. itcms dnnc by tcachcrs frnm diffcrcnt disciplincs.
Sciutinize the contents of the test done by
ciitical thinking expeits.
Sciutinize the contents of the test done by Sciutinize the contents of the test done by
ciitical thinking expeits. ciitical thinking expeits.
Phase 5 Phase 5
Revision of the Test Items Revision of the Test Items
Rcvisc thc tcst itcms bascd nn thc
suggcstinns nf critical thinking cxpcrts.
Phasc 6
Prc Try-Out nf thc Tcst
Phasc 6
Prc Try-Out nf thc Tcst
Analyze the iesult of item analysis.
Analyze the iesult of item analysis.
Rcvisc thc tcst bascd nn suggcstinns nf grnup nf
studcnts whn participatcd in thc prc try-nut nf thc
tcst.
Rcvisc thc tcst bascd nn suggcstinns nf grnup nf Rcvisc thc tcst bascd nn suggcstinns nf grnup nf
studcnts whn participatcd in thc prc try studcnts whn participatcd in thc prc try- -nut nf thc nut nf thc
tcst. tcst.
Phasc 7
Phasc 7
Actual Try
Actual Try
-
-
Out nf
Out nf
thc Tcst
thc Tcst
Analyze the iesult of item Analyze the iesult of item
analysis. analysis.
Deteimine the ieliability Deteimine the ieliability
of the test. of the test.
Phasc 8
Phasc 8
Cnnstruct
Cnnstruct
Validatinn
Validatinn
nf thc Tcst
nf thc Tcst
Using
Using
Vcrbal
Vcrbal
Rcpnrts nf
Rcpnrts nf
Thinking
Thinking
1. 1. Dctcrminc thc cxaminccs whn wnuld bc part nf scts Dctcrminc thc cxaminccs whn wnuld bc part nf scts
nf intcrvicwccs. nf intcrvicwccs.
2. 2. Dcvclnp an intcrvicwing mcthndnlngy fnr cliciting Dcvclnp an intcrvicwing mcthndnlngy fnr cliciting
trustwnrthy rcpnrts nf thinking. trustwnrthy rcpnrts nf thinking.
3. 3. Cnllcct vcrbal rcpnrts nf thinking nn itcms. Cnllcct vcrbal rcpnrts nf thinking nn itcms.
4. 4. Scnrc vcrbal rcpnrts nf thinking using thc Scnrc vcrbal rcpnrts nf thinking using thc
principlcs and critcria nf critical thinking as a principlcs and critcria nf critical thinking as a
standard, and scnring answcrs chnscn standard, and scnring answcrs chnscn
accnrding tn thc answcr kcy. accnrding tn thc answcr kcy.
6. 6. Mndify suspcctcd itcms. Mndify suspcctcd itcms.
7. 7. Retry the revised or the replaced items Retry the revised or the replaced items
with steps 3 with steps 3- -6. 6.
5. Cnmparc thinking nn itcms tn pcrfnrmancc nn Cnmparc thinking nn itcms tn pcrfnrmancc nn
thcm and systcmatically accumulatc thcsc thcm and systcmatically accumulatc thcsc
cnmparisnns acrnss cxaminccs fnr cach Itcm. cnmparisnns acrnss cxaminccs fnr cach Itcm.

Dr. David Hitchcnck frnm McMastcr

Dr. David Hitchcnck frnm McMastcr
Univcrsity, Hamiltnn Canada
Univcrsity, Hamiltnn Canada

Dr. Dnna Warrcn frnm Thc Univcrsity

Dr. Dnna Warrcn frnm Thc Univcrsity
nf Wiscnnsin, Stcvcns Pnint, Wiscnnsin,
nf Wiscnnsin, Stcvcns Pnint, Wiscnnsin,
USA
USA

Dr. Susana Nuccctclli, frnm St. Clnud

Dr. Susana Nuccctclli, frnm St. Clnud
Statc Univcrsity, St. Clnud, Minncsnta,
Statc Univcrsity, St. Clnud, Minncsnta,
USA
USA
Dr. David Hitchcnck Dr. David Hitchcnck' 's samplc nf cvaluatinn nn itcm 23: s samplc nf cvaluatinn nn itcm 23:
Item 23 (undei Section 2: ciedibility judgment test ):
A. A. Mary, a prnfcssnr nf Binlngy, whn is sn apprcciativc nf naturc s Mary, a prnfcssnr nf Binlngy, whn is sn apprcciativc nf naturc spnttcd pnttcd
snmcthing pcrching nn a branch nf a trcc partly cnvcrcd by snmc snmcthing pcrching nn a branch nf a trcc partly cnvcrcd by snmc trcc trcc
lcavcs, said, lcavcs, said, " "That nnc pcrching nn thc branch nf a trcc must bc a That nnc pcrching nn thc branch nf a trcc must bc a
hcn. hcn." "
B. B. Rita alsn a naturc lnvcr lnnkcd at thc samc nbjcct which was par Rita alsn a naturc lnvcr lnnkcd at thc samc nbjcct which was partly tly
cnvcrcd by snmc trcc lcavcs, said, cnvcrcd by snmc trcc lcavcs, said, " "That is nnt a hcn. That is nnt a hcn. It must bc a It must bc a
rnnstcr. rnnstcr." "
C. C. Ncithcr statcmcnt is mnrc bclicvablc. Ncithcr statcmcnt is mnrc bclicvablc.
C ( ThIs one Is trIcky, because one mIght suppose
C ( ThIs one Is trIcky, because one mIght suppose
that Mary
that Mary
'
'
s expertIse gIves her an edge In
s expertIse gIves her an edge In
determInIng whether a bIrd perchIng on a tree Is a
determInIng whether a bIrd perchIng on a tree Is a
hen or a rooster, but the fact that the object Is
hen or a rooster, but the fact that the object Is
partIy covered by tree Ieaves suggests that neIther
partIy covered by tree Ieaves suggests that neIther
can see enough to be sure, gIven that even Iay
can see enough to be sure, gIven that even Iay
peopIe can dIstInguIsh a rooster from a hen. By the
peopIe can dIstInguIsh a rooster from a hen. By the
way, do hens and roosters, perch on trees? Perhaps
way, do hens and roosters, perch on trees? Perhaps
the exampIe couId be made more reaIIstIc.)
the exampIe couId be made more reaIIstIc.)
Rcgarding Dircctinns nn Scctinn 1 nf thc Tcst:
Rcgarding Dircctinns nn Scctinn 1 nf thc Tcst:
Shc suggcstcd:
Shc suggcstcd:
In some of the passages, the underIIned
sentence doesn't strIke me as concIusIon.
How about modIfyIng the InstructIons as foIIows:
Each itcm prcscnts a sct nf statcmcnts, nnc nf which is undcrlin Each itcm prcscnts a sct nf statcmcnts, nnc nf which is undcrlincd. Dn cd. Dn
nnt bc cnnccrncd with whcthcr nr nnt thcsc statcmcnts arc truc. nnt bc cnnccrncd with whcthcr nr nnt thcsc statcmcnts arc truc.
Mark Items 1 through 5 accordIng to the foIIowIng system:
If the underIIned statement foIIows necessarIIy
from the other statements gIven, mark n.
If the underIIned statement contradIcts the other
statements gIven, mark B.
If the underIIned statement neIther foIIows
necessarIIy nor contradIcts the other statements
gIven, mark C.
Dear Prof. Lopez, I had a generaI Iook at
Dear Prof. Lopez, I had a generaI Iook at
your materIaIs for the vaIIdatIon test In
your materIaIs for the vaIIdatIon test In
crItIcaI thInkIng. They seem very weII
crItIcaI thInkIng. They seem very weII
thought and go beyond what I teach here at
thought and go beyond what I teach here at
St. CIoud State UnIversIty. I have no
St. CIoud State UnIversIty. I have no
suggestIons to make because your materIaIs
suggestIons to make because your materIaIs
are as good as they are.
are as good as they are.
Best wIshes, Susana.
Best wIshes, Susana.
Vcrbatim Transcriptinns nf thc vcrbal rcpnrts nf thinking nf snm Vcrbatim Transcriptinns nf thc vcrbal rcpnrts nf thinking nf snmc c
studcnts in sclcctcd itcms in ccrtain scctinns nf thc tcst. studcnts in sclcctcd itcms in ccrtain scctinns nf thc tcst.
Itcm 17 ( Dcductinn) Studcnt A:
Itcm 17 ( Dcductinn) Studcnt A:
Letter n kasI Is based on what the speaker saId ahhh. yun taIaga yung
justIfIcatIon kung bakIt yung ano po death penaIty Is effectIve. KasI after nung
InImpose yung death penaIty In a matter of 30 days daw bumaba yung crIme
rate ng bansa, So, cIearIy, parang yun dIn yung, death penaIty nga ang dahIIan
kung bakIt ahhh. bumababa ung crIme rate. I mean effectIve sIya. KasI nga,
after nung InImpose 'yon dun sa bansa, bumaba yung percentage ng crIme rate
nIIa. So Ietter n.
(My answer Is Ietter n because based on what the speaker saId that Is reaI
justIfIcatIon why death penaIty Is effectIve. Because after the ImposItIon of death
penaIty, In a matter of 30 days crIme rate In that country went down. So,
cIearIy, death penaIty Is the reason why crIme rate went down. I mean that Is
effectIve because after Its ImposItIon In that country percentage of crIme rate
went down. So Ietter n.)
Itcm 21 (Crcdibility) Studcnt D:
Itcm 21 (Crcdibility) Studcnt D:
RegardIess of theIr jobs.ahhh.Mateo beIng an
agrIcuIturIst and Mary beIng a BIoIogy professor my
ratIonaIIzatIon reIIed more on.ahhh.the source of the
InformatIon where Mateo got hIs InformatIon whIch Is
from the company that manufactures DEF powder. So
more or Iess the InformatIon that they wIII be sendIng
out are of course favourabIe to theIr product. nnd then
for the second one, It Is an Independent group. So
probabIy, theIr.ahhh.theIr resuIts are
more.ahhh.unIIkeIy to be bIased. So my answer for
number 21 Is Ietter B.
Itcm 54 (Inductinn) Studcnt H:
ThIs Is aIso n. It affIrms that nBC peIIets are
ThIs Is aIso n. It affIrms that nBC peIIets are
IethaI. But It onIy shows that two rats dIed from
IethaI. But It onIy shows that two rats dIed from
the group and It doesn
the group and It doesn
'
'
t taIk about other rats. The
t taIk about other rats. The
rats In the group that dIdn
rats In the group that dIdn
'
'
t have nBC peIIets were
t have nBC peIIets were
heaIthy. The InformatIon about the other group
heaIthy. The InformatIon about the other group
that had nBC peIIets two rats dIed. So, I am gonna
that had nBC peIIets two rats dIed. So, I am gonna
assume that a factor that they dIed was because of
assume that a factor that they dIed was because of
nBC peIIets. The onIy thIng Is
nBC peIIets. The onIy thIng Is
.
.
No
No
.
.
No
No
.
.
There
There
Is no other InformatIon about the other rats from
Is no other InformatIon about the other rats from
the second group wIth nBC peIIets. But from the
the second group wIth nBC peIIets. But from the
InformatIon gIven, I go wIth n.
InformatIon gIven, I go wIth n.
Thc intcrnal cnnsistcncy
Thc intcrnal cnnsistcncy
mcasurc nf rcliability nf Thc
mcasurc nf rcliability nf Thc
CEU
CEU
-
-
Lnpcz Critical Thinking
Lnpcz Critical Thinking
Tcst was cstablishcd using thc
Tcst was cstablishcd using thc
Kudcr
Kudcr
-
-
Richardsnn Fnrmula 20
Richardsnn Fnrmula 20
(KR
(KR
-
-
20). Thc nbtaincd KR
20). Thc nbtaincd KR
-
-
20
20
cncfficicnt was .68.
cncfficicnt was .68.

23 numbcr nf itcms cnnsidcrcd as

23 numbcr nf itcms cnnsidcrcd as
vcry difficult
vcry difficult

44 numbcr nf itcms cnnsidcrcd

44 numbcr nf itcms cnnsidcrcd
difficult
difficult

15 numbcr nf itcms cnnsidcrcd

15 numbcr nf itcms cnnsidcrcd
mndcratcly difficult
mndcratcly difficult

5 numbcr nf itcms cnnsidcrcd

5 numbcr nf itcms cnnsidcrcd
casy
casy

50 numbcr nf itcms cnnsidcrcd

50 numbcr nf itcms cnnsidcrcd
mndcratcly discriminating
mndcratcly discriminating

36 numbcr nf itcms cnnsidcrcd

36 numbcr nf itcms cnnsidcrcd
discriminating
discriminating

1 numbcr nf itcm cnnsidcrcd

1 numbcr nf itcm cnnsidcrcd
vcry discriminating
vcry discriminating