SeparatingProgrammingSheepfromNonProgrammingGoats

blog.codinghorror.com /separatingprogrammingsheepfromnonprogramminggoats/

Abunchofpeoplehavelinkedtothisacademicpaper,whichproposesawaytoseparateprogrammingsheepfromnonprogramminggoatsincomputerscience
classeslongbeforethestudentshaveevertouchedaprogramoraprogramminglanguage:

Allteachersofprogrammingfindthattheirresultsdisplaya'doublehump'.Itisasiftherearetwopopulations:thosewhocan[program],andthose
whocannot[program],eachwithitsownindependentbellcurve.Almostallresearchintoprogrammingteachingandlearninghaveconcentratedon
teaching:changethelanguage,changetheapplicationarea,useanIDEandworkonmotivation.Noneofitworks,andthedoublehumppersists.We
haveatestwhichpicksoutthepopulationthatcanprogram,beforethecoursebegins.Wecanpickapartthedoublehump.Youprobablydon'tbelieve
this,butyouwillafteryouhearthetalk.Wedon'tknowexactlyhow/whyitworks,butwehavesomegoodtheories.

Iwasn'tawarethatthedichotomybetweenprogrammersandnonprogrammerswassopronouncedatthisearlystage.DanBricklintouchedonthistopicinhisessay,
WhyJohnnyCan'tProgram.Butevidentlyit'scommonknowledgeamongstthosewhoteachcomputerscience:

Despitetheenormouschangeswhichhavetakenplacesinceelectroniccomputingwasinventedinthe1950s,somethingsremainstubbornlythesame.
Inparticular,mostpeoplecan'tlearntoprogram:between30%and60%ofeveryuniversitycomputersciencedepartment'sintakefailthefirst
programmingcourse.Experiencedteachersarewearybutneverobliviousofthisfactbrighteyedbeginnerswhobelievethattheoldonesmusthave
beendoingitwronglearnthetruthfrombitterexperienceandsoithasbeenforalmosttwogenerations,eversincethesubjectbeganinthe1960s.

Youmaythinkthetestthey'reproposingtodetermineprogrammingaptitudeiscomplex,butit'snot.Here'squestionone,verbatim:
Read the following statements and tick the box next to the correct answer.
int a = 10;
int b = 20;
a = b;
The new values of a and b are:
[ ] a = 20 b = 0
[ ] a = 20 b = 20
[ ] a = 0 b = 10
[ ] a = 10 b = 10
[ ] a = 30 b = 20
[ ] a = 30 b = 0
[ ] a = 10 b = 30
[ ] a = 0 b = 30
[ ] a = 10 b = 20
[ ] a = 20 b = 10

Thistestseemstrivialtoprofessionalprogrammers,butremember,it'sintendedforstudentswhohaveneverlookedatalineofcodeintheirlives.Theother12
questionsareallvariationsonthesameassignmenttheme.

Theauthorsofthepaperpositthattheprimaryhurdlesincomputerscienceare..

1.assignmentandsequence
2.recursion/iteration
3.concurrency*

..inthatorder.Thus,westartbytestingtheveryfirsthurdlenoviceprogrammerswillencounter:assignment.Thetestresultsdividedthestudentscleanlyintothree
groups:

44%ofstudentsformedaconsistentmentalmodelofhowassignmentworks(evenifincorrect!)
39%studentsneverformedaconsistentmodelofhowassignmentworks.
8%ofstudentsdidn'tgiveadamnandlefttheanswersblank.

Thetestwasadministeredtwiceonceatthebeginning,beforeanyinstructionatall,andagainafterthreeweeksofclass.Thestrikingthingisthattherewasvirtuallyno
movementatallbetweenthegroupsfromthefirsttosecondtest.Eitheryouhadaconsistentmodelinyourmindimmediatelyuponfirstexposuretoassignment,thefirst
hurdleinprogrammingorelseyouneverdevelopedone!

Theauthorsfoundanextremelyhighlevelofcorrelationbetweensuccessatprogrammingandformingaconsistentmentalmodel:
 Clearly,Dehnahdi'stestisnotaperfectdividerofprogrammingsheepfromnonprogramminggoats.Nevertheless,ifitwereusedasanadmissions
barrier,andonlythosewhoscoredconsistentlywereadmitted,thepass/failstatisticswouldbetransformed.Inthetotalpopulation32outof61
(52%)failedinthefirsttestconsistentgrouponly6outof27(22%).Webelievethatwecanclaimthatwehaveapredictivetestwhichcanbetaken
priortothecoursetodetermine,withaveryhighdegreeofaccuracy,whichstudentswillbesuccessful.Thisis,sofarasweareaware,thefirsttesttobe
abletoclaimanydegreeofpredictivesuccess.

Ihighlyrecommendreadingthroughthedraftpaper(pdf),whichwasremarkablyentertainingforwhatIthoughtwasgoingtobeadry,academicpaper.Instead,itreads
likeablogentry.It'sfilledwithinterestinginsightslikethisone:

Ithastakenussometimetodaretobelieveinourownresults.Itnowseemstous,althoughweareawarethatatthispointwedonothavesufficientdata,
andsoitmustremainaspeculation,thatwhatdistinguishesthethreegroupsinthefirsttestistheirdifferentattitudestomeaninglessness.

Formallogicalproofs,andthereforeprogramsformallogicalproofsthatparticularcomputationsarepossible,expressedinaformalsystemcalleda
programminglanguageareutterlymeaningless.Towriteacomputerprogramyouhavetocometotermswiththis,toacceptthatwhateveryou
mightwanttheprogramtomean,themachinewillblindlyfollowitsmeaninglessrulesandcometosomemeaninglessconclusion.Inthetest
theconsistentgroupshowedapreacceptanceofthisfact:theyarecapableofseeingmathematicalcalculationproblemsintermsofrules,andcanfollow
thoseruleswheresoevertheymaylead.Theinconsistentgroup,ontheotherhand,looksformeaningwhereitisnot.Theblankgroupknowsthatitis
lookingatmeaninglessness,andrefusestodealwithit.

Everyoneshouldknowhowtouseacomputer,butnoteveryoneneedstobeaprogrammer.Butit'sstillalittledisturbingthattheactofprogrammingseemsliterally
unteachabletoasizablesubsetofincomingcomputersciencestudents.Evidentlynoteveryoneisasfascinatedbymeaninglessrulesandmeaningless
conclusionsasweareIcan'timaginewhynot.

*whichIhopetomastersometimebetweennowandmydeath