CPSC 490

Backtracking: Introduction

Brute Force and Backtrac ki n g

Mostoftheproblemsincomputersciencearenotsolvableinpolynomialtime(atleastasfaraswe know). This is true both in the theoretical sense and in practice. When faced with one of these problems,therearefewoptions:giveup,trybruteforceorsettleforanapproximatesolution.Inthis section,wewillfocusonthesecondoption.Thereareasurprisingnumberofdifficultproblemsthat havebruteforcealgorithmsthatareabletosolvefairlylargeprobleminstances.

Estimat i n g the Runni n g Time

Firstofall,weneedawayofestimatingtherunningtimeofagivenalgorithm.Themostobvioustool isthebigOnotation.Butwhatdoesitmeanifanalgorithmtakes O n 2 time?Howbigcannbe beforethealgorithmbecomesimpractical?Obviously,thatdependsonyourCPUspeedandtype,your RAM,theoperating systemandlotsofothercomplicateddetails.Thatwasthemainpointbehindthe bigOnotationtoavoidthinkingaboutthesedetails!ThisiswhythebigOnotationhasahidden constantthatismeanttoabsorbthedelayscausedbyhardwareandsoftwaredifferences. Actually,thesedifferencesarenotthatgreat,especiallyifwearetalkingaboutbruteforcealgorithms thatareexponentialintime. 2log n secondsisverydifferentfrom 2000log n secondsfora lotofinterestingvaluesofn.However, 22 n secondsisaboutthesameas 10002n secondswhen n=100 inthesensethatbotharelongerthanourlifetime.Soifwehavea O 2n algorithmand wanttorunitonaproblemofsize100,itdoesn'tmatterwhatkindofcomputerweareusingitwill notwork. Wecangofurtherandnotethatprocessorspeedshavereachedaplateau.Moore'slawthatpromises exponentialgrowthofCPUspeedsnolongerholds. Intelhasannouncedthattheyhaveabandoned plans of creating a 4GHz CPU for a while and will instead focus on finding ways to put several processorsononechip.ApplehasputoffthereleaseoftheG5Powerbookbecausetheyarehaving troubleswithCPUcoolingandpowerconsumption.Processorsarenotlikelytobecomeanyfasterin thenearfuture. Withthis inmind, letmemake afew"bold"statements.Ifyouhaveaaprogram that requires 1 millionoperations,itwillruninlessthanasecond.Somethingwith100millionoperationswillrunin afewseconds.With1billionoperations,preparetowaitforseveralminutes.By"operations",Imean evaluatingasimpleexpression,readingorstoringavariable,etc.Thinkofonesimplelineinyour programasoneoperation.Thisisbasicallytrueforallmoderncomputersifyouwantyourprogram toruninafewseconds,youbettermakesureitisnotexecutingmorethanafewhundredmillionlines of code (if you imagine unrolling all of the loops and functions calls so that there are no jumps anywhereinthecode). Thissimpleruleisveryvague,andIamsurethattherearelotsofexampleswhenitdoesnothold,but it'sclosetothetruthanditisveryuseful.Forinstance,wecanuseittoanswertheoriginalquestion, "Howbigcannbeifweplantorunasimple O n 2 algorithmonit?"Theansweris,"About10000 ifyouwantittoterminateinafewseconds."Thisisacrudeestimate,butitgivesaconcretenumber toworkwith.Thetruenumbermightbesomewherebetween5000and20000youcantryitonyour computerbywritingasimpleloop.Whatifanalgorithmrequires 2n operations?Inthiscase,the answeris,"27,giveortake2." Thepurposeoftheruleistogiveacrude,butconcreteestimateoftherunningtimeinplaceswhere

CPSC 490

Backtracking: Introduction

thebigOnotationdoesnotgiveenoughinformation.Armedwiththislittletool,let'slookatafew techniquesforapplyingbruteforcetooneclassicproblem.

Backtrac ki n g with DFS

Byfarthemostusefultechniqueinproblemsolvingis"divideandconquer".Backtracking isoneofits simplestapplications.Wewanttofindasolutiontoaprobleminstance.Thesolutioniscomplicated andconsistsofmanysmallparts.Sowewilltryallpossibilitiesforpickingthefirstpart.Foreachof these, we will try all possibilities of picking the second part. And so on until we finally build a completesolution.Orperhapswearelookingforthebestsolution.Thenwewillgenerateallsolutions partbypartandrememberthebestsolutionwefind. Hereisaveryfamousproblem.Howmanywaysaretheretoplace8queensonachessboard sothat notwoqueensattackeachother?Achessboardisan8by8grid.Queenscanbeplacedinthecellsof thegrid.Twoqueensareundermutualattackiftheysharearow,acolumnoradiagonal.Hereisone possiblesolution: X X X X X X X X Asolutionconsistsof8partstheindividualpositionsofthe8queens.Thisisourbasisfordivide andconquer.Selectalocationforqueen#1(64possibilities).Foreachofthose,selectalocationfor queen#2(63possibilities),etc.Whenwehaveall8queensplaced,checkthatnotwosharearow,a columnoradiagonal.Iftheydonot,thenwehaveavalidconfigurationcountit.Attheend,return thenumberofvalidconfigurationsfound. Thisworks,butit's ridiculouslyslow. There are64*63*...*57 configurations tocheck,andittakes about8*8operationstocheckeachone,foratotalofover 1016 operations.Thereisnohopethat thisprogramwillfinishrunninginanykindofreasonabletime.Butatleastitworks;let'sseehowwe canspeeditup. Weknowthatnotwoqueenscanshareacolumn,andthereare8ofthem,soeachcolumnhasexactly onequeenonit.Thisgivesusadifferentwaytodivideupasolution.Asolutionis8numbersinthe rangefrom0to7givingtherowsonwhichthe8queensreside,readingfromlefttoright. Wecan representthesolutionabovebythe8numbers(0,4,7,5,2,6,1,3). Let'srunthesamealgorithm,butthistime,wewillhaveonly8possibilitiesforqueen1,7possibilities for queen 2, etc. After we have placed all 8 queens, we still have to check that no two share a diagonal.Thatstillrequiresabout8*8operations,butnowwehaveonly8*7*...*1configurationsto check.Hereiswhatanimplementationmightlooklike.

CPSC 490 Example1:

Backtracking: Introduction

introw[8],used[8]; intqueens(inti){ if(i==8){ //Placedall8queens.Verifydiagonals. for(intj=0;j<8;j++) for(intk=j+1;k<8;k++){ if(row[j]+j==row[k]+k)return0; if(row[j]j==row[k]k)return0; } return1; } inttotal=0; for(row[i]=0;row[i]<8;row[i]++) { //ensurewearenotsharingarow if(used[row[i]])continue; //placethequeenandrecurseontherest used[row[i]]=true; total+=queens(i+1); //marktherowasunusedagain used[row[i]]=false; } returntotal; }

Tousethefunctionqueens(),firstclearused[]tobeallfalseandthencallqueens(0).row[i]isthe rowinwhichqueenincolumniresides.used[r]istrueifsomequeenwehavealreadyplacedusesrow r.Onceireaches8,weverifythatnotwoqueensshareadiagonalandreturn1.queens(0)finishesin about30millisecondsandreturns92.That'squiteanimprovement. Youmayhavenoticedthatweareessentiallygeneratingallthe8!permutationswiththisfunction. Theexactnumberofoperationsrequiredisabout20million(8!*8*36)thereare8!recursivecalls, eachonerequiring8iterationsoftheloop,and36operationstocheckforavalidconfiguration.This canalsobedonewithoutanyrecursion,simplyusingnext_permutation(). NotehowsimilarthisimplementationistoDFS.Ifweconsidereachpartialconfiguration(whereonly thefirstiqueenshavebeenassignedrows)avertex,thenweadoingadepthfirstsearchonacertain configurationgraph.ItisusefultothinkofbacktrackingasgraphsearchorDFSbecausewewillbe abletoapplysomeoftheideasfromgraphtheoryhere.Forinstance,thinkaboutwhatitwouldmean touseBFSinthiscaseinsteadofDFS.Whichwayismoreefficient?Wewillcomebacktothisidea later. Itmayseemlikeawastetoexplore8!(40320)configurationsjusttofindthe92thatwearelooking for.Ifweincreasethesizeoftheboardto10x10,theprogramwillrequireabout5seconds.Fora 13x13board,Iwaitedforhalfandhourforittofinishandgaveup.Inthenextsectionwewillseeone veryusefultechniquethatwillallowustoimprovethealgorithmsothatitworksonlargerboard sizes.Formoreinfoonthisclassicproblem,see http://www.research.att.com/cgibin/access.cgi/as/njas/sequences/eisA.cgi?Anum=A000170