Вы находитесь на странице: 1из 4

WhatisaplainEnglishexplanationofBigOnotation?

1 2 next

Quicknote,thisisalmostcertainlyconfusingBigOnotation(whichisanupperbound)with
Thetanotation(whichisatwosidebound).Inmyexperiencethisisactuallytypicalof
discussionsinnonacademicsettings.Apologiesforanyconfusioncaused.

BigOcomplexitycanbevisualizedwiththisgraph:

ThesimplestdefinitionIcangiveforBigOnotationisthis:

BigOnotationisarelativerepresentationofthecomplexityofanalgorithm.

Therearesomeimportantanddeliberatelychosenwordsinthatsentence:

relative:youcanonlycompareapplestoapples.Youcan'tcompareanalgorithmtodo
arithmeticmultiplicationtoanalgorithmthatsortsalistofintegers.Butacomparisonof
twoalgorithmstodoarithmeticoperations(onemultiplication,oneaddition)willtellyou
somethingmeaningful
representation:BigO(initssimplestform)reducesthecomparisonbetween
algorithmstoasinglevariable.Thatvariableischosenbasedonobservationsor
assumptions.Forexample,sortingalgorithmsaretypicallycomparedbasedon
comparisonoperations(comparingtwonodestodeterminetheirrelativeordering).This
assumesthatcomparisonisexpensive.Butwhatifcomparisonischeapbutswapping
isexpensive?Itchangesthecomparisonand
complexity:ifittakesmeonesecondtosort10,000elementshowlongwillittakeme
tosortonemillion?Complexityinthisinstanceisarelativemeasuretosomethingelse.

Comebackandrereadtheabovewhenyou'vereadtherest.

ThebestexampleofBigOIcanthinkofisdoingarithmetic.Taketwonumbers(123456and
789012).Thebasicarithmeticoperationswelearntinschoolwere:

addition
subtraction
multiplicationand
division.

Eachoftheseisanoperationoraproblem.Amethodofsolvingtheseiscalledanalgorithm.

Additionisthesimplest.Youlinethenumbersup(totheright)andaddthedigitsinacolumn
writingthelastnumberofthatadditionintheresult.The'tens'partofthatnumberiscarried
overtothenextcolumn.

Let'sassumethattheadditionofthesenumbersisthemostexpensiveoperationinthis
algorithm.Itstandstoreasonthattoaddthesetwonumberstogetherwehavetoaddtogether
6digits(andpossiblycarrya7th).Ifweaddtwo100digitnumberstogetherwehavetodo100
additions.Ifweaddtwo10,000digitnumberswehavetodo10,000additions.

Seethepattern?Thecomplexity(beingthenumberofoperations)isdirectlyproportionalto
thenumberofdigitsninthelargernumber.WecallthisO(n)orlinearcomplexity.

Subtractionissimilar(exceptyoumayneedtoborrowinsteadofcarry).

Multiplicationisdifferent.Youlinethenumbersup,takethefirstdigitinthebottomnumberand
multiplyitinturnagainsteachdigitinthetopnumberandsoonthrougheachdigit.Soto
multiplyourtwo6digitnumberswemustdo36multiplications.Wemayneedtodoasmanyas
10or11columnaddstogettheendresulttoo.

Ifwehavetwo100digitnumbersweneedtodo10,000multiplicationsand200adds.Fortwo
onemilliondigitnumbersweneedtodoonetrillion(1012)multiplicationsandtwomillionadds.

Asthealgorithmscaleswithnsquared,thisisO(n2)orquadraticcomplexity.Thisisagood
timetointroduceanotherimportantconcept:

Weonlycareaboutthemostsignificantportionofcomplexity.

Theastutemayhaverealizedthatwecouldexpressthenumberofoperationsas:n2+2n.But
asyousawfromourexamplewithtwonumbersofamilliondigitsapiece,thesecondterm(2n)
becomesinsignificant(accountingfor0.0002%ofthetotaloperationsbythatstage).

Onecannoticethatwe'veassumedtheworstcasescenariohere.Whilemultiplying6digit
numbersifoneofthemis4digitandtheotheroneis6digit,thenweonlyhave24
multiplications.Stillwecalculatetheworstcasescenarioforthat'n',i.ewhenbothare6digit
numbers.HenceBigOnotationisabouttheWorstcasescenarioofanalgorithm

TheTelephoneBook

ThenextbestexampleIcanthinkofisthetelephonebook,normallycalledtheWhitePagesor
similarbutit'llvaryfromcountrytocountry.ButI'mtalkingabouttheonethatlistspeopleby
surnameandtheninitialsorfirstname,possiblyaddressandthentelephonenumbers.

Nowifyouwereinstructingacomputertolookupthephonenumberfor"JohnSmith"ina
telephonebookthatcontains1,000,000names,whatwouldyoudo?Ignoringthefactthatyou
couldguesshowfarintheS'sstarted(let'sassumeyoucan't),whatwouldyoudo?

Atypicalimplementationmightbetoopenuptothemiddle,takethe500,000thandcompareit
to"Smith".Ifithappenstobe"Smith,John",wejustgotreallucky.Farmorelikelyisthat"John
Smith"willbebeforeorafterthatname.Ifit'safterwethendividethelasthalfofthephone
bookinhalfandrepeat.Ifit'sbeforethenwedividethefirsthalfofthephonebookinhalfand
repeat.Andsoon.

Thisiscalledabinarysearchandisusedeverydayinprogrammingwhetheryourealizeitor
not.

Soifyouwanttofindanameinaphonebookofamillionnamesyoucanactuallyfindany
namebydoingthisatmost20times.Incomparingsearchalgorithmswedecidethatthis
comparisonisour'n'.

Foraphonebookof3namesittakes2comparisons(atmost).
For7ittakesatmost3.
For15ittakes4.

For1,000,000ittakes20.

Thatisstaggeringlygoodisn'tit?

InBigOtermsthisisO(logn)orlogarithmiccomplexity.Nowthelogarithminquestion
couldbeln(basee),log10,log2orsomeotherbase.Itdoesn'tmatterit'sstillO(logn)justlike
O(2n2)andO(100n2)arestillbothO(n2).

It'sworthwhileatthispointtoexplainthatBigOcanbeusedtodeterminethreecaseswithan
algorithm:

BestCase:Inthetelephonebooksearch,thebestcaseisthatwefindthenamein
onecomparison.ThisisO(1)orconstantcomplexity
ExpectedCase:AsdiscussedabovethisisO(logn)and
WorstCase:ThisisalsoO(logn).

Normallywedon'tcareaboutthebestcase.We'reinterestedintheexpectedandworstcase.
Sometimesoneortheotherofthesewillbemoreimportant.
Backtothetelephonebook.

Whatifyouhaveaphonenumberandwanttofindaname?Thepolicehaveareversephone
bookbutsuchlookupsaredeniedtothegeneralpublic.Orarethey?Technicallyyoucan
reverselookupanumberinanordinaryphonebook.How?

Youstartatthefirstnameandcomparethenumber.Ifit'samatch,great,ifnot,youmoveon
tothenext.Youhavetodoitthiswaybecausethephonebookisunordered(byphone
numberanyway).

Sotofindanamegiventhephonenumber(reverselookup):

BestCase:O(1)
ExpectedCase:O(n)(for500,000)and
WorstCase:O(n)(for1,000,000).

TheTravellingSalesman
Thisisquiteafamousproblemincomputerscienceanddeservesamention.Inthisproblem
youhaveNtowns.Eachofthosetownsislinkedto1ormoreothertownsbyaroadofa
certaindistance.TheTravellingSalesmanproblemistofindtheshortesttourthatvisitsevery
town.

Soundssimple?Thinkagain.

Ifyouhave3townsA,BandCwithroadsbetweenallpairsthenyoucouldgo:

ABC
ACB
BCA
BAC
CAB
CBA

Wellactuallythere'slessthanthatbecausesomeoftheseareequivalent(ABCandC
BAareequivalent,forexample,becausetheyusethesameroads,justinreverse).

Inactualitythereare3possibilities.

Takethisto4townsandyouhave(iirc)12possibilities.
With5it's60.
6becomes360.

Thisisafunctionofamathematicaloperationcalledafactorial.Basically:

5!=54321=120
6!=654321=720
7!=7654321=5040

25!=252421=15,511,210,043,330,985,984,000,000

50!=504921=3.041409321064

SotheBigOoftheTravellingSalesmanproblemisO(n!)orfactorialorcombinatorial
complexity.

Bythetimeyougetto200townsthereisn'tenoughtimeleftintheuniversetosolvethe
problemwithtraditionalcomputers.

Somethingtothinkabout.

PolynomialTime

AnotherpointIwantedtomakequickmentionofisthatanyalgorithmthathasacomplexityof
O(na)issaidtohavepolynomialcomplexityorissolvableinpolynomialtime.

O(n),O(n2)etcareallpolynomialtime.Someproblemscannotbesolvedinpolynomialtime.
Certainthingsareusedintheworldbecauseofthis.PublicKeyCryptographyisaprime
example.Itiscomputationallyhardtofindtwoprimefactorsofaverylargenumber.Ifitwasn't,
wecouldn'tusethepublickeysystemsweuse.
Anyway,that'sitformy(hopefullyplainEnglish)explanationofBigO(revised).

editedFeb28'15at16:55 answeredJan28'09at11:18
user289086 cletus
430k 122 788 883

468 WhiletheotheranswersfocusonexplainingthedifferencesbetweenO(1),O(n^2)etal....yoursisthe
onewhichdetailshowalgorithmscangetclassifiedinton^2,nlog(n)etc.+1foragoodanswerthat
helpedmeunderstandBigOnotationaswellYewLongJan28'09at11:42

14 onemightwanttoaddthatbigOrepresentsanupperbound(givenbyanalgorithm),bigOmegagivea
lowerbound(usuallygivenasaproofindependentfromaspecificalgorithm)andbigThetameansthat
an"optimal"algorithmreachingthatlowerboundisknown.mdmFeb2'09at19:16

17 Thisisgoodifyou'relookingforthelongestanswer,butnotfortheanswerthatbestexplainsBigOina
simplemanner.kirk.burlesonJul16'10at18:21

147 1:Thisisblatantlywrong:_"BigOhisrelativerepresentationofcomplexityofalgorithm".No.BigOhisan
asymptoticupperboundandexistsquitewellindependentofcomputerscience.O(n)islinear.No,you
areconfusingBigOhwiththeta.lognisO(n).1isO(n).Thenumberofupvotestothisanswer(andthe
comments),whichmakesthebasicmistakeofconfusingThetawithBigOhisquiteembarassing...
AryabhattaMay24'11at4:44

57 "Bythetimeyougetto200townsthereisn'tenoughtimeleftintheuniversetosolvetheproblemwith
traditionalcomputers."Whentheuniverseisgoingtoend?IsaacJun18'12at10:43