Multiprocessing FromWikipedia,thefreeencyclopedia (Redirectedfrom{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multi processing&redirect=no"\o"Multiprocessing"}) Jumpto:{HYPERLINK"http://en.wikipedia.org/wiki/Multi processing"\l"columnone"},{HYPERLINK "http://en.wikipedia.org/wiki/Multiprocessing"\l"searchInput"} Multiprocessingistheuseoftwoormore{HYPERLINK "http://en.wikipedia.org/wiki/CPU"\o"CPU"}(CPUs)withina singlecomputersystem.Thetermalsoreferstotheabilityofa systemtosupportmorethanoneprocessorand/ortheabilityto allocatetasksbetweenthem.{HYPERLINK "http://en.wikipedia.

org/wiki/Multiprocessing"\l"cite_note-"} Therearemanyvariationsonthisbasictheme,andthedefinitionof multiprocessingcanvarywithcontext,mostlyasafunctionofhow CPUsaredefined({HYPERLINK "http://en.wikipedia.org/wiki/Multicore_(computing)"\o"Multi core(computing)"}ononedie,multiplechipsinonepackage, multiplepackagesinonesystemunit,etc.).

Multiprocessingsometimesreferstotheexecutionofmultiple concurrentsoftwareprocessesinasystemasopposedtoasingle processatanyoneinstant.However,theterms{HYPERLINK "http://en.wikipedia.org/wiki/Computer_multitasking"\o "Computermultitasking"}or{HYPERLINK "http://en.wikipedia.org/wiki/Computer_multitasking"\o "Computermultitasking"}aremoreappropriatetodescribethis concept,whichisimplementedmostlyinsoftware,whereas multiprocessingismoreappropriatetodescribetheuseofmultiple hardwareCPUs.Asystemcanbebothmultiprocessingand multiprogramming,onlyoneofthetwo,orneitherofthetwo. Contents [{HYPERLINK"javascript:toggleToc()"}]

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"Types"}

o

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"Processor_symmetry"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi

processing"\l "Instruction_and_data_streams"}
o

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"Processor_coupling"}

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l "Software_implementation_issues"}

o

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"SISD_multiprocessing"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"SIMD_multiprocessing"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"MISD_multiprocessing"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l "MIMD_multiprocessing"}

{HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"See_also"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"External_links"} {HYPERLINK "http://en.wikipedia.org/wiki/Multi processing"\l"References"}

[{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:Types"}]Types [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:Processorsymmetry"}] Processorsymmetry Inamultiprocessingsystem,allCPUsmaybeequal,orsomemay bereservedforspecialpurposes.Acombinationofhardwareand operatingsystemsoftwaredesignconsiderationsdeterminethe symmetry(orlackthereof)inagivensystem.Forexample, hardwareorsoftwareconsiderationsmayrequirethatonlyoneCPU

respondtoallhardwareinterrupts,whereasallotherworkinthe systemmaybedistributedequallyamongCPUs;orexecutionof kernelmodecodemayberestrictedtoonlyoneprocessor(eithera specificprocessor,oronlyoneprocessoratatime),whereasuser modecodemaybeexecutedinanycombinationofprocessors. Multiprocessingsystemsareofteneasiertodesignifsuch restrictionsareimposed,buttheytendtobelessefficientthan systemsinwhichallCPUsareutilized. SystemsthattreatallCPUsequallyarecalled{HYPERLINK "http://en.wikipedia.org/wiki/Symmetric_multiprocessing"\o "Symmetricmultiprocessing"}(SMP)systems.Insystemswhereall CPUsarenotequal,systemresourcesmaybedividedinanumberof ways,including{HYPERLINK "http://en.wikipedia.org/wiki/Asymmetric_multiprocessing"\o "Asymmetricmultiprocessing"}(ASMP),{HYPERLINK "http://en.wikipedia.org/wiki/Nonuniform_memory_access"\o "Nonuniformmemoryaccess"}(NUMA)multiprocessing,and {HYPERLINK"http://en.wikipedia.org/wiki/Computer_cluster"\o "Computercluster"}multiprocessing(qq.v.). [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio

n=edit&section="\o"Editsection:Instructionanddatastreams"}] Instructionanddatastreams Inmultiprocessing,theprocessorscanbeusedtoexecuteasingle sequenceofinstructionsinmultiplecontexts({HYPERLINK "http://en.wikipedia.org/wiki/SIMD"\o"SIMD"}orSIMD,often usedin{HYPERLINK "http://en.wikipedia.org/wiki/Vector_processing"\o"Vector processing"}),multiplesequencesofinstructionsinasinglecontext ({HYPERLINK"http://en.wikipedia.org/wiki/MISD"\o"MISD"}or MISD,usedfor{HYPERLINK "http://en.wikipedia.org/wiki/Redundancy_(engineering)"\o "Redundancy(engineering)"}infailsafesystemsandsometimes appliedtodescribe{HYPERLINK "http://en.wikipedia.org/wiki/Pipelining"\o"Pipelining"}or {HYPERLINK"http://en.wikipedia.org/wiki/Hyperthreading"\o "Hyperthreading"}),ormultiplesequencesofinstructionsin multiplecontexts({HYPERLINK "http://en.wikipedia.org/wiki/MIMD"\o"MIMD"}or{HYPERLINK "http://en.wikipedia.org/wiki/MIMD"\o"MIMD"}). [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio

n=edit&section="\o"Editsection:Processorcoupling"}] Processorcoupling TightlycoupledmultiprocessorsystemscontainmultipleCPUsthat areconnectedatthebuslevel.TheseCPUsmayhaveaccesstoa centralsharedmemory(SMPor{HYPERLINK "http://en.wikipedia.org/wiki/UMA"\o"UMA"}),ormay participateinamemoryhierarchywithbothlocalandshared memory({HYPERLINK"http://en.wikipedia.org/wiki/NUMA"\o "NUMA"}).The{HYPERLINK "http://en.wikipedia.org/wiki/IBM_p"\o"IBMp"}Regattais anexampleofahighendSMPsystem.{HYPERLINK "http://en.wikipedia.org/wiki/Intel"\o"Intel"}{HYPERLINK "http://en.wikipedia.org/wiki/Xeon"\o"Xeon"}processors dominatedthemultiprocessormarketforbusinessPCsandwerethe onlyx86optionuntilthereleaseof{HYPERLINK "http://en.wikipedia.org/wiki/AMD"\o"AMD"}'s{HYPERLINK "http://en.wikipedia.org/wiki/Opteron"\o"Opteron"}rangeof processorsin2004.Bothrangesofprocessorshadtheirownonboard cachebutprovidedaccesstosharedmemory;theXeonprocessors viaacommonpipeandtheOpteronprocessorsviaindependent pathwaystothesystem{HYPERLINK "http://en.wikipedia.org/wiki/RAM"\o"RAM"}.

Chipmultiprocessors,alsoknownas{HYPERLINK "http://en.wikipedia.org/wiki/Multicore_(computing)"\o"Multi core(computing)"}computing,involvesmorethanoneprocessor placedonasinglechipandcanbethoughtofthemostextremeform oftightlycoupledmultiprocessing.Mainframesystemswith multipleprocessorsareoftentightlycoupled. Looselycoupledmultiprocessorsystems(oftenreferredtoas {HYPERLINK"http://en.wikipedia.org/wiki/Computer_cluster"\o "Computercluster"})arebasedonmultiplestandalonesingleor dualprocessor{HYPERLINK "http://en.wikipedia.org/wiki/Commodity_computer"\o "Commoditycomputer"}interconnectedviaahighspeed communicationsystem({HYPERLINK "http://en.wikipedia.org/wiki/Gigabit_Ethernet"\o"Gigabit Ethernet"}iscommon).ALinux{HYPERLINK "http://en.wikipedia.org/wiki/Beowulf_cluster"\o"Beowulf cluster"}isanexampleofa{HYPERLINK "http://en.wikipedia.org/wiki/Loose_coupling"\o"Loose coupling"}system. Tightlycoupledsystemsperformbetterandarephysicallysmaller thanlooselycoupledsystems,buthavehistoricallyrequiredgreater initialinvestmentsandmay{HYPERLINK

"http://en.wikipedia.org/wiki/Depreciation"\o"Depreciation"} rapidly;nodesinalooselycoupledsystemareusuallyinexpensive commoditycomputersandcanberecycledasindependent machinesuponretirementfromthecluster. Powerconsumptionisalsoaconsideration.Tightlycoupledsystems tendtobemuchmoreenergyefficientthanclusters.Thisisbecause considerableeconomiescanberealizedbydesigningcomponentsto worktogetherfromthebeginningintightlycoupledsystems, whereaslooselycoupledsystemsusecomponentsthatwerenot necessarilyintendedspecificallyforuseinsuchsystems. [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:Softwareimplementation issues"}]Softwareimplementationissues {HYPERLINK "http://en.wikipedia.org/wiki/Fly nn%s_taxonomy"\o"Flynn's taxonomy"} Single

Multiple

Instruction Instruction

{HYPERLIN {HYPERLIN K Single Data K "http://en. "http://en. wikipedia.o wikipedia.o rg/wiki/SIS rg/wiki/MI D"\o "SISD"}

SD"\o "MISD"}

{HYPERLIN {HYPERLIN K Multiple Data K "http://en. "http://en. wikipedia.o wikipedia.o rg/wiki/SI rg/wiki/MI MD"\o "SIMD"}

MD"\o "MIMD"}

Thisbox:{HYPERLINK "http://en.wikipedia.org/wiki/Te mplate:Flynn%s_Taxonomy"\o "Template:Flynn's Taxonomy"}{HYPERLINK "http://en.wikipedia.org/wiki/Te mplate_talk:Flynn%s_Taxonom

y"\o"Templatetalk:Flynn's Taxonomy"}{HYPERLINK "http://en.wikipedia.org/w/index .php?title=Template:Flynn%s_T axonomy&action=edit"} Seealso:{HYPERLINK "http://en.wikipedia.org/wiki/Software_lockout"\o"Software lockout"} [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:SISDmultiprocessing"}]SISD multiprocessing Mainarticle:{HYPERLINK"http://en.wikipedia.org/wiki/SISD"\o "SISD"} Ina{HYPERLINK"http://en.wikipedia.org/wiki/SSID"\o"SSID"} computeroneprocessorsequentiallyprocessesinstructions,each instructionprocessesonedataitem. [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio

n=edit&section="\o"Editsection:SIMDmultiprocessing"}]SIMD multiprocessing Mainarticle:{HYPERLINK"http://en.wikipedia.org/wiki/SIMD"\o "SIMD"} Ina{HYPERLINK"http://en.wikipedia.org/wiki/SIMD"\o"SIMD"} computeroneprocessorhandlesastreamofinstructions,eachone ofwhichcanperformcalculationsinparallelonmultipledata locations. SIMDmultiprocessingiswellsuitedto{HYPERLINK "http://en.wikipedia.org/wiki/Parallel_processing"\o"Parallel processing"},inwhichaverylargesetofdatacanbedividedinto partsthatareindividuallysubjectedtoidenticalbutindependent operations.Asingleinstructionstreamdirectstheoperationof multipleprocessingunitstoperformthesamemanipulations simultaneouslyonpotentiallylargeamountsofdata. Forcertaintypesofcomputingapplications,thistypeofarchitecture canproduceenormousincreasesinperformance,intermsofthe elapsedtimerequiredtocompleteagiventask.However,a drawbacktothisarchitectureisthatalargepartofthesystemfalls idlewhenprogramsorsystemtasksareexecutedthatcannotbe dividedintounitsthatcanbeprocessedinparallel.

Additionally,programsmustbecarefullyandspeciallywrittento takemaximumadvantageofthearchitecture,andoftenspecial optimizingcompilersdesignedtoproducecodespecificallyforthis environmentmustbeused.Somecompilersinthiscategoryprovide specialconstructsorextensionstoallowprogrammerstodirectly specifyoperationstobeperformedinparallel(e.g.,DOFORALL statementsintheversionof{HYPERLINK "http://en.wikipedia.org/wiki/FORTRAN"\o"FORTRAN"}usedon the{HYPERLINK"http://en.wikipedia.org/wiki/ILLIAC_IV"\o "ILLIACIV"},whichwasaSIMDmultiprocessing{HYPERLINK "http://en.wikipedia.org/wiki/Supercomputer"\o "Supercomputer"}). SIMDmultiprocessingfindswideuseincertaindomainssuchas {HYPERLINK"http://en.wikipedia.org/wiki/Computer_simulation" \o"Computersimulation"},butisoflittleuseingeneralpurpose desktopandbusinesscomputingenvironments.[{HYPERLINK "http://en.wikipedia.org/wiki/Wikipedia:Citation_needed"\o "Wikipedia:Citationneeded"}] [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:MISDmultiprocessing"}]MISD multiprocessing

Mainarticle:{HYPERLINK"http://en.wikipedia.org/wiki/MISD"\o "MISD"} MISDmultiprocessingoffersmainlytheadvantageofredundancy, sincemultipleprocessingunitsperformthesametasksonthesame data,reducingthechancesofincorrectresultsifoneoftheunits fails.MISDarchitecturesmayinvolvecomparisonsbetween processingunitstodetectfailures.Apartfromtheredundantand failsafecharacterofthistypeofmultiprocessing,ithasfew advantages,anditisveryexpensive.Itdoesnotimprove performance.Itcanbeimplementedinawaythatistransparentto software.Itisusedin{HYPERLINK "http://en.wikipedia.org/wiki/Array_processor"\o"Array processor"}andisimplementedinfaulttolerantmachines. [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:MIMDmultiprocessing"}]MIMD multiprocessing Mainarticle:{HYPERLINK"http://en.wikipedia.org/wiki/MIMD"\o "MIMD"} MIMDmultiprocessingarchitectureissuitableforawidevarietyof tasksinwhichcompletelyindependentandparallelexecutionof

instructionstouchingdifferentsetsofdatacanbeputtoproductive use.Forthisreason,andbecauseitiseasytoimplement,MIMD predominatesinmultiprocessing. Processingisdividedintomultiple{HYPERLINK "http://en.wikipedia.org/wiki/Thread_(computer_programming)" \o"Thread(computerprogramming)"},eachwithitsownhardware processorstate,withinasinglesoftwaredefinedprocessorwithin multipleprocesses.Insofarasasystemhasmultiplethreads awaitingdispatch(eithersystemoruserthreads),thisarchitecture makesgooduseofhardwareresources. MIMDdoesraiseissuesofdeadlockandresourcecontention, however,sincethreadsmaycollideintheiraccesstoresourcesinan unpredictablewaythatisdifficulttomanageefficiently.MIMD requiresspecialcodingintheoperatingsystemofacomputerbut doesnotrequireapplicationchangesunlesstheprograms themselvesusemultiplethreads(MIMDistransparenttosingle threadedprogramsundermostoperatingsystems,iftheprograms donotvoluntarilyrelinquishcontroltotheOS).Bothsystemand usersoftwaremayneedtousesoftwareconstructssuchas {HYPERLINK "http://en.wikipedia.org/wiki/Semaphore_(programming)"\o "Semaphore(programming)"}(alsocalledlocksorgates)toprevent

onethreadfrominterferingwithanotheriftheyshouldhappento crosspathsinreferencingthesamedata.Thisgatingorlocking processincreasescodecomplexity,lowersperformance,andgreatly increasestheamountoftestingrequired,althoughnotusually enoughtonegatetheadvantagesofmultiprocessing. Similarconflictscanariseatthehardwarelevelbetweenprocessors (cachecontentionandcorruption,forexample),andmustusuallybe resolvedinhardware,orwithacombinationofsoftwareand hardware(e.g.,cacheclearinstructions). [{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:Seealso"}]Seealso

{HYPERLINK"http://en.wikipedia.org/wiki/BC"\o "BC"} {HYPERLINK "http://en.wikipedia.org/wiki/Symmetric_multiprocessing"\o "Symmetricmultiprocessing"} {HYPERLINK "http://en.wikipedia.org/wiki/Asymmetric_multiprocessing" \o"Asymmetricmultiprocessing"}

{HYPERLINK"http://en.wikipedia.org/wiki/Multi core_(computing)"\o"Multicore(computing)"} {HYPERLINK"http://en.wikipedia.org/wiki/BMDFM"\o "BMDFM"}BinaryModularDataflowMachine,aSMPMIMD RuntimeEnvironment {HYPERLINK"http://en.wikipedia.org/wiki/Software_lockout" \o"Softwarelockout"}

[{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:Externallinks"}]External links

{HYPERLINK"http://multicore.ning.com/"}

[{HYPERLINK "http://en.wikipedia.org/w/index.php?title=Multiprocessing&actio n=edit&section="\o"Editsection:References"}]References