Multiprocessors

COMP 211 Computer Systems Organization and Architecture

ACL 20130627

Introduction

As their name suggest, multiprocessor systems

utilizes more than one processor to improve
performance.

Early multiprocessor systems used multiple

processor to improve throughput by executing
independent jobs on different processors.

Introduction
Dividing

a single programs work among multiple

processors, multiprocessors can achieve greater
performance than is possible with a single
processor in any given fabrication technology.

The

terms multitasking or multiprogramming are

more appropriate to describe this concept, which is
implemented
mostly
in
software,
whereas
multiprocessing is more appropriate to describe the
use of multiple hardware CPUs. A system can be
both multiprocessing and multiprogramming, only
one of the two, or neither of the two of them.

VS

Multi core
a single computing component with two or more
independent actual central processing units (called
"cores"), which are the units that read and execute
program instructions.

Multiprocessor
It is a tightly coupled computer system
having two or more processing units (Multiple
Processors) each sharing main memory and
peripherals, in order to simultaneously process
programs.

Microprocessors Speedup

Designer of uniprocessor systems,

performance in terms of speedup.

measure

measure

Similarly, multiprocessor architects

performance in terms of speedup.

Microprocessors Speedup

Ideal Microprocessors Speedup

Linear Speedup
The execution time of program on an nprocessor system would be 1/nth of the execution
time on a one-processor system
Super Linear Speedup
It occurs when programs are sometimes more
efficient on multiprocessor systems than on
uniprocessor systems.

Limitations of Speedup
Interprocessor communication
Synchronization
Load Balancing

Limitations of Speedup
Interprocessor communication

Whenever one processor generates (computes) a

value that is needed by the fraction of the program
running on another processor, that value must be
communicated to the processors that need it, which
takes time

On a uniprocessor system, the entire program

runs on one processor, so there is no time
lost to interprocessor communication

Limitations of Speedup
Synchronization

It is often necessary to synchronize the processors

to ensure that they have all completed some phase
of the program before any processor begins working
on the next phase of the program.

Limitations of Speedup
Load Balancing

In many parallel applications, it is difficult to divide

the program across the processors such that each
processors chunk of the work take the same amount
of time.

Some of the processors completes their task early

and are idle waiting for the others to finish.
This results in increasing the overall
execution time.

Multiprocessor Systems
Multiprocessors consist of processors connected
by a communications network.
CPU

CPU

CPU

CPU

Network
CPU

CPU

CPU

CPU

ASSIGN:
A.What is multithreading?
Advantage and disadvantage

B. Given Types Of Multiprocessors: (two

columns)

1.PROVIDE ILLUSTRATION
2.GIVE ATLEAST TWO EXAMPLES EACH

Types of Multiprocessors
Centralized Memory System
Distributed Memory System

Centralized Memory System

There is only one memory system for the entire
multiprocessor, and memory references from all of
the processors go to that memory system.
CPU

CPU
CPU

CPU
CPU

Network

Memory

CPU
CPU

Centralized Memory System

Advantages:
All data in the memory is accessible by all
processors
There will never be a problem with multiple
copies of a given datum.
It is a better design when the number of
processors in the system is small.
A single memory may be able to meet the
bandwidth demands of the processors.
Disadvantage:
The bandwidth does not increase as the
number of processors increase.

Distributed Memory System

Each processor has its own memory system,
which it can access directly. In obtaining the data
that is stored in some other processors memory, a
processor must communicate with it to request the
data.
Memory

Memory

Memory

Memory

CPU

CPU

CPU

CPU

Network
CPU

CPU

CPU

CPU

Memory

Memory

Memory

Memory

Distributed Memory System

Advantages:
Each processor has its own memory system.
There is more total bandwidth in the memory
system than in a centralized memory system, and
that the latency to complete a memory request is
lower.
Used when the latency of the network is
unacceptably long enough for the CMS to handle.
Disadvantage:
Only some of the data in the memory
is directly accessible by each processor, since a
processor can only read and write its
local memory system.

Programming Models
Two programming models used in Multiprocessors

Message-Passing System
Shared-Memory System

Message-Passing Systems

It is structured as a set of communicating

processes, with no shared mutable state.

It can involve rendezvous between sender and

receiver.
Synchronous
Sender waits for receiver
Asynchronous Sender continues immediately
after sending a message

It allows easy system upgrade

Shared-Memory Systems

It can provide a single address space that all of

the processors can read and write.

The communications happen is implicit.

The memory system handles inter-processor
communication by allowing all processors to
see data written by any processor.

Memory Consistency Models

Defines when memory operations executed on one

processor become visible on other processors.
Two Consistency Models
Strong Consistency
- dictates that the memory system act exactly
as if there were only one memory in the computer
that different processors take turns using.

Relaxed Consistency

- allowing different processors to have different

values for a given datum until the program requests
that all memories made consistent.

Cache Coherence

It defines how data can be shared and replicated

across processors.

It also defines the specific set of actions that are

executed to keep each processors view of memory
system consistent.

Bus-based Shared-Memory System

A bus is used as the communications network that

connects the processors to each other and the
centralized memory system.

These are used to allow a variable number of

processors to communicate with each other without
changing the hardware and the bus makes it easy to
implement cache coherence.

Advantage:
Memory references can actually complete faster
if some other processors has a copy of the required
line
Disadvantage:
The bandwidth available over the bus does not
grow as the number of processors in the system
increases.

Bus-based Shared-Memory System

Processor

Processor

Processor

Processor

Cache

Cache

Cache

Cache

Memory Bus

Memory Bus

MAIN MEMORY

ASSIGNMENT:
WHAT IS SISD, MISD, SIMD, MIMD (PROVIDE
ILLUSTRATION