Chapter 1

Introduction To Advanced
Computer Architecture And
Parallel Processing

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.1 Four Decades of Computing

Feature

Batch

Time-Sharing

Desktop

Network

Decade

1960s

1970s

1980s

1990s

Location

Terminal Room

Desktop

Mobile

Users

Computer
Room
Experts

Specialists

Individuals

Groups

Data

Alphanumeric

Text, numbers

Font,Graphs

Multimedia

Objective

Calculate

Access

Present

Communicate

Interface

Punched Card

Keyboard, CRT

See & Point

Ask & Tell

Operation

Process

Edit

Layout

Orchestrate

Connectivity

None

LAN

Internet

Owners

Corporate
Computer Centers

Peripheral
Cable
Divisional IS
Shops

Dept. End
Users

Everyone

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.2 Flynns Taxonomy of Computer

Architecture
Single Instruction Single Data (SISD)
Single Instruction Multiple Data (SIMD)
Multiple Instruction Multiple Data (MIMD)
Multiple Instruction Single Data (MISD)

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.2 Flynns Taxonomy of Computer

Architecture
Single Instruction Single Data (SISD)
Conventional single-processor von Neumann
computers are classified as SISD systems.
Instruction Stream

I/O

Control
Unit

Processor
(P)

Instruction Stream

Advanced Computer Architecture and Parallel Processing

Memory
(M)
Data Stream

Hesham El-Rewini & Mostafa Abd-El-Barr

1.3 SIMD Architecture

Single Instruction Multiple Data (SIMD)
Consists of 2 parts:
a front-end Von Neumann computer.
A processor array: connected to the memory bus of
the front end.
Data Stream
P1

M1

Instruction Stream
Data loaded
from front end

Control
Unit
Data Stream
Program loaded
from front end

Advanced Computer Architecture and Parallel Processing

Pn

Mn

Hesham El-Rewini & Mostafa Abd-El-Barr

1.3 SIMD Architecture

SIMD Scheme 1
Each processor has its own local memory.
Control Unit

P1

P2

P3

Pn-1

M1

M2

M3

Mn-1

Pn

Mn

Interconnection Network

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.3 SIMD Architecture

SIMD Scheme 2
Processors and memory modules communicate with
each other via interconnection network.
Control Unit

P1

P2

P3

Pn-1

Pn

Interconnection Network

M1

M2

Advanced Computer Architecture and Parallel Processing

M3

Mn-1

Mn

Hesham El-Rewini & Mostafa Abd-El-Barr

1.4 MIMD Architecture

Multiple Instruction Multiple Data (MIMD)
Made of multiple processors and multiple memory
modules connected together via some
interconnection network.
2 broad categories:
Shared memory
Message passing

Instruction Stream
Control
Unit-1

Data Stream
P1

M1

Instruction Stream

Instruction Stream
Control
Unit-n

Data Stream
Pn

Mn

Instruction Stream

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.4 MIMD Architecture

Shared Memory versus Message Passing
Architecture
M

Interconnection Network

Interconnection Network

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.4 MIMD Architecture

Shared Memory Organization
Inter-processor coordination is accomplished by
reading and writing in a global memory shared by all
processors.
Typically consists of servers that communicate
through a bus and cache memory controller.
M

Interconnection Network

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.4 MIMD Architecture

Message Passing Organization
Each processor has access to its own local memory.
Communications are performed via send and receive
operations.
Message passing multiprocessors employ a variety of
static networks in local communications.
Interconnection Network

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.5 Interconnection Networks (INs)

Mode of Operation
Synchronous:
a single global clock is used by all components in the system
(lock-step manner).

Asynchronous:
No global clock required
Hand shaking signals are used to coordinate the operation of
asynchronous systems.

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.5 Interconnection Networks (INs)

Control Strategy
Centralized: one central control unit is used to control
the operations of the components of the system.
Decentralized: the control function is distributed
among different components in the system.

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.5 Interconnection Networks (INs)

Switching Techniques
Circuit switching: a complete path has to be
established prior to the start of communication
between a source and a destination.
Packet switching: communication between a source
and a destination takes place via messages divided
into smaller entities, called packets

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.5 Interconnection Networks (INs)

Topology
Describes how to connect processors and memories
to other processors and memories.
Static: direct fixed links are established among nodes
to form a fixed network.
Dynamic: connections are established when needed.

Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr

1.6 Summary
A number of concepts and system configurations
related to obtaining high-performance computing
via parallelism were introduced.
Flynns taxonomy was provided.
An introduction to SIMD and MIMD systems was
given.
Shared-memory and message passing systems
and their interconnection networks were
introduced.
Advanced Computer Architecture and Parallel Processing

Hesham El-Rewini & Mostafa Abd-El-Barr