American Journal of Computer Science and Engineering

2014; 1(2): 12-17

Published online August 20, 2014 (http://www.openscienceonline.com/journal/ajcse)

Architecture of parallel processing in computer

organization
Prabhudev S Irabashetti1, Gawali Anjali B.2, Betkar Akshay S.2
1
Department of Computer, University of Pune, Maharashtra, India
2
Vishwabharati Academy’s college of Engg, University of Pune, Maharashtra, India

Email address
irabashetti@gmail.com (P. S. Irabashetti), Anjaligawali8@gmail.com (Gawali Anjali B.),
Akshaybetkar74@gmail.com (Betkar Akshay S.)

To cite this article

Prabhudev S Irabashetti, Gawali Anjali B., Betkar Akshay S.. Architecture of Parallel Processing in Computer Organization, American
Journal of Computer Science and Engineering. Vol. 1, No. 2, 2014, pp. 12-17

Abstract
This paper review the reporting of parallel processing in processor organization. A parallel processing becomes more
trendy, the oblige for improvement in parallel processing in processor organization becomes even more significant. Here
we evaluate each multiple processor organization, 1) SISD: single instruction single data, 2) SIMD: Single instruction
multiple data 3) MISD: Multiple instruction single data and 4) MIMD: Multiple instruction multiple data along with
Vector Processor/Array Processor, Symmetric Multiprocessing, NUMA and Cluster.
Keywords
Data, Instruction, IS: Instruction Set, CU: Control Unit, MU: Memory Unit PU: Processing Unit, LM: Local Memory,
PE: Processing Element, SMP: Symmetric Multiprocessor, NUMA: Non-Uniform Memory Access

instruction cycle, but multiple sets of operands may be

1. Introduction
The nomenclature of computer systems proposed by M. J.
Flynn in 1966 has remained the focal point in the field.
This is based on the notion of instruction and data streams
that can be simultaneously manipulated by the machine.
Parallelism is to reduce the turnaround time but even
increase the CPU time due to overhead and increase the
required memory due to duplicate data and instructions.
Writing and debugging a parallel program is much more
complicated than a sequential program, at least an order-of-
magnitude. Different architectures, different programming
models are suitable for different applications and so the
characteristics of the applications should make the decision
for the selection of parallel hardware architecture and also
the parallelization of the applications. A stream is just a
sequence of items (instruction or data) SISD: A type of
computer architecture in which there is a single instruction
cycle, and operands are fetched in serial fashion into a
single processing unit before execution. SIMD: A type of
multiprocessor architecture in which there is a single
fetched to multiple processing units and may be operated
upon simultaneously within a single instruction cycle.
MISD: is a type of parallel computing architecture where
many functional units perform different operations on the
same data. MIMD: A type of multiprocessor architecture in
which several instruction cycles may be active at any given
time, each independently fetching instructions and
operands into multiple processing units and operating on
them in a concurrent fashion.
2. Classification of Parallel
Processor Architectures
2.1. SISD Model
In computing, SISD (single instruction, single data) is a
term referring to a computer architecture in which a Single
processor, a uniprocessor, executes a single instruction
stream, to operate on data stored in a single memory. SISD
is one of the four main classifications. In this system
classifications are based upon the number of concurrent
13 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization

instructions and data streams present in the computer • Single Data: Only one data stream is being used as
architecture. SISD can have concurrent processing input during any one clock cycle
characteristics. Instruction fetching and pipelined execution • Deterministic execution
of instructions are common examples found in most • This is the oldest and even today, the most common
modern SISD computer type of computer
• A serial (non-parallel) computer • Examples: older generation mainframes,
• Single Instruction: Only one instruction stream is minicomputers and workstations; most modern day
being acted on by the CPU during any one clock PCs.
cycle

Figure 1. Taxonomy of Parallel Processor Architectures.

Figure 2. SISD model.

Figure 3. SIMD model.

• A type of parallel computer

2.2. SIMD Model
Single Instruction stream Multiple Data stream. is a class
of Parallel computer. It describes computers with multiple
processing elements that perform the same operation on
multiple data points simultaneously. Thus, such machines
exploit data level parallelism. SIMD is particularly
applicable to common tasks like adjusting the contrast in a
digital image or adjusting the volume of digital audio. Most
modern CPU designs include SIMD instructions in order to
improve the performance of multimedia use.
• Single Instruction: All processing units execute the
same instruction at any given clock cycle
• Multiple Data: Each processing unit can operate on
a different data element
• Best suited for specialized problems characterized
by a high degree of regularity, such as
graphics/image processing.
• Synchronous (lockstep) and deterministic execution
• Two varieties: Processor Arrays and Vector
Pipelines
Examples:
• Processor Arrays: Connection Machine CM-2,
MasPar MP-1 & MP-2, ILLIAC IV
• Vector Pipelines: IBM 9000, Cray X-MP, Y-MP &
 American Journal of Computer Science and Engineering 2014, 1(2): 12-17
C90, Fujitsu VP, NEC SX-2, Hitachi S820, ETA10
• Most modern computers, particularly those with
graphics processor units (GPUs) employ SIMD
instructions and execution units
2.2.1. Advantages
An application that may take advantage of SIMD is one
where the same value is being added to (or subtracted from)
a large number of data points, a common operation in many
multimedia applications. One example would be changing
the brightness of an image. Each pixel of an image consists
of three values for the brightness of the red (R), green (G)
and blue (B) portions of the color. To change the brightness,
the R, G and B values are read from memory, a value is
added to (or subtracted from) them, and the resulting values
are written back out to memory.
With a SIMD processor there are two improvements to
this process. For one the data is understood to be in blocks,
and a number of values can be loaded all at once. Instead of
a series of instructions saying "get this pixel, now get the
next pixel", a SIMD processor will have a single instruction
that effectively says "get n pixels" (where n is a number
that varies from design to design). For a variety of reasons,
this can take much less time than "getting" each pixel
individually, as with traditional CPU design.
Another advantage is that SIMD systems typically
include only those instructions that can be applied to all of
the data in one operation. In other words, if the SIMD
system works by loading up eight data points at once, the
add operation being applied to the data will happen to all
eight values at the same time. Although the same is true for
any super-scalar processor design, the level of parallelism
in a SIMD system is typically much higher.
2.3. MISD Model
In computing, MISD (multiple instruction, single data) is
a type of parallel computing architecture where many
functional units perform different operations on the same
data. Pipeline architectures belong to this type, though a
purist might say that the data is different after processing
by each stage in the pipeline. Fault-tolerant computers
executing the same instructions redundantly in order to
detect and mask errors, in a manner known as task
replication, may be considered to belong to this type. Not
many instances of this architecture exist, as MIMD and
SIMD are often more appropriate for common data parallel
14
techniques. Specifically, they allow better scaling and use
of computational resources than MISD does. However, one
prominent example of MISD in computing are the Space
Shuttle flight control computers.
Figure 4. MISD model.
• A type of parallel computer
• Multiple Instructions: Each processing unit operates
on the data independently via separate instruction
streams.
• Single Data: A single data stream is fed into
multiple processing units.
• Few actual examples of this class of parallel
computer have ever existed. One is the
experimental Carnegie-Mellon C.mmp computer
(1971).
• Some imaginable uses might be:
o multiple frequency filters operating on a single
signal stream
o Multiple cryptography algorithms attempting to
crack a single coded message.
2.4. MIMD Model
In computing, MIMD (multiple instructions, multiple
data) is a technique employed to achieve parallelism.
Machines using MIMD have a number of processors that
function asynchronously and independently. At any time,
different processors may be executing different instructions
on different pieces of data. MIMD architectures may be
used in a number of application areas such as computer-
aided design/computer-aided manufacturing, simulation,
modeling, and as communication switches. MIMD
machines can be of either shared memory or distributed
memory categories. These classifications are based on how
MIMD processors access memory Shared memory
machines may be of the bus-based, extended, or
hierarchical type. Distributed memory machines may have
15 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization
hypercube or mesh interconnection schemes. A multi-core
CPU is an MIMD machine.
Figure 5. MIMD Model.
• A type of parallel computer
• Multiple Instruction: Every processor may be
executing a different instruction stream
• Multiple Data: Every processor may be working
with a different data stream
• Execution can be synchronous or asynchronous,
deterministic or non-deterministic
• Currently, the most common type of parallel
computer - most modern supercomputers fall into
this category.
• Examples: most current supercomputers, networked
parallel computer clusters and "grids", multi-
processor SMP computers, multi-core PCs.
• Note: many MIMD architectures also include
SIMD execution sub-components
2.5. Comparison of SIMD and MIMD
• SIMD computers require less hardware than MIMD
computers because they have only one control unit.
• SIMD computers require less memory because only
one copy of the program needs to be stored. In
contrast, MIMD computers store the program and
operating system at each processor.
• However, the relative unpopularity of SIMD
processors as general purpose compute engines can
be attributed to their specialized hardware
architectures, economic factors, design constraints,
product life-cycle, and application characteristics.
• In contrast, platforms supporting the SPMD
paradigm can be built from inexpensive off-the-
shelf components with relatively little effort in a
short amount of time.
• SIMD computers require wide design effort
resulting in larger product development times.
Since the underlying serial processors change so
rapidly, SIMD computers suffer from fast
obsolescence. Their regular nature of many
applications also makes SIMD architectures less
suitable
2.6. Vector Processor/Array Processor
The Central Processing Unit (CPU), implements
instruction set containing instructions that operate on an
array of data having at least two indices. Single vector
instruction specifies a great deal of work like as executing
the whole loop. In addition, vector processors provide
vector instructions. With the use of single processor these
multiple instructions can execute with the help of
Pipelining.
• The computation of each result is independent of
the computation of previous results, allowing a very
deep pipeline without any data hazards.
• A single vector instruction specifies a tremendous
amount of work – it is the same as executing an
entire loop. Thus, the instruction bandwidth
requirement is reduced.
• Control hazards are non-existent because an entire
loop is replaced by a vector instruction whose
behavior is predetermined.
Figure 6. Vector Processor.
• There are two primary types of architectures for
vector processors: vector-register processors and
memory- memory vector processors.
In a vector-register processor, all vector
operations—except load and store—are among the
vector registers.
In a memory-memory vector processor, all vector
operations are memory to memory
Advantages of Vector Processor:
• Each result is independent of previous results by
allowing deep pipelines and high clock rates.
• A single vector instruction performs a great deal of
work - meaningless fetches and fewer branches.
• Vector instructions access memory a block at a time
which allows memory latency to be amortized over
many elements.
 American Journal of Computer Science and Engineering 2014, 1(2): 12-17
• Vector instructions access memory with known
patterns, which allows multiple memory banks to
simultaneously supply operands.
• Less memory access = faster processing time.
2.7. Symmetric Multiprocessing (SMP)
In computing, symmetric multiprocessing or SMP
involves a multiprocessor computer hardware architecture
where two or more identical processors are connected to a
single shared main memory and are controlled by a single
OS instance. Most common multiprocessor systems today
use an SMP architecture. In the case of multi-core
processors, the SMP architecture applies to the cores,
treating them as separate processors. Processors may be
interconnected using buses, crossbar switches or on-chip
Mesh network. The bottleneck in the scalability of SMP
using buses or crossbar switches is the bandwidth and
power consumption of the interconnect among the various
processors, the memory, and the disk arrays. Mesh
architectures avoid these bottlenecks, and provide nearly
linear scalability to much higher processor counts at the
sacrifice of programmability.
2.7.1. SMP Advantages
• Performance: If some work can be done in parallel
• Availability: Since all processors can perform the
same functions, failure of a single processor does
not halt the system
• Incremental growth: User can enhance performance
by adding additional processors
• Scaling: Vendors can offer range of products based
on number of processors
Figure 7. Symmetric Multiprocessing System.
2.8. Non Uniform Memory Access (NUMA)
NUMA refers to the designing of computer memory in
multiprocessor system while the memory access time
depends on the memory location. Under NUMA processor
can access the local memory faster than the non-local
memory. NUMA architectures logically follow in scaling
from symmetric multiprocessing (SMP) architectures.
16
NUMA can remove the problem, arrives when number of
processors accesses the shared memory. For that NUMA
provides the separate memory for each processor to
eliminate demerits of tightly coupled systems. NUMA
systems include additional hardware or software to move
data between memory banks. This operation slows the
processors attached to those banks, so the overall speed
increase due to NUMA depends heavily on the nature of
the running tasks.
Figure 8. NUMA.
2.9. Cluster
A cluster is a type of parallel or distributed processing
system, which consists of a collection of interconnected
stand-alone computers cooperatively working together as a
single, integrated computing resource.
– A typical cluster:
– Network: Faster, closer connection than a typical
network (LAN)
– Low latency communication protocols
– Looser connection than SMP
Clusters are responsible to improve the performance of
the system and provide the availability of the processor,
while typically being much more cost-effective than single
computers of comparable speed or availability. Clustering
relies on the centralized management approach to make
nodes available.
2.9.1. Advantages of Clusters
• Low Cost: Customers can eliminate the cost and
complexity of procuring
• Elasticity: We can add and remove computer
resources to meet the size and time requirements
for your workloads.
• Run Jobs Anytime, Anywhere: You can launch
compute jobs using simple APIs or management
tools and automate workflows for maximum
efficiency and scalability. You can increase your
speed of innovation by accessing computer
resources in minutes instead of spending time in
queues.
17 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization

3. Conclusion [4] http://www.cse.unsw.edu.au/~cs4211/04s1/seminars/rompo.

pdf
In this paper we have talked about the Architectures of all
[5] http://groups.engin.umd.umich.edu/vi/w3_workshops/VI_wi
parallel processing type in computer organization. We also nter04_ganesan.pdf
explained the types of parallel processing with example and
advantages of Single Instruction Multiple Data and
evaluation of SIMD and MIMD. We presented detailed Biography
explanation about Vector processing, SMP (symmetric Prabhudev S Irabashetti received his
multiprocessor), NUMA (Non-uniform memory access), M.Tech. degree from Davangere
Cluster and its advantages University, Davangere in June 2012. He is
presently working as Assistant Professor in
Computer Department, Vishwabharati
Academy’s College of Engg Ahmednagar,
Maharashtra, India. He is currently guiding
B.E and M.E. students of VACOE
Ahmedanagar. He has presented 2 papers in National
Conference, and published 2 papers in International Journals

Gawali Anjali B.(BE Computer Engg.)

Student- Vishwabharti Academy’s College
of Engg., Ahmednagar. University of Pune-
Pune. Maharastra-India
Figure 9. Clustering of Processors.

References Betkar Akshay S. (BE Computer Engg.)

Student- Vishwabharti Academy’s College
[1] Introduction to Parallel Processing, Cornell Theory Center of Engg., Ahmednagar. University of Pune-
Workshop, 1997.
Pune. Maharastra-India
[2] O. Serlin, The Serlin Report On Parallel Processing, No.54,
pp. 8-13, November 1991.
[3] Advances in Parallel Computing from the Past to the Future,
Dr.P Sammulal, Volume 1, Issue 4, September 2013