Multiprocessors Interconnection Networks

INs Taxonomy
Interconnection Networks Static 1D 2D HC Bus-based Single Multiple Dynamic Switch-based SS MS
Cross bar

Bus-based Dynamic INs

static interconnection networks.

2.2 2.2.1

BUS-BASED DYNAMIC INTERCONNECTION NETWORKS Single Bus Systems

Single Bus Systems

A single bus is considered the simplest way to connect multiprocessor systems. Figure 2.2 shows an illustration of a single bus system. In its general form, such a system consists of N processors, each having its own cache, connected by a

Figure 2.2

Example single bus system.

Single Bus Systems

Figure 2.1

A topology-based taxonomy for interconnection networks.

Each PE has its own cache - reducing the

2.2

are established on the y as needed. Static networks can be further classied according to their interconnection pattern as one-dimension (1D), two-dimension (2D), or hypercube (HC). Dynamic networks, on the other hand, can be classied based on interconnection scheme as bus-based versus switch-based. Bus-based networks can further be classied as single bus or multiple buses. Switch-based dynamic networks can be classied according to the structure of the interconnection network as single-stage (SS), multistage (MS), or crossbar networks. Figure 2.1 illustrate this taxonomy. In the following sections, we study the different types of dynamic and static interconnection networks.

BUS-BASED DYNAMIC INTERCONNECTION NETWORKS Single Bus Systems

processor/memory trafc

2.2.1

Typically, 2-50 PEs N/W complexity - O(1) Time complexity - O(N) 1 PE access the bus at a given time

A single bus is considered the simplest way to connect multiprocessor systems. Figure 2.2 shows an illustration of a single bus system. In its general form, such a system consists of N processors, each having its own cache, connected by a

Figure 2.2

Example single bus system.

modules connected to all buses. The multiple bus with single bus memory connection has each memory module connected to a specic bus. The multiple bus with partial bus memory connection has each memory module connected to a subset of buses. The multiple bus with class-based memory connection has memory modules grouped into classes whereby each class is connected to a specic subset of buses. A class is just an arbitrary collection of memory modules. One can characterize those connections using the number of connections required and the load on each bus as shown in Table 2.2. In this table, k represents the number of classes; g represents the number of buses per group, and Mj represents the number of memory modules in class j.

Example of Commercial SB Machines

Characteristics of Some Commercially Available Single Bus Systems Maximum No. of Processors 4 8 30 Clock Rate 180 MHz 112 MHz 167 MHz Maximum Memory 4,096 MB 2,048 MB 30,720 MB

TABLE 2.1

Machine Name HP 9000 K640 IBM RS/6000 R40 Sun Enterprise 6000

Processor PA-8000 PowerPC 604 UltraSPARC 1

Bandwidth 960 MB/s 1,800 MB/s 2,600 MB/s

Multiple Bus Systems

P P P P P P

Multiple bus with full bus-memory connection (MBFBMC)

Multiple Bus Systems

P P P P P P

Multiple bus with single bus-memory connection (MBSBMC)

Multiple Bus Systems

Multiple bus with partial bus-memory connection (MBPBMC)

Multiple Bus Systems

Multiple bus with class-based memory connection (MBCBMC)

Bus Synchronization
Synchronous The time for any transaction over a sync
bus us know in advance account

Devices take the transaction time into Asynchronous Depends on the availability of data and
the readiness of devices to initiate bus transaction

Bus Synchronization
Requesting Bus Master Current Bus Master

Predened-

Bus
Bus Re quest Bus Grant Bus Busy

priority schemes

(a)
Bus Re quest

Bus Grant

Bus Busy

(b)

random simple rotating equal LRU

Figure 2.4

Bus handshaking mechanism (a) the scheme; and (b) the timing.

Switch-based Dynamic INs

Crossbar Networks
the outputs of the network. The simplest switching element that can be used is the

Figure 2.5

An 8 8 crossbar network (a) straight switch setting; and (b) diagonal switch setting.

Crossbar Networks
Time complexity - O(1) Crossbar is a nonblocking allows a multiple i/o
connection pattern to be achieved simultaneously Network complexity - O(N2)

the outputs of the network. The simplest switching element that can be used is the

Figure 2.5 An 8 8 crossbar network (a) straight switch setting; and (b) diagonal switch setting.

Single-Stage Networks

Straight

Exchange

Upper-broadcast

Lower-broadcast

Figure 2.6 The different settings of the 2 2 SE.

lish communication between a given input (source) to a ), data has to be circulated a number of times around the connection pattern for interconnecting the inputs and the network is the Shufe Exchange. Two operations are ened using an m bit-wise address pattern of . p1 p0 , as follows:

Shufe-Exchange Network

E( pm1 pm2 . . . p1 p0 ) pm1 pm2 . . . p1 p0

S( pm1 pm2 . . . p1 p0 ) pm2 pm3 . . . p1 p0 pm1

e (S) and exchange (E) operations, data is circulated fr it reaches its destination. If the number of inputs, for ex single-stage IN is N and the number of outputs, for exampl mber of SEs in a stage is N/2. The maximum length of a

2.3

SWITCH-BASED INTERCONNECTION NETWORKS

27

The Cube Network dened as follows:

Cube Network

The interconnection pattern used in the cube network is

Ci ( pm1 pm2 pi1 pi pi1 p1 p0 ) pm1 pm2 pi1 pi pi1 p1 p0 Consider a 3-bit address (N 8), then we have C2 (6) 2, C1 (7) 5 and C0 (4) 5. Figure 2.7 shows the cube interconnection patterns for a network with N 8. The network is called the cube network due to the fact that it resembles the interconnection among the corners of an n-dimensional cube (n log2 N) (see Fig. 2.16e, later). The Plus Minus 2 i (PM2I) Network connection functions dened as follows: The PM2I network consists of 2k inter-

PM2i (P) P 2i mod N(0

Figure 2.7

i , k) i , k)

PM2i (P) P 2i mod N(0

The cube network for N 8 (a) C0; (b) C1; and (c) C2 .

For example, consider the case N 8, PM21 (4) 4 21 mod 8 6. Figure 2.8 shows the PM2I for N 8. It should be noted that PM2(k1) (P)

The Plus Minus 2 i (PM2I) Network i connection functions dened as follows:

Plus-Minus 2 (PM2I) Network

PM2i (P) P 2i mod N(0 PM2i (P) P 2i mod N(0 i , k) i , k)

The PM2I network consists of 2k inter-

For example, consider the case N 8, PM21 (4) 4 21 mod 8 6. Figure 2.8 shows the PM2I for N 8. It should be noted that PM2(k1) (P) PM2(k1) (P)8P, 0 P , N. It should also be noted that PM22 C2 . This last a observation indicates that it should be possible to use the PM2I network to perform at least part of the connections that are parts of the Cube network (simulating the Cube network using the PM2I network) and the reverse is also possible. Table 2.3 provides the lower and the upper bounds on network simulation times for the b three networks PM2I, Cube, and Shufe Exchange. In this table the entries at the intersection of a given row and a given column are the lower and the upper
c

Figure 2.8 The PM2I network for N 8 (a), PM20 for N 8; (b) PM21 for N 8; and (c) PM22 for N 8.

Multistage Networks

ISC 1

ISC x-1

Switches

Switches

Switches

Figure 2.9

Multistage interconnection network.

Figure 2.10

An example 8 8 Shufe Exchange network (SEN).

The Banyan Network

Figure 2.11

An 8 8 Banyan network.

The Omega Network

Figure 2.12

The Omega network for N 8.

Blockage in MINs
Blocking networks Rearrangeable networks Non-blocking networks

2.4

STATIC INTERCONNECTION NETWORKS

33

000 001 010 011 100 101 110 111 (a) 000 001 010 011 100 101 110 111

000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111 (b)

010

Figure 2.13 Illustration of the rearrangeability of the Benes network (a) Benes network with two simulataneously established paths; and (b) the rearrangement of connection 110 ! 100 in order to satisfy connection 101 ! 001.

connection 110 ! 100, it will not be possible to establish the connection 101 ! 001 unless the connection 110 ! 100 is rearranged as shown in part (b) of the gure. Nonblocking Networks Nonblocking networks are characterized by the prop-

Figure 2.14

A three-stage Clos network.

Static Interconnection Networks

Completely Connected Networks

1 2

Figure 2.15

Example completely connected network.

Limited Connection Networks

(a)

(b)

(c)

(d)

(e)

Figure 2.16 Examples of static limited connected networks (a) a linear array network; (b) a ring network; (c) a two-dimensional array (mesh) network; (d ) a tree network; and (e) a three-cube network.

Cube-Connected Networks

Figure 2.17 A 4-cube.

Mesh-Connected Networks

Figure 2.18

A 3 3 2 mesh network.

The k-ary n-Cube Networks

K= 8 (a)

(b)

Figure 2.19 Examples of k-ary n-cube networks (a) 8-ary 1-cube (8 nodes ring) network; and (b) 8-ary 2-cube (eight 8-node rings) network.

Analysis & Performance Metrics

Dynamic Networks

Figure 2.20

An 8 8 crossbar network with a single-point failure.

Dynamic Networks
P 1 P2 PN

B1 B2 BB

M1 M2 MM

Figure 2.21

Example multiple bus system.

TABLE 2.4 Network Bus Multiple bus MINs Crossbar

Performance Comparison of Dynamic Networks Delay (Latency) O(N) O(mN) O(log N) O(1) Cost (Complexity) O(1) O(m) O(N log N) O(N 2) Blocking Yes Yes Yes No Degree of Fault Tolerance 0 (m 2 1) 0 0

Static Networks
Degree of a node Diameter Symmetric

TABLE 2.5 Network CCNs Linear array Binary tree n-cube 2D-mesh k-ary n-cube

Performance Characteristics of Static Networks Degree (d) N21 2 3 log2 N 4 2n Diameter (D) 1 N21 2(dlog2 Ne 1) log2 N 2(n 1) Nbk=2c Cost (No. of Links) N(N 1)=2 N21 N21 nN=2 2(N n) nN Symmetry Yes No No Yes No Yes Worst Delay 1 N log2 N log2 p N N k log2 N