Parallel

Computing
Second Edition

MORESHWAR R BHUJADE
Professor
Department of Computer Science and Engineering
Indian Institute of Technology Bombay
Mumbai, India

New Age Science Limited

The Control Centre, 11 A Little Mount Sion
Tunbridge Wells, Kent TNI 1YS, UK
www.newagescience.co.uk • e-mail: info@newagescience.co.uk
 CONTENTS

Preface to the Second Edition vii

Preface to the First Edition ix

INTRODUCTION 1
1.1 Parallel Computing 1
1.2 Parallel Architectures 7
1.3 Architectural Classification Schemes 16
1.4 Performance of Parallel Computers 20
1.5 Performance Metrics for Processors 27
1.6 Parallel Programming Models 31
1.7 Parallel Algorithms 34
1.8 Distributed Processing 35
Exercises 36
References 37

PIPELINE PROCESSING 40
2.1 Introduction 40
2.2 Pipeline Performance 44
2.3 Arithmetic Pipelines 47
2.4 Pipelined Instruction Processing 59
2.5 Pipeline Stage Design 66
2.6 Hazards 68
2.7 Dynamic Instruction Scheduling 75
2.8 Memory Systems Used in Pipelined Processors 86
2.9 Pipeline Scheduling Theory 92
2.10 High Performance Processor Designs 122
2.11 Branch Prediction 137
Exercises 150
References 159
X CONTENTS

SYNCHRONOUS PARALLEL PROCESSING 163

3.1 Introduction 163
3.2 Example-SIMD Architecture and Programming Principles 166
3.3 SIMD Parallel Algorithms 172
3.4 Data Mapping and Memory in Array Processors 181
3.5 Case Studies of SIMD Parallel Processors 186
Exercises 192
References 193

INTERCONNECTION NETWORKS 194

4.1 Introduction to Permutations 194
4.2 Elementary Permutations Used in Interconnection Networks 198
4.3 Network Classifications 203
4.4 Complete (Nonblocking) Networks: From Cross Bars to ... 204
4.5 Commonly Used Interconnection Networks 215
Exercises 226
References 227

NEUROCOMPUTING: PARALLEL ARCHITECTURES FOR COGNITIVE

FUNCTIONS 228
5.1 Introduction 228
5.2 Artificial Neuron Model (Perceptron) 232
5.3 Neural Network as Classifiers 236
5.4 Learning by Perceptrons 242
5.5 Supervised Training of Perceptron Network 247
5.6 SLT Model 253
5.7 Hopfield Network 260
Exercises 263
References 264
INDEX 267