PARALLEL PROGRAMMING

3003

OBJECTIVE
To identify the scope for parallelism in a program, understand the various parallel
programming models and the challenges involved in parallel programming, learn the
basics of OpenMP and MPI and programming with them, and the programming of
heterogeneous systems using CUDA and OpenCL
UNIT I

FUNDAMENTALS OF PARALLEL COMPUTING

Need for Parallel Computing Parallel Computer Models ILP, TLP and Data
Parallelism Parallel Programming Overview Processes, Tasks and Threads
Parallel Programming Models Shared Memory Programming Message Passing
Paradigm Interaction and Communication Interconnection Networks.
UNIT II

CHALLENGES OF PARALLEL PROGRAMMING

Identifying Potential Parallelism Techniques for Parallelizing Programs Issues

Cache Coherence issues Memory Consistency Models Maintaining Memory
Consistency Synchronization Issues Performance Considerations.
UNIT III SHARED MEMORY MODELS AND OPENMP PROGRAMMING

OpenMP Execution Model Memory Model and Consistency OpenMP Directives

Run Time Library Routines Handling Data and Functional Parallelism
Performance Considerations.
UNIT IV MPI PROGRAMMING

The MPI Programming Model MPI Basics Circuit Satisfiability Global

Operations Asynchronous Communication Collective Communication Other
MPI Features Performance Issues Combining OpenMP and MPI.
UNIT V

PROGRAMMING HETEROGENEOUS PROCESSORS

GPU Architecture Basics of CUDA CUDA Threads CUDA Memories

Synchronization Handling Performance Issues Application Development.
Introduction to OpenCL.
TOTAL: 45

TEXT BOOKS:
1.

2.
3.
4.

John L. Hennessey and David A. Patterson, Computer Architecture A

quantitative approach, Morgan Kaufmann / Elsevier Publishers, 4 th. Edition,
2007.
Peter S. Pacheco, An Introduction to Parallel Programming, Morgan
Kaufmann, 2011.
Michael J Quinn, Parallel programming in C with MPI and OpenMP, Tata
McGraw Hill, 2003.
David B. Kirk and Wen-mei W. Hwu, Programming Massively Parallel
Processors, Morgan Kaufmann, 2010.

REFERENCES:
1.

2.
3.

4.

5.
6.

Ananth Grama, George Karypis, Vipin Kumar and Anshul Gupta,

Introduction to Parallel Computing, Second Edition, Pearson Education
Limited, 2003.
Shameem Akhter and Jason Roberts, Multi-core Programming, Intel Press,
2006.
Ian Foster, Designing and Building Parallel Programs: Concepts and Tools
for Parallel Software Engineering, Addison Wesley Longman Publishing
Co., USA, 1995.
David E. Culler, Jaswinder Pal Singh, Parallel Computing Architecture: A
hardware/ Software approach , Morgan Kaufmann / Elsevier Publishers,
1999.
OpenMP Programmers Manual.
MPI Programmers Manual