Parallel and Distributed Computing

Zhiguo Zhang Winter, 2009

Course Objectives:
As multithreaded and multi-core processors become the mainstream computing platform, writing parallel programs to exploit these processors computing potential becomes increasingly important. This class will teach students concepts and techniques in parallel and distributed computing. This course will concentrate on the basic concepts, models, programming, system design and software engineering issues associated with the development of parallel/distributed (enterprise) applications or system. Many interesting programming projects will be assigned. Parallel and distributed programming tools such as Pthreads, OpenMP, MPI, nVidia / CUDA will be involved.

WWW: http://sist.sysu.edu.cn/~lnszzg/09w.PDC
Email: lnszzg@mail.sysu.edu.cn Submission: 070000000.name.chapter01.hw.ver01.zip

Prerequisites:
This course requires courses in operating system, programming (C, C++). The mastering of computer networks and computer architectures is preferred. In general, you should have basic C/C++ programming skills and concepts such as threads, cache, etc.

Principal Texts:
Foundations of Multithreaded, Parallel, and Distributed Programming, Gregory R. Andrews, Addison Wesley, 2000, ISBN 0-201-35752-6.

Supplemental Texts:
Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers, Barry Wilkinson and Michael Allen, Prentice Hall, 1999, ISBN 0-13-671710-1. (China Machine Press, 2002) Distributed Computing Principles and Applications, M. L. Liu, Scalable Parallel Computing: Technology, Architecture, Programming, Kai Hwang and Zhiwei Xu, McGraw-Hill, 1999

Parallel Programming with MPI, Peter Pacheco, Morgan Kaufmann, 1996, ISBN 1-55860-339-5. Java Network Programming and Distributed Computing, David Reilly and Michael Reilly, Addison Wesley, 2002, ISBN 0-201-71037-4. Java Distributed Computing Jim Farley, O'Reilly Publishers, 1998. ISBN 1-56592-206-9. Designing and Building Parallel Programs: Concepts and Tools for Parallel Software Engineering, Ian Foster, Addison Wesley, ISBN 0-201-57594-9. Introduction to Parallel Computing: Design and Analysis of Algorithms, Vipin Kumar, Ananth Grama, Anshul Gupta, and George Karypis, Benjamin Cummings, 1993, ISBN 0-8053-3170-0.

Grading Policy:
1. Homework, Lab And Class Projects (Individual): 60% 2. Final Exam: 40% Bonus (10%): I will randomly give 10 simple quizzes in classes. The major purpose of quizzes is to check your attendance.

Tentative Lecture Outline:

Parallel and Distributed Computing Basic concepts Computing Platforms Processes and synchronization Critical sections and locks Semaphores and Pthreads library Monitors and Java Multiprocessor implementations Parallel Computing Overview of parallel computing Barriers, data parallel algorithms, bag of tasks Semaphores and Pthreads library Message passing and MPI library Lab 1 (Parallel Programming) Distributed Computing

Distributed parallel computing Clients and servers using message passing RPC and rendezvous Programming distributed systems: Ada, SR, Ada, Java sockets and RMI Distributed implementations Distributed computing paradigms Lab 2 (Distributed Computing) Networked Computing Cluster Computing Internet Applications (Client-Server and Distributed Objects) Enterprise Computing (J2EE) Assignment 2 (Distributed Computing) Formal Models CSP, SDL, Lotos, CTL and model checking

Writing (Report) Guideline:

Title Introduction Conceptual Model Detailed Computing Model(s) or Architecture Software Architecture (Implementation) Programming Considerations and Examples Summary References

Final Exam:
Multiple Choices Programming Explanation and design