Main course text books#
The course does not strictly follow any particular text book. However, most of the concepts treated in the course are presented (sometimes from different perspectives and/or using different methodologies and principles) in the ones listed below. Please note that some of these books are available for free from O’Reilly for ANU students. You will need to enter your ANU email address at this site to get access.
-
Parallel Programming: techniques and applications using networked workstations and parallel computers, B. Wilkinson and M. Allen. Prentice Hall 2nd edition (March 4, 2004). ISBN 0131405632. (Amazon link Pearson Link)
-
Introduction to Parallel Computing, 2nd Ed., A. Grama, A. Gupta, G. Karypis, V. Kumar, Addison-Wesley 2003, ISBN 0201648652. Available for free in electronic form from here.
-
Introduction to High Performance Computing for Scientists and Engineers (Chapman & Hall/CRC Computational Science) Georg Hager and Gerhard Wellein, 2011.
-
High Performance Computing. Modern Systems and Practices, T. Sterling, M. Anderson and M. Brodowicz. Morgan Kaufmann, 2018. Available for free in electronic form from here.
-
Parallel Programming: Concepts and Practice, Bertil Schmidt et al, Morgan Kaufmann, 1st edn 2017. Covers the course’s (and more) programming paradigms nicely. Available for free in electronic form from here.
-
Principles of Parallel Programming, C. Lin and L. Snyder. Pearson, 2009. International Edition, - ISBN 978-0-321-54942-6 or US Edition - ISBN 978-0-321-48790-2 (Amazon link, Pearson link)
Other recommended books and reports#
- Computer Systems: A Programmer’s Perspective, R.E. Bryant and D. O’Hallaron, Pearson/Prentice Hall, ISBN 0-13-178456-0. This is an excellent book on machine architecture and related issues. It is not a basic text book, but if you understand all the material in this book, you will have an excellent grounding in computer systems. You do not need to buy this text for the course, but if you want to know more about the details of what happens at the single CPU level then read this book.
- Using MPI: Portable Parallel Programming with the Message-Passing Interface W. Gropp, W. Lusk, A. Skjellum, MIT Press 1999, ISBN 0-262-57134-X. Probably the first text book on MPI1. OK, but there are lots of on line reference MPI books, so don’t buy it for this course!
- Using MPI-2: Advanced Features of the Message-Passing Interface W. Gropp, W. Lusk, R. Thakur, MIT Press 1999, ISBN 0-262-057133-1. More worth buying than the MPI1 book - because MPI2 has some more tricky features. Here for reference - don’t go and buy this book for the course!
- Programming with POSIX Threads, D.R. Butenhof, Addison-Wesley 1997, ISBN 0-201-63392-2. If you want to get serious programming pthreads then you probably want to buy a book. This book is quite good - but again, you don’t need it for the course.
- CUDA by Example: An Introduction to General-Purpose GPU Programming, Jason Sanders, Edward Kandrot. NVIDIA, 1st Edn, 2010. This is one of several books about programming GPUs. It has a lot of examples, starting from the most basic and working up.
- Programming Massively Parallel Processors: A Hands-on Approach, D. Kirk and W. Hwu, Morgan Kaufmann, 2nd Edn, 2013. Worth looking at to find out how GPUs actually work and how to get high performance out of them for real-world problems such as MRI reconstruction.
- What Every Programmer Should Know About Memory, Ulrich Drepper, 2007, online. A comprehensive reference on CPU and memory, intersecting in particular with the Shared Memory sections of the course.
C Programming book#
- Stephen Kochan’s book Programming in C. Available for free electronically for ANU students.
- Effective C: An Introduction to Professional C Programming by Robert C. Seacord. Available for free electronically for ANU students.
Tutorials#
- OpenMP Tutorial from LLNL (US).
- MPI Tutorial from LLNL (US).
- POSIX threads Tutorial from LLNL (US).