Parallel Programming

Course no.
252-0029-00L
Semester
Spring 2017
Lecturer
Thorsten Hoefler, Martin Vechev
Head TA
Tobias Grosser, Pavol Bielik
Assistants
Timon Gehr, Daniele Spampinato, Vytautas Astrauskas, Johannes de Fine Licht,
Konstantin Taranov, Tobias Gysi, Michael Rabinovich, Andre Kahles,
Srdan Krstic, Dimitri Stanojevic, Jingyi Wang, Julian Croci and Nicolas Ochsner
Lecture
Tue 10-12 and Wed 13-15 [ETF E 1]
Exercise
Wed 15-17 or Fri 10-12 [Student assigment to exercise groups]


Overview

The purpose of this course is to introduce students to parallel programming. By the end of the course students will be able to design and implement working parallel programs in traditional (e.g., Java Threads) and emerging parallel programming models. Moreover, students will master fundamental concepts in parallelism and be able to reason about the correctness, performance, and the construction of parallel programs using different parallel programming paradigms (e.g., task parallelism, data parallelism) and mechanisms (e.g., threads, tasks, locks, communication channels). Finally, the course will examine how parallel programming methodologies can be applied in different algorithmic domains by investigating parallelization of algorithms.

Topics include:
  • Basic parallel programming concepts
  • Parallel programming using Java
  • Synchronization techniques
  • Case studies of building parallel programs starting from sequential algorithms

Course Content

Main text and reference book

  • Introduction to Java Programming, 2014. Daniel Liang. ISBN-13: 9780133813463
  • Java Concurrency in Practice, 2006. Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, Doug Lea. ISBN-13: 9780321349606
  • The Art of Multiprocessor Programming, 2011. Maurice Herlihy, Nir Shavit. Morgan Kaufmann. Also available online in the ETH network.

Related resources, text and reference books

Introduction to Java books (freely available)

Presentation Schedule

All course materials are password protected using your ETHZ credentials.
  DateTitleSlides
  Feb 21 Motivation, topics, organization PDF
  Feb 22 Java recap PDF
  Feb 28-29 Introduction to Threads and Synchronization PDF
  Mar 7-8 Introduction to Threads and Synchronization (continued) PDF
  Mar 14 Parallel Architectures PDF
  Mar 15 Parallel Concepts PDF
  Mar 21 Parallel Models PDF
  Mar 28 Fork Join PDF
  April 4 Shared Memory Concurrency: Races, Atomicity PDF
  April 11 Data Races, Solving Mutual Exclusion with Atomic Registers PDF
  April 12 Solving Mutual Exclusion for many processes, Hardware Primitives for mutual exclusion PDF
  April 25 Beyond Locks: Deadlocks, Semaphores PDF
  April 26 Beyond Locks II: Producer-/Consumer, Monitors PDF
  May 2 Readers/Writers Lock, Lock Granularity: Coarse Grained, Fine Grained, Optimal and Lazy Synchronization PDF
  May 3 Skip lists and without Locks I PDF
  May 9 Without Locks II PDF
  May 10 A bit of Concurrency Theory: Linearizability, Sequential Consistency, Consensus PDF
  May 16 Sequential Consistency, Consensus + Transactional Memory PDF
  May 17 Transactional Memory & Programming based on Message Passing PDF
  May 23 Programming Models based on Message Passing II, CSP and MPI PDF
  May 24 Parallel Sorting (A taste of algorithms!) PDF
  May 30 Exploiting Parallelism in Compilers and Code Generators PDF
  May 31 Exploiting Parallelism in Compilers and Code Generators II PDF

Exercises

The assignment of students into exercise groups.
Students should bring their laptops to the exercise sessions as most of the exercises involve programming.
  WeekTitleDue DateAssignmentSolution
  1 Introduction - PDF
  2 Java Threads 7.3 PDF PDF PDF
  3 Java Threads II 13.3 PDF PDF PDF
  4 Parallel Models 20.3 PDF PDF
  5 Task Parallelism 27.3 PDF PDF PDF PDF
  6 Task Parallelism II 3.4 PDF PDF PDF
  7 Synchronization and Resource Sharing 10.4 PDF PDF PDF PDF
  8 More on Synchronization 24.4 PDF PDF PDF
  9 Beyond Locks 2.5 PDF PDF PDF
  10 Advanced Synchronization Mechanisms 9.5 PDF PDF PDF
  11 Advanced Synchronization Mechanisms 16.5 PDF PDF PDF PDF
  12 Transactional Memory 23.5 PDF PDF PDF
  13 Message Passing 30.5 PDF PDF PDF

Exams and Grading

There is a written, centralized exam after the end of the semester. Exercise sessions are not graded.

  • 100% of grade made up from written exam

Old Exam