This course was held at Tsinghua University in Summer 2017. The CMU version of this course is 15-418/618.
From smart phones, to multi-core CPUs and GPUs, to the world's largest supercomputers and web sites, parallel processing is ubiquitous in modern computing. The goal of this course is to provide a deep understanding of the fundamental principles and engineering trade-offs involved in designing modern parallel computing systems as well as to teach parallel programming techniques necessary to effectively utilize these machines. Because writing good parallel programs requires an understanding of key machine performance characteristics, this course will cover both parallel hardware and software design.
Students are expected to be strong C/C++ programmers as there will be exposure to a variety of "C-like" parallel programming languages in this course. The course assumes that students have basic knowledge of systems level topics, such as the representation of programs as machine code, how a processor executes an instruction stream, memory hierarchy design, memory management, and simple networking. While some knowledge of computer architecture would be helpful for understanding the material in this course, it is not a prerequisite. Previous experience with operating systems, distributed systems, or compilers may prove to be very helpful.
There is no required textbook for the course. Luckily, in general these days there are plenty of great parallel programming resources available for free on the web. However, if you wish to obtain a textbook for additional reading, you may find the following books helpful.
There is no final exam for this course.
Programming assignments. Students will complete three programming assignments. These assignments will involve programming in C-like languages for CPUs and GPUs such as C, ISPC, and CUDA. Each assignment will take about one week. Programming assignment grades will be based on code performance and will be automatically graded vis script so that students know an estimate of their grade at the time of submission.
Written assignments. Students will complete four written assignments. These assignments will take about four days each, and will graded to provide students feedback. Students will then have one opportunity to correct their original written handin and resubmit corrected assignments for 90% of the original credit. There will be further opportunities for students to earn up to 100% of the original credit.
- Programming Assignments (3): 20% + 20% + 20% = 60%
- Written Assignments (4): 10% + 10% + 10% + 10% = 40%
Students in this course are absolutely encouraged to talk to each other, to the TAs, to the instructors, or to anyone else about course assignments. Any assistance, though, must be limited to discussion of the problems and sketching general approaches to a solution. Each student must write their own code and produce their own writeup. Consulting another student's solution, or solutions from the internet, is prohibited on assignments. These and any other form of collaboration constitute cheating. If you have any question about whether some activity would constitute cheating, just be cautious and ask the instructors before proceeding!
You may not supply code, assignment writeups, or exams you complete during the course to other students in future instances of this course or make these items available (e.g., on the web) for use in future instances of this course (just as you may not use work completed by students who've taken the course previously). You are welcome to use source control hosts like Github for your assignments, however please be sure to make your programming assignment repositories private.