CSE822/CMSE822 Parallel Computing

Homeworks/Pre-class/In-class assignments of CSE822 Parallel Computing taken by Prof William (Bill) Punch at Michigan State University (MSU) in Fall 2021

Schedule

1. Lectures - Tuesday/Thursday 10:20-11:40 AM in Room EB1230
2. Office Hours - Monday/Wednesday 3:00-5:00 PM on Zoom 9299924290/395525

Requirements

1. Ubuntu 18.04
2. C++
3. MSU HPCC Access

Other platforms have not been tested.

Books

1. Intro to HPC - Victor Eijkhout, Edmond Chow, Robert van de Geijn pdf, html
2. Parallel Programming for Science and Engineering - Victor Eijkhout pdf, html

Lectures

Date	Wk/Day	Pre-class Slides	Video	Book	Submit	In-class Topics	Slides	Homework
02-09	0/1					Introduction, Logistics	Intro
07-09	1/1	Pointers			PointerCode	HPCC Usage
09-09	1/2		CPU	1.2	Exer 1.1	Branching		HW1 Released
14-09	2/3	C-arrays, 2D-arrays	Cache, VM	1.3-1.4	Exer 1.12	Cache	Cache
16-09	2/4			1.6, 1.7.1- 1.7.4, 2.1- 2.5	Exer 1.18, 2.3, 2.10, 2.16	Arithmetic Intensity, Amdahl Law, Scalability	Roofline	HW2 Released
21-09	3/5	Command line args	Thread till 12:30		Effect of .join	Threads1	Threads1
23-09	3/6		Thread 12:30 to 28:30		.join/.detach, copy/move	Threads2	Threads2	HW3 Released
27-09	4/7				Avoid race, deadlock, detect race	Concurrency1	Concurrency1
29-09	4/8		Concurrency from 28:30			Concurrency2	Concurrency2	HW4 Released
05-10	5/9				async default, future advantages	OpenMP	Tasks
07-10	5/10		OpenMP Intro,False sharing example	Chap 17	PP Exer 17.3	OpenMP	OpenMP	HW5 Released
12-10	6/11		For loops	Chap 18, 19,20	stringstream, thread local	OpenMP	OpenMP2
14-10	6/12		OpenMP Sync	Chap 23 till 23.3.1	Critical/Atomic/Reduction	OpenMP Sync	OpenMP Sync	HW6 HW7 Released
19-10	7/13		Worksharing 29:00-38:00 Tasks	PP 19.3 21.1 24	PP Exer 19.3	OpenMP Schedule Task	OpenMP3
21-10	7/14		Git		Add Func Template	Func Templates	Func Templates
19-11	11/20			-	Cuda compile on HPCC	Cuda	GPU
23-11	12/21		Vector_add	-	Diff between C malloc, C++ new, cuda cudaMalloc, cudaMallocManaged; sync GPU + CPU	Cuda	Cuda
25-11			Thanksgiving break
30-11	13/22		Vector_add	-	nvprof on profile.cu	More Cuda	More Cuda	HW 11 released
02-12	13/23		Tiled Matrix_Mul	-	nvvp ./a.out	Cuda Matrix Mul	Mat Mul

Debugging Segmentation Fault

1. gdb

• Compile your application with -g, then you'll have debug symbols in the binary file.

g++ -g cpp_file.cpp -o binary.o

• Use gdb to open the gdb console.
• Type

file binary.o

• Type

run argument1 argument2

• Do something to cause a Segmentation Fault.
• Type bt to get a stack trace of the Segmentation Fault.

HPCC Stuff

1. HPCC Access
2. SLURM scheduling and SLURM job management
3. Module basics
4. HPCC Tutorial Series
5. HPCC Video Series