Course Syllabus 18847G: Special Topics in Computer Systems: Computational Problem Solving for Engineers Fall 2020
Location: TBA
Class Lecture: TBA Number of Units: 12
Prerequisites: 18213, 18202 or equivalent
Course Description: Computer systems accessible to students and engineers have become incredibly powerful, starting from current laptops, desktops, and servers, through cloud computing systems up to high performance computing systems. At the same time, engineering students utilize problem solving environments like Matlab, languages like Python, and systems like Torch, Spark, and TensorFlow in their research and coursework and will resort to C/C++ only when needed. First commercial quantum computing (annealing) systems are becoming available and machine learning can consume any amount of available computing power.
This course covers how to solve problem in this new computing environment. It discusses how to scale problems from the initial concept stage usually executed on one's laptop computer in Python or Matlab to more powerful computing systems like school servers, GPU accelerated systems, Condor high throughput clusters, cloud computing platforms, and high performance computing systems. It further discusses how to transition the software to parallel and accelerated C/C++ when it becomes necessary.
This course addresses a wide range of computational and informatics problems from traditional numerical simulation and symbolic data processing to emerging AI and machine learning problems. It reviews the necessary computer architecture and systems as well as engineering mathematics background. The course covers the most important scalable parallel algorithms used in engineering computing, and discusses software platforms providing problemspecific and general implementations. Further, it discusses the hardware and software infrastructure available to CMU students. It discusses algorithm analysis from the numerical and complexity perspective, parallelization approaches and scalability, algorithm optimization, evaluation and presentation of results. Finally, it discusses practical issues like obtaining real data and generating synthetic test data, as well as housing and moving of data.
Students completing the course will be able to productively solve engineering problems on current computer systems and develop algorithms from the concept stage to efficientenough scaledup implementations necessary to solve large scale problem instances. Students will obtain the necessary understanding of applicable algorithms and their scalability, as well as the underlying mathematics and numerical analysis. They will be able to leverage modern (parallel) computer systems as well as systems with accelerators and utilize applicable parallelization strategies as well as available software libraries and standard software systems. PhD students will use their own research problem as case study while MS students will solve assigned homeworks and do a final project.
Tentative Course Calendar
Date  Day  Class Activity 
August 
26  Mon.  Semester & Mini1 Classes Begin 
27  Tues.  State of computing: What is the current state of the art from embedded devices through desktops, servers, and consumer systems all the way to HPC and supercomputing 
29  Thurs.  Computer architecture: Relevant computer architecture concepts 
September 
2  Mon.  Labor Day; No Classes 
3  Tues.  Software stack: ISA, operating system, virtualization, messaging, GPU programming 
5  Thurs.  Mathematics for Engineers: The central role of numerical linear algebra 
10  Tues.  Algorithm analysis, scalability, complexity: Getting answers in time 
12  Thurs.  Parallelization: Sequential vs. parallel algorithms, scalability vs. performance 
17  Tues.  Need for speed: Principles of code optimization, when and how to optimize code 
19  Thurs.  PhD Student Projects: overview presentations 
24  Tues.  Cancelled, CMU TOC 
26  Thurs.  The ECE Computing Environment: number cluster, Andrew systems, capability machines, GPU access, cloud access, Pittsburgh Supercomputing Center and XSEDE 
October 
1  Tues.  Cloud computing and HPC: Amazon EC2/Windows Azure/Google Cloud, Computational Grids, Scientific Workflows, Computing Centers 
3  Thurs.  Desktop to big iron and the Cloud: Hadoop/Spark, TensorFlow, Matlab and Python, R 
8  Tues.  Scalable algorithms: Numerical linear algebra, CNNs/DNNs, FFTs 
10  Thurs.  Scalable algorithms: ODE and PDE solvers, discretizations 
15  Tues.  Scalable algorithms: Graph algorithms and sparse numerical linear algebra 
17  Thurs.  Scalable algorithms: Discrete and continuous optimization 
18  Fri.  MidSemester Break; No Classes 
22  Tues.  Scalable algorithms: Informatics, symbolic computing, higher level AI algorithms 
24  Thurs.  Scalable algorithms: Statistics: Monte Carlo, MCMC, statistical machine learning 
29  Tues.  Scalable algorithms: Deep Learning, Big Data Analytics 
31  Thurs.  Numerical Analysis: How good are your answers? How to make them better? 
November 
5  Tues.  PhD Student Projects: Project checkin/status report 
7  Thurs.  Data: Obtaining real data sets, data visualization, data bases, data stores, file systems 
12  Tues.  Field trip: Pittsburgh Supercomputing Center or ECE Computing/Cyert Hall 
14  Thurs.  Guest Lecture: Nick Nystrom, Interim Director, Pittsburgh Supercomputing Center 
19  Tues.  Cancelled. Oneonone project meetings 
2022   Thanksgiving Holliday; No Classes 
26  Tues.  From productivity to performance: C++, OpenMP, MPI, CUDA, Autotuning 
28  Thurs.  Beyond Classical Computing: Quantum computing, quantum annealing 
December 
3  Tues.  PhD Student Project Presentations: 
5  Thurs.  Research Talk: SPIRAL: Formal Software Synthesis of Computational Kernels 
913   Final Examinations (no final exam in this class) 
