Supercomputing (B.Sc.) ====================== Format ------ The seminar takes place in the following time slot: Mon, 12:00PM - 2:00PM. All of our meetings are face-to-face in room 3220, EAP2. The seminar has two parts. During the first part we read selected chapters of the book `Introduction to High Performance Computing for Scientists and Engineers `__. Earlier chapters are presented by the teaching staff, later chapters may be selected as student seminar topics. The second part discusses recent supercomputing research papers. Students may also choose any of the papers given below as their seminar topic. The general format of the seminar is similar to that of a reading group. This means that all participants read the book chapter(s) or paper *before* attending the respective sessions. A single person, either a student or somebody of the teaching staff, becomes an expert in the topic. This person presents the topic in 30 minutes and leads the discussion afterwards. Student Papers -------------- All participants write a scientific paper about their chosen seminar topic. The paper has to be submitted via email four weeks after the respective topic was discussed in the seminar. Use the ACM proceedings `template `__ with the sigconf option for your paper. The paper should be 4-6 pages in length (excl. references). You may write your paper in either English or German. Supervision ----------- Preparing presentations and writing scientific papers is hard. You may ask for advise at any time! Start early and keep in touch with your advisor! Two meetings with your advisor are mandatory: * The first meeting should be at least one week before your presentation. * The second meeting should be at least one week before your paper submission deadline. Schedule -------- +----------+----------------------------------------------------------+ | Date | What? | +==========+==========================================================+ | 04/03 | Kickoff | +----------+----------------------------------------------------------+ | 04/10 | Deadline for choosing a topic | +----------+----------------------------------------------------------+ | 04/17 | Modern Processors (Ch. 1) | +----------+----------------------------------------------------------+ | 04/24 | Basic optimization techniques for serial code (Ch. 2) | +----------+----------------------------------------------------------+ | 05/08 | Data access optimization (Ch. 3) | +----------+----------------------------------------------------------+ | 05/15 | Parallel Computers (Ch. 4) | +----------+----------------------------------------------------------+ | 05/22 | Basics of Parallelization (Ch. 5) | +----------+----------------------------------------------------------+ | 06/05 | OpenMP (Ch. 6) | +----------+----------------------------------------------------------+ | 06/12 | STRONGHOLD: Fast and Affordable Billion-Scale Deep | | | | | | Learning Model Training | +----------+----------------------------------------------------------+ | 06/19 | Locality and NUMA (Ch. 8) | +----------+----------------------------------------------------------+ | 06/26 | Message Passing Interface (Ch. 9 and 10) | +----------+----------------------------------------------------------+ | 06/27 | Get Together (06:30 PM, | | | `Daheme im Garten `__ ) | +----------+----------------------------------------------------------+ Topics ------ Select any of the following chapters/papers as your seminar topic. Additionally, you may also suggest a topic/paper. Topics will be given out on a first come, first served basis. * Introduction to High Performance Computing for Scientists and Engineers (`book `__): * Data Access Optimization (Ch. 3) * Parallel Computers (Ch. 4) * Basics of Parallelization (Ch. 5) * OpenMP (Ch. 6 and 7) * Locality and NUMA (Ch. 8) * Message Passing Interface (Ch. 9 and 10) * `SC22 `__: * HammingMesh: A Network Topology for Large-Scale Deep Learning (`paper `__, AI) * CA3DMM: A New Algorithm Based on a Unified View of Parallel Matrix Multiplication (`paper `__) * DeepSpeed-Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale (`paper `__, AI) * SpDISTAL: Compiling Distributed Sparse Tensor Computations (`paper `__, AI) * STRONGHOLD: Fast and Affordable Billion-Scale Deep Learning Model Training (`paper `__, AI) * Lessons Learned on MPI+Threads Communication (`paper `__) * `IPDPS23 `__: * Accelerating CNN inference on long vector architectures via co-design (`preprint `__, AI) * Exploiting Sparsity in Pruned Neural Networks to Optimize Large Model Training (`preprint `__, AI) * `MLSYS23 `__: * Reducing Activation Recomputation in Large Transformer Models (`preprint `__, AI) * Efficiently Scaling Transformer Inference (`preprint `__, AI) * On Optimizing the Communication of Model Parallelism (`preprint `__, AI) AI Summer School 2023 --------------------- The `AI Summer School 2023 `__ will host poster sessions where all attendees present cutting-edge research on artificial intelligence. Preparing a poster presentation for the summer school may count as a seminar presentation. Only AI-related topics may be presented at the summer school.