Tensor Computations (M.Sc.)
===========================

The seminar will take place weekly in an 1 1/2h time slot.
All of our meetings will be face-to-face.

The general format of the seminar is similar to that of a reading group.
This means that all participants read the paper before attending the respective sessions.
A single person, either a student or somebody of the teaching staff, becomes an expert in the topic. 
This person presents the topic in 30 minutes and leads the discussion afterwards.

Student Papers
--------------
Each participant is required to write a scientific paper on their selected seminar topic.
The paper has to be submitted via email four weeks after the respective topic was discussed in the seminar.
Use the ACM proceedings `template <https://www.acm.org/publications/proceedings-template>`__ with the sigconf option for your paper.
The length of the paper should be between 7 and 10 pages (excl. references).
You may write your paper in either English or German.

Supervision
-----------
Preparing presentations and composing scientific papers can be challenging.
You may ask for advise at any point in time!
Start early and keep in touch with your advisor!

Two meetings with your advisor are mandatory:
  * The first meeting should be at least one week before your presentation.
  * The second meeting should be at least one week before your paper submission deadline.

Schedule
--------

+----------+-----------------------------------------------------------------------+
| Date     | What?                                                                 |
+==========+=======================================================================+
| 10/16    | Kickoff                                                               |
+----------+-----------------------------------------------------------------------+
| 10/30    | Anatomy of High-Performance Matrix Multiplication                     |
+----------+-----------------------------------------------------------------------+
| 11/06    | BLIS: A Framework for Rapidly Instantiating BLAS Functionality        |
+----------+-----------------------------------------------------------------------+
| 11/13    | A Design of a High-Performance GEMM-like Tensor-Tensor Multiplication |
+----------+-----------------------------------------------------------------------+
| 11/20    | High-Performance Tensor Contraction without Transposition             |
+----------+-----------------------------------------------------------------------+
| 12/11    | The Tensor Algebra Compiler                                           |
+----------+-----------------------------------------------------------------------+
| 12/18    | A Configurable Cloud-Scale DNN Processor for Real-Time AI             |
+----------+-----------------------------------------------------------------------+
| 02/05    | Ansor: generating high-performance tensor programs for deep learning  |
+----------+-----------------------------------------------------------------------+

Topics
------

Select any of the following papers as your seminar topic.
Additionally, you may also suggest your own paper.
Topics will be given out on a first come, first served basis.

* Anatomy of High-Performance Matrix Multiplication (`preprint <https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf>`__)
* BLIS: A Framework for Rapidly Instantiating BLAS Functionality (`paper <https://dl.acm.org/doi/pdf/10.1145/2764454>`__)
* High-Performance Tensor Contraction without Transposition (`paper <https://dl.acm.org/doi/10.1137/16M108968X>`__)
* LazyTensor: combining eager execution with domain-specific compilers (`preprint <https://arxiv.org/abs/2102.13267>`__)
* A Design of a High-Performance GEMM-like Tensor-Tensor Multiplication (`preprint <https://arxiv.org/pdf/1607.00145.pdf>`__, `paper <https://dl.acm.org/doi/10.1145/3157733>`__)
* SpDISTAL: Compiling Distributed Sparse Tensor Computations (`paper <https://dl.acm.org/doi/abs/10.5555/3571885.3571963>`__)
* The Tensor Algebra Compiler (`paper <http://tensor-compiler.org/files/kjolstad-oopsla17-tensor-compiler.pdf>`__)
* HAOTuner: A Hardware Adaptive Operator Auto-Tuner for Dynamic Shape Tensor Compilers (`paper <https://ieeexplore.ieee.org/document/10160123>`__)
* GraphTensor: Comprehensive GNN-Acceleration Framework for Efficient Parallel Processing of Massive Datasets (`paper <https://doi.ieeecomputersociety.org/10.1109/IPDPS54959.2023.00011>`__)
* An Efficient 2D Method for Training Super-Large Deep Learning Models (`paper <https://ieeexplore.ieee.org/document/10177476>`__)
* Dynamic Tensor Linearization and Time Slicing for Efficient Factorization of Infinite Data Streams (`paper <https://ieeexplore.ieee.org/document/10177430>`__)
* Optimizing High-Performance Linpack for Exascale Accelerated Architectures (`preprint <https://arxiv.org/abs/2304.10397>`__)
* HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity (`preprint <https://arxiv.org/abs/2305.12718>`__)
* A Configurable Cloud-Scale DNN Processor for Real-Time AI (`paper <https://doi.org/10.1109/ISCA.2018.00012>`__)
* Ansor: generating high-performance tensor programs for deep learning (`paper <https://dl.acm.org/doi/abs/10.5555/3488766.3488815>`__)