Practical Course on High-Performance Computing
This practical course comprises of a crash course on the basics of High-Performance Computing, which is delivered during a one-week block tutorial. Including hands-on exercises, it will cover theoretical knowledge regarding parallel computing, high-performance computing, supercomputers, and the development and performance analysis of parallel applications. Practical demonstrations will encourage you to utilize the GWDG cluster system to execute existing parallel applications, start developing your own parallel application using MPI and OpenMP, and to analyze the performance of these applications to ensure they run efficiently.
On the first day of the tutorial, we will help you form groups of three to four people to work on the exercises and form a learning community.
For students, we will present on the last day of the tutorial a group assignment that you will have to solve in pairs.
If you are just interested to learn about parallel programming and don't need credits, you can join the block tutorial part of the course and gain a certificate.
This course is suitable for Bachelor and Master students and the block course is suitable for GWDG academy participants. We aim to form suitable learning groups for all attendees.
Key information
Contact | Julian Kunkel | ||
Location | Broadcast: https://meet.gwdg.de/b/jul-tff-dq1-rt0 Breakout: https://meet.gwdg.de/b/jul-ffv-ljs-7u5 | ||
Time | Full week: Tuesday. 02.04.24 - Friday. 05.04.24 - 09:00 - 18:00, Half Day Monday. 08.04.24 - Tuesday. 09.04.24 - 14:00 - 18:00, Final presentation Monday. 16:15-17:45 starting 17.06.24 | ||
Language | English | ||
Module | Modul M.Inf.1829: Praktikum High-Performance Computing (6C) Modul B.Inf.1803: Fachpraktikum I (5C) Modul B.Inf.1804: Fachpraktikum II (5C) Modul B.Inf.1805: Fachpraktikum III (5C) Modul B.Inf.1833: Fachpraktikum Data Science (9C) Modul B.Inf.1834: Fachpraktikum Data Science I (klein) (5C) Modul B.Inf.1835: Fachpraktikum Data Science II (klein) (5C) |
||
SWS | 6 | ||
Credits | 5,6,9 (depending on the course) | ||
Contact time | up to 84 hours (63 full hours), depending on the course | ||
Independent study | up to 186 hours | ||
Tutors | Jonathan Decker, Julian Kunkel |
Please note that we plan to record sessions (lectures and seminar talks) with the intent of providing the recordings via BBB to other students but also to publish and link the recordings on YouTube for future terms. If you appear in any of the recordings via voice, camera or screen share, we need your consent to publish the recordings. See also this Slide.
Learning Objectives
The students will be able to
- Construct parallel processing schemes from sequential code using MPI and OpenMP
- Justify performance expectations for code snippets
- Sketch a typical cluster system and the execution of an application
- Characterize the scalability of a parallel application based on observed performance numbers
- Analyze the performance of a parallel application using performance analysis tools
- Describe the development and executions models of MPI and OpenMP
- Construct small parallel applications that demonstrate features of parallel applications
- Demonstrate the usage of an HPC system to load existing software packages and to execute parallel applications and workflows
- Demonstrate the application of software engineering concepts
- Construct a small quantum computer program
Agenda
Block Course
The block seminar contains sessions that contain short lectures followed by hands-on exercises that teach the most relevant aspects of the concepts and tools. The hands-on typically starts with a short tutorial as a walk-through followed by guided learning based on a provided worksheet with exercises and descriptions. Participants can take breaks as necessary during guided learning.
Computer science students, please attend still your lectures, this year the easter vacations are unfortunately located such that scheduling is difficult.
This part is attended by BSc/MSc students and GWDG academy participants
Everyone, please prepare via the information given in this sheet Prep exercise
Tuesday 02.04.2024
Introductory day – optional for computer students
- 09:00 - 9:45 Welcome, Organization of the day and study groups – Julian Kunkel Slides
- 09:45 - 12:00 Crash Course for Linux Shell – Kevin Lüdemann Slides
- Exercise (login to the cluster) Exercise
- 12:00 - 13:00 Lunch Break
- 13:00 - 14:00 Git – Jonathan Decker Slides
- Exercise Exercise
- 14:00 - 17:00 Crash Course for Linux – Kevin Lüdemann Slides
- Lecture Overview Linux System/Folders/Editors
- Exercise Compilation of applications Exercise
Wednesday 03.04.2024
- 09:00 - 09:50 Welcome, Organization of the practical course and study groups – Julian Kunkel Slides
- 09:50 - 10:30 Overview on parallel computing – Julian Kunkel
- 10:30 - 12:00 Using a cluster for remote computation – introduction – Azat Khuziyakhmetov Slides
- Exercise
- 12:00 - 13:00 Lunch Break
- 13:00 - 14:30 Managing Software using Spack – Azat Khuziyakhmetov Slides
- 16:00 - 18:00 Using Slurm to run applications on the cluster – Azat Khuziyakhmetov Slides
Thursday 04.04.2024
- 12:00 - 13:00 Lunch Break
- 13:00 - 14:00 Quantum Computing - A brief introduction for the curious – Lourens van Niekerk
- 15:00 - 18:00 OpenMP (Exercise)
Friday 05.04.2024
- 09:00 - 10:00 Basic Principles of Parallel Computing – Oswald Haan Slides
- 10:00 - 11:00 MPI: General Introduction Slides
- 11:00 - 12:00 MPI: Point to Point Communication (1) Slides
- 12:00 - 13:00 Lunch Break
- 13:00 - 14:00 MPI: Point to Point Communication (2) Exercise
- 14:00 - 16:00 Parallel Application: Calculation of pi Slides
Monday 08.04.2024
Tuesday 09.04.2024
- 14:00 - 15:00 Benchmarking – Aasish Kumar Sharma Slides Exercise 1 + 2
- 15:00 - 16:30 Introduction performance engineering – Aasish Kumar Sharma Slides
- 16:30 - 17:30 Joint performance analysis exercise Exercise - Task 3 - Exercise
- Exercise: How to write a benchmark
- Theoretic assessment of benchmarking results
- 17:30 - 18:00 Description of the group assignment – Julian Kunkel, Jonathan Decker Sheet
Registration
Students, please register using StudIP. GWDG Academy participants, please register there.
Relevant for Students to Obtain the Credits
This remaining part is mandatory for BSc and MSc students to obtain the credits but can be skipped if you just want to join the block course.
In order to obtain the credits, you will parallelize a non-trivial problem of your choice using the concepts and tools learned during the block course. As an alternative option, you could choose from an administrative topic that we derive from a practical problem at the GWDG.
Firstly, you will decide upon a problem you like to solve, then you will create a sequential solution to this problem, and lastly, you apply the experience of the block course to parallelize your application and analyze its scalability. You need to prepare a presentation for your fellow students as well as document your solution in a report. Both the presentation and the report are due at the end of the term and will be assessed and marked.
Examination
To obtain the credits, students must develop their (own) group project. A presentation of 20 min and report (max 15 pages per group member) must be created. The mark consists of 30% presentation and 70% report.
The report must be submitted as a PDF file. The source code must be handed in as well. Optimally as a Gitlab or Github repository.
Project Meetings
This attendance of the virtual project meetings is mandatory for BSc and MSc students to obtain the credits.
There will be three project meetings in which the groups introduce their results. These meetings are not marked but provide valuable feedback toward your project. In the first seminar, the project idea and rough implementation plan are shared. The second seminar will introduce a report and gives you the opportunity to ask questions regarding the preparation of the report. In the final meeting, the project results are shared.
Dates
- 22.04.2024 - Submit the project title to Jonathan and group composition - we will then assign a supervisor to you
- 27.05.2024 16:15-17:45 (https://meet.gwdg.de/b/jul-tff-dq1-rt0) - Project introduction: Students present their project idea and project plan (5 (+5) min per group)
- 17.06.2024 16:15-17:45 - Result presentation (20 min per group)
- Group 1 - David Alexandre Silva, Jan Lenke, Claas Kochanke - Wild-Fire-Simulation using C++
- 24.06.2024 16:15-17:45 - Result presentation (20 min per group)
- Group 2 - Ughur Mammadzada - Simulating multi-star systems using Python (and maybe C++)
- Group 3 - Sebastian Thurm - MPI for <https://github.com/N720720/lindemann>
- 01.07.2024 16:15-17:45 - Result presentation (20 min per group)
- Group 7 - Siddharth Simediya, Sunny Jain - Poker Simulation using MPI
- Group 5 -Frederik Hennecke, Pascal Brockmann - Parallel SAT solver in C/C++
- 08.07.2024 16:15-17:45 - Result presentation (20 min per group)
- Group 6 - Karan Sharma, Pranay Bhatia - Parallel Ray Tracing
- Group 4 - Karthik Banjan, Hanumanth Padmanabhan - Fractal Generation Algorithms in Rust
- 30.09.2024 Last day of term - Submission of your report + slides + code + repo jonathan.decker@uni-goettingen.de
- Make sure your repo is accessible either internally or publicly
Example Projects
Projects should cover either the creation and parallelization of an interesting problem such as:
- Determine optimal moves (search tree method) for games.
- (Simple) predator-prey relationship of a closed system with animal migration.
- Cars in the traffic of a city network and resulting traffic jams.
- Astrophysical computations.
- Skat, Go or robot simulation
- Longest path problem
- Solving large logical formulas
- Algorithms from bioinformatics
- Strategies for placing airplane passengers
Most important is the correct parallelization (possibly with alternatives) and evaluation. Detailed knowledge of numerics or a precise representation of the model is not required.
Alternatively, we provide a list of specific topics that are motivated by our needs in the data center, these may cover administrative aspects as well. We are open to further suggestions.
Examples can be found in the course from last year or here. Have a look at the results and videos generated.
Content of the Project
For a generic parallelization project, some contents should be worked on and accordingly included in the presentation and elaboration, the aspects are:
- Concepts of the underlying (application) model.
- Parallelization scheme (communication pattern, distribution of data & tasks).
- Parallelization should be done with MPI (optional: shared memory parallelization with threads or OpenMP).
- Performance analysis of the sequential code (does it behave as expected).
- Scaling behavior of the parallel version
- Speedup diagrams
- Potential analysis with Vampire/Sunshot
- Performing an optimization of the parallel version (communication scheme etc.)
Project Topics
This is a list of topics for the individual projects that are motivated by our needs in the data center. We welcome it if you propose your own simulation, parallelization, or administrative topic. Even for our suggested topics, you will have some room for developing the topic in the direction of your choice.
Group | Student | Supervisor | Topic | Submissions |
1 | David Alexandre Silva | Martin Paleico | Wild-Fire-Simulation using C++ | Report Slides Code |
1 | Jan Lenke | Martin Paleico | Wild-Fire-Simulation using C++ | |
1 | Claas Kochanke | Martin Paleico | Wild-Fire-Simulation using C++ | |
2 | | Martin Paleico | Simulating multi-star systems using Python (and maybe C++) | |
2 | Ughur Mammadzada | Martin Paleico | Simulating multi-star systems using Python (and maybe C++) | Report Slides Code |
2 | | Martin Paleico | Simulating multi-star systems using Python (and maybe C++) | |
3 | Sebastian Thurm | Christian Köhler | MPI for <https://github.com/N720720/lindemann> | Report Slides |
4 | Karthik Banjan | Artur Wachtel | Fractal Generation Algorithms in Rust | Report Slides |
4 | Hanumanth Padmanabhan | Artur Wachtel | Fractal Generation Algorithms in Rust | |
5 | Frederik Hennecke | Zoya Masih | Parallel SAT solver in C/C++ | Report Slides Code |
5 | Pascal Brockmann | Zoya Masih | Parallel SAT solver in C/C++ | |
6 | Karan Sharma | Zoya Masih | Parallel Ray Tracing | Report Slides Code |
6 | Pranay Bhatia | Zoya Masih | Parallel Ray Tracing | |
7 | Sunny Jain | Julian Kunkel | Poker Simulation using MPI | Report Slides Code |
7 | Siddharth Simediya | Julian Kunkel | Poker Simulation using MPI |