====== Practical Course on High-Performance Computing ====== This practical course comprises two parts: firstly, a crash course on the basics of High-Performance Computing is delivered during a one-week block tutorial. Including hands-on exercises, it will cover theoretical knowledge regarding parallel computing, high-performance computing, supercomputers, and the development and performance analysis of parallel applications. Practical demonstrations will encourage you to utilize the GWDG cluster system to execute existing parallel applications, start developing your own parallel application using MPI and OpenMP, and to analyze the performance of these applications to ensure they run efficiently. On the first day of the tutorial, we will help you form groups of three four people to work on the exercises. We will form a learning community that will blend into the second part of the course. For students, we will present on the last day of the tutorial a group assignment that you will have to solve in **pairs**. **If you are just interested to learn about parallel programming and don't need credits, you can join only the first part of the course and gain a certificate.** This course is suitable for Bachelor and Master students and the block course is suitable for [[https://www.gwdg.de/academy|GWDG academy]] participants. We aim to form suitable learning groups for all attendees. ===== Key information ===== || Contact || [[about:people:julian_kunkel|Julian Kunkel]] || || Location || [[https://meet.gwdg.de/b/jul-tff-dq1-rt0|Virtual in BBB]] || || Time || Full week: Monday 25.04 - Friday 29.04, 9:00 - 18:00, Final presentation tbd. || || Language || English || || Module || Modul M.Inf.1829: Praktikum High-Performance Computing || || SWS || 6 || || Credits || 5,6,9 (depending on the course) || || Contact time || up to 84 hours (63 full hours), depending on the course || || Independent study || up to 186 hours || || Tutors || Sven Bingert, Marcus Boden, Christian Boehme, [[about:people:jonathan_decker|Jonathan Decker]], Laura Endter, Oswald Haan, [[about:people:julian_kunkel|Julian Kunkel]], Hendrik Nolte, Jack Ogaja, Vanessa End, Ruben Kellner || {{:teaching:summer_term_2022:praktikum_high-performance_computing.pdf |Module description}} ===== Learning Objectives ===== The students will be able to * Construct parallel processing schemes from sequential code using MPI and OpenMP * Justify performance expectations for code snippets * Sketch a typical cluster system and the execution of an application * Characterize the scalability of a parallel application based on observed performance numbers * Analyze the performance of a parallel application using performance analysis tools * Describe the development and executions models of MPI and OpenMP * Construct small parallel applications that demonstrate features of parallel applications * Demonstrate the usage of an HPC system to load existing software packages and to execute parallel applications and workflows * Demonstrate the application of software engineering concepts ===== Agenda ===== ==== Block Seminar ==== The block seminar contains sessions that contain short lectures followed by hands-on exercises that teach the most relevant aspects of the concepts and tools. The hands-on typically starts with a short tutorial as a walk-through followed by guided learning based on a provided worksheet with exercises and descriptions. Students can take breaks as necessary during guided learning. **Please prepare your PC/laptop following the instructions in our {{ :teaching:summer_term_2022:pchpc_preparation_exercise_0.pdf |preparation exercise sheet}}**. For transferring files between your home machine and the HPC compute cluster, see the following instructions: {{ :teaching:summer_term_2022:pchpc_transferring_files_guide.md |Transferring Files}} **We understand that it is not possible for everyone to participate in all sessions this week at all times. If you have, for example, other lectures at the same time you should attend them. If you are present for at least 80% of the course it is no problem. It is of course in your own interest to visit the sessions.** The videos of the event are available in [[https://www.youtube.com/watch?v=hqJFTYlM3XE&list=PLvcoSsXFNRbntX3_qH7LPW40vCrqhWBKG|YouTube]]. //This part is attended by BSc/MSc students and GWDG academy participants// === Monday 25.04.2022 === * 09:00 - 10:15 **Welcome**, Organization of the practical course -- //Julian Kunkel// {{ :teaching:summer_term_2022:pchpc_welcome_slides.pdf |Slides}} * 10:15 - 11:00 **Crash Course for Linux** 1/2 -- //Jonathan Decker// {{ :teaching:summer_term_2022:pchpc_linux_crash_course_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_linux_crash_course_exercise_1.pdf |Exercise}} * ~11:00 Break * 11:05 - 12:00 **Crash Course for Linux** 2/2 * 12:00 - 12:45 Lunch Break * 12:45 - 13:45 **Cluster introduction** -- //Marcus Boden// {{ :teaching:summer_term_2022:pchpc_cluster_intro_slides.pdf |Slides}} * 13:45 - 14:40 **Managing Software using Spack** 1/2 -- //Laura Endter// {{ :teaching:summer_term_2022:pchpc_spack_slides.pdf |Slides}} * ~14:40 - Break * 14:45 - 15:45 **Managing Software using Spack** 2/2 * 15:45 - 17:00 **Using Slurm to run applications on the cluster** 1/2 -- //Marcus Boden// {{ :teaching:summer_term_2022:pchpc_slurm_slides.pdf |Slides}} * ~ 17:00 Break * 17:05 - 18:00 **Using Slurm to run applications on the cluster** 2/2 === Tuesday 26.04.2022 === * 09:00 - 10:00 **Debugging with GDB + Valgrind** -- //Jack Ogaja// {{ :teaching:summer_term_2022:pchpc_debug_gdb_valgrind_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_debug_gdb_valgrind_tutorial.pdf |Tutorial}} {{ :teaching:summer_term_2022:pchpc_matrix_init_cn.c |Code1}} {{ :teaching:summer_term_2022:pchpc_matrix_init_rw.c |Code2}} {{ :teaching:summer_term_2022:pchpc_debug_gdb_valgrind_exercise_solution.md |Solution}} * 10:00 - 11:00 **Basic Principles of Parallel Computing** -- //Oswald Haan// {{ :teaching:summer_term_2022:pchpc_parallel_basics_slides.pdf |Slides}} * ~11:00 Break * 11:05 - 11:30 **MPI: General Introduction** {{ :teaching:summer_term_2022:pchpc_mpi_intro_slides.pdf |Slides}} * 11:35 - 12:00 **Exercise: Compiling and Running MPI-Programs** 1/2 {{ :teaching:summer_term_2022:pchpc_mpi_intro_exercise.pdf |Exercise}} * 12:00 - 12:45 Lunch Break * 12:45 - 13:45 **Exercise: Compiling and Running MPI-Programs** 2/2 * 13:45 - 14:45 **MPI: Point to Point Communication** {{ :teaching:summer_term_2022:pchpc_mpi_p2p_slides.pdf |Slides}} * ~14:45 Break * 14:45 - 16:00 **Exercise: Point to Point** {{ :teaching:summer_term_2022:pchpc_mpi_p2p_exercise.pdf |Exercise}} * ~16:00 Break * 16:05 - 16:30 **Parallel Application : Calculation of pi** {{ :teaching:summer_term_2022:pchpc_mpi_applications_slides.pdf |Slides}} * 16:30 - 17:00 **Exercise** * 17:00 - 18:00 **MPI: Collective Communication** {{ :teaching:summer_term_2022:pchpc_mpi_collective_slides.pdf |Slides}} === Wednesday 27.04.2022 === * 09:00 - 10:00 **Exercise: Collectives** -- //Oswald Haan// {{ :teaching:summer_term_2022:pchpc_mpi_collective_exercise.pdf |Exercise}} * 10:00 - 11:00 **Parallel Application: Matrix-Vector Multiplication** {{ :teaching:summer_term_2022:pchpc_mpi_applications_slides.pdf |Slides}} * ~11:00 Break * 11:05 - 11:35 **Exercise** * 11:35 - 12:30 **Parallel Application: Heat Equation** 1/2 {{ :teaching:summer_term_2022:pchpc_mpi_applications_slides.pdf |Slides}} * 12:00 - 12:45 Lunch Break * 12:45 - 13:30 **Parallel Application: Heat Equation** 2/2 * 13:30 - 14:00 **Exercise** * ~14:00 Break * 14:05 - 16:00 **Python single-node Parallelization** -- //Hendrik Nolte// {{ :teaching:summer_term_2022:pchpc_python_local_parallelization_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_python_local_parallelization_tutorial.pdf |Tutorial}} * ~16:00 Break * 16:05 - 18:00 **Python multi-node Parallelization (MPI)** {{ :teaching:summer_term_2022:pchpc_python_mpi_parallelization_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_python_mpi_parallelization_tutorial.pdf |Tutorial}} {{ :teaching:summer_term_2022:pchpc_python_parallelization_solutions.zip |Solutions}} === Thursday 28.04.2022 === * 9:00 - 11:00 **Programming with pthreads** -- //Vanessa End// {{ :teaching:summer_term_2022:pchpc_pthreads_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_pthreads_exercise.pdf |Exercise}} {{ :teaching:summer_term_2022:pchpc_pthreads_codes.zip |Code}} * ~ 11:00 Break * 11:05 - 12:00 **OpenMP** 1/3 -- //Sven Bingert// {{ :teaching:summer_term_2022:pchpc_openmp_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_openmp_exercise_1.pdf |Exercise}} {{ :teaching:summer_term_2022:pchpc_openmp_code_ex_1.zip |Code}} * 12:00 - 12:45 Lunch Break * 12:45 - 15:00 **OpenMP** 2/3 {{ :teaching:summer_term_2022:pchpc_openmp_exercise_2.pdf |Exercise}} {{ :teaching:summer_term_2022:pchpc_openmp_code_ex_2.zip |Code}} * ~15:00 Break * 15:05 - 16:00 **OpenMP** 3/3 * ~16:00 Break * 16:05 - 18:00 **Quantum Computing - A brief introduction for the curious** -- //Christian Boehme// {{ :teaching:summer_term_2022:pchpc_quantum_computing_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_quantum_computing_code.zip |Code}} {{ :teaching:summer_term_2022:pchpc_qiskit.sh |Install Script}} === Friday 29.04.2022 === * 9:00 - 10:00 **Introduction to benchmarking and performance engineering** -- //Julian Kunkel// {{ :teaching:summer_term_2022:pchpc_bench_perf_engineering.pdf |Slides}} * 10:00 - 11:00 **Node Level Performance Analysis (NLPA): LIKWID performance tool-suite** 1/2 -- //Jack Ogaja// {{ :teaching:summer_term_2022:pchpc_likwid_slides.pdf |Slides}} {{ :teaching:summer_term_2022:pchpc_likwid_tutorial.pdf |Tutorial}} {{ :teaching:summer_term_2022:pchpc_nlpe_with_likwid_code.zip |Code}} * ~11:00 Break * 11:05 - 11:35 **Node Level Performance Analysis (NLPA): LIKWID performance tool-suite** 2/2 * 11:35 - 12:00 **NLPA Exercises: Using LIKWID Wrapper, Process Affinity & Performance Counters tools** 1/2 * 12:00 - 12:45 Lunch Break * 12:45 - 13:15 **NLPA Exercises: Using LIKWID Wrapper, Process Affinity & Performance Counters tools** 2/2 * 13:15 - 15:00 **Performance Analysis with Vampir** 1/3 * ~15:00 Break * 15:05 - 17:00 **Performance Analysis with Vampir** 2/3 * ~17:00 Break * 17:05 - 17:30 **Performance Analysis with Vampir** 3/3 * 17:30 - 18:00 **Description of the group assignment** -- //Julian Kunkel// {{ :teaching:summer_term_2022:pchpc_student_assignment.pdf |Assignment}} ===== Registration ===== The deadline for registration is April 10th. Students, please register using StudIP. GWDG Academy participants, please register there. ===== Relevant for Students to Obtain the Credits ===== This remaining part is mandatory for BSc and MSc students to obtain the credits but can be skipped if you just want to join the block course. In order to obtain the credits, you will parallelize a non-trivial problem of your choice using the concepts and tools learned during the block course. As an alternative option, you could choose from an administrative topic that we derive from a practical problem at the GWDG. Firstly, you will decide upon a problem you like to solve, then you will create a sequential solution to this problem, and lastly, you apply the experience of the block course to parallelize your application and analyze its scalability. You need to prepare a presentation for your fellow students as well as document your solution in a report. Both the presentation and the report are due at the end of the term and will be assessed and marked. ===== Examination ===== To obtain the credits, students must develop their (own) group project. A presentation of 15 min (per group member) and report (max 15 pages per group member) must be created. The mark consists of 30% presentation and 70% report. The report must be submitted as a PDF file. The source code must be handed in as well. Optimally as a Gitlab or Github repository. ==== Project Meetings ==== //This attendance of the virtual project meetings is mandatory for BSc and MSc students to obtain the credits.// There will be three project meetings in which the groups introduce their results. These meetings are not marked but provide valuable feedback toward your project. In the first seminar, the project idea and rough implementation plan are shared. The second seminar will introduce a report and gives you the opportunity to ask questions regarding the preparation of the report. In the final meeting, the project results are shared. Subsequent meetings are held in our [[https://meet.gwdg.de/b/jul-tff-dq1-rt0|BBB Room]]. === Dates === * End of May - Submit the project title to Julian and group composition - we will then assign a supervisor to you * 2022-06-21 17:15-18:00 - **Project introduction**: Students present their project idea and project plan (5 (+5) min per group) * 2022-07-12 17:15-18:00 - **Q&A session for the assignment** -- Jonathan Decker, Julian Kunkel * Report and presentation: requirements and assessment criteria * Suggested structure * Q&A opportunity * 2022-09-13 14:15-18:00 - **Result presentation** Students present their project results (15 min per **student**) * 2022-09-31 Last day of term - Submission of your report to the supervisor ===== Example Projects ===== Projects should cover either the creation and parallelization of an interesting problem such as: * Determine optimal moves (search tree method) for games. * (Simple) predator-prey relationship of a closed system with animal migration. * Cars in the traffic of a city network and resulting traffic jams. * Astrophysical computations. * Skat, Go or robot simulation * Longest path problem * Solving large logical formulas * Algorithms from bioinformatics * Strategies for placing airplane passengers Most important is the correct parallelization (possibly with alternatives) and evaluation. Detailed knowledge of numerics or a precise representation of the model is not required. Alternatively, we provide a list of specific topics that are motivated by our needs in the data center, these may cover administrative aspects as well. We are open to further suggestions. Examples from a similar course with previous works can be found [[https://hps.vi4io.org/teaching/hamburg/sommersemester_2015/parallele_programmierung#ergebnisse|here]]. Have a look at the results and videos generated. ===== Content of the Project ===== For a generic parallelization project, some contents should be worked on and accordingly included in the presentation and elaboration, the aspects are: * Concepts of the underlying (application) model. * Parallelization scheme (communication pattern, distribution of data & tasks). * Parallelization should be done with MPI (optional: shared memory parallelization with threads or OpenMP). * Performance analysis of the sequential code (does it behave as expected). * Scaling behavior of the parallel version * Speedup diagrams * Potential analysis with Vampire/Sunshot * Performing an optimization of the parallel version (communication scheme etc.) ===== Project Topics ===== This is a list of topics for the individual projects that are motivated by our needs in the data center. We welcome it if you propose your own simulation, parallelization, or administrative topic. Even for our suggested topics, you will have some room for developing the topic in the direction of your choice. ==== ZombieSim ==== * Niklas Bauer * Abdullah Amawi * Maaike Bierenbroodspot Supervisor: Julian Kunkel Presentation: [[https://youtu.be/4mJRSwNK1ew|Youtube Link]]\\ Report: {{ :teaching:summer_term_2022:pchpc_report_zombiesim_by_abdullah_amawi_maaike_bierenbroodspot.pdf |Download}}\\ Slides Abdullah: {{ :teaching:summer_term_2022:pchpc_presentation_zombiesim_by_abdullah_amawi.pdf |Download}}\\ Slides Maaike: {{ :teaching:summer_term_2022:pchpc_presentation_zombiesim_by_maaike_bierenbroodspot.pdf |Download}}\\ Repo: [[https://github.com/amawi/ZombieSim|Github]]\\ Code: {{ :teaching:summer_term_2022:pchpc_code_zombiesim.zip |Download}} ==== Kullback-Leibler-Divergenz in der Bioinformatik ==== * Tim van den Berg * Vincenz Dumann Supervisor: Marcus Boden Presentation: [[https://youtu.be/T_kKnIUKgaA|Youtube Link]]\\ Report: {{ :teaching:summer_term_2022:pchpc_report_kullback_leibler_by_vincenz_dumann_tim_van_den_berg.pdf |Download}}\\ Slides: {{ :teaching:summer_term_2022:pchpc_presentation_kullback_leibler_by_vincenz_dumann_tim_van_den_berg.pdf |Download}}\\ Repo: [[https://gitlab-ce.gwdg.de/tim.vandenberg/practical_hpc|GWDG Gitlab]]\\ Code: {{ :teaching:summer_term_2022:pchpc_code_practical_hpc.zip |Download}} ==== Simulation der Laplace Gleichung ==== * Philipp Müller * Jakob Hördt Supervisor: Jonathan Decker Presentation: [[https://youtu.be/zlNUa7aB7rw|Youtube Link]]\\ Report: {{ :teaching:summer_term_2022:pchpc_report_laplace_gleichung_by_jakob_hoerdt_philipp_mueller.pdf |Download}}\\ Slides: {{ :teaching:summer_term_2022:pchpc_presentation_laplace_gleichung_by_jakob_hoerdt_philipp_mueller.pdf |Download}}\\ Repo: [[https://gitlab.gwdg.de/j.hoerdt/parallaplacation/|GWDG Gitlab]]\\ Code: {{ :teaching:summer_term_2022:pchpc_code_parallaplacation.zip |Download}} ==== Solar system and the dynamics of this N-Body system ==== * Aaron Nagel * Yannik Feldner Supervisor: Jack Ogaja Presentation: [[https://youtu.be/bI8rlLmhohs|Youtube Link]]\\ Report: {{ :teaching:summer_term_2022:pchpc_report_n-body_system_with_swing_by_aaron_nagel_yannik_feldner.pdf |Download}}\\ Slides: {{ :teaching:summer_term_2022:pchpc_presentation_n-body_system_with_swing_by_feldner_and_nagel.pdf |Download}}\\ Repo: [[https://gitlab-ce.gwdg.de/hpc-feldner-nagel/n-body-system|GWDG Gitlab]]\\ Code: {{ :teaching:summer_term_2022:pchpc_code_n-body-system.zip |Download}}\\ Video: {{ :teaching:summer_term_2022:pchpc_report_video_n-body_system_with_swing_by_aaron_nagel_yannik_feldner.zip |Download}} ==== Parallel Deep Learning pipelines using Go and MPI ==== * Silin Zhao Supervisor: Patrick Michaelis Presentation: [[https://youtu.be/2siZQBvRPuY|Youtube Link]]\\ Report: {{ :teaching:summer_term_2022:pchpc_report_go_deep_learning_mpi_by_silin_zhao.pdf |Download}}\\ Slides: {{ :teaching:summer_term_2022:pchpc_presentation_go_deep_learning_mpi_by_silin_zhao.pdf |Download}}\\ Repo: [[https://github.com/scofild429/go_mpi_network|Github]]\\ Code: {{ :teaching:summer_term_2022:pchpc_code_go_mpi_network.zip |Download}}