BoF: Analyzing Parallel I/O

Parallel I/O performance can be a critical bottleneck for applications, yet users are often ill-equipped for identifying and diagnosing I/O performance issues. Increasingly complex hierarchies of storage hardware and software deployed on many systems only compound this problem. Tools that can effectively capture, analyze, and tune I/O behavior for these systems empower users to realize performance gains for many applications.

In this BoF, we form a community around best practices in analyzing parallel I/O and cover recent advances to help address the problem presented above, drawing on the expertise of users, I/O researchers, and administrators in attendance.

The primary objectives of this BoF are to: 1) highlight recent advances in tools and techniques for monitoring I/O activity in data centers, 2) to discuss experiences and limitations of current approaches, 3) to discuss and derive a roadmap for future I/O tools with the goal to capture, assess, predict and optimize I/O.

The BoF is held in conjunction with the Supercomputing conference. The official announcement is listed here.

The BoF is powered by the Virtual Institute for I/O and DECICE ¹⁾.

The BoF is organized by

• Shane Snyder (ANL, USA), ssnyder@mcs.anl.gov
• Hendrik Nolte (GWDG, Germany), hendrik.nolte@gwdg.de
• Julian Kunkel (Georg-August-Universität Göttingen/GWDG), julian.kunkel@gwdg.de

We have a series of (8 minute) talks followed by a longer discussion.

• Welcome – Shane Snyder
  Slides
• The network testing mode in elbencho v3.0 – Sven Breuner (VAST)
  Slides
• The HPC IOAnalysis Repository – Andre Brinkman (Uni Mainz)
  Slides
  HPC application developers and administrators need to understand the complex interplay between compute clusters and storage systems to make effective optimization decisions. Ad hoc investigations of this interplay based on isolated case studies can lead to conclusions that are incorrect or difficult to generalize. The I/O Trace Initiative aims to improve the scientific community's understanding of I/O operations by building a searchable collaborative archive of I/O traces from a wide range of applications and machines, with a focus on high-performance computing and scalable AI/ML. This initiative advances the accessibility of I/O trace data by enabling users to locate and compare traces based on user-specified criteria. It also provides a visual analytics platform for in-depth analysis, paving the way for the development of advanced performance optimization techniques. By acting as a hub for trace data, the initiative fosters collaborative research by encouraging data sharing and collective learning.
• MangoIO – Radita Liem (RWTH)
  Slides
• Understanding Storage Performance using Benchmarking - Experiences at GWDG – Hendrik Nolte (GWDG)
  Slides