BoF: Analyzing Parallel I/O

Parallel application I/O performance often does not meet user expectations. Additionally, slight access pattern modifications may lead to significant changes in performance due to complex interactions between hardware and software. These challenges call for sophisticated tools to capture, analyze, understand, and tune application I/O.

In this BoF, we will highlight recent advances in monitoring tools to help address this problem. We will also encourage community discussion to compare best practices, identify gaps in measurement and analysis, and find ways to translate parallel I/O analysis into actionable outcomes for users, facility operators, and researchers.

The BoF is held in conjunction with the Supercomputing conference. The schedule is listed here.

The BoF is organized by

• Julian Kunkel (DKRZ, Germany), kunkel@dkrz.de
• Phil Carns (ANL, USA), carns@mcs.anl.gov

• Introduction and agenda – Julian Kunkel kunkel@dkrz.de
  • Slides
• SIOX: A flexible approach – Julian Kunkel kunkel@dkrz.de
  • Slides – SIOX src
• Darshan: state of the project and new features – Phil Carns carns@mcs.anl.gov and Shane Snyder ssnyder@mcs.anl.gov
  • Slides – Project Home
• The Dashboard: HPC I/O Analysis Made Easy – Huong Luu luu1@illinois.edu
  • Slides – DarshanSQL src – DarshanVis src
• How predictable are HPC application, and why should we care? – Matthieu Dorier mdorier@anl.gov
  • Slides
• Parallel I/O Monitoring at JSC – Wolfgang Frings w.frings@fz-juelich.de
  • Slides– Project Home
• Tools and techniques towards a holistic understanding of I/O demands at NERSC – Glenn Lockwood glock@lbl.gov
  • Slides
• Monitoring plans @ DKRZ and discussion about monitoring standards – Julian Kunkel kunkel@dkrz.de
  • Slides
• Group discussion and panel (15 min)

• Julian Kunkel Julian is responsible for several projects in the research division at DKRZ. He has been working on tracing environments and tools for client and server-side I/O since many years.
• Phil Carns Phil is a principal software development specialist in the Mathematics and Computer Science division of ANL. He is the technical lead of the Darshan project and a key contributor to a variety of related storage simulation and prototyping activities.
• Shane Snyder Shane is also a software developer in MCS. His research interests include parallel file systems, I/O middleware systems, and HPC I/O workload characterization.
• Huong Luu Huong is a postdoctoral scholar in the computer science department at the University of Illinois at Urbana – Champaign. She also analyzes and improves application I/O behavior at the NCSA as part of the Petascale Application Improvement Discovery (PAID) program. Her research interests include analysis and automated tuning of HPC I/O.
• Matthieu Dorier Matthieu is a postdoctoral appointee at Argonne National Laboratory. His research interests include efficient I/O, data management and storage for high-performance simulations, data-intensive computing, data analytics and visualization. He has published several research papers in top HPC conferences, including SC, IPDPS and Cluster.
• Wolfgang Frings is a research scientist at Jülich Supercomputing Centre of Forschungszentrum Jülich. His research interest focuses on parallel I/O, I/O middleware, and HPC system monitoring. He is author of several software tools used at many HPC centers. Among these are SIONlib, a library to support task-local parallel file I/O on large-scale systems and LLview, a batch system monitoring software.
• Glenn Lockwood Glenn is a member of NERSC's Advanced Technologies Group at Lawrence Berkeley National Laboratory who specializes in data-intensive computing. His work focuses on characterizing the I/O demands of the NERSC workload throughout the entire I/O stack, ranging from application to back-end disk and flash, to guide future systems' requirements.