Due to COVID-19, the workshop will be organized as a free virtual event using video conferencing, the videos of the presentations will be published on this page.

Managing scientific data at a large scale is challenging for both scientists and the host data centre.

The storage and file systems deployed within a data centre are expected to meet users' requirements for data integrity and high performance across heterogeneous and concurrently running applications.

With new storage technologies and layers in the memory hierarchy, the picture is becoming even murkier. To effectively manage the data load within a data centre, I/O experts must understand how users expect to use the storage and what services they should provide to enhance user productivity.

In this workshop, we bring together I/O experts from data centres and application workflows to share current practices for scientific workflows, issues, and obstacles for both hardware and the software stack, and R&D to overcome these issues. We seek to ensure that a systems-level perspective is included in these discussions.

The workshop content is built on two tracks with calls for papers/talks:

• Research paper track – Requesting submissions regarding state-of-the-practice and research about I/O in the data centre (see our topic list).
• Talks from I/O experts – Requesting submissions of talks.

We are excited to announce that research papers will be published in Springer LNCS open access and extended manuscripts in the Journal of High-Performance Storage as well. Contributions to both tracks are peer-reviewed and require submission of the respective research paper or idea for your presentation via Easychair (see the complete description in Track: Research Papers).

The workshop is held in conjunction with the ISC-HPC during the ISC workshop day. Note that the attendance to ISC workshops requires a workshop pass. See also our last year's workshop web page.

This workshop is powered by the Virtual Institute for I/O, the Journal of High-Performance Storage, ESiWACE ¹⁾.

The workshop is organised by

• Julian Kunkel (Department of Computer Science, University of Reading, UK), j.m.kunkel@reading.ac.uk
• Jay Lofstead (Sandia National Lab, USA), gflofst@sandia.gov
• Jean-Thomas Acquaviva (DDN, France), jtacquaviva@ddn.com

The videos are available in YouTube. Please see our workshop summary paper.

Times are listed in BST (GMT+1), CEST is +1 hour, -6 hours for US Central (CDT)

• 9:45 Welcome to the HPC IODC workshop – Julian Kunkel, Jay Lofstead, Jean-Thomas Acquaviva
  Slides – Video
  This talk provides an introduction to the HPC IODC workshop providing the motivation and wider scope behind the workshop.
• 10:00 Research paper session – chair Jean-Thomas Acquaviva
  • Characterizing I/O Optimization Effect Through Holistic Log Data Analysis of Parallel File Systems and Interconnects – Yuichi Tsujita (RIKEN)
    Slides – Video
    Recent HPC systems utilize parallel file systems such as GPFS and Lustre to ope with the huge demand of data-intensive applications. Although most of the HPC systems provide performance tuning tools on compute nodes, there is not enough chance to tune I/O activities on parallel file systems including high speed interconnects among compute nodes and file systems. We propose an I/O performance optimization framework using log data of parallel file systems and interconnects in a holistic way for effective use of HPC systems including I/O nodes and parallel file systems. We demonstrate our framework at the K computer with two I/O benchmarks for the original and the enhanced MPI-IO implementations. Its I/O analysis has revealed that I/O performance improvements achieved by the enhanced MPI-IO implementation are due to effective utilization of parallel file systems and interconnects among I/O nodes compared with the original MPI-IO implementation.
  • Investigating the Overhead of the REST Protocol to Reveal the Potential for Using Cloud Services for HPC Storage – Frank Gadban (University of Hamburg)
    Slides – Video
    In this paper, we investigate the overhead of the REST protocol via HTTP compared to the HPC-native communication protocol MPI when storing and retrieving objects. Albeit we compare the MPI for a communication use case, we can still evaluate the impact of data communication and, therewith, the efficiency of data transfer for data access patterns. We accomplish this by modeling the impact of data transfer using measurable performance metrics. Hence, our contribution is the creation of a performance model based on hardware counters that provide an analytical representation of data transfer over current and future protocols. We validate this model by comparing the results obtained for REST and MPI on two different cluster systems, one equipped with Infiniband and one with Gigabit Ethernet. The evaluation shows that REST can be a viable, performant and resource-efficient solution, in particular for accessing large files.
  • Classifying Temporal Characteristics of Job I/O Patterns Using Machine Learning Techniques – Eugen Betke (DKRZ)
    Slides – Video
    Every day, supercomputers execute 1000s of jobs with different characteristics. Data centers monitor the behavior of jobs to support the users and improve the infrastructure, for instance, by optimizing jobs or by determining guidelines for the next procurement. The classification of jobs into groups that express similar run-time behavior aids this analysis as it reduces the number of representative jobs to look into.
    This work utilizes machine learning techniques to cluster and classify parallel jobs based on the similarity in their temporal I/O behavior. Our contribution is the qualitative and quantitative evaluation of different I/O characterizations and similarity measurements and the development of a suitable clustering algorithm.
    In the evaluation, we explore I/O characteristics from monitoring data of one million parallel jobs and cluster them into groups of similar jobs. Therefore, the time series of various IO statistics is converted into features using different similarity metrics that customize the classification. When using general-purpose clustering techniques, suboptimal results are obtained. Additionally, we extract phases of IO activity from jobs. Finally, we simplify the grouping algorithm in favor of performance. We discuss the impact of these changes on the clustering quality.
• 11:30 Research talks – chair Julian Kunkel
  • A Reinforcement Learning Strategy to Tune Request Scheduling at the I/O Forwarding Layer
    Slides – Video – Jean Luca Bez, Francieli Zanon Boito, Ramon Nou, Alberto Miranda, Toni Cortes, Philippe O. A. Navaux
    I/O optimization techniques can improve performance for the access patterns they were designed to target, but they often decrease for others. Moreover, these techniques usually depend on the precise tune of their parameters, which commonly falls back to the users. We propose an approach to tune parameters dynamically at runtime based on the I/O workload observed by the system. Our focusing is on the I/O forwarding layer as it is transparent to applications and file system independent. Our approach uses a reinforcement learning technique to make the system capable of learning the best parameter value to each observed access pattern during its execution, eliminating the need for a complex and time-consuming training phase. We evaluate our proposal for the TWINS scheduling algorithm designed for the I/O forwarding layer seeking to reduce contention and coordinate accesses to the data servers. We demonstrate our approach can reach a precision of 88% on the parameter selection in the first hundreds of observations of an access pattern, achieving 99% of the optimal performance.
  • Data Systems at Scale in Climate and Weather: Activities in the ESiWACE Project – Julian Kunkel (University of Reading)
    Slides – Video
    The ESiWACE project aims to enable global eddy-resolving weather and climate simulations on the upcoming (pre-)Exascale supercomputers. In this talk, a selection of efforts to mitigate the effects of the data deluge from such high-resolution simulations is introduced. In particular, we describe the advances in the Earth System Data Middleware (ESDM), which enables scalable data management and supports the inhomogeneous storage stack. ESDM which provides a NetCDF compatible layer at a high-performance and portable-portable fashion. A selection of performance results is given and ongoing efforts for workflow support and active storage are discussed.
  • Phobos a scale-out object store implementing tape library support – Patrice Lucas (CEA), Philippe Deniel (CEA), Thomas Leibovici(CEA)
    Slides – Video
    Phobos is an open source scale-out distributed object store providing access to multiple backends from flash and hard drives to tape libraries. Very large datasets can be efficiently managed on inexpensive storage media without giving up performance, scalability or fault-tolerance. Phobos is designed to offer several data layouts, such as mirroring or erasure coding. IOs through tape drives are optimized by dedicated resource scheduling policies. Developed at CEA, Phobos is in production since 2016 to manage the France Genomique multi-petabyte dataset at TGCC.

• 13:00 Virtual Lunch break
• 14:00 Expert talks – chair Jean-Thomas Acquaviva
  • The ALICE data management pipeline – Massimo Lamanna (CERN)
    Slides – Video
    ALICE is a major experiment at the CERN LHC with more than 1500 physicists, engineers and technicians, including around 350 graduate students, from 154 physics institutes in 37 countries across the world. ALICE primarily focuses on the study of high-energy nucleus-nucleus collisions. This allows the physicists to study strongly interacting matter at the highest energy densities reached so far in the laboratory with the goal to understand mechanisms and phenomena in particle physics and astrophysics.
    After 7 years of data taking, the experiment is currently being upgraded with new detectors providing novel computing challenges notably in high data rates, processing and storage needs. In this talk, I will describe the computing challenges we need to solve in order to fully benefit from the performance of the new detectors. I will discuss and present the overall design of the ALICE computing farm O^2. Particular emphasis will be given to the technical choices in setting up a 60-PB disk farm to sustain rates of the order of 100 GB/s to the mass storage during the online data-processing.
  • Accelerating your Application I/O with UnifyFS – Kathryn Mohror (Lawrence Livermore National Laboratory)
    Slides – Video
    UnifyFS is a user-level file system that is highly-specialized for fast shared file access on high performance computing (HPC) systems with distributed burst buffers. UnifyFS delivers significant performance improvements over general purpose file systems by supporting the specific needs of HPC workloads with reduced POSIX semantics support called “lamination semantics.” In this talk, we will give an introductory overview of how to use the lightweight UnifyFS file system to improve the I/O performance of HPC applications. We will describe how UnifyFS works with burst buffers, the benefits and limitations of lamination semantics, and how users can incorporate UnifyFS into their jobs. Finally, we will detail the current implementation status of UnifyFS and our plans for the future.
  • How to recognise I/O bottlenecks and what to do about them – Rosemary Francis (Ellexus)
    Slides – Video
    Dr Rosemary Francis is CEO and technical founder of Ellexus, the I/O profiling company. Rosemary will be sharing industry perspectives on how to recognise I/O bottlenecks and what to do about them. The delicate and often dynamic balance between I/O, CPU and memory can hide some easy wins in terms of improving throughput on-prem and reducing costs in the cloud. Equally, improving I/O is also about reducing the load on shared storage and not just about the incremental improvements of individual applications.
• 15:30 Discussion of hot topics – chair Julian Kunkel
• 16:00 Expert talks – chair Jay Lofstead
  • Managing Decades of Scientific Data in Practice at NERSC – Glenn Lockwood (NERSC)
    Slides – Video
    The National Energy Research Scientific Computing Center (NERSC) has been operating since 1974 and has been storing and preserving user data continuously for over 45 years as a result. This has resulted in NERSC building significant expertise in how to store and manage user data for long periods of time–a decade or more–and the practical factors that must be considered when data must be retained for longer than the lifetime of the physical components of the data center, including the entire data center facility itself. As the relevance of HPC extends beyond modeling and simulation and the usable lifetime of data extends from months to years or decades, these best practices in long-term data stewardship are likely to become more important to more HPC facilities. To this end, we present here some of the practical considerations, best practices, and lessons learned from managing the scientific data of NERSC's thousands of users over a period of four decades.
  • Portable Validations of Scientific Explorations with Container-native Workflows – Ivo Jimenez (UC Santa Cruz)
    Slides – Video
    Researchers working in computer, computational or data science often find it difficult to reproduce experiments from artifacts like code, data, diagrams and results which are left behind by previous researchers. The code developed on one machine often fails to run on other machines due to differences in hardware architecture, OS, software dependencies, among others. This is accompanied by the difficulty in understanding how artifacts are organized, as well as in using them in correct order. Software container technology such as Docker, can solve most of the practical issues of portability, and in particular, container-native workflow engines can significantly aid experimenters in their work. In this talk, we introduce Popper, a container-native workflow engine that executes each step of a workflow in a separate dedicated container without assuming the presence of a Kubernetes cluster or any cloud based Kubernetes service. With Popper, researchers can build and validate workflows easily in almost any environment of their choice including local machines, SLURM based HPC clusters, CI services or Kubernetes based cloud computing environments. To exemplify the suitability of this workflow engine, we present three case studies where we take examples from Machine Learning and High Performance Computing and turn them into Popper workflows. We also discuss how Popper can be used to aid in preparing artifacts associated with article submissions to conferences and journals, and in particular give an overview of the Journal of High-Performance Storage, a new eJournal that combines open reviews, living papers, digital reproducibility, and open access.
  • Tuning I/O Performance on Summit: HDF5 Write Use Case Study – Xie Bing (Oak Ridge National Laboratory)
    Slides – Video
    The HDF5 I/O library is widely used in HPC across a variety of domain sciences for its simplicity, flexibility, and rich performance-tuning space. In this work, we address an observed HDF5 write performance issue on Summit at OLCF, which in particular is the poor write performance of HDF5 with the default configuration. To identify the performance issue, we developed an I/O benchmarking methodology to profile the HDF5 performance on Summit across scales, compute-node allocations, I/O configurations and times. We developed a solution to the issue by altering the HDF5 alignment configuration which resulted in a 100x write performance improvement for VPIC benchmark. We expect our methodology and solution to be applicable to other platforms and technologies.
• 17:30 Discussion of hot topics – chair Jay Lofstead
• 18:00 End

We will provide the link for the video conference to registered attendees. Fill the linked form, to register for the workshop.

• Thomas Boenisch (High-performance Computing Center Stuttgart)
• Suren Byna (Lawrence Berkeley National Laboratory)
• Matthew Curry (Sandia National Laboratories)
• Sandro Fiore (CMCC)
• Wolfgang Frings (Juelich Supercomputing Centre)
• Javier Garcia Blas (Carlos III University)
• Adrian Jackson (The University of Edinburgh)
• Ivo Jimenez (University of California, Santa Cruz)
• Anthony Kougkas (Illinois Institute of Technology)
• Glenn Lockwood (Lawrence Berkeley National Laboratory)
• Jay Lofstead (Sandia National Laboratories)
• Carlos Maltzahn (University of California, Santa Cruz)
• Suzanne McIntosh (New York University)
• Maria Perez (Technical University of Madrid)
• Robert Ross (Argonne National Laboratory)
• George S. Markomanolis (Oak Ridge National Laboratory)
• Feiyi Wang (Oak Ridge National Laboratory)
• Bing Xie (Oak Ridge National Lab)

The workshop is integrated into ISC-HPC. We welcome everybody to join the workshop, including:

• I/O experts from data centres and industry.
• Researchers/Engineers working on high-performance I/O for data centres.
• Domain scientists and computer scientists interested in discussing I/O issues.
• Vendors are also welcome, but their presentations must align with data centre topics (e.g. how do they manage their own clusters) and not focus on commercial aspects.

The call for papers and talks is already open. We accept early submissions and typically proceed with them within 45 days. We particularly encourage early submission of abstracts such that you indicate your interest in submissions.

You may be interested in joining our mailing lists at the Virtual Institute for I/O.

We especially welcome participants that are willing to give a presentation about the I/O of the representing institutions' data centre. Note that such presentations should cover the topics mentioned below.

CFP text

The research track accepts papers covering state-of-the-practice and research dedicated to storage in the data centre.

Proceedings will appear in ISC's post-conference workshop proceedings in Springers LNCS. Extended versions have a chance for acceptance in the first issue of the JHPS journal. We will apply the more restrictive review criteria from JHPS and use the open workflow of the JHPS journal for managing the proceedings. For interaction, we will rely on Easychair, so please submit the metadata to EasyChair before the deadline.

For the workshop, we accept papers with up to 12 pages (excluding references) in LNCS format. You may already submit an extended version suitable for the JHPS in JHPS format. Upon submission, please indicate potential sections for the extended version (setting a light red background colour). The JHPS template can be easily converted to the LNCS Word format such that the effort is minimal for the authors to obtain both publications. Alternatively, you can use the Springer LNCS LaTeX or Word template and convert it to a Google Doc. See the Manuscript Preparation, Layout & Templates, Springer.

For accepted papers, the length of the talk during the workshop depends on the controversiality and novelty of the approach (the length is decided based on the preference provided by the authors and feedback from the reviewers). We also allow virtual participation (without attending the workshop personally). All relevant work in the area of data centre storage will be published with our joint workshop proceedings. We just believe the available time should be used best to discuss controversial topics.

The relevant topics for papers cover all aspects of data centre I/O, including:

• Application workflows
• User productivity and costs
• Performance monitoring
• Dealing with heterogeneous storage
• Data management aspects
• Archiving and long term data management
• State-of-the-practice (e.g., using or optimising a storage system for data centre workloads)
• Research that tackles data centre I/O challenges

• 2020-02-24: Submission deadline: AoE ²⁾
  • Note: The call for papers and talks is already open.
  • We appreciate early submissions of abstracts and full papers and review them within 45 days.
• 2020-04-15: Extended Submission deadline: AoE ³⁾ due to the Coronavirus
  • Please submit abstracts asap.
• 2020-05-03: Author notification
• 2020-05-10: Camera-ready papers for JHPS (It depends on the author's ability to incorporate feedback into their submission in the incubator.)
• 2020-06-10: Pre-final submission for ISC (Papers to be shared during the workshop. We will also use the JHPS papers, if available.)
• 2020-06-25: Workshop
• 2020-07-24: Camera-ready papers for ISC ⁴⁾ – As they are needed for ISC's post-conference workshop proceedings. We embrace the opportunity for authors to improve their papers based on the feedback received during the workshop.

The main acceptance criterion is the relevance of the approach to be presented, i.e., the core idea is novel and worthwhile to be discussed in the community. Considering that the camera-ready version of the papers is due after the workshop, we pursue two rounds of reviews:

1. Acceptance for the workshop (as a talk).
2. Acceptance as a paper *after* the workshop, incorporating feedback from the workshop.

After the first review, all papers undergo a shepherding process.

The criteria for The Journal of High-Performance Storage are described on its webpage.

The topics of interest in this track include, but are not limited to:

• A description of the operational aspects of your data centre
• A particular solution for specific data centre workloads in production

We also accept industry talks, given that they are focused on operational issues on data centres and omit marketing.

We use Easychair for managing the interaction with the program committee. If you are interested in participating, please submit a short (1/2 page) intended abstract of your talk together with a brief Bio.

• Submission deadline: 2020-04-10 AoE
• Author notification: 2020-05-03

The following list of items should be tried to be integrated into a talk covering your data centre, if possible. We hope your site's administrator will support you to gather the information with little effort.

1. Workload characterisation
   1. Scientific Workflow (give a short introduction)
      1. A typical use-case (if multiple are known, feel free to present more)
      2. Involved number of files/amount of data
   2. Job mix
      1. Node utilisation (related to peak-performance)
2. System view
   1. Architecture
      1. Schema of the client/server infrastructure
         1. Capacities (Tape, Disk, etc.)
      2. Potential peak-performance of the storage
         1. Theoretical
         2. Optional: Performance results of acceptance tests.
      3. Software/Middleware used, e.g. NetCDF 4.X, HDF5, …
   2. Monitoring infrastructure
      1. Tools and systems used to gather and analyse utilisation
   3. Actual observed performance in production
      1. Throughput graphs of the storage (e.g., from Ganglia)
      2. Metadata throughput (Ops/s)
   4. Files on the storage
      1. Number of files (if possible, per file type)
      2. Distribution of file sizes
3. Issues/Obstacles
   1. Hardware
   2. Software
   3. Pain points (what is seen as the most significant problem(s) and suggested solutions, if known)
4. Conducted R&D (that aim to mitigate issues)
   1. Future perspective
   2. Known or projected future workload characterisation
   3. Scheduled hardware upgrades and new capabilities we should focus on exploiting as a community
   4. Ideal system characteristics and how it addresses current problems or challenges
   5. What hardware should be added
   6. What software should be developed to make things work better (capabilities perspective)
   7. Items requiring discussion