HPC-IODC: HPC I/O in the Data Center Workshop

Managing scientific data at large scale is challenging for scientists but also for the host data center. The storage and file systems deployed within a data center are expected to meet users' requirements for data integrity and high performance across heterogeneous and concurrently running applications.

With new storage technologies and layers in the memory hierarchy, the picture is becoming murkier. To effectively manage the data load within a data center, I/O experts must understand how users expect to use these new storage technologies and what services they should provide in order to enhance user productivity. We seek to ensure a systems-level perspective is included in these discussions.

In this workshop we bring together I/O experts from data centers and application workflows to share current practices for scientific workflows, issues and obstacles for both hardware and the software stack, and R&D to overcome these issues. A common structure of the talks will be provided to focus on relevant aspects and streamline the discussion. Short scientific papers related to the topic are welcome for submission and will be published under open access.

The workshop is held in conjunction with the ISC-HPC. See the Agenda Planer

The workshop is organized by

• Julian Kunkel (DKRZ, Germany), kunkel@dkrz.de
• Jay Lofstead (Sandia National Lab, USA), gflofst@sandia.gov
• Colin McMurtrie (CSCS, Switzerland), cmurtrie@cscs.ch

The following list of items should be tried to be integrated into the talk if possible. We hope your sites admin will support you to gather the information with little effort.

1. Workload characterization
   1. Scientific Workflow (give a short introduction)
      1. A typical use-case (if multiple are known, feel free to present more)
      2. Involved number of files / amount of data
   2. Job mix
      1. Node utilization (rel. to peak-performance)
2. System view
   1. Architecture
      1. Schema of the client/server infrastructure
         1. Capacities (Tape, Disk, etc.)
      2. Potential peak-performance of the storage
         1. Theoretical
         2. Optional: performance results of acceptance tests.
      3. Software / Middleware used, e.g. NetCDF 4.X, HDF5, …
   2. Monitoring infrastructure
      1. Tools and systems used to gather and analyse utilization
   3. Actual observed performance in production
      1. Throughput graphs of the storage (e.g. from Ganglia)
      2. Metadata throughput (Ops/s)
   4. Files on the storage
      1. Number of files (if possible per file type)
      2. Distribution of file sizes
3. Issues / Obstacles
   1. Hardware
   2. Software
   3. Pain points (what is seen as the biggest problem(s) and suggested solutions, if known)
4. Conducted R&D (that aim to mitigate issues)
   1. Future perspective
   2. Known or projected future workload characterization
   3. Scheduled hardware upgrades and new capabilities we should focus on exploiting as a community
   4. Ideal system characteristics and how it addresses current problems or challenges
   5. what hardware should be added
   6. what software should be developed to make things work better (capabilities perspective)
   7. Items requiring discussion to work through how to address

Talks and (short) proceedings will be published on this webpage. Short papers should be no longer than 5 pages (two column) but we are quite flexible. Additionally, we aim to write a journal paper containing the synthesis of the workshop: current practice, obstacles and observations from the data centers, that is co-authored by workshop participants.

• June 10th: Submission deadline of the paper draft (just send the PDF to hpc-iodc-submissions@wr.informatik.uni-hamburg.de)
• June 30th: Author notification with feedback
• July 7th: Camera-ready publication that will be published on the web-page

The workshop is integrated into ISC-HPC. We welcome everybody to joint the workshop, including:

• I/O experts from data centers and industry.
• Interested domain scientists and computer scientists interested in discussing I/O issues.
• Vendors.

You may be interested to join our open mailing list HPC-IODC-15 which is open to discuss HPC-I/O topics.

We especially welcome participants that are willing to give a presentation about the I/O of the representing institutions data center. If you are interested to give a presentation, please send a short email to hpc-iodc-submissions@wr.informatik.uni-hamburg.de. Note that all presentations should cover the topics mentioned above – sketch a rough presentation outline in your email. (For vendors: do not focus on commercial aspects in your talk).

Agenda is in preparation.

• 9:00 Welcome – Julian Kunkel – Slides
• 9:15 Summary of the DoE Storage Systems and Input/Output Workshop 2014 – Jay Lofstead (Sandia) – Slides
• 10:00 I/O at the German Climate Computing Center (DKRZ) – Julian Kunkel (DKRZ) – Slides
• 10:30 I/O studies at CSCS – Colin McMurtrie (CSCS) – Slides
• 11:00 Coffee Break
• 11:30 I/O in an Industrial Scientific Computing environment – Gerd Büttner (Airbus) – Slides
• 12:00 I/O at the HLRS – Thomas Bönisch (HLRS) – Slides
• 12:30 Discussion round (Benchmarking) – Jay Lofstead
• 13:00 Lunch
• 14:00 I/O at the University of Dresden – Michael Kluge (ZIH Dresden) – Slides
• 14:30 Activities Towards High Availability of Parallel I/O at the K computer – Yuichi Tsujita (RIKEN) – Slides
• 15:00 I/O at Argonne – Florin Isaila (Argonne National Laboratory & University Carlos III) – Slides
• 15:30 I/O at JSC – Wolfgang Frings (Jülich Supercomputing Centre) – Slides
• 16:00 Coffee Break
• 16:30 Percipient Storage: A Big Data and Extreme Compute Storage Architecture – Sai Narasimhamurthy (Seagate) – Slides
• 17:00 Discussion round 2 – Colin McMurtrie
• 17:30 Feedback and Farewell – Julian Kunkel
• 18:00 End