HPC-IODC: HPC I/O in the Data Center Workshop
Managing scientific data at large scale is challenging for scientists but also for the host data center. The storage and file systems deployed within a data center are expected to meet users' requirements for data integrity and high performance across heterogeneous and concurrently running applications.
With new storage technologies and layers in the memory hierarchy, the picture is becoming murkier. To effectively manage the data load within a data center, I/O experts must understand how users expect to use these new storage technologies and what services they should provide in order to enhance user productivity. We seek to ensure a systems-level perspective is included in these discussions.
In this workshop we bring together I/O experts from data centers and application workflows to share current practices for scientific workflows, issues and obstacles for both hardware and the software stack, and R&D to overcome these issues.
The workshop content is build on two tracks with calls for papers/talks:
- research paper presentation – you'll need to submit a short paper regarding relevant research for I/O in the datacenter.
- talks from I/O experts – you'll need to submit a rough outline for your talk.
Contributions to both tracks are peer reviewed and require submission of the respective research paper or idea for your presentation via Easychair (see the descriptions below).
We will have a keynote by Robert Ross with the topic: “Studying I/O in the Data Center: Observing and Simulating I/O for Fun and Profit”.
Rob Ross from Argonne National Laboratory serves as deputy director of the Scientific Data Management, Analysis, and Visualization Institute. He lead and was involved in many storage projects such as PVFS2, Darshan and CODES, and published more than 100 papers in the area of HPC-I/O.
The workshop is held in conjunction with the ISC-HPC. See also our last year's workshop web page.
Please also see our BoF for the community project VI4IO at ISC.
| Date | Thursday June 23th, 2016 | ||
| Venue | Frankfurt, Germany, Details about the ISC-HPC venue | ||
| Mailinglist | HPC-IODC-16 | ||
| Contact | Dr. Julian Kunkel |
Organization
The workshop is organized by
- Julian Kunkel (DKRZ, Germany), kunkel@dkrz.de
- Jay Lofstead (Sandia National Lab, USA), gflofst@sandia.gov
- Colin McMurtrie (CSCS, Switzerland), cmurtrie@cscs.ch
Program committee
- Wolfgang Frings (Jülich Supercomputing Center)
- Javier Garcia Blas (University Carlos III of Madrid)
- Rob Ross (Argonne National Laboratory)
- Carlos Maltzahn (University of California, Santa Cruz)
- Kathryn Mohror (Lawrence Livermore National Laboratory)
- Xiaosong Ma (North Carolina State University, Oak Ridge National Laboratory)
- Julian Kunkel (DKRZ, Germany)
- Jay Lofstead (Sandia National Laboratory)
- Colin McMurtrie (CSCS, Switzerland)
Participation
The workshop is integrated into ISC-HPC. We welcome everybody to joint the workshop, including:
- I/O experts from data centers and industry.
- Researchers/Engineers working on high-performance I/O for data centers.
- Interested domain scientists and computer scientists interested in discussing I/O issues.
- Vendors are also welcome, but their presentations must align with data center topics (e.g. how do they manage their own clusters) and not focus on commercial aspects.
You may be interested to join our open mailing list HPC-IODC-16 which is open to discuss HPC-I/O topics.
We especially welcome participants that are willing to give a presentation about the I/O of the representing institutions data center. Note that such presentations should cover the topics mentioned below.
Track: research papers
We accept short papers with up to 10 pages (excl. references) in LNCS format. Please see the instructions and templates for authors provided by Springer.
Our targeted proceedings are ISC's post-conference workshop proceedings in Springers LNCS. We use Easychair for managing the proceedings and PC interaction.
Paper Deadlines
- Submission deadline: 28-02-2016 AoE
- Author notification: 23-03-2016
- Workshop: 23-06-2016
- Camera-ready papers: 23-07-2016 – As they are needed for ISC's post-conference workshop proceedings. We embrace the chance for authors to improve their papers based on the feedback received during the workshop.
Track: Talks by I/O experts
The topics of interest in this track include but are not limited to:
- A description of the operational aspects of your data center
- A particular solution for certain data center workloads in production
We use Easychair for managing the acceptance and PC interaction. If you are interested to participate please submit a short (1/2 page) intended abstract of your talk together with a short Bio.
Deadlines for the submission of the abstract
- Submission deadline: 28-02-2016 AoE
- Author notification: 23-03-2016
Content
The following list of items should be tried to be integrated into a talk covering your data center, if possible. We hope your sites administrator will support you to gather the information with little effort.
- Workload characterization
- Scientific Workflow (give a short introduction)
- A typical use-case (if multiple are known, feel free to present more)
- Involved number of files / amount of data
- Job mix
- Node utilization (rel. to peak-performance)
- System view
- Architecture
- Schema of the client/server infrastructure
- Capacities (Tape, Disk, etc.)
- Potential peak-performance of the storage
- Theoretical
- Optional: performance results of acceptance tests.
- Software / Middleware used, e.g. NetCDF 4.X, HDF5, …
- Monitoring infrastructure
- Tools and systems used to gather and analyse utilization
- Actual observed performance in production
- Throughput graphs of the storage (e.g. from Ganglia)
- Metadata throughput (Ops/s)
- Files on the storage
- Number of files (if possible per file type)
- Distribution of file sizes
- Issues / Obstacles
- Hardware
- Software
- Pain points (what is seen as the biggest problem(s) and suggested solutions, if known)
- Conducted R&D (that aim to mitigate issues)
- Future perspective
- Known or projected future workload characterization
- Scheduled hardware upgrades and new capabilities we should focus on exploiting as a community
- Ideal system characteristics and how it addresses current problems or challenges
- what hardware should be added
- what software should be developed to make things work better (capabilities perspective)
- Items requiring discussion to work through how to address
Agenda
The Agenda is currently in preparation.
- 09:10 Keynote: Studying I/O in the Data Center: Observing and Simulating I/O for Fun and Profit
Robert Ross, Argonne National Laboratory
presentation - 10:00 Delta: Data Reduction for Integrated Application Workflows and Data Storage
Jay Lofstead, Gregory Jean-Baptiste and Ron Oldfield
presentation - 10:30 The Effect of Python and NetCDF on the Read Performance when using HPC Parallel Filesystems
Matthew Jones, Jon Blower, Bryan Lawrence and Annette Osprey
presentation - 11:00 – 11:30 Coffee break
- 12:00 Analyzing the High Performance Parallel I/O on LRZ’s HPC Systems – Sandra Mendez
presentation - 13:00 – 14:00 Lunch
- 16:00 – 16:30 Coffee break
- 16:30 Analyzing Data Properties using Statistical Sampling Techniques – Illustrated on Scientific File Formats and Compression Features
Julian Kunkel
presentation - 17:00 An Overview of the Sirocco Parallel Storage System
Matthew Curry, H. Lee Ward, Geoff Danielson and Jay Lofstead
presentation - 17:30 Discussion round + Farewell
- 18:00 End