MD-Workbench
This MPI parallel benchmark mimics the concurrent access to typically small objects. It comes with the following features:
- deterministic access pattern mimicking interactive users or producer/consumer queues;
- configurable working set size to fit into a cache of a given size or exceed it;
- performance regression testing by preserving the working set between runs;
- support for various storage backends (POSIX, MPI-IO, S3, MongoDB, PostgreSQL), and
- report of throughput and latency statistics including timing individual I/O operations.
It therewith overcomes certain limitations of HPC metadata benchmarks and provides additional quantifiable measures.
Key Information
Contact | Dr. Julian Kunkel | ||
Repository | Public on GitHub | ||
URL | MD-Workbench @ VI4IO |
Publications
- Understanding Metadata Latency with MDWorkbench (Julian Kunkel, George S. Markomanolis), 2019-01-25 BibTeX DOI PDF
Talks
- Tracking User-Perceived I/O Slowdown via Probing (Dr. Julian Kunkel), HPC IODC Workshop, ISC HPC, Frankfurt, Germany, 2019-06-20 Presentation
- Understanding Metadata Latency with MDWorkbench (Dr. Julian Kunkel), WOPSSS, Frankfurt, Germany, 2018-06-28 Presentation