ESiWACE

The Centre of Excellence in Simulation of Weather and Climate in Europe (ESiWACE) is a H2020 funded project.

Within the project, our research group is responsible for DKRZ's contribution to work package 4 (Exploitability).

Please see the official web page of ESiWACE for further information.

Contact (for WP4 contribution of DKRZ) Dr. Julian Kunkel

  • Modelling costs for storage methods and understanding these
  • Modelling tape archives and costs
  • Flexible disk storage layouts for earth system data
  • DKRZ
  • STFC
  • ECMWF
  • CMCC
  • Seagate

The design deliverable architecture-d4.2.pdf

ESDM builds upon a data model similar to NetCDF and utilizes a self-describing on-disk data format for storing structured data. We aim to deliver the NetCDF integrated version by the end of the ESiWACE1 project. This improvement can then be used as a drop-in replacement for typical use-cases without changing anything from the application perspective. While our current version utilises the manual configuration by data-center experts, the ultimate long-term goal is to employ machine learning to automatise the decision making and reduce the burden for users and experts.

Publications

  • Toward Understanding I/O Behavior in HPC Workflows (Jakob Lüttgau, Shane Snyder, Philip Carns, Justin M. Wozniak, Julian Kunkel, Thomas Ludwig), 2019-02-11 BibTeX DOI
  • Beating data bottlenecks in weather and climate science (Bryan N. Lawrence, Julian Kunkel, Jonathan Churchill, Neil Massey, Philip Kershaw, Matt Pritchard), 2019-01-25 BibTeX URL PDF
  • Cost and Performance Modeling for Earth System Data Management and Beyond (Jakob Lüttgau, Julian Kunkel), 2019-01-25 BibTeX DOI PDF
  • A Survey of Storage Systems for High-Performance Computing (Jakob Lüttgau, Michael Kuhn, Kira Duwe, Yevhen Alforov, Eugen Betke, Julian Kunkel, Thomas Ludwig), 2018-04 BibTeX URL DOI PDF
  • Poster: Adaptive Tier Selection for NetCDF and HDF5 (Jakob Lüttgau, Eugen Betke, Olga Perevalova, Julian Kunkel, Michael Kuhn), 2017-11-14 BibTeX PDF
  • Simulation of Hierarchical Storage Systems for TCO and QoS (Jakob Lüttgau, Julian Kunkel), 2017 BibTeX DOI PDF
  • Poster: Modeling and Simulation of Tape Libraries for Hierarchical Storage Systems (Jakob Lüttgau, Julian Kunkel), 2016-11-15 BibTeX URL PDF

Talks

  • Impressum
  • Privacy
  • research/projects/hamburg/esiwace.txt
  • Last modified: 2019-02-17 12:07
  • (external edit)