ESiWACE2

The Centre of Excellence in Simulation of Weather and Climate in Europe (ESiWACE2) is an H2020 funded project and successor of the ESiWACE project.

Within the project, the research group is responsible for various contribution to work packages and particularly WP 4.

Please see the official web page of ESiWACE for further information.

Contact Dr. Julian Kunkel

  • Deutsches Klimarechenzentrum GmbH (coordinator)
  • Centre National de la Recherche Scientifique
  • European Centre for Medium-Range Weather Forecasts
  • Barcelona Supercomputing Center
  • Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V./ Max-Planck-Institut für Meteorologie
  • Sveriges meteorologiska och hydrologiska institut
  • Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique
  • National University of Ireland Galway (Irish Centre for High End Computing)
  • Met Office
  • Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici
  • The University of Reading
  • Science and Technology Facilities Council
  • BULL SAS
  • Seagate Systems UK Limited
  • ETH Zürich
  • The University of Manchester
  • Netherlands eScience Center
  • Federal Office of Meteorology and Climatology
  • DataDirect Networks
  • Mercator Océan

While we are involved in various work packages, the main focus lies is WP4 which will provide the necessary toolchain to handle data at pre-exa-scale and exa-scale, for single simulations, and ensembles.

Specifically, we will

  1. Support data reduction in ensembles and avoid un-necessary subsequent data manipulations by providing tools to carry out ensemble statistics “in-flight” and compress ensemble members on the way to storage.
  2. Provide tools to: a) transparently hide complexity of multiple-storage tiers from applications at runtime by developing middleware that lies between the familiar NetCDF interface and storage, and prototype commercially credible storage appliances which can appear at the backend of such middleware, and; b) support manual migration of semantically important content between primary storage on disk, tape, and object stores, including appropriate user-space caching tools (thus allowing some portable data management within weather and climate workflows).

The work will build upon the generated prototypes build in ESiWACE-1.

architecture-d4.2.pdf

Publications

Talks

  • The Need for Next Generation Semantic Interfaces to Process Climate/Weather Workflows (Dr. Julian Kunkel), SIG IO UK, University of Reading, UK, 2018-06-06 Presentation
  • Towards Intelligent Storage Systems (Dr. Julian Kunkel), Computer Science Workshop, University of Reading, UK, 2018-05-08 Presentation
  • Exploiting the Heterogeneous Storage Landscape in a Data Center (Dr. Julian Kunkel), Per3s: Performance and Scalability of Storage Systems Workshop, Rennes, France, 2018-01-12 Presentation
  • Exploiting Weather and Climate Data at Scale (WP4) (Dr. Julian Kunkel), ESiWACE General Assembly, Berlin, Germany, 2017-12-12 Presentation