====== Support for I/O in Workflow Engines ======

| Begin | Anytime |
| Supervisor | [[about:people:julian kunkel]] |

This work will be embedded in the [[https://aces.cs.reading.ac.uk/|ACES research group]].

If you are interested in this topic or similar topics, contact [[about:people:julian kunkel]].

===== Description =====
Workflow engines orchestrate tasks defined as dependency graphs efficiently. In High-Performance Computing, such engines are, for example, used to define the workflows of generating a weather prediction and execute them on supercomputers consisting of 1000s of servers. However, the current workflow engines neglect the requirements for input/output (I/O) in their description and, thus, lack the opportunity of optimization.
In this work, the CYLC workflow engine is extended to cover I/O requirements in the description and graphical representation and develop a first scheduling algorithm that aims to minimize I/O between steps.

===== Methods =====

Firstly, a model and graphical representation are developed describing the relevant facets of I/O that should be covered.
Next, the existing description language is extended with the model.
A scheduling algorithm is developed (e.g., in Python) and evaluated that considers the 
The developed scheduler will act as a prototype showcasing the benefits when including I/O for the existing CYLC community.

===== Beneficial Knowledge =====

While all skills needed to complete this project can be obtained during the time of the project, some skills are beneficial:
  * Programming languages: Python (intermediate level)
  * Linux (intermediate)

An MSc candidate is expected to bring soft skills (in decreasing order of importance):
  * Communication
  * Problem-Solving
  * Time management