Jakob Hördt

Biography

Jakob is a research assistant in the group of Julian Kunkel. He has a Master's degree in computer science from University of Göttingen.

Research Interests

  • Distributed Systems
  • HPC

Projects

Teaching

Open Thesis Topics

Developing an Inference Engine with WebGPUApply

WebGPU is an emerging graphics API for modern GPU use in the browser. In this thesis the potential of it for AI inference directly in the browser is explored.

Innovating on network protocols for AI inferenceApply

The defacto standard API for LLM inference tasks is the OpenAI API. However, it is not optimal in terms of bandwidth and other characteristics. For example, images for are base64 encoded in the protocol, wasting bandwidth and CPU cycles. The SSE protocol used for streaming also incurs overhead. Furthermore, the protocol's stateless nature requires the whole conversation to be sent on each request. This could be mitigated for example by exploiting modern Compression Dictionary Transport or HPACK/QPACK. In this work, such approaches are implemented and evaluated in comparison to the existing protocols for common AI inference tasks.

Theses

  • An HPC FaaS Runtime based on HPX and Modern Lightweight Isolation, Jakob Hördt (Master's Thesis), Advisors: Prof. Dr. Julian Kunkel, Sven Bingert, 2023-09, BibTeX URL

All publications as BibTex