Jakob Hördt
Biography
Jakob is a research assistant in the group of Julian Kunkel. He has a Master's degree in computer science from University of Göttingen.
Research Interests
- Distributed Systems
- HPC
Projects
Teaching
Open Thesis Topics
Developing an Inference Engine with WebGPUApply
WebGPU is an emerging graphics API for modern GPU use in the browser. In this thesis the potential of it for AI inference directly in the browser is explored.
Innovating on network protocols for AI inferenceApply
The defacto standard API for LLM inference tasks is the OpenAI API. However, it is not optimal in terms of bandwidth and other characteristics. For example, images for are base64 encoded in the protocol, wasting bandwidth and CPU cycles. The SSE protocol used for streaming also incurs overhead. Furthermore, the protocol's stateless nature requires the whole conversation to be sent on each request. This could be mitigated for example by exploiting modern Compression Dictionary Transport or HPACK/QPACK. In this work, such approaches are implemented and evaluated in comparison to the existing protocols for common AI inference tasks.
Theses
All publications as BibTex