Theses
2023
Running Kubernetes Workloads on Rootless HPC Systems using Slurm
| Author | Sören Metje |
| Type | Master's Thesis |
| Advisors | Prof. Dr. Julian Kunkel, Stefanie Mühlhausen |
| Date | 2023-12 |
| Material | BibTeX URL |
An HPC FaaS Runtime based on HPX and Modern Lightweight Isolation
Enhancing Tree Segmentation in Large Forest Point Clouds with Synthetic Data
| Author | Ali Doosthosseini |
| Type | Master's Thesis |
| Advisors | Prof. Dr. Julian Kunkel, Prof. Dr. Alexander Ecker |
| Date | 2023-09 |
| Material | BibTeX |
2022
Learning to Attack: Automated Red Teaming Using Deep Reinforcement Learning
| Author | Sadegh Keshtkar |
| Type | Master's Thesis |
| Advisors | Sahin Albayrak |
| Date | 2022-05 |
| Abstract | This thesis investigated the application of Deep Reinforcement Learning (DRL) to automated offensive assessment and tried to build a DRL agent, in a specific sense, a Dueling Double Deep Q Network (D3QN), which is capable of learning to attack an unknown network. |
| Material | BibTeX |
2023
Implementation of a Liquid Neural Network Control System for Multi-Join Cyber Physical ARM
| Author | Michael Bidollahkhani |
| Type | Master's Thesis |
| Advisors | Ferhat Atasoy, Abdellatef Hamdan |
| Date | 2023-06 |
| Material | BibTeX |
2016
Extract and mining government services, especially USO and their impacts on the development of rural communities using data mining algorithms and artificial intelligence
| Author | Michael Bidollahkhani |
| Type | Bachelor's Thesis |
| Advisors | I. Soleimani, A. Shahbahrami |
| Date | 2016 |
| Material | BibTeX |
2022
Knowledge Component Extraction and Representation from Video Sequences
| Author | Felix Stein |
| Type | Master's Thesis |
| Advisors | Prof. Dr. Jens Grabowski |
| Date | 2022-01-26 |
| Material | BibTeX |
2010
Design, Construction and Validation of an articulated solar panel for CubeSats
| Author | Patrick Höhn |
| Type | Master's Thesis |
| Advisors | Dr Johnny Ejemalm |
| Date | 2010 |
| Material | BibTeX |
2008
Entwicklung und Bau eines ausklappbaren Sonnensegels für den Cube-Satelliten Uwe 2
| Author | Patrick Höhn |
| Type | Diplomarbeit |
| Advisors | Prof. Dr.-Ing. Winfried Perseke |
| Date | 2008 |
| Material | BibTeX |
2022
Interactive Data Center Digital Twin using Virtual Reality
| Author | Lars Quentin |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2022-12-15 |
| Abstract | Virtual Reality (VR) allows users to experience and interact with digital environments in an immersive way and, therewith, can provide a more intuitive way of interacting with computers using more natural human-machine interaction paradigms. Additionally, virtual environments can provide various advantages for data visualization, as the digital space is not constrained by real-world physical limitations. Furthermore, leveraging the third dimension and the spatial features, VR can be a viable alternative to traditional visualization. The goal of this thesis is the interaction and visualization paradigms to the realm of data centers by creating a server room as a digital twin using the Unity game engine. By integrating the live utilization metrics of a real High-Performance Computing (HPC) cluster into the virtual world, the VR server room can be used to analyze the current data and reason about its implications. This integration is done by incorporating the data into several places in the virtual environment. Following the creation of this digital twin, the application was evaluated against traditional, web-based data dashboards by conducting a user acceptance study. The results imply that the VR alternative improved the user experience and reduced the analysis complexity compared to the web version, concluding that VR can be a valuable addition to traditional metric visualization for the application of data center monitoring. |
| Material | Thesis BibTeX |
Tree species classification from airborne LiDAR using individual crown delineation and machine learning
| Author | Hauke Kirchner |
| Type | Master's Thesis |
| Advisors | Dr. N. Knapp |
| Date | |
| Material | BibTeX |
Taxonomical read assignment with filtered spaced \ word matches at varying taxonomic levels
| Author | Hauke Kirchner |
| Type | Bachelor's Thesis |
| Advisors | Prof. Morgenstern |
| Date | |
| Material | BibTeX |
2022
The Potential of Serverless Kubernetes-Based FaaS Platforms for Scientific Computing Workloads
| Author | Jonathan Decker |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2022-01-31 |
| Abstract | Serverless computing has emerged as a new execution model for cloud computing with many advantages such as improved resource sharing and utilization as well as reduced complexity for developers that under serverless only need to write function code without worrying about server infrastructure. These advantages are also interesting for other fields of computing, for instance, High-Performance Computing (HPC) as improving resource sharing on expensive compute clusters would be a great benefit and serverless could even work as a drop-in replacement for existing HPC function. In order to utilize serverless on-premise, one needs to choose a serverless platform to deploy from the various, avail- able open source platforms, however, with so many rapidly evolving open source serverless platforms, it has become difficult to draw comparisons between them. Furthermore, there are no standardized benchmarks for serverless platforms, therefore, in order to perform a systematic analysis, three workloads inspired by scientific computing were designed and implemented in this thesis to compare the performance and usability of three representa- tive open source Kubernetes-based serverless platforms. The workloads include an image processing task as well as an adaption of dgemm() for HPL and were developed and tested on a cluster of Virtual Machines (VMs) provided by the Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG)1. It was observed that the performance of these platforms can be adequate when compared to expected hardware limits, but overall there are still many opportunities for improvements in terms of performance and usability. |
| Material | BibTeX URL |
2021
On comparative results concerning the Grundy and b-chromatic number of graphs
| Author | Zoya Masih |
| Type | PhD Thesis |
| Advisors | Manouchehr Zaker |
| Date | 2021-10-12 |
| Abstract | The Grundy and b-chromatic number of graphs are two important chromatic parameters. The Grundy number of a graph G, denoted by Γ(G) is the worst case behavior of greedy (First-Fit) coloring procedure for G and the b-chromatic number, denoted by b(G), is the maximum number of colors used in any color-dominating coloring of G. Since the nature of these colorings are different, they have been studied widely but separately in the literature. This dissertation firstly presents some preliminary concepts. Then the two chromatic numbers are compared for Trees, Cacti, (K4,C4)-free graphs and K2q-free graphs. In each family, a function f will be introduced, such that, for every graph G of the family, the inequality Γ(G)≤f(b(G)) holds. The aforesaid function, is also presented for graphs with sufficiently large girth. |
| Material | BibTeX |
Open source vehicle ECU diagnostics and testing platform
| Author | Ashcon Mohseninia |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2021-04-29 |
| Abstract | With the complexity of electronics in consumer vehicles, there currently only exists proprietary tools produced by various OEMs to diagnose their own vehicles. Each OEM has its own tool, and there is no easy way for a consumer to diagnose their own vehicle. This project will explore the possibility of creating an entirely open source diagnostics software stack that will work with all ready existing diagnostic adapters that utilize the Passthru API (Which is used for a PC to communicate with a diagnostic adapter plugged into a vehicles OBD-II port). Additionally, this project will also explore creating an entirely open source Passthru API driver for an open source OBD-II adapter, whilst additionally porting the API from Win32 only to UNIX based operating systems such as Linux and OSX, allowing for a wider target audience compared to traditional diagnostic applications and adapters which only target Windows. |
| Material | Thesis BibTeX |
2020
Performance Analysis of Convolutional Neural Network Applying Quantum Annealing
| Author | Aasish Kumar Sharma |
| Type | Master's Thesis |
| Advisors | Sanjeeb Prasad Pandey |
| Date | 2020-12-30 |
| Abstract | The paper presents a case study on Quantum Annealing in relation to Simulated Annealing. At first it includes theoretical details for Annealing method, Simulated and Quantum Annealing techniques for solving problems where search space is discrete. Onward, based on variant performance parameters, from various experimental results, conducted using quantum annealer (like D-Wave) in compare to different classical counter fits verifies Quantum Annealing as an optimized technique against Simulated Annealing. |
| Material | BibTeX URL |
2019
Beamer Slide Composition
| Author | Joseph Dyer |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2019-06-09 |
| Abstract | This report describes the action taken to improve the reuse of individual slides from presentations, intending to aid the dissemination of knowledge especially in an academic setting. This goal is to be achieved by managing large collections of presentations made with LaTex, using machine learning to automate archiving and improve retrieval. This report documents the design and development process of a candidate created to prove the usefulness of such a system. |
| Material | Thesis BibTeX |
Recognition of Company Mergers Using Interactive Labeling and Machine Learning Methods
| Author | Anne Lorenz |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Doris Birkefeld |
| Date | 2019-05-08 |
| Abstract | Being informed about current business events is essential for decision makers in a company. The aim of this thesis is to develop a strategy for recognizing company mergers in news articles by applying common and experimental Machine Learning methods. For this text classification problem, an interactive human-computer labeling technique is explored in order to manually label an unclassified data set in a more efficient way. Using class probabilities based on the Naive Bayes algorithm, the iterative approach accelerates the data labeling process by propagating labels through the data set. Through experimental research on this practical application problem, it is found that the proposed labeling technique is eight times faster than conventional labeling, because the number of articles to be labeled can be reduced. On the resulting data set, a common Support Vector Machines model achieves a Recall score of 86.9% and a Precision score of 86.1%. The presented incremental method that is simple to implement is not only suitable for text classification problems, but universal for all kinds of large unclassified data sets, as, e.g., in image classification and speech recognition. |
| Material | Thesis BibTeX |
Accessibility Assistance for the Interactive Navigation of Texts
| Author | Imad Hamoumi |
| Type | Master's Thesis |
| Advisors | Patricio Farrell, Dr. Julian Kunkel |
| Date | 2019-01-28 |
| Abstract | A mass increase of information has been observed in the last years. The web has undergone a phase of rapid growth regarding content and became an important medium. This growth led to new challenges for users to satisfy their information needs, as well as for service providers to store content and make it easily accessible. Researches have proposed new approaches that improve the quality of search machines, the accuracy of ranking algorithms, and the performance of storing systems. However, they neglect the role of the interface that visually presents this information and assists the user in finding it. Thus, these approaches are only usable if the interface used is optimised to serve the users. This thesis describes the dominant approach used today to search and explore text data. Then, it proposes a navigation model that improves these approaches. In order to proof the theory of this navigation model, three interfaces are implemented. These interfaces integrate the data of a service that is used to search for research data. This service serves as case study to evaluate the results. The evaluation is conducted with test users and discussed at the end of the thesis. |
| Material | Thesis BibTeX |
Entwicklung eines computergesteuerten Akteurs für das kooperative Spielen in einem dyadischen Geschicklichkeitsspiel
| Author | Melanie Budde |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2019-01-17 |
| Material | Thesis BibTeX |
2018
Comparing Naïve Bayesian Networks to Support Vector Machines to predict Stock Prices Based on Press Release Sentiment
| Author | Max Lübbering |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Patricio Farrell |
| Date | 2018-12-04 |
| Abstract | In this master thesis, we trained and evaluated two models, namely a naïve Bayesian network and a support vector machine, to predict stock price trends based on count vectorized press releases published in the after hours. Additionally, we introduced a trivial classifier to put the results of the other two into perspective. The stock price trend prediction was solved in two steps: In the first step, we built for each of the three algorithms a classifier in order to predict the impact of a press release. For training and evaluation, every press release was assigned the label impact, if the stock price had changed at least 8% from the exchange closing time to exchange opening time plus an offset of 5 hours and no impact, otherwise. In the second step, all press releases with no impact were discarded. The remaining ones were reassigned to the classes sentiment and no sentiment based on the direction of their impact. Afterwards, we built a model for each of the three algorithms to predict the sentiment of press releases. After applying grid search on an extensive grid, the impact models of the naïve Bayesian network, the support vector machine and trivial classifier had an accuracy of 76%, 78% and 77%, respectively. The balanced dataset contained 919 training samples and 231 samples in the holdout set. These high accuracies show that im- pact prediction is per se possible, even though count vectorization destroys all the semantics. The sentiment models of the naïve Bayesian network, the support vector machine and trivial classifier had an accuracy of 47%, 53% and 53%, respectively. The balanced training set contained 426 samples and the holdout set 108. The weak results reveal that sentiment prediction is far more complex than impact prediction and cannot be captured by the word frequency in a document. As part of our press release exploration, we demonstrated that over night press releases cause inefficiencies in the market for the entire next trading day. As a consequence, we provided an example that clearly contradicts with the theoretical efficient market hypothesis. If our models become more reliable, these inefficiencies can be exploited. |
| Material | Thesis BibTeX |
Understanding Customer Behaviour to Optimize Product Sorting for E-Commerce Websites
| Author | Amina Voloder |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Antonia Wrobel |
| Date | 2018-08-07 |
| Abstract | This thesis investigates product-specific and user-specific characteristics that influence the sales in order to develop a novel sorting algorithm for application in the field of e-commerce, through the analysis of customer preferences and the nature of a given store’s products, to improve the personalisation of online shopping systems. The algorithm optimises the order of displayed products by their purchase probability, determining which products users are most likely to purchase. This is determined by investigating the correlation between product sales and: product seasons, the time of day, and the devices users own. It was found that products could be classified as being sold well in particular months or regardless of the month. Similarly, products could be sold well at particular times of the day or regardless of the time. It was also determined that users of Apple devices should have more expensive products promoted to them, as they typically purchase greater quantities of expensive products. The algorithm is evaluated by means of visual and quantitative comparison against the standard sorting algorithm used within an e-commerce system by novomind AG. Test results indicate a discernible variation in the sorting order of products, as well as an increase in the variety of the eight highest sorted products. The full contribution of the algorithm to the sorting optimisation is verifiable through real-world A/B testing. |
| Material | Thesis BibTeX |
Comparison of Compiler’s Intermediate Representations and Input/Output Access Patterns with String Kernels
| Author | Raul Torres |
| Type | PhD's Thesis |
| Advisors | Thomas Ludwig, Dr. Julian Kunkel, Manuel F. Dolz |
| Date | 2018-06-18 |
| Abstract | Kernel methods aim for the detection of stable patterns robustly and efficiently from a finite data sample by embedding data items into a space of higher dimensionality where data points have linear relations. Strings kernels apply this methodology to find relationships between string objects by checking for the number of shared substrings and using this measure as a similarity score. Due to the low number of studies conducted in the area of code comparison and Input/Output (I/O) access pattern recognition using string kernels, the goal of this thesis is to propose a suitable, general representation, as well as the corresponding strategies of comparison based on kernel methods, such that they can be used successfully to determine a reliable similarity measure among a collection of programs. Therefore, we propose different conversion strategies from these original sources to a weighted string representation; the defined representation is a collection of tokens whose weights allow the modulation of the contribution of each token in the calculation of the overall similarity. The resulting strings are compared with a new family of kernel functions, which correspond to the major contribution of this thesis: the kastx spectrum kernel family. In order to create a similarity measure among two strings, these kernels are based on the longest common substrings; the idea behind this approach is to give more relevance to the largest common pieces of code rather than to small and disperse code instructions. The size of the valid matching substrings is controlled by the cut weight, a parameter given by the user, that specifies the minimum weight that those substrings should have. We tested our methodology in two scenarios: i) pattern recognition in I/O traces, and ii) comparison of intermediate representations of a compiler. In the first scenario, the clustering analysis showed that the proposed kernels managed to conform clusters that reflected the similarity of patterns taken from two popular I/O benchmarks. For the second scenario, a set of C functions was organized in four different classes, according to their purpose; clustering analysis here also showed a cluster organization that reflected the affinity among functions of the same class. These new kernels obtained similar, and in some cases, better results when compared to the blended spectrum kernel [SC04], a string kernel with widespread use in cheminformatics problems. The provided kernels will enrich the spectra of available string kernel functions on the literature and might be used in the future in similarity studies, not only in the field of computer science, but also in other areas like cheminformatics, bioinformatics or natural language processing. |
| Material | Thesis BibTeX |
Large-Scale Accessibility Analysis Using OpenStreetMap Data
| Author | Martin Poppinga |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2018-06-06 |
| Abstract | By using accessibility analyses, the quality of coverage of institutions can be determined, giving knowledge how many persons can be reached within a given time. Such analyses have a variety of use-cases in the public, private and commercial sector. Existing approaches showed precise results only for smaller regions or were at the expense of precision in a small-scale context. Additionally, the used datasets and software frameworks are often proprietary, which hinders reproducibility in research. In this thesis, it is shown that large-scale accessibility analyses are possible using open data while retaining a high precision. Also, the data cleaning and preparation is presented. Further is demonstrated that obtained information from different analyses can be combined for more complex analyses and can be used for solving location optimization problems. For evaluation, different studies for Germany are sketched, and the outcomes are shown. Examples are the coverage of fire stations, hospitals, stroke units or charging stations for electric cars. The results show that it is possible to achieve a more precise resolution as existing approaches, enabling analyses using small-scaled aspects while maintaining country-sized expansion and acceptable computation times. This advancement enables, for example, further studies in spatial and urban research. |
| Material | Thesis BibTeX |
Vector Folding for Icosahedral Earth System Models
| Author | Jonas Tietz |
| Type | Bachelor's Thesis |
| Advisors | Nabeeh Jumah, Dr. Julian Kunkel |
| Date | 2018-03-26 |
| Abstract | The performance of High Performance Computing (HPC) applications become increasingly bound by the access latency of main memory. That is why strategies to minimize accesses to memory and maximize the use of the caches are crucial for any serious HPC application. There is lots of research on the topic of trivial rectangular grids, like using SIMD (single instruction multiple data) instructions, to operate on multiple values at once, or cache blocking techniques, which try to divide the grids into chunks, which fit into the cache. Additionally, there are new interesting techniques for minimizing loads in stencil computations like vector folding. This thesis will take a look at the theoretical performance benefits, especially vector folding in conjunction with an icosahedral grid, and test them in a simple test case. As a result the performance improved slightly in comparison over traditional vectorization techniques. |
| Material | Thesis BibTeX |
Enabling Single Process Testing of MPI in Massive Parallel Applications
| Author | Tareq Kellyeh |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Christian Hovy |
| Date | 2018-03-20 |
| Abstract | While many parallel programming models exist, the dominant model is MPI. It has been considered as the de facto standard for building parallel programs that use message passing. In spite of this popularity, there is a lack of tools that support testing of MPI programs. When considering unit testing, it is not widely applied to scientific programs, even though it is an established practice in professional software development. However, with MPI, the communicated data comes from different processes which increases the effort of creating small test units. In this thesis, a solution to reduce the effort of testing massive parallel applications is developed. By applying this solution, any selected piece of MPI parallelized code that forms a part of such applications can be tested. The used method is based on the technique: Capture and Replay. This technique extracts data while executing the application and uses this data as an input for the MPI communications in the test phase. The structures, that contain the extracted data, are generated automatically. As a step towards enabling Unit Testing of MPI applications, this thesis supports the user in writing appropriate test units and executing them by a single process solely. In this way, repeating the expensive parallel execution of MPI programs can be avoided. This step is considered as the most important contribution of this thesis. |
| Material | Thesis BibTeX |
Modeling and Performance Prediction of HDF5 data on Objectstorage
| Author | Roman Jerger |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Jakob Lüttgau |
| Date | 2018-03-15 |
| Material | Thesis BibTeX |
2017
Verarbeitung von Klimadaten mit Big-Data-Werkzeugen
| Author | Alexander Erhardt |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2017-07-31 |
| Abstract | Die Verarbeitung und Analyse von Klimadaten umfassen heutzutage größere Datenmengen, die sehr oft strukturiert innerhalb der NetCDF-Dateien aufbewahrt werden. Die Verarbeitungsprozesse der Datenanalyse benötigen komplexe leistungsfähige Systemen mit größerem Berechnungspotential, um die Datenverarbeitung in akzeptabler Zeit ausführen zu können. Moderne Big-Data-Werkzeuge bieten gut strukturierte Plattformen für die Verarbeitung wissenschaftlicher Daten innerhalb der NetCDF-Dateien. In dieser Arbeit werden mögliche Alternativen der Verwendung von Big-Data-Werkzeugen erläutert, die eine Möglichkeit schaffen, die vom Nutzer angeforderte Verarbeitungsabläufeinnerhalb einer Weboberfläche auszuführen und die Ergebnisse mit Hilfe einer grafischen Datendarstellung begutachten zu können. Auf der Basis des entwickelten Systems wird untersucht, inwiefern die aktuellen Werkzeuge für interaktive Analyse der Klimadaten geeignet sind. Dabei werden sämtliche Berechnungsprozesse mittels SciSparks auf einem Cluster von Berechnungsknoten ausgeführt. Die Steuerung dieser Prozessen sowie Visualisierung der Verarbeitungsergebnisse ermöglicht Apache Zeppelin innerhalb einer Webschnittstelle. Es wird untersucht, inwiefern genannte Werkzeuge angeforderte Voraussetzungen bereits erfüllen können. Diese Systeme werden durch einige Komponenten erweitert, um einen Prototyp des vorgestellten Ansatzes zu entwickeln. Somit werden auf der Basis theoretischer Grundlagen die aufgesetzten Komponenten in einem System mit einer Benutzerwebschnittstelle zusammengefasst. Dabei wurde vorhandene SciSparkFunktionalität mit den implementierten CDO-Operatoren und dem Stencil-Verfahren für ein-, zwei- und dreidimensionale NetCDF-Variablen erweitert. Zum Schluss wird gezeigt, wie effizient eine Ausführung der unterschiedlichen Prozessabläufe in dem entwickelten System sein kann und welche Einschränkungen auf die Software und Hardware ungeeignet beziehungsweise nicht leistungsfähig genug sind. |
| Material | Thesis BibTeX |
Interactive Recommender Systems For A Professional Social Network
| Author | Mirko Köster |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2017-06-09 |
| Abstract | In this thesis, we research interactive recommender systems and present a method to offer interactive recommendations in the form of recommender settings. Specifically, this is done in the domain of job recommendations at XING, a professional social network. These settings allow users to tune some aspects of the job recommender system, i.e. their preferred career level, whether they are willing to commute or even move to a new location, and which topics (skills, jobroles and disciplines) they like or dislike. These topics are explicitly not taken from the users’ profiles, as profiles on XING rather reflect the CV of the user, i.e. things that the user did in the past but not what the user aims to work on in the future. Instead, we generate the topics from the job recommendations we already offer, which are influenced by the users’ profiles, their behavior on the platform as well as from their previously specified recommender settings. These topics can thus be seen as a summary of the users’ job recommendations. By tweaking the recommendation settings, the actual job recommendations immediately change which in turn has an influence on the selectable topics thus allowing the user to interactively refine the recommendation settings and explore the item space. We implemented our recommender settings approach in the back-end of the actual job recommendation service, thus turning XING’s job recommender into an interactive recommender service. Moreover, we implemented a prototype application that allows users to experience the interactive job recommendations. Given both the adjusted job recommender service and our prototype, we conducted both a large-scale quantitative evaluation as well as a user study in which we collected qualitative feedback and analyzed the impact on user satisfaction. |
| Material | Thesis BibTeX |
In-situ Transformation for Input/Output Access Patterns by Applying Building Blocks of Optimization Schemas
| Author | Daniel Schmidtke |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2017-04-19 |
| Abstract | This thesis is about the finding of optimization strategies, that can be applied by in-situ transformation of input/output access patterns and the classification of these strategies. The found optimizations are then being implemented in SIOX and FUSE and evaluated with different benchmarks. The optimization strategies found in this thesis are a demonstration of the possibilities that can be achieved using in-situ transformation. |
| Material | Thesis BibTeX |
Extracting Semantic Relations from Wikipedia using Spark
| Author | Hans Ole Hatzel |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2017-02-02 |
| Abstract | In this work, the full text of both the German and the English Wikipedia were used for two subtasks. 1. Finding Compound Words 2. Finding Semantic Associations of Words The approach to the first task was to find all nouns in the Wikipedia and evaluate which of those form compounds with any other nouns that were found. PySpark was used to work through the whole Wikipedia dataset and the performance the part-of-speech tagging operation on the whole dataset was good. In this way, a huge list of nouns was created which could then be used to check it for compound words. As this involved checking each noun against every other noun the performance was not acceptable, with the analysis of the whole English Wikipedia taking over 200 hours. The data generated from the first subtasks was then for the task of both generating and solving CRA tasks. CRA tasks could be generated at a large scale. CRA tasks were solved with an accuracy of up to 33%. The second subtask was able to cluster words based on their semantics. It was established that this clustering works to some extend and that the vectors representing the words therefor have some legitimacy. The second subtask’s results could be used to perform further analysis on how the difficulty of CRA tasks behaves with how words are related to each other. |
| Material | Thesis BibTeX |
2016
Adaptive Selection of Lossy Compression Algorithms Using Machine Learning
| Author | Armin Schaare |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Anastasiia Novikova |
| Date | 2016-11-29 |
| Abstract | This goal of this thesis was to evaluate machine learning model’s ability for their use as an automatic decision feature for compression algorithms. Their task would be to predict which compression algorithms perform best on what kind of data. For this, artificially generated data, itself, and its compression was analyzed, producing a benchmark of different features, upon which machine learning models could be trained. The models’ goal was to predict the compression and decompression throughput of algorithms Additionally, models had to correctly attribute data to the algorithm producing the best compression ratios. Machine learning approaches under consideration were Linear Models, Decision Trees and the trivial Mean Value Model as a comparison baseline. It was found, that Decision Trees performed significantly better than Linear Models which in turn were slightly better than the Mean Value approach. Nevertheless, even Decision Trees did not produce a satisfying result which could be reliably used for practical applications. |
| Material | Thesis BibTeX |
Evaluation von alternativen Speicherszenarien für hierarchische Speichersysteme
| Author | Marc Perzborn |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-10-31 |
| Abstract | Ziel der vorliegenden Bachelorarbeit war es, das Simulationsprogramm FeO auf seine Korrektheit zu überprüfen und zu verbessern. Dazu wurden verschiedene Szenarien simuliert. Die Ergebnisse bestätigen zum großen Teil die Annahmen. Im Cache gespeicherte Informationen können schneller Ausgegeben werden, als nicht im Cache gespeicherte. Bei wenig verbauten Laufwerken müssen lesende Anfragen auf nicht gecachte Informationen warten, wenn jedes Laufwerk belegt ist. Das Speichermanagement eines vollen Cache funktioniert einwandfrei. Bei einem Cache mit freiem Speicherplatz wird nicht wie in einem realen System reagiert. Die Verarbeitungszeiten für Anfragen auf nicht gecachte Informationen variiert, wenn verschiedene Komponenten des Bandarchives, beispielsweise die Generation der Laufwerke, die Anzahl der Laufwerke des Bandarchives oder die Bandbreite von Komponenten, verändert werden. |
| Material | Thesis BibTeX |
Quality Control of Meteorological Time-Series with the Aid of Data Mining
| Author | Jennifer Truong |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-10-30 |
| Abstract | This thesis discusses the topic quality controls in the meteorological field and in particular optimize them by adjustment and construction of an automated pipeline for the quality checks. Three different kinds of pipelines are developed through this thesis: The most general one has the focus on high error detection with a low false positive rate. But a categorized pipeline is also designed, which classify the data in “good”, “bad” and “doubtful”. Furthermore a fast fault detection pipeline is derived from the general pipeline to make it possible to react nearline to hardware fails. In this thesis general fundamentals about meteorological coherence, statistical analysis and quality controls for meteorology are described. After that the approach of this thesis are lead by the development of the automated pipeline. Meteorological measurements and their corresponding quality controls got explored to optimize them. Beside an optimization of existing quality controls, new automated tests are developed within this thesis. The evaluation of the designed pipeline shows that the quality of the pipeline depends on the input parameters. The more information we have for the input the better is the pipeline working. But the specialty of the pipeline is that it works with any kind of input, so it is not limited to strict input parameters. |
| Material | Thesis BibTeX |
Characterizing Literature Using Machine Learning Methods
| Author | Jan Bilek |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-10-14 |
| Abstract | In this thesis, we explore the classical works by famous authors available in Project Gutenberg – a free online ebook library. The contemporary computational power enables us to analyze thousands of books and find similarities between them. We explore the differences between books and genres with respect to features such as proportion of stop words, the distribution of part of speech classes or frequencies of individual words. Using this knowledge, we create a model which predicts book metadata, including author or genre, and compare the performance of different approaches. With multinomial naive Bayes model, we reached 74.1 % accuracy on the author prediction task out of more than 1 400 authors. For other metadata, the random forest classifier achieved the best results. Through most predictive features, we try to capture what is typical for individual genres or epochs. As a part of the analysis, we create Character Interactions model that enables us to visualize the interactions between characters in the book and define the main or central character of the book. |
| Material | Thesis BibTeX |
Untersuchung von Interaktiven Analyse- und Visualisierungsumgebungen im Browser für NetCDF-Daten
| Author | Sebastian Rothe |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-07-21 |
| Abstract | Simulations- und Messergebnisse von Klimamodellen umfassen heutzutage oftmals große Datenmengen, die beispielsweise in NetCDF-Dateien als spezielle Datenstrukturen abgelegt werden können. Die Analyse dieser Messergebnisse benötigt meist komplexe und leistungsstarke Systeme, die es dem Nutzer ermöglichen, die Datenmenge an Simulationsergebnissen beispielsweise in tabellarischer Form oder durch grafische Repräsentation anschaulich darzustellen. Moderne Cloud-Systeme bieten dem Nutzer die Möglichkeit, Ergebnisse zu speichern und beispielsweise über das Internet weltweit verfügbar zu machen. Dieses Verfahren hat allerdings auch den Nachteil, dass dazu erst die gesamte Ergebnisdatei aus dem Cloud-System angefordert werden muss, bevor sie analysiert werden kann. Diese Arbeit befasst sich mit der Untersuchung eines alternativen Ansatzes, bei dem es für den Nutzer möglich sein soll, über eine Webanwendung erste Analysen auf serverseitig ausgeführten Werkzeugen durchzuführen, deren Ergebnisse dann im Webbrowser veranschaulicht werden können. Basis dieser ReDaVis (Remote Data Visualizer) genannten Anwendung bilden die Softwaresysteme OpenCPU und h5serv. Die Voranalysen arbeiten auf kleinen Teilmengen der Daten. Sie sollen Aufschluss darüber geben, ob detailliertere Analysen auf dem Gesamtdatensatz lohnenswert sind. Es soll untersucht werden, inwiefern vorhandene Tools diesen Ansatz bereits umsetzen können. Einige dieser Komponenten werden dann verwendet und durch eigene Komponenten ergänzt, um einen Software-Prototyp des vorgestellten Ansatzes entwickeln zu können. Dazu werden zunächst theoretische Grundlagen genauer erläutert, die dann dazu verwendet werden, die eingesetzten Komponenten als Webanwendung zusammenfassen zu können. Die Anwendung unterstützt neben Visualisierungstechniken zur grafischen Repräsentation der Datensätze auch die Möglichkeit, verschiedene aufeinanderfolgende Funktionen in Form einer Pipeline auf einen Datensatz anzuwenden. Es wird gezeigt, inwiefern die unterschiedlichen Konstellationen an Komponenten zusammenarbeiten können oder durch Einschränkungen auf Software- und Hardwareebene ungeeignet sind beziehungsweise mit Blick auf heute weit verbreitete Alternativen nicht leistungsfähig genug arbeiten. |
| Material | Thesis BibTeX |
Automation of manual code optimization via DSL-directed AST-manipulation
| Author | Jonas Gresens |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-06-27 |
| Abstract | Program optimization is a crucial step in the development of performance critical applications but in most cases only manually realizable due to its complexity. The substantial structural changes to the source code reduce the readability and maintainability and complicate the ongoing development of the applications. The objective of this thesis is to examine the advantages and disadvantages of an AST-based solution to the conflicting relationship between performance and structural code quality of a program. For this purpose a prototype is developed to automate usually manual optimizations based on instructions by the user. The thesis covers the design and implementation as well as the evaluation of the prototype for the usage as a tool in software development. As a result this thesis shows the categorical usability of the AST-based approach and the need for further investigation. |
| Material | Thesis BibTeX |
Modeling and Simulation of Tape Libraries for Hierarchical Storage Management Systems
| Author | Jakob Lüttgau |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2016-04-09 |
| Abstract | The wide variety of storage technologies (SRAM, NVRAM, NAND, Disk, Tape, etc.) results in deep storage hierarchies to be the only feasible choice to meet performance and cost requirements when dealing with vast amounts of data. In particular long term storage systems employed by scientific users are mainly reliant on tape storage, as they are still the most cost-efficient option even 40 years after their invention in the mid-seventies. Current archival systems are often loosely integrated into the remaining HPC storage infrastructure. However, data analysis tasks require the integration into the scratch storage systems. With the rise of exascale systems and in situ analysis also burst buffers are likely to require integration with the archive. Unfortunately, exploring new strategies and developing open software for tape archive systems is a hurdle due to the lack of affordable storage silos, the resulting lack of availability outside of large organizations and due to increased wariness requirements when dealing with ultra durable data. Eliminating some of these problems by providing virtual storage silos should enable community-driven innovation, and enable site operators to add features where they see fit while being able to verify strategies before deploying on test or production systems. The thesis asseses moderns tape systems and also puts their development over time into perspective. Subsequently, different models for the individual components in tape systems are developed. The models are then implemented in a prototype simulation using discrete event simulation. It is shown that the simulation can be used to approximate the behavior of tape systems deployed in the real world and to conduct experiments without requiring a physical tape system. |
| Material | Thesis Presentation BibTeX |
2015
Vorhersage von E/A-Leistung im Hochleistungsrechnen unter der Verwendung von neuronalen Netzen
| Author | Jan Fabian Schmid |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2015-12-17 |
| Abstract | Die Vorhersage der Laufzeit von Dateizugriffen im Hochleistungsrechner ist wichtig für die Entwicklung von Analysewerkzeugen, die Wissenschaftler bei der effizienten Nutzung der gegebenen Ressourcen unterstützen können. In dieser Bachelorarbeit wird das parallele Dateisystem eines Hochleistungsrechners analysiert und unter dem Einsatz künstlicher neuronaler Netze werden verschiedene Ansätze zur Modellierung der Ein-/Ausgabe-Leistung entwickelt und getestet. Dabei erreichen die entwickelten künstlichen neuronalen Netze bei der Vorhersage von Zugriffszeiten geringere Modellabweichungen gegenüber den tatsächlichen Zugriffszeiten als lineare Modelle. Es stellt sich heraus, dass der entscheidende Faktor für eine gute Modellierung des Ein-/Ausgabe-Systems darin liegt, zwischen gleichartigen Dateizugriffen, die allerdings zu verschiedenen Zugriffszeiten führen, zu unterscheiden. Die Laufzeitdifferenzen zwischen Dateizugriffen mit gleichen Aufrufparametern können durch die unterschiedliche Verarbeitung im System erklärt werden. Da diese Verarbeitungspfade nicht bekannt oder aus direkt messbaren Attributen ableitbar sind, zeigt sich, dass die Vorhersage der Zugriffszeiten eine nicht triviale Aufgabe ist. Ein Ansatz besteht darin, periodische Verhaltensmuster des Systems auszunutzen, um den Verarbeitungspfad eines Zugriffs vorauszusagen. Dieses periodische Verhalten gezielt für genauere Vorhersagen zu verwenden, erweist sich allerdings als schwierig. Um eine Näherung der Verarbeitungspfade zu bestimmen, wird in dieser Bachelorarbeit ein Verfahren eingeführt, bei dem die Residuen eines Modells zur Erstellung von Klassen genutzt werden, welche wiederum mit den Verarbeitungspfaden korrelieren sollten. Bei der Analyse dieser Klassen können Hinweise auf ihren Zusammenhang mit den Verarbeitungspfaden gefunden werden. So sind Modellierungen, die diese Klassenzuordnungen verwenden, in der Lage, wesentlich genauere Vorhersagen zu machen als andere Modelle. Die Vorhersage der Laufzeit von Dateizugriffen im Hochleistungsrechner ist wichtig für die Entwicklung von Analysewerkzeugen, die Wissenschaftler bei der effizienten Nutzung der gegebenen Ressourcen unterstützen können. In dieser Bachelorarbeit wird das parallele Dateisystem eines Hochleistungsrechners analysiert und unter dem Einsatz künstlicher neuronaler Netze werden verschiedene Ansätze zur Modellierung der Ein-/Ausgabe-Leistung entwickelt und getestet. Dabei erreichen die entwickelten künstlichen neuronalen Netze bei der Vorhersage von Zugriffszeiten geringere Modellabweichungen gegenüber den tatsächlichen Zugriffszeiten als lineare Modelle. Es stellt sich heraus, dass der entscheidende Faktor für eine gute Modellierung des Ein-/Ausgabe-Systems darin liegt, zwischen gleichartigen Dateizugriffen, die allerdings zu verschiedenen Zugriffszeiten führen, zu unterscheiden. Die Laufzeitdifferenzen zwischen Dateizugriffen mit gleichen Aufrufparametern können durch die unterschiedliche Verarbeitung im System erklärt werden. Da diese Verarbeitungspfade nicht bekannt oder aus direkt messbaren Attributen ableitbar sind, zeigt sich, dass die Vorhersage der Zugriffszeiten eine nicht triviale Aufgabe ist. Ein Ansatz besteht darin, periodische Verhaltensmuster des Systems auszunutzen, um den Verarbeitungspfad eines Zugriffs vorauszusagen. Dieses periodische Verhalten gezielt für genauere Vorhersagen zu verwenden, erweist sich allerdings als schwierig. Um eine Näherung der Verarbeitungspfade zu bestimmen, wird in dieser Bachelorarbeit ein Verfahren eingeführt, bei dem die Residuen eines Modells zur Erstellung von Klassen genutzt werden, welche wiederum mit den Verarbeitungspfaden korrelieren sollten. Bei der Analyse dieser Klassen können Hinweise auf ihren Zusammenhang mit den Verarbeitungspfaden gefunden werden. So sind Modellierungen, die diese Klassenzuordnungen verwenden, in der Lage, wesentlich genauere Vorhersagen zu machen als andere Modelle. |
| Material | Thesis Presentation BibTeX |
Automatisches Lernen der Leistungscharakteristika von Paralleler Ein-/Ausgabe
| Author | Eugen Betke |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2015-06-27 |
| Abstract | Die Leistungsanalyse und -optimierung sind seit dem Beginn der elektronischen Datenverarbeitung notwendige Schritte in den Qualitätssicherungs- und Optimierungszyklen. Sie helfen eine qualitative und performante Software zu erstellen. Insbesondere im HPC-Bereich ist dieses Thema wegen der steigender Softwarekomplexität sehr aktuell. Die Leistungsanalysewerkzeuge helfen den Prozess wesentlich zu vereinfachen und zu beschleunigen. Sie stellen die Vorgänge verständlich dar und liefern Hinweise auf mögliche Verbesserungen. Deren Weiterentwicklung und Entwicklung neuer Verfahren ist deshalb essentiell für diesen Bereich. Das Ziel dieser Arbeit ist zu untersuchen, ob E/A-Operationen mit Hilfe von maschinellen Lernen automatisch den richten Cachetypen zugeornet werden können. Zu diesem Zweck werden Methoden entwickelt, die auf den CART-Entscheidungsbäumen und kMeans-Algorithmen basieren und untersucht. Die erhofften Ergebnisse wurden auf diese Weise nicht erreicht. Deswegen werden zum Schluss die Ursachen indentifiziert und diskutiert. |
| Material | Thesis Presentation BibTeX |
Optimization of non-contiguous MPI-I/O Operations
| Author | Enno David Zickler |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2015-01-29 |
| Abstract | High performance computing is an essential part for most science departments. The possibilities expand with increasing computing resources. Lately data storage becomes more and more important, but the development of storage devices can not keep up with processing units. Especially data rates and latencies are enhancing slowly, resulting in efficiency becoming an important topic of research. Programs using MPI provide the possibility to get more efficient by using more information about the file system. In this thesis, advanced algorithms for optimization of non-contiguous MPI-I/O operations are developed by considering well-known system specifications like data rate, latency, or block and stripe alignment, maximum buffer size or the impact of read-ahead-mechanisms. Access patterns combined with these parameters will lead to an adaptive data sieving for non-contiguous I/O operations.The parametrization can be done by machine learning concepts, which will provide the best parameters even for unknown access pattern. The result is a new library called NCT, which provides a view based access on non-contiguous data at a POSIX level. The access can be optimized by data sieving algorithms whose behavior could easily be modified due to the modular design of NCT. Existing data sieving algorithms were implemented and evaluated with this modular design. Hence, the user is able to create new advanced data sieving algorithms using any parameters he regards useful. The evaluation shows many possibilities for where such an algorithm improves the performance. |
| Material | Thesis Presentation BibTeX |
2014
Gröbner bases and combinatorics for binary codes
| Author | Zoya Masih |
| Type | Master's Thesis |
| Advisors | Hossein Sabzrou |
| Date | 2014-09-21 |
| Abstract | In this dissertation, based on the paper “M. Borges-Quantana, M.A. Borges-Ternard, P. Fitzpatrick, and E. Martínez-Moro, Gröbner bases and combinatorics for binary codes AAECC 19 (2006) 393-411”, we introduce a binomial ideal derived from a binary linear code. We present some applications of Gröbner basis of this ideal with respect to an ordering compatible with the total degree. Finally we discuss some algorithms for computing the Gröbner basis. |
| Material | BibTeX |
Halbautomatische Überprüfung von kollektiven MPI-Operationen zur Identifikation von Leistungsinkonsistenzen
| Author | Sebastian Rothe |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2014-04-09 |
| Abstract | Computersimulationen werden heutzutage vermehrt dazu genutzt, wissenschaftliche Experimente in virtuellen Umgebungen auszuführen. Um die Ausführungsdauer zu re- duzieren, werden parallele Programme entwickelt, die auf Rechenclustern ausgeführt werden. Programme, die auf mehrere Computersysteme verteilt sind, nutzen meist den MPI-Standard (Message Passing Interface), um den Nachrichtenaustausch zwischen den Rechnern realisieren zu können. Aufgrund des komplexen Aufbaus der Rechencluster wird die verfügbare Hardware allerdings oftmals nicht ideal ausgenutzt. Es existiert damit Optimierungspotential, das genutzt werden kann, um die Laufzeit der Applikationen weiter zu verringern. Leistungsanalysen bilden hierbei die Basis, um Schwachstellen im System oder in den genutzten MPI-Implementationen aufzudecken und sie später zu optimieren. Diese Arbeit befasst sich mit der Entwicklung des Analysewerkzeugs pervm (performance validator for MPI), das sich auf die Untersuchung der kollektiven Operationen von MPI konzentriert und dadurch Leistungsinkonsistenzen aufdecken soll. Dafür werden theoretische Grundlagen genauer erläutert, die dann dazu verwendet werden, das Zusammenspiel der benötigten Komponenten des Analysewerkzeugs zu erklären. Die Ausführung von pervm lässt sich in die Mess- und die Auswertungsphase unterteilen. Es können die Ausführungszeiten der eigentlichen MPI-Operation sowie verschiedener Algorithmen, die unterschiedlich effiziente Ausführungsmöglichkeiten einer kollektiven Operation beschreiben, ermittelt werden. Neben der Analyse dieser Messergebnisse bietet die Auswertungsphase des Werkzeugs zusätzlich die Möglichkeit, die theoretische Ausführungsdauer eines Algorithmus auf einem gegebenen System anhand dessen Leistungswerte zu simulieren. Die beschriebenen Ausführungsmöglichkeiten liefern zahlreiche Ansätze zur Identifikation von Leistungsengpässen. Es wird gezeigt, inwiefern bei der Verwendung der kollektiven MPI-Operation Rückschlüsse auf den genutzten Algorithmus gezogen werden können. Referenzalgorithmen mit kürzeren Ausführungszeiten im Vergleich zur MPI-Operation liefern Hinweise auf weitere Inkonsistenzen in der Implementation der genutzten MPI-Bibliothek. |
| Material | Thesis BibTeX |
Analyse und Optimierung von nicht-zusammenhängende Ein-/Ausgabe in MPI
| Author | Daniel Schmidtke |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Michaela Zimmer |
| Date | 2014-04-07 |
| Abstract | Das Ziel dieser Arbeit ist es, das Potential von Datasieving zu evaluieren und in Optimierungen nutzbar zu machen. Dazu werden die folgenden Ziele definiert. 1. Systematische Analyse der erzielbaren Leistung. 2. Transparente Optimierung. 3. Kontextsensitive Optimierung. |
| Material | Thesis BibTeX |
Automatic Analysis of a Supercomputer's Topology and Performance Characteristics
| Author | Alexander Bufe |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2014-03-18 |
| Abstract | Although knowing the topology and performance characteristics of a supercomputer is very important as it allows for optimisations and helps to detect bottleneck, no universal tool to determine topology and performance characteristic is available yet. Existing tools are often specialised to analyse either the behaviour of a node or of the network topology. Furthermore, existing tools are unable to detect switches despite their importance. This thesis introduces an universal method to determine the topology (including switches) and an efficient way to measure the performance characteristics of the connections. The approach of the developed tool is to measure the latencies first and then to compute the topology by analysing the data. In the next step, the gained knowledge of the topology is used to parallelise the measurement of the throughput in order to decrease the required time or to allow for more accurate measurements. A general approach to calculate latencies of connections that cannot be measured directly based on linear regression is introduced, too. At last, the developed algorithm and measurement techniques are validated on several test cases and a perspective of future work is given. |
| Material | Thesis BibTeX |
Flexible Event Imitation Engine for Parallel Workloads
| Author | Jakob Lüttgau |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2014-03-18 |
| Abstract | Evaluating systems and optimizing applications in high-performance computing (HPC) is a tedious task. Trace files, which are already commonly used to analyse and tune applications, also serve as a good approximation to reproduce workloads of scientific applications. The thesis presents design considerations and discusses a prototype implementation for a flexible tool to mimic the behavior of parallel applications by replaying trace files. In the end it is shown that a plugin based replay engine is able to replay parallel workloads that use MPI and POSIX I/O. It is further demonstrated how automatic trace manipulation in combination with the replay engine allows to be used as a virtual lab. |
| Material | Thesis BibTeX |
2013
Simulation of Parallel Programs on Application and System Level
| Author | Julian Kunkel |
| Type | PhD Thesis |
| Advisors | Thomas Ludwig |
| Date | 2013-07-30 |
| Abstract | Computer simulation revolutionizes traditional experimentation providing a virtual laboratory. The goal of high-performance computing is a fast execution of applications since this enables rapid experimentation. Performance of parallel applications can be improved by increasing either capability of hardware or execution efficiency. In order to increase utilization of hardware resources, a rich variety of optimization strategies is implemented in both hardware and software layers. The interactions of these strategies, however, result in very complex systems. This complexity makes assessing and understanding the measured performance of parallel applications in real systems exceedingly difficult. To help in this task, in this thesis an innovative event-driven simulator for MPI-IO applications and underlying heterogeneous cluster computers is developed which can help us to assess measured performance. The simulator allows conducting MPI-IO application runs in silico, including the detailed simulations of collective communication patterns, parallel I/O and cluster hardware configurations. The simulation estimates the upper bounds for expected performance and therewith facilitates the evaluation of observed performance. In addition to the simulator, the comprehensive tracing environment HDTrace is presented. HDTrace offers novel capabilities in analyzing parallel I/O. For example, it allows the internal behavior of MPI and the parallel file system PVFS to be traced. While PIOsimHD replays traced behavior of applications on arbitrary virtual cluster environments, in conjunction with HDTrace it is a powerful tool for localizing inefficiencies, conducting research on optimizations for communication algorithms, and evaluating arbitrary and future systems. This thesis is organized according to a systematic methodology which aims at increasing insight into complex systems: The information provided in the background and related-work sections offers valuable analyses on parallel file systems, performance factors of parallel applications, the Message Passing Interface, the state-of-the-art in optimization and discrete-event simulation. The behavior of memory, network and I/O subsystem is assessed for our working group's cluster system, demonstrating the problems of characterizing hardware. One important insight of this analysis is that due to interactions between hardware characteristics and existing optimizations, performance does not follow common probability distributions, leading to unpredictable behavior of individual operations. The hardware models developed for the simulator rely on just a handful of characteristics and implement only a few optimizations. However, a high accuracy of the developed models to explain real world phenomenons is demonstrated while performing a careful qualification and validation. Comprehensive experiments illustrate how simulation aids in localizing bottlenecks in parallel file system, MPI and hardware, and how it fosters understanding of system behavior. Additional experiments demonstrate the suitability of the novel tools for developing and evaluating alternative MPI and I/O algorithms. With its power to assess the performance of clusters running up to 1,000 processes, PIOsimHD serves as virtual laboratory for studying system internals. In summary, the combination of the enhanced tracing environment and a novel simulator offers unprecedented insights into interactions between application, communication library, file system and hardware. |
| Material | Thesis BibTeX URL |
2012
Effiziente Verarbeitung von Klimadaten mit ParStream
| Author | Moritz Lahn |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2012-06-28 |
| Abstract | In Zusammenarbeit mit der ParStream GmbH wird in dieser Arbeit untersucht in wieweit sich die von ParStream entwickelte Datenbank zur effizienteren Verarbeitung von Klimadaten nutzen lässt. Für die Auswertung der Klimadaten verwenden Wissenschaftler oftmals das Climate Data Operators Programm (CDO). Das CDO Programm ist eine Sammlung von vielen Operatoren zur Auswertung von Daten die von Klimasimulationen bzw. Erd-System Modellen stammen. Die Auswertung mit diesem Programm ist sehr zeitintensiv. Dieser Ausgangspunkt begründet die Motivation zur Nutzung der ParStream Datenbank, die mit einem eigens entwickelten spaltenorientierten Bitmap Index und einer komprimierten Indexstruktur, Anfragen an eine große Datenbasis parallel und sehr effizient verarbeiten kann. Mit dem beschleunigten Abruf der Daten eröffnen sich neue Möglichkeiten im Bereich der Echtzeit-Analyse, die bei der interaktiven Visualisierung von Klimadaten hilfreich sind. Als Ergebnis dieser Arbeit wird untersucht welche CDO Operatoren mit der ParStream Datenbank umsetzbar sind. Einige Operatoren werden zu Demonstrationszwecken mit der ParStream Datenbank umgesetzt. Die Leistungsvorteile werden durch Tests verifiziert und zeigen eine effizientere Verarbeitung von Klimadaten mit der ParStream Datenbank. Es hat sich herausgestellt, dass ParStream bei einigen Operatoren die Ergebnisse zwischen 2x und 20x schneller ausliefern kann als das CDO Programm. Als ein weiteres Ergebnis stellte sich bei der Klassifizierung der CDO Operatoren heraus, dass die meisten Operationen direkt durch SQL abgebildet werden können. Der Industriepartner stimmt einer Veröffentlichung des PDFs nicht zu. |
| Material | BibTeX |
Replay Engine for Application Specific Workloads
| Author | Jörn Ahlers |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel |
| Date | 2012-04-12 |
| Abstract | Today many tools exist which are related to the processing of workloads. All of these have their specific area where they are used. Despite their differences they also have functions regarding the creation and execution of workloads in common. To create a new tool it is always needed to implement all of these functions even when they were implemented before in another tool. In this thesis a framework is designed and implemented that allows replaying of application specific work-loads. This gets realized through a modular system which allows to use existing modules in the creation of new tools to reduce development work. Additionally a function is designed to generate parts of the modules by their function headers to further reduce this work. To improve the generation, semantical information can be added through comments to add advanced behavior. To see that this approach is working examples are given which show the functionality and evaluate the overhead created through the software. Finally additional work that can be done to further improve this tool is shown. |
| Material | Thesis BibTeX |
2010
Crossmedia File System MetaFS -- Exploiting Performance Characteristics from Flash Storage and HDD
| Author | Leszek Kattinger |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Olga Mordvinova |
| Date | 2010-03-23 |
| Abstract | Until recently, the decision which storage device is most suitable, in aspects of costs, capacity, performance and reliability has been an easy choice. Only hard disk devices offered requested properties. Nowadays rapid development of flash storage technology, makes these devices competitive or even more attractive. The great advantage of flash storage is, apart from lower energy consumption and insensitivity against mechanical shocks, the much lower access time. Compared with hard disks, flash devices can access data about a hundred times faster. This feature enables a significant performance benefit for random I/O operations. Unfortunately, the situation at present is that HDDs provide a much bigger capacity at considerable lower prices than flash storage devices, and this fact does not seem to be changing in the near future.Considering also the wide-spread use of HDDs, the continuing increase of storage density and the associated increase of sequential I/O performance, the incentive to use HDDs will continue. For this reason, a way to combine both storage technologies seems beneficial. From the view of a file system, meta data is often accessed randomly and very small, in contrast a logical file might be large and is often accessed sequentially. Therefore, in this thesis a file system is designed and implemented which places meta data on an USB-flash device and data on an HDD. The design also considers, how meta data operations can be optimized for a cheep low-end USB flash device, which provide flash media like fast access times but also characteristic low write rates, caused by the block-wise erase-before-write operating principle. All measured file systems show a performance drop for meta data updates on this kind of flash devices, compared with their behavior on HDD. Therefore the design focused on the possibility to map coherent logical name space structures (directories) close to physical media characteristics (blocks). To also check impacts by writes of data sizes equal or smaller then the selected block size, the option to write only full blocks or current block fill rates was given. The file system was implemented in the user space and operate through the FUSE interface. Despite of the overhead caused by this fact, the performance of write associated meta data operations (like create/remove) was better or equal than of those file systems used for benchmark comparison. |
| Material | Thesis BibTeX Sources |
2009
Tracing Internal Behavior in PVFS
| Author | Tien Duc Tien |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2009-10-05 |
| Abstract | Nowadays scientific computations are often performed on large cluster systems because of the high performance they deliver. In such systems there are many reasons for bottlenecks which are related to both hardware and software. This thesis defines and implements metrics and information used for tracing events in MPI applications in conjunction with the parallel file system PVFS in order to localize bottlenecks and determine system behavior. They are useful for the optimizations of the system or applications. After tracing, data is stored in trace files and can be analyzed via the visualization tool Sunshot. There are two experiments made in this thesis. The first experiment is made on a balanced system. In this case Sunshot shows a balanced visualization between nodes, i.e. the load between nodes looks similar. Moreover, in connection with this experiment the new metrics and tracing information or characteristics are discussed in detail in Sunshot. In contrast, the second experiment is made on an unbalanced system. In this case Sunshot shows where bottlenecks occurred and components which are related. |
| Material | Thesis BibTeX |
Simulation-Aided Performance Evaluation of Input/Output Optimizations for Distributed Systems
| Author | Michael Kuhn |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2009-09-30 |
| Material | Thesis BibTeX URL |
Design and Implementation of a Profiling Environment for Trace Based Analysis of Energy Efficiency Benchmarks in High Performance Computing
| Author | Stephan Krempel |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2009-08-31 |
| Material | Thesis BibTeX |
Model and simulation of power consumption and power saving potential of energy efficient cluster hardware
| Author | Timo Minartz |
| Type | Master's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2009-08-27 |
| Material | Thesis BibTeX URL |
2008
Ergebnisvisualisierung paralleler Ein/Ausgabe Simulation im Hochleistungsrechnen
| Author | Anton Ruff |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2008-05-31 |
| Material | BibTeX |
2007
Container-Archiv-Format für wahlfreien effizienten Zugriff auf Dateien
| Author | Hendrik Heinrich |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2007-09-30 |
| Material | Thesis BibTeX |
Directory-Based Metadata Optimizations for Small Files in PVFS
| Author | Michael Kuhn |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2007-09-03 |
| Material | Thesis BibTeX URL |
Towards Automatic Load Balancing of a Parallel File System with Subfile Based Migration
Benchmarking of Non-Blocking Input/Output on Compute Clusters
| Author | David Büttner |
| Type | Bachelor's Thesis |
| Advisors | Dr. Julian Kunkel, Thomas Ludwig |
| Date | 2007-04-24 |
| Material | Thesis BibTeX URL |