Geometric Approaches to Big Data Modeling and Performance Prediction

Show simple item record

dc.contributor Helsingin yliopisto, Matemaattis-luonnontieteellinen tiedekunta, Tietojenkäsittelytieteen laitos fi
dc.contributor University of Helsinki, Faculty of Science, Department of Computer Science en
dc.contributor Helsingfors universitet, Matematisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap sv
dc.contributor.author Goetsch, Peter
dc.date.issued 2018
dc.identifier.uri URN:NBN:fi-fe201804208657
dc.identifier.uri http://hdl.handle.net/10138/273494
dc.description.abstract Big Data frameworks (e.g., Spark) have many configuration parameters, such as memory size, CPU allocation, and the number of nodes (parallelism). Regular users and even expert administrators struggle to understand the relationship between different parameter configurations and the overall performance of the system. In this work, we address this challenge by proposing a performance prediction framework to build performance models with varied configurable parameters on Spark. We take inspiration from the field of Computational Geometry to construct a d-dimensional mesh using Delaunay Triangulation over a selected set of features. From this mesh, we predict execution time for unknown feature configurations. To minimize the time and resources spent in building a model, we propose an adaptive sampling technique to allow us to collect as few training points as required. Our evaluation on a cluster of computers using several workloads shows that our prediction error is lower than the state-of-art methods while having fewer samples to train. en
dc.language.iso eng
dc.publisher Helsingin yliopisto fi
dc.publisher University of Helsinki en
dc.publisher Helsingfors universitet sv
dc.title Geometric Approaches to Big Data Modeling and Performance Prediction en
dc.type.ontasot pro gradu -tutkielmat fi
dc.type.ontasot master's thesis en
dc.type.ontasot pro gradu-avhandlingar sv
dct.identifier.urn URN:NBN:fi-fe201804208657

Files in this item

Total number of downloads: Loading...

Files Size Format View
Peter Goetsch Master's Thesis 6-June-2018.pdf 923.0Kb PDF View/Open

This item appears in the following Collection(s)

Show simple item record