November 30, 2018
11:00 - 12:00
Louvain-la-Neuve
Paul Otlet Room - Réaumur building a.327
Towards self-managed, re-configurable streaming dataflow systems
By Vasiliki (Vasia) Kalavri, postdoctoral fellow at the Systems Group, ETH Zurich.
Next-generation stream processing systems will not only be scalable and reliable, but also autonomous, flexible, and able to automatically re-configure running applications without downtime. Automatic re-configuration will allow stream processors to dynamically mitigate skew and stragglers, switch execution plans on-the-fly, vary resource allocation, and support code updates without the need to halt and restart applications.
In this talk I will share my research group’s recent work in this area. I will present SnailTrail (NSDI’18), an online critical path analysis module that detects bottlenecks and provides insights on streaming application performance, and DS2 (OSDI’18), an automatic scaling controller for streaming systems which uses lightweight instrumentation to estimate the true processing and output rates of individual dataflow operators.
Vasia is a postdoctoral fellow at the Systems Group, ETH Zurich, where she is working on distributed stream processing and large-scale graph analytics. She did her PhD at KTH, Stockholm, and UCLouvain, Belgium, where she was an EMJD-DC fellow. Her thesis, "Performance Optimization Techniques and Tools for Distributed Graph Processing" received the IBM Innovation Award 2017 by FNRS. During her PhD, she also spent time at DIMA TU Berlin, Telefonica Research Barcelona, and data Artisans. Vasia is a PMC member of Apache Flink and a core developer of its graph processing API, Gelly.