SPEAr: Expediting Stream Processing with Accuracy Guarantees
DocUID: 2020-008 Full Text:
PDF Author: Nikos R. Katsipoulakis, Alexandros Labrinidis, Panos K. Chrysanthis
Abstract: Stream Processing Engines (SPEs) are used for realtime and continuous processing with stateful operations. This type of processing poses numerous challenges due to its associated complexity, unpredictable input, and need for timely results. As a result, users tend to overprovision resources, and online scaling is required in order to overcome overloaded situations. Current attempts for expediting stateful processing are impractical, due to their inability to uphold the quality of results, maintain performance, and reduce memory requirements. In this paper, we present the SPEAr system, which can expedite processing of stateful operations automatically by trading accuracy for performance. SPEAr detects when it can accelerate processing by employing online sampling and accuracy estimation at no additional cost. We built SPEAr on top of Storm and our experiments indicate that it can reduce processing times by more than an order of magnitude, use more than an order of magnitude less memory, and offer accuracy guarantees in real-world benchmarks.
Keywords: Watermarking, Buffer Storage, Storms, Real-time systems, Memory Management, Manuals, Silicon, data handling, memory requirements, SPEAr system, stateful operations, online sampling, stream processing engines, SPEs, online scaling, stateful processing
Published In: 36th International Conference on Data Engineering
ISBN: 978-1-7281-2903-7
Pages: 1105-1116
Place Published: Dallas, Texas
Year Published: 2020
DOI: 10.1109/ICDE48307.2020.00100
Project: SPEAr Subject Area: Memory Management, Stream Processing
Publication Type: Conference Paper
Sponsor: Others