CONFLuEnCE: Implementation and Application Design
DocUID: 2011-005 Full Text:
PDF Author: Panayiotis Neophytou, Panos K. Chrysanthis, Alexandros Labrinidis
Abstract: Data streams have become pervasive and data production rates are increasing exponentially, driven by advances in technology, for example the proliferation of sensors, smart phones, and their applications. This fact effectuates an unprecedented opportunity to build real-time monitoring and analytics applications, which when used collaboratively and interactively, will provide insights to every aspect of our environment, both in the business and scientific domains. In our previous work, we have identified the need for workflow management systems which are capable of orchestrating the processing of multiple heterogeneous data streams, while enabling their users to interact collaboratively with the workflows in real time. In this paper, we describe CONFLuEnCE (CONtinuous workFLow ExeCution Engine), which is an implementation of our continuous workflow model. CONFLuEnCE is built on top of Kepler, an existing workflow management system, by fusing stream semantics and stream processing methods as another computational domain. Furthermore, we explicate our experiences in designing and implementing real-life business and scientific continuous workflow monitoring applications, which attest to the ease of use and applicability of our system.
Keywords: workflow, continuous workflows, data streams, monitoring applications
Published In: Proc. of the 7th International Conference on Collaborative Computing: Networking, Applications and Worksharing
Pages: pp. 181-190
Place Published: Orlando, Florida
Year Published: 2011
Project: AstroSelf, CONFLuEnCE Subject Area: Workflow Management, Data Integration, Data Streams
Publication Type: Conference Paper
Sponsor: NSF IIS-0534531, NSF OIA-1028162