Enabling Real-Time Business Intelligence

September 5, 2016 - New Delhi, India

Celebrating 10 years of bridging Academic and Industrial Innovation!

Invited Talks

Data Systems that are Easy to Design, Tune and Use in Real-Time

By Stratos Idreos

How far away are we from a future where a data management system sits in the critical path of everything we do? Already today we often go through a data system in order to do several basic tasks, e.g., to pay at the grocery store, to book a flight, to find out where our friends are and even to get coffee. Businesses and sciences are increasingly recognizing the value of storing and analyzing vast amounts of data. Other than the expected path towards an exploding number of data-driven businesses and scientific scenarios in the next few years, in this talk we also envision a future where data becomes readily available and its power can be harnessed by everyone in real-time. What both scenarios have in common is a need for new kinds of data systems that “just work”; they are easy to design, easy to tune, and easy to use in real-time. We will discuss this vision and our recent efforts towards 1) adaptive data systems that can adapt to data and access patterns on-the-fly, 2) self-designing data systems that make it easy to spin-off and test new data system architectures in near real-time and 3) curious data systems that make it easy to explore data in real-time even if we do not know what queries to ask.

About the speaker:

Stratos Idreos

Stratos Idreos is an assistant professor of Computer Science at Harvard University where he leads DASlab, the Data Systems Laboratory@Harvard SEAS. Stratos works on data system architectures with emphasis on how we can make it easy to design efficient data systems as applications and hardware keep evolving and on how we can make it easy to use these systems even for non-experts. For his doctoral work on Database Cracking, Stratos won the 2011 ACM SIGMOD Jim Gray Doctoral Dissertation award and the 2011 ERCIM Cor Baayen award. He is also a recipient of an IBM zEnterpise System Recognition Award, a VLDB Challenges and Visions best paper award and an NSF Career award. In 2015 he was awarded the IEEE TCDE Early Career Award from the IEEE Technical Committee on Data Engineering for his work on adaptive data systems.

Being Smart: The Role of Timely Analytics

By Krithi Ramamritham

These days, unless something has the epithet "smart" attached to it, it is nothing. Smart Energy solutions promise cleaner, cheaper and more reliable energy. Smart Cities promise better quality of life for its citizens. We will argue that for a "system" to be SMART, it should Sense Meaningfully, Analyze and Respond Timely. Using real-world examples from the domains of Smart Energy and Smart Cities, this talk will illustrate the central role of data in being SMART.

About the speaker:

Krithi Ramamritham

Prof. Krithi Ramamritham holds a Ph.D. degree in Computer Science from the University of Utah. He did his B.Tech (Electrical Engineering) and M.Tech (Computer Science) degrees from IIT Madras. After a long stint at the University of Massachusetts, he moved to IIT Bombay as the Vijay and Sita Vashee Chair Professor in the Department of Computer Science and Engineering. During 2006-2009, he served as Dean (R&D) at IIT Bombay. He currently heads IIT Bombay's new Center for Urban Science and Engineering. Prof. Ramamritham's research explores timeliness and consistency issues in computer systems, in particular, databases, real-time systems, and distributed applications. His recent work addresses these issues in the context of sensor networks, embedded systems, mobile environments and smart grids. During the last few years he has been interested in the use of Information and Communication Technologies for creating tools aimed at socio-economic development.

He is a Fellow of the IEEE, ACM, Indian Academy of Sciences, National Academy of Sciences, India, and the Indian National Academy of Engineering. He was honored with a Doctor of Science (Honoris Causa) by the University of Sydney. He is also a recipient of the Distinguished Alumnus Award from IIT Madras. Twice he received the IBM Faculty Award. He just received the 2015 Outstanding Technical Contributions and Leadership Award from the IEEE Technical Committee for Real-Time Systems and IEEE's CEDA Outstanding Service Award. He has been associated with the editorial board of various journals. These include IEEE Embedded Systems Letters and Springer's Real-Time Systems Journal (Editor-in-Chief), IEEE Transactions on Knowledge and Data Engineering, IEEE Transactions on Parallel and Distributed Systems, IEEE Transactions on Mobile Computing, IEEE Internet Computing, ACM Computing Surveys and the VLDB (Very Large Databases) Journal.

Ubiq: A Scalable and Fault-tolerant Log Processing Infrastructure

By Manpreet Singh

Most of today’s Internet applications generate vast amounts of data (typically, in the form of event logs) that needs to be processed and analyzed for detailed reporting, enhancing user experience and increasing monetization. In this paper, we describe the architecture of Ubiq, a geographically distributed framework for processing continuously growing log files in real time with high scalability, high availability and low latency. The Ubiq framework fully tolerates infrastructure degradation and data center-level outages without any manual intervention. It also guarantees exactly-once semantics for application pipelines to process logs as a collection of multiple events. Ubiq has been in production for Google’s advertising system for many years and has served as a critical log processing framework for several dozen pipelines. Our production deployment demonstrates linear scalability with machine resources, extremely high availability even with underlying infrastructure failures, and an end-to-end latency of under a minute.

This was in collaboration with Ashish Gupta, Haifeng Jiang, Venkatesh Basker, Manish Bhatia, Vinny Ganeshan, Shan He, Scott Holzer, Monica Lenart, Navin Melville, Shivakumar Venkataraman, Tianhao Qiu, Namit Sikka, Alexander Smolyanov, Yuri Vasilevski, and Divy Agrawal

About the speaker:

Manpreet Singh

Manpreet is the Uber Technical Lead for Data Processing in Data Infrastructure and Analysis team at Google. He has conceptualized, designed, implemented and launched multiple large-scale distributed systems, such as Photon and Ubiq, during the last 10 years at Google. Prior to that, he earned PhD in Computer Science at Cornell University.