Bringing complex event processing to Spark streaming

Thursday, April 19
2:00 PM - 2:40 PM
Room I

Complex event processing (CEP) is about identifying business opportunities and threats in real time by detecting patterns in data and taking appropriate automated action. Example business use cases for CEP include location-based marketing, smart inventories, targeted ads, Wi-Fi offloading, fraud detection, churn prediction, fleet management, predictive maintenance, security incident event management, and many more. While Spark Streaming provides a distributed resilient framework for ingesting events in real time, effort is still needed to build CEP applications. This is because CEP use cases require correlation of events, which in turn requires us to treat every incoming event as a discrete occurrence in time. Spark Streaming treats the entire batch of events as single occurrence. Many CEP use cases also require alerts to be fired even when there is no incoming event. An example of such use case is to fire an alert when an order-shipped event is NOT received within the SLA times following an order-received event. At Oracle we have adopted a few neat techniques like running continuous query engines as long running tasks, using empty batches as triggers, etc. to bring complex event processing to Spark Streaming.

Join us to learn more on CEP for Spark, the fastest growing data processing platform in the world.

Presentation Video

SPEAKERS

Prabhu Thukkaram
Senior Director, Product Development
Oracle
PRABHU THUKKARAM Senior Director, Product Development Oracle USA Inc Prabhu Thukkaram heads engineering for Oracle Stream Analytics cloud service and manages software development for the product in U.S., Hungary, India, and China. As product owner and engineering lead he provides technical evangelism and design guidance for the Oracle Stream Analytics product. In addition to managing a large technical team in diverse geographies, Prabhu is an expert in distributed systems, service oriented architectures, data integration, big data analytics, and business intelligence software. Prabhu has more than 24 years of experience in building enterprise software products, beginning his career at a startup in India as Software Engineer where he and his team built a platform to emulate expensive IBM mainframe environments on low-cost Unix systems. Subsequently Prabhu joined Oracle in 1993 contributing to many of Oracle’s Analytics products. He has a Masters in Computer Science from University of Colorado and Masters in Business Administration from University of Santa Clara. Prabhu also has several patents in the areas of stream processing and distributed systems and speaks every year at Oracle Open World.
Hoyong Park
Architect
Oracle, America
Hoyong is an Architect at Oracle where he leads development of the Stream Analytics Plaform focusing on architecture and design for the next generation self-service Stream Analytics cloud service. He has over 25 years of software engineering experience and has been working on Real-time Streaming and Business Intelligence products at Oracle for last 13 years. He has wide experience in developing highly distributed systems and scalable technologies. Before entering the industry, he spent three years in a PhD program at Korea Advanced Institute of Science and Technology, primarily researching scientific and high-performance computing. He is an expert in geo-spatial technologies and has presented at several symposiums. Hoyong also holds 17+ patents and has been a primary inventor in areas of stream processing, distributed computing, and geo spatial processing.