Omid: scalable and highly available transaction processing for Apache Phoenix

Omid: scalable and highly available transaction processing for Apache Phoenix

Tuesday, June 19
11:50 AM - 12:30 PM
Executive Ballroom 210B/F

Apache Phoenix is an OLTP and operational analytics for Hadoop. To ensure operations correctness, Phoenix requires that a transaction processor guarantees that all data accesses satisfy the ACID properties. Traditionally, Apache Phoenix has been using the Apache Tephra transaction processing technology. Recently, we introduced into Phoenix the support for Apache Omid—an open source transaction processor for HBase that is used at Yahoo at a large scale.

A single Omid instance sustains hundreds of thousands of transactions per second and provides high availability at zero cost for mainstream processing. Omid, as well as Tephra, are now configurable choices for the Phoenix transaction processing backend, being enabled by the newly introduced Transaction Abstraction Layer (TAL) API. The integration requires introducing many new features and operations to Omid and will become generally available early 2018.

In this talk, we walk through the challenges of the project, focusing on the new use cases introduced by Phoenix and how we address them in Omid.

Presentation Video


Ohad Shacham
Senior Research Scientist
Yahoo Research, Oath
Ohad Shacham is Senior Research Scientist at Yahoo Research. He works on scalable big data and search platforms. Most recently, he focused on extending the Omid transaction processing system with high availability and integrating Omid with Apache Phoenix. Ohad received his PhD in concurrent software verification from Tel-Aviv University CS in 2012. Prior to Yahoo, Ohad lead the SAT based formal verification activities at IBM Research and worked on automatic software vectorization at Intel.
James Taylor