Audi‘s Hadoop Journey into the Hybrid Cloud

Audi‘s Hadoop Journey into the Hybrid Cloud

Wednesday, March 20
11:50 AM - 12:30 PM
Room 124-125

After having run Hadoop on-premise in production for some time, we decided to build a Hadoop platform in AWS to extend the on-premise Hadoop cluster to a hybrid platform.

In this presentation, we first briefly state our motivation and requirements for building a cloud platform. Moving to the cloud not only offers new technical possibilities, it also helped us to make your way of working more agile. We explain how we setup a team of internal and external experts, defined an agile working mode and how this approach worked for us.

Then we explain our choice of tools (Terraform, Ambari Blueprints) and how we used them to automate the deployment of our cloud platform. We also take a quick glance into Cloudbreak. We share what worked well but we also mention some of the pitfalls we encountered.

Finally we look into how we connected the cloud and on-premise platform to build a hybrid platform. Here we look especially into user management and our Kerberos setup. We conclude our presentation by giving a glimpse into the next steps on our roadmap.

Presentation Video

SPEAKERS

Carsten Herbe
Big Data Architect
Audi Business Innovation GmbH
Carsten works as a Big Data Architect at Audi Business Innovation GmbH. Audi Business Innovation GmbH, a subsidiary of Audi, is a small company focused on developping new mobility services as well as innovative IT solutions for Audi. Carsten has more than 10 year experience in delivering Data Warehouse and BI solutions to his customers. He started working with Hadoop in 2013 and since then he has focused on both big data infrastructure and solutions. Currently Carsten is helping Audi to extend their Big Data platform based on Hadoop and Kafka to the cloud. Further, as an solution architect he is responsible for developing and running analytical applications on that platform.