Using Ranger and Atlas in the Real World

Using Ranger and Atlas in the Real World

Thursday, March 21
11:40 AM - 12:30 PM
Room 124-125

What's it like to implement Ranger and Atlas on a data lake? As a data custodian I have implemented Ranger to protect sensitive data and to make sure our raw data is not altered. I've created all kinds of policies to do that. But I also gained a lot of experience using the Ranger REST API. In this session I will tell how to work with that.

Atlas is a nice product to store metadata. I call it the search engine for metadata in your data lake. But what kind of metadata do you store in it? What is useful? And who needs to access it? Atlas also comes with a REST API, which was a little harder to figure out, but in the end I managed to do a lot with it. In this session I'll explain how to use it as well.

I've used Ranger and Atlas for a year now, so expect a lot of insights for beginners as well as intermediate users.

SPEAKERS

Marcel-Jan Krijgsman
Data Custodian
Port of Rotterdam
I am a data custodian and data engineer at Port of Rotterdam. It is my job to protect data in our data lake and to make sure, that when users may see it, it can easily be found. We use Ranger for security and Atlas to store metadata. I've familiarized myself quite a bit with these two products. Not only using the UI, but also very much with the REST API of both products. It hadn't always been like this. In 2016 I decided to leave my 20 years of Oracle database experience behind to learn Big Data and become a data engineer. While learning, I started making Youtube videos on things I learned along the way. I experimented with open source products, like Hadoop, MongoDB and ElasticSearch, and showed my work at my Youtube channel (https://www.youtube.com/channel/UCVt-roCRXgsNtIb0WjfTa4g).