Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP Databases

Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP Databases

Thursday, May 23
2:00 PM - 2:40 PM
Marquis Salon 12

At Walmart Labs, we get close to 200 million customers every week across our 11000+ stores & online all over the world. As part of our data lake initiatives, we started a full-fledged migration to Hadoop based solutions for all our data needs at lower cost than traditional RDBMS/MPP solutions. While we have seen significant success in migrating to Hadoop based Data Lake solutions from traditional RDBMS based data warehouses, one challenge that we have faced is around migrating end users to Hadoop due to query latency issues. To solve this problem and to reduce the cost of the solution, Walmart Labs started using Hive LLAP.

In this session, we will introduce you to Hive LLAP, its architecture, best practices for deployment to achieve sub-second query performance and its cost comparison with traditional RDBMS systems for the same use case.

SPEAKERS

Naveen Peddamail
Sr. Manager - Software Engineering
WalmartLabs
I am an Engineering Manager with 6+ Years of Data Engineering, Data Science & Analytics Experience across Retail, Finance & Marketing industries. In my current role at WalmartLabs, I lead the engineering team for the customer experience domain and we have been primarily working on a data lake initiative to bring all the various data assets at Walmart under one single platform. I have worked across various database systems and have been extensively working on Hadoop stack in my past 4 years at Walmart. As part of our data lake migration, I worked building end to end data platforms on Hadoop and also worked on evaluating multiple Query acceleration layers on top of Hadoop and as part of this we have evaluated Druid, LLAP, Spark, Kinetica etc to power our BI platforms. My Speaking & Presentation Experience: @NWA IISE conference: Topic: 'Data Cafe: Enabling Real Time Insights Through Visualization' @Bentonville Data Science Meetup: ' Data Cafe: Ask Me Anything - Bot Framework using NLP' My work at WalmartLabs was featured on Forbes as one of the "The Most Practical Big Data Use Cases Of 2016" https://www.forbes.com/sites/bernardmarr/2016/08/25/the-most-practical-big-data-use-cases-of-2016/#1a1206531625
Abhishek Gupta
Software Engineer - 2
Walmart
This is Abhishek Gupta with around 5 years of professional experience in IT Industry, currently working in Walmart Labs as a Software Engineer 2. At Walmart, I am primarily working in the Data Lake Initiative of Walmart; mainly involved in building the data pipeline by gathering requirements from business, understanding the business rules, performing ETL and developing reports. So, I'm an active user of engineering tools and technologies such as Hadoop, Hive, Spark etc. Prior to this, I had worked for more than 2 years in the area of data warehousing and business intelligence at AIG (American International Group). Apart from this, I have pursued my Master's in Management Information Systems with the specialization in Data Analytics from the University of Arizona. Being a President at IET (Institution of Engineering & Technology) during my under graduation, I have given talks on various topics to engineering students. Apart from this, I have also organized and led career events during my Master's program at the University of Arizona. Moreover, I recently had the chance to be a host at the "Open Data Science Conference West 2018". https://www.linkedin.com/feed/update/urn:li:activity:6466493531757383680