Kudu as Storage Layer to Digitize Credit Processes

Kudu as Storage Layer to Digitize Credit Processes

Technical
 
Big Compute and Storage
Wednesday, March 20
2:50 PM - 3:30 PM
Room 118-119

With HDFS and HBase, there are two different storage options available in the Hadoop ecosystem. Both have their strengths and weaknesses. However, neither HDFS nor HBase can be used universally for all kinds of workloads. Usually this leads to complex hybrid architectures. Kudu is a very versatile storage layer which fills this gap and simplifies the architecture of Big Data systems.

A large German bank is using Kudu as storage layer to fasten their credit processes. Within this system, financial transactions of millions of customers are analysed by Spark jobs to categorize transactions and to calculate key figures. In addition to this analytical workload, several frontend applications are using the Kudu Java API to perform random reads and writes in real-time.

The presentation will cover these topics:
- Business and technical requirements
- Data access patterns
- System architecture
- Kudu data modelling
- Kudu architecture for High Availability
- Experiences from development and operations

SPEAKERS

Olaf Hein
Department Head & Principal Consultant
ORDIX AG
With more than 20 years working in the IT industry, Olaf has earned experiences as architect, developer, administrator, trainer and project manager in many different areas. Storing and processing huge amounts of data, was always a focal point of his work. At ORDIX AG, he is responsible for Big Data and Data Warehouse technologies and solutions. He has built up a powerful team of Big Data consultants, created several training courses, speaks at conferences and regularly publishes technical articles. Talks in the past: Cloudera Sessions, München 2017: Fast analytics on fast data - Kudu als Storage Layer für Banking Applikationen DOAG, Nürnberg 2017: Big Data - Quickstart mit Hadoop und der Oracle Big Data Platform Big Data Summit, Hanau 2018: Fast analytics on fast data - Kudu als Storage Layer für Banking Applikationen Strata Data Converence, London 2018: Fast analytics on fast data - Kudu as storage layer for banking applications DOAG Big Data Days, Dresden 2018: Fast analytics on fast data - Digitalisierung von Kreditprozessen mit Kudu Upcoming talks: IT Tage, Frankfurt2018: Fast analytics on fast data - Digitalisierung von Kreditprozessen mit Kudu Publications: Big Data - Informationen neu gelebt (Teil VII): Apache Kudu; ORDIX news 2/2017 Informationen neu gelebt (Teil II): Apache Cassandra; ORDIX news 2/2015 Informationen neu gelebt (Teil I): Wie big ist Big Data?; ORDIX news 1/2015 Neuerungen in der Oracle Database 12c (Teil V): Erweiterungen im DWH-Umfeld; ORDIX news 3/2014 Dokumentenschredder: Zerlegen und Zusammensetzen von XML-Dokumenten mit dem DB2 XML Extender; XML Magazin Ausgabe 1.2004