Teams, tools, and practices for scalable and resilient data value at Klarna Bank

Teams, tools, and practices for scalable and resilient data value at Klarna Bank

Thursday, April 19
4:50 PM - 5:30 PM
Europe

Manage your data infrastructure like this:
(i) don’t drown the infra teams in domain data specifics
(ii) build robust low-latency lookup facilities to feed online services
(iii) always take stress out of the equation

At Klarna bank we do online decisions on risk, fraud and ID. Over a hundred data sources are being processed by over a hundred analysts and over a hundred batch jobs. Three data infrastructure engineering teams are operating and developing this data lake: Core team, apps team, and performance team. The total head count is less than a dozen.

To keep afloat, we’ve distilled the following practices: (i) The immutability and recomputation properties of the Lambda/Kappa architectures (ii) continuously delivered and automated infrastructure, (iii) tooling to empower producers and consumers of data to be accountable and self-sufficient, and (iv) proactively improve efficiency of data users.

We’ll talk about some of these practices and tools we have built during several years of running banking applications on Hortonworks Hadoop. Ecosystem components we’ll touch include Kafka, Avro, Hive, Oozie, ELK, Ranger, and Ansible. Tools developed by us include HiveRunner, tooling for data import, along with continuous delivery of data pipelines.

Presentation Video

SPEAKERS

Erik Zeitler
Senior Data Engineer, PhD
Klarna Bank AB
Erik Zeitler holds a PhD from Uppsala University, where he performed research on highly scalable data stream management systems. This work is published at premier conferences such as EDBT and VLDB. Erik has been architecting Klarna's data infrastructure for analytics and real time services. He regularly serves as a speaker at universities, meetups and trade shows. Erik regularly serves as an expert advisor for research and development at the European Commission.
Per Ullberg
Lead Software Engineer
Klarna Bank
Per Ullberg is a dedicated lead java developer with a strong interest in quality-abling processes and tools. Per works at the Klarna Bank in the teams that implements and enables the lambda architecture as a long term solution for analytics at Klarna. Per has earlier worked at companies like NasdaqOMX, Unibet and Bwin with a strong track record of enabling quality by introducing testability to domains where testability previously was lacking. Per strongly believes that the biggest influencers to product quality are developers.