Deep Learning with DL4J on Apache Spark: Yeah it’s Cool, but are You Doing it the Right Way?

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it the Right Way?

Thursday, March 21
2:50 PM - 3:30 PM
Room 118-119

DeepLearning4J (DL4J) is a powerful Open Source distributed framework that brings Deep Learning to the JVM (it can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers). It can be used on distributed GPUs and CPUs. It is integrated with Hadoop and Apache Spark. ND4J is a Open Source, distributed and GPU-enabled library that brings the intuitive scientific computing tools of the Python community to the JVM. Training neural network models using DL4J, ND4J and Spark is a powerful combination, but the overall cluster configuration can present some unespected issues that can compromise performances and nullify the benefits of well written code and good model design. In this talk I will walk through some of those problems and will present some best practices to prevent them. The presented use cases will refer to DL4J and ND4J on different Spark deployment modes (standalone, YARN, Kubernetes). The reference programming language for any code example would be Scala, but no preliminary Scala knowledge is mandatory in order to better understanding the presented topics.

SPEAKERS

Guglielmo Iozzia
Big Data Delivery Lead
Optum (UnitedHealth)
I am currently a Big Data Delivery Lead at Optum (UnitedHealth Group) and based in Dublin (Ireland). Me and my teams deal with projects in the PI (fraud, waste and abuse, claims processing) and the healthcare space. I worked previously at IBM Ireland, where I switched my career path from Test Automation to Analytics and Machine Learning. I am passionate about coding, Big Data, AI/ML/DL, test automation, Open Source, DevOps and cooking (home made pizza is my speciality). I share my tech thoughts through my blog (http://googlielmo.blogspot.ie/) and DZone (https://dzone.com/users/2532948/virtualramblas.html) where I am a Golden Member. During 2018 I have presentend to several international conferences such as DataWorks Summit Berlin, Google I/O Extended, Predictive Analytics World for Industry 4.0 and many others. My first book "Hands-on Deep Learning with Apache Spark" (https://tinyurl.com/y7d98s64) is going to be released in December 2018.