DataWorks Summit in Washington, DC

Artificial intelligence (AI) is transforming every industry. Data science and machine learning are opening new doors in process automation, predictive analytics, and decision optimization. This track offers sessions spanning the entire data science lifecycle: development, test, and production.

You’ll see examples of innovative analytics applications and systems for data visualization, statistics, machine learning, cognitive systems, and deep learning. We’ll show you how to use modern open source workbenches to develop, test, and evaluate advanced AI models before deploying them. You’ll hear from leading researchers, data scientists, analysts, and practitioners who are driving innovation in AI and data science.

Sample technologies: TensorFlow, Keras, Apache Spark, PyTorch, Apache MXNet, Theano, DL4J, R, scikit-learn, DSX, Apache Zeppelin

Apache Hadoop continues to drive data management innovation at a rapid pace. Hadoop 3.0 adds container management to YARN, an object store to HDFS, and more. This track presents these advances and describes projects in incubation and the industry initiatives driving innovation in and around the Hadoop platform.

You’ll learn about key projects like HDFS, YARN, and related technologies. You’ll interact with technical leads, committers, and experts who are driving the roadmaps, key features, and advanced technology research around what is coming next and the extended open source big compute and storage ecosystem.

Sample technologies: Apache Hadoop (YARN, HDFS, Ozone), Apache Kudu, Kubernetes, Apache BookKeeper

A hybrid, multi-cloud data architecture that optimizes information placement and processing between on-premises data centers and the cloud is critical to scale and flexibility. But it must also provide a global and integrated view of all your data with consistent operations, governance, and security.

This track provides the latest best practices on how to build modern data architectures. You’ll learn about key open source projects, including Apache Ambari, Cloudbreak, and related technologies and how they integrate with the latest cloud offerings to enable transformative changes. You’ll interact with technical leads, committers, and experts who are driving research, key features, and roadmaps in the extended open source big data architecture.

Sessions cover the full deployment lifecycle, how to set up and manage high-availability configurations, and how DevOps practices can help speed solutions into production. You’ll learn how to manage data across the edge, the data center, and the cloud. And you’ll hear cutting-edge best practices for large-scale deployments.

Sample technologies: Apache Ambari, Cloudbreak, DataPlane Service, AWS, Azure,GCP

Cybersecurity is a data problem. With escalating volumes of logs, network traffic, and IoT-driven endpoints to watch, it has become a big data problem. Finding attacks faster to prevent intrusions from turning into advanced persistent threats requires capturing and processing streaming data in real time.

In this track, you’ll learn how to leverage big data and stream processing to improve your cybersecurity. Experts will explain how to scale with analytics on more data and react in real time. They’ll show how a strategic focus on data can make your security staff more efficient, how machine learning can augment their skills, and how community, automation, and scalability can help you fight off the threats.

Sample technologies: Apache Metron, Apache Spot

Data engineers and architects use multiple engines to process data in the most appropriate way, from batch ETL, to interactive SQL, to low latency NoSQL. Sessions will cover the SQL engines and tools that help users to derive the most from their data on premises and in the cloud and enrich their enterprise data warehouse (EDW). You’ll learn how NoSQL stores like Apache HBase are adding transactional capabilities that bring traditional operational data store (ODS) workloads to Hadoop and why data preparation is a key workload. You’ll meet Apache community rock stars and learn how these innovators are building the applications of the future.

Sample technologies: Apache Hive, Apache Tez, Apache ORC, Apache Druid, Apache HBase, Apache Phoenix

Your data lake contains a growing volume of diverse enterprise data, so a breach could be catastrophic. Privacy violations and regulatory infractions can damage your corporate image and long-term shareholder value. Government and industry regulations demand you properly secure and govern your data to assure compliance and mitigate risks. But as Hadoop and streaming applications emerge as a critical foundation of a modern data architecture, enterprises face new requirements for protection and governance.

In this track, you’ll learn about the key enterprise requirements for governance and security of the extended data plane. You’ll hear best practices, tips, tricks, and war stories on how to secure and govern your big data infrastructure.

Sample technologies: Apache Ranger, Apache Sentry, Apache Atlas, and Apache Knox

The rapid proliferation of sensors and connected devices is fueling an explosion of data. Streaming data allows algorithms to dynamically adapt to new patterns in data, which is critical in applications like fraud detection and stock price prediction. Deploying real-time machine learning models in data streams enables insights and interactions not previously possible.

In this track you’ll learn how to apply machine learning to capture perishable insights from streaming data sources and how to interface with devices at the “jagged edge.” Sessions present new strategies and best practices for real-time data ingestion and analysis. Presenters will show how to use these technologies to develop IoT solutions and how to combine historical with streaming data to build dynamic, real-time predictive systems for actionable insights.

Sample technologies: Apache Nifi, Apache Storm, Streams Messaging Manager, Streaming Analytics Manager, Apache Flink, Apache Spark Streaming, Apache Beam, Apache Pulsar and Apache Kafka