Running Enterprise Workloads with an open source Hybrid Cloud Data Architecture

Running Enterprise Workloads with an open source Hybrid Cloud Data Architecture

Tuesday, October 16
2:05 PM - 2:50 PM
Room 301

Cloud is turbocharging the Enterprise IT landscape with agility and flexibility. And now, discussions of cloud architecture dominate Enterprise IT. Cloud is enabling many ephemeral on-demand use cases which is a game-changing opportunity for analytic workloads. But all of this comes with the challenges of running enterprise workloads in the cloud securely and with ease.

With the convergence of cloud, IoT, and big data technologies, enterprises increasingly have their data spread across multiple data lakes on-prem and in cloud data lake stores in many geographies and across multiple public cloud vendor platforms, for example, due to regulatory and compliance mandates that limit cross-border data transfer. With the proliferation of data types and sources in this complex landscape, the process of discovery, provisioning, and running relevant workloads on this data to gather insights has become more complex. Additionally, gaining global visibility into the business context, usage, and trustworthiness of data requires a centralized view of all data and metadata, security controls, data access, and monitoring.

All of these challenges create a significant chasm between initial data capture and subsequent data insights generation to drive value creation. Therefore, enterprises now require a “global insight fabric” that can find a happy medium between adequate rules and policies of data governance while providing a trusted environment for users to collaborate and share data responsibly in order to create value.

In this talk, we will outline how Hortonworks DataPlane Service(DPS) can help customers build a global insight fabric that can span storing and analyzing data within data centers to implementing an open source hybrid architecture that takes advantage of cloud's elasticity and new use cases. We will get a personal view of the challenges faced in safely moving data from on-premises data centers into multiple public clouds, safeguarding it through replication, and then applying consistent security and governance policies across diverse environments to deliver trusted data and insights to the business. We will highlight how DataPlane Service can help enterprises with this hybrid architectural journey, and how open source architectures are enabling this transformation across enterprises.


Alan Gates
Alan is a founder of Hortonworks and an original member of the engineering team that took Pig from a Yahoo! Labs research project to a successful Apache open source project. Alan is PMC member on Apache Hive, Pig, and many other Apache projects. As part of the Apache Incubator PMC he has mentored many new Apache communities. Alan has a BS in Mathematics from Oregon State University and a MA in Theology from Fuller Theological Seminary. He is also the author of Programming Pig, a book from O’Reilly Press. Follow Alan on Twitter: @alanfgates.