Apache Knox – Hadoop Security Swiss Army Knife

Apache Knox - Hadoop Security Swiss Army Knife

Wednesday, March 20
2:50 PM - 3:30 PM
Room 131-132

While traditional on-prem systems have always been a target from internal and external attackers, recent times have seen increased attacks on Hadoop cloud deployments. Hadoop systems are going to be increasingly targeted due to the large volume of data that it stores. Many Hadoop installations on cloud are publicly accessible without any security measures which pose threat to exfiltration of large datasets and possibly crypto-mining on this infrastructure with its huge distributed compute capability.

Apache Knox provides multiple layers of security related to authentication, service-level authorization and web application security controls out of the box for multiple Hadoop components.

Apache Knox provides configuration to prevent common OWASP Top 10 security risks e.g. Cross-site Request Forgery (CSRF), Cross Site Scripting (XSS), MIME Content Type sniffing, Clickjacking, etc. We will also discuss controls like HTTP Strict Transport Security which prevents SSL Downgrade attacks and CORS filter for allowing applications to make cross domain requests only to specifically allowed hosts through XHR. Support to include/exclude Cipher suites and exclude SSL protocols enables compliance with hardening guidelines provided by CIS for application servers.

Knox has several supported authentication mechanisms with Kerberos underneath e.g. LDAP over SSL, AD, PAM based auth for Unix users, integration with Identity Providers like Okta, etc. Also, capabilities like Trusted Proxy, Single Sign-On auth, Hostmap Provider, Identity Assertion Provider, Client Authentication enhances the overall security posture.

We will also cover the typical kill-chain methodology tailored to Hadoop ecosystem which will help formulate the preventive measures against future compromises.

Presentation Video


Krishna Pandey
Staff Software Engineer
An InfoSec Generalist. CISSP. My more than a decade long work experience revolves around all aspects of security mainly Secure-SDLC, Source Code Analysis, Vulnerability Assessment, Penetration Testing for Web Applications, Architecture Review, Incident Response, ISMS Compliance, Doing and facilitating 3rd Party Audits. Managed multiple Federal Data Center Operations, O/S and Application Hardening, Linux System Administration. Solution Deployment and Integration for Federal and various State Governments. Contributor to Apache Knox, Apache Zeppelin and Apache Spark. Have also years of experience in leading and managing a team for monitoring, securing and ensuring "Availability Round-the-Clock" for National Critical Infrastructure. Solving Brain-Teasing needle-in-haystack production issues (Architecture, Application, System & Network) and incorporating new requirements. Conducting Vulnerability Analysis and analyzing VA reports for suggesting corrective and preventive actions (Hotfixes/ CVEs / Design Change/ Hardening/ Patching /Upgrades) to Engineering and Operations team. Panelist for Big Data Security Work Group. Designing Solution Architecture and Capacity Planning for highly-available applications on Cloud/Data Centre environment.
Larry McCay
Sr. Development Manager and Security Architect
Larry is a Senior Development Manager and Architect on the Hortonworks security team. He is also a committer and PMC member for the Apache Knox and Apache Ranger projects, committer for Apache Hadoop and contributor to security aspects of multiple Hadoop related projects. He is a veteran in the enterprise middleware space with a specialization in platform management and security. Larry has extensive experience in the Java EE application server technologies and has served on various expert groups for JSRs within the JCP for Java EE security. He has worked on various webservices technologies and stacks including SOAP and REST with a focus on security.