Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform

Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform

Tuesday, June 19
11:50 AM - 12:30 PM
Meeting Room 230A

Overview of USAAs decision drivers and journey to migrate from our IBM BigInsights Hadoop platform to Hortonworks Data Platform. Many obstacles challenged our ability to respond to demands of our business data needs and analytic capabilities. In addition to migrating 1.5 PB (500 TB useable) of data to our new HDP environment, we were introducing a new security model with Kerberos/AD integration, data governance, as well as many new HDP services that were unavailable within our BigInsights platform. We'll discuss overall scope of work for this year long journey and our approach in bringing enterprise adoption of a new Hadoop platform. We still have many efforts under way to further enhance our data delivery patterns, information governance process and procedures, and optimized consumption within our HDP platform, but we are now better positioned to provide a flexible, secure, and managed Hadoop platform with focused innovation to meet USAAs strategic initiatives.

Presentation Video


Lisa Coleman
Technical Architect
Lisa is currently serving USAA and its members in their Chief Technology Office as a Technical Architect over Data and Analytics Infrastructure. She has focus in setting architectural direction for our big data infrastructure and analytic appliances. Prior to this role, she has devoted her expertise in the implementation and support of USAAs data warehousing and analytic systems as a Lead DBA.
Robert Tucker
Software Developer & Integrator Lead
Robert is currently a Hadoop Administrator at USAA. He began his big data journey in 2011 when as part of a team of 4 he helped to bring Hadoop into USAA. He is a certified Cloudera Hadoop Administrator and has experience working with a few different Hadoop distributions including IBM BigInsights and Hortonworks HDP. He also worked a bit with the MapR and Pivotal distributions. Since 2011 he has assisted in building and maintaining the USAA Hadoop clusters. He also assisted in identifying the skills needed, growing the team of Hadoop Administrators and is always willing to take the time to train others on the job. He has also assisted in developing a chargeback model, lead efforts to perform upgrades (not a small effort with BigInsights and GPFS!), lead an effort to combine clusters and assisted in identifying and purchasing the next generation hardware for the clusters. When he had spare time he enjoyed setting up his own clusters at home though with his wife and six kids (including 7 month old twins) there is little time or budget left for that now.