Intelligently collecting data at the edge—intro to Apache MiNiFi

Intelligently collecting data at the edge—intro to Apache MiNiFi

Thursday, April 19
11:00 AM - 11:40 AM
Room II

Apache NiFi provided a revolutionary data flow management system with a broad range of integrations with existing data production, consumption, and analysis ecosystems, all covered with robust data delivery and provenance infrastructure. Now learn about the follow-on project which expands the reach of NiFi to the edge, Apache MiNiFi. MiNiFi is a lightweight application which can be deployed on hardware orders of magnitude smaller and less powerful than the existing standard data collection platforms. With both a JVM compatible and native agent, MiNiFi allows data collection in brand new environments — sensors with tiny footprints, distributed systems with intermittent or restricted bandwidth, and even disposable or ephemeral hardware. Not only can this data be prioritized and have some initial analysis performed at the edge, it can be encrypted and secured immediately. Local governance and regulatory policies can be applied across geopolitical boundaries to conform with legal requirements. And all of this configuration can be done from central command & control using an existing NiFi with the trusted and stable UI data flow managers already love.

Expected prior knowledge / intended audience: developers and data flow managers should have passing knowledge of Apache NiFi as a platform for routing, transforming, and delivering data through systems (a brief overview will be provided). The talk will focus on extending the data collection, routing, provenance, and governance capabilities of NiFi to IoT/edge integration via MiNiFi.

Presentation Video


Andy LoPresto
Sr. Member of Technical Staff
Andy LoPresto is a Sr. Member of Technical Staff at Hortonworks working on the Hortonworks DataFlow team. In this role he serves as both a Committer and Project Management Committee Member for Apache NiFi, an open source, robust, secure data routing and delivery system. Andy focuses on security concerns within NiFi including identity management, TLS negotiation, data protection, access control, encryption and hashing. Andy is also involved with the sub-project, Apache MiNiFi, which drives edge data collection, including secure command and control and immediate data provenance and governance. He has presented about NiFi at DataWorks Summits in Singapore, Tokyo, Melbourne, Berlin, Sydney, and San Jose, FOSDEM '17 in Brussels, and the OpenIoT Summit 2017.