Detecting real-time market manipulation in decentralized cryptocurrency exchanges

Detecting real-time market manipulation in decentralized cryptocurrency exchanges

Wednesday, June 20
4:40 PM - 5:30 PM
Grand Ballroom 220B

A "decentralized exchange" is a currency exchange which lives and is run completely as a smart contract on the blockchain with no central authority or party running the backend. Funds are held in a smart contract and secured with a public/private key pair, such that each buy/sell/withdraw can only be invoked by the wallet owner and not by the central cluster admin.

The smart contract itself is run on the Ethereum Virtual Machine, which is comprised of hundreds of thousands of nodes that run independently on people's personal computers (and GPU farms!) but store every event on a public ledger. This enables a powerful platform for Investors, but also for money launderers, and "pump and dump" schemers.

For this demo, we will use popular data science tools to analyze EtherDelta's books—a cryptocurrency exchange with over 1 billion USD worth of funds in the "smart contract"—and leverage this publicly available dataset to expose which "coin" may be associated with scams, as they happen.

From a technology stack, we will showcase how events on a blockchain can be analyzed in modern big data architectures. These events could be the logs of a smart contract execution, for which we'll show how to leverage Spark via a Jupyter or Zeppelin Notebook to perform ETL using the power of a remote Hadoop cluster. This will cover our experiences in the slowness and limitations of querying data directly from the blockchain, and how a Kafka producer/consumer model works well for analyzing granular level applications running on one of the many blockchains/tangles arising in the crypto-currency decentralized compute world.

Presentation Video


Kat Petre
Ambari Product Owner
Kat Petre is a technology rebel. Strong open source supporter, early adopter and fully dedicated to being in a state of constant learning, she recently left her Product Specialist role in MapR field organization to join the open source efforts of making Ambari great again. Previously, she served as Solution Architect in Cloudera's professional services team, specializing in building secure yet usable, highly available distributed big data solutions, to detect the scarce meaningful signals from the datalakes of noise. Always looking for company in tackling the interesting problems with big data technologies, her ultimate goal is to create a simulation of the real world, fed by real time data, and overclock it. Because the people that are crazy enough to think they can change the world, are the ones that do.
Jesus Alvarez
Advisory Software Engineer
Jesus Alvarez is a Technical Evangelist with a passion for Data Science, Security, and Crypto-Currency. Currently an advisory engineer, building integration tools to allow IBM DSX to integrate with open and closed source components. Pioneer to the Hadoop ecosystem, building installers for "big data accelerators" in 2012 at IBM. Deep understanding for the importance of security, having spent 4 years in Healthcare IT/Software Design. Contributor to Apache Knox, Ambari, and an array of data science notebooks and tools. Veteran in the realm of Crypto-Currency, Jesus was pushing the limits of home electric breakers in 2012-2014 running GPU and ASIC miners and architecting high frequency cryptocurrency arbitrage bots since the era before Mt-Gox and Cryptsy went dark.