Observing Intraday Indicators Using Real-Time Tick Data on Apache Superset and Druid

Observing Intraday Indicators Using Real-Time Tick Data on Apache Superset and Druid

Thursday, March 21
2:00 PM - 2:40 PM
Room 127-128

The Central Bank of the Republic of Turkey is primarily responsible for steering the monetary and exchange rate policies in Turkey.

One of the major core functions of the Bank is market operations. In this context, analyzing and interpreting real-time tick data related to money market instruments has become not only a requirement but also a challenge.

For this use case, an API provided by one of the financial data vendors has been used to gather real-time tick data and data routing has been orchestrated by Apache NiFi.

Gathered data is being transferred to Kafka topics and then handed off to Druid for real-time indexing tasks.

Indicators such as effective cost, bid-ask spread, price impact measures, return reversal are calculated using Apache Storm and finally visualized by means of Apache Superset in order to provide decision-makers with a new set of tools.

SPEAKERS

ZEKERIYA BESIROGLU
Strategy Advisor
Komtas
Zekeriya Besiroglu has progressive experience(+18 years) in IT. Zekeriya is one of the few people in the EMEA area, having knowledge and accepted as expert in RAC&Exadata & Exalogic & Big Data &Cloud and Engineered Systems. He is Oracle ACE. 7+ yrs experience in deploying and managing the multi-node development, testing and production Hadoop cluster with different Hadoop components (HIVE, PIG, SQOOP, OOZIE, FLUME, HCATALOG, HBASE,COUCHBASE, ZOOKEEPER,NIFI) using Cloudera Manager and Ambari. Provide architectural and technical guidance to help customers understand the cloud, and make best use of the Amazon Web Services (AWS) cloud computing platform to build scalable, robust, and secure applications. •Strong knowledge in configuring Name Node High Availability and Name Node Federation. •Familiar with importing and exporting data using Sqoop from RDBMS MySQL, Oracle, Teradata and also using fast loaders and connectors Experience. •Experience in using Flume to stream data into HDFS Programming in Python, Matlab, SQL, R, C#, Mathematica • Tools: Pandas, NumPy, SciPy, scikit-learn, matplotlib, ggplot, Highcharts, Tableau, LaTex • Classification and Clustering, Predictive Modeling, Regression, Dimensionality Reduction (PCA, SVD), •Ensemble Methods (Boosting, Bagging), Anomaly Detection • Deep experience with AI/DL/ML platformsWorking knowledge of things like Sci-kit Learn, Tensorflow, Cntk, caffe2 with ML/DL/AI Independent Software Vendors AWS Big Data Services - EMR, Redshift, DynamoDB, RDS, Kinesis, Data Pipeline Hortonworks Certified Instructor Hortonworks Certified Hadoop Administrator Certified Spark Developer Certified R Programmer Oracle Certified RAC Expert&EXADATA Certified Bea Weblogic,Websphere,SOA System Admin http://zekeriyabesiroglu.com
Central Bank of the Republic of Turkey
Big Data Team
Central Bank of the Republic of Turkey
Emre Tokel - Big Data Team Leader Central Bank of the Republic of Turkey Emre has 15+ years of experience in software development. He has taken role as developer and project manager in various projects. For 2 years now, he has been involved in big data and data intelligence studies within the Bank. Emre has been leading the big data team since last year and is responsible for the architecture of the Big Data Platform, which is based on Hortonworks technologies. He has an MBA degree and is pursuing his Ph.D in finance. Besides IT, he is a divemaster and teaching SCUBA. Kerem Basol - Big Data Engineer Central Bank of the Republic of Turkey Kerem has 10+ years of experience in software development including mobile, back-end and front-end. For the past two years, he focused on big data technologies and currently working as a big data engineer. Kerem is responsible for data ingestion and building custom solution stacks for business needs using the Big Data Platform, which is based on Hortonworks technologies. He holds an MS degree in CIS from UPENN. M. Yağmur Sahin - Big Data Engineer Central Bank of the Republic of Turkey Yağmur has been developing software for 10 years. Being experienced in software development, he has completed his masters degree in 2016 on distributed stream processing where he was first introduced with big data technologies. For the last 2 years, he has been designing and implementing big data solutions for the Bank using Hortonworks Data Platform. Yağmur is also pursuing his Ph.D at Medical Informatics department of METU. He loves running and hopefully will complete a marathon in coming years.