Greater Toronto Area, Canada
Machine Learning and Data Analytics using Jupyter, Python , Scala, Spark, AWS, EMR, S3, Kinesys, Elastic Cache Current & Previous Projects: 1. Comcast Product & Services Recommendation Engine 2. Customer Lifetime Value & Survival Modeling 3. X1 Real Time Analytic Platform 4. Data Quality Project using Anomaly Detection
Designing & Optimizing Batch & Streaming Spark Jobs using Spark 2.1, Scala, Kafka , Cassandra. • Developed a shared Streaming Spark Job to filter device information for IOT devices which will help to reduce 30% of processing time of other jobs to perform similar Task. Spark 1.6, Kafka, Cassandra • Designed a prototype for using Spark Job Server for sharing NamedRDDs amongst 4 different Jobs. • Implemented custom Serialization using Google Protocol Buffers for Kafka messages with Spark Streaming • Implemented Vault fallback mechanism for storing Cassandra Auth secrets.
Big Data Analytics Web Service Development