Bengaluru, Karnataka, India
I’m a Senior Data Engineer with 10 years of experience designing, building, and optimizing scalable data platforms and distributed processing systems across cloud and on-prem environments. I specialize in end-to-end data pipeline architecture — from data modeling and ingestion to transformation, orchestration, and deployment — ensuring reliability, performance, and cost efficiency at scale. My technical toolkit includes AWS Cloud (Glue, EMR, Redshift, Lambda), Apache Spark, Kafka, Hadoop, Hive, HBase, and NoSQL databases. I’m also exploring how Generative AI can augment data engineering workflows to automate insights and enhance decision-making. At Intuit, I’ve contributed to modernizing legacy pipelines into cloud-native architectures, improving data reliability and enabling faster analytics delivery for key business functions. I’m passionate about solving complex data challenges — building systems that scale seamlessly, empower data scientists, and drive intelligent business outcomes. Outside of work, I enjoy mentoring aspiring data engineers and exploring new generative AI tools for data automation.
• Designed and deployed real-time data pipelines on AWS using Spark, Kafka, and Glue, processing 5TB+ daily across multiple business domains. • Migrated legacy ETL workflows to cloud-native architecture (EMR + S3 + Lambda), reducing compute cost by 35%. • Built reusable data quality framework in Python and Airflow, improving reliability across 150+ pipelines. • Partnered with data science teams to deliver AI/ML-ready datasets for analytics and experimentation. • Build Generative AI–driven data automation for intelligent monitoring and pipeline optimization.
• Designed and deployed real-time data pipelines on AWS using Spark, Kafka, and Glue, processing 5TB+ daily across multiple business domains. • Migrated legacy ETL workflows to cloud-native architecture (EMR + S3 + Lambda), reducing compute cost by 35%. • Built reusable data quality framework in Python and Airflow, improving reliability across 150+ pipelines. • Partnered with data science teams to deliver AI/ML-ready datasets for analytics and experimentation. • Build Generative AI–driven data automation for intelligent monitoring and pipeline optimization.
• Developed data lakehouse solutions integrating structured and semi-structured data from multiple sources (RDBMS, API, and streaming). • Led migration of legacy pipelines to AWS EMR improving performance by 40% and reducing cost by 25%. • Partnered with Data Science and BI teams to deliver near-real-time dashboards leveraging Snowflake / Redshift / Athena. • Implemented CI/CD for data pipelines with Git, Jenkins, and Terraform, improving deployment reliability. • Mentored a 3-member team of junior data engineers on cloud best practices and Spark optimization.
• Built robust ETL workflows in Informatica and Python to automate ingestion from 20+ financial data sources. • Designed SQL-based data marts enabling efficient reporting and ad-hoc analytics for product and operations teams. • Optimized Oracle and Teradata queries, reducing execution time by 60%. • Contributed to the design of a data quality framework improving data validation coverage by 35%. • Collaborated cross-functionally with business analysts to translate financial reporting requirements into technical specifications.
• Developed data acquisition pipelines for fund and market data ingestion into enterprise warehouses. • Automated ETL scheduling and monitoring through Unix shell scripts and Python, reducing manual interventions. • Worked with stakeholders to streamline data validation and transformation rules for portfolio analytics.
• Assisted in building and testing ETL components for client financial data migration. • Created SQL stored procedures and data validation scripts for QA environments. • Gained hands-on experience with data modeling and integration workflows.