Greater Delhi Area
I am a highly proactive Data Engineer with 6.9 years of experience building scalable data infrastructure, automating ETL pipelines, and delivering high-impact business solutions. I excel in collaborative, fast-paced environments where ownership and a "get things done" mindset are vital. My expertise spans cloud ecosystems (AWS, GCP), data platforms (Snowflake, Databricks), and orchestration tools (Airflow, Prefect, GitHub Actions). I have a proven track record of optimizing data ecosystems, evidenced by cutting Snowflake query times by 93% (from 5+ hours to 25 minutes) and engineering over 20 concurrent pipelines with 99.9% uptime. Pragmatic and curious, I bridge the gap between complex data engineering and business utility. For instance, I built an AI-powered conversational chatbot using LLMs, a Snowflake MCP server, and a semantic layer, allowing non-technical users to run advanced data queries in plain English. Holding certifications as a GCP Professional Data Engineer, SnowPro Advanced Data Engineer, and Databricks Associate, I bring rigorous technical validation to my hands-on experience. I am eager to apply my skills in pipeline automation, cloud migrations, and emerging AI technologies to drive immediate value for your engineering team.
Optimized Snowflake query performance by 93% (5+ hours → 20-25 minutes) and automated S3 data pipeline using Apache Airflow, enabling faster UAT cycles and reducing infrastructure costs.
Engineered and optimized over 20+ data pipelines using Python, Spark, SparkSQL, and Airflow to process and validate large-scale datasets, ensuring 99.9% uptime. Collaborated with cross-functional teams to streamline ETL workflows, improving data transformation efficiency by 30%. Enhanced data quality with custom cleaning and validation scripts, reducing errors by 40%. Designed scalable data lake architecture using AWS S3, Glue, and SparkSQL, boosting data accessibility and processing speed by 25%. Implemented schema validation processes, maintaining data type consistency and reducing deployment errors. Built robust data integration pipelines connecting APIs, databases, and S3, reducing manual intervention by 50%. Deployed Slack-integrated alert systems for pipeline monitoring, achieving faster issue resolution times. Utilized Snowflake and BigQuery to manage and analyze terabytes of data, enabling actionable insights via Power BI dashboards. Migrated and processed data across cloud platforms (GCP to AWS) with optimized storage formats like Parquet, JSON, and Avro. Developed and executed high-throughput batch and streaming workflows on Databricks with SparkSQL, improving data processing efficiency. Designed and maintained robust data pipelines and workflows using GitHub Actions for CI/CD integration. Also collaborated in Agile environment.