Noida, Uttar Pradesh, India
Data Engineer with 3+ years of experience building scalable ETL pipelines, optimizing large-scale data workflows, and enabling analytics across enterprise datasets. Currently working at Nagarro on high-volume pharma data platforms, leveraging Databricks, Apache Spark, SQL, AWS (S3, Redshift), and Python to design and maintain reliable data pipelines and data processing workflows. Databricks Certified Data Engineer Associate with hands-on experience in PySpark, Spark SQL, workflow automation, data quality optimization, and cloud-based data engineering solutions. Tech stack & interests: • Databricks & Delta Lake • PySpark & Spark SQL • SQL & Data Modeling • ETL & Workflow Automation • AWS Cloud Data Platforms • Power BI & Analytics • Data Quality & Troubleshooting I enjoy solving complex data challenges, improving pipeline reliability, and building scalable systems that support analytics and business decision-making efficiently. Currently focused on advancing deeper into modern Data Engineering and distributed data processing architectures.
Key Responsibilities as a Data Engineer • Data Pipeline Development: Designed and built scalable ETL pipelines using PySpark and SQL on AWS (S3, Redshift), processing high-volume pharma datasets and reducing reporting turnaround time by 40%. • Workflow Orchestration: Developed and managed data workflows using AWS Managed Workflows for Apache Airflow (MWAA), enabling reliable scheduling, monitoring, and dependency management of ETL jobs. • Data Transformation: Implemented robust transformation logic to clean, standardize, and enrich datasets based on business rules, ensuring consistency across reporting layers. • Performance Optimization: Tuned ETL pipelines and SQL queries to improve processing efficiency, scalability, and resource utilization. • Data Quality & Validation: Built data validation frameworks and automated quality checks to ensure data accuracy and integrity across pipelines. • Stakeholder Enablement: Delivered curated datasets and supported reporting use cases, enabling business teams to derive actionable insights. • Documentation & Maintenance: Created detailed documentation for data workflows, mappings, and pipeline architecture to support maintainability and knowledge transfer. • Troubleshooting & Monitoring: Diagnosed and resolved pipeline failures, data inconsistencies, and performance bottlenecks to ensure SLA adherence.
Automated business reporting using Power BI, Power Apps, and SharePoint, cutting down manual effort by 80%. Created optimized Power BI dashboards and data models using Power Query and DAX, driving SLA tracking and operational visibility.