Gyanpur, Uttar Pradesh, India
Data Engineer with 3+ years of experience building scalable data pipelines and data-driven solutions. I specialize in Python, SQL, PySpark, Databricks, and Azure-based data platforms, focusing on designing efficient ETL pipelines, distributed data processing, and data transformation workflows. My work centers around end-to-end data engineering, including data ingestion, transformation, validation, and pipeline orchestration. I have hands-on experience with Spark architecture, DataFrame and RDD APIs, Databricks workflows, Delta Lake, and Azure Data Factory, enabling reliable and scalable processing of large datasets. I focus heavily on data quality, governance, lineage, and performance optimization, ensuring that data pipelines are reliable, efficient, and production-ready. My approach combines strong analytical thinking, debugging skills, and automation, helping improve operational efficiency and reduce manual processes. Technically, I work across the modern data stack including Python, SQL, PySpark, Spark SQL, Databricks, Azure Data Fundamentals, ETL Design, Data Warehousing, and Cloud Data Platforms. I also leverage tools such as Git, Jupyter Notebook, Power BI, MySQL, and Excel for analytics, reporting, and version control. Beyond data engineering, I have experience using Pandas, NumPy, Scikit-learn, Matplotlib, Seaborn, and Streamlit for data analysis, visualization, and machine learning experimentation. I am particularly interested in building scalable data infrastructure, optimizing big data workflows, and enabling organizations to make better decisions through reliable data systems. Core Skills & Keywords: Data Engineering • ETL Pipelines • PySpark • Spark SQL • Databricks • Delta Lake • Azure Data Factory • Data Warehousing • Distributed Data Processing • Big Data • Python • SQL • Data Pipeline Optimization • Data Governance • Data Quality • Data Validation • Git • Power BI • Pandas • NumPy • Machine Learning • Cloud Data Platforms