Pune Division, Maharashtra, India
I thrive on uncovering patterns in data and engineering scalable solutions. With a passion for big data engineering, I specialize in designing, building, and optimizing data pipelines that enable seamless data migration, processing, and analytics across GCP and Azure. My expertise spans Hadoop, Apache Spark, PySpark, SQL, and NoSQL databases, allowing me to handle large-scale data processing efficiently. I leverage Apache Airflow for workflow orchestration, ensuring automation, monitoring, and reliability in pipeline execution. Additionally, I work extensively with BigQuery, Dataflow, Dataproc, Azure Data Factory, Synapse Analytics, and Databricks, building cost-effective and high-performance data solutions for real-time & batch data processing. Beyond work, I am a continuous learner, always exploring new technologies and optimizing data solutions. In my free time, I enjoy traveling, going to the gym, and cooking. I also love connecting with professionals to discuss big data, cloud technologies, and emerging trends in AI and analytics. Tech Stack: Big Data Technologies: Hadoop | Apache Spark | PySpark | SQL | NoSQL | ETL/ELT Pipelines Cloud Platforms: GCP | Azure | BigQuery | Dataflow | Dataproc | Azure Data Factory | Synapse Analytics | Databricks Workflow Orchestration & Optimization: Apache Airflow | Automation | Monitoring | Query Optimization Programming Languages: Python | SQL | Javascript
- Designed, developed, and orchestrated data pipelines for multiple value streams, enabling seamless cloud migration to GCP and Azure. - Built scalable ETL/ELT pipelines to process and migrate 100TB+ of data, optimising performance and cost efficiency. - Integrated diverse data sources (RDBMS, NoSQL, cloud storage, APIs) and ensured smooth data movement across on-prem and cloud environments. - Implemented workflow orchestration using Apache Airflow, automating end-to-end pipeline execution and reducing processing time by 40%. - Optimized data processing using Apache Spark, enhancing compute efficiency by 35% for large-scale data transformations. - Ensured data reliability, monitoring, and governance through logging, alerting, and error handling mechanisms. - Collaborated with cross-functional teams to streamline data migration strategies, supporting business intelligence and analytics initiatives.
- Developed an in-house data ingestion platform using VueTS, Vuex (Frontend), Java (Backend), Apache Spark (Data Processing), and Airflow (Pipeline Orchestration). - Enabled ingestion from diverse sources (RDBMS, filesystem, cloud object storage) across on-prem DWH and cloud, processing 1TB+ data daily. - Designed an intuitive UI, allowing non-data engineers to create, manage, and orchestrate data pipelines, reducing onboarding time by 40%. - Implemented Kubernetes-based auto-scaling, improving resource utilization efficiency by 30% under high workloads. - Facilitated large-scale data migration across multiple DT streams, accelerating cloud adoption and reducing data transfer time by 50%.
• Designed and implemented data ingestion pipelines to integrate financial data from multiple business domains including insurance, loans, and investments into a centralized data lake on ADLS. • Developed and optimized ETL workflows using ADLS , ADF and databricks to ensure efficient data transformation, validation, and loading into analytics-ready layers • Collaborated with analytics teams to deliver curated datasets and dashboards supporting key business KPIs.
Key Projects: 1. T-Mobile (Frontend Revamp Project) • Led a team of 5 to migrate frontend from jQuery to Angular 2. • Built modular and dynamic components integrated with Adobe Experience Manager (AEM). 2. GSTN (Government of India Project) • Contributed to the development of the GSTN Tax Filing Portal from scratch using Angular 1.5 and TypeScript. • Participated in client requirement gathering and UI design.
The Entrepreneurship Cell is a non-profit organization run by the students of IIT Bombay that aims at manifesting the latent entrepreneurial spirit of the young students. I as member of E-Cell IIT-B hosts various workshops, speaker sessions, innovative games,competitions for aspiring entrepreneurs and students in our city and support them by providing necessary resources such as seed funding, mentoring, consultancy and networking.