Amit Kumar Bairagi

Data Engineer

Barcelona, Catalonia, Spain

About

With a bachelor's degree in information technology and extensive experience in ETL, I am a data pipeline expert who can deliver efficient and scalable data solutions. I have worked with various data sources and destinations, such as AWS, SFTP, SQL Server, Oracle, Snowflake, and websites via API, using tools such as Informatica PowerCenter, Python, Unix Shell Scripting, Jira, and Snowflake DB. As a Senior ETL Architect at PCGI Consulting, I led offshore teams in Agile-driven ETL projects, collaborating with onshore leads and project stakeholders. I established a robust data pipeline framework for ETL, integrating data from diverse sources into databases such as AWS RDS, Redshift, and Snowflake. I leveraged Snowflake DB, Python, and AWS EC2 for standardization, automation, and orchestration of data loads. I also implemented data quality and performance checks using Python in Snowflake, and utilized Snowflake DBT for transforming flat data. I have handled data in investment banking and healthcare sectors, transforming raw data into strategic insights with technical precision. I am passionate about learning new skills and technologies, and I strive to deliver high-quality results within cross-functional teams.

Experience

  • Data Engineer DB/AWS Cloud at Allianz Technology
    Aug 2024 - Present · 1 yr 11 mos

  • Senior Architect– ETL Python at PCGI Consulting
    Oct 2022 - Sep 2023 · 1 yr

    Led offshore teams in Agile-driven ETL projects, collaborating with Onshore Leads and Project Stakeholders. Orchestrated seamless data delivery by coordinating with offshore teams. Established a robust Data Pipeline framework for ETL, integrating data from diverse sources (AWS, SFTP, EMAIL, websites via API) into databases such as AWS RDS, Redshift, and Snowflake. Leveraged Snowflake DB, Python, and AWS EC2 for standardization, automation, and orchestration of data loads. Implemented data quality and performance checks using Python in Snowflake. Utilized Snowflake DBT for transforming flat data and created SOPs for efficient monitoring by the operations team. orchestrated data delivery through GitHub version control.

  • Consultant at Genpact Headstrong Capital Markets
    Jun 2016 - Oct 2022 · 6 yrs 5 mos

    Set up ETL process using Informatica Power Centre , Python, and Unix Shell Scripting for data extraction from various sources including FTP, SFTP, SQL Server DB, websites (using API), local servers, AWS, RDS, and Data Files via Email. Loaded extracted data into Oracle, Mongo DB, Snowflake, and SQL Server Databases. Created procedures and functions in Oracle, integrated with Informatica for data transformation, data quality, and auditing. Developed Python scripts for transforming/extracting Excel files into CSV format. Used Unix Scripting for validation of flat files, reading varying file names, and connecting to FTP/SFTP servers. Created Autosys Jobs for orchestrating data loads, Integrating GitHub for version control

  • Consultant at Dell International Services India Private Limited
    Jun 2015 - Nov 2015 · 6 mos

    Leveraged ETL framework using Informatica to connect various databases like SQL Server and Oracle, loading data into Oracle.Created orchestration in Informatica for end-to-end load using SCDs for historical and incremental data load, maintaining relationships between tables.Developed procedures and functions for use in Informatica, focusing on data quality, data relationship checks, auditing data flow, error detection, and handling.

  • Associate at Infosys
    May 2014 - Jun 2015 · 1 yr 2 mos

    Automated process to streamline forecast order handling, replacing manual processing from third-party administrators.Processed forecast orders through an automated system, enhancing accuracy and reducing complexity.Managed forecast order information in flat files, converting them to XML files for processing in the Supply Chain system.Ensured efficient communication with third-party systems for timely delivery of forecast order information to customers.