Greater Melbourne Area
A Lead Data Engineer who is passionate about transforming and optimizing the way data is used in companies and ensuring architectural design meets the needs of business objectives. I have several years’ experience in building data intensive applications, tackling challenging architectural, scalability problems and other big data issues.
*AWS *Redshift *Terraform *dbt *Devops *Airflow *AWS ECS *Docker *Data architechture and Design *Data Modelling *Data Governance *ML and MLOps
*Databricks *AWS *Redshift *Snowflake *Airflow *Azure Devops *Terraform
End to end delivery of data driven insights to help end user make correct decision • Implemented Spark Sql to update queries based on the business requirements • Data analysis, profiling, messaging, transformation, staging and processing for NAB • Adapted functionality in S3 and Redshift to allow incremental loading by adding staging layer and scripting Spark SQL code to integrate staged data with the files • Designed and developed Extract-Transform-Load (ETL) routines and event driven data pipelines • Creating S3 buckets also managing policies for S3 buckets and Utilized S3 bucket and Glacier for storage and backup on AWS • Work closely with ML Engineers and Data Scientists to collect training data and measuring products success • Worked on design and development of in-house reporting which had multiple components like Oracle, AWS S3, Python, Spark SQL, Redshift, Tableau, OBIEE and AWS Quicksight • Providing sales, performance and customer insight and behavior reports to 7000+ bankers across NAB on weekly basics • Developed dashboard and reports using Tableau and AWS Quicksight managing all aspects of the technical development • Solution Designer for the OFSAA, BI implementation and Analytics projects • Providing support to business-critical reporting modules like CRE, FTP etc. • Timely delivery of regulatory reporting APRA, FSA and RBNZ • Creation of complex OBIEE Reports, BI Publisher Reports, Dashboards, KPIs, Actions, Agents, Scorecards and custom report consulting, development and deployment • Creation Oracle Data Integrator (ODI) interfaces and packages and data transformation from loading data into OFSAA Common Staging Area (CSA) from various Legacy Source Systems • Preparing Unix shell scripts for the interface execution and automation • Responsible for automation and building CICD pipeline for automated code deployment using GitHub, Jenkins and Artifactory