Greater Vancouver Metropolitan Area
Microsoft Certified: Fabric Data Engineer Associate with a strong background in building scalable data solutions using Snowflake, Azure Databricks, Microsoft Fabric, Data Factory, and Delta Lake. Skilled in designing robust ETL/ELT pipelines with PySpark, SQL, and Python, and optimizing big data workflows in cloud-based lakehouse architectures. Proficient in handling real-time and batch data, ensuring data quality, security, and governance using CI/CD (Azure DevOps) and Apache Airflow. Adept at collaborating in Agile/Scrum environments and delivering business-ready data for analytics and BI tools like Power BI and Tableau. Passionate about enabling data-driven decisions through modern, efficient, and reliable data engineering practices. • Proficient in Big Data environment with hands-on experience in utilizing Azure Databricks, Spark, and Delta Lake for large-scale data processing of structured and semi-structured data. • Experienced working on Databricks Unified Data Analytics, Databricks Workspace User Interface, Managing Databricks Notebooks, Delta Lake with Python, and Delta Lake with Spark SQL. • Hands-on experience in designing and developing scalable data pipelines using Azure Data Factory (ADF), Azure Databricks (ADB), Delta Lake, and Snowflake to process high-volume financial transactions. • Strong understanding of Big Data Security and governance, including encryption, RBAC, and compliance with PCI DSS, SOX, and GDPR standards. • Proven expertise in building ELT workflows using PySpark, SQL, and ADF for integrating data from core banking systems, credit bureaus, and payment processors. • Skilled in developing optimized data models and partitioning strategies for Big Data lakes in Azure Data Lake Storage (ADLS) and Snowflake, improving query performance and storage efficiency. • Strong experience working with Azure Data Lake Storage Gen2 (ADLS) and Delta Lake to store and manage large-scale structured and semi-structured financial data. • Skilled in implementing data quality and monitoring frameworks using Apache Airflow, Great Expectations, and Azure Monitor to ensure pipeline reliability and compliance. • Built real-time data streaming and analytics solutions using Kafka, Databricks Structured Streaming, and Delta Live Tables for fraud detection and transaction monitoring. • Migrated legacy ETL processes from on-premise systems to Azure Synapse Analytics and Snowflake, reducing operational costs and improving performance.
Working as Senior Data Engineer, developing SQL Server to Azure Databricks migration solutions using Azure Databricks Lakehouse and end to end aanalytic solutions using Microsoft Fabric.
Project Overview: Ally Financial is a leading digital financial service company offering an array of deposit and mortgage products and services. This role was part of the Enterprise Payments and Fraud Prevention division, managing a portfolio of liquidity/ Credit risk regulatory data projects. Responsibilities: • Developed and maintained ETL pipelines using Fivetran and Azure Data Factory to ingest and transform data. Optimized notebooks and data pipelines using Python, Spark, and Scala • Built and deployed high-performance ELT workflows using Azure Data Factory, Apache Spark, and SQL, automating complex transformations across regulatory and liquidity datasets. • Collaborated in the design of distributed systems and streaming architectures using Spark Streaming and Kafka to ingest real-time transactional data and fraud alerts. • Implemented secure and reusable credential handling using Azure Key Vault, integrated with Databricks, ADF, and other pipeline components. • Automated infrastructure provisioning using Terraform and enabled seamless code integration and deployment through CI/CD pipelines in Azure DevOps, AWS CodePipeline, and GitHub Actions. • Designed analytical data models and curated datasets in Azure Synapse and Power BI to support dashboards for fraud detection, risk exposure, and compliance tracking. • Developed Python scripts for data validation, transformation, and automation tasks within ETL pipelines, improving pipeline efficiency and reducing manual intervention. • Orchestrated data workflows using Apache Airflow and managed version control, collaboration, and CI/CD deployment using Git and Azure DevOps. • Leveraged Terraform for infrastructure-as-code (IaC) to provision and manage Azure resources for data platform deployment and automation. •Conducted Snowflake performance tuning, developed virtual warehouses, and utilized Streams & Tasks for near real-time data processing and Time Travel for auditability.
Project Overview: Worked within Citibank’s Global Data Services team, supporting a portfolio of Big Data regulatory reporting and capital markets modernization projects. The role involved building scalable data pipelines and delivering secure, governed, and analytics-ready datasets. Responsibilities: • Developed scalable ETL/ELT pipelines using Azure Data Factory, PySpark, and SQL to ingest, transform, and validate data from multiple banking systems into Snowflake. • Designed and implemented data lakehouse architecture using Azure Data Lake Storage (ADLS) and Delta Lake, enabling both batch and near real-time processing. • Built and optimized data models (star/snowflake schemas) in Snowflake, supporting regulatory and liquidity risk reporting requirements. • Utilized Streams & Tasks, Time Travel, and Virtual Warehouses in Snowflake to implement incremental loads, rollback capabilities, and cost-efficient compute scaling. • Created parameterized notebooks in Azure Databricks for large-scale data transformations and advanced analytics use cases using Spark (Scala/PySpark). • Ensured data security and governance through Role-Based Access Control (RBAC), masking policies, and lineage tracking via Azure Purview. • Enabled BI and analytics integration by exposing curated datasets to Power BI and Tableau, supporting teams across finance and risk. • Followed CI/CD best practices using Azure DevOps, Git, and Terraform for code versioning, pipeline deployment, and infrastructure automation. • Participated in Agile Scrum ceremonies, collaborated with cross-functional teams, and maintained detailed technical documentation for all data engineering solutions
Project Overview: Worked on a project portfolio worth CAD 25 million of data mapping, data transformation & migration projects post RBC- HSBC merger using MS Azure Databricks Big Data platform and other large-scale technology implementations for top North American Financial institutions. Responsibilities: • Designed, developed, and maintained scalable data pipelines using Databricks and Apache Spark • Implemented ETL/ELT workflows for processing structured and unstructured data. • Optimized and fine-tuned Spark jobs for performance and cost-efficiency. • Collaborated with Data Scientists, Analysts, and Engineers to integrate data from various sources. • Worked with Delta Lake, notebooks, and MLflow to support analytics and machine learning initiatives. • Ensured Data Quality, Governance, and Compliance standards are met. • Monitored and troubleshooted data pipelines in production environments. • Automated data validation and pipeline testing processes. • Participated in code reviews, architecture discussions, and Agile ceremonies
Project Overview: This role was responsible for managing a highly complex Organizational change project of the implementation of a new Electronic Medical Record (EMR) system for the University Health Network, the largest hospital chain in Ontario, funded by the provincial government. Responsibilities: • Worked closely with project managers and system integrators to align data engineering deliverables with key EMR rollout milestones. • Collaborated with EMR implementation teams and clinical stakeholders to define data requirements and support real-time reporting needs across departments. • Designed and implemented secure, scalable ETL pipelines using Azure Data Factory, Azure Databricks, and ADLS Gen2 to integrate clinical and administrative data from multiple legacy systems. • Built and maintained reusable Delta Lake tables and data models to support longitudinal patient analytics and care coordination dashboards. • Leveraged Spark (PySpark) on Databricks to handle large volumes of semi-structured EMR data for downstream analytics and reporting in Power BI. • Applied data governance, security (Key Vault), and compliance standards in alignment with healthcare regulations such as PHIPA and provincial Health Ontario mandates. • Integrated hospital data into the centralized Azure Synapse environment for enterprise-wide querying and regulatory reporting.