Singapore
Profile summary, please use the chatbot: https://tanyanchonglinkedin-cgdfavgmg8e8dyav.southeastasia-01.azurewebsites.net/
Architected and built distributed image-processing pipelines on Azure Databricks using Apache Spark, applying Adaptive Query Execution, selective disk and memory caching, and vectorised Pandas UDFs to improve processing throughput Engineered both structured and semi-structured data pipelines on Azure Databricks to ingest API and questionnaire data into Bronze, Silver, and Gold layers, applying schema validation, deduplication, and incremental merges Developed and deployed Azure Machine Learning batch pipelines for large-scale, cost-efficient, deep learning and LLM-based image and video processing and inference Built event-driven orchestration using Azure Durable Functions, Storage Queues, and triggers to decouple processing stages and support retries, failure recovery, and asynchronous execution Standardized Databricks deployments across development and production using Databricks Asset Bundles and CI/CD, improving deployment consistency and reproducibility Migrated pipeline authentication to Azure Managed Identities and Service Principals, eliminating shared secrets and securing cross-subscription connectivity through Private Endpoints and RBAC Benchmarked inference workloads across Azure CPU and GPU compute SKUs, identifying configurations that balanced processing performance and infrastructure cost
Developed and maintained PySpark and Scala pipelines that transformed raw gaming activity data into structured datasets for finance reporting and downstream analytics Contributed to reusable Scala components for common data transformations, reducing duplicated implementation across analytics workflows Supported scheduled batch-processing workflows in a legacy Hadoop environment using Oozie and Python Developed an end-to-end workflow to streamline user onboarding onto the data platform, including connectivity to existing data streams and setup of the required Spark and Hadoop processing environment
Built and maintained Python ETL pipelines to extract data from Lazada Open Platform, Google Analytics, AWS S3, and other external APIs, transform it into analysis-ready datasets, and load it into PostgreSQL Designed PostgreSQL tables and data-processing workflows that consolidated data from multiple business systems for reporting and downstream analytics Developed and deployed interactive R Shiny applications on Ubuntu using Docker and ShinyProxy, connecting curated PostgreSQL datasets to business-facing dashboards Built Tableau dashboards and administered Tableau Online, enabling self-service reporting across the organisation
Developed SQL Server data models and reporting datasets using complex stored procedures, transforming and aggregating transactional data into summary tables and combining them with detailed fact data to produce analysis-ready views Built and maintained SSIS ETL pipelines to extract, transform, and load data across operational and reporting systems Developed data transformations and analytical models to support CRM loyalty programme analysis and business reporting Ran point simulations to optimise loyalty programme tiering structures, directly informing business pricing decisions
Migrated operational reporting workflows from legacy Excel-based processes to MySQL, designing relational data models to store and organise solar-site sensor and meter data Built data preparation and reporting workflows that transformed operational data into structured datasets for Tableau dashboards and automated reporting Developed R-based outlier detection to identify anomalous site readings and support solar-site maintenance and operational monitoring