WP. Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
- Collaborated with senior data engineers to develop and implement ETL processes to ingest data from various sources, transform it into usable formats, and load it into Alibaba Cloud MaxCompute. - Conducted in-depth data analysis, generating insights that informed key business decisions, and provided recommendations for optimization. - Assisted in real-time data processing by leveraging Apache Flink to handle incoming data streams from the MLFF system, efficiently integrating data into Alibaba MaxCompute Hologres. - Managed hot and cold storage strategies, optimizing data retention and access speed for different data types based on usage patterns. - Monitored, maintained, and troubleshooted data pipeline issues under the guidance of senior data engineers.
- Collaborated with senior data engineers to develop and implement ELT processes to extract data from various sources, transform it into usable formats, and load it into target database(GCP). - Developed a robust data ingestion pipeline using existing internal functions and library, and Apache Airflow, automating the extraction, transformation, and loading of data from multiple sources into Google Cloud. - Contributed to the creation of data visualization dashboards using Looker Studio to communicate key metrics and trends to stakeholders effectively. - Monitored, maintained, and troubleshooted data pipeline issues with senior data engineers.