Singapore
Big data engineer at TikTok Search | Singaporean
Data infrastructure • Developed mid-layer data infrastructure for TikTok search domain, improving the efficiency of data development and downstream usage. By optimizing the data pipeline with both big data techniques and business insights, the overall data processing duration and compute resource usage have been reduced by more than 65% Business Support • Built both real-time and batch data pipelines, A/B test metric systems, and dashboards for multiple TikTok vertical search domains' core metrics
- Designed and optimized a RAG framework to extract insights from unstructured corporate reports. - Developed and benchmarked evaluation metrics (correctness & completeness) for model performance. - Conducted iterative solution refinement for RAG framework with business teams and optimized document ingestion and retrieval pipelines, improving accuracy and efficiency by fine-tuning hyperparameters for enhanced model performance.
•Evaluated and benchmarked the company's current system against open source and other available algorithms based on speed, user-friendliness, and performance matrix, by applying the algorithms on multiple datasets with more than 200,000 images in different scenes such as detection and classification. •Identified 20+ issues and proposed improvements for the computer vision deep learning algorithm and Graphic UI.
•Developed an Ultra-wideband vital sensing simulation in Python, using Numpy and Object-Oriented Programming techniques; carried out thousands of rounds of automated simulation for parameter optimisation and improved the detection accuracy by 55%. •Conducted literature reviews of various state-of-the-art digital signal processing algorithms and applied 6 algorithms to the project; constructed a real-time dynamic Graphical User Interface for interactive simulation with PyQt.