The Randstad, Netherlands
• Leading a Team of 9 Data Engineers on Data Engineering best practices and soft skills development • Successfully lead the team through a full data migration from AWS to GCP in 6 months starting from the very complex dependencies landscape of the in house Logistics Analytics • Managing Requirements and Priotization for Stakeholders while also leading the team on efficient AGILE practices • Taking a proactive and leading role in defining JET data platforms by cross collaborating with different platform teams in technical discussions and vision shaping • Lead development of an harmonised Single Source of Truth in DBT for global logistics data at company level for different delivery partners to fuel harmonised KPIs for reporting • Supporting a healthy and balanced team dynamic with the aim of promoting an inclusive, flexible and work-life balanced workplace
• Coaching team members on best practices on data processing and software engineering through peer review and constructive feedback • Leading on the breakdown of high level deliverables into measurable and workable unit with clear business value for the team • Identifying objectives and analyzing feasibility for the quarterly planning in joint effort with the PO • Contributing to promoting the InnerSource culture within the organization and worked as projects maintainer and reviewer • Contributed to a reusable Azure blueprint for a self service data platform on both infrastructure and application layers • Guided and supported 15+ teams using reusable Data Platform blueprints on best practices and ad hoc use cases • Developed Extendible self service Ingestion Framework based on Python, Azure functions and Databricks with a customizable JSON Interface per data source • Developed a self service ETL frameworks with a JSON interface based on Python/Apache Spark running on Azure Databricks to enable data value creation for multiple Data Teams • Deployed a centralized orchestration solution for Data Workflows with multiple Apache Airflow instances deployed with Helm on a multi tenancy Azure Kubernetes cluster
• Enabled multi cloud real-time data stream between AWS and GCP for real time orders analysis by leveraging on Apache Kafka, Kafka Connect and Google Pubsub • Coaching team members on Data Engineering Best Practices • Contributing shaping up team long term goals and feasibility • Supporting team lead in team management and guidance
• Built a self-service Data Platform to promote a Data Centric view, powered by an Apache Airflow Cluster deployed on AWS in High Availability using Puppet and Cloudformation. • Built process to automatically allocate company costs using Apache Beam on GCP Dataflow • Built Data Lake on Google BigQuery powered by Deployment Manger and Apache Airflow. • Built a Customers Deduplication model with Spark and Graph Theory on GCP Dataproc to cluster multiple user accounts of the same customer • Gave guest lecture about Data Dynamics at Erasmus University to MSc students • Built Smart Metrics for autoscaling of Apache Airflow workers to face peak periods automatically using Amazon Cloudwatch and Puppet • Collaborated with Business Analysts to rewrite Marketing data flows on Apache Airflow • Built Observability dashboards for data pipelines on Datadog and Splunk
• Managed a Cloudera Hadoop Cluster to host the Big Data infrastructure used for Data Ingestion, ETL and Analytics • Built Near Realtime data ingestion system of vehicles trajectories with Apache Kafka • Automated ETL with scheduled Apache Hive jobs on Apache Oozie • Built Insights reporting web portal in AngularJS with Java backend