Amsterdam, North Holland, Netherlands
Full-Stack Data Engineering, Delta Lake Warehousing, and DevOps Architecture. DevOps & Platform Design: Designing and managing the DevOps platform lifecycle, establishing deployment workflows and environment configuration standards. Environment Management: Automating containerized application orchestration across isolated ACC and PRD bare-metal nodes. Delta Lake & Warehousing: Implementing an on-premises Delta Lake storage framework using the Medallion architecture to serve as the analytical data warehouse layer. Python Software Engineering: Building modular data processing pipelines using Object-Oriented Python (OOP) with strict type-checking. Orchestration & Dashboards: Deploying Apache Airflow setups for workflow orchestration and Streamlit applications for the data serving layer.
Working on the config driven data analytics platform, enabling users to onboard data with ease. My responsibilities include: - Infrastructure as Code with Terraform and orchestration with Terragrunt - Configuration model development with Pydantic - Spark optimisation in Databricks - Release management - CI/CD pipeline development - Logging and alerting - Parametrised unit testing and e2e testing
Operating company: Beerwulf Core Data Engineering Skills • Translating Business Logic to Data Solutions; • Data Modelling (Data Vault, Kimball); • Managing ETL/ELT Pipelines; • Building, managing and improving the data platform (IaC); • Machine Learning Engineering; • Unit testing (pytest); Cloud & Infrastructure Skills • Cloud platform skills (Azure, Databricks) • Infrastructure as Code (Terraform) DevOps & Automation Skills • Azure DevOps • CI/CD Pipelines & automated testing Projects: • With the team we’re developing a new landing zone, Data Vault model, and a dimensional analysis layer, to replace the current structure; • Implementing infrastructure as code with Terraform to manage both Unity Catalog resources (schemas, permissions, external storage) and Databricks workspace resources (clusters, compute configurations); • Maximizing test coverage with extra care for code quality; • Developing and improving ML models for forecasting.
• Development of dashboards (Tableau/Power BI); • Development and maintenance of data warehouses (Azure SQL, SSMS); • Automation of data pipelines (ETL); • Data visualization and analysis (Python); • Predictive modelling/ machine learning. Projects: • Creating fully automated dashboards. In the process, files are transformed and loaded into the data warehouse. Via live connection, the dashboards shows multiple visualisations that give insight in the behaviour of assets based on statistics. • Development of a dashboards for province in the Netherlands, where deformation of multiple assets is shown. Multiple times series are fitted, and predictions are compared with actual values.
• Data Analysis (relational databases (SQL) and R); • Forecasting initial delivery quantities of new releases; • Coordinating flows of goods and information; • Process optimization; • Development of KPI dashboards (Power BI). Personal projects: • Clustering customers and products with machine learning; • Development of predictive models for predicting sales of new releases.