Poland
My work in telemetry team is focused around creating ETL/ELT jobs with the help of Apache Airflow to build data models inside our data warehouse in SQL server where we use them for reporting purposes. Main delivery from our team that I'm a huge part of is providing a Unified Data Platform to integrate data from Bitbucket, Jenkins, CS vendors, Kubernetes, LDAP, Copilot and other tools for mapping lineage and ownership at domain, application, and developer levels. Besides writing data pipelines I act as a db admin and I'm also maintaining micro services related to data models that I host on k8s which I deploy with simple Jenkins jobs.
Recent projects: - Developed and migrated data model for security reporting that covers the whole company. The closest description is ELT supported data medallion with microservice enrichment. It solved identifying ownership of vulnerabilities, increased speed of fixing weak points and saved millions annually in licences. - Developed analytic data mart for AI adoption metrics. Cause of this company saves a lot of money on licence magemennt. - Migrated airflow service to helm charts which increased speed of onboarding new tenants to the application, - Developed key features in Airflow 3.x including secure plugins, custom SSO authorization and custom API authentication, - Developed streamlit app for our team in k8s which gives others developers more option over the type of reports that they want to create.
Providing L2 suport for core application