Post by ONCORE AI

Millions of historical television, radio and video recordings are now accessible through De Schatkamer, the new online media archive of Nederlands Instituut voor Beeld en Geluid. If you're interested in how platforms like these are built behind the scenes, Tweakers recently published an excellent technical deep dive into the architecture behind De Schatkamer. It's a great read for anyone curious about the engineering that powers large-scale digital platforms. While visitors experience a fast and intuitive search platform, a substantial data infrastructure operates behind the scenes. More than 1.3 million media assets, metadata from multiple source systems, editorial updates and complex rights information all need to be processed, synchronised and made searchable. ONCORE AI was responsible for designing and building the platform's core data infrastructure, working closely with the technology team at Beeld & Geluid and our partner Hypersolid. Our contribution included the end-to-end metadata pipeline and the search architecture that powers the archive, ensuring the entire foundation is fully AI-ready. For those interested in the technical architecture, some of the key building blocks include: - The Metadata Pipeline: An automated architecture built with Apache Airflow and dbt that ingests, cleanses, and structures fragmented data from multiple internal source systems. - CMS Sync: Moving away from slow batch processing so that editorial updates flow through the entire chain and are live within minutes. - Enterprise Search Engineering: A high-performance solution built on Elasticsearch 9.x, fine-tuned to deliver instant retrieval for the public. - Future-Proof Foundation: A cloud-native, containerized infrastructure that prevents vendor lock-in and creates the clean data foundation required to seamlessly embed operational AI workflows moving forward. We're proud to have contributed to a project like De Schatkamer. Seeing it come to life has been incredibly rewarding, and we'd like to sincerely thank the teams at Nederlands Instituut voor Beeld en Geluid and Hypersolid for the great collaboration throughout the project. 👉 Curious about the technology behind De Schatkamer? Read the full Tweakers article here: https://lnkd.in/e-azA26g #OncoreAI #DataEngineering #DataArchitecture #AIReady #AIworkflow