Post by Matthias Seibold

[Watch with sound 🔊] In our IPCAI 2026 paper "Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes", we introduce a novel framework for generating 4D audio-visual representations of surgical scenes by detecting acoustic events and projecting acoustic localization information from a phased microphone array onto dynamic point clouds obtained from a RGB-D camera. This work introduces the first approach for spatial sound localization in dynamic surgical scenes, marking a significant advancement towards multimodal surgical scene representations. By integrating acoustic and visual data, the proposed framework enables richer contextual understanding and provides a foundation for future intelligent and autonomous surgical systems. Thanks to all co-authors and supporters! Author list: Jonas Hein, Lazaros Vlachopoulos, Maurits Olthof, Bastian Sigrist, Philipp Fürnstahl, Matthias Seibold Society for the Advancement of Applied Computer Science (GFaI) e.V. - Michael Markus Ackermann, Carsten Hessenius, Andy Meyer Norsonic Brechbühl AG - Matthias Brechbühl Swiss National Science Foundation SNSF Universitätsklinik Balgrist Research In Orthopedic Computer Science OR-X

Video Content