作者：Ege Özsoy Tobias Czempiel Felix Holm Chantal Pellegrini Nassir Navab
现代手术是在复杂而动态的环境中进行的，包括医护人员、患者和设备之间不断变化的互动。因此，手术室（OR）的整体建模是一项具有挑战性但至关重要的任务，有可能优化手术团队的绩效，并有助于开发新的手术技术，以改善患者的预后。将手术场景整体表示为语义场景图（SGG），其中实体被表示为节点，它们之间的关系被表示为边缘，这是细粒度语义OR理解的一个很有前途的方向。我们首次提出使用时间信息进行更准确和一致的整体OR建模。具体来说，我们引入了记忆场景图，其中先前时间步骤的场景图充当指导当前预测的时间表示。我们设计了一种端到端的架构，可以智能地融合我们轻量级的时间信息
Modern surgeries are performed in complex and dynamic settings, including ever-changing interactions between medical staff, patients, and equipment. The holistic modeling of the operating room (OR) is, therefore, a challenging but essential task, with the potential to optimize the performance of surgical teams and aid in developing new surgical technologies to improve patient outcomes. The holistic representation of surgical scenes as semantic scene graphs (SGG), where entities are represented as nodes and relations between them as edges, is a promising direction for fine-grained semantic OR understanding. We propose, for the first time, the use of temporal information for more accurate and consistent holistic OR modeling. Specifically, we introduce memory scene graphs, where the scene graphs of previous time steps act as the temporal representation guiding the current prediction. We design an end-to-end architecture that intelligently fuses the temporal information of our lightweight memory scene graphs with the visual information from point clouds and images. We evaluate our method on the 4D-OR dataset and demonstrate that integrating temporality leads to more accurate and consistent results achieving an +5% increase and a new SOTA of 0.88 in macro F1. This work opens the path for representing the entire surgery history with memory scene graphs and improves the holistic understanding in the OR. Introducing scene graphs as memory representations can offer a valuable tool for many temporal understanding tasks.
论文链接：http://arxiv.org/pdf/2303.13293v1
更多计算机论文：http://cspaper.cn/