作者：Chong Bao Yinda Zhang Bangbang Yang Tianxing Fan Zesong Yang Hujun Bao Guofeng Zhang Zhaopeng Cui
尽管在使用用户友好的工具（如Photoshop、语义笔画甚至文本提示）进行2D编辑方面取得了巨大成功，但3D领域的类似功能仍然有限，要么依赖于3D建模技能，要么只允许在少数类别中进行编辑。在本文中，我们提出了一种新颖的语义驱动的NeRF编辑方法，它使用户能够用单个图像编辑神经辐射场，并以高保真度和多视图一致性忠实地提供编辑后的新视图。为了实现这一目标，我们提出了一个先验引导编辑场来编码3D空间中的细粒度几何和纹理编辑，并开发了一系列技术来帮助编辑过程，包括使用代理网格进行循环约束以便于几何监督，使用颜色合成机制来稳定语义驱动的纹理编辑，以及使用基于特征聚类的正则化来保持不相关内容不变
Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories.In this paper, we present a novel semantic-driven NeRF editing approach, which enables users to edit a neural radiance field with a single image, and faithfully delivers edited novel views with high fidelity and multi-view consistency.To achieve this goal, we propose a prior-guided editing field to encode fine-grained geometric and texture editing in 3D space, and develop a series of techniques to aid the editing process, including cyclic constraints with a proxy mesh to facilitate geometric supervision, a color compositing mechanism to stabilize semantic-driven texture editing, and a feature-cluster-based regularization to preserve the irrelevant content unchanged.Extensive experiments and editing examples on both real-world and synthetic data demonstrate that our method achieves photo-realistic 3D editing using only a single edited image, pushing the bound of semantic-driven editing in 3D real-world scenes. Our project webpage: https://zju3dv.github.io/sine/.
论文链接：http://arxiv.org/pdf/2303.13277v1
更多计算机论文：http://cspaper.cn/