作者：Ce Zheng Guo-Jun Qi Chen Chen
人体网格恢复（HMR）为游戏、人机交互和虚拟现实等各种现实世界应用提供了丰富的人体信息。与基于单个图像的方法相比，基于视频的方法可以利用时间信息，通过引入人体运动先验来进一步提高性能。然而，像VIBE这样的多对多方法存在运动平滑和时间不一致的问题。而像TCMR和MPS-Net这样的多对一方法依赖于未来的框架，这是非因果的，并且在推理过程中时间效率低下。为了应对这些挑战，提出了一种新颖的基于扩散驱动变压器的视频HMR框架（DDT）。DDT被设计用于从输入序列中解码特定的运动模式，增强运动平滑度和时间一致性。作为一种多对多的方法，我们的DDT解码器输出所有帧的人体网格，使DDT在时间效率至关重要的现实世界应用中更可行
Human mesh recovery (HMR) provides rich human body information for variousreal-world applications such as gaming, human-computer interaction, and virtualreality. Compared to single image-based methods, video-based methods canutilize temporal information to further improve performance by incorporatinghuman body motion priors. However, many-to-many approaches such as VIBE sufferfrom motion smoothness and temporal inconsistency. While many-to-one approachessuch as TCMR and MPS-Net rely on the future frames, which is non-causal andtime inefficient during inference. To address these challenges, a novelDiffusion-Driven Transformer-based framework (DDT) for video-based HMR ispresented. DDT is designed to decode specific motion patterns from the inputsequence, enhancing motion smoothness and temporal consistency. As amany-to-many approach, the decoder of our DDT outputs the human mesh of all theframes, making DDT more viable for real-world applications where timeefficiency is crucial and a causal model is desired. Extensive experiments areconducted on the widely used datasets (Human3.6M, MPI-INF-3DHP, and 3DPW),which demonstrated the effectiveness and efficiency of our DDT.
论文链接：http://arxiv.org/pdf/2303.13397v1
更多计算机论文：http://cspaper.cn/