作者：Willi Menapace Aliaksandr Siarohin Stéphane Lathuilière Panos Achlioptas Vladislav Golyanik Elisa Ricci Sergey Tulyakov
游戏引擎是计算机图形学中强大的工具。他们的力量是以巨大的发展代价换来的。在这项工作中，我们提出了一个框架来训练类似游戏引擎的神经模型，仅从单眼注释视频中获得。其结果——可学习游戏引擎（LGE）——保持场景、对象和代理的状态，并能够从可控视点渲染环境。与游戏引擎类似，它对游戏的逻辑和潜在的物理规则进行建模，通过指定高级别和低级别的动作序列，用户可以玩游戏。最吸引人的是，我们的LGE解锁了导演模式，在这种模式下，游戏是通过在幕后策划，以语言和所需状态的形式为特工指定高级动作和目标来进行的。这需要学习由我们的动画模型封装的“游戏AI”，以使用高级约束在场景中导航，对抗对手，制定赢得分数的策略。钥匙托莱阿
Game engines are powerful tools in computer graphics. Their power comes atthe immense cost of their development. In this work, we present a framework totrain game-engine-like neural models, solely from monocular annotated videos.The result-a Learnable Game Engine (LGE)-maintains states of the scene, objectsand agents in it, and enables rendering the environment from a controllableviewpoint. Similarly to a game engine, it models the logic of the game and theunderlying rules of physics, to make it possible for a user to play the game byspecifying both high- and low-level action sequences. Most captivatingly, ourLGE unlocks the director’s mode, where the game is played by plotting behindthe scenes, specifying high-level actions and goals for the agents in the formof language and desired states. This requires learning “game AI”, encapsulatedby our animation model, to navigate the scene using high-level constraints,play against an adversary, devise the strategy to win a point. The key tolearning such game AI is the exploitation of a large and diverse text corpus,collected in this work, describing detailed actions in a game and used to trainour animation model. To render the resulting state of the environment and itsagents, we use a compositional NeRF representation used in our synthesis model.To foster future research, we present newly collected, annotated and calibratedlarge-scale Tennis and Minecraft datasets. Our method significantly outperformsexisting neural video game simulators in terms of rendering quality. Besides,our LGEs unlock applications beyond capabilities of the current state of theart. Our framework, data, and models are available athttps://learnable-game-engines.github.io/lge-website.
论文链接：http://arxiv.org/pdf/2303.13472v1
更多计算机论文：http://cspaper.cn/