Abstract

This paper introduces a novel compact shape representation from captured volumetric video sequences of people. 4D volumetric video achieves highly realistic reproduction, replay and free-viewpoint rendering of actor performance from multiple view video acquisition. A variational encoder-decoder is trained on 3D skeletal motion sequences of an actor performing multiple motions to learn a generative model of the dynamic 4D shape. This provides a compact encoded representation capable of high-quality synthesis of the 4D shapes with two orders of magnitude compression.

Paper

Deep 4D Shape Representation: Learning 4D Volumetric Video from Skeletal Motion
João Regateiro, Marco Volino, and Adrian Hilton
ACM SIGGRAPH European Conference on Visual Media Production (CVMP) 2019



Citation

    @inproceedings{Regateiro:3DV:2019,
        AUTHOR = "Regateiro, Joao, and Volino, Marco, and Hilton, Adrian",
        TITLE = "Deep 4D Shape Representation: Learning 4D Volumetric Video from Skeletal Motion",
        BOOKTITLE = "ACM SIGGRAPH European Conference on Visual Media Production (CVMP) 2019",
        YEAR = "2019",
    }

Data

Public datasets used in this paper can be found in the CVSSP3D Publications and Datasets.

Acknowledgments

This research was supported by the EPSRC Audio-Visual Media Research Platform Grant (EP/P022529/1). The authors would also like to thank Adnane Boukhayma for providing the ’Thomas’ dataset used for evaluation.