Sida Peng Ph.D. Student 3D Vision Group College of Computer Science and Technology Zhejiang University Email: pengsida at zju dot edu dot cn [Google Scholar]

Short Bio

I am a fifth year (2018-) Ph.D. student in Computer Science at Zhejiang University, supervised by Prof. Xiaowei Zhou. I obtained my bachelor degree in Information Engineering from Zhejiang University in 2018. My research focuses on 3D reconstruction, rendering, and 3D generation. My CVPR 2021 paper was selected as the CVPR best paper candidate, and I have 5 papers accepted as CVPR Highlight/Oral. I received the 2020 CCF-CV Excellent Young Researcher Award (3 in China were awarded) and was selected as the 2022 Apple Scholar in AI/ML (15 in the world were awarded).

Preprints

• Ponder: Point Cloud Pre-training via Neural Rendering
  Di Huang, Sida Peng, Tong He, Xiaowei Zhou, Wanli Ouyang
  [Paper][Project Page]

• Deep Generative Models on 3D Representations: A Survey
  Zifan Shi*, Sida Peng*, Yinghao Xu*, Yiyi Liao, Yujun Shen (* equal contribution)
  [Paper][Project Page]

• Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos
  Sida Peng, Zhen Xu, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao, Xiaowei Zhou
  [Paper][Supplementary Material][Project Page][Code]

Publications

• Representing Volumetric Videos as Dynamic MLP Maps
  Sida Peng*, Yunzhi Yan*, Qing Shuai, Hujun Bao, Xiaowei Zhou (* equal contribution)
  CVPR 2023.
  [Paper][Project Page][Code]

• Learning Neural Volumetric Representations of Dynamic Humans in Minutes
  Chen Geng*, Sida Peng*, Zhen Xu*, Hujun Bao, Xiaowei Zhou (* equal contribution)
  CVPR 2023.
  [Paper][Project Page]

• Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask
  Shangzhan Zhang, Sida Peng, Tianrun Chen, Linzhan Mou, Haotong Lin, Kaicheng Yu, Yiyi Liao, Xiaowei Zhou
  CVPR 2023.
  [Paper][Project Page]

• AutoRecon: Automated 3D Object Discovery and Reconstruction
  Yuang Wang, Xingyi He, Sida Peng, Haotong Lin, Hujun Bao, Xiaowei Zhou
  CVPR 2023 Highlight.
  [Paper][Project Page][Code]

• Learning Human Mesh Recovery in 3D Scenes
  Zehong Shen, Zhi Cen, Sida Peng, Qing Shuai, Hujun Bao, Xiaowei Zhou
  CVPR 2023.
  

• Grid-guided Neural Radiance Fields for Large Urban Scenes
  Linning Xu*, Yuanbo Xiangli*, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin (* equal contribution)
  CVPR 2023.
  [Paper][Project Page]

• Neural Scene Chronology
  Haotong Lin, Qianqian Wang, Ruojin Cai, Sida Peng, Hadar Elor, Xiaowei Zhou, Noah Snavely
  CVPR 2023.
  

• DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
  Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, Sergey Tulyakov
  CVPR 2023 Highlight.
  [Paper][Project Page][Code]

• Learning 3D-aware Image Synthesis with Unknown Pose Distribution
  Zifan Shi*, Yujun Shen*, Yinghao Xu, Sida Peng, Yiyi Liao, Sheng Guo, Qifeng Chen, Dit-Yan Yeung (* equal contribution)
  CVPR 2023.
  [Paper][Project Page][Code]

• Implicit Neural Representations with Structured Latent Codes for Human Body Modeling
  Sida Peng, Chen Geng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Xiaowei Zhou, Hujun Bao
  TPAMI 2023.
  [Paper][Code]

• TotalSelfScan: Learning Full-body Avatars from Self-Portrait Videos of Faces, Hands, and Bodies
  Junting Dong, Qi Fang, Yudong Guo, Sida Peng, Qing Shuai, Hujun Bao, Xiaowei Zhou
  NeurIPS 2022.
  [Paper][Project Page][Code]

• Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
  Haotong Lin*, Sida Peng*, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, Xiaowei Zhou (* equal contribution)
  SIGGRAPH Asia 2022.
  [Paper][Project Page][Code]

• MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images
  Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, Xin Tong
  TPAMI 2022.
  [Paper][Project Page][Code]

• NerfCap: Human Performance Capture with Dynamic Neural Radiance Fields
  Kangkan Wang, Sida Peng, Xiaowei Zhou, Jian Yang, and Guofeng Zhang
  TVCG 2022.
  [Paper]

• Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
  Yuan Liu, Yilin Wen, Sida Peng, Cheng Lin, Xiaoxiao Long, Taku Komura, Wenping Wang
  ECCV 2022.
  [Paper][Project Page][Code]

• Learning to Estimate Object Poses without Real Image Annotations
  Haotong Lin*, Sida Peng*, Zhize Zhou, Xiaowei Zhou (* equal contribution)
  IJCAI 2022.
  [Paper][Code]

• Novel View Synthesis of Human Interactions from Sparse Multi-view Videos
  Qing Shuai, Chen Geng, Qi Fang, Sida Peng, Wenhao Shen, Hujun Bao, Xiaowei Zhou
  SIGGRAPH 2022.
  [Paper][Project Page][Code]

• Neural 3D Scene Reconstruction with the Manhattan-world Assumption
  Haoyu Guo*, Sida Peng*, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou (* equal contribution)
  CVPR 2022 Oral presentation.
  [Paper][Supplementary Material][Project Page][Code]

• 3D-aware Image Synthesis via Learning Structural and Textural Representations
  Yinghao Xu, Sida Peng, Ceyuan Yang, Yujun Shen, Bolei Zhou
  CVPR 2022.
  [Paper][Project Page][Code]

• Neural Rays for Occlusion-aware Image-based Rendering
  Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, Wenping Wang
  CVPR 2022.
  [Paper][Project Page][Code]

• Towards Efficient and Photorealistic 3D Human Reconstruction: A Brief Survey
  Lu Chen, Sida Peng, Xiaowei Zhou
  Visual Informatics 2022.
  [Paper]

• Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies
  Sida Peng*, Junting Dong*, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao (* equal contribution)
  ICCV 2021.
  [Paper][Supplementary Material][Project Page][Code][Extension]

• Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans
  Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou
  CVPR 2021 Best paper candidate.
  [Paper][Supplementary Material][Project Page][Code]

• PVNet: Pixel-wise Voting Network for 6DoF Object Pose Estimation
  Sida Peng*, Xiaowei Zhou*, Yuan Liu, Haotong Lin, Qixing Huang, Hujun Bao (* equal contribution)
  TPAMI 2020.
  [Paper][Code]

• Deep Snake for Real-Time Instance Segmentation
  Sida Peng, Wen Jiang, Huaijin Pi, Xiuli Li, Hujun Bao, Xiaowei Zhou
  CVPR 2020 Oral presentation.
  [Paper][Code]

• GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs
  Yuan Liu, Zehong Shen, Zhixuan Lin, Sida Peng, Hujun Bao, Xiaowei Zhou
  NeurIPS 2019.
  [Paper][Supplementary Material][Project Page][Code]
  

• PVNet: Pixel-wise Voting Network for 6DoF Estimation
  Sida Peng*, Yuan Liu*, Qixing Huang, Xiaowei Zhou, Hujun Bao (* equal contribution)
  CVPR 2019 Oral presentation.
  [Paper][Supplementary Material][Project Page][Code]

Honors

• Outstanding Graduates of Zhejiang Province
  2023
• National Scholarship
  2022
• Apple Scholars in AI/ML PhD Fellowship
  2022
• National Scholarship
  2021
• Style3D Graduate Fellowship
  2021
• First place in China Graduate AI Innovation Competition
  2021
• National Scholarship
  2020
• CCF-CV Excellent Young Researcher Award
  2020
• First place in China Graduate AI Innovation Competition
  2019
• National Scholarship
  2019
• Chen TianZhou Educational Foundation Scholarship
  2019
• Chiang Chen Industrial Charity Foundation Grant
  2019
• Graduate Student Scholarship
  2019

Teaching Assistant

• Computational Photography
  Spring, 2023
• Computational Photography
  Spring, 2022
• Computational Photography
  Spring, 2021
• Computational Photography
  Spring, 2020
• Computational Photography
  Spring, 2019