Currently, he is a research fellow at Nanyang Technological University (NTU) Singapore, working with Prof. Ziwei Liu. Prior to this, he worked as a Research Scientist at the Sea AI Lab of Sea Group.

He completed Ph.D. degree in 2021 at CASIA, supervised by Prof. Tieniu Tan, co-supervised by Prof. Liang Wang and Prof. Wei Wang. He was a visting student from Sep, 2019 to Sep, 2020 in Learning and Vision Lab at National University of Singapore, working with Prof. Jiashi FENG.

His research lies at the intersection of deep learning and computer vision, including vision-based human perception (pose and action), few-shot learning, self-supervised learning, semi-supervised learning, in-context learning and video generation.

If you are seeking research interns (AIGC), feel free to contact me!

Remote cooperations are welcome (ChenYang.Si.Mail@gmail.com).

Contact

  • Email:
    • ChenYang.Si.Mail@gmail.com
    • chenyang.si@ntu.edu.sg
    • sicy@sea.com
    • chenyang.si@cripac.ia.ac.cn

Publications

  • FreeU: Free Lunch in Diffusion U-Net
    Chenyang Si, Ziqi Huang, Yuming Jiang, Ziwei Liu
    CVPR, 2024
    [pdf] [project page] [code] [video]

  • VBench: Comprehensive Benchmark Suite for Video Generative Models
    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, Ziwei Liu
    CVPR, 2024
    [pdf] [project page] [code] [video]

  • VideoBooth: Diffusion-based Video Generation with Image Prompts
    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, Ziwei Liu
    CVPR, 2024
    [pdf] [project page] [code] [video]

  • Towards Language-Driven Video Inpainting via Multimodal Large Language Models
    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, Chen Change Loy
    CVPR, 2024
    [pdf] [project page]

  • Scaling Supervised Local Learning with Augmented Auxiliary Networks
    Chenxiang Ma, Jibin Wu, Chenyang Si, KC Tan
    ICLR, 2024

  • FreeInit: Bridging Initialization Gap in Video Diffusion Models
    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, Ziwei Liu
    Tech report, 2023
    [pdf] [project page] [code] [video]

  • Lavie: High-quality video generation with cascaded latent diffusion models
    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, Ziwei Liu
    Tech report, 2023
    [pdf] [project page] [code]

  • Frequency-Enhanced Data Augmentation for Vision-and-Language Navigation
    Keji He, Chenyang Si, Zhihe Lu, Yan Huang, Liang Wang, Xinchao Wang
    NeurIPS, 2023
    [pdf]

  • FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation
    Jingwen Guo, Hong Liu, Shitong Sun, Tianyu Guo, Min Zhang, Chenyang Si*
    (ICCV), 2023
    [pdf]

  • Semantic Prompt for Few-Shot Learning
    Wentao Chen*, Chenyang Si*, Zhang Zhang, Liang Wang, Zilei Wang, Tieniu Tan
    (CVPR), 2023
    [pdf] [code] [video]

  • Metaformer baselines for vision
    Weihao Yu, Chenyang Si, Pan Zhou, Mi Luo, Yichen Zhou, Jiashi Feng, Shuicheng Yan, Xinchao Wang
    TPAMI, 2023
    [pdf] [code]

  • Federated Zero-Shot Learning with Mid-Level Semantic Knowledge Transfer
    Shitong Sun, Chenyang Si, Shaogang Gong, Guile Wu
    Tech report, 2022
    [pdf]

  • Mugs: A Multi-Granular Self-Supervised Learning Framework
    Pan Zhou, Yichen Zhou, Chenyang Si, Weihao Yu, Teck Khim Ng, Shuicheng Yan
    NeurIPS Workshop, 2022
    [pdf] [code]

  • Transformed Autoencoder: Pre-training with Mask-Free Encoder and Transformed Decoder
    Yichen Zhou, Pan Zhou, Chenyang Si, Weihao Yu, Teck Khim Ng, Shuicheng YAN
    NeurIPS Workshop, 2022
    [[pdf]]

  • Inception Transformer
    Chenyang Si, Weihao Yu, Pan Zhou, Yichen Zhou, Xinchao Wang, Shuicheng Yan
    (NeurIPS, Oral), 2022
    [pdf] [code]

  • Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
    Peng Wang, Jun Wen, Chenyang Si, Yuntao Qian, Liang Wang
    (TIP), 2022
    [pdf]

  • Metaformer is actually what you need for vision
    Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, Shuicheng Yan
    (CVPR), 2022
    [pdf] [code]

  • Generalizable Person Re-Identification via Self-Supervised Batch Norm Test-Time Adaption
    Ke Han, Chenyang Si, Yan Huang, Liang Wang, Tieniu Tan
    (AAAI), 2022
    [pdf]

  • Few-Shot Learning with Part Discovery and Augmentation from Unlabeled Images
    Wentao Chen, Chenyang Si, Wei Wang, Liang Wang, Zilei Wang, Tieniu Tan
    (IJCAI), 2021
    [[pdf]]

  • Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognition
    Chenyang Si, Xuecheng Nie, Wei Wang, Liang Wang, Tieniu Tan, Jiashi Feng
    (ECCV), 2020
    [pdf] [video]

  • Unsupervised Motion Representation Learning from Skeleton Sequence via Neighbor Reconstruction
    Chenyang Si, Wei Wang, Liang Wang, Tieniu Tan, Jiashi Feng

  • Skeleton-Based Action Recognition with Hierarchical Spatial Reasoning and Temporal Stack Learning Network
    Chenyang Si, Ya Jing, Wei Wang, Liang Wang, Tieniu Tan
    (Pattern Recognition), 2020
    [pdf]

  • Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search
    Ya Jing, Chenyang Si, Junbo Wang, Wei Wang, Liang Wang, Tieniu Tan
    (AAAI), oral, 2020
    [pdf]

  • An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition
    Chenyang Si, Wentao Chen, Wei Wang, Liang Wang, Tieniu Tan
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019
    [pdf]

  • Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning
    Chenyang Si, Ya Jing, Wei Wang, Liang Wang, Tieniu Tan
    European Conference on Computer Vision (ECCV), 2018
    [pdf]

  • Pose-Based Two-Stream Relational Networks for Action Recognition in Videos
    Wei Wang, Jinjin Zhang, Chenyang Si, Liang Wang
    (arxiv), 2018
    [pdf]

  • Multistage adversarial losses for pose-based human image synthesis
    Chenyang Si, Wei Wang, Liang Wang, Tieniu Tan
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR,), spotlight, 2018
    [pdf]

Teaching Experience

  • Teaching Assistant
    Deep Learning
    Mar. 2019-Jun. 2019
    University of Chinese Academy of Sciences