I am Xiangtai and I work as a Research Fellow at MMLab@NTU S-Lab advised by Prof.Chen Change Loy.
I obtained my PhD degree at Peking University (PKU). My PhD supervisor is Prof.Yunhai Tong. I obtained my bachelor degree at Beijing University of Posts and Telecommunications (BUPT).
I am working on the following research directions:
π Pixel-Wised Scene Understanding for Video/Image Scene Understanding, including (Semantic/Instance/Panoptic) Segmentation and Object Detection, zero/few shot variants.
π General deep learning Method with its application, including Vision Transformer, Efficient Model Design, Neural Collapse.
π Vision meets language, including Open Vocabulary Learning, Visual Prompting, Visual Grounding.
Previously, I did some research on Image/Video Semantic/Instance/Panoptic Segmentation as well as several related problems during my PhD.
π₯ News
- 2022.11οΌTwo paper on Video Scene Understanding is accepted by T-PAMI.
- 2022.09οΌOne paper on Neural Collapse is accepted by NeurIPS-2022.
- 2022.08οΌ Β ππ Join the MMLab@NTU S-Lab! Our four works (Video K-Net, PanopticPartFormer, FashionFormer, and PolyphonicFormer in CVPR-22/ECCV-22) code are all released. Check out my github homepage.
- 2022.07οΌ Β ππ Our SFNet-Lite (extension of SFNet-ECCV20) achieve the best mIoU and speed trade-off. on multiple driving datasets. SFNet-Lite can obtain 80.1 mIoU while running at 50 FPS, 78.8 mIoU while running at 120 FPS. Code.
- 2022.07: Β ππ Three papers are accepted by ECCV-2022. One paper is accepted by ICIP-2022.
- 2022.07: Β ππ Graduated From PKU.
- 2022.03: Β ππ Video K-Net is accepted by CVPR-2022 as oral presentation.
π Selected Publications
Full Publications Per Year can be found in Here.
* means equal contribution.
Code can be found in this.
Selected Conference
- Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation, Xiangtai Li, Shilin Xu, Yibo Yang, Guangliang Cheng, Yunhai Tong, Dacheng Tao, ECCV 2022 The first unified part-aware panoptic segmentation model | Code
- Fashionformer: A Simple, Effective and Unified Baseline for Human Fashion Segmentation and Recognition, Shilin Xu*, Xiangtai Li*, Jingbo Wang, Guangliang Cheng, Yunhai Tong, Dacheng Tao, ECCV 2022 | Code
- PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation, Haobo Yuan*, Xiangtai Li*, Yibo Yang, Guangliang Cheng, Jing Zhang, Yunhai Tong, Lefei Zhang, Dacheng Tao, ECCV 2022 Winner of ICCV-2021 BMTT workshop, The first unified depth aware video panoptic segmentation model | Code
- Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation, Xiangtai Li*, Wenwei Zhang*, Jiangmiao Pang*, Kai Chen, Guangliang Cheng, Yunhai Tong, Chen Change Loy, CVPR 2022 (Oral, top2%) The first unified video segmentation model and codebase for VPS, VIS, VSS | Code
- Semantic Flow for Fast and Accurate Scene Parsing, Xiangtai Li, Ansheng You, Zhen Zhu, Houlong Zhao, Maoke Yang, Kuiyuan Yang, Yunhai Tong, ECCV 2020 (Oral, top2%) The first real time model over 80\% mIoU on Cityscapes test set. | Code
- GFF: Gated Fully Fusion for Semantic Segmentation, Xiangtai Li, Houlong Zhao, Lei Han, Yunhai Tong, Kuiyuan Yang, AAAI 2020 (Oral) | Code
Selected Journal
- TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers , Qianyu Zhou*, Xiangtai Li* , Lu He, Yibo Yang, Guangliang Cheng, Yunhai Tong, Lizhuang Ma, Dacheng Tao, T-PAMI-2022 End-to-End Vision Transformer for Video Object Detection | Code
- Improving Video Instance Segmentation via Temporal Pyramid Routing, Xiangtai Li, Hao He, Yibo Yang, Henghui Ding, Kuiyuan Yang, Guangliang Cheng, Yunhai Tong, Dacheng Tao T-PAMI-2022 The first dynamic network for video scene understanding | Code
Selected Arxiv
- Towards Robust Referring Image Segmentation, Jianzong Wu, Xiangtai Li, Xia Li, Henghui Ding, Yunhai Tong, Dacheng Tao, arxiv The first benchmark for Robust Referring Image Segmentation | Project
- SFNet: Faster, Accurate, and Domain Agnostic Semantic Segmentation via Semantic Flow , Xiangtai Li, Jiangning Zhang, Yibo Yang, Guangliang Cheng, Yunhai Tong, Kuiyuan Yang, Dacheng Tao, arxiv | Code
π Honors and Awards
- National Scholarship, Ministry of Education of China in PKU (year 2020-2021) (year 2019-2020)
- President Scholarship of PKU (year 2020-2021)
- 2017, 2022 Beijing Excellent Graduates
- 2017, 2022 BUPT/PKU Excellent Graduates
- 2021.11 Winner of Segmenting and Tracking Every Point and Pixel: 6th Workshop on ICCV-2021 Track2 (Project Leader and First Author)
π Educations
- 2017.09 - 2022.06, PhD in Peking University (PKU)
- 2013.09 - 2017.06, Bachelor in Beijing University of Posts and Telecommunications (BUPT)
π¬ Invited Talks
- 2022.05 Invited talk on Panoptic Segmentation and Beyond in Baidu PaddleSeg Group
- 2021.12 Invited talk on Video Segmentation in DiDi Auto-Driving Group
- 2021.10 Invited talk on Aligned Segmentation HuaWei Noah Auto-Driving Group
π» Internships and Professional activities
- SenseTime, mentored by Dr.Guangliang Cheng and Dr.Jianping Shi.
- JD AI (remote cooperation), mentored by Dr.Yibo Yang and Prof.Dacheng Tao.
- DeepMotion (Now Xiaomi Car), mentored by Dr.Kuiyuan Yang.
- Regular Conference Reviewer for CVPR, ICCV, ECCV, ICLR, AAAI, NeurIPS, ICML, IJCAI and Journal Reviewer For IEEE-TIP, IEEE-TPAMI, IJCV.
- I am lucky mentored and also collaborate by Dr.Kuiyuan Yang, Prof.Li Zhang, Dr.Guangliang Cheng, Dr.Yibo Yang, Prof.Dacheng Tao, Prof.Zhouchen Lin, Mr.Xia li, Dr.Jiangmiao Pang.