蔡志鹏（Zhipeng Cai）
Senior AI Research scientist at Meta, Menlo Park, California, USA.
Education: PhD of Computer Science at The University of Adelaide, Australia. Supervisor: Prof. Tat-Jun Chin & Prof. David Suter.
Email: czptc2h@gmail.com
Google Scholar
I have supervised Ph.D students/interns from various countries. Send me an email if you are interested in internships/collaborations.

About me

    I am interested in general machine learning and computer vision problems. During PhD, I was interested in robust geometric perception, which estimates computer vision models (correspondences between images, poses, 3D reconstructions) given outlier contaminated data. I was specifically interested in designing efficient algorithms that have optimality guarantees, i.e., guarantee to return the best solution. After graduation, my interests shift towards a mixture of learning and vision, where I study various problems such as 1) learning-based perception (feature matching, finding correspondences, pose estimation, depth estimation etc) 2) Continual Learning 3) Generative models (e.g., novel view synthesis, multi-modal generation). My work has been selected as one of the 12 best papers at ECCV'18.

News

Award

Publication (Check the ArXiv version for modifications after publication)

2024

MIDGArD: Modular Interpretable Diffusion over Graphs for Articulated Designs
Quentin Leboutet, Nina Wiedemann, Zhipeng Cai , Michael Paulitsch, Kai Yuan
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2024
[Paper]
Slack-Free Spiking Neural Network Formulation for Hypergraph Minimum Vertex Cover
Tam Ngoc-Bang Nguyen, Anh-Dzung Doan, Zhipeng Cai , Tat-Jun Chin
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2024
[Paper]
Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai , Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, Shaojie Shen
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
[ArXiv preprint] [Project page] [Code] [Huggingface Demo]
Revisiting Test Time Adaptation under Online Evaluation
Motasem Alfarra, Hani Itani, Alejandro Pardo, Shyma Alhuwaider, Merey Ramazanova, Juan C. Pérez, Zhipeng Cai , Matthias Müller, Bernard Ghanem
Forty-first International Conference on Machine Learning (ICML) 2024
[ArXiv preprint] [Code]
L-MAGIC: Language Model Assisted Generation of Images with Coherence
Zhipeng Cai , Matthias Müller, Reiner Birkl, Diana Wofk, Shao-Yen Tseng, JunDa Cheng, Gabriela Ben-Melech Stan, Vasudev Lal, Michael Paulitsch
The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
[Paper] [Project page] [Code] [Huggingface Demo] [Intel Featured Blog] [Intel Labs Linedin]
LiSA: LiDAR Localization with Semantic Awareness
Bochun Yang, Zijun Li, Wen Li, Zhipeng Cai * , Chenglu Wen, Yu Zang, Matthias Müller, Cheng Wang *
*: Equal corresponding author
The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024, selected as highlight (3.6% acceptance rate)
[Paper] [Code] [Intel Featured Blog] [Intel Labs Linedin]
GIM: Learning Generalizable Image Matcher From Internet Videos
Xuelun Shen+, Zhipeng Cai+, *, Wei Yin+, Matthias Müller, Zijun Li, Kaixuan Wang, Xiaozhi Chen, Cheng Wang*
+: Equal contribution, *: Equal corresponding author.
Twelfth International Conference on Learning Representations (ICLR 2024), spotlight (5% acceptance rate) presentation.
[ArXiv preprint] [Project page] [Code] [Huggingface Demo] [Intel Blog Post]

2023

SimCS: Simulation for Online Domain-Incremental Continual Segmentation
Motasem Alfarra, Zhipeng Cai , Adel Bibi, Bernard Ghanem, Matthias Muller
Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI 2024)
[ArXiv preprint]
LDM3D-VR: Latent Diffusion Model for 3D VR
Gabriela Ben-Melech Stan, Diana Wofk, Estelle Aflalo, Shao-Yen Tseng, Zhipeng Cai , Michael Paulitsch, Vasudev Lal
NeurIPS 2023 Workshop on Diffusion Models
[Paper]
CorresNeRF: Image Correspondence Priors for Neural Radiance Fields.
Yixing Lao, Xiaogang Xu, Zhipeng Cai, Xihui Liu, Hengshuang Zhao.
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2023
[Paper]
E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning
Xiuhong Lin, Changjie Qiu, Zhipeng Cai, Siqi Shen, Yu Zang, Weiquan Liu, Xuesheng Bian, Matthias Müller, Cheng Wang
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2023
[Paper] [Code]
GSDDet: Ground Sample Distance Guided Object Detection for Remote Sensing Images
Yunuo Yang, Zhipeng Cai , Pinqing Song, Yu Zang, Guanjie Huang, Ming Cheng, Cheng Wang
IEEE Transactions on Geoscience and Remote Sensing (TGRS)
[Paper]
CLNeRF: Continual Learning Meets NeRF
Zhipeng Cai , Matthias Müller
International Conference on Computer Vision (ICCV) 2023
[Paper] [Code] [dataset]
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai , Xiaozhi Chen, Kaixuan Wang, Gang Yu, Chunhua Shen
International Conference on Computer Vision (ICCV) 2023
[ArXiv preprint] [Code]
Online Continual Learning Without the Storage Constraint
Ameya Prabhu, Zhipeng Cai , Puneet Dokania, Philip Torr, Vladlen Koltun, Ozan Sener
[ArXiv preprint] [Code]

2022

SimCS: Simulation for Online Domain-Incremental Continual Segmentation
Motasem Alfarra, Zhipeng Cai , Adel Bibi, Bernard Ghanem, Matthias Muller
CVPR Workshop on Continual Learning (CLVision) 2023
[ArXiv preprint]
Improving Information Retention in Large Scale Online Continual Learning
Zhipeng Cai, Vladlen Koltun, Ozan Sener
[ArXiv preprint]

2021

Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data
Zhipeng Cai, Ozan Sener, Vladlen Koltun
International Conference on Computer Vision (ICCV) 2021
[ArXiv preprint] [Code]

2020

Consensus Maximization: Theoretical Analysis and New Algorithms
Zhipeng Cai
Ph.D thesis (Supervised by: Prof. Tat-Jun Chin and Prof. David Suter)
[PDF]
Globally Optimal and Efficient Vanishing Point Estimation in Atlanta World
Haoang Li, Pyojin Kim, Ji Zhao, Kyungdon Joo, Zhipeng Cai, Zhe Liu, Yun-Hui Liu
European Conference on Computer Vision (ECCV) 2020
[Paper]

2019

Robust fitting in computer vision: easy or hard?
Tat-Jun Chin, Zhipeng Cai, Frank Neumann
International Journal on Computer Vision (IJCV), Special Issue on Best of ECCV 2018.
[Paper]
Consensus Maximization Tree Search Revisited
Zhipeng Cai, Tat-Jun Chin, Vladlen Koltun
International Conference on Computer Vision (ICCV) 2019, oral (4.6% acceptance rate)
[Arxiv preprint] [Code]
Practical Optimal Registration of Terrestrial LiDAR Scan Pairs
Zhipeng Cai, Tat-Jun Chin, Alvaro Parra Bustos, Konrad Schindler
ISPRS Journal of Photogrammetry and Remote Sensing, 2019.
[Arxiv preprint] [PDF] [Code]

2018

Deterministic consensus maximization with biconvex programming
Zhipeng Cai, Tat-Jun Chin, Huu Le, David Suter
European Conference on Computer Vision (ECCV) 2018, oral (2.4% acceptance rate) presentation.
[Arxiv preprint] [Code] [Slides]
Robust fitting in computer vision: easy or hard?
Tat-Jun Chin, Zhipeng Cai, Frank Neumann
European Conference on Computer Vision (ECCV) 2018, oral presentation.
Selected as one of the 12 best papers from the conference (0.4% acceptance rate)
[Arxiv preprint] [Slides]

2016

Spatial-Related Traffic Sign Inspection for Inventory Purposes Using Mobile Laser Scanning Data
Chenglu Wen, Jonathan Li, Huan Luo, Yongtao Yu, Zhipeng Cai, Hanyun Wang, Cheng Wang
IEEE Transactions on Intelligent Transportation Systems, 2016
[PDF]
Patch-based semantic labeling of road scene using colorized mobile LiDAR point clouds
Huan Luo, Cheng Wang, Chenglu Wen, Zhipeng Cai, Ziyi Chen, Hanyun Wang, Yongtao Yu, Jonathan Li
IEEE Transactions on Intelligent Transportation Systems, 2016
[PDF]

2015

Occluded Boundary Detection for Small-footprint Ground-borne LIDAR Point Cloud Guided by Last-echo
Zhipeng Cai, Cheng Wang, Chenglu Wen, Jonathan Li
IEEE Geoscience and Remote Sensing Letters, 2015
[PDF]
3D-PatchMach: an Optimization Algorithm for Point Cloud Completion
Zhipeng Cai, Cheng Wang, Chenglu Wen, Jonathan Li
Second IEEE International Conference on Spatial Data Mining and Geographical Knowledge Services(ICSDM2015)
[PDF]

2012

Automatic road extraction from mobile laser scanning data
Hanyun Wang, Zhipeng Cai, Huan Luo, Cheng Wang, Peng Li, Wentao Yang, Suoping Ren, Jonathan Li
International Conference on Computer Vision in Remote Sensing (CVRS), 2012
[PDF]
Scale invariant kernel-based object tracking
Peng Li, Zhipeng Cai, Hanyun Wang, Zhuo Sun, Yunhui Yi, Cheng Wang, Jonathan Li
International Conference on Computer Vision in Remote Sensing (CVRS), 2012
[PDF]
Cascade framework for object extraction in image sequences
Peng Li, Zhipeng Cai, Cheng Wang, Zhuo Sun, Hanyun Wang, Jonathan Li
International Conference on Computer Vision in Remote Sensing (CVRS), 2012
[PDF]

Patent:

Working experience

Academic Service

Last Updated on 22th Nov, 2016

Published with GitHub Pages

Contact GitHub API Training Shop Blog About © 2016 GitHub, Inc. Terms Privacy Security Status Help