PUBLICATION

Preprint

(* indicates corresponding author)

1. Jie Yang, Ailing Zeng*, Ruimao Zhang*, Lei Zhang "UniPose: Detecting Any Keypoints“, arXiv preprint arXiv:2310.08530 (2023). ( The first unified framework, named UniPose, to detect keypoints of any articulated (e.g., human and animal), rigid, and soft objects via visual or textual prompts for fine-grained vision understanding and manipulation. ) 【PDF】


2. Shunlin Lu, Ling-Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang*, Lei Zhang, Heung-Yeung Shum* "HumanTOMATO: Text-aligned Whole-body Motion Generation“, arXiv preprint arXiv:2310.12978 (2023). ( A novel text-aligned whole-body motion generation framework that can generate high-quality, diverse, and coherent facial expressions, hand gestures, and body motions simultaneously. ) 【PDF】


3. Jie Yang, Bingliang Li, Fengyu Yang, Ailing Zeng*, Lei Zhang, Ruimao Zhang*, "Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model“, arXiv preprint arXiv:2305.12252 (2023). ( We introduce a novel scheme, DiffHOI, which leverages both the generative and representation capacities of pre-trained text-to-image diffusion models to enhance the performance of HOI detection tasks. ) 【PDF】


4. Jiong Wang, Fengyu Yang, Wenbo Gou, Bingliang Li, Danqi Yan, Ailing Zeng, Yijun Gao, Junle Wang, Ruimao Zhang*, "FreeMan: Towards Benchmarking 3D Human Pose Estimation in the wild“, arXiv preprint arXiv:2309.05073 (2023). ( We present FreeMan, the first large-scale, real-world multi-view dataset. FreeMan was captured by synchronizing 8 smartphones across diverse scenarios. It comprises 11M frames from 8000 sequences, viewed from different perspectives. These sequences cover 40 subjects across 10 different scenarios, 27 locations, each with varying lighting conditions. ) 【PDF】【Project】

Newly Accepted Articles

(* indicates corresponding author)

1. Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, Lei Zhang, "Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset“, Proc. of Conference on Neural Information Processing Systems ( NeurIPS ), Track Datasets and Benchmarks, 2023 ( A large-scale 3D expressive whole-body human motion dataset (over 10M frames) with SMPL-X, text, audio, and RGB modalities.) 【Homepage】【Code】


2. Siyue Yao, Mingjie Sun, Bingliang Li, Fengyu Yang, Junle Wang, Ruimao Zhang*, "Dance with You: The Diversity Controllable Dancer Generation via Diffusion Models“, Proc. of ACM International Conference on Multimedia ( ACM MM ), 2023 ( We introduce a novel multi-dancer synthesis task called partner dancer generation, aiming to synthesize virtual human dancers capable of performing the dance with users. ) 【PDF】【Dataset】


3. Jie Yang, Ailing Zeng*, Feng Li, Shilong Liu, Ruimao Zhang*, Lei Zhang, "Neural Interactive Keypoint Detection“, Proc. of IEEE International Conference on Computer Vision ( ICCV ), 2023 ( The first end-to-end neural interactive keypoint detection framework significantly reduces 10+ times labeling costs. ) 【PDF】 【Code】 【Youtube】


4. Yiran Qin, Chaoqun Wang, Zijian Kang, Ningning Ma, Zhen Li, Ruimao Zhang*., "SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection“, Proc. of IEEE International Conference on Computer Vision ( ICCV ), 2023 ( The first work discusses how to perform multimodal fusion for 3D detection by constructing the supervisory signal. ) 【PDF】 【Code】


5. Jie Yang, Chaoqun Wang, Zhen Li, Junle Wang, Ruimao Zhang*, "Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains“, Proc. of IEEE International Conference on Computer Vision and Pattern Recognition ( CVPR ), 2023 ( We have answered the question: how could we leverage the mutual benefits of data from multiple labeling granularities to improve the performance of a given human parsing network with the specific architecture! ) 【PDF】 【Code】


6. Ye Zhu, Jie Yang, Siqi Liu, Ruimao Zhang*, "Inherent Consistent Learning for Accurate Semi-supervised Medical Image Segmentation", Proc. of Conference on Medical Imaging with Deep Learning( MIDL ), 2023 ( Oral ) ( A novel plug-and-play module for effectively improving the performance of semi-supervised segmentation, especially for small organs and lesions. ) 【PDF】 【Code】


7. Jie Yang, Ye Zhu, Chaoqun Wang, Zhen Li, Ruimao Zhang*, "Toward Unpaired Multi-modal Medical Image Segmentation via Learning Structured Semantic Consistency“, Proc. of Conference on Medical Imaging with Deep Learning( MIDL ), 2023 ( A novel scheme to learn the mutual benefits of different modalities to achieve better segmentation results for unpaired multi-modal medical images. ) 【PDF】


8. Jie Yang, Ailing Zeng*, Shilong Liu, Feng Li, Ruimao Zhang*, Lei Zhang, "Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation“, Proc. of International Conference on Learning Representations( ICLR ), 2023 、 ( We reformulate the task of multi-person pose estimation with two explicit box detection processes and achieve a new SOTA! ) 【PDF】【Code】【Youtube】

Recent Selected Publications ( See Full List )

(* indicates corresponding author)

1. Jie Yang, Ailing Zeng*, Feng Li, Shilong Liu, Ruimao Zhang*, Lei Zhang, "Neural Interactive Keypoint Detection“, Proc. of IEEE International Conference on Computer Vision ( ICCV ), 2023 【PDF】 【Code】


2. Yiran Qin, Chaoqun Wang, Zijian Kang, Ningning Ma, Zhen Li, Ruimao Zhang*, "SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection“, Proc. of IEEE International Conference on Computer Vision ( ICCV ), 2023 【PDF】 【Code】


3. Jie Yang, Chaoqun Wang, Zhen Li, Junle Wang, Ruimao Zhang*, "Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains“, Proc. of IEEE International Conference on Computer Vision and Pattern Recognition ( CVPR ), 2023 【PDF】【Code】


4. Ye Zhu, Jie Yang, Siqi Liu, Ruimao Zhang*, "Inherent Consistent Learning for Accurate Semi-supervised Medical Image Segmentation“, Proc. of Conference on Medical Imaging with Deep Learning( MIDL ), 2023 ( Oral ) 【PDF】【Code】


5. Jie Yang, Ailing Zeng*, Shilong Liu, Feng Li, Ruimao Zhang*, Lei Zhang, "Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation“, Proc. of International Conference on Learning Representations( ICLR ), 2023 【PDF】【Code】【Youtube】


6. Yuanfeng Ji, Haotian Bai, Jie Yang, Chongjian Ge, Ye Zhu, Ruimao Zhang*, Zhen Li*, Lingyan Zhang, Wanling Ma, Xiang Wan, Ping Luo*, "AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation“, Proc. of Conference on Neural Information Processing Systems ( NeurIPS ), 2022 ( Oral ) 【PDF】【AMOS Challenge】


7. Haotian Bai, Ruimao Zhang*, Jiong Wang, Xiang Wan, "Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration“, Proc. of Europe Conference on Computer Vision( ECCV ), 2022 【PDF】【Code】【Youtube】


8. Ping Luo, Ruimao Zhang*, Jiamin Ren, Zhanglin Peng, Jingyu Li, "Switchable Normalization for Learning-to-Normalize Deep Representation", IEEE Transactions on Pattern Analysis and Machine Intelligence ( T-PAMI ), 43(2):712-728, 2021 ( IF:17.86 ) 【PDF】【Code】


9. Ruimao Zhang, Zhanglin Peng, Lingyun Wu, Zhen Li, Ping Luo, " Exemplar Normalization for Learning Deep Representation ", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition ( CVPR ), 2020 【PDF】【Supp】


10. Ruimao Zhang, Liang Lin, Guangrun Wang, Meng Wang, Wangmeng Zuo, “Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions”, IEEE Transactions on Pattern Analysis and Machine Intelligence ( T-PAMI ), 41(3):596 - 610, 2019 ( IF:17.86 ) 【PDF】


11. Ruimao Zhang, Jingyu Li, Hongbin Sun, Yuying Ge, Ping Luo, Xiaogang Wang, Liang Lin, “SCAN: Self-and-Collaborative Attention Network for Video Person Re-identification”, IEEE Transactions on Image Processing ( T-IP ), 28(10):4870-4882, 2019 【PDF】【Code】

MEMBER

Ph.D. Students

Chaoqun Wang

Ph.D., since 2021, co-supervised with Prof. Tianwei Yu

Scene Understanding, Video Analysis, Multimodal Learning

M.S.: Nanjing Univ. of Sci. & Tech.

B.S.: Huazhong Univ. of Sci. & Tech.

Yiran Qin

Ph.D., since 2021, co-supervised with Prof. Zhen Li

Scene Understanding, Embodied AI, Large Visual Language Model

M.S.: not applicable

B.E.: Shandong University (Top 10%)

Jie Yang

Ph.D., since 2021, co-supervised with Prof. Zhen Li

Human Centric Visual Perception and Generation

M.S.: not applicable

B.E.: Harbin Engineering Univ. (Top 1%)

Ruiying Liu

Ph.D., since 2022, co-supervised with Prof. Tianshu Yu

AI+Science, Deep Graph Representation Learning

M.S.: Xiamen University

B.E.: Xiamen University

Shunlin Lu

Ph.D., since 2023, co-supervised with Prof. Benyou Wang

Multi-modal Learning, Human Centric Understanding

M.S.: University of Southern California

B.E.: Wuhan University of Technology

Bohao Li (Merit Scholarship)

Ph.D., since 2023, co-supervised with Prof. Shuang Li

Multi-modal Learning, Visual Understanding and Reasoning

M.S.: Univ. of Chinese Academy of Sci.

B.E.: Wuhan University (Top 5%)

MPhil Students

Bingliang Li

Master, since 2022, master student in School of Data Science

Scene Understanding, Referring Image Segmentation

Fengyu Yang

Master, since 2022, master student in School of Data Science

Human Centric 3D Perception, Synthesis and Animation

Research Assistants

Hanqi Jiang

B.E., Beijing Jiaotong Univeristy

Cross-modal Learning, Transformer, CLIP

Jiayu Chang

B.E., Central South Univeristy

Human Centric Visual Perception and Generation

Alumni

TEACHING