SurgIntent
A benchmark and knowledge-guided multi-agent framework for understanding the clinical intent behind surgical actions and the objectives they prepare for.
Third-year Ph.D. student · HKUST(GZ)
Pengfei Hao郝鹏飞
Medical AI / Surgical Multimodal Large Language Models
I am a third-year Ph.D. student in Robotics and Autonomous Systems at HKUST(GZ), advised by Prof. Lei Zhu.
My research focuses on medical AI and surgical multimodal large language models. I am particularly interested in surgical scene understanding from images and videos, reasoning in medical multimodal large language models, and multi-agent systems.
Previously, I received my M.Phil. from HKUST(GZ) and my B.Eng. from Xi’an Jiaotong University.
Surgical AI · A conceptual illustration
RESEARCH INTERESTS
Research Areas.
01
Surgical Scene Understanding
Understanding surgical images and videos, including anatomy, instruments, actions, and surgical intent.
02
Medical Multimodal Large Language Models
Integrating medical images and clinical information in multimodal large language models for medical understanding and reasoning.
01 / RESEARCH
Recent Work.
Surgery-R1
Reasoning multimodal models for surgical question answering and localization, trained with supervised fine-tuning and reinforcement learning.
Surgical-MambaLLM
Mamba2-enhanced cross-modal integration and instrument-aware perception for visual question localized-answering in robotic surgery.
02 / UPDATES
Recently.
Submitted , our work on surgical intent understanding, to ICLR 2027.
RESEARCHBRAF-MLLM, our work on thyroid cancer BRAF V600E mutation prediction, has been accepted to MICCAI 2026.
CONFERENCEOur work on knowledge-guided surgical visual reasoning appears in IEEE JBHI, Vol. 29, Issue 12.
PUBLICATIONSurgical-MambaLLM appears at MICCAI 2025.
CONFERENCE03 / RESEARCH RECORD
Publications & manuscripts.
8 works
04 / BACKGROUND
Education &
background.
A background in mechanical engineering and robotics, with research interests in surgical scene understanding and medical multimodal large language models.
2024 — Present
Ph.D. in Robotics and Autonomous Systems
The Hong Kong University of Science and Technology (Guangzhou)
Advised by Prof. Lei Zhu2022 — 2024
M.Phil. in Robotics and Autonomous Systems
The Hong Kong University of Science and Technology (Guangzhou)
2018 — 2022
B.Eng. in Mechanical Engineering
Xi’an Jiaotong University
Outstanding Undergraduate ThesisBeyond research, I enjoy singing, football, basketball, and badminton. I previously served as president of the Feeling Music Club at Xi’an Jiaotong University.
OUTSIDE THE LABLET’S CONNECT
Always happy to talk research.
Surgical scene understanding and medical multimodal large language models.