About Me
I am actively seeking research and algorithm roles in 3D perception and embodied intelligence, especially work that brings these capabilities into real-world physical systems. I am Zi Fang, a Ph.D. candidate in Mechanical Engineering at the Robotics Institute, Shanghai Jiao Tong University, expecting to graduate in December 2026.
Download public CV (PDF)EmailGoogle Scholar
Research Interests
My doctoral research focuses on ultrasound-guided thyroid puncture. It connects freehand-ultrasound observation, continuous 3D reconstruction, multi-sweep spatial alignment, tissue–instrument semantic modeling, and a navigation coordinate chain. I care not only about benchmark results, but also about whether these methods form a verifiable system through calibration, phantom experiments, and robotic prototypes.
3D Perception and Continuous Representation
Multi-sensor pose estimation, 2D/3D registration, NeRF/3DGS, continuous neural fields, and 3D semantic reconstruction.
Embodied Spatial Intelligence
Unifying images, poses, continuous scene representations, and tissue–instrument geometry into spatial coordinates that robots can interpret, plan with, and act upon.
Robotic Systems and Autonomous Planning
Learning-guided path planning, robot mechanisms, and state perception, with closed-loop perception, planning, and execution validated in physical tasks.
- 3D Perception
- Embodied Intelligence
- Multi-sensor Fusion
- Neural Fields
- Robot Planning
- Robot Learning
Research Assistant
This chatbot connects local-material static retrieval with RAG. You can ask about my research interests, project details, and related work.
Project Experience
My projects follow a technical chain from multi-sensor state perception and pose/deformation optimization to continuous 3D representation, planning, and physical execution. The four directions below mirror the project structure in my CV, with freehand ultrasound and compact robotic systems serving as concrete experimental testbeds.
Instrument and image pose estimation with joint pose, deformation, and representation optimization for 2D/3D deformable registration
Stereo near-infrared cameras and an IMU estimate probe pose, while an N-wire model provides spatiotemporal calibration between the probe and ultrasound image. Multi-sweep discrepancy is separated into sweep-level global rigid bias, frame-level high-frequency jitter, and force-conditioned diffeomorphic deformation.
Hierarchical feature matching, Lie-group B-spline trajectory constraints, and a force-conditioned Morph field address these errors and are jointly optimized with NeRF/3DGS representations to map images, instruments, and tissue into a shared canonical space.
Ultrasound canonicalization and enhancement with multi-task 2D semantic segmentation
Manifold analysis of foundation-model features filters low-quality observations caused by unstable probe contact. Image-domain response modeling and acquisition-condition simulation then convert B-mode images from different devices and acquisition settings into a more stable canonical echo-intensity proxy.
A shared encoder and task branches jointly segment the thyroid, nodules, vessels, and needle. Cross-task semantic guidance, anatomical constraints, class-imbalance handling, and tubular topology supervision provide reliable 2D semantic inputs for subsequent 3D reconstruction.
3D inverse rendering and semantic fields with NeRF/3DGS and acoustic priors
Image-pose pairs define a continuous 3D field, while ultrasound propagation priors such as attenuation, backscatter, and direction-dependent reflection enter a differentiable renderer so that the implicit representation captures both image intensity and medium properties.
A continuous semantic field replaces discrete voxel writing and fuses multi-view 2D probability observations into anatomy that can be queried at arbitrary coordinates, providing a dense, continuous, and multi-view-consistent spatial representation for path planning.
Origami puncture robot and puncture-path planning
I am developing a compact five-DoF robot for head-and-neck puncture. An origami-chain three-DoF parallel stage is combined with a lower five-bar two-DoF mechanism to adjust needle entry pose within a small form factor.
Planning operates in multi-tissue 3D anatomy: BiT* searches for asymptotically optimal paths that respect flexible-needle motion and avoid risk structures, while learned models predict candidate entry points and non-uniform sampling regions.
Publications
GLA-NeRF: global-local aligned neural radiance fields for multi-sweep freehand 3D ultrasound
GLA-NeRF models sweep-level global rigid bias and frame-level local jitter separately, combining hierarchical localization, continuous-trajectory regularization, and neural-rendering error to refine multi-sweep poses and learn a canonical representation.
UPI-NeRF: ultrasonic-physics-informed neural radiance fields for freehand 3D ultrasound
UPI-NeRF strengthens the implicit representation and builds an explicit ultrasound renderer from Rayleigh backscatter, microfacet reflection, and direction parameterization, improving texture, interfaces, and view-dependent appearance in freehand 3D ultrasound.
From B-Mode Images to Canonical Echo-Intensity Representations for Robust Thyroid Ultrasound Segmentation
EIDC estimates an image-domain imaging response through phase-only initialization, domain-level prototypes, and image-level residual adaptation, then combines acquisition-condition augmentation to produce a more stable canonical echo-intensity proxy from public B-mode images.
Prediction for Loosening Life of Bolted Joints Using IMUs With Dimensionality Reduction
Multi-domain features from bonded-IMU vibration signals pass through outlier handling, denoising, imputation, standardization, and dimensionality reduction before remaining bolt-loosening life is predicted for proactive maintenance.
Neural-Guided RRT*: Learning-Based Planning of Entry Point and Puncture Path for Steerable Bevel-Tip Needle Insertion
Two 3D U-Nets predict candidate entry points and path-probability distributions to guide non-uniform RRT* sampling, while kinematically feasible spatial arcs extend the tree for steerable bevel-tip needles.
- Design, Modeling, and Validation of a 6-DoF Wearable Puncture Robot
· IEEE Robotics and Automation Letters, 2026. - Coarse-to-VoI: A Two-Stage Framework for Coronary Artery Segmentation in 3D Computed Tomography Angiography
· International Conference on Computer Vision, Image Processing and Applications, 2025. - Reinforcement Learning-Based Cooperative Fault-Tolerant Control for Multi-Actuator System With Uncertain Parameters and State Constraints
· IEEE Transactions on Aerospace and Electronic Systems, 2025. - Design and Control of a Robotic System for Coronary Interventions
· IEEE International Conference on Robotics and Biomimetics (ROBIO), 2025. - Design and Implementation of a 4-DOF Wearable Assisted Puncture Robot
· International Conference on Intelligent Robotics and Applications, 184–196, 2025. - A Novel Deep Learning Enhanced Particle Swarm Optimization for Puncture Path Planning
· International Conference on Intelligent Robotics and Applications, 166–174, 2025. - Prior skeleton based online deep reinforcement learning for coronary artery centerline extraction
· Proceedings of the Institution of Mechanical Engineers, Part H 237(5), 557–570, 2023. - Fault Diagnosis Method for Industrial Robots based on Dimension Reduction and Random Forest
· 27th International Conference on Mechatronics and Machine Vision in Practice, 2021.
Education
- 2021 — 2026Ph.D. Candidate · Mechanical Engineering
School of Mechanical Engineering, Shanghai Jiao Tong University · Robotics Institute
Dissertation: implicit 3D thyroid-ultrasound reconstruction and puncture-navigation systems - 2017 — 2021B.Eng. · Mechanical Engineering
School of Mechanical Engineering, Shanghai Jiao Tong University
Pilot honors program · Outstanding graduate of the school - 2014 — 2017High School
Zhenhai High School of Ningbo
Skills
- LLM applications
- Vibe coding, OpenClaw, RAG, and Harness
- Artificial intelligence
- PyTorch and Lightning with dual-GPU deployment, TensorFlow, JAX, MATLAB, and Simulink
- Robotic 3D perception
- NeRF, 3DGS, Diffusion, VLA, PyBullet, and Isaac robot simulation
- Embedded and mechanical design
- PyQt and C# interface development, STM32, Altium, and CATIA electrical modeling; CATIA and SolidWorks mechanical modeling, Ansys static simulation, and Adams dynamic simulation
- Languages
- English-taught undergraduate curriculum; CET-4: 595, CET-6: 547