About Me
I am actively seeking research and algorithm roles in 3D perception and embodied intelligence, especially work that brings these capabilities into real-world physical systems. I am Zi Fang, a Ph.D. candidate in Mechanical Engineering at the Robotics Institute, Shanghai Jiao Tong University, expecting to graduate in December 2026.
Download public CV (PDF)EmailGoogle Scholar
Research Interests
My dissertation, Research on Neural-Field-Based Three-Dimensional Reconstruction of Thyroid Ultrasound and a Puncture Planning System, uses freehand ultrasound to study perception from irregular 2D observations to continuous 3D space. Acoustic scales and imaging conditions constrain neural fields; joint pose–representation optimization aligns multiple sweeps; multi-view semantics recover tissue structure and finite needle segments. I connect these methods with tracking, calibration, and interactive software to build a verifiable spatial perception system.
3D Perception and Continuous Representation
Multi-sensor pose estimation, 2D/3D registration, NeRF/3DGS, continuous neural fields, and 3D semantic reconstruction.
Embodied Spatial Intelligence
Unifying images, poses, continuous scene representations, and tissue–instrument geometry into spatial coordinates that robots can interpret, plan with, and act upon.
Robotic Systems and Autonomous Planning
Learning-guided path planning, robot mechanisms, and state perception, with closed-loop perception, planning, and execution validated in physical tasks.
- 3D Perception
- Embodied Intelligence
- Multi-sensor Fusion
- Neural Fields
- Robot Planning
- Robot Learning
Research Assistant
This chatbot connects local-material static retrieval with RAG. You can ask about my research interests, project details, and related work.
Project Experience
My projects follow a technical chain from multi-sensor state perception and pose/deformation optimization to continuous 3D representation, planning, and physical execution. The four directions below mirror the project structure in my CV, with freehand ultrasound and compact robotic systems serving as concrete experimental testbeds.
Instrument and image pose estimation with joint pose, deformation, and representation optimization for 2D/3D deformable registration
Stereo near-infrared tracking, N-wire probe calibration, and needle-tip pivot calibration establish separate probe–image and instrument–tip references. The dissertation separates sweep-level rigid offsets from frame-level perturbations, using hierarchical matching and uncertainty weighting to constrain global alignment.
Lie-group B-splines, statistical–kinematic regularization, and continuous-trajectory self-distillation constrain local updates during joint optimization with the neural field. Related projects also explore IMU fusion, force-conditioned Morph fields, and NeRF/3DGS deformable registration.
Ultrasound canonicalization and enhancement with multi-task 2D semantic segmentation
USF-MAE features, UMAP, and a random forest identify valid image columns under weak probe contact. EIDC hierarchical effective-response compensation and acquisition-condition simulation then produce more consistent canonical image-domain intensity proxies across B-mode sources.
A shared encoder and task branches support partial labels. Source-task transfer, CBAM semantic guidance, and anatomical constraints improve learning and structural consistency at different stages. Missing labels are excluded from supervision rather than treated as background, providing well-defined 2D observations for multi-view modeling.
3D inverse rendering and semantic fields with NeRF/3DGS and acoustic priors
Image-pose pairs constrain a continuous 3D representation. Finite acoustic footprints, scale-aware encoding, and imaging-plane conditioning account for irregular observations, while Rayleigh scattering and direction-parameterized reflection support differentiable rendering. The dissertation evaluates continuous neural fields; related projects also explore 3DGS.
The semantic field represents position, scale, direction, and acquisition state, with separate anatomy and needle outputs. Geometric soft targets, topology constraints, and robust fitting recover finite needle segments. Four workbenches integrate acquisition/calibration, 2D semantics, sequence registration, and 3D reconstruction/needle analysis for interactive validation.
Origami puncture robot and puncture-path planning
I am developing a compact five-DoF robot for head-and-neck puncture. An origami-chain three-DoF parallel stage is combined with a lower five-bar two-DoF mechanism to adjust needle entry pose within a small form factor.
Planning operates in multi-tissue 3D anatomy: BiT* searches for asymptotically optimal paths that respect flexible-needle motion and avoid risk structures, while learned models predict candidate entry points and non-uniform sampling regions.
Publications
GLA-NeRF: global-local aligned neural radiance fields for multi-sweep freehand 3D ultrasound
GLA-NeRF models sweep-level global rigid bias and frame-level local jitter separately, combining hierarchical localization, continuous-trajectory regularization, and neural-rendering error to refine multi-sweep poses and learn a canonical representation.
UPI-NeRF: ultrasonic-physics-informed neural radiance fields for freehand 3D ultrasound
UPI-NeRF strengthens the implicit representation and builds an explicit ultrasound renderer from Rayleigh backscatter, microfacet reflection, and direction parameterization, improving texture, interfaces, and view-dependent appearance in freehand 3D ultrasound.
From B-Mode Images to Canonical Echo-Intensity Representations for Robust Thyroid Ultrasound Segmentation
EIDC estimates an image-domain imaging response through phase-only initialization, domain-level prototypes, and image-level residual adaptation, then combines acquisition-condition augmentation to produce a more stable canonical echo-intensity proxy from public B-mode images.
Prediction for Loosening Life of Bolted Joints Using IMUs With Dimensionality Reduction
Multi-domain features from bonded-IMU vibration signals pass through outlier handling, denoising, imputation, standardization, and dimensionality reduction before remaining bolt-loosening life is predicted for proactive maintenance.
Neural-Guided RRT*: Learning-Based Planning of Entry Point and Puncture Path for Steerable Bevel-Tip Needle Insertion
Two 3D U-Nets predict candidate entry points and path-probability distributions to guide non-uniform RRT* sampling, while kinematically feasible spatial arcs extend the tree for steerable bevel-tip needles.
- Design, Modeling, and Validation of a 6-DoF Wearable Puncture Robot
· IEEE Robotics and Automation Letters, 2026. - Coarse-to-VoI: A Two-Stage Framework for Coronary Artery Segmentation in 3D Computed Tomography Angiography
· International Conference on Computer Vision, Image Processing and Applications, 2025. - Reinforcement Learning-Based Cooperative Fault-Tolerant Control for Multi-Actuator System With Uncertain Parameters and State Constraints
· IEEE Transactions on Aerospace and Electronic Systems, 2025. - Design and Control of a Robotic System for Coronary Interventions
· IEEE International Conference on Robotics and Biomimetics (ROBIO), 2025. - Design and Implementation of a 4-DOF Wearable Assisted Puncture Robot
· International Conference on Intelligent Robotics and Applications, 184–196, 2025. - A Novel Deep Learning Enhanced Particle Swarm Optimization for Puncture Path Planning
· International Conference on Intelligent Robotics and Applications, 166–174, 2025. - Prior skeleton based online deep reinforcement learning for coronary artery centerline extraction
· Proceedings of the Institution of Mechanical Engineers, Part H 237(5), 557–570, 2023. - Fault Diagnosis Method for Industrial Robots based on Dimension Reduction and Random Forest
· 27th International Conference on Mechatronics and Machine Vision in Practice, 2021.
Education
- 2021 — 2026Ph.D. Candidate · Mechanical Engineering
School of Mechanical Engineering, Shanghai Jiao Tong University · Robotics Institute
Dissertation: Research on Neural-Field-Based Three-Dimensional Reconstruction of Thyroid Ultrasound and a Puncture Planning System - 2017 — 2021B.Eng. · Mechanical Engineering
School of Mechanical Engineering, Shanghai Jiao Tong University
Pilot honors program · Outstanding graduate of the school - 2014 — 2017High School
Zhenhai High School of Ningbo
Skills
- LLM applications
- Vibe coding, OpenClaw, RAG, and Harness
- Artificial intelligence
- PyTorch and Lightning with dual-GPU deployment, TensorFlow, JAX, MATLAB, and Simulink
- Robotic 3D perception
- NeRF, 3DGS, Diffusion, VLA, PyBullet, and Isaac robot simulation
- Embedded and mechanical design
- PyQt and C# interface development, STM32, Altium, and CATIA electrical modeling; CATIA and SolidWorks mechanical modeling, Ansys static simulation, and Adams dynamic simulation
- Languages
- English-taught undergraduate curriculum; CET-4: 595, CET-6: 547