loading page

Augmented reality-based interactive scheme for robot-assisted percutaneous renal puncture navigation
  • +4
  • Yiwei Zhuang,
  • Shuyi Wang,
  • Hua Xie,
  • Wei Qing,
  • Haoliang Li,
  • Yuhan Shen,
  • Yichun Shen
Yiwei Zhuang
University of Shanghai for Science and Technology
Author Profile
Shuyi Wang
University of Shanghai for Science and Technology

Corresponding Author:[email protected]

Author Profile
Hua Xie
Children's Hospital of Shanghai
Author Profile
Wei Qing
University of Shanghai for Science and Technology
Author Profile
Haoliang Li
University of Shanghai for Science and Technology
Author Profile
Yuhan Shen
University of Shanghai for Science and Technology
Author Profile
Yichun Shen
University of Shanghai for Science and Technology
Author Profile

Abstract

In this paper, we present an Augmented Reality (AR)-based application combined with a robotic system for percutaneous renal puncture navigation interaction and demonstrate its technical feasibility. Our system provides an intuitive interaction scheme between the surgeon and the robot without the need for traditional external input devices, and applies an image-target-based 3D registration scheme to transform the coordinate system between Hololens2 and the robot without using additional tracking devices. Users can visualise the abdominal puncture phantom and obtain 3D depth information of the lesion site by wearing Hololens2 and control the robot directly using buttons or gestures. To investigate the accuracy and feasibility of the proposed interaction scheme, six subjects were recruited to complete 3D registration alignment accuracy experiments, and puncture positioning accuracy experiments using ultrasound unaided navigation, AR unaided navigation and AR robotic navigation. The results showed that the average alignment error of 3D registration was 3.61±1.05 mm. The average positioning errors of ultrasound freehand navigation, AR freehand navigation and AR robotic navigation were 7.67±2.00 mm, 6.13±1.07 mm and 5.52±0.37 mm, respectively; the average puncture times were 34.86±1.67 s, 22.40±2.07 s, and 29.41±1.37 s.
Submitted to Computer Animation and Virtual Worlds
16 Apr 2024Assigned to Editor
16 Apr 2024Submission Checks Completed
15 Jul 2024Review(s) Completed, Editorial Evaluation Pending
26 Aug 2024Editorial Decision: Revise Minor
11 Sep 20241st Revision Received
12 Sep 2024Assigned to Editor
12 Sep 2024Submission Checks Completed
27 Sep 2024Reviewer(s) Assigned
21 Oct 2024Review(s) Completed, Editorial Evaluation Pending
23 Oct 2024Editorial Decision: Revise Minor
20 Nov 20242nd Revision Received
22 Nov 2024Submission Checks Completed
22 Nov 2024Assigned to Editor
24 Nov 2024Reviewer(s) Assigned