Augmented reality-based interactive scheme for robot-assisted
percutaneous renal puncture navigation
Abstract
In this paper, we present an Augmented Reality (AR)-based application
combined with a robotic system for percutaneous renal puncture
navigation interaction and demonstrate its technical feasibility. Our
system provides an intuitive interaction scheme between the surgeon and
the robot without the need for traditional external input devices, and
applies an image-target-based 3D registration scheme to transform the
coordinate system between Hololens2 and the robot without using
additional tracking devices. Users can visualise the abdominal puncture
phantom and obtain 3D depth information of the lesion site by wearing
Hololens2 and control the robot directly using buttons or gestures. To
investigate the accuracy and feasibility of the proposed interaction
scheme, six subjects were recruited to complete 3D registration
alignment accuracy experiments, and puncture positioning accuracy
experiments using ultrasound unaided navigation, AR unaided navigation
and AR robotic navigation. The results showed that the average alignment
error of 3D registration was 3.61±1.05 mm. The average positioning
errors of ultrasound freehand navigation, AR freehand navigation and AR
robotic navigation were 7.67±2.00 mm, 6.13±1.07 mm and 5.52±0.37 mm,
respectively; the average puncture times were 34.86±1.67 s, 22.40±2.07
s, and 29.41±1.37 s.