Intraoperative Patient Specific Volumetric Reconstruction and 3D
Visualization for Laparoscopic Liver Surgery
Abstract
Despite the benefits of minimally invasive surgery, interventions such
as laparoscopic liver surgery present unique challenges, like the
significant anatomical differences between preoperative images and
intraoperative scenes due to pneumoperitoneum, patient pose, and organ
manipulation by surgical instruments. To address these challenges, we
propose a method for intraoperative 3D reconstruction of the surgical
scene, including vessels and tumors, without altering the surgical
workflow. The technique combines Neural Radiance Field (NeRF)
reconstructions from tracked laparoscopic videos with ultrasound 3D
compounding. We evaluate the accuracy of our reconstructions on a
clinical laparoscopic liver ablation dataset, consisting of laparoscope
and patient reference poses from optical tracking, laparoscopic and
ultrasound videos, as well as preoperative and intraoperative CTs. We
propose a solution to compensate for liver deformations due to pressure
applied during ultrasound acquisitions, improving the overall accuracy
of the 3D reconstructions compared to the ground truth intraoperative CT
with pneumoperitoneum. We train a unified NeRF from the ultrasound and
laparoscope data, which allows real-time view synthesis providing
surgeons with comprehensive intraoperative visual information for
laparoscopic liver surgery.