Instrument Contact Force Estimation Using Endoscopic Image Sequence and 3D Reconstruction Model

Abstract

This paper presents a method to estimate contact force for minimally invasive surgery (MIS) using endoscopic imagery. In order to provide surgeons the information of contact force as surgical instruments get contact to human tissue, we developed a method to calculate tissue deformation by reconstructing 3D model using real-time image processing. The force information will be fed back to a haptic device to allow the surgeon to feel contact force as well as to show the force value on monitor for the surgeon. We propose a 3D reconstruction method to model the tissue surface under deformation, and use the deformation data to obtain contact force near the instrument tip. Using image tracking, we locate a surgical instrument in the endoscopy scene, then determine the deformation area in the image scene. Several experiments validate the effectiveness of method of deformation calculation.