Idea Highlight A: Stereo-based Tracking of Deformable Biological Surfaces

Researchers at the Engineering Research Center for Computer-Integrated Surgical Systems and Technology are developing dynamically updated geometric models of internal organs. These models will help solve one of the major complications in current surgical procedures, namely the fact that the organ is changing shape during the procedure, either as a result of breathing or the beat of the heart or because the organ deforms when affected by surgical instruments. For example, with the advent of coronary stabilization techniques, off-pump coronary artery bypass grafting (CABG) has become more popular in recent years. However, the stabilizer does not immobilize the heart completely, which leads to prolonged operating time. A stabilization system that tracks and compensates for the residual motion of the heart during robot-assisted CABG would have significant impact.

These researchers are developing a multi-camera image-processing algorithm that directly estimates and tracks deformable biological surfaces. The algorithm makes use of the same data the surgeon currently sees: a stream of stereo image pairs acquired from an operating microscope or endoscope. The algorithm computes surface geometry by optimizing the parameters of a function describing the relationship between the images of the left and right “eyes.” From this relationship, it is possible to compute the geometry of the observed surface.

By performing a surface-based optimization, the algorithm is able to operate extremely efficiently and can, therefore, process images at the rate they are acquired. As a result, the algorithm can be used in a number of surgical applications requiring dynamic models the surgical field, including establishing safety regions, developing virtual fixtures for guidance, and measuring mechanical properties of various tissues and organs.

To test the algorithm, the researchers processed image sequences of beating pig hearts obtained by the stereo endoscope used in the da Vinci robotic surgery system. This procedure is illustrated in the figure below. The figure also shows that the algorithm was able to track the deformation of the heart, as well as the respiration of the lungs, with sub-pixel accuracy. Future directions for this research include investigating the utility of this technique for the assessment of regional myocardial functions.



The endoscope in the da Vinci system is used to acquire the heart motion sequence (L). Tracking of the heart surface shows the heart beatings on top of the lung respirations (R).


The figure above shows the reconstructed geometry of a patch of the heart during one frame of the sequence. The image has been mapped onto the disparity surface to show the relationship between the geometry and the area of the tracked surface.


<< back to Highlights main menu