Endoscope Video Registration for Skull Base Surgery

The focus of this project is to study the problem of registering an endoscopic video sequence to a preoperative CT scan with applications to sinus surgery and transnasal skull base surgery. The main goal of this project is to be able to accurately track the location of the endoscope in real-time using vision techniques and thus be able to determine endoscope location within the sinus passage. The advantages to being able to do real-time visual tracking are quite substantial as current tool tracking systems for sinus surgery require additional equipment beyond the endoscope camera that are subject to line-of-sight issue or electro-magnetic distortion depending on the modality of tracking use.

Publications

  • D. J. Mirota, M. Ishii, and G. D. Hager, "Vision-Based Navigation in Image-Guided Interventions," Annual Review of Biomedical Engineering, vol. 13, iss. 1, pp. 297-319, 2011.
    bibtex Go to document
    @ARTICLE{Mirota2011a,
      author = {Mirota, Daniel J. and Ishii, Masaru and Hager, Gregory D.},
      title = {Vision-Based Navigation in Image-Guided Interventions},
      journal = {Annual Review of Biomedical Engineering},
      year = {2011},
      volume = {13},
      pages = {297-319},
      number = {1},
      abstract = {The trend toward minimally invasive surgical interventions has created new challenges for visualization during surgical procedures. However, at the same time, the introduction of high-definition digital endoscopy offers the opportunity to apply methods from computer vision to provide visualization enhancements such as anatomic reconstruction, surface registration, motion tracking, and augmented reality. This review provides a perspective on this rapidly evolving field. It first introduces the clinical and technical background necessary for developing vision-based algorithms for interventional applications. It then discusses several examples of clinical interventions where computer vision can be applied, including bronchoscopy, rhinoscopy, transnasal skull-base neurosurgery, upper airway interventions, laparoscopy, robotic-assisted surgery, and Natural Orifice Transluminal Endoscopic Surgery (NOTES). It concludes that the currently reported work is only the beginning. As the demand for minimally invasive procedures rises, computer vision in surgery will continue to advance through close interdisciplinary work between interventionists and engineers.},
      doi = {10.1146/annurev-bioeng-071910-124757},
      eprint = {http://www.annualreviews.org/doi/pdf/10.1146/annurev-bioeng-071910-124757},
      owner = {dmirota},
      timestamp = {2011.06.08},
      url = {http://arjournals.annualreviews.org/eprint/iQXnrba9rj8NAVAnC8uD/full/10.1146/annurev-bioeng-071910-124757}
    }
  • H. Wang, D. Mirota, and G. D. Hager, "A Generalized Kernel Consensus Based Robust Estimator," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, iss. 1, pp. 178-184, 2010.
    bibtex Go to document Go to document
    @ARTICLE{Wang2010,
      author = {Hanzi Wang and Daniel Mirota and Gregory D. Hager},
      title = {A Generalized Kernel Consensus Based Robust Estimator},
      journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year = {2010},
      volume = {32},
      pages = {178-184},
      number = {1},
      address = {Los Alamitos, CA, USA},
      doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.148},
      file = {Wang2010:Wang2010.pdf:PDF;tpami09.pdf:tpami09.pdf:PDF},
      issn = {0162-8828},
      owner = {dmirota},
      publisher = {IEEE Computer Society},
      timestamp = {2009.08.19},
      url = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.148}
    }
  • D. Mirota, H. Wang, R. H. Taylor, M. Ishii, and G. D. Hager, "Toward Video-Based Navigation for Endoscopic Endonasal Skull Base Surgery," in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2009, 2009, pp. 91-99.
    bibtex Go to document Go to document
    @INPROCEEDINGS{Mirota2009a,
      author = {Daniel Mirota and Hanzi Wang and Russell H. Taylor and Masaru Ishii and Gregory D. Hager},
      title = {Toward Video-Based Navigation for Endoscopic Endonasal Skull Base Surgery},
      booktitle = {Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2009},
      year = {2009},
      editor = {Guang-Zhong Yang and David Hawkes and Daniel Rueckert and Alison Noble and Chris Taylor},
      volume = {5761},
      series = {Lecture Notes in Computer Science},
      pages = {91--99},
      publisher = {Springer},
      file = {57610091.pdf:57610091.pdf:PDF},
      isbn = {978-3-642-04267-6},
      location = {Heidelberg},
      owner = {dmirota},
      timestamp = {2009.10.21},
      url = {http://www.springerlink.com/content/u267048356864772/}
    }
  • D. Mirota, R. H. Taylor, M. Ishii, and G. D. Hager, "Direct Endoscopic Video Registration for Sinus Surgery," in Medical Imaging 2009: Visualization, Image-guided Procedures and Modeling. Proceedings of the SPIE, 2009, pp. 72612k‑1-72612k‑8.
    bibtex Go to document Go to document
    @CONFERENCE{Mirota2009,
      author = {Daniel Mirota and Russell H. Taylor and Masaru Ishii and Gregory D. Hager},
      title = {Direct Endoscopic Video Registration for Sinus Surgery},
      booktitle = {Medical Imaging 2009: Visualization, Image-guided Procedures and Modeling. Proceedings of the SPIE},
      year = {2009},
      volume = {7261},
      pages = {72612K‑1 -- 72612K‑8},
      month = {February},
      file = {:Mirota2009.pdf:PDF;SPIE2009_sinus.pdf:SPIE2009_sinus.pdf:PDF},
      fundingagency = {NIH},
      owner = {dmirota},
      timestamp = {2009.03.01},
      url = {http://dx.doi.org/10.1117/12.812334}
    }
  • D. Abretske, D. Mirota, G. D. Hager, and M. Ishii, "Intelligent frame selection for anatomic reconstruction from endoscopic video," in Applications of Computer Vision (WACV), 2009 Workshop on, 2009, pp. 1-5.
    bibtex Go to document Go to document
    @INPROCEEDINGS{Abretske2009,
      author = {Abretske, D. and Mirota, D. and Hager, G.D. and Ishii, M.},
      title = {Intelligent frame selection for anatomic reconstruction from endoscopic video},
      booktitle = {Applications of Computer Vision (WACV), 2009 Workshop on},
      year = {2009},
      pages = {1 -5},
      month = {dec.},
      abstract = {Using endoscopic video, it is possible to perform 3D reconstruction of the anatomy using the well known epipolar constraint between matched feature points. Through this constraint, it is possible to recover the translation and rotation between camera positions and thus reconstruct the 3D anatomy by triangulation. However, these motion estimates are not stable for small camera motions. In this work, we propose a covariance estimation scheme to select pairs of frames which give rise to stable motion estimates, i.e. minimal variance with respect to pixel match error. We parameterize the essential matrix using a minimal 5 parameter representation and estimate motion covariance based upon the estimated feature match variance. The proposed algorithm is applied to endoscopic video sequences recorded in porcine sinus passages in order to extract stable motion estimates.},
      doi = {10.1109/WACV.2009.5403052},
      file = {:Abretske2009.pdf:PDF},
      issn = {1550-5790},
      journal = {Applications of Computer Vision (WACV), 2009 Workshop on},
      keywords = {3D anatomy;3D reconstruction;anatomic reconstruction;camera positions;endoscopic video sequences;epipolar constraint;feature match variance estimation;feature point matching;intelligent frame selection;motion covariance estimation;pixel match error;porcine sinus passages;image matching;image reconstruction;image sequences;medical image processing;motion estimation;},
      owner = {dmirota},
      timestamp = {2010.10.04},
      url = {http://dx.doi.org/10.1109/WACV.2009.5403052}
    }
  • H. Wang, D. Mirota, G. Hager, and M. Ishii, "Anatomical reconstruction from endoscopic images: Toward quantitative endoscopy," American Journal of Rhinology, vol. 22, iss. 1, pp. 47-51, 2008.
    bibtex Go to document Go to document
    @ARTICLE{Wang2008,
      author = {Wang, Hanzi and Mirota, Daniel and Hager, Gregory and Ishii, Masaru},
      title = {Anatomical reconstruction from endoscopic images: Toward quantitative endoscopy},
      journal = {American Journal of Rhinology},
      year = {2008},
      volume = {22},
      pages = {47-51},
      number = {1},
      month = {January/February},
      owner = {Daniel Mirota},
      pdf = {http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2850107/pdf/nihms184174.pdf},
      timestamp = {2008.03.06},
      url = {http://dx.doi.org/10.2500/ajr.2008.22.3129}
    }
  • H. Wang, D. Mirota, M. Ishii, and G. D. Hager, "Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator," in Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 2008, pp. 1-7.
    bibtex Go to document Go to document
    @INPROCEEDINGS{Wang2008a,
      author = {Hanzi Wang and Mirota, Daniel and Ishii, Masaru and Hager, Gregory D.},
      title = {Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator},
      booktitle = {Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on},
      year = {2008},
      pages = {1-7},
      month = {June},
      doi = {10.1109/CVPR.2008.4587687},
      file = {Wang2008a.pdf:Wang2008a.pdf:PDF},
      issn = {1063-6919},
      journal = {Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on},
      keywords = {endoscopes, image sequences, medical image processing, motion estimationadaptive scale kernel consensus estimator, endoscopic image sequences, feature tracking algorithm, robust motion estimation, structure recovery},
      owner = {dmirota},
      timestamp = {2008.09.14},
      url = {http://dx.doi.org/10.1109/CVPR.2008.4587687}
    }