Zachary Pezzementi :: Research
My research focuses on novel applications of automated sensing to both fully automated and human-cooperative robotic systems. Recent projects work toward modeling, analyzing, and providing guidance for robotic surgery, manipulation, and exploration tasks, using vision and touch sensing. My thesis work focuses on object recognition using array-type tactile force sensors.
Human Detection and Tracking in Agriculture
We developed a large-scale datset and benchmark for evaluating person detection capabilities in off-road environments, specifically focused on agriculture. The dataset includes nearly 100k labeled frames of stereo video with GPS localization. We evaluated several leading person detection approaches and presented one of our own as well.See the NREC project page for details.
Manipulating and Perceiving Simultaneously
The goal of this project is to develop a system, consisting of a robotic hand equipped with tactile sensors, capable of autonomously exploring an environment and identifying objects that have been encountered before, while manipulating the unknown objects as necessary. The ability to explore an unknown object using solely haptic information requires expansion of the state of the art both in object recognition and in manipulation, in addition to the application of simultaneous localization and mapping techniques to the haptic domain. Our approach focuses first on the adaptation of feature-based object recognition methods from the computer vision domain to haptic object recognition.
Visual Tracking of Articulated Objects
Many objects encountered in the real world can be described as kinematic chains of parts with roughly uniform appearance characteristics. We developed a GPU-accelerated method for tracking such objects in single- or multi-channel (eg, stereo) video streams in diverse domains. The method consists, in brief, of modeling the appearance of the various object parts, then rendering a 3D model of the target object geometry from each view, and measuring the consistency of the resulting image with an appearance class probability map derived from the video images. It's been demonstrated in both surgical and generic settings.
Virtual Fixtures for Human-Machine Cooperative Manipulation
We suggest that dynamics beyond the first order are important in a number of tasks in both open and minimally-invasive surgery. In response, we have designed guidance virtual fixtures which focus not just on the position of the tool, but also its velocity. These fixtures are intended for use in providing guidance to replicate motions, such as those of an expert surgeon demonstrating a procedure to a novice.For more information, see the Human Machine Collaborative Systems overview.
We are interested in modeling and understanding the underlying structures in surgical motions. We would like to eventually use this understanding to create benchmarks for surgical skill evaluation, to develop methods for better surgical training and to automate the documentation of surgeries for libraries.See the Surgical Modeling project website for more details.
Retinal OCT Registration
Optical coherance tomography is a non-invasive imaging modality analogous to ultrasound using light rays. Registration of pre-operative OCT images to more familiar intra-operative fundus images allows precise location of pathologies which would otherwise be invisible.
The following pertains to all of the IEEE publications below:© IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
- Zachary Pezzementi, Trenton Tabor, Peiyun Hu, Jonathan K. Chang, Deva Ramanan, Carl Wellington, Benzun P. Wisely Babu, and Herman Herman. "Comparing Apples and Oranges: Off-Road Pedestrian Detection on the NREC Agricultural Person-Detection Dataset". arXiv preprint arXiv:1707.07169.
- Project Page
- Trenton Tabor, Zachary Pezzementi, Carlos Vallespi, and Carl Wellington. "People in the Weeds: Pedestrian Detection Goes Off-road". IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Purdue University, West Lafayette, IN, October 2015.
- Project Page
- Zachary Pezzementi and Gregory D. Hager. "Tactile Object Recognition and Localization Using Spatially-Varying Appearance". In International Symposium on Robotics Research (ISRR), Flagstaff, AZ, August 2011.
- Zachary Pezzementi, Erion Plaku, Caitlin Reyda, and Gregory D. Hager. "Tactile Object Recognition From Appearance Information". In IEEE Transactions on Robotics, Special Issue on Robotic Sense of Touch, 27:3, June 2011. pp. 473-487.
- Zachary Pezzementi, Caitlin Reyda, and Gregory D. Hager. "Object mapping, recognition, and localization from tactile geometry". In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 2011. pp. 5942-5948.
- Zachary Pezzementi, Erica Jantho, Lucas Estrade, Gregory Hager. "Characterization and Simulation of Tactile Sensors". In Haptics Symposium, Waltham, MA, March 2010. pp. 199-205.
- Poster -- Finalist for Best Poster Award
- James C. Gwilliam, Zachary Pezzementi, Erica Jantho, Allison M. Okamura, Steven Hsiao. "Human vs. Robotic Tactile Sensing: Detecting Lumps in Soft Tissue". In Haptics Symposium, Waltham, MA, March 2010. pp. 21-28.
- Oral Presentation
- Zachary Pezzementi, Sandrine Voros, and Gregory D. Hager. "Articulated Object Tracking by Rendering Consistent Appearance Parts". In IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan, May 2009. pp. 3940-3947.
- Oral Presentation (and associated videos)
- Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Allison Okamura. "Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: A Learning-based Method". In Haptics Symposium, Reno, NE, March 2008. pp. 209-215.
- Oral Presentation (and associated video)
- Ioana Fleming, Sandrine Voros, Balazs Vagvolgyi, Zach Pezzementi, James Handa, M.D., Russell Taylor, Gregory Hager. "Intraoperative Visualization of Anatomical Targets in Retinal Surgery". In Workshop on Application of Computer Vision (WACV), Copper Mountain, CO, January 2008. pp.1-6.
- Zachary Pezzementi, Allison Okamura, Gregory D. Hager. "Dynamic Guidance with Pseudoadmittance
Virtual Fixtures". In IEEE International Conference on Robotics and
Automation (ICRA), Rome, Italy, April 2007. pp. 1761-1767.
- Oral Presentation (w/o videos)
- Geoffrey Hollinger, Zachary Pezzementi, Benjamin Mitchell, Yavor
Georgiev, Anthony Manfredi, and Bruce Maxwell. "Design of a Social Mobile Robot Using
Emotion-Based Decision Mechanisms". In IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), Beijing, China, Fall 2006. pp.
- Zachary Pezzementi. "Object recognition using tactile array sensors." PhD thesis, Johns Hopkins University, Department of Computer Science, Baltimore, MD, May 2011.
- Geoffrey Hollinger, Zachary Pezzementi, Alexander Flurie, and Bruce Maxwell. "Design and Construction of an Indoor Robotic Blimp for Urban Search and
Rescue Tasks". Swarthmore College Senior Design Thesis, Spring 2005.
The following code was developed over the course of my research and seemed potentially useful to a wider audience, so I've released it under GPL-v3. Please refer to the documentation of each for detailed info.
- MAPS-TFSS - Tactile Force Sensor Simulator
Library Page | Documentation | Download
- MAPS-PF - Generic particle filtering classes
Documentation | Download
This page first went online January 2007. Last updated 10/24/10. Copyright Zachary Pezzementi