Hybrid Image-Plane/Stereo (HIPS) manipulation is a method of processing image data, and of controlling a robotic manipulator arm in response to the data, that enables the manipulator arm to place an end-effector (an instrument or tool) precisely with respect to a target (see figure). Unlike other stereoscopic machine-vision-based methods of controlling robots, this method is robust in the face of calibration errors and changes in calibration during operation.
In this method, a stereoscopic pair of cameras on the robot first acquires images of the manipulator at a set of predefined poses. The image data are processed to obtain image-plane coordinates of known visible features of the end-effector. Next, there is computed an initial calibration in the form of a mapping between (1) the image-plane coordinates and (2) the nominal threedimensional coordinates of the noted end-effector features in a reference frame fixed to the main robot body at the base of the manipulator. The nominal three-dimensional coordinates are obtained by use of the nominal forward kinematics of the manipulator arm — that is, calculated by use of the currently measured manipulator joint angles and previously measured lengths of manipulator arm segments under the assumption that the arm segments are rigid, that the arm lengths are constant, and that there is no backlash. It is understood from the outset that these nominal three-dimensional coordinates are likely to contain possibly significant calibration errors, but the effects of the errors are progressively reduced, as described next.
As the end-effector is moved toward the target, the calibration is updated repeatedly by use of data from newly acquired images of the end-effector and of the corresponding nominal coordinates in the manipulator reference frame. By use of the updated calibration, the coordinates of the target are computed in manipulator-reference- frame coordinates and then used to the necessary manipulator joint angles to position and orient the end-effector at the target with respect to the same kinematic model from the calibration step. As the end-effector/target distance decreases, the computed coordinates of the end-effector and target become more nearly affected by the same errors, so that the differences between their coordinates become increasingly precise. When the end-effector reaches the target, the remaining effective position error is the distance that corresponds to more than about one pixel in the stereoscopic images of the target.
This work was done by Eric Baumgartner and Matthew Robinson of Caltech for NASA’s Jet Propulsion Laboratory. For further information, access the Technical Support Package (TSP) free on-line at www.techbriefs.com/tsp under the Information Sciences category. NPO-30492
This Brief includes a Technical Support Package (TSP).
Hybrid Image-Plane/Stereo Manipulation
(reference NPO30492) is currently available for download from the TSP library.
Don't have an account?
Overview
The document is a Technical Support Package from NASA's Jet Propulsion Laboratory, focusing on Hybrid Image-Plane/Stereo (HIPS) Manipulation, which is a technique used for precise manipulation in robotic systems, particularly in aerospace applications. The package outlines the advancements and methodologies related to the HIPS approach, emphasizing its significance in enhancing the capabilities of robotic manipulators.
Key highlights include the extensive testing conducted with the LEMUR robot, which features a 3 degree-of-freedom (d.o.f.) manipulator. This manipulator is designed to operate in environments with significant kinematic uncertainty, such as actuator backlash. To improve image analysis, a small feature or "cue" is added to the end-effector, allowing for better tracking and manipulation of objects.
The document details the calibration process of the camera systems used in conjunction with the manipulator. It explains that the camera calibration is performed relative to the manipulator's reference frame, ensuring that the image-plane appearance of targets can be accurately specified by ground operators. The manipulator generates camera models that are updated in real-time throughout its trajectory, allowing for locally accurate positioning and manipulation of targets.
One of the notable aspects of the HIPS technique is that it does not require continuous real-time vision. Instead, it relies on discrete samples of the image-plane appearance of the end-effector, which simplifies the processing requirements and enhances the system's efficiency. The in-situ calibration of the manipulator-generated camera model is also highlighted, ensuring precise positioning throughout the mission's duration.
Additionally, the document discusses the Sample Return Rover (SRR), a fast and lightweight rover equipped with a 4-wheel rocker suspension. This rover is designed for the retrieval of samples cached by a science rover and features autonomous rendezvous and docking capabilities. Like the LEMUR robot, it also utilizes a 3 d.o.f. manipulator with a gripping end-effector and a wide field-of-view stereo pair.
Overall, the Technical Support Package provides a comprehensive overview of the HIPS Manipulation techniques, showcasing their potential applications in robotic systems for space exploration and other fields, while also offering insights into the calibration and operational methodologies that enhance robotic performance in complex environments.
