Hand-eye coordination through endpoint closed-loop and learned endpoint open-loop visual servo control

Abstract

We propose a hand–eye coordination system for a humanoid robot that supports bimanual reaching. The system combines endpoint closed-loop and open-loop visual servo control. The closed-loop component moves the eyes, head, arms, and torso, based on the position of the target and the robot’s hands, as seen by the robot’s head-mounted cameras. The open-loop component uses a motor–motor mapping that is learnt online to support movement when visual cues are not available.