An Integrated Perception, Navigation, and Manipulation Pipeline for Semi-Autonomous Door Opening with a Mobile Manipulator

Abstract

Autonomous door opening is a key capability for service robots. It requires integration of perception, navigation, localization, and manipulation. This paper presents a semi-autonomous door-opening pipeline for a mobile manipulator. The system requires manual initialization of the robot pose in RViz; it autonomously detects and approaches the door, estimates the 3D handle position, aligns the mobile base, performs the pre-grasp positioning, and executes handle manipulation and door opening. The system was evaluated on a real mobile manipulator in 20 trials: 10 nominal operation trials, 5 trials with varying initial robot-to-door distance, and 5 trials varying initial end-effector height. In nominal condition, door perception, navigation, 3D localization, alignment, and pre-grasp positioning succeeded 10/10, while complete door opening succeeded 8/10. The system successfully handled the initial robot-to-door distance in four of five trials. However, changes in the initial end-effector height reduced performance, completing only one of five trials. Failures occurred mainly in the contact and interaction between the handle and the gripper. Physical handle manipulation is the main limitation of the current system.