This project provides an automatic pipeline of creating a real-world 6D pose detection dataset with the help of a wandering UR5 robotic arm on a mobile platform. The 3D model of the training object and its initial offset to the robot base are to be provided. The dataset consits of RGB images, depth images, segmentation masks and 6D poses for the training object.
This automatic pipeline is mainly targeted at industrial manipulating or quality inspection tasks, where a deep learning model is used to identify/inspect products with accurate known models. The considered application flowchart for the project is shown as in the figure below:
In this project, the UR5 robot is mounted on a mobile platform and holds an Intel D435 RGBD camera to wander around the object of interest with the following pattern:
|
|
The RGB and depth images are captured by the sensor and the ground truth of object’s 6D poses, instance/segmentation mask and 3D bounding boxes are calculated from the camera matrix corresponding to robot’s joint states in real time.
Experiment:
|
|
Result:
To evaluate the quality of ground truth generated by the method, we annotated the ground truth directly from RGB images once again and checked the BF-Score and IoU of real ground truth with the ground truth generated by our method. The subsegment with YCB bottle got an average score of 0.5156 (BF) and 0.8451 (IoU). The subsegment with Lego car got an average score of 0.8618 (BF) and 0.8551 (IoU).
Demo:
A simple show case in Gazebo simulation:
More details to be found in the code.
