Deep Grasping – artificial intelligence perfects the art of bin-picking

In short

If machine learning methods are to be used in industry, robots will have to learn faster than before how to grasp components safely. In the research project Deep Grasping, scientists from Fraunhofer IPA and the University of Stuttgart have developed a virtual learning environment. Here neural networks are trained and then transferred to real robots.

In detail

If robots learn to grasp objects from experience, the learning time is still too long to use artificial intelligence for handling tasks economically in industries. With the help of the developed virtual learning environment, robots now train their neural networks even before they are put into operation: The bin-picking processes are now only simulated. The pre-trained networks are then transferred to the real robot, which saves time.

For the project a dataset for neural networks was created, which shows typical bin-picking scenarios and, thanks to its size, is able to train even deep neural networks. Within this framework, neural learning methods for object recognition with 2D and 3D data for the separation of workpieces by robots were developed and verified on test platforms. The image processing algorithms teach themselves new workpieces in order to reduce the need for experts when it comes to commissioning or changing workpieces. Within the research project an approach for 6D object pose estimation was developed, which provides robust and accurate estimations based on simulated and real data within a few milliseconds.

The project was based on two applications: the separation of chaotically stored, rigid workpieces, as is often done in industry when loading machines, and the handling of stacked, sometimes yielding objects, which are typically found in retail.