Fast, flexible 3D image processing
The IPA researchers worked on several development focuses as part of the project, during the course of which they expanded their expertise as well as the resulting range of 3D image processing-based products and services. One core aspect was the creation of what is known as semantic digital twinning of retail products. This involves using sensors to capture any product in a scanning station, for example. The product is then modeled three-dimensionally based on a wide range of information such as shape, color, and texture. The modeling process, in which Fraunhofer IPA collaborated with the Kaptura company, also involved machine learning methods, a subfield of AI (Artificial Intelligence).
With the help of the technologies developed, it is possible to model wide product ranges quickly and in 3D with high resolution texture. During the modeling process, semantic information such as logos, nutritional value or quantitative information on products is also recognized and digitally collected. The models generated are the basis, for example, for digital twins of a real store, a web store or automated product analyses, such as those carried out in laboratories.
In addition, a scalable object recognition solution optimized for retail has been created, enabling automation solutions for production and logistics through precise optical product identification. Application examples include robot-based bin picking or bin packing, i.e. the packing of goods of all types and shapes. The latter task, also known as “order picking” or “pick & pack”, in which goods are removed from crates or shelves and neatly packed into a target carton, is becoming increasingly important, especially in these times of booming online retail. The object recognition solution developed to support these processes can be taught fully automatically on digital 3D product models in less than an hour for each new product.