Definitions

Artificial Intelligence (AI) provides machines with capabilities comparable to intelligent human behavior. The generic term AI encompasses problem-solving methods, including logic and planning procedures, which would require human intelligence. Probably the best known and most researched and applied method is machine learning.

Machine Learning (ML) enables machines and robots to generate knowledge by experience, which they then use to solve problems. The key to ML is data, which transforms the concept of traditional data processing. Until now, a program had to be written for a computer that uses input data to generate output data. Thanks to machine learning, a computer can now program itself. From the training data entered, machines learn to derive patterns and regularities autonomously and even evaluate unknown data correctly. To do this, a machine receives input data and, depending on the machine learning method, also output data, and then automatically creates the required program based on this data. For this type of learning there are again different methods, with Deep Learning being the most common.

Deep Learning (DL) uses deep artificial neural networks to acquire knowledge from data. Artificial neural networks are based on the processes taking place in the human brain, where each nerve cell (neuron) is connected to an average of 10,000 other neurons. In artificial neural networks, the neurons are arranged in layers with connections to the neurons of adjacent layers. To train a network well, it must receive large volumes of input data and, to start with, the correct output information. During the training step an optimization problem is solved, in which the discrepancy between the specified output data and the output generated by the network is reduced step by step. This teaches it to understand the data patterns and later to reach the correct result even with unknown data sets.

Reinforcement Learning (RL) is a form of machine learning in which an RL Agent is used to solve sequential problems that require decisions. RL is applied to problem setting when the quality of individual actions cannot be assessed directly, but rather the result of a series of activities. The task is represented by a reward signal.

The RL Agent interacts with its environment, receives rewards signals from it, and optimizes its decision accordingly to maximize its overall result (the sum of reward signals). For example, the RL Agent can be used to control a robot arm so that it reaches a moving target. The reward signal can be the distance between the robot arm and the target.

Recent breakthroughs have been achieved by combining Deep Learning with RL, whereby the RL agent is represented by a functional approximator (like a neural network). Deep RL has been successfully implemented to achieve superhuman performances in games like Go, Atari, etc. However, the application of Deep RL in an industrial context is extremely limited.

At Fraunhofer IPA we are promoting the use of Deep RL in industry. Some applications we are currently working on include:

• Smart gripper systems e.g. bin-picking
• Sensor-guided bin-picking
• Model-based simulation of manufacturing systems
• Learning parameters to optimize production processes
• Robust programming of robots in inaccurate environments
• Safe robot navigation
• Safe human-machine collaboration
• Improvements in product design, resource design (e.g. robot grippers)
• Order planning
• Fleet management for AGVs
• Maintenance planning