“EmAsIn” stands for “Emotion-sensitive Assistance Systems for Reactive Psychological Interaction with People”. Within the scope of the project, the team will study how people with illnesses can overcome special psychological or cognitive challenges in their everyday lives with the aid of targeted human-machine interaction and which aids can improve their quality of life. For optimal results, the partners are actively collaborating with people suffering from dementia or affective disorders and jointly investigating technological approaches and assistance functions.

User specific Human-Machine Interaction

Along with illness-related situations, the new system should be able to recognize social behavior patterns and nonverbal clues in human communication. In particular, the project team will study personalized information transfer concepts and new user interface concepts that specifically provide support functions and encouraging stimuli in situations recognized by the system as critical. The project also aims to extend existing assistance systems in this direction and upgrade them with new functions that detect and interpret implicit human communication signals, such as emotions or gestures, and that are able to derive and perform suitable support actions.

Recognize emotions reliably and react appropriately

For many years, Fraunhofer IPA has been working on the development of technical assistance systems that provide informational and actuatory support to elderly users. An example project is “safe@home”, in which the scientists realized a touchless and discrete assistance system for private environments using stationary 3D sensors. The system detects falls as well as immobility and derives suitable actions, such as informing neighbors or an emergency call center. Mobile robots like the Care-O-bot^® or the emergency assistance „MoBiNa“ are able to connect with those stationary sensors and can drive autonomously to the person in need of help after a fall. At the same time, they serve as communication platforms for the direct conversation with the emergency center.

In the current project, scientists of Fraunhofer IPA are further developing those technologies, specifically the stationary sensors for the detection of emotions as well as the mobile robotic system as an interaction and information interface. The challenge is to realize reliable emotion recognition. Whereas falls can be detected by motion sequences, emotions are expressed by gestures or the behavior of a person. The assistant systems must interpret those and adjust their reaction accordingly.