Examples demonstrating the interaction of sensing and cognition

Building partly on the Center for Sensing and Cognition, the Center for Humans and Technology (MeTech) was established at Chemnitz University of Technology in January 2023. Consequently, this page is not updated anymore.

In both cases – in sensing and in cognition – some processes are covered better by humans, some by technology. Hence, goal-oriented collaboration of humans and technology provides multiple synergies, and those can be furthered by a profound scientific understanding of human and technological sensing and cognition. This shall be demonstrated by six prototypical examples.

Example 1: Driver assistance

Systems for driver assistance – in contrast to pure driving assistance systems – require the driver to be part of the closed control loop. This makes them a prototypical and wide-spread example for an application that unites sensing and cognition. First, technical sensors capture the state of the environment (road conditions, vehicle state, traffic, etc.) and possibly the state of the driver (vigilance, distraction, etc.). A first step of technical cognition follows: sensory data are assigned value, are classified and prioritized. Then, information display towards the driver is decided upon, taking the constraints of the human sensory systems into account. For example, warning signals must be alarming and intuitive, but not permanently distracting. Based on these data, human cognition stays in command for deciding on driving manoeuvres, unless they violate hard physical constraints, as monitored and evaluated by a second stage of technical cognition. Consequences of actions are in turn captured by technical sensors, thereby closing the control loop that includes the human user (“human-in-the-loop”).

Example 2: Human-robot collaboration in production

Human-robot collaboration is another prime example for the “human-in-the-loop” concept. In contrast to classical industrial production, human and robot act simultaneously in a shared workspace, and in an ideal scenario they interact collaboratively. The state of the workspace needs to be captured by technical sensors and evaluated by technical cognition. Data are then processed to be communicated to the human user, taking the constraints of human sensory processing in acquiring presented information into consideration. Knowledge of human cognition, in particular its social aspects, allows human-robot interaction to be intuitive and lowers the requirements for system‑specific training.

Example 3: Lighting systems and sensors change our environment

Human sensing and cognition is challenged by technological progress. Our visual environment and the development of novel light-emitting diodes (LEDs) present a prototypical example: not only do these LEDs replace conventional light sources (“Retrofit”), but they also exhibit qualitatively new properties, such as a controlled modulation of colour and brightness. Explorative systems combine light and sound to address several human sensory modalities at the same time, and are wirelessly connected to a sensory network. Target-oriented development in this dynamic research area requires a holistic approach, which takes into account the properties of human sensory systems and the capability of human cognition to adapt to novel environments as well as the potential of technical sensing and cognition, such as the development of light sources that adapt to ambient lighting conditions (e.g., adaptive front lights).

Example 4: Age-appropriate human-technology interaction

The demographic change inflicts a special demand on innovative technology to remain accessible for all age groups. Across the lifespan, human sensory systems as well as human cognition undergo a substantial change. As a prerequisite for keeping technology readily accessible for humans of all age groups, and to build upon parameters like experience in a constructive fashion, the sensory effects of aging must experimentally be isolated from the cognitive effects. Only then, technical sensors and – by means of intelligent control through technical cognition – interaction and communication between technology and the human user can be adapted to become age-appropriate. This is exemplified by the automatic detection of support requirements of elderly users when using technology and the subsequent targeted intervention.

Example 5: Augmented human sensing

Optimal technological support (“intelligent sensors”) makes itself available whenever human sensory or cognitive capabilities are impaired – either temporarily (e.g., sensory overload) or permanently (e.g., reduced sensory sensitivity caused by disease). The replacement of an inoperative part of the human sensor by a technical sensor, such as in the case of the cochlear implant, belongs to this category. Even in healthy humans, sensory systems face restrictions: some physical quantities (radioactivity, electromagnetic radiation outside the visible spectrum, etc.) cannot be sensed or can be sensed only with poor resolution. This is a typical case for augmentation by technical sensors. In the ideal case, sensor technology performs “intelligent” preprocessing (technical cognition) to present the sensory data to human cognition in a naturalistic way; that is, as closely matched as possible to the actual human sensory organs. To meet this goal, human sensory systems, which in most cases are the input channel to human cognition for the technical data, as well as the processing of novel signals by human cognition need to be understood.

Example 6: Scene perception and understanding

In scene recognition and understanding, human sensing and cognition outperform their technical counterparts still by far. To provide users who suffer from corresponding impairments with adequate technical support (see case 5) and to achieve complex scene perception in dangerous situations by technological solutions, technical sensors and cognition should eventually be equipped with vastly improved capabilities. To reach this objective, a better understanding of the sensory and cognitive processes that allow humans to perceive, recognize and understand complex scenes is needed. Then, the transfer to the technical system can build upon principles inspired by the biological system.