Person Detection and Localisation
with RGB-D Data

Person detection by means of optical sensors opens up a wide range of applications:

• Automatic monitoring of public areas
• Customer behaviour and personal attendance in retail as well as in public institutions (museums, clubs, etc.)
• Pedestrian detection in the automotive industry
• AAL (Ambient Assisted Living): Measuring a person's mobility, making social contacts, etc.

With the help of person detection algorithms it is possible to determine the number of people in a room as well as their respective three-dimensional position and extent. Fig. 1 shows the results of an algorithm that processes RGB-D data. The position data are validated with the help of a reference measurement system.

Fig. 2 shows an example application in the area of AAL. Using the position data, long-term statistics on a person's location and duration can be generated. The location inside the home, the times the person spends outside the home and the mobility, i.e. how often and how long the person is moving/circulating, can be recorded and evaluated later.
The analysis of the statistics, for example by the attending physician, can give information about strong changes to the usual daily routine or apparent reduction in mobility of a person living alone. The reasons for this may be a change in the state of health. With the help of the possibility of a later analysis by a doctor, a change in the condition of a person can be determined, which might have gone unnoticed under certain circumstances.

Pose Estimation with RGB-D Data

In addition to the position of a person, the rough posture (standing, sitting, lying) of the person is recognized using methods of machine learning. Fig. 3 shows the points extracted from the point cloud and associated with the respective persons. Based on these points, a person's pose can be determined using meaningful feature vectors (Fig. 4).

The inclusion of information on physical maintenance in long-term statistics allows more accurate conclusions to be drawn about a person's mobility. In addition, various movements of daily life, such as the movement of a bottle to the mouth, are learned and recognized.