Cameras measuring three-dimensional are relatively new. They additionally link image data to information concerning the distance. In contrast to stereo cameras, they work in an active way and obtain information through measuring the distance between camera and object. This happens by means of a modulated infrared light source which is sent out by the camera. The signal is reflected by an object and enters the camera's sensor. That way, information about the distance can directly be calculated.

Currently, two systems are being evaluated and used for the processing of sensor signals: The PMD sensor (Photonic Mixer Device) by PMD Technologies and a 3D CMOS camera, which was developed within the european project UseRCams. Both systems could be applied in collision detection, in monitoring the blind spot or in the protection of pedestrians.

The CAN-Bus is a digital standard in communication technology, which is currently established area-wide within the automotive industry and sensor technology. In order to access the information on the one hand of the sensors integrated by the manufacturer and on the other hand of the measurement devices installed additionally, another entity is necessary - the CAN gateway. It channels the data of different CAN buses as well as of other media (RS232) and records them. By isolating the connected segments from each other, the gateway assures a completely interference-free communication infrastructure within the vehicle.

Some sensors and actuators are not natively equipped with an appropriate gateway to connect them to a CAN bus. In order to nevertheless allow a simple and consistent connection, a CAN interface module has been developed. It consists of a microcontroller with a CAN interface (Atmel AT90CAN128), of digital and analog inputs and outputs, and a corresponding periphery. By means of the microcontroller, the user is able to gather sensor data, to run a preprocessing, and to communicate the collected data on the CAN bus. Furthermore, actuators or displays can be accessed via the CAN bus.

The Geographic Information System (GIS) has access to digitally stored maps with detailed information about the course of the road. Furthermore, it provides data concerning road markings, curbs, and footpaths. In the field of sensor data fusion, digital maps are considered to be sensors. They further provide valuable information about the transportation system, constitution and condition of traffic routes, and striking objects within the vehicle's environment (e.g. buildings, trees, street lighting). 

 

The vehicle's interior is subject to current research as well. Two cameras measure the three-dimensional direction of the driver's head and identify the line of vision. In order to make these information available also in the dark, the cameras are supported by two infrared illuminators. The eye and head tracker are usually applied above the steering wheel.

Based on the data about the head movements, line of vision, and movement of the eyelids, information about the driver's exhaustion can be deduced.

This camera is used for detecting the vehicle's environment especially at good visibility during the day. The measurement of color information is mainly relevant for the detection and recognition of facilities conducting the traffic (e.g. road signs, traffic lights).

A far infrared camera is a sensor which is able to measure thermal radiation. Therefore, it is also known as thermal imaging camera. The far infrared camera is used for the recognition of the vehicle's environment in situations with bad visibility (e.g. in the dark). It is especially applicable for the detection of other road users (vehicles, bicyclists, pedestrians). Advantages of this camera are its passive sensor principle (it does not need any external illumination) and its ability to detect differences in temperature. Living objects like human beings or animals can therefore be detected especially well. A far infrared camera is for instance applied within the actual night-vision system of BMW.

Three far infrared sensors are responsible for sensor signal processing. One is a thermal imaging camera based on the pyroelectric sensor system with a resolution of 160 x 120 px. Two more cameras are equipped with a micro-bolometric sensor and have a resolution of 320 x 240 px. Both types of camera are able to detect heat radiation within a wavelength range of 7 µm - 14µm.

The speed sensor collects data regarding the vehicle's velocity and the driving direction (forwards or backwards). The yaw rate sensor primarily detects the vehicle's rotational speed (angular velocity). Altogether, the two sensors are responsible for the on-board identification of the vehicle's change of position over time.

The Global Positioning System (GPS) is a navigation and positioning system based on satellites. A GPS system is nowadays used in almost every vehicle. In current research, GPS is one of the most important sensors for estimating the position and proper motion of a vehicle. The spatial accuracy of standard GPS sensors is about 15 meters. In order to precisely estimate the proper motion (of a vehicle), the GPS is supplemented by data from inertial sensors. An improved accuracy of GPS sensors can be obtained by incorporating additional adjustment data which are transmitted to the GPS receiver via terrestrial radio signals. Such systems procure an accuracy of up to 0.2 meters and are called Differential GPS (DGPS).

A highly precise DGPS is available in addition to to the prevalent GPS modules. This DGPS delivers ground truth for sensor data fusion. By means of this high-precision information one can for instance scrutinize and optimize the algorithms of signal processing (for object recognition, proper movement estimation etc.). The system provides position measurements with an accuracy of up to 0.01 meters.

The Global System for Mobile Communication (GSM) allows for mobile phones to transmitt local data to a control center. The data, which have been locally collected at the vehicle's position, are transmitted to a center, which interrelates them to data of other mobile measuring systems, and then, in purified form, makes them available once again to the road users. 

An inertial sensor is able to measure three-dimensional movements in a highly precise way. The system detects translations and rotations concerning the three spatial axes. Inertial sensors are either mechanical or micro-mechanical systems. Their advantage lies in the high-precision measurement of acceleration relative to their own coordinate system. As their mode of operation does not allow long-term absolute positioning, they are combined with GPS sensors in order to estimate a vehicle's position.

The laser measuring system LMS 221 is a system working completely non-contact. By means of a laser beam it scans the environment two-dimensional. For each measurement, information about the distance from the laser to the object as well as the object's value of reflection are displayed. The laser scanner scans with a maximum frequency of 75 Hz (1 degree resolution) and has a maximum angular resolution of 0.25 degrees. The resolution of the distance value is very high compared to those of radar sensors. The laser measuring system is used to detect objects and the roadside within the vehicle's surroundings.

These illuminators emit light within the near infrared area (~ 800nm). Light in this wavelength is not visible for humans. The illuminators are employed to support the near infrared camera. With the aid of this additional illumination, an infrared camera can recognize objects up to a distance of 150 meters. Infrared lights like these are applied for example within the active night vision system of Mercedes Benz.

We appreciate Automotive Lighting supporting our research by sponsoring these illuminators.

The near infrared camera is used to collect data about the vehicle's environment especially in situations with bad visibility (e.g. in the dark). It is applicable for the recognition of the roadside as well as for the detection of other road users (vehicles, bicyclists, pedestrians). In order to augment the camera's capacity, one can apply external infrared illuminators.

Radars are sensors measuring actively. They send electromagnetical signals within the gigahertz frequency range (GHz). Radar sensor systems are important in the field of the recognition of a vehicle's environment, as they are able to provide data about position and velocity vectors from several objects at the same time. Their lower measurement accuracy can be compensated by means of a fusion with other sensors (e.g. Lidar). When it comes to object detection within the close-up range, short-range radars with a frequency of 24 GHz and a reach of up to 60 meters are applied. Long-range radar sensors work by means of 77 GHz signals and are able to identify targets up to a distance of 150 meters.

A stereo camera combines two single cameras into a sensor system. This way it can generate data about the third dimension (distance) in addition to the two-dimensional image information. The way a stereo camera works is similar to that of the human vision. The conjunction of two image sensors allows the calculation of distance information for every picture element. This is an advantage compared to singular cameras, which are not able to provide such information about the distance of the objects within the camera's field of vision. The stereo camera works passive and generates its information from the algorithmic conjunction of the two single images.

Throughout the last years, the equipment of traffic routes with sensor technology has constantly been increasing. However, an area-wide detection of the traffic condition is hardly obtainable this way. That is why vehicles are equipped with different sensor technology measuring the current traffic situation (e.g. traffic jam) or, like in this case, the actual state of the road surface (dry, water, ice). The road condition sensor is an active sensor, which is able to detect the weather-dependent state of the road by means of electromagnetic impulses and signal analysis.