Synthetic Data Generation using 3D Computer Graphics

For modern computer vision applications, neural networks are often the key to success. Such systems are trained with huge amounts of data with the aim to learn different variations of objects. Depending on the domain, it can happen that the required training quantities are not available and already existing data sets cannot be adapted. One approach to counteract this problem is the generation of synthetic data. In the context of computer vision applications, such data can be generated using 3D computer graphics. This usually has the advantage that the creation of large amounts of data is more cost-effective and efficient.

A work group of the Chair of Digital and Circuit Technology is investigating various methods for the generation of synthetic data for computer vision applications and how these can be used in the field of AAL (Ambient Assisted Living) or for autonomous driving. Another field of research is the optimization of synthetic data for realistic abstraction.

The video in figure 1 visualizes possible scenarios in the area of indoor activities and autonomous driving.

Fig 2: synthetically generated point cloud

Figure 2 illustrates the generation of point clouds from synthetic data as it would be generated by a 3D camera system within an apartment. In addition to the two-dimensional camera images, point clouds provide spatial information and can thus support object detection and classification. Synthetically generated point clouds enrich real 3D data sets, which are needed to develop and train classification algorithms.

Current research emphasis and open topics are:

• Generation of synthetic data using Unity3D, Unreal Engine and Blender

• Domain adaptation of synthetic data using Generative Adversarial Networks and Style Transfer Methods

• omnidirectional images