Automation Technology
Superpixel

Superpixel

Image pixels are the base unit in most image processing tasks. However, they are a consequence of the discrete representation of images and not natural entities. Superpixels are the result of perceptual grouping of pixels, or seen the other way around, the results of an image oversegmentation. Superpixels carry more information than pixels and align better with image edges than rectangular image patches. Superpixel can cause substantial speed-up of subsequent processing since the number of superpixels of an image varies from 25 to 2500, in contrast to hundreds of thousands of pixels. A superpixel segmentation of an input image is illustrated in the following figure:

superpixels.jpg

Our research interests cover the evaluation of properties of superpixel segmentation algorithms in benchmarks as well as designing new algorithms.

If you are interested in mobile robot localization, you can have a look at the application of superpixel vocabularies for appearance change prediction across season.

Superpixel Benchmark

Superpixels are becoming increasingly popular in many computer vision applications. Their benefit is shown in applications like object recognition, segmentation and automatic photo pop-up. Downsides of using superpixel segmentation as preprocessing step are the computational effort for the computation of superpixels and more importantly the risk of loosing meaningful image edges by placing them inside a superpixel. Depending on the application and the used superpixel algorithm, subsequent processing steps can struggle with a non lattice arrangement of the superpixels. Therefore, the careful choice of the superpixel algorithm and its parameters for the particular application are crucial.

We provide a Matlab toolboxe for comparing superpixel algorithms. It combines and extends our two previously published benchmark toolboxes. The toolbox includes metrics to evaluate the segmentation quality, runtime, stability and compactness. The results of 14 available algorithms are provided together with the toolbox (in the numerical_results subfolder). Some example results are shown below, for details have a look at my thesis .

Prerequisites to run the benchmark are:

Details and evaluation of 14 available superpixel implementations can be found in this thesis: The benchmark and some the metrics were firstly presented in the following papers:

Superpixel Algorithms

A major insight from our previous work on extensive comparison of superpixel segmentation algorithms is the existence of several trade-offs for such algorithms. The most intuitive is the trade-off between segmentation quality and runtime. However, there exist many more between these two and a multitude of other performance measures. We designed two new superpixel segmentation algorithms, based on existing algorithms, that provide better balanced trade-offs. Better balanced means, that we increase one performance measure by a large amount at the cost of slightly decreasing another. The proposed new algorithms are expected to be more appropriate for many real time computer vision tasks. The first proposed algorithm, Preemptive SLIC, is a faster version of SLIC, running at frame-rate (30 Hz for image size 481x321) on a standard desktop CPU. The speed-up comes at the cost of slightly worse segmentation quality. The second proposed algorithm is Compact Watershed. It is based on Seeded Watershed segmentation, but creates uniformly shaped superpixels similar to SLIC in about 10 ms per image.

C++ source code and Matlab wrappers for both algorithms are available:

Details and evaluation of the algorithms can be found in our ICPR paper:

Related Publications

Neubert, P. (2015) Superpixels and their Application for Visual Place Recognition in Changing Environments. Dissertation: TU Chemnitz

Neubert, P. & Protzel, P. (2014) Compact Watershed and Preemptive SLIC: On improving trade-offs of superpixel segmentation algorithms. In Proc. of Intl. Conf. on Pattern Recognition (ICPR)

Neubert, P. & Protzel, P. (2013) Evaluating Superpixels in Video: Metrics Beyond Figure-Ground Segmentation.. In Proc. of British Machine Vision Conference (BMVC)

Neubert, P. & Protzel, P. (2012) Superpixel Benchmark and Comparison. In Proc. of Forum Bildverarbeitung