Intelligent Frame Extractor
IFE (Intelligent Frame Extractor) is a modular, cross-platform (C++/Python) software tool developed by the Chair of Computer Engineering that applies deep-learning and conventional object detection algorithms to video and image data from Indoor Flight Center (IFC) projects. It combines detection, heatmap visualization, automated PDF reporting, and video shortening in one application, letting new detection algorithms be plugged in as reusable extension modules for different drone and inspection use cases.
Purpose & Motivation
The different research projects carried out at the Indoor Flight Center (e.g. multicopter and power line inspection work) generate large volumes of video and image data that need to be analyzed for detected objects and events. IFE was created to give these projects a common, reusable data-processing tool rather than requiring each project to build its own pipeline. Its modular design specifically addresses the need to plug in new object detection algorithms as projects and their requirements evolve, allowing the same core application to adapt across different use cases.
Technical Description
Software and Architecture
- Modular architecture built for reusability and portability
- Core supports both C++ and Python
- New object detection algorithms (and their associated GUI features) can be integrated as extension modules
- Supports both deep-learning-based object detection algorithms and conventional image processing algorithms
- Uses a standardized data serialization format to log and store detection results
Key Technologies
- Object detection (deep learning–based and classical computer vision approaches)
- Data serialization for logging and portability of results
- PDF-based report generation from serialized detection data
Capabilities
IFE combines data analysis, image processing, and media playback into one tool:
Data Analysis
- Data serialization / logging of detection results, supporting reusability and portability
- Automated inspection report generation (PDF output) based on serialized data
- Heatmap generation to highlight frames/regions where objects were detected, with support for multiple object types
Image Processing
- Modular integration of image processing algorithms and their GUI components (C++/Python)
- Standardized serialization format across algorithms
Media / Operation
- Playback of images and video, including support for 4K resolution
- Zoom and drag controls to visually validate detection results
- Data projection: an overlay layer displaying detection results on top of the video, supporting multiple simultaneous objects
- Video shortening: automatically produces a condensed version of long videos containing only the relevant (detection-positive) segments, reducing dataset size
Research Context
IFE is used across several projects at the Chair of Computer Engineering, each applying its modular detection pipeline to a different domain:
- APOLI: IFE was first released to support the APOLI project, where it serves as a post-processing application for videos captured during power line inspection flights. Using both feature-based and deep-learning-based algorithms, IFE enables intelligent data processing aimed at inspecting insulators for burn marks.
- AFORI: In response to the European spruce bark beetle infestation, which has severely damaged forests across Germany and Saxony, IFE is used with deep-learning-based algorithms to localize regions of forests in Saxony affected by bark beetle infestation, supporting authorities' efforts to contain the spread.
- CE (Teaching): IFE also supports students in the Chair's own courses, who use it while developing image processing algorithms. Students can access a range of existing filters and integrate their own algorithms into the application, so that their work can be reused by future student cohorts.
