Forschungsseminar

Introduction to Deep Reinforcement Learning and Successor Representations

Erik Syniawa

Thu, 30. 4. 2026, A10.368 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

In this talk I introduce basic deep reinforcement learning (RL) concepts from the ground up, covering imitation learning, policy gradients, and actor-critic methods, before asking what these approaches get wrong. Standard model-free RL conflates two distinct problems: learning where a policy goes and how rewarding those places are. This coupling makes reward adaptation expensive and transfer across tasks structurally difficult. I motivate the Successor Representation (Dayan, 1993) as a principled decoupling of dynamics from reward, and show how Successor Features (Barreto et al., 2017) can scale this idea to large state spaces. I close with a brief outlook on latent self-predictive representations as a modern version of these principles. All concepts are developed intuitively from first principles, (hopefully) making them accessible to everyone. But a small disclaimer: there will be math and there is no elegant way to fully hide a Bellman equation ;)

Bridging the Latency Gap: A Neurocomputational Model of Predictive Remapping

Nikolai Stocks

Thu, 23. 4. 2026, A10.368 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Visual perception remains remarkably stable across saccadic eye movements despite rapid and large displacements of the retinal image. This stability is thought to rely, in part, on predictive remapping processes that update spatial representations in anticipation of eye movements. However, the precise functional role and temporal characteristics of these mechanisms remain debated. In this study, we investigate predictive remapping within a neurocomputational model of spatial perception based on quasi-multiplicative gain fields. The model integrates retinal input with corollary discharge (CD) and proprioceptive eye-position (PC) signals to generate distributed spatial representations. We emulate a visual foraging paradigm and reproduce receptive field dynamics reported in electrophysiological recordings from lateral intraparietal area (LIP). Our results show that CD-driven predictive remapping is sufficient to account for the rapid transition between pre- and postsaccadic representations observed experimentally, particularly during the early postsaccadic interval. By systematically manipulating model components, we demonstrate that the latency period following saccade offset - during which bottom-up visual input is not yet available - imposes a fundamental computational constraint that necessitates predictive mechanisms. In contrast, presaccadic activity patterns are not well explained by CD-based remapping and are more consistent with attentional or top-down influences. Together, these findings suggest a functional dissociation between presaccadic modulation and postsaccadic continuity. Predictive remapping, driven by corollary discharge, appears to primarily serve to bridge the latency-induced gap in visual processing, ensuring a seamless transition between perceptual states across saccades.

Reframing Continual Motor Learning in Robotics: Hypernetwork-Driven Control of Spiking Reservoir Dynamics

Johannes Hofele

Thu, 16. 4. 2026, A10.368 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Der zunehmende Einsatz von Robotersystemen in dynamischen Umgebungen erfordert Regelungsstrategien, die nicht nur präzise, sondern auch anpassungsfähig sind und kontinuierlich lernen können. Herkömmliche Regelungsmethoden und rein datengesteuerte Ansätze haben oft Schwierigkeiten, auf neue Aufgaben zu verallgemeinern, und neigen bei sequenziellem Training zu katastrophalem Vergessen. Inspiriert von biologischen Lernmechanismen untersucht diese Arbeit ein Steuerungsframework, das kontextabhängige Hypernetzwerke mit Spiking-Reservoirs für die Echtzeit-Motorsteuerung kombiniert. Der Ansatz nutzt ein Hypernetzwerk, um die rekurrenten Gewichte eines Spiking-Reservoirs zu generieren, was eine aufgabenabhängige Modulation der Netzwerkdynamik ermöglicht. Die Architektur wird an einem Roboterarm mit zwei Gelenken evaluiert, der mehrere Trajektorienabbildungsaufgaben ausführt, darunter lineare, kreisförmige und Acht-förmige Bewegungen. Die Ergebnisse zeigen, dass das Modell einfache Trajektorien zuverlässig mit hoher Genauigkeit lernen und reproduzieren kann, während komplexere Bewegungen aufgrund der limitierten Ausdruckskraft des Reservoirs weiterhin eine Herausforderung darstellen. Darüber hinaus zeigt das System eine teilweise Fähigkeit, katastrophales Vergessen zu mildern, indem es unter kontinuierlichen Lernbedingungen Schlüsselmerkmale zuvor gelernter Aufgaben beibehält. Insgesamt unterstreicht diese Arbeit das Potenzial von hypernetzwerkgesteuerten Spiking-Systemen für adaptive Motorsteuerung und kontinuierliches Lernen, während sie gleichzeitig aktuelle Einschränkungen hinsichtlich Ausdruckskraft und Skalierbarkeit transparent aufzeigt.

Informationsveranstaltung für Studenten für das Sommersemester

Fred Hamker

Thu, 9. 4. 2026, A10.368

Camera Simulation Credibility Assessment for Automotive Perception Approaches

Arya Anup

Tue, 31. 3. 2026, https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Synthetic data is widely used for testing and validation for Autonomous driving tasks. This thesis investigates different methods to assess credibility of using synthetic data for real-world perception tasks in autonomous driving. A structured multi-level evaluation framework is conducted on object detection, multi-object tracking and semantic segmentation tasks using the dataset pairs KITTI-VKITTI 2 and Cityscapes-Synscapes. Leave-One-Out (LOO) training splits are used to analyze robustness across varying scene compositions. For assessing input-level similarity between real and synthetic training splits, Frechet Inception Distance (FID) and Kernel Inception Distance (KID) is used. Task-level performance differences are assessed using bootstrap-based statistical equivalence testing. Distribution-level and scene-level deviations are further analyzed to characterize the magnitude and structure of performance shifts. Featurelevel representation similarity between models trained on real and synthetic data is assessed using Centered Kernel Alignment (CKA) metrics. For the selected dataset pairs, the results indicate measurable feature distribution differences between real and synthetic training data that exceed intra-domain baselines. At the task level, aggregate performance metrics such as mAP@0.5, MOTA, and mIoU frequently fail statistical equivalence under stricter tolerance margins, whereas dominant object categories remain comparatively stable in some settings. Performance deviations are consistent across splits, and representation analysis reveals strong alignment in early network layers with increasing divergence in higher task-specific layers. However, no consistent split-wise relationship is observed between input-level similarity, representation similarity, and task-level performance differences. The findings demonstrate that credibility assessment requires integrating multiple complementary evaluation levels, as no single metric fully explains sim-to-real performance behavior.

Sehen trotz Augenbewegung: Simulation transsakkadischer Orientierungsintegration in einem neurocomputationalen Modell

Jonas Stefaniak

Wed, 18. 3. 2026, A10.368 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Das menschliche visuelle System erzeugt trotz ständiger Augenbewegungen eine stabile Wahrnehmung der Umwelt. Ein zentraler Mechanismus hierfür ist die transsakkadische Integration: die Fähigkeit, Merkmalsinformation über Sakkaden hinweg aufrechtzuerhalten und mit postsakkadischer Information zu einer kohärenten Repräsentation zusammenzuführen. Diese Arbeit implementiert das Experiment von Stewart und Schütz (2019) zur transsakkadischen Orientierungsdiskrimination in ein bestehendes neurocomputationales Modell des parietalen Cortex (TU Chemnitz, Professur Künstliche Intelligenz). Das Modell kombiniert einen dorsalen Pfad zur räumlichen Koordinatentransformation über Gain-Fields mit einem ventralen Pfad zur Merkmalsverarbeitung und wurde um orientierungsselektive Entscheidungspopulationen (V4_L, V4_R) erweitert. Die Simulationsergebnisse zeigen qualitativ zwei zentrale Muster der experimentellen Vorlage: einen transsakkadischen Vorteil gegenüber prä- und postsakkadischen Einzelpräsentationen sowie eine Abhängigkeit der Integration vom Referenzrahmen (spatiotop > retinotop). Der zugrundeliegende Mechanismus liegt in der Koordinatentransformation über LIP und Xh: Prä- und postsakkadische Information konvergiert nur dann, wenn der Stimulus spatiotop konstant bleibt. Da der Corollary-Discharge-Pfad präsakkadischen Input benötigt, um über das LIP Aktivität an der Zielposition aufzubauen, wurde für die postsakkadische Bedingung ein Platzhalter über eine zusätzliche Population implementiert, die im LIP einen konstanten Aufmerksamkeitsfokus unabhängig vom präsakkadischen Stimulus aufrechterhält.

Exploring Reinforcement learning strategies in automated wheel truing

Toni Rozsahegyi

Wed, 18. 3. 2026, A10.368 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Spoked bicycle wheel truing, correcting rim deviations by adjusting spoke tensions, presents a challenging testbed for reinforcement learning due to its high-dimensional state space, non-local action effects, and multi-objective constraints. The central question is whether RL agents can learn effective truing policies in simulation, comparing value-based (Rainbow DQN), policy-gradient (PPO), and model-based (TD-MPC2) approaches across different state representations, action spaces, and reward functions. A systematic ablation study reveals substantial performance differences across these design choices, with some results challenging conventional intuitions about state representation and action space formulation. The insights from these experiments inform the selection and optimization of a candidate policy for future sim-to-real transfer to physical truing machines.

OBM Pinpointing with Machine Learning

Nandini Mehta

Mon, 16. 3. 2026, https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Euro 7 introduces stricter real-driving emission requirements and shifts diagnostics from component-level fault detection to system-level emission monitoring. Although On-Board Monitoring (OBM) can detect excessive emissions, it often cannot determine which subsystem is responsible, particularly when faults are soft, combined, or distributed. This thesis addresses that challenge by proposing a compact data-driven method for subsystem-level fault pinpointing. Using a large multi-vehicle dataset of quasi-stationary diesel powertrain snapshots, the study applies feature selection and correlation analysis to reduce the input space. A baseline autoencoder is first used for anomaly detection, followed by a supervised contrastive autoencoder that learns a structured latent space where similar fault families cluster together and different faults are clearly separated. A lightweight classifier is then trained on these embeddings to identify emission-relevant subsystems. The proposed method achieves 94.16% accuracy on more than 370,000 validation samples, demonstrating a practical and interpretable solution for Euro 7 OBM-OBD diagnostic integration.

Development of Deep Learning Methods for Estimating Facial Landmarks in an In-Cabin Sensing Ground Truth System

Gaurav Rana

Tue, 10. 3. 2026, https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

This thesis investigates development of deep learning methods for estimating facial landmarks in an in-cabin sensing ground truth system using facial landmarks detection in near-infrared (NIR) imagery for in-cabin driver monitoring systems. Reliable landmark localisation is essential for downstream safety critical functions such as blink detection, yawning detection, and head pose estimation. However, models trained primarily on RGB data often struggle with illumination variations and domain differences present in NIR environments. To address this challenge, a hybrid deep learning architecture was developed consisting of two stages. The first stage adapts the Infrared face detector for face localisation, while the second stage employs a stacked hourglass network to estimate facial landmarks through heatmap regression. The model was evaluated using two output settings : a full 68 landmarks scheme and a reduced set of 30 critical landmarks designed for driver monitoring applications. A dedicated NIR dataset with subject-level splits was created, and the models were optimized using targeted training strategies and NIR-specific data augmentation. Experimental results show that the proposed approach significantly improves landmark localisation accuracy compared to pre-trained baselines, while also reducing failure rates. In particular, the reduced landmark configuration provides highly accurate and stable landmark estimates suitable for real-time cabin sensing applications.

Biologisch Inspirierte Active Vision

Jan Chevalier

Tue, 17. 2. 2026, A10.367 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Das aktive Sehen bei Robotern kann durch biologisch inspirierte Ansätze gelöst werden. Es werden 2 verschiedene Ansätze für das selbständige Zurechtfinden eines Roboters vorgestellt. Einmal ein Modell, welches sich selbst ein internes Modell der Umwelt macht ("A Hierarchical System for a Distributed Representation of the Peripersonal Space of a Humanoid Robot", M. Antonelli, 2014). Außerdem ein Modell, welches an 3 verschieden Roboterköpfen getestet wurde ("A Portable Bio-Inspired Architecture for Efficient Robotic Vergence Control", A. Gibaldi, 2017). Des Weiteren wird ein Ansatz der biologisch inspirierten Objekterkennung vorgestellt ("Active Vision : on the relevance of a Bio-inspired approach for object detection", K. Hoang, 2019).

Part 2: Why did I drive to work on my day off: Perspectives on goal-directed and habitual behavior with implications for future AI systems

Erik Syniawa

Tue, 3. 2. 2026, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Although modern AI systems excel at complex tasks such as language generation, they lack the adaptive efficiency that characterizes human behavior. Humans seamlessly balance deliberate planning with automatic responses depending on context and computational demands. In this talk we will discuss recent frameworks suggesting that goal-directed and habitual control are integrated rather than separate systems. Butz et al. (2025) propose a computational account where context inference optimizes cognitive effort, while Hamker et al. (2025) propose an anatomical perspective through interacting cortico-basal ganglia loops and learned shortcuts. I will outline the potential synergies and incompatibilities that could arise in integrating these frameworks. Finally, I will discuss how principles from these frameworks particularly context integration and shortcuts could potentially inspire more efficient AI architectures and how these principles relate to already published AI approaches.

Interacting cortico - basal ganglia - thalamocortical loops in behavioral control

Fred Hamker

Tue, 27. 1. 2026, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

I will present a new framework explaining how the brain controls goal-directed behavior, habits, and their learning processes. I propose that goal selection is implemented through the limbic loop, which connects the hippocampus to the ventral basal ganglia. This loop retrieves goal plans from memory, allowing the brain to 'virtualize' potential outcomes based on prior experiences. Once a goal is selected, plan execution is carried out by other, hierarchically organized cortico-basal ganglia circuits, with lower-level loops being guided by objectives set by higher-level ones. I will also present findings from a neuro-computational model applied to a well-known two-stage decision-making task. Our results highlight the central role of hippocampal replay in learning and behavior, particularly in distinguishing between model-based and model-free learning strategies.

Explaining the Interaction Between Criticality and Evolutionary Success in Recurrent Neural Network Models for Dynamical Control

Idris Hafsa

Tue, 20. 1. 2026, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Neural networks operating in real-world environments are often exposed to noisy and unpredictable inputs, which can lead to unstable behavior. This work investigates how the dynamical regime of recurrent neural networks influences their robustness under such perturbations, with a focus on the concept of criticality. Recurrent networks were trained in a controlled environment and subjected to external disturbances to evaluate their stability and performance. The results indicate that classical indicators of criticality do not provide a reliable predictor of robustness or performance. Instead, performance is strongly shaped by emergent behavioral strategies and task-specific dynamics, highlighting the need for alternative approaches to studying stability and robustness in recurrent neural systems.

Why did I drive to work on my day off: Perspectives on goal-directed and habitual behavior with implications for future AI systems

Erik Syniawa

Tue, 16. 12. 2025, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Although modern AI systems excel at complex tasks such as language generation, they lack the adaptive efficiency that characterizes human behavior. Humans seamlessly balance deliberate planning with automatic responses depending on context and computational demands. In this talk we will discuss recent frameworks suggesting that goal-directed and habitual control are integrated rather than separate systems. Butz et al. (2025) propose a computational account where context inference optimizes cognitive effort, while Hamker et al. (2025) propose an anatomical perspective through interacting cortico-basal ganglia loops and learned shortcuts. I will outline the potential synergies and incompatibilities that could arise in integrating these frameworks. Finally, I will discuss how principles from these frameworks particularly context integration and shortcuts could potentially inspire more efficient AI architectures and how these principles relate to already published AI approaches.

Perisaccadic attentional updating in Area V4: A neurocomputational approach

Julia Bergelt

Tue, 18. 11. 2025, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Psychophysical studies with human subjects investigated the dynamics of spatial visual attention around the time of saccades. They revealed that attention pointers remap against saccade direction, but at the same time show a lingering at the irrelevant retinotopic position after saccade. Involved brain areas have remained elusive. However, recordings from neurons in visual area V4 of macaques, obtained from a novel experimental design, con rm remapping of spatial attention to a position opposite to the saccade direction shortly before saccade onset (Marino & Mazer, 2018). Unexpectedly, in comparison with behavioral data from human subjects, a neural correlate of lingering of spatial visual attention has not been observed in V4. To better understand the underlying computational mechanisms of attentional updating in V4, we developed a neurocomputational model of perisaccadic visual attention in area V4 by incorporating perisaccadic spatio-temporal signals from the frontal eye field (FEF) and the lateral intraparietal area (LIP). When we test this model on the same task, our obtained data replicates the observation of predictive remapping of spatial attention pointers in V4. Further, our model provides an intuitive explanation for the lack of lingering attention in the neural recordings: As the attended location is only cued during instruction trials, endogenous attention is required, which might have faded away due to a continuous stream of visual input in each trial. By deactivating different pathways in the neurocomputational model, our model predicts that the attentional enhancement measured in V4 originates from spatial updating and not from a saccade target attention pointer.

Exploring Deep Learning Approaches for 3D Deformation: Toward Finite Element Method Distillation

Aida Farahani

Tue, 11. 11. 2025, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Thesis test presentation: The thesis introduces neural frameworks for modeling 3D deformations with the goal of enhancing the predictive accuracy and computational efficiency of traditional Finite Element Method (FEM) simulations. It explores two complementary approaches: a single-step deformation model that employs implicit neural representations and signed distance fields to approximate FEM-based deformations up to 400× faster than conventional simulations, and a multi-step deformation model that formulates deformation as a sequential decision-making process within a deep reinforcement learning framework. Supported by two custom datasets, DefBeam and DefCube, the study demonstrates that AI-driven methods can effectively complement FEM by accelerating simulations and improving accuracy in applications such as material design, virtual prototyping, and industrial forming.

Bridging Conflicting Views on Eye Position Signals: A Neurocomputational Approach to Perisaccadic Perception

Nikolai Stocks

Tue, 4. 11. 2025, A10.208.1 and https://webroom.hrz.tu-chemnitz.de/gl/mic-cv7-ptw

Saccades are an integral component of visual perception, yet the accuracy and role of eye position signals in the brain remain unclear. The classical model of perisaccadic perception posits that the dorsal visual system combines an imperfect eye position signal with visual input, leading to systematic perisaccadic mislocalizations under specific experimental conditions. However, neurophysiological studies of eye position information have produced seemingly conflicting results. One team of researchers observed the eye position signal directly in gain-field neurons in the lateral intraparietal area (LIP) and found them incompatible with the classical model. In contrast, another team reported evidence for an eye position signal consistent with the classical model, even showing that accurate eye position can be decoded from neural activity. We modeled two subpopulations of neurons in LIP receiving input from two different sources, one representing the corollary discharge containing predictive presaccadic signals, the other representing a slowly updating proprioceptive eye position signal. By decoding eye position from the neural activity of these subpopulations, we observed the model containing sufficient information to allow the decoder to accurately predict and track the perisaccadic eye position. Our findings reconcile the apparent contradiction between the different neurophysiological studies by providing a unified framework for understanding eye position signals in perisaccadic perception. Our results suggest that a combination of a late-updating proprioceptive signal and a predictive corollary discharge is sufficient for accurately decoding eye position.