Öffentliche Verteidigung im Rahmen des Promotionsverfahrens
Herr Dipl.-Inf. Robert Herms
Fakultät für Informatik
"Effective Speech Features for Cognitive Load Assessment: Classification and Regression"
17:00 Uhr, Straße der Nationen 62, Böttcher-Bau, 1/368a (neu: A10.368.1)
Speech contains a multitude of information and has been identified to be a potential modality to measure the user's cognitive load. The focus of this thesis is on the effectiveness of speech features for automatic cognitive load assessment, with particular attention being paid to new perspectives of this research area. A new cognitive load database, called CoLoSS, is introduced containing speech recordings of users who performed a learning task. The CoLoSS corpus, together with the CLSE database in which two variants of the Stroop test and a reading span task are employed, forms the basis for the evaluations. Various acoustic features from different categories including prosody, voice quality, and spectrum are investigated in terms of their relevance. Moreover, Teager energy parameters, which have proven highly successful in stress detection, are introduced for cognitive load assessment and it is demonstrated how automatic speech recognition technology can be used to extract potential indicators of the user's cognitive load. As a further contribution, three hand-crafted feature sets are proposed. The suitability of the extracted features is systematically evaluated by recognition experiments with speaker-independent systems designed for three-class classification (low, medium, and high cognitive load). Various configurations in terms of combinations of features, filters for feature selection, feature normalisation methods, and model parameters are tested. To prove the generalisation ability of the proposed feature sets, cross-corpus experiments are carried out. Additionally, a novel approach to speech-based cognitive load modelling is introduced, whereby the load is represented as a continuous quantity and its prediction can thus be regarded as a regression problem.