Johannes Tröger, Elisa Mallick, Nicklas Linz, Inez Ramakers, Silke Kern, Ingmar Skoog, Gonzalo Sánchez Benavides, Oriol Grau, Juan Domingo Gispert, Stefanie Köhler, Stefan Teipel & Alexandra König
* Poster presented at the 17th Clinical Trials on Alzheimer’s Disease (CTAD), Madrid (Spain)
Abstract
Background: Clinical trials for Alzheimer’s disease are large-scale, multinational, and involve numerous sites across different countries. While these trials adhere to overarching inclusion and quality criteria, population characteristics and procedures inevitably vary by location. Therefore, cognitive measures used in these trials must be validated for suitability across diverse study populations and regions. This ensures their effectiveness as reliable endpoints, pre-selection criteria, or screening tools. Hence, we present data from an automatic digital cognitive assessment the ki: digital speech biomarker for cognition (SB-C) from four different clinical cohorts across four different European countries. The aim of this study is to derive and validate one global cut-off for differentiating between cognitively impaired and cognitively healthy subjects and report the respective performance.
Methods: For this research we used a sample from four different memory clinic populations: the Swedish H70 Birth Cohort study (N=696), the Dutch DeepSpA study (N=134), the German PROSPECT-AD DESCRIBE-AD study (N=96). Additionally, we used a sample of participants with Subjective Cognitive Impairment (SCI): the Spanish PROSPECT-AD Beta-AARC study (N=59). Clinical labels (SCI, MCI, Dementia) of all participants were provided from each cohort site and based on clinical diagnostic routine procedure. We automatically extract the SB-C score and its subscores (executive function, memory, semantic memory, processing speed) from SVF and RAVLT speech recordings using ki:elements’ proprietary speech analysis pipeline including automatic speech recognition and feature extraction. We performed (1) inferential statistics comparing cognitively healthy (Healthy Controls or SCI) and cognitively impaired (Mild Cognitive Impairment (MCI) or mild dementia) groups based on the SB-C Cognition Score in each cohort of the sample and (2) determined a global cutoff to differentiate between cognitively healthy and cognitively impaired groups on the whole sample and apply it in the four different cohorts separately. For (1) we performed a non-parametric Kruskal-Wallis test to compare SB-C scores of both cognitively healthy and cognitively impaired groups to check for general feasibility. For (2), we applied an optimal cutoff procedure aimed at maximizing the sum of sensitivity and specificity in discriminating between cognitively healthy and cognitively impaired groups.
Results: The Kruskal-Wallis test revealed a significant difference of the SB-C score between the diagnosis groups in each cohort (Cognitively Healthy > Cognitively Impaired; see Table 2). The optimal cutoff analysis showed that a cutoff of 0.42 in the SB-C Cognition Score differentiated between diagnosis groups with a balanced accuracy and a ROC AUC score of 0.77 (Figure 1). Sensitivity and specificity of the classification were 0.84 and 0.71 respectively. Applied in the four cohorts separately, the cutoff of 0.42 results in balanced accuracies of 0.76 or 0.77, sensitivities greater than 0.61 and specificities greater than 0.68 (Figure 2 and Table 3), and most of the patients of PROSPECT-AD Barcelona Beta study are classified as cognitively unimpaired (Figure 2).
Conclusion: The SB-C reaches classification performance of .75 or higher across all four cohorts for separating cognitively healthy from cognitively impaired subjects. Despite multilingual cohorts (Dutch, Swedish, German & Spanish) in different research contexts i.e. memory clinic based inclusion, longitudinal register cohort or pre-clinical research focused) one global cut-off demonstrated good and more important robust performance. This serves as important evidence to demonstrate the fit-for-purpose of such a digital automatic cognitive speech measure in the area of clinical AD clinical trials.
