Abstract
Background:
Although social cognitive dysfunction is a major feature of the behavioral variant of frontotemporal dementia (bvFTD), quantitative measurement of social behavior changes is poorly available in clinical settings.
Objective:
The aim of the study is to evaluate diagnostic accuracy of social-emotional questionnaires in distinguishing bvFTD from healthy control (HC) subjects and Alzheimer’s disease (AD) patients.
Methods:
We enrolled 29 bvFTD, 24 AD, and 18 HC subjects matched for age, sex, and education. Two informant-based measures of socio-emotional sensitivity and empathy (i.e., revised Self-Monitoring Scale (rSMS) and Interpersonal Reactivity Index (IRI)) were administered. One-way ANOVA was performed to compare groups, whereas Receiver Operating Characteristics (ROC) curve analysis tested questionnaire ability in distinguishing groups. A short version of IRI (sIRI) was obtained by excluding the non-contributing subscale (i.e., personal distress).
Results:
Compared to HC and AD, bvFTD showed significantly lower scores in rSMS and IRI scores, except for IRI personal distress subscale. The sIRI showed an excellent performance in early diagnosis (bvFTD versus HC = AUC 0.95). Both sIRI and rSMS showed good performance in distinguishing bvFTD from AD (AUC 0.83).
Conclusions:
ROC analyses support the usefulness of informant social questionnaires in memory clinics and their potential value in screening procedures for research eligibility in forthcoming trials. In the timely diagnosis of bvFTD patients, IRI and rSMS may supply crucial information for the early detection of signs and symptoms affecting social-emotional skills, which might otherwise be underrecognized.
Keywords
INTRODUCTION
The investigation of social-cognition skills has gained crucial importance in clinical settings, especially for the early and differential diagnosis of neurocognitive disorders. A large body of evidence supports an early disruption of socio-emotional brain networks in patients with the behavioral variant of frontotemporal dementia (bvFTD), a complex neurocognitive disorder that primarily affects the social brain [1–3]. Although current diagnostic criteria [4] include a qualitative assessment of “loss of empathy and sympathy” and “inappropriate social behavior” among the six core features of the clinical diagnosis, they do not require objective quantification of socio-cognitive dysfunctions. Growing literature highlighted the limitations of these criteria published twelve years ago and their poor ability to early distinguish bvFTD from Alzheimer’s disease (AD) or primary psychiatric disorders [5, 6]. In parallel, different studies proved that the quantitative measurement of social-cognition skills is of paramount importance to early identify bvFTD in clinics [5, 8].
Although changes in social behavior and affective regulation have been shown to be predominant from the earliest stages of the disease [1, 9], the neuropsychological test batteries used to assess suspected bvFTD cases do not routinely include instruments that target social-cognition abilities [10, 11]. The quantification of social deficits is generally limited to a small number of academic memory clinics with a particular interest in this field [12, 13]. According to the most recent edition of the American Psychiatric Association’s Diagnostic and Statistical Manual for Mental Disorders (DSM-5) [14], social cognition should be considered one of six core components of neurocognitive functioning that should be quantified, alongside perceptual-motor function, language, learning and memory, complex attention, and executive functions. For this reason, there has been much debate among clinical researchers on the implementation of social measures in memory clinics, and large international initiatives have been launched to overcome current limitations in the field [12, 15].
Quantitative neuropsychological tools assessing social-cognition subdomains encompass both informant-based and patient-administered measures. The revised Self-Monitoring Scale (rSMS) [16] and the Interpersonal Reactivity Index (IRI) [17] represent very well-known and often-used informant-based questionnaires in experimental research. According to literature [18–23], these instruments are among the most widely used to obtain indirect information on empathy and socio-emotional sensitivity, due to their ease and speed of administration, as well as their good psychometric characteristics, including internal consistency, retest reliability, and construct validity [24–29], even though they are now considered dated instruments. Moreover, such instruments proved useful in the diagnostic framework of bvFTD and in disease monitoring [30, 31]. However, objective patient-administered social cognitive tasks have the advantage of providing information on those individuals in very early disease stages or at-risk (e.g., carriers of known causative mutations), who usually refer alone to the specialist and for whom caregiver support is not required. Limited clinical availability and heterogeneous, non-systematic use of socio-cognitive measures in clinical research result in poor comparability of diagnostic performances among studies [32]. Choosing the right task to assess social cognition in neurocognitive patients is still a critical point, as no clinical recommendations are available at present.
Among all social-cognition facets, empathy and social self-monitoring are core subdomains early disrupted by frontotemporal lobar degeneration [33, 34]. A theoretical model [35–38] splits empathic skills, recognizing two components: cognitive and affective empathy. Cognitive empathy is partly reliant on the mentalizing network and on crucially engaging the prefrontal cortex, whereas the affective component of empathy is likely mediated by the activation of limbic and temporal lobe structures [30, 39]. In the study of Hsieh et al. [40], the empathic loss reported in bvFTD patients was significantly associated with a less-caring relationship with the caregiver. The striking inability of bvFTD patients to emotionally connect and understand the point of view of their caregiver is, unsurprisingly, associated with a poorer quality of the relationship in this patient-caregiver dyad compared to an AD cohort [40]. bvFTD is also characterized by alterations in self-monitoring due to the progressive degeneration in fronto-median networks, right anterior insula, and medial thalamus, which may lead to changes in personality [33]. Indeed, as reported from the caregiver’s perspective, it has been described as a progressive decline in self-monitoring ability as the disease advances from asymptomatic to severe phases [31]. Up to this point, informant-based questionnaires indirectly investigating empathy and the ability to modulate one’s behavior according to the demands of the social context (e.g., IRI and rSMS) are the most commonly used measures in clinical research for the assessment of such cognitive skills. Although these measures were originally designed as patient-administered, informant-based versions are considered to be more reliable in the assessment of neurocognitive disorders due to potential lack of insight by these patients [18, 41].
This study is part of a line of research that aims to provide evidence of the clinical maturity of social cognitive measures. The primary objective of this work is to evaluate the clinical performances of the above-mentioned quantitative measures (i.e., IRI and rSMS) in distinguishing bvFTD from healthy control (HC) subjects and AD patients. We thus aim to assess specificity, sensitivity, and diagnostic accuracy of the two social measures, assuming optimal discriminatory performance for both questionnaires.
MATERIALS AND METHODS
Participants
A total of 71 participants were included in the present study, sub-grouped into 29 probable bvFTD, 24 prodromal or mild AD, and 18 cognitively HC subjects matched for age, education, and sex. Patients were consecutively enrolled at the Department of Clinical Neurosciences, San Raffaele Hospital (Milan, Italy) between 2016 and 2018. We included only patients in early disease stages (i.e., Clinical Dementia Rating scale (CDR) global score≤1) and with a confirmed clinical diagnosis according to current diagnostic criteria [4, 43]. Expert clinicians blind to the performance on social questionnaires performed the clinical diagnosis. Standard neurological examination, full neuropsychological testing battery, and neuroimaging assessment (i.e., MRI and/or FDG-PET) provided support for the diagnosis. Since no biological marker of FTD was available in our clinical research settings, we considered a 2-year mean follow-up as reference standard for confirmation of the initial classification.
The HC sample included subjects recruited at local community centers. We included only subjects with negative history of neuropsychiatric disorders, negative neurological examination, taking no medication interfering with neurobehavioral functioning, with CDR global score = 0 and Mini-Mental State Examination (MMSE) raw score > 28 (if≥9 years of education) or > 27 (if≤8 years of education). A close informant (e.g., spouse) was interviewed to confirm the subject’s normal daily functioning.
All patients or informants/caregivers, and each HC, gave informed consent to the experimental procedure that had been approved by the local ethical committee. See Table 1 for the demographic and clinical features of the sample.
Demographic and clinical features of the sample
bvFTD, behavioral variant of frontotemporal dementia; AD, Alzheimer’s disease; HC, healthy control subjects; CDR-GS, Clinical Dementia Rating scale –global score; CDR SoB, Clinical Dementia Rating scale –sum of boxes; FTLD-modified CDR SoB, Clinical Dementia Rating scale modified for frontotemporal lobar degeneration –sum of boxes; MMSE, Mini-Mental State Examination; *p < 0.05; **p < 0.01; ***p < 0.001.
Socio-emotional questionnaires
The socio-emotional assessment, which lasted approximately 15 min, was carried out using the IRI [17] and rSMS [16] questionnaires administered to the informant (i.e., the caregiver of bvFTD and AD patients or a close family member of HC subjects) with support from the examiner. These measures investigate, respectively, cognitive and affective empathic skills and the ability to adapt one’s behavior to the social context.
The IRI is a questionnaire comprising 28 items included in 4 subscales: Perspective Taking (PT, the ability to spontaneously imagine the cognitive perspectives of others) and Fantasy (FS, the ability to project oneself in fictional contexts, e.g., books or movies) for the assessment of cognitive empathy; Empathic Concern (EC, the other-centered emotional responses elicited by the perception of the emotional states of others) and Personal Distress (PD, the self-centered emotional responses due to stressful situations or negative emotional states of others) for the assessment of affective empathy [17, 27]. Each item is scored according to a 5-point Likert scale. The higher the empathic competence, the higher the score recorded on IRI-PT, IRI-FS, and IRI-EC subscales. In contrast, in the IRI-PD subscale, higher scores correspond to higher personal distress, which usually preclude pro-social behavior [27].
The rSMS is a 13-item questionnaire developed by Lennox and Wolfe [16], divided into two subscales: the first (i.e., rSMS-Ability –rSMS-A) explores the ability to modify one’s social behavior, while the second (i.e., rSMS-Sensitivity –rSMS-S) investigates individual sensitivity to the expressive behavior of others. Each item is scored on a 5-point Likert scale with some reverse scores [31]. Overall, a higher score obtained from the sum of the individual items not only reflects stronger self-monitoring skills but also indicates a greater sensitivity to the expressive behavior of others.
For each patient, the total scores of the two scales (IRI global score or IRI-GS and rSMS global score or rSMS-GS) were calculated, as well as the scores of the individual subscales: IRI-PT, IRI-F, IRI-EC, and IRI-PD; rSMS-A and rSMS-S. Moreover, according to statistical comparison results (see Results section), we computed a short version of the IRI (i.e., short IRI –sIRI), by including the sum of the scores of only the IRI-PT, IRI-F, and IRI-EC subscales.
Statistical analyses
Statistical analysis was performed using SPSS software (https://www.spss.it/). Statistical significance was established at p < 0.05 for all statistical tests. Descriptive statistics were calculated, including frequency and percentage for categorical variables, mean and standard deviation for pseudo-continuous variables and median and interquartile range for those variables that were not normally distributed. Comparisons between groups regarding demographics, clinical characteristics, and measures of social cognition were performed by chi-square test and ANOVA tests using Bonferroni correction for post-hoc comparisons. Due to the non-normal data distribution, a non-parametric test (Mann-Whitney U test) was used to compare clinical features between the two patient groups. The effect size for the ANOVA test was eta-squared, which ranges from 0 to 1. Commonly encountered interpretation values in published literature are: 0.01–0.06 (small effect), 0.06–0.14 (moderate effect), and > = 0.14 (large effect) [44].
ROC (receiving operating curve) analysis was used to assess the ability of the socio-emotional questionnaires collected by the informant to distinguish bvFTD patients from HC subjects and AD patients. The analysis was performed considering both global scores (IRI-GS and rSMS-GS), the subscale scores (IRI-PT, IRI-F, IRI-EC, IRI-PD, rSMS-A, and rSMS-S) and the short version of the IRI questionnaire (sIRI). AUC values of IRI-GS and sIRI were compared using DeLong’s test [45]. For those measures that showed significant discriminative effect (p < 0.05), the cut-off score was derived from the Youden index (Sensitivity+Specificity –1).
RESULTS
Clinical and demographic profile of the sample
As expected, no significant differences were found in the demographic variables (age, sex, and education) among the groups. Nor were significant differences found for disease duration and CDR global scores, CDR sum of boxes (CDR SOB) and CDR modified for frontotemporal lobar degeneration–sum of boxes (FTLD-modified CDR SOB) between patient groups. Both patient groups showed reduced MMSE scores compared to HC (F(2.71) = 31.4, p < 0.001), with AD patients showing significantly lower scores than bvFTD (p = 0.003). For details on comparisons among groups regarding demographic and clinical features, see Table 1.
Socio-emotional profiles
Comparisons of the socio-emotional performance among the three groups revealed significant differences in IRI and rSMS global and subscale measures, except for the IRI-PD, which did not show any significant difference among groups (see Figs. 1 and 2). In details, post hoc analyses showed significantly lower scores in bvFTD compared to HC on IRI-GS (p < 0.001), IRI-PT (p < 0.001), IRI-F (p < 0.001), and IRI-EC (p = 0.003), as well as on rSMS-GS (p < 0.001) and rSMS-S (p < 0.001). No significant effect was found for IRI-PD. Compared to AD, patients with bvFTD showed the same profile of significantly reduced scores in all measures: IRI-GS (p < 0.001), IRI-PT (p < 0.001), IRI-F (p = 0.02), IRI-EC (p = 0.008), rSMS-GS (p < 0.001), and both rSMS-A (p = 0.007) and rSMS-S (p < 0.001). In all measures considered, post-hoc analyses found no significant differences between HC and AD.

Comparison between groups at IRI Global Score (A), Fantasy (B), Empathic Concern (C), Perspective Taking (D), and Personal Distress (E) subscales; *p < 0.05; **p < 0.01; ***p < 0.001.

Comparison between groups at rSMS Global Score (A), Ability (B) and Sensitivity (C) subscales; *p < 0.05; **p < 0.01; ***p < 0.001.
Given the results of IRI comparisons, a short version of the questionnaire (short IRI, sIRI) was obtained by retaining the most-contributing subscales found to be significantly different in bvFTD than in HC and AD, i.e., IRI-F, IRI-EC, and IRI-PT.
See Table 2 for details on group comparisons in socio-emotional questionnaires.
Socio-emotional informant questionnaires performances among groups
bvFTD, behavioral variant of frontotemporal dementia; AD, Alzheimer’s disease; HC, healthy control subjects; MMSE, Mini-Mental State Examination; GS, global score; IRI, interpersonal reactivity index; F, fantasy; EC, empathic concern; PT, perspective taking; PD, personal distress; rSMS, revisited Self-Monitoring Scale; A, ability; S, sensitivity; *p < 0.05; **p < 0.01; ***p < 0.001.
Diagnostic accuracy profile of socio-emotional measures
The ROC analysis showed good discriminating performance in bvFTD versus HC for both IRI and rSMS global scores (IRI-GS: Area Under the Curve (AUC) = 0.89, p < 0.001, Standard Error (SE) = 0.04, Confidence Interval (CI) 95% = 0.81–0.98, accuracy = 82%; rSMS-GS: AUC = 0.83, p < 0.001, SE = 0.06, CI 95% = 0.72–0.95, accuracy = 77%). Considering subscale scores, sIRI also showed excellent performance (AUC = 0.95, p < 0.001, SE = 0.02, CI 95% = 0.90–1, accuracy 91%), followed by IRI-PT (AUC = 0.95, p < 0.001, SE = 0.02, CI 95% = 0.90–1, accuracy = 88%) and rSMS-S (AUC = 0.90, p < 0.001, SE = 0.04, CI 95% = 0.81–0.99, accuracy = 84%).
Comparably, ROC analyses on patient groups showed that global scores of both IRI and rSMS significantly distinguish bvFTD from AD (IRI-GS: AUC = 0.81, p < 0.001, SE = 0.06, CI 95% = 0.70–0.93, accuracy = 78%; rSMS-GS: AUC = 0.83, p < 0.001, SE = 0.06, CI 95% = 0.71–0.95, accuracy = 81%). Even here, sIRI showed good performance (AUC = 0.83, p < 0.001, SE = 0.05, CI 95% = 0.72–0.94, accuracy = 80%) followed by rSMS-S (AUC = 0.86, p < 0.001, SE = 0.05, CI 95% = 0.75–0.97, accuracy = 82%) and IRI-PT (AUC = 0.85, p < 0.001, SE = 0.05, CI 95% = 0.74–0.95, accuracy = 78%). The rSMS-A subscale had the poorest performance in distinguishing bvFTD from AD (AUC = 0.73, p = 0.003, SE = 0.07, CI 95% = 0.59–0.87, accuracy = 69%).
Finally, consistent with the one-way ANOVA comparison analyses, IRI-PD failed to significantly differentiate bvFTD from HC and AD.
For details on the ROC curves, see Table 3 and Fig. 3.
Coordinates for ROC curve in bvFTD versus HC and AD
bvFTD, behavioral variant of frontotemporal dementia; AD, Alzheimer’s disease; GS, Global Score; IRI, Interpersonal Reactivity Index; F, fantasy; EC, empathic concern; PT, perspective taking; PD, personal distress; rSMS, revisited Self-Monitoring Scale; A, ability; S, sensitivity; sIRI, short Interpersonal Reactivity Index.

Receiving Operating Characteristic curve for the short Interpersonal Reactivity Index (sIRI), the Interpersonal Reactivity Index global score (IRI-GS) and the revised Self-Monitoring Scale global score (rSMS-GS) in the behavioral variant of frontotemporal dementia versus healthy controls (A) and Alzheimer’s disease (B) groups.
Additionally, DeLong’s test proved no significant difference between AUC values of IRI-GS and sIRI, both for bvFTD versus HC and bvFTD versus AD, thus having similar clinical performances (Table 4).
DeLong’s test on pairwise comparison of ROC curves
DISCUSSION
Despite the huge effort made by the International FTD Consortium in drawing up more stringent criteria for the diagnosis of bvFTD [4], the in vivo early and differential diagnosis of this syndrome is still a challenge. The lack of specific biological markers for frontotemporal dementia represents a main issue for clinicians. In the timely diagnosis of bvFTD, the role of close family members and informants is of utmost relevance for the early detection of signs and symptoms affecting social-emotional skills, which might otherwise be underrecognized. People who live daily alongside bvFTD patients and who see day-to-day changes in their characters or personalities, therefore, represent an invaluable source of information about suspect socio-emotional deficits and for monitoring changes over time. Nonetheless, qualitative reports of altered social behaviors or lack of adequate empathic responses are not sufficiently accurate to capture subtle changes nor to classify multifaced social-cognition deficits. The use of quantitative social-cognition measures adds relevant information to reach a timely diagnosis. This latter is of particular relevance for the patient-caregiver dyad in preventing the dramatic impact that the disease has on the quality of life of the entire family unit [46].
The clinical validity of socio-emotional measures in bvFTD diagnosis has been claimed for a few patient-administered tasks [7, 47–51] and informant-based questionnaires [31, 40], particularly in terms of discriminating bvFTD from HC. The poor availability of socio-emotional measures validated and standardized for use in clinical settings prevents a full evaluation of the clinical performances of such measures in real-life scenarios. In this study, we provided information on the clinical performances of IRI and rSMS questionnaires, finding excellent diagnostic accuracy for both socio-emotional measures and suggesting that these measures are a valuable aid in the discrimination of bvFTD from both HC and AD. The investigation of social-emotional abilities in bvFTD using informant-based questionnaires not only makes it possible to objectify, albeit indirectly, the presence of social-cognition deficits in the patient, but also guarantees a preliminary qualitative exploration of their impact on caregiver quality of life. Thus, it can represent the first step in further exploring burden and psychological wellbeing in caregivers via validated instruments (e.g., Zarit Burden Interview [52, 53], Burden Scale for Family Caregivers [54] and the Neuropsychiatric Inventory Questionnaire [55]). Caregivers of bvFTD patients experience higher levels of distress than AD caregivers [46, 57] and need long-term psychoeducational support [58].
In line with other studies on social-cognition measures [7, 60], we provided higher sensitivity than specificity of social questionnaires according to the cut-off score derived from the Youden index analysis. This claims the additional role of other neuropsychological measures of improving the diagnostic ability of cognitive testing. The use of multiple cognitive measures has been widely recommended for the differential diagnosis of bvFTD [41, 62]. Combination of multiple social-cognition tools as that reported in the Social cognition and Emotional Assessment –SEA battery [63], including theory of mind, emotion recognition, reversal/extinction skills, behavioral control ability, and apathy, may improve the diagnostic performance of cognitive markers. In two studies [48, 59], the short version of emotion recognition and theory-of-mind tasks (i.e., the Mini-SEA) accurately classified bvFTD from AD patients in 82.5% and 87.9% of cases. Additionally, as reported in a clinical study from our group [61], the combined use of social and non-social cognitive tasks, together with behavioral scales, can correctly classify 85% of bvFTD and 90% of AD cases.
According to current literature [30, 64], our findings highlight a preserved pattern of personal distress. The IRI-PD subscale consists of items that measure feelings such as fear, apprehension, and discomfort that are experienced when witnessing the negative experiences of others. Theoretically, this subscale reflects a primitive form of empathy that interferes with an effective empathic response. It tends to decrease when the other empathic subscales increase (particularly the IRI-PT subscale) and is negatively related to measures of overall social functioning [30]. Indeed, as shown by different experimental and structural MRI studies, other empathic dimensions (e.g., perspective taking or empathic concern) are affected earlier in bvFTD [40, 64–66]. For this reason, the IRI-PD subscale might not be informative for a bvFTD diagnosis. The sIRI questionnaire shows comparable high accuracy levels with respect to the long version (see Table 4). Nonetheless, the sIRI could better capture useful information for a timely diagnosis of socio-cognitive changes in neurocognitive disorders being faster and easy-to-administer in memory clinic settings. It is worth mentioning that the IRI scale was not developed specifically for the population of interest, and new measures are emerging from literature as possible alternatives. However, given the evidence in the literature on the validity of this scale in this specific patient population, we chose to assess the clinical performance of this measure in support of clinical research findings.
The rSMS questionnaire is a valid measure in bvFTD diagnosis [31, 67–69], since it reflects alterations in the Salience Network (SN) functional connectivity [67], a key network supporting socio-emotional functioning and mediating sensitivity and responsiveness to subtle nonverbal social cues during face-to-face interactions [70]. The cortical nodes of the SN include the insula and the anterior cingulate cortex, fundamental brain structures for predicting socio-emotional sensitivity and human emotional awareness [71, 72]. Notably, we proved better diagnostic performance for the sensitivity subscale (rSMS-S) compared to the ability subscale (rSMS-A), which appears to be the most informative measure in early detecting SN alterations.
In conclusion, although the limits of the study are mainly related to the small sample size and the lack of pathological confirmation to support the initial classification, both socio-emotional informant-based questionnaires should be recommended for use in clinical settings. In addition, although the little predictive utility of the personal distress subscale in the differential diagnosis of neurocognitive disorders (e.g., [30, 64]), the exclusion of PD subscale from the IRI questionnaire is to acknowledge as a potential limitation due to construct validity threat.
Empathy skills and the ability to modulate one’s behavior according to the demands of social context are core cognitive aspects early affected in bvFTD [34]. Given the absence, to the best of our knowledge, of validated tests capable of obtaining an objective measure of empathic skills and behavioral adaptability, IRI and rSMS offer at present unique sources of clinical information. The use of these measures would benefit from the introduction of normative data for the reference population and of validated language-adapted versions, which should be promoted. Upcoming research should also focus on testing clinical performances of socio-emotional questionnaires in the whole frontotemporal lobar degeneration spectrum comparing the behavioral variant with other variants, as well as on investigating performances of individuals with the behavioral presentation of AD and with psychiatric disorders in comparison to those of bvFTD patients. This latter is of particular relevance in view of current evidence [73, 74] and in support of better inclusion of socio-cognitive neuropsychological instruments in the diagnostic framework of bvFTD. On the other hand, future perspectives should also consider the adoption of more ecological/behavioral tasks, (e.g., [75]), assessing empathy in patients using real-life scenarios, as already applied in other neurocognitive populations (e.g., [76]).
Footnotes
ACKNOWLEDGMENTS
We acknowledge the Italian Association for Frontotemporal Dementia (AIMFT) which has always encouraged and supported our work. We also thank patients and caregivers for their help in fostering our research.
FUNDING
The authors have no funding to report.
CONFLICT OF INTEREST
Chiara Cerami is an Editorial Board Member of this journal but was not involved in the peer-review process nor had access to any information regarding its peer review.
DATA AVAILABILITY
The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restriction.
