Certainty-weighted recognition memory: Potential applications for early detection and metacognition

Abstract

Background

Memory loss is a core feature of typical Alzheimer's disease (AD) and amnestic mild cognitive impairment (aMCI). Standard memory tests such as word lists assess verbal episodic memory with delayed recall and recognition. However, actual memory fidelity is likely variable, continuous, and has a subjective component.

Objective

We investigated dual-processing models of episodic memory (recollection versus familiarity) using confidence ratings in a “judgment of knowing” paradigm (JOK).

Methods

This paradigm was applied to the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) memory test as part of neuropsychological evaluation at University of Pittsburgh Alzheimer's Disease Research Center (ADRC), to generate novel indices of memory function to improve sensitivity to early memory problems and provide a memory awareness metric. On recognition testing, participants rated how sure they were of their yes/no responses to each item. We derived novel variables related to memory and metacognition, including an Accuracy-Certainty Index and the Relative Certainty Index.

Results

In this sample of 347 participants (185 with AD, 55 with MCI, 111 cognitively unimpaired), CERAD Delayed Recall was the best single variable for discriminating groups, although multiple certainty variables also discriminated groups well.

Conclusions

The addition of certainty indices to a standard verbal memory task increased discriminative power between groups, particularly between cognitively normal controls and MCI or AD.

Keywords

Alzheimer's disease judgment of knowing sensitivity/specificity tests/measurement

Introduction

Many neurologic conditions have some degree of impaired awareness of neurologic deficits, which has been termed lack of insight, unawareness of deficit, or anosognosia. Anosognosia is seen frequently in patients with dementia, which has a significant impact on both the person, their family, and their care.¹ Up to 80% of patients with Alzheimer's disease (AD) dementia have some degree of anosognosia² and numerous studies have found this deficit to be even more severe in behavioral variant frontotemporal dementia (bvFTD).^3–5

Anosognosia has also been described in patients with mild cognitive impairment (MCI),⁶ although results are mixed, with some studies failing to find an association.⁷ The mechanism of anosognosia in MCI, like other conditions, can be varied. Some studies have found an association with learning and memory⁸ while others have shown associations with behavioral symptoms⁹ and deficits in self-appraisal.¹⁰ Mechanistically, it is both possible and likely that lack of awareness for a deficit could arise from either memory-related or memory-independent processes. Every major cognitive framework^2,11,12 includes both memory and executive processes, but beyond that, theories may disagree. Accordingly, when researchers¹³ performed a systematic review of structural and functional investigations of the basis of anosognosia they found a predictable degree of heterogeneity. Specifically, some studies have found associations with frontal regions, others with temporo-parietal, and others found areas within both regions that were associated with anosognosia in patients with dementia.^14,15

Despite being important for practical purposes, anosognosia measures are less commonly included in standard neuropsychological batteries. Anosognosia is associated with higher levels of caregiver burden,^16–18 poor medication adherence,¹⁹ impaired financial abilities^20,21 and scam susceptibility,²² and worse ability to predict one's own competency at a driving simulation task.²³

Anosognosia has historically been studied as a distinct concept from metacognition, although the two are clearly related and should likely be conceptualized as interconnected.²⁴ Metacognition, the process by which we monitor and update our self-representation of various cognitive abilities, is an important and well-studied concept within cognitive psychology.²⁵ Similar paradigms have been used to study the self-awareness of neurologic deficits in a number of neurologic disorders.^11,12,26 Due to the complex nature of self-awareness, it is likely that different assessment methodologies tap into different metacognitive processes. Some of these methodologies include feeling of knowing²⁷ (FOK; participants predict their future ability to recall learned information) and retrospective confidence ratings²⁸ (CONF; participants rate their confidence in the accuracy of a previously made recognition decision). When FOK and CONF have been evaluated, they show activation in different regions of the brain, further supporting the idea that while both are measurements of memory monitoring, they are the result of different cognitive processes.²⁹ The same researchers found that retrospective metamemory judgments have been studied extensively in cognitively unimpaired study participants and have been found to rely more on strength of the memory itself, ease of retrieval, and cue-related factors. While it may seem intuitive that memory-dependent processes would be more discriminative in disorders that primarily involve memory failure, the literature on task-specific metamemory judgements in AD is mixed. Some studies have demonstrated that patients with AD have impaired global metacognitive abilities but intact task-specific metacognitive abilities when compared to age-matched controls,^30,31 but others have demonstrated clear deficits in task-related metamemory judgments.^32,33 Given the compelling clinical and functional implications of poor metacognition, and this mixed literature on task-specific metacognition, clarification of these mixed results is needed.

Many studies of metamemory in both normal and impaired individuals use long or difficult tasks that are designed to differentiate between related cognitive components of a complex task such as metamemory judgment. However, to introduce the concept of metamemory assessment into routine clinical practice, it would be best if the task was familiar to clinicians, required minimal addition of time to the clinical encounter, and was useful in detecting neurodegeneration. Thus, we sought to create a simple addition to a standard verbal learning task (the CERAD word memory list)³⁴ that could increase discriminative power among cognitively unimpaired controls, those with MCI, and those with AD dementia. In our modified CERAD recognition task, we asked participants to indicate whether they had encountered the word previously and how certain they were of the accuracy of their answer. We use the term ‘certainty’ in place of confidence to highlight the pragmatic clinical re-purposing of the retrospective confidence rating strategy frequently employed in metamemory tasks. This approach brings metamemory assessment into the clinical encounter and helps to increase the discriminative power of standard memory tasks. Thus, our aim was to create a standard verbal learning task that could increase discriminative power among cognitively unimpaired controls, those with MCI, and those with AD dementia.

Methods

Participants/collection

Study design for data acquisition

All data collection was done in compliance with guidelines on human participants research and approved by University of North Carolina and University of Pittsburgh institutional review boards, and a data use agreement was approved. The certainty-weighted recognition paradigm was implemented on the CERAD word-list memory test as part of the standard neuropsychological evaluation at University of Pittsburgh Alzheimer's Disease Research Center (ADRC). On delayed recognition testing, participants rated how sure they were of each of their yes/no responses to the 20-item recognition list (10 targets, 10 foils) on a 4-point scale: Very sure (4), More than 50% sure (3), Less than 50% sure (2) and Not sure at all (1). Four points were used rather than a dichotomous “certain versus uncertain” decision in order to extend the scale for weighting answers. These data were used to derive several novel variables related to recognition memory and metacognition. These included the Accuracy-Certainty Index (ACI), (TCC), Total Incorrect Certainty (TIC), Mean Correct Certainty (MCC), and Mean Incorrect Certainty (MIC). Additionally, a putative metacognitive metric for AD dementia, the Relative Certainty Index (RCI), was generated by dividing the MIC score by the MCC score. Terms and definitions are provided in Table 1. Gathering certainty ratings from participants added an average of 4.15 min to administration time for the CERAD, and an average of 3.24 min scoring time per administration.

Table 1.

List of certainty-weighted memory variables with their calculations.

Total Correct Certainty (TCC)	Sum of certainty ratings (range 0–4 each) for correct responses
Total Incorrect Certainty (TIC)	Sum of certainty ratings for incorrect responses
Accuracy-Certainty Index (ACI)	TCC - TIC
Mean Correct Certainty (MCC)	Average certainty rating for participant's correct responses
Mean Incorrect Certainty (MIC)	Average certainty rating for participant's incorrect responses
Relative Certainty Index (RCI)	MIC/MCC

The test was administered to consecutive participants between February 2002 and April 2004: 111 normal controls (NC), 51 MCI, 185 patients with probable AD dementia per the original McKhann criteria.³⁵ These criteria were applied for each participant at the time of the study. Of note, five participants in the NC group had Clinical Dementia Ratings of 0.5 but were still considered unimpaired and kept in the NC group.

As part of the assessment battery, clinicians working with the participants completed the Hamilton Depression Scale³⁶ after a semi-structured clinical interview with the participant. This scale shows strong reliability and validity for use with older adults.³⁷ Study partners (e.g., caregivers, close family members) also completed the Blessed Rating Scale,³⁸ a scale of 22-items which can be divided into three sections: A) changes in performance of everyday activities; B) changes in habits including self-care; and C) changes in personality, interest, and drives. This scale also shows strong reliability and validity for use with older adults.³⁸ Unfortunately, we do not have records as to the informant's relationship to the study participants in each case.

Statistical methods

Data were analyzed using SPSS version 23. Assumptions of all tests were evaluated and considered met (for example, on our main ANCOVA we evaluated homogeneity of slopes, normality of residuals, variable co-linearity, homogeneity of variable, and normal distribution of variables, along with assessment of outliers). We assessed differences in demographic and other factors among diagnostic groups of AD, MCI, and NC using one-way between-subjects analysis of variance (ANOVA) for continuous variables and Chi-square tests for categorical variables. Pearson correlations were used to evaluate relationships between these variables and outcome variables.

Analysis of covariance (ANCOVA) compared outcome measures between groups (MCI, AD dementia, NC), adjusting for age and education level. Effect sizes were evaluated using partial ή². For significant results, pairwise group comparisons were made with Tukey's Test.

To compare accuracy alone (Total Correct Recognition) directly against accuracy combined with certainty (ACI), logistic regressions and ROC curves examined how the two predictors predicted group members pairwise (NC versus MCI, NC versus AD, MCI versus AD).

Pearson correlations examined potential relationships between Hamilton Depression Scale (Ham-D) scores with recognition certainty outcomes, to explore any potential relationship and establish a type of discriminant validity, verifying that certainty recognition is something separate from confidence related to depression. Correlations also assessed for relationships between Blessed Dementia Rating Scale subscale scores and certainty outcomes, to establish a convergent and ecological validity for certainty outcomes.

Results

Preliminary analyses

The clinical groups (AD dementia, MCI, and NC) differed on multiple demographic and other variables (Table 2). Pearson correlations determined that of those demographic variables, only age and education showed significant (p < 0.05) relationships with CERAD Word List variables (which included nearly every variable, including learning trial scores, accuracy scores for recognition, and certainty ratings).

Table 2.

Demographics and other characteristics in sample by cognitive diagnosis.

			NC (N = 111)				MCI (N = 51)				AD dementia (N = 185)
			N	Mean (%)	sd	Range	n	Mean (%)	SD	Range	n	Mean (%)	SD	Range	p
Demographics
	Age		110	660.9	90.2	36–87	51	690.9	90.3	51–88	185	760.2	60.8	48–90	<00.001
	Age at Onset						40	650.3	100.0	48–87	164	720.7	70.5	44–87	<00.001
	Years of Education		110	150.8	30.1	8–24	8–24	150.0	30.3	8–20	110	130.3	30.3	8–25	<00.001
	Sex														<00.001
		Female	71	640.0%			17	330.3%			123	660.5%
		Male	39	350.1%			34	660.7%			62	330.5%
	Race														00.008
		White	91	820.0%			42	820.4%			176	950.1%
		African American	18	160.2%			8	150.7%			9	40.9%
		Asian American	1	00.9%			1	20.0%			0	00.0%
Other Characteristics
	# With APOE4 allele		30	280.8			27	560.3			111	600.3			<00.001
	# With CDR >=00.5		5	50.0			40	880.9			169	990.4			< 00.001
	MMSE scores		111	280.4	10.6	22–30	51	260.8	20.6	21–30	183	190.7	40.1	9–29	<00.001
	Hamilton Depression score		10	30.9	30.9	0–12	40	40.9	40.9	0–24	168	40.8	40.0	0–20	>00.05

Min- Mental State Exam (Folstein, 1975).

ANCOVAs

Results of ANCOVAs comparing various CERAD Word List outcomes between clinical groups are provided in Table 3.

Table 3.

CERAD word list learning task performance by group.

	AD dementia (N = 185)		MCI (N = 51)		NC (N = 111)		ANCOVA		Post-Hoc (Tukey's) ps
	Mean	SD	Mean	SD	Mean	SD	p	ES	NC/MCI	MCI/AD	NC/AD
Trial 1 Recall	20.68	10.34	30.90	10.56	50.63	10.72	<00.001	00.32	<00.001	<00.001	<00.001
Trial 2 Recall	30.90	10.63	60.12	10.77	70.95	10.35	<00.001	00.46	<00.001	<00.001	<00.001
Trial 3 Recall	40.54	10.72	60.98	10.73	80.87	10.16	<00.001	00.51	<00.001	<00.001	<00.001
% Savings	80.38	180.02	420.81	270.91	770.44	190.99	<00.001	00.60	<00.001	<00.001	<00.001
Delayed List Recall	00.46	10.06	30.06	20.40	60.95	20.24	<00.001	00.65	<00.001	<00.001	<00.001
Total Correct Recog	130.70	20.85	170.47	20.43	190.35	10.16	<00.001	00.44	<00.001	<00.001	<00.001
Correct Targets Recog	50.45	30.06	80.35	10.83	90.41	10.07	<00.001	00.26	00.027	<00.001	<00.001
Correct Foils Recog	80.25	20.14	90.12	10.85	90.94	00.34	<00.001	00.14	00.021	<00.001	00.009
Correct Target Certainty	180.11	110.23	300.55	80.16	360.90	40.70	<00.001	00.36	<00.001	<00.001	<00.001
Correct Foil Certainty	260.15	100.93	330.08	90.38	390.18	20.37	<00.001	00.25	<00.001	<00.001	<00.001
Total Certainty	620.34	130.10	700.65	100.03	780.14	30.00	<00.001	00.24	<00.001	<00.001	<00.001
Total Correct Certainty	440.26	140.39	630.63	130.60	760.08	50.86	<00.001	00.49	<00.001	<00.001	<00.001
Mean Correct Certainty	30.19	00.64	30.62	00.45	30.93	00.12	<00.001	00.23	00.001	<00.001	<00.001
Total Incorrect Certainty	180.08	90.70	70.02	60.96	20.06	30.87	<00.001	00.36	<00.001	<00.001	<00.001
Mean Incorrect Certainty	20.92	00.78	20.87	00.89	30.20	90.37	ns		ns	ns	ns
Relative Certainty Index	00.92	00.16	00.81	00.24	00.83	00.24	00.007	00.04	ns	00.004	ns
Accuracy-Certainty Index	260.19	200.75	560.61	190.11	740.02	90.49	<00.001	00.50	<00.001	<00.001	<00.001

ANCOVAs are age/education adjusted.

As expected, the clinical groups differed in performance in an order which replicates well-known hierarchies of NC > MCI > AD for most outcomes. This was true for learning trial scores, delayed spontaneous recall, and various aspects of delayed recognition certainty. Certainty showed a similar pattern (the NC group was most certain of correct answers and the AD dementia group least certain), as did variables relying on a combination of accuracy and certainty (such as TCC, TIC, ACI). The RCI, designed to indicate potential anosognosia (as it is a ratio between certain on incorrect versus correct items and thus may represent lack of awareness of correctness), was higher in AD dementia than the other two groups. Largest effect sizes (magnitude of group differences) were found for later learning trial scores, delayed free recall, Total Correct Recognition, TCC, and ACI. Notably, the effect size was larger for ACI than for Total Correct Recognition, suggesting addition of certainty ratings added to group discrimination over accuracy alone.

Group classification

A logistic regression predicting group membership between NC (0) and MCI (1) using recognition memory accuracy (Total Correct Recognition) versus accuracy combined with certainty (ACI) was significant, χ²(2) = 59.798, p < 0.001, Nagelkerke R² = 0.453. The ACI was a much stronger predictor (β = -0.238, SE = 0.079, p = 0.003, Exp[B] = 1.268) than Total Correct Recognition (β = 1.020, SE = 0.622, p > 0.05, Exp[B] = 0.361). An ROC curve (Figure 1) showed stronger overall diagnostic classification accuracy for ACI (AUC = 0.823) than Total Correct Recognition (AUC = 0.798).

Figure 1.

ROC curve discriminating NC and MCI.

A logistic regression predicting group membership between NC (0) and AD (1) using recognition memory accuracy (Total Correct Recognition) versus accuracy combined with certainty (ACI) was significant, χ²(2) = 279.722, p < 0.001, Nagelkerke R² = 0.861. The ACI was a much stronger predictor (β = -0.413, SE = 0.097, p < 0.001, Exp[B] = 1.511) than Total Correct Recognition (β = 1.886, SE = 0.694, p = 0.007, Exp[B] = 0.115). An ROC curve (Figure 2) showed slightly stronger overall classification accuracy for ACI (AUC = 0.981) than Total Correct Recognition (AUC = 0.968).

Figure 2.

ROC curve discriminating NC and AD.

A logistic regression predicting group membership between MCI (0) and AD (1) using recognition memory accuracy (Total Correct Recognition) versus accuracy combined with certainty (ACI) was significant, χ²(2) = 70.226, p < 0.001, Nagelkerke R² = 0.397. The ACI was a stronger predictor (β = -0.077, SE = 0.030, p = 0.011, Exp[B] = 1.080) than Total Correct Recognition (β = 0.088, SE = 0.233, p > 0.05, Exp[B] = 0.916). An ROC curve (Figure 3) showed slightly stronger overall classification accuracy for ACI (AUC = 0.852) than Total Correct Recognition (AUC = 0.834).

Figure 3.

ROC curve discriminating MCI and AD.

Additional analyses

We found no significant correlations (p > 0.05) between Ham-D scores and any memory outcomes in our sample but found significant (p < 0.05) negative relationships between all Blessed scales and all CERAD Word List variables. When these correlations were repeated but conducted separately within clinical groups, correlations between Ham-D and outcomes were again non-significant (p < 0.05), and relationships between Blessed subscales and CERAD Word List outcomes were only significant (p < 0.05) for the following: In NC: none, in MCI: Blessed A with Total Certainty, TCC, and MCC, and in AD: Blessed A with each learning trial, Delayed Recall, Total Correct, Correct Targets, Correct Target Certainty, TCC, TIC, and ACI, and Blessed B with each learning trial. Please see the correlation matrices in Table 4.

Table 4.

Correlations (Pearson r values) between Blessed and Hamilton D scores and memory variables, by diagnosis group.

Diagnosis		Trial 1 Recall	Trial 2 Recall	Trial 3 Recall	Percent Savings	Delayed List Recall	Total Corr Recog	Corr Targ Recog	Correct Foil Recog	Corr Targ Cert	Corr Foil Cert	Tot Cert	To Corr Cert	Mean Corr Cert	Tot Incorr Cert	Mean Incorr Cert	Relative Certainty Index	Accuracy- Certainty Index
AD dementia	Blessed A	−0.232**	−0.254**	−0.302**	−0.261**	−0.263**	−0.289**	−0.208**	−0.070	−0.221**	−0.057	−0.029	−0.221**	−0.040	0.282**	0.039	0.085	−0.290**
	Blessed B	−0.224**	−0.214**	−0.256**	−0.114	−0.117	−0.075	−0.130	0.090	−0.130	0.120	0.112	−0.009	0.081	0.169*	0.169*	0.145	−0.087
	Blessed C	−0.063	−0.072	−0.035	−0.101	−0.112	0.060	0.019	0.049	0.029	0.027	0.019	0.045	0.006	−0.040	0.000	−0.011	0.051
	Blessed Total	−0.050	−0.072	−0.064	−0.108	−0.118	0.018	−0.017	0.046	−0.013	0.032	0.017	0.015	0.005	0.001	0.005	−0.007	0.010
	Ham-D	−0.186*	−0.127	−0.091	−0.176*	−0.163*	−0.135	−0.117	−0.003	−0.124	−0.011	−0.026	−0.110	−0.040	0.122	0.022	0.075	−0.136
MCI	Blessed A	0.160	0.183	0.156	−0.114	−0.039	−0.149	−0.138	−0.054	−0.284	−0.218	−0.348*	−0.321*	−0.405**	0.118	−0.064	0.134	−0.279
	Blessed B	0.115	0.177	0.169	0.078	0.114	−0.078	−0.051	−0.047	−0.113	−0.094	−0.180	−0.134	−0.153	−0.013	−0.195	−0.136	−0.096
	Blessed C	−0.142	0.015	−0.107	−0.019	−0.014	−0.177	−0.230	−0.003	−0.150	0.050	0.050	−0.051	0.113	0.205	0.019	−0.051	−0.106
	Blessed Total	−0.137	0.000	−0.121	−0.024	−0.027	−0.152	−0.217	0.015	−0.139	0.061	0.056	−0.037	0.112	0.184	0.022	−0.047	−0.089
	Ham-D	0.153	0.144	0.152	−0.108	−0.074	−0.026	−0.154	0.105	−0.165	0.067	−0.062	−0.047	−0.042	−0.002	−0.116	−0.108	−0.035
NC	Blessed A	0.100	−0.014	−0.056	0.135	0.074	−0.024	−0.028	0.027	−0.008	0.019	0.065	−0.002	0.049	0.048	0.153	0.154	−0.022
	Blessed B	0.100	−0.014	−0.056	0.135	0.074	−0.024	−0.028	0.027	−0.008	0.019	0.065	−0.002	0.049	0.048	0.153	0.154	−0.022
	Blessed C	0.100	−0.014	−0.056	0.135	0.074	−0.024	−0.028	0.027	−0.008	0.019	0.065	−0.002	0.049	0.048	0.153	0.154	−0.022
	Blessed Total	0.100	−0.014	−0.056	0.135	0.074	−0.024	−0.028	0.027	−0.008	0.019	0.065	−0.002	0.049	0.048	0.153	0.154	−0.022
	Ham-D	0.058	0.115	−0.153	0.034	−0.014	−0.206	−0.206	—	−0.207	0.106	0.087	−0.150	0.092	0.243	0.068	0.054	−0.200

*p < 0.05; **p < 0.01.

Discussion

Typical memory assessment involves use of free recall and yes/no recognition testing, with each item being considered only correct or incorrect. However, memory is a more complex process, and true memory recognition likely involves degrees of recognition, or memory “strength.” The current study presents a methodology for assessing memory certainty, combining it with accuracy scoring, and utilizing these certainty-weighted recognition scores for better diagnostic classification.

Our results show that nearly all aspects of the CERAD Word List memory test differed between NC, MCI, and AD dementia groups, including typical accuracy scores along with certainty rating variables. This is in line with prior literature that has demonstrated the sensitivity of the CERAD word list to distinguishing MCI and AD dementia.³⁹ The greatest effect sizes were found for later learning trial scores, delayed free recall, Total Correct Recognition, TCC, and ACI. Notably, the effect size was larger for ACI than for Total Correct Recognition, suggesting addition of certainty ratings adds to group discrimination over accuracy alone. This was supported by logistic regressions and ROC curves showing ACI having stronger ability to discriminate groups than Total Correct Recognition. This difference was most pronounced discriminating NC with MCI and AD groups, rather than discriminating MCI from AD.

Our findings were consistent with previous research showing delayed free recall is the most discriminating measure for detecting memory impairment.⁴⁰ Recognition memory testing adds to simple free recall by accounting for relative storage versus retrieval deficits.⁴¹ The dual-process model of recognition memory suggests that two distinct, independent processes contribute to recognition: recollection and familiarity. Recollection is closely related to recall and requires similar neuroanatomical substrates, particularly the hippocampus and prefrontal cortex. Likewise, familiarity is more closely related to implicit memory and relies more on regions surrounding the hippocampus.⁴¹

Our results show that certainty ratings provide higher resolution to recognition findings. Thus, especially in only subtly impaired patients, there are some cases where delayed recall and recognition memory alone may not be optimal. For a patient/client on the borderline of impairment, a low degree of certainty in their responses could tip the scales and suggest relatively weak memory recognition despite a reasonable accuracy score. It is important to note that addition of certainty ratings only added limited value to discriminating MCI from AD. While it appears addition of certainty ratings may be clinically useful in “dementia” differential overall, it may be less vital discriminating more impaired patients. We attribute this to the already-shown effectiveness of traditional memory testing in identifying Alzheimer's dementia. That is not to say that adding certainty ratings to assessments with these populations in unjustified, just that it will add less than it does in the discrimination between NC and other groups. Another justification for still including certainty ratings in more impaired groups aside from any addition to group classification could be the addition of different information (e.g., if further research validates these as a measure of insight, or if they correlate between with real-world functioning) to drive clinical care recommendations.

Correct certainty may be particularly important in patients with milder cognitive impairment, who may have fewer incorrect responses. However, certainty with wrong answers may play a different but critical role. More research is needed to clarify this, but it is possible that in addition to just weakened strength of a memory (and thus decreasing certainty in general), a patient could be worsening if they strongly believe they know the answer while they do not. Thus, they may have unawareness of their memory problem (anosognosia) versus confabulation due to impairment in temporal consciousness.⁴²

In addition to assisting diagnosis of MCI, certainty combined with accuracy (ACI) was related to “real-world” functioning as well. In MCI participants, informant report of performance in everyday activities (Blessed Rating Scale A) was related only to certainty measures, and not to standard memory measures (learning, delayed recall, normal recognition). Certainty ratings were not related to performance of healthy controls, but several (certainty and/or accuracy) measures were related to everyday performance for those with AD. This is in line with current literature that suggests that informant report of current functioning has standalone utility for predicting cognitive impairment.⁴³ Again, this suggests a specific relationship between certainty performance and everyday performance in the group that these measures are most effective at categorizing people with MCI.

Our methodology discriminated between groups and separately was related to functional impairment ratings as well, but the reason why is still not completely clear. One explanation could be that weakening of memory traces are not necessarily detected with simple yes/no responses. It seems less likely that those with impairment have noticed their decline and have thus lost confidence in their memory in general, resulting in decreased certainty. Since certainty ratings also related to informant report of everyday functioning, the “loss of confidence” affecting certainty performance would have to extend well beyond testing circumstances and result in everyday errors, which would be observable by others. That theory is still possibly true, although in this study, certainty ratings were not at all correlated with Ham-d depression scores, and were often not related to informant reports of changes in personality, drives, and interests. Thus, if certainty ratings were only affected by a loss of confidence, it would have to be a very specific type of confidence but also severe enough to impact daily functioning, while not resulting in significant depression. Together, these finding suggest that certainty ratings more likely reflect a true memory process rather than personality, behavioral, or emotional factors, although this also needs more research.

This study has some notable strengths. The sample size is large for each diagnostic group. The CERAD list is similar to many other commonly used list-based verbal episodic memory tests (e.g., Rey Auditory Verbal Learning Test, California Verbal Learning Test, Hopkins Verbal Learning Test, word list from the Repeatable Battery for the Assessment of Neuropsychological Status) in its use of learning trials, delayed recall, and yes/no recognition. It is reasonable to expect that adding certainty ratings to recognition testing for those measures would add diagnostic power as it does with the CERAD. Future studies could also assess use of certainty for memory measures other than word lists that involve recognition testing (e.g., stories, visual designs), and how these compared to word list certainty recognition. We will also note that while they may seem complicated at first, calculation of the certainty-weighted variables is actually somewhat simple and can easily be done by clinicians or researchers manually if needed.

There are also some limitations with this study, including the sample characteristics and external validity. The ADRCs are known to be highly selected samples which historically had been often more educated, less ethnoracially diverse, with greater socioeconomic status than target populations of community-dwelling older adults. Targeted efforts have been made to improve this,⁴⁴ although at the time these current data were collected, the difference was likely greater than now. Thus, while the ethnoracial characteristics of this sample from the Pitt ADRC were on par with the local community population, these results may not be applicable to communities with greater ethnoracial diversity or community-dwelling older adults with less education.

The lack of biological classification of etiologic diagnosis through cerebrospinal fluid analysis or postmortem neuropathological confirmation is a further limitation that could be rectified in future studies of this type of item-by-item certainty on memory tasks. This would allow a more granular distinction between those meeting clinical criteria for an amnestic dementia syndrome and amnestic MCI with and without biological evidence of Alzheimer's disease neuropathologic change. We also must mention the data were collected between 2002–2004, with McKhann criteria used at the time for diagnostic classification, and that these criteria could be seen as out-of-date today. Future studies should validate certainty ratings with more updated classification criteria.

Future studies may also consider evaluating latency or reaction time for recognition correct and incorrect responses, as they may provide additional information into awareness.

Footnotes

Acknowledgements

The authors have no acknowledgments to report.

ORCID iDs

Matthew Harris

Marissa A. Gogniat

Ethical considerations

All ethical guidelines for research were followed. This study was approved by the University of North Carolina Institutional Review Board, study # 13-0429.

Consent to participate

Written consent to participate in ADRC testing was obtained as per ADRC procedures. IRB waived additional consent for this study as data were considered archival.

Consent for publication

Not applicable

Author contribution(s)

Matthew Harris: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Project administration; Supervision; Writing – original draft; Writing – review & editing.

James R. Bateman: Conceptualization; Investigation; Writing – review & editing.

C. Elizabeth Shaaban: Conceptualization; Data curation; Project administration; Writing – review & editing.

James Becker: Methodology; Project administration; Writing – review & editing.

Steven T. DeKosky: Data curation; Methodology; Project administration; Writing – review & editing.

Oscar L. Lopez: Data curation; Investigation; Methodology; Project administration; Writing – review & editing.

Marissa A. Gogniat: Conceptualization; Methodology; Writing – review & editing.

Beth Snitz: Investigation; Methodology; Writing – review & editing.

Daniel Kaufer: Conceptualization; Data curation; Investigation; Methodology; Project administration; Supervision; Writing – original draft.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

Data not made available.

References

Ruiter

Schoonbrood

Twillert

, et al. Anosognosia in dementia: a review of current assessment instruments. Alzheimers Dement (Amst) 2020; 12: e12079.

Agnew

Morris

. The heterogeneity of anosognia for memory impairment in Alzheimer's disease: a review of the literature and a proposed model. Aging Ment Health 1998; 2: 7–19.

Banks

Weintraub

. Self-awareness and self-monitoring of cognitive and behavioral deficits in behavioral variant frontotemporal dementia, primary progressive aphasia and probable Alzheimer's disease. Brain Cogn 2008; 67: 58–68.

Rosen

Alcantar

Zakrzewski

, et al. Metacognition in the behavioral variant of frontotemporal dementia and Alzheimer’s disease. Neuropsychology 2014; 28: 436–447.

Williamson

Alcantar

Rothlind

, et al. Standardized measurement of self-awareness deficits in FTD and AD. J Neurol Neurosurg Psychiatry 2010; 81: 140–145.

Vannini

Hanseeuw

Munro

, et al. Anosognosia for memory deficits in mild cognitive impairment: insight into the neural mechanism using functional and molecular imaging. Neuroimage Clin 2017; 15: 404–414.

Kalbe

Salmon

Perani

, et al. Anosognosia in very mild Alzheimer's disease but not in mild cognitive impairment. Dement Geriatr Cogn Disord 2005; 19: 349–356.

Tremont

Alosco

. Relationship between cognition and awareness of deficit in mild cognitive impairment. Int J Geriatr Psychiatry 2011; 26: 299–306.

Vogel

Hasselbalch

Gade

, et al. Cognitive and functional neuroimaging correlate for anosognosia in mild cognitive impairment and Alzheimer's disease. Int J Geriatr Psychiatry 2005; 20: 238–246.

10.

Ries

Jabbar

Schmitz

, et al. Anosognosia in mild cognitive impairment: relationship to activation of cortical midline structures involved in self-appraisal. J Int Neuropsychol Soc 2007; 13: 450–461.

11.

McGlynn

Schacter

. Unawareness of deficits in neuropsychological syndromes. J Clin Exp Neuropsychol 1989; 11: 143–205.

12.

Rosen

. Anosognosia in neurodegenerative disease. Neurocase 2011; 17: 231–241.

13.

Zamboni

Wilcock

. Lack of awareness of symptoms in people with dementia: the structural and functional basis. Int J Geriatr Psychiatry 2010; 26: 783–792.

14.

Hallam

Chan

Gonzalez Costafreda

, et al. What are the neural correlates of meta-cognition and anosognosia in Alzheimer's disease? A systematic review. Neurobiol Aging 2020; 84: 250–264.

15.

Mondragon

Maurits

De Dyn

. Functional neural correlates of anosognoia in mild cognitive impairment and Alzheimer's disease: a systematic review. Neuropsychol Rev 2019; 29: 139–165.

16.

Kelleher

Tolea

Galvin

. Anosognosia increases caregiver burden in mild cognitive impairment. Int J Geriatr Psychiatry 2016; 31: 799–808.

17.

Seltzer

Vasterling

Yoder

, et al. Awareness of deficit in Alzheimer's disease: relation to caregiver burden. Gerontologist 1997; 37: 20–24.

18.

Turró-Garriga

Garre-Olmo

Vilalta-Franch

, et al. Burden associated with the presence of anosognosia in Alzheimer's disease. Int J Geriatr Psychiatry 2013; 28: 291–297.

19.

Cotrell

Wild

Bader

. Medication management and adherence among cognitively impaired older adults. J Gerontol Soc Work 2006; 47: 31–46.

20.

Okonkwo

Wadley

Griffith

, et al. Awareness of deficits in financial abilities in patients with mild cognitive impairment: going beyond self-informant discrepancy. Am J Geriatr Psychiatry 2008; 16: 650–659.

21.

Mottola

Wilson

, et al. Metamemory and financial decision making in older adults without dementia. Neuropsychology 2022; 36: 35–43.

22.

Sunderaraman

Cosentino

. Integrating the constructs of anosognosia and metacognition: a review of recent findings in dementia. Curr Neurol Neurosci Rep 2017; 17: 27.

23.

Wild

Cotrell

. Identifying driving impairment in Alzheimer disease: a comparison of self and observer reports versus driving evaluation. Alzheimer Dis Assoc Disord 2003; 17: 27–34.

24.

Ernst

Moulin

Souchay

, et al. Anosognosia and metacognition in Alzheimer's disease: insights from experimental psychology. In: Dunlosky

Tauber

(eds) The Oxford handbook of metamemory. Oxford: Oxford University Press, 2016, pp.451–472.

25.

Nelson

. Metamemory, psychology of. In: Smelser

Baltes

(eds) International encyclopedia of the social and behavioral sciences. Oxford, UK: Pergamon, 2001, pp.9733–9738.

26.

Cosentino

Stern

. Metacognitive theory and assessment in dementia: do we recognize our areas of weakness? J Int Neuropsychol Soc 2005; 11: 910–919.

27.

Metcalfe

Dunlosky

. Metamemory. In: Byrne

(ed.) Learning and memory: a comprehensive reference. New York: Academic Press, 2008, pp.349–362.

28.

Yang

Luo

. Retrospective confidence rating about memory performance is affected by both retrieval fluency and non-decision time. Metacogn Learn 2022; 17: 651–681.

29.

Chua

Schacter

Sperling

. Neural correlates of metamemory: a comparison of feeling-of-knowing and retrospective confidence judgments. J Cogn Neurosci 2009; 21: 1751–1765.

30.

Gallo

Cramer

Wong

, et al. Alzheimer's disease can spare local metacognition despite global anosognosia: revisiting the confidence-accuracy relationship in episodic memory. Neuropsychologia 2012; 50: 2356–2364.

31.

Pappas

Sunderland

Weingartner

, et al. Alzheimer's disease and feeling-of-knowing for knowledge and episodic memory. J Gerontol 1992; 47: P159–P164.

32.

Cosentino

Metcalfe

Butterfield

, et al. Objective metamemory testing captures awareness of deficit in Alzheimer's disease. Cortex 2007; 43: 1004–1019.

33.

Dodson

Spaniol

O'Connor

, et al. Alzheimer's disease and memory-monitoring impairment: Alzheimer's patients show a monitoring deficit that is greater than their accuracy deficit. Neuropsychologia 2001; 49: 2609–2618.

34.

Morris

Heyman

Mohs

. The Consortium to Establish a Registry for Alzheimer's Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer's disease. Am Acad Neurol 1989; 39: 1159–1165.

35.

McKhann

Drachman

Folstein

, et al. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology 1984; 34: 939–944.

36.

Hamilton

. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960; 23: 56–61.

37.

Mulsant

Sweet

Rifai

, et al. The use of the Hamilton Rating Scale for depression in elderly patients with cognitive impairment and physical illness. Am J Geriatr Psychiatry 1994; 2: 220–229.

38.

Blessed

Tomlinson

Roth

. The association between quantitative measures of dementia and of senile changes in the cerebral gray matter of elderly subjects. Br J Psychiatry 1968; 114: 797–811.

39.

Kanchanatawan

Tangwongchai

Supasitthumrong

, et al. Episodic memory and delayed recall are significantly more impaired in younger patients with deficit schizophrenia than in elderly patients with amnestic mild cognitive impairment. PLoS One 2018; 13: e0197004.

40.

Zhao

Zhou

, et al. Short-term delayed recall of auditory verbal learning test is equivalent to long-term delayed recall for identifying amnestic mild cognitive impairment. PLoS One 2012; 7: e51157.

41.

Yonelinas

. The nature of recollection and familiarity: a review of 30 years of research. J Mem Lang 2002; 46: 441–517.

42.

Belli

Nicoletti

Radicchi

, et al.

Confabulations in cases of dementia: atypical early sign of Alzheimer's disease or misleading feature in dementia diagnosis?

Front Psychol 2020; 11: 553886.

43.

Denny

Bartley

Edwards

, et al. Ad8 patient-informant discrepancy predicts insight and cognitive impairment in Alzheimer's disease. Geriatr Nurs 2021; 42: 262–267.

44.

Williams

Meisel

William

, et al. An interdisciplinary outreach model of Arican American recruitment for Alzheimer's disease research. Gerontologist 2011; 51: S134–S141.