Investigating self-report and neuropsychological assessments of cognitive flexibility in people with and without persistent pain: An online,cross-sectional,observational study

Abstract

Introduction

People with persistent pain experience problems modifying their cognition and behaviours when task or environmental demands change – abilities otherwise known as cognitive flexibility. However, limitations and inconsistent results of previous studies raise concerns over the quality of that evidence. We aimed to determine whether people with and without persistent pain differ on two assessments that are commonly used to assess cognitive flexibility. We also examined the relationship between the two assessments and explored whether people with and without persistent pain are distinguishable based on their scores on these assessments.

Methods

Participant demographics and symptoms of anxiety and depression were assessed. Participants completed the Cognitive Flexibility Inventory (CFI) and the Wisconsin Card Sorting Test (WCST). Multiple linear regression on the two outcome variables: CFI (total score) and WCST (% perseverative responses) was applied using backward stepwise selection. Both outcomes were calculated as a standardised proportion of the outcome scale and log-odds transformed to meet the model assumptions. Correlation analysis and logistic regression were used to investigate our secondary and exploratory aims.

Results

Data were available from 128 participants with persistent pain and 68 pain-free controls. After adjusting for covariates, no differences were found between people with and without persistent pain on either assessment of cognitive flexibility. No significant correlations were detected between the two assessments in either group. The probability of having persistent pain was also not associated with scores on either or both assessments.

Conclusion

‘Cognitive flexibility’ appears similar in people with and without persistent pain.

Keywords

Chronic pain executive function cognitive flexibility neuropsychology self-report

Introduction

Persistent pain, characterised by pain that exceeds the normal time frame required for bodily tissue to heal after injury (i.e. usually >3 months),^1,2 is a worldwide health problem.³ Persistent pain affects approximately 20% of the population^4–6 and interferes with many aspects of everyday life including physical, emotional and social functioning.⁷ Despite major advances in our knowledge and understanding of persistent pain, important gaps remain. Particularly, the role of cognitive factors in the development of and recovery from persistent pain remains to be fully understood. Uncovering such factors has the potential to lead to more effective and tailored strategies to prevent and treat persistent pain.

One potentially important factor is what is commonly known as ‘cognitive flexibility’. Cognitive flexibility is thought to be a multifaceted construct⁸ that broadly refers to the ability to adjust thoughts, and cognitive and behavioural strategies when task or environmental demands change.^9,10 There are two approaches to assessing cognitive flexibility – self-report questionnaires and neuropsychological assessments. The Cognitive Flexibility Scale (CFS),¹¹ the Cognitive Flexibility Inventory (CFI)¹² and the Shift subscale of the Behavior Rating Inventory of Executive Function (BRIEF)¹³ are commonly used self-report assessments of cognitive flexibility. Commonly used neuropsychological tests of cognitive flexibility include the Wisconsin Card Sorting Test (WCST),¹⁴ the Trail Making Test (TMT)¹⁵ and the Stroop Test.¹⁶

Several studies have investigated how people with persistent pain perform on neuropsychological assessments of cognitive flexibility, but those studies have yielded contrasting results. Some suggest people with persistent pain perform worse than pain-free controls;^17–19 others suggest they do not.^20–25 One study²⁶ found that people with fibromyalgia and people with musculoskeletal pain performed better on the Stroop Test (used here and by others e.g. Johnco et al.²⁷ as a marker of cognitive flexibility) than pain-free controls, once depression and premorbid intelligence had been statistically controlled for. The mixed findings are difficult to interpret because most studies are underpowered,^{17,18,20,22,23,25,26} use a combination of assessments and outcome variables without correcting for multiple comparisons,^18,22,24,26 or provide insufficient information about the task being used or how outcome variables were calculated.^19,23,26 Further, the assortment of neuropsychological assessments and outcome variables that are used to assess cognitive flexibility poses great challenges when comparing results across studies. When data are combined, meta-analyses reveal that people with persistent pain perform worse than pain-free controls on neuropsychological tests of cognitive flexibility,^28–31 but high risk of bias of constituent studies leaves some degree of uncertainty. Therefore, although the general pattern of systematic evidence indicates that people with persistent pain are impaired on neuropsychological assessments of cognitive flexibility, there are significant uncertainties around that evidence.

There is a building body of evidence that, although these self-report and neuropsychological assessments both purport to be capturing ‘cognitive flexibility’, they are unlikely to be tapping into shared, or similar, constructs.^32,33 In people with and without fibromyalgia, there appears to be a lack of association between self-reported cognitive and behavioural flexibility (i.e. Shift subscale of the BRIEF) and neuropsychological tests of cognitive flexibility (i.e. TMT B-A and WCST sum of perseverative responses).²¹ Our two meta-analytic reviews also failed to detect a relationship between self-report questionnaires and neuropsychological tests of ‘cognitive flexibility’ in both clinical (11 studies)³³ and non-clinical (21 studies)³² cohorts.

With the exception of two cross-sectional studies,^21,34 no research to date has examined how people with persistent pain differ from pain-free controls when self-evaluating their ability to be cognitively and behaviourally flexible. Baker et al.³⁴ found that people with persistent pain reported worse functioning on the Shift subscale of the BRIEF than pain-free controls did, whereas Gelonch et al.²¹ reported no differences between people with and without fibromyalgia on both self-report and neuropsychological assessments of cognitive flexibility, after adjusting for depression and anxiety.

It is possible that there may be distinguishable groups of people with persistent pain that can be identified on the basis of their performance on self-report and neuropsychological assessments of cognitive flexibility. Indeed, a study by Attal et al.³⁵ found that in a cohort of people undergoing surgical procedures for osteoarthritis and breast cancer, the presence of chronic post-surgical pain at 6 and 12-month was significantly predicted by performance on the TMT (part B). However, in people undergoing knee arthroplasty or noncardiac chest surgery, performance on the TMT (B-A) and Stroop (interference T-scores) did not predict the incidence of persistent post-surgical pain 6-month later.³⁶ To our knowledge, there have been no attempts to predict the presence of persistent pain based on concurrent self-report and neuropsychological assessments of ‘cognitive flexibility’. It seems reasonable to suggest that there may be subgroups of people who are characterised by different patterns of self-reported and neuropsychological assessment results, but more evidence is needed to fill this gap in knowledge. Identifying such patterns may be important because if such groups exist, it would aid in the development of therapeutic approaches that are specifically targeted to individual needs.

The current study had three aims. Our first aim was to determine how people with persistent pain perform on a self-report questionnaire and on a neuropsychological assessment of cognitive flexibility relative to pain-free controls, after controlling for known confounders. We hypothesised that people with persistent pain would, on average, perform worse than their pain-free counterparts on both a self-report and a neuropsychological assessment of cognitive flexibility, even after controlling for confounders. Our second aim was to determine the relationship between self-reported cognitive flexibility and neuropsychologically assessed cognitive flexibility in people with and without persistent pain. Based on previous findings, we hypothesised that the two assessments would not relate. Our third, and exploratory aim, was to explore whether the probability of having persistent pain (or not) is associated with scores obtained on two concurrent cognitive flexibility assessments. No specific hypothesis was generated around this exploratory aim. For a full description of our a priori predictions, refer to the statistical analysis plan, which can be accessed via Open Science Framework (OSF) (https://osf.io/9q8ut/).

Materials and methods

The University of South Australia's Human Research Ethics Committee (HREC) granted ethical approval for this online, cross-sectional study (approval no. 202439). The Checklist for Reporting Results of Internet E-Surveys (CHERRIES)³⁷ and the STROBE guidelines^38,39 were adhered to during the reporting of this study (see Supplemental Files 1 and 2). An a priori protocol was registered and locked under embargo on OSF (https://osf.io/9q8ut/). Deviations from the protocol or statistical analysis plan were recorded and were either outlined in this manuscript, or uploaded and time-stamped on OSF.

Participants

Participants from the general population were recruited using an online, convenience sampling strategy. The study was advertised via recruitment flyers, social media posts, announcements on the Universty of South Australia's Research Volunteers website, and emails that were distributed using previously consented participant mailing-lists and the Research Participation Sign-up System, managed by the School of Psychology, Social Work & Policy, University of South Australia. Participants with or without persistent pain (i.e. pain experienced most days for three months or longer)^1,2 were invited to take part in an online survey. Participants were eligible if they were aged ≥ 18 years, had adequate ability to read and understand English, had normal or corrected-to-normal (e.g. glasses or contact lenses) vision, and had access to, and competency with, a computer/laptop or smartphone with internet access. Participants were excluded if they had colour vision deficiencies.

Measures

Primary outcomes

Self-reported cognitive flexibility – the Cognitive Flexibility Inventory (CFI)

The CFI is a 20-item scale that aims to assess: (i) the tendency to realise that challenging situations are indeed controllable; (ii) the ability to perceive different explanations for life incidences and human behaviour; and (iii) the capability to produce multiple alternative solutions to difficult scenarios.¹² The CFI consists of two subscales: the Alternatives subscale (13 items) and the Control subscale (7 items). Respondents are required to indicate the degree to which they agree or disagree with various statements. Each item is rated on a 7-point Likert scale from 1 (strongly disagree) to 7 (strongly agree), with some items being reversed scored. Higher scores on the CFI are considered to reflect greater self-reported cognitive flexibility. The CFI has previously demonstrated high internal consistency, adequate test-retest reliability over a 7-week time frame (r = 0.81, p < .001), and convergent validity with the Cognitive Flexibility Scale.¹² We selected the CFI because unlike other available questionnaires, it was developed with the intention of assessing components of cognitive flexibility that are thought to be adaptive when faced with stressful circumstances.¹² Here, the CFI’s total score was used as an outcome of self-reported cognitive flexibility.

Neuropsychologically assessed cognitive flexibility – the Wisconsin Card Sorting Test (WCST)

The WCST¹⁴ is a card-sorting task that has been widely used to assess cognitive flexibility.^27,40 Participants are instructed to ‘match’ 128 response cards, one by one, to one of four stimulus cards that vary in terms of their colour, form and number – see Figure 1. An electronic version of the WCST, which was scored in accordance with the standardised manual,¹⁴ was administered via Inquisit Web – an online software program hosted by Millisecond.⁴¹ Participants were able to match a response card to one of the four multidimensional stimulus cards by clicking on it with either a computer mouse or with their fingertip via a touch-screen. Participants were not provided with direction on how to correctly sort the response cards, but relied only on written feedback (‘correct’ or ‘wrong’) displayed on the screen to determine the correct sorting principle with which to sort the cards by. After 10 correct responses were made in succession, the correct sorting principle changed and participants were required to determine the new sorting principle through a process of trial and error. In line with the recommendations outlined by Miles et al.⁴² perseverative responses (percentage score) were used as an indicator of neuropsychological cognitive flexibility. Perseverative responses are continual responses made in accordance to a stimulus dimension that does not match the correct sorting principle, that is, the response is incorrect. However, since the cards are multidimensional and at times ambiguous – that is, a response card has the potential to match a stimulus card on multiple dimensions (e.g. colour and number) – perseverative responses may also be correct. A higher percentage of perseverative responses is thought to reflect poorer neuropsychologically assessed cognitive flexibility. We selected the WCST over other neuropsychological assessments because it most closely aligns with the definition of cognitive flexibility and is widely used in cognitive flexibility research. However, the validity and reliability of the WCST remains to be fully established. Sub-optimal psychometric properties of the WCST may partly stem from varying scoring procedures and administration modalities (i.e. manual or computerised), matters we have covered elsewhere.⁴²

Figure 1.

An illustrative example of the WCST. The single card with one yellow triangle is one of 128 response cards. The four bottom cards from the left-hand side to right hand side: one red triangle, two green stars, three yellow crosses and four blue circles are the stimulus cards.

Surveyed demographics

Participants responded to questions relating to age, sex assigned at birth, current gender identity, education level, current activity/occupation, financial situation, place of residence, country of residence, country of birth, refugee status, identification with ethnic or racial groups, mental health, neurological conditions and medication use. Using adaptive questioning, all participants were then asked whether they have persistent pain, defined as ‘pain experienced most days for 3 months or longer’. Participants who responded ‘yes’ to this question were presented with additional questions relating to pain intensity, duration, location(s) and diagnosis (if applicable).

Hospital Anxiety and Depression Scale (HADS)

The HADS is a questionnaire that is used to assess symptoms of anxiety and depression. The HADS consists of 14 items rated on a 4-point Likert scale ranging from 0 to 3. The 14 items are divided into two subscales: 7 items assess depressive symptoms and 7 items assess anxiety symptoms. The scores from each subscale range from 0 to 21, with higher scores reflecting more severe symptoms: 0–7 (normal), 8–10 (borderline abnormal) and 11–21 (abnormal).^43,44 The two subscales of the HADS are internally consistent (HADS-D ranging from 0.67 to 0.90; HADS-A ranging from 0.68 to 0.93) and the HADS is a valid and reliable scale, with high sensitivity and specificity (range: 0.70 to 0.90).⁴⁵

Procedure

The online survey was developed and administered using Inquisit Web.⁴¹ Participants voluntarily accessed the online survey by following a link that prompted them to install the Inquisit 6 Player application (version 6.5.1) on their chosen device (either a computer or smartphone). Once Inquisit was installed, participants were instructed to click the ‘start’ button, which directed them to an information sheet and consent form. Informed consent was obtained electronically from each participant by selecting the ‘I wish to proceed’ button at the bottom of the page before commencing the study. The introductory page informed respondents about the purpose of the study, the eligibility criteria, the estimated time required for survey completion, the management of data and confidentiality, and that participation was not mandatory even after agreeing to take part. Participants were asked to enter a 6-digit alphanumerical ID code and were then asked to respond to 3 screening questions to confirm their eligibility: (1) are you at least 18 years of age? (2) are you colour-blind? and (3) can you read and understand English? Where participants were not eligible for this study, they were redirected to a page that notified them that they were ineligible to participate in this study and thanked them for their interest.

Eligible participants were asked whether they wished to receive a summary of the results and whether they would like to be contacted about future research being conducted – responding ‘yes’ to either or both items prompted participants to enter their preferred email address in a text-box. Personal information (i.e. email addresses) were removed prior to data analysis and stored separately from the data in a password protected file. Participants were then directed to a demographic questionnaire, self-report questionnaires and the WCST. Data for the Immune Status Questionnaire (ISQ)⁴⁶ were collected as part of a larger study, but were not analysed or reported here. The number of questionnaire items was limited to three per page. Questionnaire items were distributed across a different number of screen pages depending on group allocation – 41 pages for people with persistent pain and 36 pages for pain-free controls. Due to the nature of the WCST, presented screen pages varied across participants; the maximum number was 130 (including instruction and completion screen pages). It was compulsory to respond to every question that was presented in the survey, except for demographic questions relating to participants’ life (age, education level, financial situation etc.); however, participants could quit the survey at any time by pressing ‘Alt + E’ on their computer/laptop keyboard or by closing the application on their smartphone. The self-report questionnaires (i.e. CFI, HADS and ISQ) and the neuropsychological test (i.e. WCST) were presented in a counterbalanced order, such that some participants received these tests in the order presented above, while others received the tests in the reversed order (i.e. WCST, ISQ, HADS, CFI). The demographic questionnaire was always presented first. Participants were able to change their responses using a ‘Back’ button up until the point of submitting each of the self-report questionnaires; it was not possible for participants to change their responses during the completion of the WCST.

Upon study completion (or where consent was withdrawn part way through the study), participants were directed to a debriefing form that outlined the true purpose and aims of the study. Participants were then provided with information relating to mental health services, instructions on how to uninstall Inquisit from their device and contact information of the project’s primary investigator. Participants were given up to 1 week after study completion to withdraw their data from being included in the final analysis if they wished to do so. No honorariums were offered for study completion. A small group of colleagues within our laboratory piloted the survey, wherein they provided feedback on the clarity of the demographic items and the functionality of the survey as a whole. All recruitment materials and the full online survey are provided in the Supplementary material (see Supplemental Files 3–9).

Data analysis

Sample size calculation

Sample size calculations were made under the assumptions of the analysis to be performed on the two primary outcomes (see primary analysis for further details) to ensure sufficiently powered primary analyses. Mean and standard deviation estimates of the outcome variables (WCST perseverative responses and CFI total score) in previous studies^12,27,47 were used. A detectable increase of 5% for WCST perseverative responses (%) in the persistent pain group than that of the pain-free control group was used, and a detectable decrease of 6 (similarly 5% of the scale) for the CFI (total score) in the persistent pain group than that of the pain-free control group was used. A required sample size of at least 120 would achieve 95% statistical power at the level of significance of 0.05. The sample size calculations were performed using simulation^48–50 of a beta distributed response of a linear model data generating process because both outcomes are approximately continuous, skewed and bounded. Sensitivity analysis of the required sample size was also performed using alternative model specifications and a range in the number of covariates and their relative effects. The empirical calculations were verified using analytical multiple linear regression power analysis methods available in G*Power 3^51,52 and the pwr package⁵³ in R (version 4.1.3).⁵⁴ Sample size calculations were undertaken prior to data analysis, but not prior to data collection. Full details regarding the sample size calculation can be found on OSF (https://osf.io/9q8ut/).

Statistical analysis plan

Data analysis was performed using R (version 4.2.2).⁵⁴ Raw data were imported into R using the readr⁵⁵ package. Data manipulation was performed using the tidyverse⁵⁶ and janitor⁵⁷ packages. Data visualisation and figures were created using ggplot2,⁵⁸ ggpubr,⁵⁹ ggstatsplot⁶⁰ and jtools,⁶¹ while diagnostic plots were generated using the performance⁶² package. The Classic Color Blind colour palette via the ggthemes⁶³ package; the Darjeeling1 colour palette via the wesanderson⁶⁴ package; the Okabe-Ito colour palette⁶⁵ via the colorblindr⁶⁶ package; and the base R grDevices⁵⁴ package were used in our attempt to make data visualisation more accessible to people who are colour-blind. Descriptive tables were created using gtsummary.⁶⁷ Linear and logistic regression outputs were interpreted using the report⁶⁸ package. Model equations were adapted from the equatiomatic⁶⁹ package. Other packages that were used include writexl,⁷⁰ skimr,⁷¹ car,⁷² and parameters.⁷³

We decided a priori that (i) the same user ID indicated duplicate participants and that we would retain the first entry for analysis and exclude any subsequent entries; (ii) data collected prior to participant withdrawal (i.e. non-completion of primary outcomes) would not be analysed and (iii) data would be excluded from statistical analyses if 10 or more identical responses were given in succession on the CFI. No data were missing for any of the variables used to perform our primary analyses, so multiple imputation was not required (refer to the statistical analysis plan on OSF: https://osf.io/9q8ut/). Data for each of the reported non-binary identities did not meet our predetermined sample size (i.e. >5 participants). To protect the privacy and safety of participants, we needed to either (i) combine non-binary gender identities into a third group (providing the reported non-binary gender identities were similar enough to warrant being combined) or (ii) remove them from data analysis. Using The Trans Language Primer,⁷⁴ we found that the reported non-binary identities were similar, so we decided to collapse them into a third group, namely ‘transgender and non-binary individuals’. Unless otherwise stated, statistical significance was set at $α$ = 0.05. Annotated R scripts that were used to undertake all aspects of data analysis can be accessed via OSF (https://osf.io/9q8ut/).

Primary analysis

Multiple linear regression

Separate regression analyses were carried out to (i) determine whether people with persistent pain perform worse, on average, than pain-free controls on a neuropsychological assessment of cognitive flexibility and (ii) determine whether self-reported cognitive flexibility is lower, on average, in those with persistent pain than in pain-free controls, after adjusting for confounding variables in both models. The following variables were entered into each model as covariates: age, gender (male, female, non-binary or transgender individuals), education level (high school, tertiary education, university higher degree), medication use (no, yes), HADS depression and anxiety scores, as well as test administration order ([order 1]: CFI first, WCST last vs [order 2]: WCST first, CFI last), and modality of survey completion (computer/laptop vs smartphone). The first level listed in each of the categorical covariates refers to the reference level used. Multiple linear regression with backwards stepwise selection was used to ensure model parsimony to avoid overfitting.⁷⁵ The predictor variable of interest was the grouping variable – that is, healthy control (HC) or persistent pain (PP); the outcome variables were the CFI (total score) and the WCST (% perseverative responses), calculated as proportions along their associated scales, herein, referred to as Cognitive Flexibility Inventory proportion of total score (CFI proportion) and WCST proportion of % perseverative responses (WCST proportion), respectively. These proportional outcome variables were transformed via log-odds transformation to satisfy the assumptions of multiple linear regression. Proportions equal to 0 or 1, which would result in undefined logit transformed values, were handled by adding (or subtracting) a half of the smallest unit of the scale. The assumptions of linearity, homoscedasticity of the residuals, influential observations, normality of the residuals and multicollinearity were inspected. Initially, all predictor variables were entered into each regression model to create the full initial models. Predictors with p-values >.05 were then removed in a stepwise manner – removing the predictor with the largest p-value first – until only significant predictors remained in both models.⁷⁶ Since the effect of group (i.e. PP or HC) on the two outcome variables (CFI and WCST proportion) was of primary interest, this variable remained in the final models irrespective of the p-value.

The aforementioned covariates were selected based on prior literature. Indeed, performance on the WCST has been found to be influenced by gender,⁷⁷ age and education.^14,78 Some studies have reported that poor cognitive performance is associated with depression and anxiety.^79,80 Further, medications including antidepressants, opioids and anticonvulsants have been found to have both enhancing and detrimental effects on cognitive function.^81–83 On the other hand, gender,⁸⁴ and symptoms of depression and anxiety have been found to influence scores on the CFI.^12,27,85,86 For the purposes of consistency, and that not every demographic characteristic has been previously investigated for its effects on both the CFI or WCST, we decided to enter each of these variables as covariates in the regression models of the two outcomes variables.

The full initial regression models for the i = 1, 2…, 196 participants take the form:

logit (Y_{i}) = β_{0} + β_{1} ({Group}_{i} = Persistent pain) + β_{2} ({Age}_{i}) + β_{3} ({Educationlevel}_{i} = Tertiary) + β_{4} ({Educationlevel}_{i} = University higher degree) + β_{5} ({Gender}_{i} = Female) + β_{6} ({Gender}_{i} = Non ‐ binary and / or transgender individuals) + β_{7} ({Medication}_{i} = yes) + β_{8} ({HADS_anxiety}_{i}) + β_{9} ({HADS_depression}_{i}) + β_{10} ({Test order}_{i} = Order 2) + β_{11} ({Modality}_{i} = Smartphone) + ϵ_{i}

where:

• $Y_{i}$ is the outcome variable (a proportion of the outcome scale): either CFI total score or WCST % perseverative responses) for the $i^{t h}$ participant,

• $β_{0}$ is an intercept (the predicted value of $l o g i t (Y_{i})$ when all predictors are 0),

• $β_{j}$ is the coefficient of the jth predictor for $j = 1,2, . . ., 11$ (model coefficients),

• the categorical predictor variables take the form $(X_{i} = l)$ for level $l$ of predictor $X$ and takes the value 1 when $X_{i} = l$ for participant $i$ (and 0 otherwise), and

• $ϵ_{i} \sim N (0, σ)$ (independently and identically distributed normal error) for participant i.

Sensitivity analyses were carried out with univariate outliers – CFI or WCST proportion values that fell below Q1 - 1.5 × IQR or exceeded Q3 + 1.5 × IQR – removed, which was visually confirmed via box-plots.

Secondary and exploratory analyses

Correlations

Currently, the relationship between self-reported and neuropsychological assessments of cognitive flexibility remains uncertain in people with persistent pain. To investigate the relationship between a self-report and neuropsychological assessment of cognitive flexibility, raw scores on the CFI subscales (i.e. Alternatives and Control) were correlated with raw perseverative response (%) scores on the WCST in people with and without persistent pain. Separate correlation analyses were performed for the persistent pain group and pain-free control group. Holm–Sidak adjusted significance thresholds (based on the sorted p-values) were applied to account for multiple comparisons and reduce the risk of Type I errors.⁸⁷ Holm–Sidak adjusted 95% confidence intervals of the estimated coefficients were also calculated. The assumptions of linearity or homoscedasticity were independently assessed by two authors (CAH, TS), and a third author (GLM) was conferred with when consensus could not be reached. Effect sizes for correlation coefficients were considered small (0.1–0.29), medium (0.3–0.49), and large (>0.5) as recommended by Cohen.⁸⁸ Sensitivity analyses were carried out with univariate outliers – CFI (Alternatives and Control subscales) or WCST (% perseverative responses) values that fell below Q1 - 1.5 × IQR or exceeded Q3 + 1.5 × IQR – removed, which was visually confirmed via box-plots.

Logistic regression

Logistic regression was used to explore the probability of the presence of persistent pain based on scores from a self-report and neuropsychological assessment of cognitive flexibility simultaneously. Predictor variables were centred and scaled based on sampled distributions of the observed values. A backward model reduction process was used, with non-significant model terms being removed in a stepwise manner.

The full initial logistic regression model was:

logit (p_{i}) = β_{0} + β_{1} ({CFIproportion}_{i}) + β_{2} ({WCSTproportion}_{i}) + β_{3} ({CFIproportion}_{i} \times {WCSTproportion}_{i})

Y_{i} \sim Bernoulli (p_{i})

where:

• $i = 1,2, . . ., 196$ is the participant number,

• $p_{i}$ is the probability of person $i$ having persistent pain, that is, $p_{i} = P ({group}_{i} =^{“} persistent {pain}^{”})$ where ${group}_{i}$ is the group ( $“ persistent pain ”$ or $“ healthy control ”$ ) of person $i$ and $P (x)$ is the probability of event $x$ ,

• $β_{0}$ is an intercept (the predicted value of $l o g i t (Y_{i})$ when all predictors are 0),

• $C F I p r o p o r t i o n_{i}$ is the CFI proportion of the total score for the $i^{t h}$ participant (centred and scaled),

• $W C S T p r o p o r t i o n_{i}$ is the WCST proportion of % perseverative responses for the $i^{t h}$ participant (centred and scaled), and

• $Y_{i}$ is the observed PP status of person $i$ , that is, $Y_{i} = 1$ if person $i$ is in the group PP and $Y_{i} = 0$ if person $i$ is not in the group PP (i.e. HC).

Results

Sample characteristics

A total of 241 participants (excluding duplicates) accessed the online survey between December 8th 2021 and April 9th 2022. Data from 196 participants were available for analysis (see Figure 2). Participant characteristics are summarised for each group separately using descriptive statistics in Table 1. Continuous variables are presented as values of central tendency (median) and dispersion (interquartile range). The minimum and maximum range of values for continuous variables are also provided. Dichotomous and categorical variables, as well as missing values, are presented as counts and percentages. Mental health conditions were broadly categorised by the primary investigator (CAH) using the Diagnostic Statistical Manual (DSM)-5-TR⁸⁹ Complex post-traumatic stress disorder and misophonia were not otherwise categorised because they are not formally recognised in the current version of the DSM.⁸⁹ Two experienced physiotherapists (GLM and CB) categorised pain diagnoses. Additional characteristics of the sample are provided in the Supplementary material (see Supplemental File 10).

Figure 2.

Participant flow diagram.

Table 1.

Participant characteristics (N = 196).

Characteristic	HC, (n = 68)	PP, (n = 128)
Age (in years)
Median (IQR)	36.0 (28.8, 44.2)	51.0 (38.0, 59.2)
Range	22–86	18–79
Sex
Female	48 (71%)	105 (83%)
Male	20 (29%)	21 (17%)
Missing	-	2
Gender
Female	48 (71%)	103 (80%)
Male	19 (28%)	20 (16%)
Non-binary and transgender individuals	1 (1.5%)	5 (3.9%)
Education level
Secondary school (i.e. year 12 or equivalent)	3 (4.4%)	16 (12%)
Tertiary education (e.g. trade qualification/Certificate/Diploma/Bachelor’s degree)	33 (49%)	72 (56%)
University higher degree (e.g. masters, PhD)	32 (47%)	40 (31%)
Medications
No	56 (82%)	63 (49%)
Yes	12 (18%)	65 (51%)
CFI (alternatives subscale)
Median (IQR)	74.0 (68.8, 78.0)	75.0 (69.0, 78.0)
Range	54–90	44–91
CFI (control subscale)
Median (IQR)	36.5 (31.0, 41.0)	35.0 (28.0, 40.0)
Range	14–49	8–49
CFI (total score)
Median (IQR)	110.0 (100.8, 117.2)	109.5 (99.0, 117.0)
Range	73–139	54–135
WCST perseverative responses (sum)
Median (IQR)	13.0 (7.0, 24.0)	18.5 (11.0, 27.0)
Range	3–76	3–120
WCST perseverative responses (%)
Median (IQR)	11.9 (8.2, 19.6)	16.1 (11.1, 22.1)
Range	4–59	3–94
WCST perseverative errors (sum)
Median (IQR)	12.0 (7.0, 23.0)	16.0 (10.0, 24.0)
Range	3 - 60	3 - 91
WCST perseverative errors (%)
Median (IQR)	11.2 (8.2, 18.0)	14.3 (10.3, 19.5)
Range	4–47	3–71
HADS-depression
Median (IQR)	3.0 (2.0, 6.0)	7.0 (4.0, 10.2)
Range	0–13	0–19
HADS-anxiety
Median (IQR)	6.0 (3.0, 8.0)	9.0 (5.0, 12.0)
Range	0–17	0–20
Pain duration (in months)
Median (IQR)	-	68 (24, 141)
Range	-	2–600
Missing	-	4
Pain intensity
Median (IQR)	-	6 (4, 7)
Range	-	1–10
Missing	-	-
Pain location (primary)
Abdominal	-	5 (4%)
Back	-	35 (27%)
Foot or leg	-	17 (13%)
Hand or arm	-	11 (9%)
Head	-	7 (5%)
Hip	-	10 (8%)
I find it difficult to identify my worst area of pain	-	24 (19%)
Neck	-	14 (11%)
Pelvic	-	5 (4%)
Pain location (secondary)
Abdominal	-	24 (19%)
Back	-	60 (47%)
Foot or leg	-	57 (45%)
Hand or arm	-	58 (45%)
Head	-	35 (27%)
Hip	-	64 (50%)
Neck	-	65 (51%)
Pelvic	-	29 (23%)
Unknown	-	1 (1%)
Pain diagnosis
Musculoskeletal	-	89 (70%)
Neuropathic	-	20 (16%)
Nociceptive	-	57 (45%)
Nociplastic	-	90 (70%)
Non-musculoskeletal	-	12 (9%)
Unknown	-	1 (1%)

Note: CFI = Cognitive Flexibility Inventory; HADS = Hospital Anxiety and Depression Scale; HC = healthy control; IQR = interquartile range; PP = persistent pain; WCST = Wisconsin Card Sorting Test. Values and percentages for pain location (secondary) and pain diagnosis do not add up to the total n for the persistent pain group (128 or 100%) because participants could report having more than one secondary location of pain or their diagnosis could be classified into more than one category.

Group comparisons for assessments of cognitive flexibility

Multiple linear regression models were used to assess group differences on two outcomes of cognitive flexibility (CFI and WCST proportion), after accounting for the effects of the aforementioned covariates. Diagnostic plots were inspected for each of the backward selection models and no major deviations from statistical assumptions were observed (see Supplemental File 11).

Tables 2 and 3 show the final model reached from the final backward selection process for each outcome variable – CFI proportion and WCST proportion – respectively. No significant differences were found between people with and without persistent pain, after adjusting for covariates, on either assessment of cognitive flexibility: CFI proportion or WCST proportion. As shown in Table 2, the final model included the estimated effect of group, education level, and depression and anxiety on the log-odds values of the CFI proportion. The results indicated that 27.3% of the variance was explained by the model (

R^{2}

= 0.273). Model comparisons revealed that the model goodness of fit (as assessed by the Akaike Information Criterion (AIC) value) improved during the backward selection process. No significant differences were found between groups on the log-odds scale of the CFI proportion (

β

= 0.15, 95% CI = −0.04, 0.35), after adjusting for education level and symptoms of depression and anxiety. This finding is consistent with the unadjusted median values (see Table 1). However, greater severity of anxiety symptoms was significantly associated with lower scores on the CFI proportion (

β

= −0.04, 95% CI = −0.06, −0.01), as was depressive symptoms (

β

= −0.05, 95% CI = −0.07, −0.02). Additionally, higher educational attainment was associated with higher CFI proportion scores. Specifically, tertiary education (

β

= 0.33, 95% CI = 0.03, 0.63) and higher degree education (

β

= 0.52, 95% CI = 0.21, 0.83) were significantly different, on average, from high school education. A sensitivity analysis, with univariate outliers – defined as CFI proportion values that were more than 1.5 × IQR below Q1 or more than 1.5 × IQR above Q3 – removed from the final regression model did not change the statistical significance.

Table 2.

Final regression model for the CFI (proportion of the total score).

Parameter	Coefficient	p-value	95% confidence interval
(Intercept)	1.22	.000	(0.87, 1.57)
Group (persistent pain)	0.15	.124	(−0.04, 0.35)
Education level (tertiary)	0.33	.031	(0.03, 0.63)
Education level (university higher degree)	0.52	.001	(0.21, 0.83)
HADS anxiety score	−0.04	.003	(−0.06, −0.01)
HADS depression score	−0.05	.001	(−0.07, −0.02)

Note: CFI = Cognitive Flexibility Inventory; HADS = Hospital Anxiety and Depression Scale.

Table 3.

Final regression model for the WCST (proportion of % perseverative responses).

Parameter	Coefficient	p-value	95% confidence interval
(Intercept)	−1.82	.000	(−2.06, −1.58)
Group (persistent pain)	0.18	.156	(−0.07, 0.43)
HADS anxiety score	−0.03	.044	(−0.06, −0.00)
HADS depression score	0.04	.011	(0.01, 0.08)

Note: HADS = Hospital Anxiety and Depression Scale; WCST = Wisconsin Card Sorting Test.

Table 3 shows the results of the same process applied to the WCST proportion log-odds outcome. The model was statistically significant (compared to an intercept model), but only explained 5.9% of the variance ( $R^{2}$ = 0.059). Model comparisons revealed that the model goodness of fit (AIC value) improved throughout the backward selection procedure. No significant differences were observed between groups on the WCST proportion log-odds scale ( $β$ = 0.18, 95% CI = −0.07, 0.43). As shown in Table 1, the unadjusted median values appear to be different between the two groups, but this may be due to confounders. Greater severity of anxiety symptoms was significantly associated with better WCST proportion scores ( $β$ = −0.03, 95% CI = −0.06, −0.00), whereas greater severity of depressive symptoms was associated with worse WCST proportion scores ( $β$ = 0.04, 95% CI = 0.01, 0.08). A sensitivity analysis, with univariate outliers – defined as WCST proportion values that were more than 1.5 $\times$ IQR below Q1 or more than 1.5 $\times$ IQR above Q3 – removed from the final regression model resulted in scores of anxiety and depression no longer contributing significantly to the model. Graphical representations of the estimates for the final model parameters, as well as boxplots indicating outliers, can be found in the Supplementary material (see Supplemental Files 12-14).

Correlations between assessments of cognitive flexibility

The panel (CAH, TS, and GLM), which assessed the assumptions of linearity and homoscedasticity deemed these assumptions to be violated by majority vote (see Supplemental File 15). Spearman’s rho correlation coefficients were used. No significant correlations were found between either of the CFI subscales (i.e. Alternatives and Control) and the WCST (% perseverative responses) in the persistent pain group, nor in the pain-free control group (see Table 4). Sensitivity analyses, removing univariate outliers that were defined as WCST (% perseverative responses) and CFI (Alternatives and Control subscales) values that were more than 1.5

\times

IQR below Q1 or more than 1.5

\times

IQR above Q3, did not change the results in the pain-free control group, nor in the persistent pain group. Boxplots indicating outliers can be found in the Supplementary material (see Supplemental File 16).

Table 4.

Spearman’s rho correlation coefficients between self-reported cognitive flexibility and neuropsychological cognitive flexibility.

Group	Variables	Rho	p-value (Holm-Sidak adjusted)	95% CI (Holm-Sidak adjusted)
Persistent pain	WCST (% perseverative responses) & CFI (alternatives)	−0.09	.337	(−0.26, 0.09)
Persistent pain	WCST (% perseverative responses) & CFI (control)	0.01	1.000	(−0.21, 0.22)
Healthy control	WCST (% perseverative responses) & CFI (alternatives)	−0.10	.818	(−0.36, 0.18)
Healthy control	WCST (% perseverative responses) & CFI (control)	−0.06	1.000	(−0.36, 0.27)

Note: CFI = Cognitive Flexibility Inventory; CI = confidence interval; rho = Spearman’s rho; WCST = Wisconsin Card Sorting Test.

The probability of having persistent pain based on assessments of cognitive flexibility

A logistic regression model was fitted to explore whether the probability of having persistent pain is associated with two scores on assessments of cognitive flexibility (CFI proportion score and WCST proportion score). A backward selection process indicated that the most parsimonious model was the intercept only model (

β

= 0.63, 95% CI = 0.342, 0.932) (see Table 5). That is, the probability of having persistent pain was not associated with CFI proportion scores, nor WCST proportion scores. Refer to Supplemental File 17 for diagnostic plots for the logistic regression model.

Table 5.

Model parameter estimates of each logistic regression model fit in the backward selection process.

Parameter	Model 1 (interaction)	Model 2 (main effects)	Model 3 (WCST model)	Model 4 (intercept model)
(Intercept)	0.65 (0.36, 0.96)	0.65 (0.36, 0.96)	0.64 (0.35, 0.95)	0.63 (0.34, 0.93)
WCST	0.26 (−0.07, 0.65)	0.25 (−0.07, 0.62)	0.26 (−0.06, 0.63)
CFI	−0.22 (−0.55, 0.1)	−0.2 (−0.51, 0.1)
WCST x CFI	−0.08 (−0.53, 0.22)

Statistics presented: coefficient estimate (95% confidence interval).

Note: CFI = Cognitive Flexibility Inventory; WCST = Wisconsin Card Sorting Test.

Discussion

Summary of findings

We investigated whether people with and without persistent pain differ in their ability to implement different cognitive and behavioural strategies when faced with changing environmental, or task demands. Contrary to our hypothesis, we found no evidence to suggest that people with persistent pain are different from pain-free controls on either a self-report questionnaire (i.e. CFI) or a neuropsychological test (i.e. WCST), after adjusting for covariates. However, we found that more severe symptoms of anxiety and depression were associated with lower scores on the CFI, and higher educational attainment was associated with higher CFI scores. This finding is consistent with prior research that has demonstrated that self-reported anxiety and depression are negatively related with self-reported cognitive flexibility.^{12,27,85,86,90} Additionally, once outliers were removed, we found no association between anxiety and depressive symptoms and performance on the WCST – a finding that is also consistent with previous research.⁹⁰

Our study supports and extends the work of Gelonch et al.²¹ who found that people diagnosed with fibromyalgia are not significantly different from pain-free controls on the Shift subscale of the BRIEF, after controlling for depression and anxiety. By contrast, Baker et al.³⁴ found that people with persistent pain reported impairments on the Shift subscale of the BRIEF relative to pain-free controls. However, unlike our study and that of Gelonch et al.²¹ Baker et al.³⁴ did not control for the effects of depression and anxiety, and thus it is possible that the observed group differences in their sample could have been largely attributable to negative affect. Indeed, an inverse relationship between self-reported cognitive flexibility assessments and symptoms of depression and anxiety has been identified in people with and without comorbid depression and anxiety,²⁷ in people with and without anorexia nervosa,⁹⁰ and in undergraduate students,¹² providing support for the potential influence of negative affect on self-reported cognitive flexibility. In line with our results, Gelonch et al.²¹ also found that people with fibromyalgia were not significantly different from pain-free controls on the WCST (perseverative errors) after controlling for negative affect. Our findings also echo a number of previous studies that have found no differences between people with and without persistent pain on various neuropsychological assessments of cognitive flexibility.^20,22–25 However, our findings do not support those of large-scale meta-analyses that have demonstrated that people with persistent pain have small-to-moderate impairments on neuropsychological tests that are used to assess cognitive flexibility.^28–31,91

So why then do previous meta-analyses show a small-to-moderate difference between people with and without persistent pain on neuropsychological tests that are commonly used to assess cognitive flexibility? A drawback of meta-analyses is that they, by default, rely on the included constituent studies to account for confounding factors (depression, medications etc.), which could potentially explain the discrepant findings between the current study and previous meta-analytical findings. It is possible that failing to adjust for important confounders in meta-analyses could give the false impression that people with persistent pain have an impaired ability to adapt and implement new strategies when task or environmental demands change, when in fact, these small-to-moderate effects could disappear once relevant confounding factors are controlled for. Further, despite power calculations for meta-analyses being recommended, they are rarely undertaken.⁹² Indeed, it remains possible that previous meta-analyses could be underpowered, which could result in an overestimation of the true effect size, and in turn, misleading conclusions.^92,93

We also aimed to investigate the strength and direction of the relationship between a self-report questionnaire and a neuropsychological assessment of cognitive flexibility in people with and without persistent pain. As expected, we did not find a significant relationship between our two chosen assessments – the CFI (Alternatives and Control subscales) and the WCST (% perseverative responses) in the persistent pain group, nor in the pain-free control group. Our findings corroborate previous research that indicates that self-report and neuropsychological assessments that purport to assess cognitive flexibility are poorly related. Specifically, Gelonch et al.²¹ found no association between a self-report questionnaire (Shift subscale of the BRIEF) and multiple neuropsychological assessments of cognitive flexibility (TMT B-A and WCST perseverative responses) in people with and without fibromyalgia. Our results also support previous studies that have found little-to-no overlap between self-report and neuropsychological assessments of cognitive flexibility in non-clinical^{27,90,94–96} and other clinical cohorts, including people with comorbid depression and anxiety,²⁷ attention deficit hyperactivity disorder,⁹⁵ anorexia nervosa^90,96 and obsessive-compulsive disorder.⁹⁷ Our group was the first to undertake two comprehensive meta-analyses that investigated the relationship between a range of self-report and neuropsychological assessments of cognitive flexibility. In both clinical³³ and non-clinical³² cohorts, we found little-to-no relationship between self-report and neuropsychological assessments of cognitive flexibility. However, a recent study with a large sample (n = 246) of non-clinical, younger adults reported a small, but significant relationship (r = −0.22) between the CFS and the WCST (perseverative errors).⁹⁸ One potential explanation as to why our results conflict with that of Grant et al.⁹⁸ may be because they used different outcomes and had a much larger (and non-clinical) sample.

The lack of association between self-report and neuropsychological assessments found here and elsewhere is not confined to the construct of cognitive flexibility, but also extends to the executive function literature.⁹⁹ While it has been argued that strong correlations would be detectable if self-report and neuropsychological assessments assess the same construct,⁹⁹ others have suggested that the non-correspondence between self-report and neuropsychological assessments should not be unexpected since the nature of the two assessments are fundamentally distinct – neuropsychological assessments often provide respondents with specific instructions, and thus are assumed to capture a person’s maximum, momentary ability, whereas self-report questionnaires are thought to capture how respondents perceive their typical behaviours.¹⁰⁰ This may be true, but it seems unlikely to be the only difference between the assessment approaches likely to contribute. Our final and exploratory aim was to determine whether people with and without persistent pain are distinguishable on the basis of their scores on both a self-report and a neuropsychological assessment. Our findings contradict that of Attal et al.³⁵ but are consistent with that of Vila et al.³⁶ because we found that the probability of having persistent pain (or not) was not associated with neuropsychological, nor self-reported cognitive flexibility.

Strengths and limitations

The strengths of this study include the preregistration of our protocol, as recommended in all pain research,¹⁰¹ our statistical analysis plan, our sufficiently powered sample, and that we controlled for confounders. Another strength of this study was that we limited the number of assessments and their respective outcomes, while also controlling for multiple comparisons where appropriate. Doing so addressed the shortcomings of previous studies and minimised the risk of Type I errors. Further, we adopted gender-inclusive research practices; we made the intentional decision to avoid categorising transgender and non-binary people into a single ‘other’ category because such categories reflect ‘othering’ of non-cisgendered people.^102,103 This study also has several limitations. First, IP addresses were not collected, so as to protect the anonymity of respondents. Therefore, it is possible that some participants may have completed the survey more than once using different participant ID codes, which may have resulted in duplicate data going unidentified. Although we cannot eliminate such a possibility, this scenario seems unlikely because participants did not receive incentives for their participation. Second, we did not exclude participants who were not naive to the WCST. Given that the WCST is susceptible to practice effects when re-administered within a 12-month time interval,^104,105 prior knowledge of the rules that underpin the WCST could potentially yield better scores that are not reflective of enhanced cognitive flexibility, but rather measurement error.¹⁰⁶ Thus, there is a risk that such a decision could have skewed our results. Third, although we used a computerised version of the WCST that scores perseverative responses and perseverative errors in accordance to the Heaton et al.¹⁴ manual, the scores obtained from computerised versions may not be psychometrically equivalent to the manual version.¹⁰⁷ Fourth, although the absence of a researcher during testing is likely to reduce elements of social desirability bias when completing the questionnaires, it meant that we were unable to monitor attention/distraction and other fidelity aspects when participants were undertaking the WCST. Fifth, the categorisation of medication use as a binary (yes/no) response has limitations – in particular, the response options did not allow us to capture dose or to examine whether certain medications have differential effects on the primary outcomes. We also did not assess or control for aspects of pain-related functioning (e.g. pain-related disability, quality of life or pain interference); future investigations using self-report or neuropsychological assessments to assess cognitive flexibility in people with persistent pain conditions should consider controlling for such factors to better understand the association between persistent pain and the ability to adapt cognitive and behavioural strategies in response to changing task or environmental demands. Sixth, we relied on a convenience sampling strategy, so it is possible that our sample, and in turn our findings, were influenced by selection bias,^108,109 and our sample was dominantly WEIRD (Western, Education, Industrialised, Rich and Democratic),¹¹⁰ which limits the generalisability of our results. Seventh, that the data were collected via an online survey meant that we had little control over the comparability of the characteristics between the two samples. In addition, we acknowledge that the sample size of the persistent pain group was larger than that of the healthy control group, but data collection could only be ceased once the minimum required sample size was reached for both groups. We recognise that such scenarios are limitations of conducting online survey studies. Nevertheless, we adhered to the STROBE guidelines,^38,39 which provide guidance on how to report sample characteristics in observational studies; the guidelines advise against using inferential statistics, including significance tests, to describe the variability of sample characteristics. Instead, we decided a priori which confounders would be included in our two regression models (informed by previous literature) and removed them using backward-stepwise selection – a widely accepted statistical approach¹¹¹ – until the most parsimonious models were reached. Lastly, we did not assess the validity or reliability of the items of the demographic questionnaire due to time constraints; however, the current study would have benefited from such an undertaking to ensure that the questions were precise enough to evoke consistent information from respondents.¹⁰⁹

Implications and considerations

Although there is sufficient evidence to support the notion that persistent pain is characterised by impairments in executive function more broadly,^29,112,113 the findings of the current study suggest that the ability to adapt to new strategies in the face of changing demands, which is considered to be one aspect of executive function,^8,114 may not be the component of executive function that is problematic in people with persistent pain. It remains possible that our chosen instruments were not sensitive enough to detect problems with self-reported and neuropsychologically assessed cognitive flexibility in people with persistent pain. However, that we found no difference between people with and without persistent pain on both self-report and neuropsychological assessments that purport to capture cognitive flexibility despite having a large sample size, controlling for multiple confounders and restricting the number of assessments and outcome variables used, seems to challenge previous assertions that claim that more sensitive tests or advanced methods need to be developed to assess subtle distinctions in cognitive functioning.¹¹⁵ Even if subtle impairments on the CFI and WCST do exist between people with and without persistent pain and more sensitive tests are required to detect such impairments, the effects are probably too small to be clinically meaningful.

It is also important to consider the drawbacks of both self-report and neuropsychological assessments of ‘cognitive flexibility’. Self-report questionnaires that are designed to capture cognitive flexibility are likely influenced by the respondent’s capacity to be self-aware and report the information in an accurate and non-biased manner. Similarly, a notorious problem of neuropsychological tests of ‘cognitive flexibility’ is that they inevitably rely on other executive and non-executive processes, which makes interpreting outcomes of such tasks difficult and rather non-specific.^116,117 Although previous studies that have used the WCST report no differences in the number of perseverative errors between people with and without persistent pain conditions,^24,25 the findings of other neuropsychological tests that are thought to assess cognitive flexibility – such as the TMT – are much harder to reconcile. Some studies have identified impaired performance on part B of the TMT in people with persistent low back pain¹⁹ and fibromyalgia¹⁸ relative to pain-free controls, while other studies have found no differences between people with and without persistent pain.^20,24,26 A potential explanation for these contrasting findings may be because the TMT – part B is fundamentally different from the WCST even though they are both used to assess the same construct of ‘cognitive flexibility’. For instance, the TMT – part B requires respondents to alternate between an alpha-numeric sequence (1-A-2-B-3-C etc.) as quickly as possible, whereas the WCST requires respondents to adapt to changing rules during card-sorting. Further, the outcome variables of the WCST and TMT are different: the TMT – part B is typically reported as time in seconds, whereas cognitive flexibility targeting outcomes of the WCST are most often presented as total sum or percentage scores (i.e. perseverative responses or perseverative errors). Although it has been suggested that part B of the TMT reflects, at least in part, cognitive flexibility,¹¹⁸ the influence of baseline motor speed, which is calculated by subtracting part A from part B,¹¹⁹ is seldom controlled for. Even when it is, difference scores do not produce adequate reliability coefficients.^120,121 For these reasons, future research may wish to steer anyway from relying solely on self-report and neuropsychological assessments to uncover problems in executive function and move towards using more ecologically valid tasks (e.g. the Virtual Cooking Task¹²² or the Highway Driving Test¹²³) that require individuals to perform tasks that are relevant to everyday life. Doing so could improve functional outcomes for people with persistent pain; however, it should be noted that such tasks need to be specifically developed and validated for people with persistent pain before their implementation.

It is important to note that our findings are not immediately applicable to what is termed psychological flexibility, which is the main construct targeted by Acceptance and Commitment Therapy (ACT).¹²⁴ Cognitive and psychological flexibility originated from different theoretical backgrounds; cognitive flexibility arose out of the neuropsychology field, whereas psychological flexibility has its roots in behavioural psychology, mindfulness and acceptance-based techniques. While there is some empirical evidence that self-reported cognitive flexibility (as assessed via the CFS) and self-reported psychological flexibility (as assessed via the Acceptance and Action Questionnaire-II (AAQ-II)) are related,⁹⁸ the specific construct that underpins the AAQ-II has recently been put into question.^125,126 Thus, our findings cannot be applied to the construct of psychological flexibility. It is also important to note that although ACT is often used as an intervention to enhance psychological flexibility and facilitate improvement of pain-related functioning in people with persistent pain,¹²⁷ currently no data exist on whether people with and without persistent pain differ on outcomes of psychological flexibility. Additionally, learning about key pain concepts – that is, reconceptualising or changing the perception of pain – is valued by people who have recovered from persistent pain.^128,129 Clinically, educational interventions that aim to change one’s understanding of pain from a marker of tissue injury to a marker of the need for perceived protection are part of evidence-based pain management.¹³⁰ Set-shifting – conceptualised as a core component of cognitive flexibility¹³¹ – is impaired in people with persistent pain.²⁹ Poor cognitive flexibility may play a role in maintaining pain by undermining one’s capacity to reconceptualise the problem, or shift sets. However, we found no evidence to support the notion that people with persistent pain have poorer cognitive flexibility, leading us to suggest that other components of executive function such as decision making or working memory might play a role in the successful reconceptualisation of the meaning of pain, but future studies are needed.

Conclusion

We found no evidence that people with persistent pain and pain-free controls differ on self-report and neuropsychological assessments that purport to assess ‘cognitive flexibility’. Our study did not find evidence to support the notion that people with persistent pain are less effective than pain-free controls at adapting their cognitive and behavioural strategies when environmental or task demands change. Despite evidence that executive function is compromised in people with persistent pain, our results suggest that ‘cognitive flexibility’ is not how that compromise occurs. However, other confounding factors need to be considered in future research and ecologically valid assessments of ‘cognitive flexibility’ specifically, and executive function, more broadly, will need to be developed before definitive conclusions can be drawn.

Supplemental Material

Supplemental Material - Investigating self-report and neuropsychological assessments of cognitive flexibility in people with and without persistent pain: An online, cross-sectional, observational study

Supplemental Material for Investigating self-report and neuropsychological assessments of cognitive flexibility in people with and without persistent pain: An online, cross-sectional, observational study by Caitlin A Howlett, Tyman Stanford, Carolyn Berryman, Emma L Karran, Valeria Bellan, Scott Coussens, Stephanie Miles, and G Lorimer Moseley in British Journal of Pain.

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: GLM has received support from California Institutes of Health; Reality Health; ConnectHealth UK; Workers’ Compensation Boards in Australia, Europe and North America; and AIA Australia. Professional and scientific bodies have reimbursed him for travel costs related to presentation of research at scientific conferences/symposia. He has received speaker fees for lectures on pain and rehabilitation. He receives book royalties from NOIgroup publications, Dancing Giraffe Press & OPTP. CB has received support from Workers’ Compensation Boards in Australia and Kaiser Permanente. All remaining authors (CAH, TS, ELK, VB, SC, and SM) declare no potential conflicts of interest (personal or financial) with respect to the research, authorship, or publication of this manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: CB, ELK and GLM were supported by a National Health and Medical Research Council (NHMRC) Leadership Investigator Grant [ID 1178444] to GLM. CB was supported by a NHMRC Early Career Fellowship [ID 1127155]. ELK was supported by a project grant from Lifetime Support Authority of South Australia. These funding bodies had no role in the study design, collection, analysis or interpretation of the data, writing of the manuscript, or the decision to submit the manuscript for publication.

ORCID iD

Caitlin A Howlett

Supplemental Material

Supplemental material for this article is available online.

References

Treede

R-D

Rief

Barke

, et al. A classification of chronic pain for ICD-11. Pain 2015; 156: 1003–1007.

Merskey

Bogduk

. Classification of chronic pain: descriptions of chronic pain syndromes and definitions of pain terms. 2nd ed. Seattle, WA, USA: IASP Press, 1994.

James

Abate

, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018; 392: 1789–1858.

Pitcher

Von Korff

Bushnell

, et al. Prevalence and profile of high-impact chronic pain in the United States. J Pain 2019; 20: 146–160.

Sá

Moreira

Baptista

, et al. Prevalence of chronic pain in developing countries: Systematic review and meta-analysis. Pain Reports 2019; 4: e779.

Yong

Mullins

Bhattacharyya

. Prevalence of chronic pain among adults in the United States. Pain 2022; 163: e328–e332.

Hadi

McHugh

Closs

. Impact of chronic pain on patients’ quality of life: A comparative mixed-methods study. J Patient Exp 2019; 6: 133–141.

Dajani

Uddin

. Demystifying cognitive flexibility: Implications for clinical and developmental neuroscience. Trends Neurosci 2015; 38: 571–578.

Cañas

Quesada

Antolí

, et al. Cognitive flexibility and adaptability to environmental changes in dynamic complex problem-solving tasks. Ergonomics 2003; 46: 482–501.

10.

Scott

. Cognitive complexity and cognitive flexibility. Sociometry 1962; 25: 405–414.

11.

Martin

Rubin

. A new measure of cognitive flexibility. Psychol Rep 1995; 76: 623–626.

12.

Dennis

Vander Wal

. The Cognitive Flexibility Inventory: Instrument development and estimates of reliability and validity. Cognit Ther Res 2010; 34: 241–253.

13.

Roth

Isquith

Gioia

. Behavior rating inventory of executive function – adult version. Lutz, Florida: Psychological Assessment Resources, Inc, 2005.

14.

Heaton

Chelune

Talley

, et al. Wisconsin card sorting test manual: revised and expanded. Lutz, FL: Psychological Assessment Resources, 1993.

15.

Reitan

. The relation of the Trail Making Test to organic brain damage. J Consult Psychol 1955; 19: 393–394.

16.

Stroop

. Studies of interference in serial verbal reactions. J Exp Psychol 1935; 18: 643–662.

17.

Lee

D-H

Lee

K-J

Cho

KIK

, et al. Brain alterations and neurocognitive dysfunction in patients with complex regional pain syndrome. J Pain 2015; 16: 580–586.

18.

Tesio

Torta

DME

Colonna

, et al. Are fibromyalgia patients cognitively impaired? Objective and subjective neuropsychological evidence. Arthritis Care Res 2015; 67: 143–150.

19.

Weiner

Rudy

Morrow

, et al. The relationship between pain, neuropsychological performance, and physical function in community-dwelling older adults with chronic low back pain. Pain Med 2006; 7: 60–70.

20.

Cherry

Zettel-Watson

Shimizu

, et al. Cognitive performance in women aged 50 years and older with and without fibromyalgia. J Gerontol B Psychol Sci Soc Sci 2014; 69: 199–208.

21.

Gelonch

Garolera

Valls

, et al. Executive function in fibromyalgia: Comparing subjective and objective measures. Compr Psychiatr 2016; 66: 113–122.

22.

Moriarty

Ruane

O’Gorman

, et al. Cognitive impairment in patients with chronic neuropathic or radicular pain: An interaction of pain and age. Front Behav Neurosci 2017; 11: 100.

23.

Nadar

Jasem

Manee

. The cognitive functions in adults with chronic pain: A comparative study. Pain Res Manag 2016; 2016: 1–8.

24.

Suhr

. Neuropsychological impairment in fibromyalgia: Relation to depression, fatigue, and pain. J Psychosom Res 2003; 55: 321–329.

25.

Verdejo-Garcia

Lopez-Torrecillas

Calandre

, et al. Executive function and decision-making in women with fibromyalgia. Arch Clin Neuropsychol 2009; 24: 113–122.

26.

Walitt

Roebuck-Spencer

Bleiberg

, et al. Automated neuropsychiatric measurements of information processing in fibromyalgia. Rheumatol Int 2008; 28: 561–566.

27.

Johnco

Wuthrich

Rapee

. Reliability and validity of two self-report measures of cognitive flexibility. Psychol Assess 2014; 26: 1381–1387.

28.

Bell

Trost

Buelow

, et al. Meta-analysis of cognitive performance in fibromyalgia. J Clin Exp Neuropsychol 2018; 40: 698–714.

29.

Berryman

Stanton

Bowering

, et al. Do people with chronic pain have impaired executive function? A meta-analytical review. Clin Psychol Rev 2014; 34: 563–579.

30.

Kessels

RPC

Aleman

Verhagen

WIM

, et al. Cognitive functioning after whiplash injury: A meta-analysis. J Int Neuropsychol Soc 2000; 6: 271–278.

31.

Rathbone

Parkinson

Rehman

, et al. Magnitude and variability of effect sizes for the associations between chronic pain and cognitive test performances: A meta-analysis. Br J Pain 2016; 10: 141–155.

32.

Howlett

Wewege

Berryman

, et al. Same room - different windows? A systematic review and meta-analysis of the relationship between self-report and neuropsychological tests of cognitive flexibility in healthy adults. Clin Psychol Rev 2021; 88: 102061.

33.

Howlett

Wewege

Berryman

, et al. Back to the drawing board – the relationship between self-report and neuropsychological tests of cognitive flexibility in clinical cohorts. A systematic review and meta-analysis. Neuropsychology 2022; 36: 347–372.

34.

Baker

Gibson

Georgiou-Karistianis

, et al. Everyday executive functioning in chronic pain: Specific deficits in working memory and emotion control, predicted by mood, medications, and pain interference. Clin J Pain 2016; 32: 673–680.

35.

Attal

Masselin-Dubois

Martinez

, et al. Does cognitive functioning predict chronic pain? Results from a prospective surgical cohort. Brain 2014; 137: 904–917.

36.

Vila

Todorovic

Tang

, et al. Cognitive flexibility and persistent post-surgical pain: The FLEXCAPP prospective observational study. Br J Anaesth 2020; 124: 614–622.

37.

Eysenbach

. Improving the quality of web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004; 6: e34.

38.

Vandenbroucke

von Elm

Altman

, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. Int J Surg 2014; 12: 1500–1524.

39.

von Elm

Altman

Egger

, et al. Strengthening the Reporting of Observational studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet 2007; 370: 1453–1457.

40.

Tchanturia

Davies

Roberts

, et al. Poor cognitive flexibility in eating disorders: Examining the evidence using the Wisconsin Card Sorting Task. PLoS One 2012; 7: e28331.

41.

Millisecond Software . Inquisit 6. Seattle, WA: Millisecond Software. https://www.millisecond.com (2021).

42.

Miles

Howlett

Berryman

, et al. Considerations for using the Wisconsin Card Sorting Test to assess cognitive flexibility. Behav Res Methods 2021; 53: 2083–2091.

43.

Stern

. The Hospital Anxiety and Depression Scale. Occup Med 2014; 64: 393–394.

44.

Zigmond

Snaith

. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983; 67: 361–370.

45.

Bjelland

Dahl

Haug

, et al. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res 2002; 52: 69–77.

46.

Wilod Versprille

LJF

van de Loo

AJAE

Mackus

, et al. Development and validation of the Immune Status Questionnaire (ISQ). Int J Environ Res Publ Health 2019; 16: 4743.

47.

Greve

Stickle

Love

, et al. Latent structure of the Wisconsin Card Sorting Test: A confirmatory factor analytic study. Arch Clin Neuropsychol 2005; 20: 355–364.

48.

Arnold

Hogan

Colford

, et al. Simulation methods to estimate design power: An overview for applied research. BMC Med Res Methodol 2011; 11: 94.

49.

Green

MacLeod

CSIMR

. An R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol 2016; 7: 493–498.

50.

Kontopantelis

Springate

Parisi

, et al. Simulation-based power calculations for mixed effects modeling: ipdpower in Stata. J Stat Software 2016; 74: 1–25.

51.

Faul

Erdfelder

Lang

A-G

, et al. G * Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007; 39: 175–191.

52.

Faul

Erdfelder

Buchner

, et al. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behav Res Methods 2009; 41: 1149–1160.

53.

Champely

. pwr: Basic functions for power analysis. R package version 1.3-0, 2020. https://cran.r-project.org/package=pwr

54.

R Core Team . R: A language and environment for statistical computing. Vienna, Austria: R Core Team. https://cran.r-project.org/ (2022).

55.

Wickham

Hester

Bryan

. readr: Read rectangular text data. R package version 2.1.2, 2022. https://cran.r-project.org/package=readr

56.

Wickham

Averick

Bryan

, et al. Welcome to the tidyverse. J Open Source Softw 2019; 4: 1686.

57.

Firke

. janitor: Simple tools for examining and cleaning dirty data. R package version 2.2.0, 2023. https://cran.r-project.org/package=janitor.

58.

Wickham

. ggplot2: Elegant graphics for data analysis. New York: Springer-Verlag. https://ggplot2.tidyverse.org (2016).

59.

Kassambara

. ggpubr: ‘ggplot2’ based publication ready plots. R package version 0.4.0, 2020. https://cran.r-project.org/package=ggpubr

60.

Patil

. Visualizations with statistical details: The ‘ggstatsplot’ approach. J Open Source Softw 2021; 6: 3167.

61.

Long

. jtools: Analysis and presentation of social scientific data. R package version 2.2.0, 2022. https://cran.r-project.org/package=jtools

62.

Lüdecke

Ben-Shachar

Patil

, et al. performance: An R package for assessment, comparison and testing of statistical models. J Open Source Softw 2021; 6: 3139.

63.

Arnold

. ggthemes: extra themes, scales and geoms for ‘ggplot2’. R package version 4.2.4, 2021. https://cran.r-project.org/package=ggthemes

64.

Ram

Wickham

. wesanderson: a Wes Anderson palette generator. R package version 0.3.6, 2018. https://cran.r-project.org/package=wesanderson

65.

Okabe

Ito

. Color universal design (CUD): how to make figures and presentations that are friendly to colorblind people, 2008. https://jfly.uni-koeln.de/color/

66.

McWhite

Wilke

. colorblindr: simulate colorblindness in R figures, 2022. https://github.com/clauswilke/colorblindr

67.

Sjoberg

Whiting

Curry

, et al. Reproducible summary tables with the gtsummary package. R J 2021; 13: 570–580.

68.

Makowski

Ben-Shachar

Patil

, et al. Automated results reporting as a practical tool to improve reproducibility and methodological best practices adoption, 2020. https://easystats.github.io/report/

69.

Anderson

Heiss

Sumners

. equatiomatic: transform models into ‘LaTeX’ equations. R package version 0.3.1, 2022. https://cran.r-project.org/package=equatiomatic

70.

Ooms

. writexl: export data frames to Excel ‘xlsx’ format. R package version 1.4.2, 2023. https://cran.r-project.org/package=writexl

71.

Waring

Quinn

McNamara

, et al. skimr: compact and flexible summaries of data. R package version 2.1.5, 2022. https://cran.r-project.org/package=skimr

72.

Fox

Weisberg

. An R companion to applied regression. 3rd ed. Newcastle upon Tyne, UK: Sage. https://socialsciences.mcmaster.ca/jfox/Books/Companion/ (2019).

73.

Lüdecke

Ben-Shachar

Patil

, et al. Extracting, computing and exploring the parameters of statistical models using R. J Open Source Softw 2020; 5: 2445.

74.

The Trans Language Primer, https://translanguageprimer.com (accessed 30 June 2022).

75.

Hastie

Tibshirani

Friedman

. Model assessment and selection. In: The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer, 2009, pp. 219–259.

76.

McCarthy

Ceccucci

. Predictive models using regression. In: Applying predictive analytics. Cham: Springer, 2022, pp. 87–121.

77.

Boone

Ghaffarian

Lesser

, et al. Wisconsin Card Sorting Test performance in healthy, older adults: Relationship to age, sex, education, and IQ. J Clin Psychol 1993; 49: 54–60.

78.

Rhodes

. Age-related differences in performance on the Wisconsin Card Sorting Test: A meta-analytic review. Psychol Aging 2004; 19: 482–494.

79.

McDermott

Ebmeier

. A meta-analysis of depression severity and cognitive function. J Affect Disord 2009; 119: 1–8.

80.

Kurita

De Mattos Pimenta

Braga

, et al. Cognitive function in patients with chronic pain treated with opioids: Characteristics and associated factors. Acta Anaesthesiol Scand 2012; 56: 1257–1266.

81.

Prado

Watt

Crowe

. A meta-analysis of the effects of antidepressants on cognitive functioning in depressed and non-depressed samples. Neuropsychol Rev 2018; 28: 32–72.

82.

Khera

Rangasamy

. Cognition and pain: A review. Front Psychol 2021; 12: 1–1.

83.

Higgins

Martin

Baker

, et al. The relationship between chronic pain and neurocognitive function: A systematic review. Clin J Pain 2018; 34: 262–275.

84.

Wang

Zhang

Wiley

, et al. Gender differences in pleasure: The mediating roles of cognitive flexibility and emotional expressivity. BMC Psychiatr 2022; 22: 1–8.

85.

Wang

, et al. Relations between trait anxiety and depression: A mediated moderation model. J Affect Disord 2019; 244: 217–222.

86.

Lin

. Anxiety and depression aggravate impulsiveness: The mediating and moderating role of cognitive flexibility. Psychol Health Med 2020; 25: 25–36.

87.

Holm

. A simple sequentially rejective multiple test procedure. Scand J Stat 1979; 6: 65–70.

88.

Cohen

. Quantitative methods in psychology: A power primer. Psychol Bull 1992; 112: 155–159.

89.

American Psychiatric Association . Diagnostic and statistical manual of mental disorders. 5th ed. Washington, DC: American Psychiatric Association, 2022. DOI: 10.1176/appi.books.9780890425787.

90.

Lounes

Khan

Tchanturia

. Assessment of cognitive flexibility in anorexia nervosa - self-report or experimental measure? A brief report. J Int Neuropsychol Soc 2011; 17: 925–928.

91.

Bunk

Preis

Zuidema

, et al. Executive functions and pain. Zeitschrift für Neuropsychol 2019; 30: 169–196.

92.

Griffin

. Calculating statistical power for meta-analysis using metapower. Quant Methods Psychol 2021; 17: 24–39.

93.

Thorlund

Imberger

Walsh

, et al. The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis - A simulation study. PLoS One 2011; 6: e25491.

94.

Gonzalez

Figueroa

Bellows

, et al. A new behavioral measure of cognitive flexibility. In: Harris

(ed). Engineering psychology and cognitive ergonomics. Understanding human cognition. EPCE 2013. Lecture notes in computer science, vol 8019. Berlin, Heidelberg: Springer, 2013, pp. 297–306.

95.

Kercood

Lineweaver

Frank

, et al. Cognitive flexibility and its relationship to academic achievement and career choice of college students with and without attention deficit hyperactivity disorder. J Postsecond Educ Disabil 2017; 30: 329–344.

96.

Miles

Nedeljkovic

Sumner

, et al. Understanding self-report and neurocognitive assessments of cognitive flexibility in people with and without lifetime anorexia nervosa. Cognit Neuropsychiatry 2022; 27: 325–341.

97.

Sternheim

van Passel

Dingemans

, et al. Cognitive and experienced flexibility in patients with anorexia nervosa and obsessive compulsive disorder. Front Psychiatr 2022; 13: 868921.

98.

Grant

Cassidy

. Exploring the relationship between psychological flexibility and self-report and task-based measures of cognitive flexibility. J Context Behav Sci 2022; 23: 144–150.

99.

Toplak

West

Stanovich

. Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? JCPP (J Child Psychol Psychiatry) 2013; 54: 131–143.

100.

Wennerhold

Friese

. Why self-report measures of self-control and inhibition tasks do not substantially correlate. Collabra Psychol 2020; 6: 9.

101.

Lee

Lamb

Bagg

, et al. Reproducible and replicable pain research: A critical review. Pain 2018; 159: 1683–1689.

102.

Camerson

Stinson

. Gender (mis)measurement: Guidelines for respecting gender diversity in psychological research. Soc Personal Psychol Compass 2019; 13: e12506.

103.

Fraser

. Evaluating inclusive gender identity measures for use in quantitative psychological research. Psychol Sex 2018; 9: 343–357.

104.

Basso

Bornstein

Lang

. Practice effects on commonly used measures of executive function across twelve months. Clin Neuropsychol 1999; 13: 283–292.

105.

Basso

Lowery

Ghormley

, et al. Practice effects on the Wisconsin Card Sorting Test-64 card version across 12 months. Clin Neuropsychol 2001; 15: 471–478.

106.

McCaffrey

Westervelt

. Issues associated with repeated neuropsychological assessments. Neuropsychol Rev 1995; 5: 203–221.

107.

Steinmetz

Brunner

Loarer

, et al.

Incomplete psychometric equivalence of scores obtained on the manual and the computer version of the Wisconsin Card Sorting Test?

Psychol Assess 2010; 22: 199–202.

108.

Alessi

Martin

. Conducting an Internet-based survey: Benefits, pitfalls, and lessons learned. Soc Work Res 2010; 34: 122–128.

109.

Ball

. Conducting online surveys. J Hum Lactation 2019; 35: 413–417.

110.

Gosling

Sandy

John

, et al. Wired but not WEIRD: The promise of the Internet in reaching more diverse samples. Behav Brain Sci 2010; 33: 94–95.

111.

Hastie

Tibshirani

Friedman

. Linear methods for regression. In: The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer, 2009, pp. 58–60.

112.

Berryman

Stanton

Bowering

, et al. Evidence for working memory deficits in chronic pain: A systematic review and meta-analysis. Pain 2013; 154: 1181–1196.

113.

Moriarty

McGuire

Finn

. The effect of pain on cognitive function: A review of clinical and preclinical research. Prog Neurobiol 2011; 93: 385–404.

114.

Diamond

. Executive functions. Annu Rev Psychol 2013; 64: 135–168.

115.

McGuire

. Chronic pain and cognitive function. Pain 2013; 154: 964–965.

116.

Miyake

Friedman

Emerson

, et al. The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cogn Psychol 2000; 41: 49–100.

117.

Müller

Kerns

. The development of executive function. In: Liben

Müller

Lerner

(eds). Handbook of child psychology and developmental science: Cognitive processes. Hoboken, NJ: John Wiley & Sons, Inc., 2015, pp. 571–623.

118.

Kortte

Horner

Windham

. The Trail Making Test, Part B: Cognitive flexibility or ability to maintain set? Appl Neuropsychol 2002; 9: 106–109.

119.

Vall

Wade

. Trail Making Task performance in inpatients with anorexia nervosa and bulimia nervosa. Eur Eat Disord Rev 2015; 23: 304–311.

120.

Kopp

. Neuropsychologist must keep their eyes on the reliability of difference measures. J Int Neuropsychol Soc 2011; 17: 562–563.

121.

Zorowitz

Niv

. Improving the reliability of cognitive task measures: A narrative review. Biol Psychiatry Cogn Neurosci Neuroimaging 2023; 8: 789–797.

122.

Chicchi Giglioli

Gálvez

Granados

, et al. The virtual cooking task: A preliminary comparison between neuropsychological and ecological virtual reality tests to assess executive functions alterations in patients affected by alcohol use disorder. Cyberpsychol, Behav Soc Netw 2021; 24: 673–682.

123.

Veldhuijzen

van Wijck

AJM

Willie

, et al. Effect of chronic nonmalignant pain on highway driving performance. Pain 2006; 122: 28–35.

124.

Hayes

Luoma

Bond

, et al. Acceptance and commitment therapy: Model, processes and outcomes. Behav Res Ther 2006; 44: 1–25.

125.

Tyndall

Waldeck

Pancani

, et al. The Acceptance and Action Questionnaire-II (AAQ-II) as a measure of experiential avoidance: Concerns over discriminant validity. J Context Behav Sci 2019; 12: 278–284.

126.

Wolgast

. What does the Acceptance and Action Questionnaire (AAQ-II) really measure? Behav Ther 2014; 45: 831–839.

127.

Wicksell

Kemani

Jensen

, et al. Acceptance and commitment therapy for fibromyalgia: A randomized controlled trial. Eur J Pain 2013; 17: 599–611.

128.

Leake

Moseley

Stanton

, et al. What do patients value learning about pain? A mixed-methods survey on the relevance of target concepts after pain science education. Pain 2021; 162: 2558–2568.

129.

Leake

Mardon

Stanton

, et al. Key learning statements for persistent pain education: An iterative analysis of consumer, clinician and researcher perspectives and development of public messaging. J Pain 2022; 23: 1989–2001.

130.

Moseley

Butler

. Fifteen years of explaining pain: The past, present and future. J Pain 2015; 16: 807–813.

131.

Tchanturia

Morris

Anderluh

, et al. Set shifting in anorexia nervosa: An examination before and after weight gain, in full recovery and relationship to childhood and adult OCPD traits. J Psychiatr Res 2004; 38: 545–552.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.88 MB