Predicting normative data in healthy individuals on the computerized wisconsin card sorting test using regression models

Abstract

BACKGROUND:

Computerized neuropsychological tests provide advantages to clinicians with cost, administration, and time. However, studies have pointed out performance differences between manual and computerized versions of some neuropsychological tests. One of these is the Wisconsin Card Sorting Test (WCST). Due to the performance difference, the normative data of manual tests cannot be used for their computerized versions. Therefore, normative data searches are needed for computerized versions.

OBJECTIVE:

This study aimed to determine the norm values of WCST-CV in a healthy sample.

METHODS:

422 healthy adults aged 18–78 participated in this study. WCST-CVsub-scores are modeled by Regression Analysis based on Age and Education level to generate normative data. Among the 13 WCST scores, the regression models for WCST 2, WCST 3, WCST 4, WCST 10, and WCST 11 are significant. WCST 2, WCST 4, and WCST 11 scores are estimated with Ordinary Least Squares (OLS). However, WCST 3 and WCST 10 scores are estimated with Weighted Least Squares (WLS) due to the violation of the homoscedasticity assumption.

RESULTS:

The regression results show that p-values calculated from error increase as age and education level increase.

CONCLUSION:

As a result of our research, norm values between 18–78 years of age were produced using RA. It was determined that gender was not significant for any sub-score. Therefore, only age and education level from socio-demographic variables were included in the model.

Keywords

Regression modelling WCST-CV Turkish population normative data neuropsychological tests

1 Introduction

Neuropsychological tests have been used for a long time in the investigation of cognitive profiles occurring in healthy and neuropsychiatric disorders. Revealing these profiles gives information about the “normal-healthy” cognitive structure and helps us diagnose and follow the rehabilitation process for different mental and neurological disorders (Kessels & Hendriks, 2016; Nelson & Searl, 2002; Shan et al., 2008).

The Wisconsin Card Sorting Test (WCST) has been one of the most frequently used tests in evaluating frontal functions. To succeed in the WCST, an individual must employ many cognitive abilities concurrently, including attention, set shifting, perseveration, cognitive flexibility, inhibition of impulsive responses, problem-solving, and planning (Heaton et al., 1993). Hence, the WCST exhibits a considerably more intricate structure than other neuropsychological tests. When a deficit in cognitive abilities results from a mental or neurological disorder, there is a substantial decline in test performance compared to healthy controls (Gillan et al., 2020; Itzhaky et al., 2023; Petkus et al., 2020; Tercero et al., 2021; Venezia et al., 2018).

Psychometrically, different methods exist to compare individuals with their peers regarding specific cognitive abilities. Percentiles (Atalay & Cinan, 2007; Faustino et al., 2022) and Z and T transform scales created from Classical Test Theory are among the most common methods (Crawford, 2003; Guàrdia-Olmos et al., 2015). However, the most significant limitation of these methods is the need for a gold standard for classifying socio-demographic characteristics, especially education level and age, which are known to affect cognitive performance (Miranda et al., 2019; Rammal et al., 2019). WCST performance changes in different age groups due to age-related cognitive declines and individual differences such as education level and gender (Rhodes, 2004; Rammal et al., 2019). These variations highlight the need for age-appropriate norms and consider multiple factors when interpreting WCST results across different age groups. In this respect, the most common application is to categorize the sample arbitrarily according to age and education level and extract norm tables while creating norm values (Karakaş et al., 1999; Salthouse, 2001; Salthouse et al., 2003). However, this method can lead to wrong decisions about the individual’s cognitive performance; it does not allow us to know to what extent age and education level affect test performance. Another limitation is that age does not affect cognitive performance at the same level in every age range. In other words, while the decrease in performance in a test decreases with a particular slope, it can show a dramatic decrease after a specific age. Especially in neurodegenerative disorders, this situation is more evident as cognitive decline decreases after 65 years of age. To address these limitations, this study explores the application of regression models in cognitive assessment, drawing on recent advances in the field (Arango-Lasprilla et al., 2015; Guàrdia-Olmos et al., 2015). Using regression models also helps eliminate floor and ceiling effects often encountered in norm tables (Hartshorne & Germine, 2016).

1.1 Computerized WCST

Developing computer versions of psychological tests provides clinicians with many advantages. These advantages i) saving time, ii) minimizing errors, iii) being cheaper in the long term. However, studies show that the computer and manual versions of the WCST do not exhibit similar psychometric properties in both psychiatric and healthy samples (Çelik et al., 2021). Therefore, it is necessary to create norm tables like manual versions of computerized tests. This study aimed to generate predicting normative data for healthy individuals aged 18 –78 using regression models.

2 Methods

2.1 Participants

Data were gathered from a comprehensive sample comprising 478 adult participants. The scores derived from 47 individuals on the Beck Depression and Anxiety Scales were omitted from the dataset because they exceeded the predefined cut-off scores. Furthermore, 9 participants were excluded from the research analysis, as they discontinued their engagement with the assessment scales and testing procedures during the data collection phase. Consequently, the research was carried out, and subsequent analyses were performed utilizing the dataset derived from the remaining cohort of 422 ostensibly healthy adults aged between 18–78. The average age of the participants is 44.05±12.04. 59.5% (n = 251) of the participants were male, and the rest were female. In our study, the education year was taken into account as a continuous variable, and the average was found to be 11.08±4.34. In case of any neurological or psychiatric disease reported by the participant or noticed by the psychiatrist, the participant was not included in the study. In addition, participants with long-term alcohol substanceusage and taking drugs that may affect cognitive processes in the last month were excluded from the study. Written informed consent was obtained from all participants who voluntarily participated in the study. Ethics committee approval of the study was received by Bartın University Social and Human Sciences Ethics Committee (Approval number: 2021-SBB-0489; Approval date: 15.12.2021).

2.2 Measures and procedures

A structured survey instrument was employed to systematically collect socio-demographic data from study participants, encompassing critical variables including age, educational level, and gender. This comprehensive data-gathering instrument consisted of inquiries aimed at eliciting pertinent socio-demographic details, preferred handedness, the presence of psychiatric and neurological disorders, and an exhaustive medical history.

The Beck Depression and Beck Anxiety Scales were used for all participants to determine their current depression and anxiety levels. Those with scores above 9 and 8 on these scales, respectively, were excluded from the research (Hisli, 1988; Ulusoy et al., 1998). Similarly, the Montreal Cognitive Assessment Test (MOCA) was used to screen cognitive skills and show that especially older individuals are not at cognitive risk. Participants who scored less than 21 points on the MOCA were also excluded due to poor cognitive skills (Ozdilek & Kenangil, 2014). Lastly, WCST-CV was administered to participants who received the required scores from all scales and tests.

2.3 Wisconsin Card Sorting Test-Computerized Version (WCST-CV)

This study used the computer version of WCST developed by Çelik et al. (2021). WCST-CV allows the participant to read the instructions and complete the computerized test. Therefore, the participant must have basic computer skills (moving the cursor on the screen, etc.). In its manual application, the tester’s “True, False” feedback was arranged in WCST-CV with both auditory and visual feedback. When the participant completes the test, it calculates their performance according to the scoring directive used in Karakaş et al.’s (1999) study. Thirteen different scores were obtained through WCST administrations. These were total number of responses (WCST1), total number of errors (WCST2), total number of correct responses (WCST3), number of categories completed (WCST4), total number of perseverative responses (WCST5), total number of perseverative errors (WCST6), total number of nonperseverative errors (WCST7), percentage of perseverative responses (WCST8), number of responses used to complete the first category (WCST9), number of conceptual level responses (WCST10), percentage of conceptual level responses (WCST11), failure to maintain set (WCST12), and score of learning to learn (WCST13).

2.4 Statistical analysis

All analyses were performed in R with olsrr, lmtest and Metrics libraries.

2.5 Regression models and estimators

Regression analysis can be used for modeling between dependent variable as WCST sub-scores and independent variables, as age and education level to generate normative data (Rammal et al., 2019). OLS estimator is the optimal method for Linear Regression Analysis under specific assumptions, and in case of violation of these assumptions, it may lead to misleading results (Yildirim and Kantar, 2014; Yenilmez et al., 2018). OLS estimators are based on assumptions such as normally distributed error terms, no multicollinearity between the independent variables, and homoscedasticity of variance (Vicente et al., 2021). Kolmogorov-Smirnov is used to test the normality of error term (p-value must be greater than significance level, e.g. 0.05), Variance Inflation Factors (VIF) is used to assess multicollinearity (the value must be lower than 10) (Rivera et al., 2021), and Breusch-Pagan is used to test heteroscedasticity (p-value must be greater than significance level, e.g. 0.05). In case of violation of these assumptions, alternative regression methods must be used, such as robust regression for violation of normality (Yildirim and Kantar, 2014, 2020) and Weighted Least Squares (WLS) for heteroscedasticity (Kantar, 2016). In the regression analysis, age is taken into account as a continuous variable and education level is taken into account as a dummy variable similar to Rammal et al. (2019) (12 years and below is taken as 0, and over 12 years is taken as 1).

The linear Regression Model is defined as

$\begin{matrix} WCSTScor e_{k} \\ = β_{0} + β_{1} Age + β_{2} EducationLevel + ɛ \end{matrix}$ (1)

Where β₀, β₁, β₂ are regression coefficients, ɛ is error term k is WCST type, and k represents the WCST sub-scores as k = 2,4,11 for OLS.

The weighted Regression Model is defined as

$\begin{matrix} Weight WCST Scor e_{k} = β_{0} + β_{1} Weight Age \\ + β_{2} Weight Education Level + ɛ \end{matrix}$ (2)

Where β₀, β₁, β₂ are regression coefficients, ɛ is the error term, k represents the WCST sub-scores, and Weight should be determined according to heteroscedasticity. In this study, Weights are determined as Weight = 1/Age for k = 3 and Weight = 1/Fitted WCST Score for k = 10.

3 Results

Regression coefficients and R² values are given in Table 1. Age and education coefficients are significant at the 1% level for WCST 1, 2, 3, 4, 7, 10, and 11. There is a negative relationship between age and WCST 3, 4, 10, and 11, as well as between education level and WCST 1, 2, and 7. While establishing the regression models, gender was added to the model, but since its effect on WCST scores was found to be insignificant, it was removed from the model. This shows that gender has no effect on WCST scores, which means, it does not affect whether the scores increase or decrease. However, only WCST 2, 3, 4, 7, and 10, 11 models give more meaningful results when R² values are considered. Age and education variables were insufficient to explain the score of WCST 5, 6, 8, 9, 12 and 13. SDe represents the standard deviation of error terms.

Table 1
Regression coefficients and evaluation metrics

WCST Estimator Intercept Age Education Multiple R² Adjusted R² SDe

WCST 1 OLS 103,7100^* 0,4068^* –7,4367^* 0,1755 0,1716 14,1428

WCST 2 OLS 18,5533^* 0,8251^* –12,5971^* 0,2892 0,2858 19,5748

WCST 3 OLS 85,0278^* –0,4148^* 5,0910^ 0,1450 0,1410 14,4058

WCST 3 WLS 84,1075^* –0,3869^* 4,0881^ 0,1351 0,1310 14,4194

WCST 4 OLS 6,2411^* –0,0701^* 1,1544^* 0,2613 0,2578 1,8246

WCST 5 OLS 15,0915^* 0,1285^* –2,6417^. 0,0268 0,0222 12,9819

WCST 6 OLS 14,3658^*** 0,0944^* –2,1079^. 0,0224 0,0177 10,8433

WCST 7 OLS 4,3975 0,7100^* –10,1611^* 0,2186 0,2149 19,9421

WCST 8 OLS 12,7003^* 0,0530 –1,0877 0,0117 0,0070 8,1621

WCST 9 OLS 15,5029^* 0,3049^* –2,3243 0,0349 0,0297 20,1764

WCST 10 OLS 78,9291^* –0,6303^* 8,6685^* 0,1936 0,1897 18,9160

WCST 10 WLS 79,6290^* –0,6486^* 9,0116^* 0,1963 0,1925 18,9183

WCST 11 OLS 76,4240^* –0,7427^* 10,9989^* 0,2648 0,2613 18,5622

WCST 12 OLS 1,5711^*** –0,0009 –0,2508 0,0063 0,0015 1,4385

WCST 13 OLS 3,9374^* –0,1203^** 1,3872 0,0521 0,0451 6,7558

WCST	Estimator	Intercept	Age	Education	Multiple R²	Adjusted R²	SDe
WCST 1	OLS	103,7100^***	0,4068^***	–7,4367^***	0,1755	0,1716	14,1428
WCST 2	OLS	18,5533^***	0,8251^***	–12,5971^***	0,2892	0,2858	19,5748
WCST 3	OLS	85,0278^***	–0,4148^***	5,0910^**	0,1450	0,1410	14,4058
WCST 3	WLS	84,1075^***	–0,3869^***	4,0881^**	0,1351	0,1310	14,4194
WCST 4	OLS	6,2411^***	–0,0701^***	1,1544^***	0,2613	0,2578	1,8246
WCST 5	OLS	15,0915^***	0,1285^*	–2,6417^.	0,0268	0,0222	12,9819
WCST 6	OLS	14,3658^***	0,0944^*	–2,1079^.	0,0224	0,0177	10,8433
WCST 7	OLS	4,3975	0,7100^***	–10,1611^***	0,2186	0,2149	19,9421
WCST 8	OLS	12,7003^***	0,0530	–1,0877	0,0117	0,0070	8,1621
WCST 9	OLS	15,5029^***	0,3049^***	–2,3243	0,0349	0,0297	20,1764
WCST 10	OLS	78,9291^***	–0,6303^***	8,6685^***	0,1936	0,1897	18,9160
WCST 10	WLS	79,6290^***	–0,6486^***	9,0116^***	0,1963	0,1925	18,9183
WCST 11	OLS	76,4240^***	–0,7427^***	10,9989^***	0,2648	0,2613	18,5622
WCST 12	OLS	1,5711^***	–0,0009	–0,2508	0,0063	0,0015	1,4385
WCST 13	OLS	3,9374^*	–0,1203^**	1,3872	0,0521	0,0451	6,7558

Signif. codes: ‘***’ 0.001 ‘**’0.01 ‘*’ 0.05 ‘.’ 0.1.

Test statistics for OLS assumptions are given in Table 2. As is seen in Table 2, where VIF values are less than 10, there is no multicollinearity problem between the independent variables. However, there are non normality or heteroscedasticity problems for WCST 1, 5, 6, 7, 8, 9, 12, and 13 models. Although the OLS assumptions are not satisfied for WCST 3 and WCST 10, this issue has been overcome using WLS. In this study, Weights are determined as Weight = 1/Age for WCST 3 and Weight = 1/Fitted WCSTScore for WCST 10. Additionally, WCST 7 are not satisfied the OLS assumptions, and a suitable weight for WLS could not be identified. The OLS assumptions are satisfied by WCST 2, 4, and 11 among 13 different WCST models.

Table 2

Assumption of OLS

		Kolmogorov-Smirnov		Breusch-Pagan			Satisfying assumptions
WCST	Estimator	VIF	Statistic	p-value	Statistic	p-value
WCST 1	OLS	1,0419	0,1610	0,0000	53,3330	0,0000
WCST 2	OLS	1,0419	0,0451	0,3562	2,9851	0,2248	X
WCST 3	OLS	1,0419	0,0516	0,2116	46,9410	0,0000
WCST 3	WLS	1,0419	0,0567	0,1329	3,2925	0,1928	X
WCST 4	OLS	1,0419	0,0634	0,0670	2,2919	0,3179	X
WCST 5	OLS	1,0419	0,0639	0,0637	16,5160	0,0003
WCST 6	OLS	1,0419	0,0623	0,0759	17,6270	0,0001
WCST 7	OLS	1,0419	0,0944	0,0011	49,6230	0,0000
WCST 8	OLS	1,0419	0,0490	0,2638	25,1200	0,0000
WCST 9	OLS	1,0200	0,1758	0,0000	2,7881	0,2481
WCST 10	OLS	1,0419	0,0676	0,0424	32,8080	0,0000
WCST 10	WLS	1,0419	0,0648	0,0580	3,6927	0,1578	X
WCST 11	OLS	1,0419	0,0509	0,2242	2,2830	0,3193	X
WCST 12	OLS	1,0419	0,1790	0,0000	0,5529	0,7585
WCST 13	OLS	1,0122	0,0721	0,1183	7,0390	0,0296

3.1 Generating normative data

Normative data is used for determining the percentage of residuals between predicted and observed values. The predicted WCST score is calculated by regression equation (1) for OLS and equation (2) for WLS with the coefficient represented in Table 1. This procedure is explained on the WCST 11 score. The predicted WCST 11 score is calculated by WCST Score₁₁ = 76,4240–0,7427*Age+10,9989*Education Level where Education Level is a binary variable as 0 for below 12 years and 1 for above 12 years. For instance, a person 25 years old and having 16 years of education has a score of 26 on WCST 11. The predicted WCST score is calculated by 76,4240–0,7427*25 + 10,9989*1 and the predicted score is 68,8554. For this example, residual is a difference between predicted and observed values such as 26 –68,8554 = –42,8554. Standardized residual is calculated as residual/SDe and 42,8554 / 18,5622 = –2,30875 for this example where SDe values are given in Table 1 for each WCST score. This standardized residual also knows as z-score and the percentage can be taken any z table. The percentage of our example is 1%. In other words, the 25-year-old participant with 16 years of education should havea score of 68,8554; but the participant’s performance is 26. According to the regression model, this participant’s score is higher than his/her peers, only 1%.

An important point that clinicians should consider when evaluating the percentages obtained here is that obtaining a high score is not a good indicator. A high percentile for WCST3, WCST4, WCST 10 and WCST 11 is a good indicator, while a high percentile for WCST 2 is negative. For this reason, clinicians can go for interpretation by subtracting the percentile from 100. The example is given in the below for WCST 2.

The predicted WCST2 score is calculated as 18,5533 + 0,8251*25–12,5971*1 = 26,5837. The residual is 56 –26,5837 = 29,4163 and standardized residual is 29,4163 /19,5748 = 19,5748. The percentage of our example is 93%. It is not desirable to have more error responses in WCST 2. For this reason, it would be more accurate to subtract the percentile in the table from 100. For this example, this participant’s percentile is 7%.

Appendix pages 2–7 present the predicted values generated through the regression model. Due to the extensive nature of these pages, a user-friendly calculator has been incorporated on the first page of the appendix. This calculator simplifies the process for clinicians and researchers when assessing individuals. They can input the participant’s age, education level, and the raw score obtained from the WCST-CV. Subsequently, the calculator will automatically display the corresponding percentile of the participant’s peers for the specific score type in the percentile section.

In addition to the five score types derived from the analysis, to interpret scores from other score types within the domain of neuropsychological evaluation, mean and standard deviation values for each score type are provided across six distinct age groups and three different education levels. These values enable clinicians to perform calculations using the conventional approach.

As previously mentioned in the introduction, the selection of age and education level categories in the establishment of normative groups for neuropsychological tests is arbitrary. Initially, WCST norms were categorized into five age groups, conforming to the common approach, resulting in 12 age groups (Rammal et al., 2019; Arango-Lasprilla et al., 2015; Arango-Lasprilla et al., 2017; Shan et al., 2008). However, upon conducting group comparisons, no statistically significant disparities were observed among individuals’ WCST score types, particularly within the age range of 18 to 37. Consequently, participants within this age range were grouped. Subsequently, age groups were recategorized as illustrated in Table 3.

Table 3
The means and standard deviations of sub-scores derived from the Wisconsin Card Sorting Test-Computer Version (WCST-CV)

Ages

18–37 38–47 48–57 58–62 63–67 68>

WCST 1 Education up to 8 years 123.62±12.37 125.50±9.17 125.20±7.94 128.00±0.01 128.00±0.03 128.00±0.02

Education between 9 and 12 years 113.72±19.67 120.78±14.75 123.85±10.74 128.00±0.01 128.00±.00 128.00±0.01

13 years or more of education 105.28±19.57 114.71±16.01 120.72±12.98 128.00±0.01 128.00±.00 128.00±0.00

WCST2 Education up to 8 years 45.50±12.01 63.23±19.94 64.06±21.35 64.30±15.56 75.75±16.73 93.42±19.73

Education between 9 and 12 years 40.45±20.84 49.09±19.66 56.00±20.90 74.66±71.50 71.50±19.09 N.A.

13 years or more of education 31.08±16.88 41.82±19.86 46.84±16.72 N.A. 59.33±17.89 N.A.

WCST3 Education up to 8 years 78.12±6.51 62.26±16.76 61.35±17.78 63.70±15.56 52.25±34.57 34.57±19.73

Education between 9 and 12 years 73.27±12.14 71.69±13.31 67.85±16.12 53.33±17.78 56.50±19.09 N.A.

13 years or more of education 74.06±8.25 72.88±11.77 73.88±10.24 N.A. 68.66±17.89 N.A.

WCST4 Education up to 8 years 3.87±1.35 2.28±2.01 2.33±1.99 2.40±1.46 1.50±1.46 0.14±.37

Education between 9 and 12 years 4.58±1.56 3.61±1.99 2.85±2.11 1.66±1.52 2.00±4.00 4.00±

13 years or more of education 5.18±1.46 4.36±2.03 4.20±1.52 N.A. 2.00±.00 N.A.

WCST5 Education up to 8 years 21.62±10.07 22.89±13.89 21.89±14.61 24.35±13.42 26.00±20.44 7.57±10.51

Education between 9 and 12 years 18.40±12.03 19.69±11.71 20.42±12.23 35.33±27.06 35.00±25.45 N.A.

13 years or more of education 15,53±11,15 17,78±10,48 19,84±11,45 N.A. 27.33±27.39 N.A.

WCST6 Education up to 8 years 18.62±8.36 20.84±11.74 19.50±12.06 21.80±11.73 22.06±16.10 7.42±10.16

Education between 9 and 12 years 16.59±9.90 17.69±9.61 18.62±10.64 19.00±23.38 32.00±22.62 N.A.

13 years or more of education 14.38±9.65 15.98±8.64 18.32±9.98 N.A. 23.33±21.59 N.A.

WCST7 Education up to 8 years 26.12±7.77 41.60±25.71 43.75±26.57 41.95±19.77 53.62±31.19 85.42±28.40

Education between 9 and 12 years 23.62±14.34 31.03±18.41 36.45±19.32 37.33±45.74 37.50±.70 N.A.

13 years or more of education 16.59±9.69 25.26±16.94 28.40±12.82 N.A. 36.00±4.35 N.A.

WCST8 Education up to 8 years 14.80±6.15 16.54±9.05 15.55±9.38 17.03±9.16 17.23±12.58 5.80±7.94

Education between 9 and 12 years 13.99±7.08 14.31±7.15 14.85±8.06 23.70±17.29 25.00±17.67 N.A.

13 years or more of education 12.89±6.81 13.67±6.22 14.80±7.34 N.A. 18.23±16.87 N.A.

WCST9 Education up to 8 years 21.87±5.89 31.96±19.81 27.59±20.19 33.37±22.23 34.20±21.52 N.A.

Education between 9 and 12 years 25.25±20.92 30.31±23.67 31.48±23.80 12.50±2.12 28.50±13.43 N.A.

13 years or more of education 17.77±12.96 28.26±19.52 33.92±21.43 N.A. 37.66±38.55 N.A.

WCST10 Education up to 8 years 69.12±6.87 42.68±21.23 43.47±23.67 45.80±19.62 32.68±20.09 13.28±16.55

Education between 9 and 12 years 62.12±15.72 56.46±18.56 50.68±21.97 36.33±18.58 48.00±21.21 N.A.

13 years or more of education 66.22±10.41 60.00±18.48 60.80±15.24 N.A. 50.33±15.82 N.A.

WCST11 Education up to 8 years 56.57±9.26 34.62±18.50 35.40±20.55 35.78±15.33 25.53±15.70 10.37±12.92

Education between 9 and 12 years 57.18±19.26 48.27±18.95 41.96±20.22 28.38±14.51 37.50±16.57 N.A.

13 years or more of education 65.26±15.22 54.36±20.08 63.93±60.44 N.A. 39.32±12.36 N.A.

WCST12 Education up to 8 years 2.12±1.24 1.55±1.46 1.60±1.56 1.50±1.84 0.75±.85 0.85±1.21

Education between 9 and 12 years 1.43±1.56 1.53±1.25 1.88±1.36 1.33±1.52 1.50±2.12 N.A.

13 years or more of education 1.16±1.31 1.32±1.49 1.24±1.26 N.A. 2.33±2.08 N.A.

WCST13 Education up to 8 years 2.56±3.25 –0.47±9.71 –3.45±6.87 –6.55±7.86 –3.60±4.63 N.A.

Education between 9 and 12 years 0.65±6.49 0.15±5.46 –2.89±9.81 N.A. N.A. N.A.

13 years or more of education 7.05±0.37 1.53±6.03 –0.48±8.57 N.A. N.A. N.A.

		Ages
WCST 1	Education up to 8 years	123.62±12.37	125.50±9.17	125.20±7.94	128.00±0.01	128.00±0.03	128.00±0.02
	Education between 9 and 12 years	113.72±19.67	120.78±14.75	123.85±10.74	128.00±0.01	128.00±.00	128.00±0.01
	13 years or more of education	105.28±19.57	114.71±16.01	120.72±12.98	128.00±0.01	128.00±.00	128.00±0.00
WCST2	Education up to 8 years	45.50±12.01	63.23±19.94	64.06±21.35	64.30±15.56	75.75±16.73	93.42±19.73
	Education between 9 and 12 years	40.45±20.84	49.09±19.66	56.00±20.90	74.66±71.50	71.50±19.09	N.A.
	13 years or more of education	31.08±16.88	41.82±19.86	46.84±16.72	N.A.	59.33±17.89	N.A.
WCST3	Education up to 8 years	78.12±6.51	62.26±16.76	61.35±17.78	63.70±15.56	52.25±34.57	34.57±19.73
	Education between 9 and 12 years	73.27±12.14	71.69±13.31	67.85±16.12	53.33±17.78	56.50±19.09	N.A.
	13 years or more of education	74.06±8.25	72.88±11.77	73.88±10.24	N.A.	68.66±17.89	N.A.
WCST4	Education up to 8 years	3.87±1.35	2.28±2.01	2.33±1.99	2.40±1.46	1.50±1.46	0.14±.37
	Education between 9 and 12 years	4.58±1.56	3.61±1.99	2.85±2.11	1.66±1.52	2.00±4.00	4.00±
	13 years or more of education	5.18±1.46	4.36±2.03	4.20±1.52	N.A.	2.00±.00	N.A.
WCST5	Education up to 8 years	21.62±10.07	22.89±13.89	21.89±14.61	24.35±13.42	26.00±20.44	7.57±10.51
	Education between 9 and 12 years	18.40±12.03	19.69±11.71	20.42±12.23	35.33±27.06	35.00±25.45	N.A.
	13 years or more of education	15,53±11,15	17,78±10,48	19,84±11,45	N.A.	27.33±27.39	N.A.
WCST6	Education up to 8 years	18.62±8.36	20.84±11.74	19.50±12.06	21.80±11.73	22.06±16.10	7.42±10.16
	Education between 9 and 12 years	16.59±9.90	17.69±9.61	18.62±10.64	19.00±23.38	32.00±22.62	N.A.
	13 years or more of education	14.38±9.65	15.98±8.64	18.32±9.98	N.A.	23.33±21.59	N.A.
WCST7	Education up to 8 years	26.12±7.77	41.60±25.71	43.75±26.57	41.95±19.77	53.62±31.19	85.42±28.40
	Education between 9 and 12 years	23.62±14.34	31.03±18.41	36.45±19.32	37.33±45.74	37.50±.70	N.A.
	13 years or more of education	16.59±9.69	25.26±16.94	28.40±12.82	N.A.	36.00±4.35	N.A.
WCST8	Education up to 8 years	14.80±6.15	16.54±9.05	15.55±9.38	17.03±9.16	17.23±12.58	5.80±7.94
	Education between 9 and 12 years	13.99±7.08	14.31±7.15	14.85±8.06	23.70±17.29	25.00±17.67	N.A.
	13 years or more of education	12.89±6.81	13.67±6.22	14.80±7.34	N.A.	18.23±16.87	N.A.
WCST9	Education up to 8 years	21.87±5.89	31.96±19.81	27.59±20.19	33.37±22.23	34.20±21.52	N.A.
	Education between 9 and 12 years	25.25±20.92	30.31±23.67	31.48±23.80	12.50±2.12	28.50±13.43	N.A.
	13 years or more of education	17.77±12.96	28.26±19.52	33.92±21.43	N.A.	37.66±38.55	N.A.
WCST10	Education up to 8 years	69.12±6.87	42.68±21.23	43.47±23.67	45.80±19.62	32.68±20.09	13.28±16.55
	Education between 9 and 12 years	62.12±15.72	56.46±18.56	50.68±21.97	36.33±18.58	48.00±21.21	N.A.
	13 years or more of education	66.22±10.41	60.00±18.48	60.80±15.24	N.A.	50.33±15.82	N.A.
WCST11	Education up to 8 years	56.57±9.26	34.62±18.50	35.40±20.55	35.78±15.33	25.53±15.70	10.37±12.92
	Education between 9 and 12 years	57.18±19.26	48.27±18.95	41.96±20.22	28.38±14.51	37.50±16.57	N.A.
	13 years or more of education	65.26±15.22	54.36±20.08	63.93±60.44	N.A.	39.32±12.36	N.A.
WCST12	Education up to 8 years	2.12±1.24	1.55±1.46	1.60±1.56	1.50±1.84	0.75±.85	0.85±1.21
	Education between 9 and 12 years	1.43±1.56	1.53±1.25	1.88±1.36	1.33±1.52	1.50±2.12	N.A.
	13 years or more of education	1.16±1.31	1.32±1.49	1.24±1.26	N.A.	2.33±2.08	N.A.
WCST13	Education up to 8 years	2.56±3.25	–0.47±9.71	–3.45±6.87	–6.55±7.86	–3.60±4.63	N.A.
	Education between 9 and 12 years	0.65±6.49	0.15±5.46	–2.89±9.81	N.A.	N.A.	N.A.
	13 years or more of education	7.05±0.37	1.53±6.03	–0.48±8.57	N.A.	N.A.	N.A.

* This table is intended for clinicians seeking to interpret additional scores obtained from the WCST-CV. To ensure statistical significance, we stratified the sample by various age groups and levels of education. Consequently, some age groups exhibit more comprehensive ranges due to the absence of significant differences in WCST scores within specific age groups, notably for individuals aged 18 to 37. However, as cognitive decline becomes more pronounced in older age groups, the age categories become narrower. A similar pattern is observed for education levels. Our statistical group comparisons revealed significant differences, particularly between individuals with 1–5 years of education and those with 6–8 years of education. Nevertheless, when education levels were further subdivided, the number of observations per group became insufficient, denoted by “Not Applicable (N.A.)” in cases where the mean and standard deviation values are not presented. Categories with Not Applicable (N.A.) in their mean and standard deviation values indicate that the number of observations is less than ten people. Therefore, we did not write it here since consistent and reliable findings cannot be obtained in groups of less than ten people. An essential point to consider is that the averages of some scores, particularly in older age groups, did not exhibit significant differences concerning education level. For instance, in the 64 and older age group, the average score for the WCST1 score type was 128, but with very low standard deviations. As elucidated in the introduction, this phenomenon is attributed to the test’s limited ability to distinguish between individuals within this age range. Given that individuals in this age bracket consistently achieved the lowest scores, a ceiling effect, indicative of the test’s inability to capture fine distinctions, became evident.

4 Discussion

The primary objective of this investigation is to generate normative data for the WCST-CV for a sample of healthy Turkish adults using regression modeling. Our findings indicate that five distinct WCST-CV sub-scored conform to the prerequisites for regression analysis, with education level and age variables providing a satisfactory explanation for the variance observed. In contrast, when scrutinizing Tables 1 and 2 in tandem, it becomes evident that more than five WCST scores exhibit associations with education level and age. Nevertheless, norm scores were not derived for these particular score types due to their notably low explained variance rates. The existing literature reveals that in studies adopting a similar methodology, if the regression model attains statistical significance, estimated data are often presented without regard for the magnitude of explained variance (Arango-Lasprilla et al., 2015; Arango-Lasprilla et al., 2017). However, it is essential to emphasize that this practice represents an unsuitable approach within the broader context of regression analysis (Gujarati, 2022).

Norms for psychological tests should undergo periodic updates within the targeted sample. An illustrative example of this practice is found in the Wechsler Intelligence Tests for Children. The intelligence scores are periodically revised, considering biological factors (genetic factors) and development levels of countries change over time and, accordingly the growing accessibility to formal education commencing at increasingly early ages and technological advancements. This phenomenon is explained by the Flynn effect, which highlights a consistent upward trend in intelligence scores over time (Graves et al., 2021; Trahan et al., 2014; Bratsberg & Rogeberg 2018). The Turkish adaptation of the Wisconsin Card Sorting Test (WCST) was undertaken by Karakaş et al. in 1999. Nevertheless, this adaptation was limited to a relatively small sample of 69 healthy individuals, and it failed to address its applicability within clinical populations, particularly concerning ecological validity, which stands out as one of the pivotal methodological concerns. Furthermore, as alluded to in the introduction, the WCST ranks among the prominent neuropsychological tests internationally employed for the assessment of cognitive functioning in both healthy and clinical populations. However, its infrequent usage within our national context can be attributed to the scarcity of individuals possessing expertise in its administration and scoring, in contrast to more commonly employed tests. In light of this, our research team developed an extended version of the WCST in 2021 to facilitate its broader application in research and clinical contexts (Çelik et al., 2021). Nevertheless, psychometric investigations conducted among individuals afflicted by schizophrenia and those without such a diagnosis reveal disparities between the two versions. Consequently, this discrepancy necessitates formulating a distinct set of norms tailored explicitly to the WCST-CV, particularly when endeavors to assess an individual’s cognitive capacity within a clinical setting are at the forefront.

Using the computerized WCST version gives some advantages to clinicians, such as being more economical and minimizing errors in application and calculation. Due to these advantages, the production of the norms of the WCST-CV will enable the widespread use of this test in clinical and research. The increasing use of technology, especially in neuropsychological assessment tools, necessitates a review of norm studies. Having norm values of WCST-CV in a large sample will contribute to the widespread use of WCST-CV for both clinical and research purposes. Unlike other norm studies, we defined age as a continuous variable. Thus, we think that clinicians will make more precise decisions. Especially in the regression models, the gender variable was not significant for any score type. Therefore, gender was not included in the regression scores. Upon a comprehensive review of the literature, it becomes evident that a relatively limited number of studies demonstrate a substantial gender-related impact on WCST scores (Lineweaver et al., 1999). The consensus in the field is that gender is generally regarded as having no significant influence on WCST scores (Tercero et al., 2021; Norman et al., 2011; Caffara et al., 2004). Similarly, the previous norm study conducted in Türkiye found that gender did not affect WCST performance.

5 Conclusion

Compared to other studies, in this study, norm values for WCST-CV of healthy adults were calculated using a much larger sample population. Similarly, standardized tests such as the Beck Depression and Anxiety Scale and MOCA were used to determine whether the participants were mentally and neurologically healthy. Our aim is to establish norm values for the Turkish sample in order to popularize the use of the computer version of the WCST, which provides comprehensive information to clinicians and researchers about the frontal functions of the individual. Unlike other studies, instead of presenting norm lists/tables exceeding 200 pages for each score type, we developed a calculator. However, despite the unique aspects of our research, it also has some limitations. One of the most important limitations is that, as in other studies, education level was considered as a dummy variable. Dividing the education level into two groups: 12 years or less and above makes it difficult to investigate the real effect of education on WCST scores. For example, when Table 3 is examined, education level is 1–8; 9–12. Even when divided into more groups such as 13 or more years of education, a significant difference was found between education groups in terms of WCST scores. However, when the education level is wanted to be examined multivariately, the number of samples must be much higher than the current number, making the data collection process difficult and reducing the applicability of the research. In parallel with this, the number of healthy adults who have received 13 years or more of education in older ages is relatively lower than other groups. Therefore, as can be seen from Table 3, since the number of sample sizes is not sufficient for some specific education and age groups (at least 10 people for each age and education level were included in this study; if the number of people per group is less than 10 people, mean and standard deviation values are not given and are indicated as N.A.). A second limitation is that the norm values obtained are valid for the Turkish sample and cannot be generalized to other countries.

Footnotes

Acknowledgment

The Scientific and Technological Research Council of Turkey (TÜBİTAK) supported this study under project number 1919B012113778.

Ethics statement

The study was approved by the Bartın University Social and Human Sciences Ethics Committee (Approval number: 2021-SBB-0489; Approval date: 15.12.2021).

Conflict of interest

The authors have no known conflict of interest to disclose.

Availability of data and materials

If researchers are interested in obtaining the WCST-CV program and the calculator for estimating norm values in healthy adults, they can contact the corresponding author via email.

The Appendix part is available in the electronic version of this article: .

References

Arango-Lasprilla,

J. C.

, Rivera,

, Longoni,

, Saracho,

C. P.

, Garza,

M. T.

, Aliaga,

, Rodríguez,

, Rodríguez-Agudelo,

, Rábago,

, Sutter,

, Schebela,

, Luna,

, Ocampo-Barba,

, Galarza-Del-Angel,

, Bringas,

M. L.

, Esenarro,

, Martínez,

, García-Egan,

, Perrin,

P. B.

(2015) Modified Wisconsin Card Sorting Test (MWCST): Normative data for the Latin American Spanish speaking adult population, NeuroRehabilitation 37(4), 563–590. https://doi.org/10.3233/NRE-151280

Arango-Lasprilla,

J. C.

, Rivera,

, Nicholls,

, Aguayo Arelis,

, García de la Cadena,

, PeñalverGuia,

A. I.

, Vergara-Moragues,

, Rodriguez-Lorenzana,

, Marín-Morales,

, Soto-Añari,

, Lara,

, Rodríguez-Agudelo,

, Alcazar Tebar,

, Galarza-Del-Angel,

, Rodriguez-Irizarry,

, Ibañéz-Alfonso,

J. A.

, García-Guerrero,

C. E.

, Delgado-Mejía,

I. D.

, Pohlenz Amador,

, Sánchez-SanSegundo,

(2017) ModifiedWisconsin Card Sorting Test (M-WCST): Normative data for Spanish-speaking pediatric population, NeuroRehabilitation 41(3), 617–626. https://doi.org/10.3233/NRE-172242

Atalay,

, Cinan,

(2007) Yetişkinlerde planlamabecerisi: Londra kulesi (LKDX) testinin standardizasyon vegüvenilirlik çalişmasi, Turk Psikoloji Dergisi 22(60) 25–44.

Bratsberg,

, Rogeberg,

(2018) Flynn effect and its reversal are both environmentally caused, Proceedings of the National Academy of Sciences of the United States of America 115(26), 6674–6678. https://doi.org/10.1073/pnas.1718793115

Caffarra,

, Vezzadini,

, Dieci,

, Zonato,

, Venneri,

(2004) Modified Card Sorting Test: normative data, Journal of Clinical and Experimental Neuropsychology 26(2), 246–250. https://doi.org/10.1076/jcen.26.2.246.28087

Crawford,

J. R.

(2003). Psychometric foundations of neuropsychological assessment. In L. H. Goldstein&J. McNeil (Eds.), Clinical Neuropsychology: A Practical Guide to Assessment and Management for Clinicians (pp. 220-235). Chichester: Wiley.

Celik,

, Oguz

, M.

, Konur,

, Köktürk,

, Atasoy,

(2021) Comparison of computerized and manual versions of the Wisconsin Card Sorting Test on schizophrenia and healthy samples, Psychological Assessment 33(6), 562–567. https://doi.org/10.1037/pas0001015

Faustino,

, Oliveira,

, Lopes,

(2022) Normative scores of the Wisconsin Card Sorting Test in a sample of the adult Portuguese population, Applied Neuropsychology: Adult 29(4), 767–774. https://doi.org/10.1080/23279095.2020.1810040

Gillan,

C. M.

, Kalanthroff,

, Evans,

, Weingarden,

H. M.

, Jacoby,

R. J.

, Gershkovich,

, Snorrason,

, Campeas,

, Cervoni,

, Crimarco,

N. C.

, Sokol,

, Garnaat,

S. L.

, McLaughlin,

N. C. R.

, Phelps,

E. A.

, Pinto,

, Boisseau,

C. L.

, Wilhelm,

, Daw,

N. D.

, Simpson,

H. B.

(2020) Comparison of the Association between Goal-Directed Planning and Self-reported Compulsivity vs Obsessive Compulsive Disorder Diagnosis, JAMA Psychiatry 77(1), 77–85. https://doi.org/10.1001/jamapsychiatry.2019.2998

10.

Graves,

L. V.

, Drozdick,

, Courville,

, Farrer,

T. J.

, Gilbert,

P. E.

, Delis,

D. C.

(2021) Cohort differences on the CVLT-II and CVLT3: Evidence of a negative Flynn effect on the attention/working memory and learning trials, The Clinical Neuropsychologist 35(3), 615–632. https://doi.org/10.1080/13854046.2019.1699605

11.

Guàrdia-Olmos,

, Peró-Cebollero,

, Rivera,

, Arango-Lasprilla,

J. C.

(2015) Methodology for the development of normative data for ten Spanish-language neuropsychological tests in eleven Latin American countries, NeuroRehabilitation 37(4), 493–499. https://doi.org/10.3233/NRE-151277

12.

Gujarati,

D. N.

(2022). Basic econometrics. 1th Ed. Prentice Hall.

13.

Hartshorne,

J. K.

, Germine,

L. T.

(2016) and Fall of Different Cognitive Abilities Across the Lifespan, Psychological Science 26(4), 433–443. https://doi.org/10.1177/0956797614567339.

14.

Hisli,

(1988). Beck depresyon envanterinin geçerliğiüzerine bir çalışma. Journal of Psychology, 6, 118-122.

15.

Itzhaky,

, Chubarov,

, Keilp,

J. G.

, Shbiro,

, Levy,

, Zalsman,

(2023) Cognitive flexibility in inpatient children and adolescents with a history of suicide attempts, Psychiatry Research 321(August 2022) 115067. https://doi.org/10.1016/j.psychres.2023.115067

16.

Karakas,

, Irak,

, Kurt,

, Erzengin,

Ö.U.

(1999). Wisconsin kart eşleme testi ve Stroop testi TBAG formu:Ölçülen özellikler açısındankarşılaştırmalı analiz. 3P Dergisi, 7(3): 179–192.

17.

Kessels,

R. P. C.

, Hendriks,

M. P. H.

(2016). Neuropscyhological assessment. In. H.W. Friedman (Eds.), Encyclopedia of mental health (2nd ed., pp. 197-201). Oxford, Elsevier.

18.

Lineweaver,

T. T.

, Bond,

M. W.

, Thomas,

R. G.

, Salmon,

D. P.

(1999) A normative study of Nelson’s (1976) modified version of the Wisconsin Card Sorting Test in healthy older adults, The Clinical Neuropsychologist 13(3), 328–347. https://doi.org/10.1076/clin.13.3.328.1745

19.

Mert-Kantar,

(2016). Estimating variances in weighted leastsquares estimation of distributional parameters. Mathematical and Computational Applications, 21(2), 7.

20.

Miranda,

A. R.

, Franchetto Sierra,

, Martínez Roulet,

, Rivadero,

, Serra,

S. V.

, Soria,

E. A.

(2020) Age, education and gender effects on Wisconsin card sorting test: standardization, reliability and validity in healthy Argentinian adults, Neuropsychology, development, and cognition. Section B, Aging, Neuropsychology and Cognition 27(6), 807–825. https://doi.org/10.1080/13825585.2019.1693491

21.

Nelson,

A. P.

, Searl

M.M.

(2002). Neuropsychological assessment. In V. S. Ramachandran (Eds.,) Encyclopedia of the human brain (1st ed., pp. 579-593).

22.

Norman,

M. A.

, Moore,

D. J.

, Taylor,

, Franklin,

, Cysique,

Jr , Ake,

, Lazarretto,

, Vaida,

, Heaton,

R. K.

HNRC Group (2011) Demographically corrected norms for African Americans and Caucasians on the Hopkins Verbal Learning Test-Revised, Brief Visuospatial Memory Test-Revised, Stroop Color and Word Test, and Wisconsin Card Sorting Test 64-Card Version, Journal of Clinical and Experimental Neuropsychology 33(7), 793–804. https://doi.org/10.1080/13803395.2011.559157

23.

Ozdilek,

, Kenangil,

(2014) Validation of the Turkish version of the Montreal Cognitive Assessment Scale (MOCA-TR) in patients with Parkinson’s disease, The Clinical Neuropsychologist 28(2), 333–343. https://doi.org/10.1080/13854046.2014.881554

24.

Petkus,

A. J.

, Filoteo,

J. V.

, Schiehser,

D. M.

, Gomez,

M. E.

, Hui,

J. S.

, Jarrahi,

, McEwen,

, Jakowec,

M. W.

, Petzinger,

G. M.

(2020) Mild cognitive impairment, psychiatric symptoms, and executive functioning in patients with Parkinson’s disease, International Journal of Geriatric Psychiatry 35(4), 396–404. https://doi.org/10.1002/gps.5255

25.

Rammal,

, Chahine,

S. J.

, Rammal,

, Fares,

, Abbas,

L. A.

(2019) Modified Wisconsin Card Sorting Test (M-WCST): Normative data for the Lebanese adult population, Developmental Neuropsychology 44(5), 397–408. https://doi.org/10.1080/87565641.2019.1652828

26.

Rhodes,

M. G.

(2004) Age-related differences in performance on the Wisconsin Card Sorting Test: A metaanalytic review, Psychology and Aging 19(3), 482–494. https://doi.org/10.1037/0882-7974.19.3.482

27.

Rivera,

, Olabarrieta-Landa,

, Van der Elst,

, Gonzalez,

, Ferrer-Cascales,

, Penalver Guia,

A. I.

, Arango-Lasprilla

, J. C.

(2021) Regression-based normative data for children from latin america: phonological verbal fluency letters M, R, and P, Assessment 28(1), 264–276.

28.

Salthouse,

T. A.

(2001) Structural models of the relations between age and measures of cognitive functioning, Intelligence 29(2), 93–115. https://doi.org/10.1016/S0160-2896(00)00040-4

29.

Salthouse,

T. A.

, Atkinson,

T. M.

, Berish,

D. E.

(2003) Executive Functioning as a Potential Mediator of Age-Related Cognitive Decline in Normal Adults, Journal of Experimental Psychology: General 132(4), 566–594. https://doi.org/10.1037/0096-3445.132.4.566

30.

Shan,

I. K.

, Chen,

Y. S.

, Lee,

Y. C.

, Su,

T. P.

(2008) Adult normative data of the Wisconsin Card Sorting Test in Taiwan, Journal of the Chinese Medical Association: JCMA 71(10), 517–522. https://doi.org/10.1016/S1726-4901(08)70160-6

31.

Trahan,

L. H.

, Stuebing,

K. K.

, Fletcher,

J. M.

, Hiscock,

(2014) The Flynn effect: A meta-analysis, Psychological Bulletin 140(5), 1332–1360. https://doi.org/10.1037/a0037173

32.

Tercero,

B. A.

, Perez,

M. M.

, Mohsin,

, Moore,

R. C.

, Depp,

C. A.

, Ackerman,

R. A.

, Pinkham,

A. E.

, Harvey,

P. D.

(2021) Using a Meta-cognitive Wisconsin Card Sorting Test to measure introspective accuracy and biases in schizophrenia and bipolar disorder, Journal of Psychiatric Research 140 436–442. https://doi.org/10.1016/j.jpsychires.2021.06.016

33.

Ulusoy,

, Sahin,

N. H.

, Erkmen,

(1998). Turkish version of the Beck anxiety inventory: Psychometric properties. Journal of Cognitive Psychotherapy, 12.

34.

Venezia,

R. G.

, Gorlyn,

, Burke,

A. K.

, Oquendo,

M. A.

, Mann,

J. J.

, Keilp,

J. G.

(2018) The impact of cognitive reserve on neurocognitive performance in Major Depressive Disorder, Psychiatry Research 270(May), 211–218. https://doi.org/10.1016/j.psychres.2018.09.031

35.

Vicente,

S. G.

, Rivera,

, Barbosa,

, Gaspar,

, Dores,

A. R.

, Mascialino,

, Arango-Lasprilla,

J. C.

(2021) Normative data for tests of attention and executive functions in a sample of European Portuguese adult population, Aging, Neuropsychology, and Cognition 28(3), 418–437.

36.

Yenilmez,

İ.

, Kantar,

, Acıtas

Ş.

(2018) Estimation of censored regression model in the case of non-normal error, Sigma Journal of Engineering and Natural Sciences 36(2), 513–521.

37.

Yildirim,

, Mert-Kantar,

(2014) A Comprehensive Evaluation of M-estimators for Linear Regression Model, International Journal of Mathematics and Statistics 15(1), 104–138.

38.

Yildirim,

, Mert-Kantar,

(2020) Robust estimation approach for spatial error model, Journal of Statistical Computation and Simulation 90(9), 1618–1638.

		Ages
		18–37	38–47	48–57	58–62	63–67	68>
WCST 1	Education up to 8 years	123.62±12.37	125.50±9.17	125.20±7.94	128.00±0.01	128.00±0.03	128.00±0.02
	Education between 9 and 12 years	113.72±19.67	120.78±14.75	123.85±10.74	128.00±0.01	128.00±.00	128.00±0.01
	13 years or more of education	105.28±19.57	114.71±16.01	120.72±12.98	128.00±0.01	128.00±.00	128.00±0.00
WCST2	Education up to 8 years	45.50±12.01	63.23±19.94	64.06±21.35	64.30±15.56	75.75±16.73	93.42±19.73
	Education between 9 and 12 years	40.45±20.84	49.09±19.66	56.00±20.90	74.66±71.50	71.50±19.09	N.A.
	13 years or more of education	31.08±16.88	41.82±19.86	46.84±16.72	N.A.	59.33±17.89	N.A.
WCST3	Education up to 8 years	78.12±6.51	62.26±16.76	61.35±17.78	63.70±15.56	52.25±34.57	34.57±19.73
	Education between 9 and 12 years	73.27±12.14	71.69±13.31	67.85±16.12	53.33±17.78	56.50±19.09	N.A.
	13 years or more of education	74.06±8.25	72.88±11.77	73.88±10.24	N.A.	68.66±17.89	N.A.
WCST4	Education up to 8 years	3.87±1.35	2.28±2.01	2.33±1.99	2.40±1.46	1.50±1.46	0.14±.37
	Education between 9 and 12 years	4.58±1.56	3.61±1.99	2.85±2.11	1.66±1.52	2.00±4.00	4.00±
	13 years or more of education	5.18±1.46	4.36±2.03	4.20±1.52	N.A.	2.00±.00	N.A.
WCST5	Education up to 8 years	21.62±10.07	22.89±13.89	21.89±14.61	24.35±13.42	26.00±20.44	7.57±10.51
	Education between 9 and 12 years	18.40±12.03	19.69±11.71	20.42±12.23	35.33±27.06	35.00±25.45	N.A.
	13 years or more of education	15,53±11,15	17,78±10,48	19,84±11,45	N.A.	27.33±27.39	N.A.
WCST6	Education up to 8 years	18.62±8.36	20.84±11.74	19.50±12.06	21.80±11.73	22.06±16.10	7.42±10.16
	Education between 9 and 12 years	16.59±9.90	17.69±9.61	18.62±10.64	19.00±23.38	32.00±22.62	N.A.
	13 years or more of education	14.38±9.65	15.98±8.64	18.32±9.98	N.A.	23.33±21.59	N.A.
WCST7	Education up to 8 years	26.12±7.77	41.60±25.71	43.75±26.57	41.95±19.77	53.62±31.19	85.42±28.40
	Education between 9 and 12 years	23.62±14.34	31.03±18.41	36.45±19.32	37.33±45.74	37.50±.70	N.A.
	13 years or more of education	16.59±9.69	25.26±16.94	28.40±12.82	N.A.	36.00±4.35	N.A.
WCST8	Education up to 8 years	14.80±6.15	16.54±9.05	15.55±9.38	17.03±9.16	17.23±12.58	5.80±7.94
	Education between 9 and 12 years	13.99±7.08	14.31±7.15	14.85±8.06	23.70±17.29	25.00±17.67	N.A.
	13 years or more of education	12.89±6.81	13.67±6.22	14.80±7.34	N.A.	18.23±16.87	N.A.
WCST9	Education up to 8 years	21.87±5.89	31.96±19.81	27.59±20.19	33.37±22.23	34.20±21.52	N.A.
	Education between 9 and 12 years	25.25±20.92	30.31±23.67	31.48±23.80	12.50±2.12	28.50±13.43	N.A.
	13 years or more of education	17.77±12.96	28.26±19.52	33.92±21.43	N.A.	37.66±38.55	N.A.
WCST10	Education up to 8 years	69.12±6.87	42.68±21.23	43.47±23.67	45.80±19.62	32.68±20.09	13.28±16.55
	Education between 9 and 12 years	62.12±15.72	56.46±18.56	50.68±21.97	36.33±18.58	48.00±21.21	N.A.
	13 years or more of education	66.22±10.41	60.00±18.48	60.80±15.24	N.A.	50.33±15.82	N.A.
WCST11	Education up to 8 years	56.57±9.26	34.62±18.50	35.40±20.55	35.78±15.33	25.53±15.70	10.37±12.92
	Education between 9 and 12 years	57.18±19.26	48.27±18.95	41.96±20.22	28.38±14.51	37.50±16.57	N.A.
	13 years or more of education	65.26±15.22	54.36±20.08	63.93±60.44	N.A.	39.32±12.36	N.A.
WCST12	Education up to 8 years	2.12±1.24	1.55±1.46	1.60±1.56	1.50±1.84	0.75±.85	0.85±1.21
	Education between 9 and 12 years	1.43±1.56	1.53±1.25	1.88±1.36	1.33±1.52	1.50±2.12	N.A.
	13 years or more of education	1.16±1.31	1.32±1.49	1.24±1.26	N.A.	2.33±2.08	N.A.
WCST13	Education up to 8 years	2.56±3.25	–0.47±9.71	–3.45±6.87	–6.55±7.86	–3.60±4.63	N.A.
	Education between 9 and 12 years	0.65±6.49	0.15±5.46	–2.89±9.81	N.A.	N.A.	N.A.
	13 years or more of education	7.05±0.37	1.53±6.03	–0.48±8.57	N.A.	N.A.	N.A.