Newly developed Learning and Verbal Memory Test (TAMV-I): Normative data for Spanish-speaking pediatric population

Abstract

OBJECTIVE:

To generate normative data for the Learning and Verbal Memory Test (TAMV-I) in Spanish-speaking pediatric populations.

METHOD:

The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the TAMV-I as part of a larger neuropsychological battery. Free recall, memory delay and recognition scores were normed using multiple linear regressions and standard deviations of residual values. Age, age², sex, and mean level of parental education (MLPE) were included as predictors in the analyses.

RESULTS:

The final multiple linear regression models indicated main effects for age on all scores, such that scores increased linearly as a function of age. Age² had a significant effect in all countries except Cuba, and Puerto Rico for free recall score; a significant effect for memory delay score in all countries except Cuba and Puerto Rico; and a significant effect for recognition score in in all countries except Guatemala, Honduras, and Puerto Rico. Models showed an effect for MLPE in Chile (free recall), Honduras (free recall), Mexico (free recall), Puerto Rico (free recall, memory delay, and recognition), and Spain (free recall and memory delay). Sex affected free recall score for Cuba, Ecuador, Guatemala, Mexico, Paraguay, Peru, and Spain, memory delay score for all countries except Chile, Paraguay, and Puerto Rico, and recognition score for Ecuador, Mexico, Peru, and Spain, with girls scoring higher than boys.

CONCLUSIONS:

This is the largest Spanish-speaking pediatric normative study in the world, and it will allow neuropsychologists from these countries to have a more accurate way to interpret the TAMV-I with pediatric populations.

Keywords

Learning and Verbal Memory Test (TAMV-I)neuropsychology Spanish-speaking populations pediatric population

1 Introduction

Memory refers to the processes involved in the storage and subsequent retrieval of information. It is a complex process involving attention, coding (transformation of stimulus to a mental representation), learning, consolidation and storage of information in short or long term memory, and recovery, which can be divided in two types: memory, which can be free or with codes, or recognition (Strauss, Sherman, & Spreen, 2006).

Various tests can be used to evaluate of each of these processes. For the assessment of verbal memory in particular tests such as the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1958), Selective Reminding Test, (SRT; Buschke, 1973, Buschke & Fuld, 1974), California Verbal Learning Test, (CVLT; Delis, Kramer, Kaplan, & Ober, 1987; 2000), and the Hopkins Verbal Learning Test, (HVLT; Brandt, 1991; HVLT-R; Benedict, Schretlen, Groninger, & Brandt, 1998) are the most commonly used. Regarding verbal memory instruments designed specifically for the Spanish-speaking population, the Spanish-Complutense Verbal Learning Test (Test De Aprendizaje Verbal España-Complutense; TAVEC, Benedet, & Alejandre, 1998) and the Verbal Memory Curve (Curva de Memoria Verbal; Ardila, & Rosselli, 1992) are most widely used.

However, most of these tests have been created for adults and there is a lack of appropriate neuropsychological tools to assess learning and verbal memory in children and adolescents (Boyd, 2013), especially in Spanish. This is problematic because learning and memory are the most commonly affected cognitive processes in children and adolescents with attention and hyperactivity disorder (Martinussen, Hayden, Hogg-Johnson, & Tannock, 2005) and other pathologies, including acquired brain injury (Babikian, & Asarnow, 2009), fetal alcohol syndrome (Mattson, Crocker, & Nguyen, 2011), epilepsy (Nolan et al., 2004, Jocic-Jakubi, & Jovic, 2006), and substance use (Squeglia, Jacobus, & Tapert, 2009).

Many child and adolescent instruments in this area have evolved as an extension of adult memory tests. For example, although the RAVLT was created for adults, there are normative data for children from Israel (Vakil, Greenstein, & Blachstein, 2010), Brazil (Oliveira, Mograbi, Gabrig, & Charchat-Fichman, 2016), the Netherlands (Van den Burg & Kingma, 1999) and Australia (Forrester & Geffen, 1991). Similarly, the SRT and the HVLT-R were designed for English-speaking adults but have normative data for children and adolescents (Barr, 2003; Nagle, Everhart, Durham, McCammon, & Walker, 2006; Bruner, Joffe, Duggan, Casella, & Brandt, 1996; Roman, Gaither, & Hoeppner, 1994). Finally, the CVLT and TAVEC have specific versions for children, the California Verbal Learning Test-Children’s version (CVLT-C; Delis, Kramer, & Ober, 1994), which is a reduced version of the CVLT and can be applied to children between five and 16 years of age; and the Spanish-Complutense Verbal Learning Test-Children’s version (TAVEC-I; Benedet, Pamos, & Alejandre, 2001) applicable to children three to 16 years of age.

Despite all this, these tests have several drawbacks. First of all, most of them, with the exception of the SRT, must be purchased and sometimes require high costs for professionals, which has been reported as one of the major obstacles to regular use by Latin American (Arango-Lasprilla, Stevens, Morlett Paredes, Ardila, & Rivera, 2016) and Spanish (Olabarrieta-Landa et al., 2016) neuropsychologists. Second, most of these tests were originally created for the English-speaking population, utilizing word frequencies specific to the English language. Many of them have been translated into Spanish, but have not always been adapted, which means that the words used may not be of the same frequency in the translated language. Third, as discussed above, these tests are usually designed to be applied in adults and although some of them have children’s versions, these versions are sometimes simple abbreviations of the word lists and may not accurately reflect word frequency in younger populations.

Therefore, Rivera, Olabarrieta-Landa, and Arango-Lasprilla (2017a) developed the Learning and Verbal Memory Test (TAMV-I), a free access test created specifically to evaluate learning and memory in Spanish-speakers between the ages of 6 and 17 years of age. The TAMV-I has been shown to have good psychometric properties. Reliability was estimated by obtaining the Cronbach’s alpha (α> 0.849), the intraclass correlation coefficient (C_Cl = 0.846) and the test-retest reliability (r’s > 0.26; p’s < 0.001). As for the convergent validity, estimates of the Pearson product-moment correlation coefficient (r_xy) were used between each TAMV-I score and the memory task scores of the Rey-Osterrieth Complex Figure and the phonological verbal fluency test, obtaining coefficients scores greater than 0.295 (r’s > 0.295; p’s < 0.001; Rivera et al., 2017a). The construct validity was estimated by a confirmatory factor analysis, in which the TAMV-I scores were grouped into a single factor (Rivera et al., 2017a; Rivera, Olabarrieta-Landa, & Arango-Lasprilla, 2017b).

The TAMV-I is a test of learning and verbal memory ideal and valid for use in children and adolescents of Spanish-speaking countries. Currently, this test has normative data for Colombian population (Rivera, Olabarrieta-Landa, & Arango-Lasprilla, 2017c) but there are still many Spanish-speaking countries that do not have normative data. Thus, the objective of the present study is to obtain normative data for children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain based on multiple linear regressions.

2 Method

2.1 Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) were between 6 and 17 years of age, b) were born and currently lived in the country where the study was conducted, c) have Spanish as their mother tongue, d) an IQ≥80 on the Test of Non-verbal Intelligence (TONI-2, Brown, Sherbenou, & Johnsen, 2009), and e) scored <19 on the Children’s Depression Inventory (CDI, Kovacs, 1992).

Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s) were excluded due to its effects on cognitive performance. Participants in the study were from public or private schools, and they signed an informed consent to participate. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2 Instrument administration

The TAMV-I consists of a list of 12 words belonging to three semantic categories: clothing, furniture, and body parts. The test is administered in 4 trials. In each trial, first the examiner tells the examinee he/she will have to repeat the 12 words that will be read aloud. Immediately after reading the 12 words, the examinee is asked to list all the words he/she can remember. Even if the examinee can’t say all 12 words, all 4 trials of listing and remembering must be completed. Therefore, there is an immediate recall total learning score with a maximum of 48 points (12 per trial). After 30 minutes, the examinee is asked to remember and list the same 12 words. Thus, the total memory delayed recall score has a maximum of 12 points. After the delayed recall trial, the recognition trials begin. In each of 12 recognition trials, the examinee must identify the initial words from the list of 12 when presented with 3 other words in groups of 4 – 1 semantically related to the initial word, 1 non-semantically related, and 1 phonologically related. Thus, the total recognition score has a maximum of 12 points. The TAMV-I administration guidelines can be found in Rivera et al. (2017a).

2.3 Statistical analyses

Detailed statistical analyses used to generate the normative data for the TAMV-I scores are described in Rivera & Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure. 1) First, the free recall, memory delay and recognition scores were computed separately by means of the final multiple regression models. The full regression models included as predictors: age, age², sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had > 12 years of education or 0 if participant’s parent(s) had≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted ${\hat{y}}_{i} = B_{0} + B_{1} \cdot {(Age - {\bar{x}}_{Age by country})}_{i} + B_{2} \cdot {(Age - {\bar{x}}_{Age by country})}_{i}^{2} + B_{3} \cdot {Sex}_{i} + B_{4} \cdot {MLPE}_{i}$ . 2) Residual scores were calculated based on the final model ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). 3) Residuals were standardized using the residual Standard Deviation (SD_e) value provided by the regression model: z_i = e_i/SD_e. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied to free recall, memory delay and recognition scores separately for each country.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values are considered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

3 Results

3.1 TAMV-I free recall

The final multivariate linear regression models for the ten country-specific TAMV-I free recall scores were significant (see Table 1). In all countries, the TAMV-I free recall scores increased linearly as a function of age. The TAMV-I free recall scores were affected by a quadratic age effect for all countries except for Cuba and Puerto Rico. Children from Chile, Honduras, Mexico, Puerto Rico, and Spain whose parents had a MLPE >12 years obtained higher TAMV-I free recall scores than children whose parents had a MLPE≤12 years. The child’s sex affected TAMV-I free recall scores for Cuba, Ecuador, Guatemala, Mexico, Paraguay, Peru, and Spain, so that girls achieved higher scores than boys. The amount of variance these predictors explained in the TAMV-I free recall scores ranged from 11.8% (in Guatemala) to 38.0% (in Cuba).

Table 1
Final multiple linear regression models for TAMV-I Free recall

Country B Std. Error t Sig. R² SDe (residual)

Chile

Constant 28.175 0.562 50.109 <0.001 0.351 6.615

Age 1.398 0.099 14.058 <0.001

Age² – 0.076 0.032 – 2.355 0.019

MLPE 1.948 0.718 2.712 0.007

Cuba

Constant 33.200 0.435 76.254 <0.001 0.380 5.986

Age 1.340 0.089 15.049 <0.001

Sex – 1.324 0.615 – 2.152 0.032

Ecuador

Constant 33.233 0.600 55.403 <0.001 0.267 6.064

Age 0.982 0.102 9.615 <0.001

Age² – 0.120 0.034 – 3.557 <0.001

Sex – 1.550 0.712 – 2.177 0.030

Guatemala

Constant 34.120 0.823 41.458 <0.001 0.118 6.873

Age 0.823 0.204 4.036 <0.001

Age² – 0.149 0.059 – 2.519 0.013

Sex – 2.640 0.984 – 2.684 0.008

Honduras

Constant 30.900 0.588 52.551 <0.001 0.246 5.811

Age 0.960 0.107 8.961 <0.001

Age² – 0.125 0.034 – 3.621 <0.001

MLPE 2.054 0.691 2.970 0.003

Mexico

Constant 32.755 0.411 79.686 <0.001 0.254 5.804

Age 0.821 0.055 14.923 <0.001

Age² – 0.150 0.018 – 8.250 <0.001

MLPE 1.482 0.387 3.830 <0.001

Sex – 0.893 0.384 – 2.326 0.020

Paraguay

Constant 32.195 0.648 49.716 <0.001 0.257 6.243

Age 1.021 0.104 9.806 <0.001

Age² – 0.074 0.035 – 2.101 0.037

Sex – 1.569 0.729 – 2.153 0.032

Peru

Constant 32.224 0.720 44.738 <0.001 0.216 7.240

Age 0.949 0.121 7.869 <0.001

Age² – 0.114 0.040 – 2.896 0.004

Sex – 1.941 0.783 – 2.481 0.014

Puerto Rico

Constant 28.375 0.982 28.887 <0.001 0.232 7.034

Age 0.995 0.144 6.915 <0.001

MLPE 4.100 1.157 3.545 <0.001

Spain

Constant 35.232 0.392 89.951 <0.001 0.285 5.442

Age 0.945 0.052 18.074 <0.001

Age² – 0.124 0.017 – 7.358 <0.001

MLPE 1.624 0.364 4.463 <0.001

Sex – 1.037 0.349 – 2.967 0.003

Country	B	Std. Error	t	Sig.	R²	SDe (residual)
Chile
Constant	28.175	0.562	50.109	<0.001	0.351	6.615
Age	1.398	0.099	14.058	<0.001
Age²	– 0.076	0.032	– 2.355	0.019
MLPE	1.948	0.718	2.712	0.007
Cuba
Constant	33.200	0.435	76.254	<0.001	0.380	5.986
Age	1.340	0.089	15.049	<0.001
Sex	– 1.324	0.615	– 2.152	0.032
Ecuador
Constant	33.233	0.600	55.403	<0.001	0.267	6.064
Age	0.982	0.102	9.615	<0.001
Age²	– 0.120	0.034	– 3.557	<0.001
Sex	– 1.550	0.712	– 2.177	0.030
Guatemala
Constant	34.120	0.823	41.458	<0.001	0.118	6.873
Age	0.823	0.204	4.036	<0.001
Age²	– 0.149	0.059	– 2.519	0.013
Sex	– 2.640	0.984	– 2.684	0.008
Honduras
Constant	30.900	0.588	52.551	<0.001	0.246	5.811
Age	0.960	0.107	8.961	<0.001
Age²	– 0.125	0.034	– 3.621	<0.001
MLPE	2.054	0.691	2.970	0.003
Mexico
Constant	32.755	0.411	79.686	<0.001	0.254	5.804
Age	0.821	0.055	14.923	<0.001
Age²	– 0.150	0.018	– 8.250	<0.001
MLPE	1.482	0.387	3.830	<0.001
Sex	– 0.893	0.384	– 2.326	0.020
Paraguay
Constant	32.195	0.648	49.716	<0.001	0.257	6.243
Age	1.021	0.104	9.806	<0.001
Age²	– 0.074	0.035	– 2.101	0.037
Sex	– 1.569	0.729	– 2.153	0.032
Peru
Constant	32.224	0.720	44.738	<0.001	0.216	7.240
Age	0.949	0.121	7.869	<0.001
Age²	– 0.114	0.040	– 2.896	0.004
Sex	– 1.941	0.783	– 2.481	0.014
Puerto Rico
Constant	28.375	0.982	28.887	<0.001	0.232	7.034
Age	0.995	0.144	6.915	<0.001
MLPE	4.100	1.157	3.545	<0.001
Spain
Constant	35.232	0.392	89.951	<0.001	0.285	5.442
Age	0.945	0.052	18.074	<0.001
Age²	– 0.124	0.017	– 7.358	<0.001
MLPE	1.624	0.364	4.463	<0.001
Sex	– 1.037	0.349	– 2.967	0.003

Note. MLPE: Mean level of parental education.

3.2 TAMV-I memory delay

The final multivariate linear regression models for the ten country-specific TAMV-I memory delay scores were significant (see Table 2). In all countries, the TAMV-I memory delay scores increased linearly as a function of age. The TAMV-I memory delay scores were affected by a quadratic age effect for all countries except for Cuba and Puerto Rico. Children from Puerto Rico and Spain whose parents had a MLPE >12 years obtained higher TAMV-I memory delay scores than children whose parents had a MLPE≤12 years. The child’s sex affected TAMV-I memory delay scores for all countries except for Chile, Paraguay, and Puerto Rico, so that girls achieved higher scores than boys. The amount of variance these predictors explained in the TAMV-I memory delay scores ranged from 13.5% (in Guatemala) to 27.5% (in Cuba).

Table 2
Final multiple linear regression models for Memory delay

Country B Std. Error t Sig. R² SDe (residual)

Chile

Constant 8.353 0.185 45.152 <0.001 0.208 2.405

Age 0.349 0.036 9.831 <0.001

Age² – 0.024 0.012 – 2.052 0.041

Cuba

Constant 9.737 0.139 70.040 <0.001 0.275 1.911

Age 0.335 0.028 11.767 <0.001

Sex – 0.429 0.196 – 2.184 0.030

Ecuador

Constant 9.507 0.203 46.932 <0.001 0.189 2.048

Age 0.245 0.034 7.091 <0.001

Age² – 0.045 0.011 – 3.928 <0.001

Sex – 0.527 0.241 – 2.191 0.029

Guatemala

Constant 9.804 0.232 42.192 <0.001 0.135 1.941

Age 0.256 0.058 4.443 <0.001

Age² – 0.057 0.017 – 3.423 0.001

Sex – 0.647 0.278 – 2.329 0.021

Honduras

Constant 9.171 0.185 49.467 <0.001 0.225 1.845

Age 0.291 0.034 8.557 <0.001

Age² – 0.041 0.011 – 3.773 <0.001

Sex – 0.368 0.217 – 1.695 0.091

Mexico

Constant 9.554 0.115 82.755 <0.001 0.178 1.950

Age 0.208 0.018 11.282 <0.001

Age² – 0.049 0.006 – 8.178 <0.001

Sex – 0.382 0.128 – 2.982 0.003

Paraguay

Constant 9.501 0.143 66.501 <0.001 0.162 1.591

Age 0.185 0.026 7.012 <0.001

Age² – 0.027 0.009 – 2.987 0.003

Peru

Constant 9.255 0.213 43.407 <0.001 0.191 2.135

Age 0.215 0.036 6.002 <0.001

Age² – 0.048 0.012 – 4.094 <0.001

Sex – 0.693 0.233 – 2.978 0.003

Puerto Rico

Constant 7.426 0.348 21.363 <0.001 0.244 2.489

Age 0.381 0.051 7.469 <0.001

MLPE 1.073 0.410 2.616 0.010

Spain

Constant 9.959 0.136 73.155 <0.001 0.214 1.890

Age 0.256 0.018 14.064 <0.001

Age² – 0.041 0.006 – 7.011 <0.001

MLPE 0.375 0.126 2.961 0.003

Sex – 0.572 0.121 – 4.712 <0.001

Country	B	Std. Error	t	Sig.	R²	SDe (residual)
Chile
Constant	8.353	0.185	45.152	<0.001	0.208	2.405
Age	0.349	0.036	9.831	<0.001
Age²	– 0.024	0.012	– 2.052	0.041
Cuba
Constant	9.737	0.139	70.040	<0.001	0.275	1.911
Age	0.335	0.028	11.767	<0.001
Sex	– 0.429	0.196	– 2.184	0.030
Ecuador
Constant	9.507	0.203	46.932	<0.001	0.189	2.048
Age	0.245	0.034	7.091	<0.001
Age²	– 0.045	0.011	– 3.928	<0.001
Sex	– 0.527	0.241	– 2.191	0.029
Guatemala
Constant	9.804	0.232	42.192	<0.001	0.135	1.941
Age	0.256	0.058	4.443	<0.001
Age²	– 0.057	0.017	– 3.423	0.001
Sex	– 0.647	0.278	– 2.329	0.021
Honduras
Constant	9.171	0.185	49.467	<0.001	0.225	1.845
Age	0.291	0.034	8.557	<0.001
Age²	– 0.041	0.011	– 3.773	<0.001
Sex	– 0.368	0.217	– 1.695	0.091
Mexico
Constant	9.554	0.115	82.755	<0.001	0.178	1.950
Age	0.208	0.018	11.282	<0.001
Age²	– 0.049	0.006	– 8.178	<0.001
Sex	– 0.382	0.128	– 2.982	0.003
Paraguay
Constant	9.501	0.143	66.501	<0.001	0.162	1.591
Age	0.185	0.026	7.012	<0.001
Age²	– 0.027	0.009	– 2.987	0.003
Peru
Constant	9.255	0.213	43.407	<0.001	0.191	2.135
Age	0.215	0.036	6.002	<0.001
Age²	– 0.048	0.012	– 4.094	<0.001
Sex	– 0.693	0.233	– 2.978	0.003
Puerto Rico
Constant	7.426	0.348	21.363	<0.001	0.244	2.489
Age	0.381	0.051	7.469	<0.001
MLPE	1.073	0.410	2.616	0.010
Spain
Constant	9.959	0.136	73.155	<0.001	0.214	1.890
Age	0.256	0.018	14.064	<0.001
Age²	– 0.041	0.006	– 7.011	<0.001
MLPE	0.375	0.126	2.961	0.003
Sex	– 0.572	0.121	– 4.712	<0.001

Note. MLPE: Mean level of parental education.

3.3 TAMV-I recognition

The final multivariate linear regression models for the ten country-specific TAMV-I recognition scores were significant (see Table 3). In all countries, the TAMV-I recognition scores increased linearly as a function of age. The TAMV-I recognition scores were affected by a quadratic age effect for Chile, Cuba, Ecuador, Mexico, Paraguay, Peru, and Spain. MLPE affect TAMV-I recognition scores for Puerto Rico. The child’s sex affected TAMV-I recognition scores for Ecuador, Mexico, Peru, and Spain, so that girls achieved higher scores than boys. The amount of variance these predictors explained in the TAMV-I recognition score ranged from 4.3% (in Paraguay) to 16.5% (in Chile).

Table 3
Final multiple linear regression models for Recognition

Country B Std. Error t Sig. R² SDe (residual)

Chile

Constant 11.295 0.155 72.795 <0.001 0.165 2.017

Age 0.241 0.030 8.093 <0.001

Age² – 0.032 0.010 – 3.276 0.001

Cuba

Constant 11.797 0.068 173.088 <0.001 0.088 0.878

Age 0.074 0.013 5.664 <0.001

Age² – 0.009 0.004 – 2.087 0.038

Ecuador

Constant 11.489 0.157 73.079 <0.001 0.108 1.589

Age 0.137 0.027 5.102 <0.001

Age² – 0.020 0.009 – 2.278 0.023

Sex – 0.399 0.187 – 2.135 0.034

Guatemala

Constant 11.122 0.114 97.263 <0.001 0.051 1.625

Age 0.149 0.046 3.272 0.001

Honduras

Constant 11.122 0.114 97.263 <0.001 0.051 1.625

Age 0.149 0.046 3.272 0.001

Mexico

Constant 11.652 0.082 141.335 <0.001 0.061 1.392

Age 0.087 0.013 6.642 <0.001

Age² – 0.014 0.004 – 3.278 0.001

Sex – 0.223 0.091 – 2.443 0.015

Paraguay

Constant 11.340 0.121 93.560 <0.001 0.043 1.350

Age 0.061 0.022 2.715 0.007

Age² – 0.019 0.008 – 2.518 0.012

Peru

Constant 11.113 0.215 51.652 <0.001 0.068 2.155

Age 0.110 0.036 3.050 0.002

Age² – 0.025 0.012 – 2.145 0.033

Sex – 0.556 0.235 – 2.367 0.019

Puerto Rico

Constant 10.344 0.281 36.818 <0.001 0.078 1.972

Age 0.134 0.041 3.310 0.001

MLPE 0.803 0.329 2.439 0.016

Spain

Constant 11.962 0.058 206.020 <0.001 0.119 1.034

Age 0.089 0.010 9.130 <0.001

Age² – 0.024 0.003 – 7.473 <0.001

Sex – 0.140 0.066 – 2.132 0.033

Country	B	Std. Error	t	Sig.	R²	SDe (residual)
Chile
Constant	11.295	0.155	72.795	<0.001	0.165	2.017
Age	0.241	0.030	8.093	<0.001
Age²	– 0.032	0.010	– 3.276	0.001
Cuba
Constant	11.797	0.068	173.088	<0.001	0.088	0.878
Age	0.074	0.013	5.664	<0.001
Age²	– 0.009	0.004	– 2.087	0.038
Ecuador
Constant	11.489	0.157	73.079	<0.001	0.108	1.589
Age	0.137	0.027	5.102	<0.001
Age²	– 0.020	0.009	– 2.278	0.023
Sex	– 0.399	0.187	– 2.135	0.034
Guatemala
Constant	11.122	0.114	97.263	<0.001	0.051	1.625
Age	0.149	0.046	3.272	0.001
Honduras
Constant	11.122	0.114	97.263	<0.001	0.051	1.625
Age	0.149	0.046	3.272	0.001
Mexico
Constant	11.652	0.082	141.335	<0.001	0.061	1.392
Age	0.087	0.013	6.642	<0.001
Age²	– 0.014	0.004	– 3.278	0.001
Sex	– 0.223	0.091	– 2.443	0.015
Paraguay
Constant	11.340	0.121	93.560	<0.001	0.043	1.350
Age	0.061	0.022	2.715	0.007
Age²	– 0.019	0.008	– 2.518	0.012
Peru
Constant	11.113	0.215	51.652	<0.001	0.068	2.155
Age	0.110	0.036	3.050	0.002
Age²	– 0.025	0.012	– 2.145	0.033
Sex	– 0.556	0.235	– 2.367	0.019
Puerto Rico
Constant	10.344	0.281	36.818	<0.001	0.078	1.972
Age	0.134	0.041	3.310	0.001
MLPE	0.803	0.329	2.439	0.016
Spain
Constant	11.962	0.058	206.020	<0.001	0.119	1.034
Age	0.089	0.010	9.130	<0.001
Age²	– 0.024	0.003	– 7.473	<0.001
Sex	– 0.140	0.066	– 2.132	0.033

Note. MLPE: Mean level of parental education.

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF≤1.066; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.187 in a F_(2,298) distribution which correspond to percentile 17).

3.4 Normative procedure

Norms (e.g., a percentile score) for the different TAMV-I scores by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate an improved understanding of the procedure used to obtain the percentile associated with a score on this test. Let’s assume we need to find the percentile score for a 17-year-old Cuban boy who scored a 39 on the TAMV-I free recall and whose parent(s) have a MLPE of 14. The steps to obtain the percentile for this score are: 1) Find Cuba in Table 1, which provides the final regression models by country for the TAMV-I free recall scores. Use the B weights to create an equation that will allow you to obtain the predicted TAMV-I free recall score for this child using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Cuban sample which is equal to 11.5 years), and sex which was coded as male = 1 and female = 0. MLPE was not a significant predictor, and therefore is not included in this model. See Rivera & Arango-Lasprilla (2017) to figure out the mean age of each country’s sample. Then the result is added to the constant generated by the model in order to calculate the predicted value.

In the case of the Cuban boy, the predicted TAMV-I free recall score would be calculated using the following equation: ${\hat{y}}_{i} = 33.200 + [1.340 \cdot ({Age}_{i} - 11.5)] + (- 1.324 \cdot {Sex}_{i})$ . The boy’s age is 17. Sex was coded as male = 1 and female = 0, so in this case as the child is a male, the sex value is 1. Thus, the predicted value equation is: ${\hat{y}}_{i} = 33.200 + [1.340 \cdot (17 - 11.5)] + (- 1.324 \cdot 1) = 33.200 + 7.37 + (- 1.324) = 39.246$ . 2) In order to calculate the residual value (indicated with an e_i in the equation), we subtract the actual free recall score (he scored 39) from the predicted value we just calculated ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). In this case, it would be e_i = 39 - 39.246 = -0.246. 3) Next, consult the SD_e column in Table 1 to obtain the country-specific SD_e (residual) value. For Cuba it is 5.986. Using this value, we can transform the residual value to a standardized z score using the equation z_i = e_i/SD_e. In this case, we have -0.246/5.986 = -0.041. This is the standardized z score for a 17-year-old Cuban boy who scored a 39 on the TAMV-I free recall who has parent(s) with a MLPE of 14. 4) The last step is to use the tables available in most statistical reference books (e.g., Strauss et al., 2006). In this example, the z score (probability) of -0.041 corresponds to the 48 percentile. It is important to remember to use the appropriate tables that correspond to each test (free recall, memory delay and recognition) when performing these calculations.

3.5 User-friendly normative data

The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the TAMV-I. However, this method can be prone to human error due to the number of required computations by hand. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 39 on the TAMV-I free recall test for a Cuban boy who is 17 years old and whose parent(s) have a MLPE of 14) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and test (free recall, memory delay and recognition). In this case, the table for TAMV-I free recall score for boys from Cuba can be found in Table A2. (2) Find the appropriate age of the child, in this case, 17 years old. (3) Next, look in the 17 years’ age column to find the approximate location of the raw score obtained on the test. Within the 17 years’ column, the score of 39 obtained by this Cuban boy corresponds to an approximate percentile of 50.

The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method (48 vs. 50) because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.

4 Discussion

The assessment of learning and memory in disorders that affect children’s development is of special relevance today in child and adolescent neuropsychology. Different neurological disorders in infancy such as traumatic brain injury, epilepsy, or attention deficit and hyperactivity disorder, among others, often lead to learning and memory problems that usually affect the children’s reintegration to school and social life. For example, short-term memory problems may slow learning or understanding of tasks during classes, which may complicate the child’s achievement of primary or secondary education (Majerus & Van der Linden, 2013). In addition, children with episodic memory problems may have serious difficulties in explicitly learning and retrieving new information, resulting in forgetting tasks, daily activities, and the planning of future activities, etc. (Majerus & Van der Linden, 2013).

Despite the great importance of the evaluation of these problems in children, currently the majority of tests that exist to evaluate learning and memory have been created and standardized in adult populations and therefore, there are very few tests that focus on children and adolescents (Boyd, 2013). Moreover, most of these tests were originally created for English-speaking population, so the lists of words included were selected according to categorical norms and frequency of English words. Therefore, translating these tests does not guarantee the frequency of words will be the same in the translated language. On the other hand, sometimes learning and memory tests used for children are abbreviations of the lists of words used in the adult versions, so the tests have not been created thinking about children-specific characteristics (e.g. vocabulary development). Therefore, the purpose of the present study was to obtain normative data for TAMV-I from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Results of the study revealed that age, quadratic age, MLPE and sex influenced the performance of the TAMV-I.

Age was found to be significantly associated to free recall, memory delay and recognition, so that scores increase linearly as children become older. This pattern has also been reported in a study of the TAMV-I in children from Colombia (6 to 17 years of age; Rivera et al., 2017c) as well as normative data studies for children and adolescents looking at the HVLT (Barr, 2003), CVLT-C (Goodman, Delis, & Mattson, 1999), SRT (Morgan, 1982) and the RAVLT (Oliveira et al., 2016; Van Den Burg, & Kingma, 1999; Van der Elst, Molenberghs, van Tetering, & Jolles, 2017). The present study also determined the influence of a quadratic function of age on TAMV-I performance, finding that the most prominent increase occurs in younger children compared to older children. Van der Elst et al. (2017) also reported a quadratic function of age in their study looking at the RAVLT test in children between the ages 5 and 16, as well as did Rivera et al. (2017c) in their study of the TAMV-I in Colombian children between the ages 6 and 17.

Improved performance of the TAMV-I as age advances may correspond to the gradual development of memory in children. The increase in memory performance that occurs until late adolescence may not only be due to the biological maturation process, but to the development of resources and other cognitive processes that determine the development of memory such as memory strategies, metacognition, increased knowledge, or speed of information processing (Schneider, 2015).

Although the MLPE is not usually taken into account in normative data studies, this is a very important variable in child development studies (Meador et al., 2011; Schady, 2011). Previous research has shown educational level (Dubow, Boxer, & Huesmann, 2009, Ermisch, & Pronzato, 2010) and occupation of the parent(s) (Dubow et al., 2009) to have a great influence on future cognitive development of children. The present study found that children whose parent(s) had more than 12 years of education performed better in the free recall and memory delay test than children whose parent(s) had less than 12 years of education, without any effect on the recognition test, except in Puerto Rico. Rivera et al. (2017a,c) also reported the influence of the MLPE on free recall, memory delay and recognition of the TAMV-I. However, in a study looking at the RAVLT in children, Van der Elst et al. (2017) found no influence of this variable.

Finally, sex was also associated with performance in the free recall, memory delay and recognition scores, in that girls scored better on this test than boys. However, these results contradict most normative data studies in children looking at different memory tests, such as the HVLT, CVLT-C and RAVLT where sex differences were not found (Barr, 2003; Goodman et al., 1999; Morgan, 1982; Oliveira et al., 2016; Van der Elst et al., 2017) or were only marginally significant, thus, no normative data by sex was generated (Van Den Burg, & Kingma, 1999). Despite these findings, Rivera et al. (2017c) reported a better performance of girls compared to boys not only in the free recall and memory delay of the TAMV-I, but also in the recognition test.

4.1 Implications

A recently study found that 65% of neuropsychologists in Spain and 86% in Latin America included assessment of learning and memory processes as part of their usual daily practice (Arango-Lasprilla et al., 2016; Olabarrieta-Landa et al., 2016). However, few of these tests have been validated and have norms to evaluate memory and learning in children and adolescent populations. Therefore, the results of the present study have great clinical implications for the practice of neuropsychology in Spanish-speaking countries. The TAMV-I is a free access neuropsychological test created exclusively for Spanish-speaking children and adolescents, giving special attention to the words that would compose the lists for the learning task. In order to do that, norms for the categories in Spanish-speaking children were selected (Rivera et al., 2017a). In addition, the selection of words not only addressed the frequency of words in each category, but also the most common words among the different Spanish-speaking countries were selected, allowing their use in different countries were Spanish is spoken (Rivera et al., 2017a). On the other hand, the TAMV-I allows the evaluation of main processes involved in memory (learning, short- and long-term memory, and recognition), which allows professional to determine not only if the patient presents memory problems but also in which process presents the problems.

4.2 Limitations and future directions

The present study presents some limitations. First, only children whose mother tongue was Spanish were recruited. In addition, the TAMV-I was specially created for use in Spanish-speaking children (Rivera et al., 2017a). Therefore, caution should be taken into account when using the TAMV-I, as well as the use of its norms in children whose mother tongue is not Spanish. This is especially important because many countries in Latin America and Spain are multilingual, with Spanish being one of the most widely spoken languages (Chamoreau, 2014; Garrido Medina, 2007).

Secondly, it is not advisable to use these normative data in pediatric population of those Spanish-speaking countries in which the study was not conducted, such as Argentina, Bolivia, Panama or Venezuela, among others. Future studies should generate normative data for the TAMV-I in these countries.

Although the norms of the present study could be used by neuropsychologists in other countries to evaluate Spanish-speaking immigrant children from the countries where the sample was collected for this study, it should be used with caution since other variables such as level of acculturation, bilingualism, number of years living in the country, and so on, could influence performance. In addition, the quality of education of both the child and the parent(s) is another aspect that may influence the cognitive performance of children.

Although the size of the sample was adequate in each of the countries where the study was conducted, it is very important to note that only the sample in Chile, Mexico, Paraguay, Puerto Rico, and Spain were obtained from several regions of the country, while in the remaining countries the sample was collected from only one geographic area. Future studies should expand the sample in other geographical areas of these countries with the objective to have a greater representativeness.

Finally, it is important to keep in mind that the present study was performed with a normal healthy population. Therefore, future studies should be performed with clinical populations to establish the sensitivity and specificity of the TAMV-I.

5 Conclusions

The present study offers professionals working with children and adolescents a useful and valid tool that has norms to evaluate learning and memory in a Spanish-speaking population from 10 countries. The present study found that test performance was associated with age, quadratic age, MLPE, and sex, suggesting older children, girls, and those whose parent(s) have a MLPE >12 years perform better in the TAMV-I.

Currently, there are very few tests available to evaluate learning and memory in children and adolescents, especially for Spanish-speakers. The TAMV-I is an ideal tool for professionals who want to evaluate learning and verbal memory in Spanish-speaking children. The test was created and directed for Spanish-speaking children and adolescents, and so categories and lists of words were selected taking into account this population. The TAMV-I is a free access test, and as such, will undoubtedly benefit professionals in Spain and Latin America where the high cost of neuropsychological tests has been previously identified as a major obstacle to regular use. (Arango-Lasprilla et al., 2016, Olabarrieta-Landa et al., 2016). These norms will provide neuropsychologists in these countries a valid assessment tool that can be used in their everyday practice with pediatric populations.

Conflict of interest

None to report.

Footnotes

The Appendix tables are available in the electronic version of this article: .

References

Aiken

L. S.

, & West

S. G.

(1991), Multiple regression: Testing and interpreting interactions–Newbury Park, CA: Sage.

Arango-Lasprilla

J. C.

, Stevens

, Morlett Paredes

, Ardila

, & Rivera

(2016). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24(4), 318–330. doi: 10.1080/23279095.2016.1185423.

Ardila

, & Rosselli

(1992). Neuropsicología clínica. Medellín: Prensa Creativa.

Babikian

, & Asarnow

(2009). Neurocognitive outcomes and recovery after pediatric TBI: Meta-analytic review of the literature. Neuropsychology, 23(3), 283–296.

Barr

W. B.

(2003). Neuropsychological testing of high school athletes: Preliminary norms and test–retest indices. Archives of Clinical Neuropsychology, 18(1), 91–101.

Benedet

M. J.

, Pamos

, & Alejandre

M. A.

(2001). TAVECI: Test de aprendizaje verbal España-Complutense infantil: Manual. TEA Ediciones.

Benedet

, & Alejandre

(1998). Test de Aprendizaje Verbal España-Complutense (TAVEC). Madrid: TEA.

Benedict

R. H.

, Schretlen

, Groninger

, & Brandt

(1998). Hopkins Verbal Learning Test– Revised: Normative data and analysis of inter-form and test-retest reliability. The Clinical Neuropsychologist, 12(1), 43–55.

Boyd Thomas

(2013). Clinical Assessment of Memory in Children. En Tramontana

M.G.

& Hooper

S.R.

, (Eds.). Assessment issues in child neuropsychology. (pp. 177–204): Springer Science & Business Media.

10.

Brandt

(1991). The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. The Clinical Neuropsychologist, 5(2), 125–142.

11.

Brown

, Sherbenou

R. J.

, & Johnsen

S. K.

(2009), Test de inteligencia no verbal TONI-2. Madrid: TEA ediciones.

12.

Bruner

A. B.

, Joffe

, Duggan

A. K.

, Casella

J. F.

, & Brandt

(1996). Randomised study of cognitive effects of iron supplementation in non-anaemic iron-deficient adolescent girls. The Lancet, 348(9033), 992–996.

13.

Buschke

(1973). Selective reminding for analysis of memory and learning. Journal of Verbal Learning and Verbal Behavior, 12(5), 543–550.

14.

Buschke

, & Fuld

P. A.

(1974). Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology, 24(11), 1019–1019.

15.

Chamoreau

(2014). Diversidad lingüística en México. Amerindia - Langues du Mexique, 37(1), 3–20.

16.

Cook

R. D.

(1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. doi: 10.2307/1268249.

17.

Delis

D. C.

, Kramer

J. H.

, Kaplan

, & Ober

B. A.

(1987). CVLT: California Verbal Learning Test-Adult Version: Manual. Psychological Corporation.

18.

Delis

D. C.

, Kramer

J. H.

, Kaplan

, & Ober

B. A.

(2000). CVLT-II: California verbal learning test: Adult version. Psychological Corporation.

19.

Delis

, Kramer

, Kaplan

, & Ober

(1994). California Verbal Learning Test— Children’s version (CVLT-C. San Antonio, TX: Psychological Corporation.

20.

Dubow

E. F.

, Boxer

, & Huesmann

L. R.

(2009). Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations. Merrill-Palmer quarterly (Wayne State University. Press), 55(3), 224–249.

21.

Ermisch

, & Pronzato

(2010). Causal effects of parents’ education on children’s education (No. 2010–16). ISER Working Paper Series.

22.

Forrester

, & Geffen

(1991). Performance measures of 7– to 15-year-old children on the auditory verbal learning test. The Clinical Neuropsychologist, 5(4), 345–359.

23.

Garrido Medina

J. C.

(2007). Multilingüismo y lengua externa e interna en la política lingüística en España. Anuario de estudios filológicos, 30, 131–149.

24.

Goodman

A. M.

, Delis

D. C

, & Mattson

S. N.

(1999). Normative data for 4-year-old children on the California Verbal Learning Test-Children’s Version. Clinical Neuropsychology, 13(3), 274–282.

25.

Jocic-Jakubi

, & Jovic

N. J.

(2006). Verbal memory impairment in children with focal epilepsy. Epilepsy & Behavior, 9(3), 432–439.

26.

Kovacs

(1992). Children Depression Inventory CDI (Manual). Toronto: Multihealth.

27.

Kutner

M. H.

, Nachtsheim

C. J.

, Neter

, & Li

(2005), Applied linear statistical models (5th ed.). New York: McGraw Hill.

28.

Majerus

, & Van der Linden

(2013). Memory disorders in children. Handbook of Clinical Neurology, 111, 252–255. doi: 10.1016/B978-0-444-52891-9.00027-0.

29.

Martinussen

, Hayden

, Hogg-Johnson

, & Tannock

(2005). A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 44(4), 377–384.

30.

Mattson

S. N.

, Crocker

, & Nguyen

T. T.

(2011). Fetal alcohol spectrum disorders: Neuropsychological and behavioral features. Neuropsychology Review, 21(2), 81–101.

31.

Meador

K. J.

, Baker

G. A.

, Browning

, Clayton-Smith

, Cohen

M. J.

, Kalayjian

L. A.

, & Loring

D. W.

(2011). Relationship of child IQ to parental IQ and education in children with fetal antiepileptic drug exposure. Epilepsy & Behavior, 21(2), 147–152.

32.

Morgan

S. F.

(1982). Measuring long-term memory storage and retrieval in children. Journal of Clinical and Experimental Neuropsychology, 4(1), 77–85.

33.

Nagle

A. M.

, Everhart

D. E.

, Durham

T. W.

, McCammon

S. L.

, & Walker

(2006). Deception strategies in children: Examination of forced choice recognition and verbal learning and memory techniques. Archives of Clinical Neuropsychology, 21(8), 777–785.

34.

Nolan

M. A.

, Redoblado

M. A.

, Lah

, Sabaz

, Lawson

J. A.

, Cunningham

A. M.

, & Bye

A. M.

(2004). Memory function in childhood epilepsy syndromes. Journal of Paediatrics and Child Health, 40(1-2), 20–27.

35.

Olabarrieta-Landa

, Caracuel

, Pérez-García

, Panyavin

, Morlett-Paredes

, & Arango-Lasprilla

J. C.

(2016). The profession of neuropsychology in Spain: Results of a national survey. The Clinical Neuropsychologist, 30(8), 1335–1355.

36.

Olabarrieta-Landa

, Landa Torre

, López-Mugartza

J. C.

, Bialystok

, & Arango-Lasprilla

J. C.

(2017). Verbal fluency tests: Developing a new model of administration and scoring for Spanish language. NeuroRehabilitation.

37.

Oliveira

R. M.

, Mograbi

D. C.

, Gabrig

I. A.

, & Charchat-Fichman

(2016). Normative data and evidence of validity for the Rey Auditory Verbal Learning Test, Verbal Fluency Test, and Stroop Test with Brazilian children. Psychology & Neuroscience, 9(1), 54.

38.

Rey

(1958). L’examen clinique en psychologie.. Paris: Presse Universitaire de France.

39.

Rivera

, & Arango-Lasprilla

J. C.

(2017). Methodology for the development of normative data for Spanish Speaking pediatric population. NeuroRehabilitation, 41(3), 581–592.

40.

Rivera

, Folleco

J. A.

, Benito Sánchez

, Acosta Barreto

M. R.

, Riaño Garzón

M. E.

, Herrera Bravo

, & Arango-Lasprilla

J. C.

(2017c). Datos normativos para el Test de Aprendizaje y Memoria Verbal – Infantil (TAMV-I) en población colombiana de 6 a 7 años de edad. En Arango-Lasprilla

J. C.

, Rivera

& Olabarrieta-Landa

(Eds). Neuropsicología infantil, 339–349. Bogotá: Manual Moderno.

41.

Rivera

, Olabarrieta-Landa

,, & Arango-Lasprilla

J. C.

(2017b). Metodología para generar datos normativos en población colombiana de 6 a 7 años de edad. En Arango-Lasprilla

J. C.

, Rivera

& Olabarrieta-Landa

(Eds). Neuropsicología infantil, 163–195. Bogotá: Manual Moderno.

42.

Rivera

, Olabarrieta-Landa

,, & Arango-Lasprilla

J. C.

(2017a). Diseño y creación del Test de Aprendizaje y Memoria Verbal Infantil (TAMV-I) en población hispano hablante de 6 a 7 años de edad. En Arango-Lasprilla

J. C.

, Rivera

& Olabarrieta-Landa

(Eds). Neuropsicología infantil, 316–338. Bogotá: Manual Moderno.

43.

Roman

M. A.

, Gaither

R. A.

, & Hoeppner

(1994). Developmental Norms for Two Versions of the Selective Reminding Test. The Clinical Neuropsychologist, 8(3), 323–312.

44.

Schady

(2011). Parents’ education, mothers’ vocabulary, and cognitive development in early childhood: Longitudinal evidence from Ecuador. American Journal of Public Health, 101(12), 2299–2307.

45.

Schneider

(2015). Memory development from early childhood through emerging adulthood. Springer.

46.

Squeglia

L. M.

, Jacobus

, & Tapert

S. F.

(2009). The influence of substance use on adolescent brain development. Clinical EEG and Neuroscience, 40(1), 31–38.

47.

Strauss

, Sherman

, & Spreen

(2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary, (Third Edition). Oxford: Oxford University Press.

48.

Vakil

, Greenstein

, & Blachstein

(2010). Normative data for composite scores for children and adults derived from the Rey Auditory Verbal Learning Test. The Clinical Neuropsychologist, 24(4), 662–677.

49.

van den Burg

, & Kingma

(1999). Performance of 225 Dutch school children on Rey’s Auditory Verbal Learning Test (AVLT): Parallel test-retest reliabilities with an interval of 3 months and normative data. Archives of Clinical Neuropsychology, 14(6), 545–559.

50.

Van der Elst

, Molenberghs

, & van Tetering

, & Jolles

(2017). Establishing normative data for multi-trial memory tests: The multivariate regression-based approach. The Clinical Neuropsychologist, 17, 1–15.