Rey–Osterrieth Complex Figure – copy and immediate recall (3 minutes): Normative data for Spanish-speaking pediatric populations

Abstract

OBJECTIVE:

To generate normative data for the Rey–Osterrieth Complex Figure (ROCF) in Spanish-speaking pediatric populations.

METHOD:

The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the ROCF as part of a larger neuropsychological battery. The ROCF copy and immediate recall (3 minutes) scores were normed using multiple linear regressions and standard deviations of residual values. Age, age², sex, and mean level of parental education (MLPE) were included as predictors in the analyses.

RESULTS:

The final multiple linear regression models showed main effect for age on copy and immediate recall scores, such that scores increased linearly as a function of age. Age² affected ROCF copy score for all countries, except Puerto Rico; and ROCF immediate recall scores for all countries, except Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Models indicated that children whose parent(s) had a MLPE >12 years obtained higher scores compared to children whose parent(s) had a MLPE≤12 years for Chile, Puerto Rico, and Spain in the ROCF copy, and Paraguay and Spain for the ROCF immediate recall. Sex affected ROCF copy and immediate recall score for Chile and Puerto Rico with girls scoring higher than boys.

CONCLUSIONS:

This is the largest Spanish-speaking pediatric normative study in the world, and it will allow neuropsychologists from these countries to have a more accurate approach to interpret the ROCF Test in pediatric populations.

Keywords

Rey–Osterrieth Complex Figure neuropsychology Spanish-speaking populations pediatric population

1 Introduction

In 1941, Swiss psychologist André Rey designed the Rey-Osterrieth Complex Figure Test (ROCF) to examine visuospatial ability and memory in patients with traumatic brain injuries (Rey, 1941). The ROCF was later standardized by Paul-Alexandre Osterrieth in 1944, who proposed a scoring system for administration and collected the first set of normative data for children and adults (Shin, Park, Park, Seol, & Kwon, 2006). Since its initial validation, the ROCF has become one of the most widely used neuropsychological assessments to assess constructional and non-verbal memory abilities (Ardila & Rosselli, 1994; Rosselli & Ardila, 1991).

The ROCF uses an intricate stimulus that is asymmetrical in its design to assess cognitive performance through recognition and recall skills (Fastenau, 1996; Shin et al., 2006). Many cognitive abilities are necessary for good performance, and as such, the test is used to evaluate several different brain functions, including attention, working memory, visuospatial abilities, and planning (Watanabe et al., 2005). For example, the ROCF task involves viewing a complex figure and copying it; next, the individual reproduces it from memory, either immediately, following a delay, or both (Shin et al., 2006). The ROCF is often used to examine deficits due to traumatic brain injury in neurological patients, to test for dementia, and to study children’s cognitive development (Kasai et al., 2006; Watanabe et al., 2005). Both young, developing children and older children or adults with weaknesses in the aforementioned abilities typically find the task quite challenging (Akshoomoff, Feroleto, Doyle, & Stiles, 2002; Waber & Holmes, 1985).

The ROCF has received psychometric support for both reliability and validity in past research with pediatric populations. Reliability among pediatric populations tends to be high for both copy production (α= 0.95) and recall production (α= 0.94; Waber & Holmes, 1985). Evidence for convergent validity is also supported: six to eight year-old children’s scores on the ROCF correlate with other measures of visuospatial, constructional, and fine motor ability [e.g., Hoover Visual Organization Test (Hooper, 1983), Block Design subtest of the Wechsler Primary and Preschool Scale of Intelligence-Revised (Wechsler, 1989), the Grooved Pegboard (Matthews & Kløve, 1964; Frisk, Jakobson, Knight, & Robertson, 2005)]. The ROCF’s psychometric support has contributed to its popularity as a widely used neurological assessment.

Several variables have been found to influence test performance among pediatric populations, although age has the most notable influence (Beltrán Dulcey & Solís Uribe, 2012; De Leeuw, 2010; Rosselli & Ardila, 1991). Copy scores tend to increase between the ages of 12 and 16 (Meyers & Meyers, 1996), and children between the ages of 13 and 16 tend to perform better on copying the figure accurately than children who are nine to 12 years old (Beltrán Dulcey & Solís Uribe, 2012). Age also greatly influences ability for children and adolescents to recall the figure, with scores increasingly greatly between ages six and 12, with slower growth between ages 12 and 17 (Meyers & Meyers, 1996; Mitrushina, Boone, Razani, & D’Elia, 2005). Evidence for a gender difference on the ROCF has been equivocal. Two studies found conflicting findings, such that girls outperformed boys between the ages of eight and 12 in one study (Karapetsas & Kantas, 1991), and boys outperformed girls in the other (Ardila & Rosselli, 1994). However, other studies have not found a gender difference (Beltrán Dulcey & Solís Uribe, 2012; Demsky, Carone, Burns, & Sellers, 2000). Although educational level has been found to influence adults’ performance on the ROCF, an effect for pediatric populations has not been supported (Beltrán Dulcey & Solís Uribe, 2012; Meyers & Meyers, 1995). Finally, performance on the ROCF has also been found to be directly related to intellectual ability, such that individuals with learning disabilities exhibit little improvement on the test as they age (Waber & Bernstein, 1995).

Normative data has been derived for children between the ages of five and 14 in the US, Mexico, and Colombia (Ardila & Rosselli, 1994; Galindo & Cortes, 2003; Karapetsas & Kantas, 1991; Waber & Holmes, 1985, 1986). Using qualitative scoring systems, children’s ability to copy and recall the figure from memory was assessed on several dimensions, including accuracy (e.g., how much of the design was recalled); errors (i.e., distortions); organization (ability to format the figure; e.g., align all four sides); and style (i.e., continuity of lines; Shin et al., 2006). Between the ages of six and eight, children’s ability to copy the figure improves dramatically (Waber & Holmes, 1985), most likely due to an improvement in approach and organization. By age nine, children can reliably produce all parts of the design, and changes after that age tend to reflect the increased capacity to plan and organize reproduction of the figure (Ardila & Rosselli, 1994; Waber & Holmes, 1985). Six year-old children score between 14.5 and 16.5 on copying the image, whereas children ages 14 and above begin to score more similarly to adults, whose average score tends to be around 32 (Ardila & Rosselli, 1994; Kolb & Whishaw, 1985; Meyers & Meyers, 1996; Rosselli & Ardila, 1991; Shin et al., 2006).

In terms of recalling the figure from memory, in past research 57% of children between the ages of five and 14 were able to reproduce the design from memory immediately after copying it, and 43% were able to after a 20-minute delay (Waber & Holmes, 1986). At every age level, more errors are made when recalling the image than when copying the image; in addition, errors tend to decrease at each age level, such that five year-olds tend to make significantly more errors than older children (Waber & Holmes, 1986). In terms of organization scores, five year-old children tend to score the lowest, and these scores generally increase each year up through age 14 (Waber & Holmes, 1986). Regarding style, younger children tended to focus on specific parts of the figure, whereas older children began to represent the figure as a configural whole. It is around age six that children begin to show sensitivity to both individual features of a figure and the overall configuration. Up until that point, younger children tend to perform more accurately when recalling the left side of the figure than the right side (Karapetsas & Kantas, 1991; Waber & Holmes, 1985).

Few studies have established normative data for the ROCF with Spanish-speaking pediatric populations. Although normative data exists for Colombian (Ardila & Rosselli, 1994) and Mexican (Galindo & Cortes, 2003) pediatric populations, no comprehensive normative data exists for pediatric populations in many other Spanish-speaking countries. The present study sought to fill this gap in the literature by providing normative data for a population of children and adolescents from Latin American countries and Spain based on multiple linear regression analyses.

2 Method

2.1 Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) between 6 and 17 years of age, b) born and currently lived in a country where the study was conducted, c) Spanish as primary language, d) an IQ≥80 on the Test Of Non-verbal Intelligence (TONI-2; Brown, Sherbenou, & Johnsen, 2009), and e) a scored <19 on the Children’s Depression Inventory (CDI; Kovacs, 1992).

Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s) were excluded due to its effects on cognitive performance. Participants in the study were from public or private schools, and they signed an informed consent to participate. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2 Instrument administration

A trained examiner administered the ROCF Figure A (copy), and after 3 minutes, the immediate recall was given. To score the ROCF figure, the Spanish-language ROCF manual was used (Rey, 2009). The ROCF includes 18 elements, and the maximum score for each of the two tasks (copy and immediate recall) is 36. In terms of scoring, two points are given when the element is correctly reproduced; one point is given when the reproduction is either (a) distorted, (b) incomplete but placed properly, or (c) complete but placed poorly; and 0.5 point is credited when the element is distorted or incomplete and placed poorly. A score of 0 is given when the element is absent or is not recognizable (Osterrieth, 1944).

2.3 Statistical analyses

Detailed statistical analyses used to generate the normative data for the ROCF copy and immediate recall (3 minutes) scores are described in Rivera and Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure. 1) First, the ROCF copy and immediate recall test scores were computed separately by means of the final multiple regression models. The full regression models included the following as predictors: age, age², sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted ${\hat{y}}_{i} = B_{0} + B_{1} \cdot {(Age - {\bar{x}}_{Age by country})}_{i} + B_{2} \cdot {(Age - {\bar{x}}_{Age by country})}_{i}^{2} + B_{3} \cdot {Sex}_{i} + B_{4} \cdot {MLPE}_{i}$ . 2) Residual scores were calculated based on the final model ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). 3) Residuals were standardized using the residual Standard Deviation (SD_e) value provided by the regression model: z_i = e_iSD_e. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied for ROCF copy and immediate recall (3 minutes) scores separately for eachcountry.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values areconsidered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

3 Results

3.1 ROCF copy

The final multivariate linear regression models for the ten country-specific ROCF copy scores were significant (see Table 1). In all countries, the ROCF copy scores increased linearly as a function of age. The ROCF copy scores for all countries except for Puerto Rico were affected by a quadratic age effect. Children from Chile, Puerto Rico, and Spain whose parent(s) had a MLPE >12 years obtained higher ROCF copy scores than children whose parent(s) had an MLPE≤12 years. The child’s sex affected their ROCF copy scores in Chile and Puerto Rico, such that girls scored higher than boys. The amount of variance these predictors explained in the ROCF copy scores ranged from 25.5% (in Guatemala) to 63.6% (in Cuba).

Table 1
Final multiple linear regression models for ROCF copy

Country B Std. Error t Sig. R² SDe (residual)

Chile

Constant 24.574 0.533 46.086 <0.001 0.520 5.447

Age 1.586 0.082 19.336 <0.001

Age² –0.128 0.027 –4.774 <0.001

MLPE 1.467 0.601 2.439 0.015

Sex –1.695 0.569 –2.978 0.003

Cuba

Constant 33.332 0.287 116.031 <0.001 0.636 3.701

Age 1.301 0.055 23.632 <0.001

Age² –0.180 0.018 –9.936 <0.001

Ecuador

Constant 29.245 0.503 58.176 <0.001 0.308 5.735

Age 1.071 0.096 11.115 <0.001

Age² –0.121 0.032 –3.794 <0.001

Guatemala

Constant 31.112 0.481 64.648 <0.001 0.255 5.395

Age 1.300 0.159 8.161 <0.001

Age² –0.161 0.046 –3.489 0.001

Honduras

Constant 29.267 0.429 68.253 <0.001 0.464 5.066

Age 1.455 0.093 15.666 <0.001

Age² –0.152 0.030 –5.130 <0.001

Mexico

Constant 30.042 0.322 93.240 <0.001 0.326 6.473

Age 1.194 0.061 19.541 <0.001

Age² –0.170 0.020 –8.449 <0.001

Paraguay

Constant 29.043 0.572 50.789 <0.001 0.380 6.367

Age 1.327 0.106 12.547 <0.001

Age² –0.185 0.036 –5.140 <0.001

Peru

Constant 32.804 0.430 76.276 <0.001 0.346 5.299

Age 0.951 0.088 10.756 <0.001

Age² –0.153 0.029 –5.261 <0.001

Puerto Rico

Constant 27.567 0.917 30.077 <0.001 0.443 5.988

Age 1.445 0.124 11.646 <0.001

MLPE 2.049 0.989 2.071 0.040

Sex –4.030 0.887 –4.542 <0.001

Spain

Constant 32.684 0.296 110.437 <0.001 0.487 4.622

Age 1.232 0.044 27.759 <0.001

Age² –0.197 0.014 –13.765 <0.001

MLPE 1.578 0.309 5.105 <0.001

Country	B	Std. Error	t	Sig.	R²	SDe (residual)
Chile
Constant	24.574	0.533	46.086	<0.001	0.520	5.447
Age	1.586	0.082	19.336	<0.001
Age²	–0.128	0.027	–4.774	<0.001
MLPE	1.467	0.601	2.439	0.015
Sex	–1.695	0.569	–2.978	0.003
Cuba
Constant	33.332	0.287	116.031	<0.001	0.636	3.701
Age	1.301	0.055	23.632	<0.001
Age²	–0.180	0.018	–9.936	<0.001
Ecuador
Constant	29.245	0.503	58.176	<0.001	0.308	5.735
Age	1.071	0.096	11.115	<0.001
Age²	–0.121	0.032	–3.794	<0.001
Guatemala
Constant	31.112	0.481	64.648	<0.001	0.255	5.395
Age	1.300	0.159	8.161	<0.001
Age²	–0.161	0.046	–3.489	0.001
Honduras
Constant	29.267	0.429	68.253	<0.001	0.464	5.066
Age	1.455	0.093	15.666	<0.001
Age²	–0.152	0.030	–5.130	<0.001
Mexico
Constant	30.042	0.322	93.240	<0.001	0.326	6.473
Age	1.194	0.061	19.541	<0.001
Age²	–0.170	0.020	–8.449	<0.001
Paraguay
Constant	29.043	0.572	50.789	<0.001	0.380	6.367
Age	1.327	0.106	12.547	<0.001
Age²	–0.185	0.036	–5.140	<0.001
Peru
Constant	32.804	0.430	76.276	<0.001	0.346	5.299
Age	0.951	0.088	10.756	<0.001
Age²	–0.153	0.029	–5.261	<0.001
Puerto Rico
Constant	27.567	0.917	30.077	<0.001	0.443	5.988
Age	1.445	0.124	11.646	<0.001
MLPE	2.049	0.989	2.071	0.040
Sex	–4.030	0.887	–4.542	<0.001
Spain
Constant	32.684	0.296	110.437	<0.001	0.487	4.622
Age	1.232	0.044	27.759	<0.001
Age²	–0.197	0.014	–13.765	<0.001
MLPE	1.578	0.309	5.105	<0.001

Note. MLPE: Mean level of parental education.

3.2 ROCF immediate recall (3 minutes)

The final multivariate linear regression models for the ten country-specific ROCF immediate recall scores were significant (see Table 2). In all countries, the ROCF immediate recall score increased linearly as a function of age. The ROCF immediate recall scores were affected by a quadratic age effect for all countries except, Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Children from Paraguay and Spain whose parent(s) had a MLPE >12 years obtained higher ROCF immediate recall scores than children whose parent(s) had a MLPE≤12 years. The child’s sex affected ROCF immediate recall score for Chile and Puerto Rico, such that girls scored higher than boys. The amount of variance these predictors explained in the ROCF immediate recall scores ranged from 24.8% (in Ecuador) to 53.4% (in Cuba).

Table 2
Final multiple linear regression models for ROCF immediate recall (3 minutes)

Country B Std. Error t Sig. R² SDe (residual)

Chile

Constant 14.300 0.399 35.883 <0.001 0.491 5.507

Age 1.550 0.082 18.911 <0.001

Sex –1.205 0.567 –2.125 0.034

Cuba

Constant 24.147 0.525 46.009 <0.001 0.534 6.724

Age 2.105 0.103 20.343 <0.001

Age² –0.144 0.034 –4.227 <0.001

Ecuador

Constant 18.517 0.604 30.670 <0.001 0.248 6.889

Age 1.126 0.116 9.729 <0.001

Age² –0.102 0.038 –2.671 0.008

Guatemala

Constant 18.731 0.427 43.890 <0.001 0.287 6.036

Age 1.520 0.170 8.957 <0.001

Honduras

Constant 16.603 0.333 49.917 <0.001 0.415 5.702

Age 1.486 0.103 14.424 <0.001

Mexico

Constant 18.550 0.336 55.193 <0.001 0.312 6.745

Age 1.267 0.064 19.871 <0.001

Age² –0.111 0.021 –5.284 <0.001

Paraguay

Constant 16.247 0.616 26.356 <0.001 0.325 6.354

Age 1.210 0.106 11.406 <0.001

MLPE 2.348 0.771 3.046 0.003

Peru

Constant 21.466 0.558 38.445 <0.001 0.414 6.848

Age 1.555 0.114 13.602 <0.001

Age² –0.149 0.038 –3.960 <0.001

Puerto Rico

Constant 18.346 0.637 28.814 <0.001 0.352 6.576

Age 1.247 0.131 9.482 <0.001

Sex –4.558 0.934 –4.878 <0.001

Spain

Constant 19.893 0.414 48.108 <0.001 0.412 6.457

Age 1.573 0.062 25.306 <0.001

Age² –0.127 0.020 –6.329 <0.001

MLPE 2.501 0.432 5.785 <0.001

Country	B	Std. Error	t	Sig.	R²	SDe (residual)
Chile
Constant	14.300	0.399	35.883	<0.001	0.491	5.507
Age	1.550	0.082	18.911	<0.001
Sex	–1.205	0.567	–2.125	0.034
Cuba
Constant	24.147	0.525	46.009	<0.001	0.534	6.724
Age	2.105	0.103	20.343	<0.001
Age²	–0.144	0.034	–4.227	<0.001
Ecuador
Constant	18.517	0.604	30.670	<0.001	0.248	6.889
Age	1.126	0.116	9.729	<0.001
Age²	–0.102	0.038	–2.671	0.008
Guatemala
Constant	18.731	0.427	43.890	<0.001	0.287	6.036
Age	1.520	0.170	8.957	<0.001
Honduras
Constant	16.603	0.333	49.917	<0.001	0.415	5.702
Age	1.486	0.103	14.424	<0.001
Mexico
Constant	18.550	0.336	55.193	<0.001	0.312	6.745
Age	1.267	0.064	19.871	<0.001
Age²	–0.111	0.021	–5.284	<0.001
Paraguay
Constant	16.247	0.616	26.356	<0.001	0.325	6.354
Age	1.210	0.106	11.406	<0.001
MLPE	2.348	0.771	3.046	0.003
Peru
Constant	21.466	0.558	38.445	<0.001	0.414	6.848
Age	1.555	0.114	13.602	<0.001
Age²	–0.149	0.038	–3.960	<0.001
Puerto Rico
Constant	18.346	0.637	28.814	<0.001	0.352	6.576
Age	1.247	0.131	9.482	<0.001
Sex	–4.558	0.934	–4.878	<0.001
Spain
Constant	19.893	0.414	48.108	<0.001	0.412	6.457
Age	1.573	0.062	25.306	<0.001
Age²	–0.127	0.020	–6.329	<0.001
MLPE	2.501	0.432	5.785	<0.001

Note. MLPE: Mean level of parental education.

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF≤1.097; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.080 in a F_(2,201) distribution which correspond to percentile 6).

3.3 Normative procedure

Norms (e.g., a percentile score) for the different ROCF copy or immediate recall scores by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate the understanding of the procedure used to obtain the percentile associated with a score on this test. To find the percentile score for a 12-year-old Chilean boy who scored a 21 on the ROCF immediate recall (3 minutes) test and whose parent(s) have a mean of 14 years of education (MLPE). The steps to obtain the percentile for this score are: 1) Find Chile in Table 2, which provides the final regression models by country for the ROCF immediate recall score. Use the B weights to create an equation that will allow you to obtain the predicted ROCF immediate recall score for this child using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Chilean sample, which is equal to 11.5 years), and sex, which was coded as male = 1 and female = 0. Age² and MLPE were not significant predictors, and therefore are not included in this model. See Rivera and Arango-Lasprilla (2017) to figure out the mean age of each country’s sample. Finally, the result is added to the constant generated by the model in order to calculate the predicted value.

In the case of the Chilean boy, the predicted ROCF immediate recall score would be calculated using the following equation: ${\hat{y}}_{i} = 14.300 + [1.550 \cdot ({Age}_{i} - 11.5)] + (- 1.205 \cdot {Sex}_{i})$ The boy’s age is 12. The MLPE (14 years) is split into either 1 to 12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model, but since MLPE was not a significant predictor in this case, is not included in the model. Sex was coded as male = 1 and female = 0, so in this case as the child is a male, the sex value is 1. Thus, the predicted value equation is: ${\hat{y}}_{i} = 14.300 + [1.550 \cdot (12 - 11.5)] + (- 1.205 \cdot 1) = 14.300 + 0.775 - 1.205 = 13.870$ . 2) In order to calculate the residual value (indicated with an e_i in the equation), we subtract the actual ROCF immediate recall test score (he scored 21) from the predicted value we just calculated ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). In this case, it would be e_i = 21 - 13.870 = 7.130. 3) Next, consult the SD_e column in Table 2 to obtain the country-specific SD_e (residual) value. For Chile, it is 5.507. Using this value, we can transform the residual value to a standardized z score using the equation (z_i = e_i/SD_e). In this case, we have 7.130/5.507 = 1.295. This is the standardized z score for a 12-year-old Chilean boy who scored a 21 on the ROCF immediate recall who has parent(s) with 14 years of education (MLPE). 4) The last step is to use the tables available in most statistical reference books (e.g., Strauss, Sherman, & Spreen, 2006) to convert z scores to percentiles. In this example, the z score (probability) of 1.295 corresponds to the 90^th percentile. It is important to remember to use the appropriate tables that correspond to each test (copy vs. immediate recall) when performing these calculations.

3.4 User-friendly normative data

The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the ROCF copy or immediate recall test. However, this method can be prone to human error due to the number of hand calculations necessary. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 21 on the ROCF immediate recall test for a Chilean boy who is 12 years old whose parent(s) have 14 years of education) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and test (copy vs. immediate recall). In this case, the table for the ROCF immediate recall score for boys from Chile can be found in Table A13. (2) Find the appropriate age of the child, in this case, 12 years old. (3) Next, look in the 12 years’ age column to find the approximate location of the raw score obtained on the test. Within the 12 years’ column, the score of 21 obtained by this Chilean boy corresponds to an approximate percentile of 90.

The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.

4 Discussion

The ROCF is one of the most widely used neuropsychological tests in the world to evaluate visuospatial, visual-motor, and visual memory processes in both children and adults (Frisk et al., 2005). In Latin American countries and in Spain, this test is among the 10 most utilized neuropsychological tests by clinical neuropsychologists during their professional practice (Arango-Lasprilla, Stevens, Morlett-Paredes, Ardila, & Rivera, 2016; Olabarrieta-Landa et al., 2016). However, in spite of its great use, there are currently very few studies on validation and standardization of this test for Spanish-speakers. The vast majority of the studies have been done with adult populations, and to date very few studies have been conducted with pediatric populations. Therefore, neuropsychologists who use this test in Latin American countries or in Spain with pediatric populations usually perform their interpretation using norms from other countries (Arango-Lasprilla et al., 2016). Thus, there was an overwhelming need for normative data of the ROCF for pediatric populations in both Latin American countries and Spain (Arango-Lasprilla et al., 2016; Olabarrieta-Landaet al., 2016).

To fill this gap in the literature, the purpose of this study was to obtain normative data for the ROCF copy and immediate recall (3 minutes) scores for children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, México, Paraguay, Peru, and Puerto Rico) and Spain. The results of the study showed that there are different types of variables associated with the performance of the test such as age, quadratic age, sex, and MLPE. In general, it was found that the final regression models explained between 25.5% and 63.6% of the variance for the ROCF copy, and between 24.8% and 53.4% of the variance for the ROCF immediate recall.

Age was significantly related to the total score of both the ROCF copy and the ROCF immediate recall, so that the scores increase linearly as children become older. These results are similar to those reported in other studies in which ROCF scores were found to increase significantly with age (Frisk et al., 2005; Meyers & Meyers, 1996). Additionally, a curvilinear effect of age on the ROCF copy was shown in all countries, except Puerto Rico. Scores increased prominently between ages 6 and 13, while that increase slowly decreased after the ages 13–14 approximately, to later stabilize and resemble that of adults. Other studies, however, have found that the increase in scores begins to occur mainly between the ages of 12 and 16, being at age 17 when performance is matched to that of adults (Meyers & Meyers, 1996).

A curvilinear effect was also observed for the ROCF immediate recall for all countries, except for Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Score increased prominently between the ages 6 and 15, followed by a smaller increase between the ages 16 and 17 years of age. Past research has found that for the ROCF immediate and delay recall, there is a large increase in scores between the ages 6 to 12 followed by smaller increases between the ages of 12 and 17 (Anderson & Lajoie, 1996; Boone, Lesser, Hill-Gutierrez, Berman, & D’elia, 1993; Caffarra, Vezzadini, Dieci, Zonato, & Venneri, 2002; Chervinsky, Mitrushina, & Satz, 1992; Chiulli, Haaland, Larue, & Garry, 1995; Denman, 1984; Hartman & Potter, 1998; Kramer & Wells, 2004; Meyers & Meyers, 1995, 1996; Miatton, Wolters, Lannoo, & Vingerhoets, 2004; Pontón et al., 1996; Van Gorp, Satz, & Mitrushina, 1990).

On the other hand, the results of the present study contradict those found by Beltrán Dulcey and Solís-Uribe (2012). In their study, the ROCF copy and the ROCF immediate recall scores were not associated with age. One possible explanation might be the small sample size (141 children) used. Another potential explanation is that Beltrán Dulcey and Solís-Uribe (2012) divided the sample into two age ranges (9 to 12 years and 13 to 16) to compare the performance of these two groups using an independent measures t-test. In the present study, the variable age was analyzed as a continuous variable, and the sample sizes were symmetrical for each age. Subsequently, the effect of age on each test score was calculated by multiple linear regressions.

Sex was not associated with the test performance neither for the ROCF copy nor for the ROCF immediate recall in the vast majority of countries. These results are similar to those reported in other studies in which sex was not found to be associated with the performance of this test (Beltrán Dulcey & Solís-Uribe, 2012; Poulton, & Moffitt, 1995). However, sex was associated with copy and immediate recall scores in Chile and Puerto Rico, such that Chilean and Puerto Rican girls scored better than boys.

Despite the little attention paid to parents’ education levels in the standardization of neuropsychological tests for pediatric populations, research supports parents’ level of education as influential in children’s development of the cognitive functions (Schady, 2011). In this study, parents’ MLPE was associated with ROCF copy scores in Chile, Puerto Rico and Spain and with ROCF immediate recall scores in Paraguay and Spain. In both cases, children whose parent(s) had more than 12 years of education scored significantly higher than children whose parent(s) had an education below 12 years.

The results of the present study have notable clinical implications. Established norms for the ROCF for ten Spanish-speaking countries provide an excellent opportunity for clinical neuropsychologists to use this test as part of their neuropsychological evaluation protocols with the objective to evaluate alterations in visuospatial, visual-motor, and visual memory processes in pediatric populations between 6 and 17 years of age. Using these norms, the performance of each child can be assessed in a more exact and standardized manner according to their age, gender, and the education of their parent(s). The creation of standardized norms will improve neuropsychological evaluation and diagnosis both in normal and clinical populations, since deficits are usually present in children with learning disabilities (Kirkwood, Weiler, Bernstein, Forbes, & Waber, 2001), brain injury (Verger et al., 2000), autism (Czermainski, Riesgo, Guimarães, Salles, & Bosa, 2014), and attention and hyperactivity disorder (Rizzutti et al., 2008), among others. Furthermore, norms to value the neuropsychological performance of children with impairments will inform their prognostic course and facilitate the implementation of cognitive rehabilitation programs among populations who may benefit.

4.1 Limitations

Despite the strengths of the present study in filling a gap in the literature and providing the largest sample to validate and standardize the ROCF with Spanish-speaking populations, the results of the study should be interpreted in light of several limitations. This study presents normative data of the ROCF for nine countries in Latin America and Spain. For this reason, it is not advisable to use these norms to assess pediatric populations in Spanish-speaking countries not included in the present study. Future research should be conducted to standardize the ROCF in other Spanish-speaking countries.

Although the norms generated by the present study could be used by neuropsychologists in other countries to evaluate Spanish speaking immigrant children from any of the nine Latin American countries or Spain included in our sample, we caution this use due to the potential influence of other variables not assessed in the present study, such as level of acculturation, bilingualism, and number of years living in the country. In addition, the quality of education of both the child and the parent(s) is another aspect that may influence the cognitive performance of children.

The ROCF is one of the most used instruments in the world to measure visuospatial, visual-motor, and visual memory problems. However, it is essential to consider that no clinical diagnosis should be made based solely on the scores of this test. Scores should be integrated and interpreted as one part of a much larger battery that evaluates these processes in greater detail. Because there are a limited number of tests and norms in Latin America and Spain to evaluate these processes, standardization efforts should be made in future research for other similar assessments.

Although the size of the sample was adequate in each of the countries where the study was conducted, it is important to note that only the samples in Chile, Mexico, Paraguay, Puerto Rico, and Spain were obtained from several regions of the countries, whereas in the remaining countries the samples were collected from one geographical area. Future studies should expand data collection to other geographical areas of these countries to improve representativeness and generalizability.

Spanish was the first language for the children who participated in the present study. Despite Spanish being the first language for the majority of the population in Latin America and Spain, it is important to consider the cultural and linguistic richness of these countries. For example, the first language of many children may be completely different from Spanish (e.g., Portuguese, Euskera, Catalan, Guaraní, Maya, Quechua). For this reason, caution should be used when using these norms in children whose first language is not Spanish.

Finally, participants in the present study all represent a normal, healthy population. Future studies should be performed with clinical populations to establish the sensitivity and specificity of this test.

5 Conclusions

The ROCF is one of the most widely used neuropsychological tests for evaluating visuospatial, visual-motor, and visual memory processes in children and adolescents in Latin America and Spain. Present study findings established standardized norms for ten countries, and results indicated that age, sex, and parental education level influence ROCF scores. These variables ought to be taken into account when interpreting children’s scores. These norms provide neuropsychologists in these countries a valid assessment tool that can be used in everyday practice with pediatric populations.

Conflict of interest

None to report.

Footnotes

The Appendix tables are available in the electronic version of this article: .

References

Aiken

L. S.

, & West

S. G.

(1991). Multiple regression: Testing and interpreting interactions,. Newbury Park, CA: Sage.

Akshoomoff

N. A.

, Feroleto

C. C.

, Doyle

R. E.

, & Stiles

(2002). The impact of early unilateral brain injury on perceptual organization and visual memory. Neuropsychologia, 40(5), 539–561. doi: 10.1016/s0028-3932(01)00129-4

Anderson

V. A.

, & Lajoie

(1996). Development of memory and learning skills in school-aged children: A neuropsychological perspective. Applied Neuropsychology, 3(3-4), 128–139.

Arango-Lasprilla

J. C.

, Stevens

, Morlett Paredes

, Ardila

, & Rivera

(2016). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24(4), 318–330. doi: 10.1080/23279095.2016.1185423

Ardila

, & Rosselli

(1994). Development of language, memory, and visuospatial abilities in 5- to 12-year-old children using a neuropsychological battery. Developmental Neuropsychology, 10(2), 97–120. doi: 10.1080/87565649409540571

Beltrán Dulcey

, & Solís Uribe

(2012). Evaluación neuropsicológica en adolescentes: Normas para población de Bucaramanga. Revista Neuropsicología Neuropsiquiatría y Neurociencias, 12(2), 77–93.

Boone

K. B.

, Lesser

I. M.

, Hill-Gutierrez

, Berman

N. G.

, & D’elia

L. F.

(1993). Rey-Osterrieth Complex Figure performance in healthy, older adults: Relationship to age, education, sex, and IQ. The Clinical Neuropsychologist, 7(1), 22–28.

Brown

, Sherbenou

R. J

, & Johnsen

S. K.

(2009)., Test de inteligencia no verbal TONI-2. Madrid: TEA ediciones.

Caffarra

, Vezzadini

, Dieci

, Zonato

, & Venneri

(2002). Rey-Osterrieth complex figure: Normative values in an Italian population sample. Neurological Sciences, 22(6), 443–447.

10.

Chervinsky

A. B.

, Mitrushina

, & Satz

(1992). Comparison of four methods of scoring the Rey-Osterrieth Complex Figure Drawing Test on four age groups of normal elderly. Brain Dysfunction, 5(5), 267–287.

11.

Chiulli

S. J.

, Haaland

K. Y.

, Larue

, & Garry

P. J.

(1995). Impact of age on drawing the Rey-Osterrieth Figure. The Clinical Neuropsychologist, 9(3), 219–224.

12.

Cook

R. D.

(1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. doi: 10.2307/1268249

13.

Czermainski

F. R.

, Riesgo

R. D. S.

, Guimarães

L. S. P.

, Salles

J. F. D.

, & Bosa

C. A.

(2014). Executive functions in children and adolescents with autism spectrum disorder. Paidéia (Ribeirão Preto), 24(57), 85–94.

14.

De Leeuw

C. E.

(2010). Adjusting Rey Osterrieth Complex Figure Test recall performance for copy integrity. Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI Number: 3485983.

15.

Demsky

, Carone

D. A.

Jr , Burns

W. J.

, & Sellers

(2000). Assessment of visual-motor coordination in 6-to 11-yr.-olds. Perceptual and Motor Skills, 91(1), 311–321. doi: 10.2466/pms.2000.91.1.311

16.

Denman

(1984). Denman Neuropsychology Memory Scale: A Clinical Assessment of Immediate Recall, Short Term Memory and Long Term Memory in Verbal and Nonverbal Areas. Charleston, SC: Sydney B. Denman.

17.

Fastenau

P. S.

(1996). Development and preliminary standardization of the “Extended Complex Figure Test” (ECFT. Journal of Clinical and Experimental Neuropsychology, 18(1), 63–76. doi: 10.1080/01688639608408263

18.

Frisk

, Jakobson

L. S.

, Knight

R. M.

, & Robertson

(2005). Copy and recall performance of 6-8-year-old children after standard vs. step-by-step administration of the Rey-Osterrieth Complex Figure. Child Neuropsychology, 11(2), 135–152. doi: 10.1080/092970490911289

19.

Galindo

, & Cortes

J. F.

(2003). The ROCF and the complex figure for children in Spanish speaking populations. In Knight

J. A.

& Kaplan

(Eds.), The handbook of Rey-Osterrieth complex figure usage: Clinical and research applications. Lutz, FL: Psychological Assessment Resources.

20.

Hartman

, & Potter

(1998). Sources of age differences on the Rey-Osterrieth complex figure test. The Clinical Neuropsychologist, 12(4), 513–524.

21.

Hooper

H. E.

(1983). Hooper Visual Organization Test (HVOT). Los Angeles, CA, United States: Western Psychological Services.

22.

Karapetsas

A. B

, & Kantas

(1991). Visuomotor organization in the child: A neuropsychological approach. Perceptual and Motor Skills, 72, 211–217.

23.

Kasai

, Meguro

, Hashimoto

, Ishizaki

, Yamadori

, & Mori

(2006). Non-verbal learning is impaired in very mild Alzheimer’s disease (CDR 0.5): Normative data from the learning version of the Rey–Osterrieth Complex Figure Test. Psychiatry and Clinical Neurosciences, 60(2), 139–146. doi: 10.1111/j.1440-1819.2006.01478.x

24.

Kirkwood

M. W.

, Weiler

M. D.

, Bernstein

J. H.

, Forbes

J. H.

, & Waber

D. P.

(2001). Sources of poor performance on the Rey-Osterrieth Complex Figure Test among children with learning difficulties: A dynamic assessment approach. Clinical Neuropsychology, 15(3), 345–350.

25.

Kolb

, & Whishaw

I. Q.

(1985). Earlier is not always better: Behavioral dysfunction and abnormal cerebral morphogenesis following neonatal cortical lesions in the rat. Behoural Brain Research, 17(1), 25–43.

26.

Kovacs

(1992). Children’s depression inventory. North Tonawanda, NY: Multi-Health System.

27.

Kramer

J. H.

, & Wells

A. M.

(2004). The role of perceptual bias in complex figure recall. Journal of Clinical and Experimental Neuropsychology, 26(6), 838–845.

28.

Kutner

M. H.

, Nachtsheim

C. J.

, Neter

, & Li

(2005). Applied linear statistical models (5th ed.). New York: McGraw Hill.

29.

Matthews

C. G

, & Kløve

(1964). Instruction Manual for the Adult Neuropsychology Test Battery. Madison, WI, United States: University of Wisconsin Medical School.

30.

Meyers

J. E.

, & Meyers

K. R.

(1995). Rey Complex Figure Test and recognition trial professional manual. Psychological Assessment Resources.

31.

Meyers

J. E.

, & Meyers

K. R.

(1996). Rey Complex Figure Test and recognition trial: Professional manual, supplemental norms for children and adolescents. Lutz, FL, United States: Psychological Assessment Resources, Inc.

32.

Miatton

, Wolters

, Lannoo

, & Vingerhoets

, (2004). Updated and extended Flemish normative data of commonly used neuropsychological tests. Psychologica Belgica, 44, 189–216.

33.

Mitrushina

, Boone

K. B.

, Razani

, & D’Elia

L. F.

(2005). Handbook of normative data for neuropsychological assessment (2nd ed.). New York, NY, United States: Oxford University Press.

34.

Olabarrieta-Landa

, Caracuel

, Pérez-García

, Panyavin

, Morlett-Paredes

, & Arango-Lasprilla

J. C.

(2016). The profession of neuropsychology in Spain: Results of a national survey. Clinical Neuropsychology, 30(8), 1335–1355.

35.

Osterrieth

P. A.

(1944). Le test de copie d’une figure complexe. Arch Psychol, 30, 206–356.

36.

Pontón

M. O.

, Satz

, Herrera

, Ortiz

, Urrutia

C. P.

, Young

, ... Namerow

(1996). Normative data stratified by age and education for the Neuropsychological Screening Battery for Hispanics (NeSBHIS): Initial report. Journal of the International Neuropsychological Society, 2(02), 96–104.

37.

Poulton

R. G.

, & Moffitt

T. E.

(1995). The Rey-Osterreith Complex Figure Test: Norms for young adolescents and an examination of validity. Archives of Clinical Neuropsychology, 10(1), 47–56.

38.

Rey

(1941). L’examen psychologique dans les cas d’encephalopathie traumatique [The psychological examination of cases of traumatic encephalopathy]. Archives de Psychologie, 37, 126–139.

39.

Rey

(2009). REY: Test de copia y de reproducción de memoria de figuras geométricas complejas. Madrid: TEA ediciones.

40.

Rivera

, & Arango-Lasprilla

J. C.

(2017). Methodology for the development of normative data for Spanish Speaking pediatric population. NeuroRehabilitation, 41(3), 581–592.

41.

Rizzutti

, Sinnes

E. G.

, Scaramuzza

L. F.

, Freitas

, Pinheiro

, Palma

S. M.

, ... Muszkat

(2008). Clinical and neuropsychological profile in a sample of children with attention deficit hyperactivity disorders. Arquivos de neuro-psiquiatria, 66(4), 821–827.

42.

Rosselli

, & Ardila

(1991). Effects of age, education, and gender on the Rey-Osterrieth Complex Figure. The Clinical Neuropsychologist, 5(4), 370–376. doi: 10.1080/13854049108404104

43.

Schady

(2011). Parents’ education, mothers’ vocabulary, and cognitive development in early childhood: Longitudinal evidence from Ecuador. American Journal of Public Health, 101(12), 2299–2307.

44.

Shin

M. S.

, Park

S. Y.

, Park

S. R.

, Seol

S. H.

, & Kwon

J. S.

(2006). Clinical and empirical applications of the Rey-Osterrieth complex figure test. Nature Protocols, 1(2), 892–899. doi: 10.1038/nprot.2006.115

45.

Strauss

, Sherman

E. M.

, & Spreen

(2006). A compendium of neuropsychological tests: Administration, norms, and commentary. American Chemical Society.

46.

Van Gorp

W. G.

, Satz

, & Mitrushina

(1990). Neuropsychological processes associated with normal aging. Developmental Neuropsychology, 6(4), 279–290.

47.

Verger

, Junqué

, Jurado

M. A.

, Tresserras

, Bartumeus

, Nogues

, & Poch

J. M.

(2000). Age effects on long-term neuropsychological outcome in paediatric traumatic brain injury. Brain Injury, 14(6), 495–503.

48.

Waber

D. P.

, & Bernstein

J. H.

(1995). Performance of learning-disabled and non-learning-disabled children on the Rey-Osterrieth Complex Figure: Validation of the developmental scoring system. Developmental Neuropsychology, 11(2), 237–252. doi: 10.1080/87565649509540616

49.

Waber

D. P.

, & Holmes

J. M.

(1985). Assessing children’s copy productions of the Rey-Osterrieth Complex Figure. Journal of Clinical and Experimental Neuropsychology, 7(3), 264–280. doi: 10.1080/01688638508401259

50.

Waber

D. P.

, & Holmes

J. M.

(1986). Assessing children’s memory productions of the Rey-Osterrieth Complex Figure. Journal of Clinical and Experimental Neuropsychology, 8(5), 563–580. doi: 10.1080/01688638608405176

51.

Watanabe

, Ogino

, Nakano

, Hattori

, Kado

, Sanada

, & Ohtsuka

(2005). The Rey–Osterrieth Complex Figure as a measure of executive function in childhood. Brain and Development, 27(8), 564–569. doi: 10.1016/j.braindev.2005.02.007

52.

Wechsler

(1989). Wechsler Preschool and Primary Scale of Intelligence-Revised. Toronto, Ontario, Canada: Harcourt Brace Jovanovich, Inc.