Abstract
OBJECTIVE:
To generate normative data for the Rey–Osterrieth Complex Figure (ROCF) in Spanish-speaking pediatric populations.
METHOD:
The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the ROCF as part of a larger neuropsychological battery. The ROCF copy and immediate recall (3 minutes) scores were normed using multiple linear regressions and standard deviations of residual values. Age, age2, sex, and mean level of parental education (MLPE) were included as predictors in the analyses.
RESULTS:
The final multiple linear regression models showed main effect for age on copy and immediate recall scores, such that scores increased linearly as a function of age. Age2 affected ROCF copy score for all countries, except Puerto Rico; and ROCF immediate recall scores for all countries, except Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Models indicated that children whose parent(s) had a MLPE >12 years obtained higher scores compared to children whose parent(s) had a MLPE≤12 years for Chile, Puerto Rico, and Spain in the ROCF copy, and Paraguay and Spain for the ROCF immediate recall. Sex affected ROCF copy and immediate recall score for Chile and Puerto Rico with girls scoring higher than boys.
CONCLUSIONS:
This is the largest Spanish-speaking pediatric normative study in the world, and it will allow neuropsychologists from these countries to have a more accurate approach to interpret the ROCF Test in pediatric populations.
Keywords
Introduction
In 1941, Swiss psychologist André Rey designed the Rey-Osterrieth Complex Figure Test (ROCF) to examine visuospatial ability and memory in patients with traumatic brain injuries (Rey, 1941). The ROCF was later standardized by Paul-Alexandre Osterrieth in 1944, who proposed a scoring system for administration and collected the first set of normative data for children and adults (Shin, Park, Park, Seol, & Kwon, 2006). Since its initial validation, the ROCF has become one of the most widely used neuropsychological assessments to assess constructional and non-verbal memory abilities (Ardila & Rosselli, 1994; Rosselli & Ardila, 1991).
The ROCF uses an intricate stimulus that is asymmetrical in its design to assess cognitive performance through recognition and recall skills (Fastenau, 1996; Shin et al., 2006). Many cognitive abilities are necessary for good performance, and as such, the test is used to evaluate several different brain functions, including attention, working memory, visuospatial abilities, and planning (Watanabe et al., 2005). For example, the ROCF task involves viewing a complex figure and copying it; next, the individual reproduces it from memory, either immediately, following a delay, or both (Shin et al., 2006). The ROCF is often used to examine deficits due to traumatic brain injury in neurological patients, to test for dementia, and to study children’s cognitive development (Kasai et al., 2006; Watanabe et al., 2005). Both young, developing children and older children or adults with weaknesses in the aforementioned abilities typically find the task quite challenging (Akshoomoff, Feroleto, Doyle, & Stiles, 2002; Waber & Holmes, 1985).
The ROCF has received psychometric support for both reliability and validity in past research with pediatric populations. Reliability among pediatric populations tends to be high for both copy production (α= 0.95) and recall production (α= 0.94; Waber & Holmes, 1985). Evidence for convergent validity is also supported: six to eight year-old children’s scores on the ROCF correlate with other measures of visuospatial, constructional, and fine motor ability [e.g., Hoover Visual Organization Test (Hooper, 1983), Block Design subtest of the Wechsler Primary and Preschool Scale of Intelligence-Revised (Wechsler, 1989), the Grooved Pegboard (Matthews & Kløve, 1964; Frisk, Jakobson, Knight, & Robertson, 2005)]. The ROCF’s psychometric support has contributed to its popularity as a widely used neurological assessment.
Several variables have been found to influence test performance among pediatric populations, although age has the most notable influence (Beltrán Dulcey & Solís Uribe, 2012; De Leeuw, 2010; Rosselli & Ardila, 1991). Copy scores tend to increase between the ages of 12 and 16 (Meyers & Meyers, 1996), and children between the ages of 13 and 16 tend to perform better on copying the figure accurately than children who are nine to 12 years old (Beltrán Dulcey & Solís Uribe, 2012). Age also greatly influences ability for children and adolescents to recall the figure, with scores increasingly greatly between ages six and 12, with slower growth between ages 12 and 17 (Meyers & Meyers, 1996; Mitrushina, Boone, Razani, & D’Elia, 2005). Evidence for a gender difference on the ROCF has been equivocal. Two studies found conflicting findings, such that girls outperformed boys between the ages of eight and 12 in one study (Karapetsas & Kantas, 1991), and boys outperformed girls in the other (Ardila & Rosselli, 1994). However, other studies have not found a gender difference (Beltrán Dulcey & Solís Uribe, 2012; Demsky, Carone, Burns, & Sellers, 2000). Although educational level has been found to influence adults’ performance on the ROCF, an effect for pediatric populations has not been supported (Beltrán Dulcey & Solís Uribe, 2012; Meyers & Meyers, 1995). Finally, performance on the ROCF has also been found to be directly related to intellectual ability, such that individuals with learning disabilities exhibit little improvement on the test as they age (Waber & Bernstein, 1995).
Normative data has been derived for children between the ages of five and 14 in the US, Mexico, and Colombia (Ardila & Rosselli, 1994; Galindo & Cortes, 2003; Karapetsas & Kantas, 1991; Waber & Holmes, 1985, 1986). Using qualitative scoring systems, children’s ability to copy and recall the figure from memory was assessed on several dimensions, including accuracy (e.g., how much of the design was recalled); errors (i.e., distortions); organization (ability to format the figure; e.g., align all four sides); and style (i.e., continuity of lines; Shin et al., 2006). Between the ages of six and eight, children’s ability to copy the figure improves dramatically (Waber & Holmes, 1985), most likely due to an improvement in approach and organization. By age nine, children can reliably produce all parts of the design, and changes after that age tend to reflect the increased capacity to plan and organize reproduction of the figure (Ardila & Rosselli, 1994; Waber & Holmes, 1985). Six year-old children score between 14.5 and 16.5 on copying the image, whereas children ages 14 and above begin to score more similarly to adults, whose average score tends to be around 32 (Ardila & Rosselli, 1994; Kolb & Whishaw, 1985; Meyers & Meyers, 1996; Rosselli & Ardila, 1991; Shin et al., 2006).
In terms of recalling the figure from memory, in past research 57% of children between the ages of five and 14 were able to reproduce the design from memory immediately after copying it, and 43% were able to after a 20-minute delay (Waber & Holmes, 1986). At every age level, more errors are made when recalling the image than when copying the image; in addition, errors tend to decrease at each age level, such that five year-olds tend to make significantly more errors than older children (Waber & Holmes, 1986). In terms of organization scores, five year-old children tend to score the lowest, and these scores generally increase each year up through age 14 (Waber & Holmes, 1986). Regarding style, younger children tended to focus on specific parts of the figure, whereas older children began to represent the figure as a configural whole. It is around age six that children begin to show sensitivity to both individual features of a figure and the overall configuration. Up until that point, younger children tend to perform more accurately when recalling the left side of the figure than the right side (Karapetsas & Kantas, 1991; Waber & Holmes, 1985).
Few studies have established normative data for the ROCF with Spanish-speaking pediatric populations. Although normative data exists for Colombian (Ardila & Rosselli, 1994) and Mexican (Galindo & Cortes, 2003) pediatric populations, no comprehensive normative data exists for pediatric populations in many other Spanish-speaking countries. The present study sought to fill this gap in the literature by providing normative data for a population of children and adolescents from Latin American countries and Spain based on multiple linear regression analyses.
Method
Participants
The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) between 6 and 17 years of age, b) born and currently lived in a country where the study was conducted, c) Spanish as primary language, d) an IQ≥80 on the Test Of Non-verbal Intelligence (TONI-2; Brown, Sherbenou, & Johnsen, 2009), and e) a scored <19 on the Children’s Depression Inventory (CDI; Kovacs, 1992).
Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s) were excluded due to its effects on cognitive performance. Participants in the study were from public or private schools, and they signed an informed consent to participate. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.
Instrument administration
A trained examiner administered the ROCF Figure A (copy), and after 3 minutes, the immediate recall was given. To score the ROCF figure, the Spanish-language ROCF manual was used (Rey, 2009). The ROCF includes 18 elements, and the maximum score for each of the two tasks (copy and immediate recall) is 36. In terms of scoring, two points are given when the element is correctly reproduced; one point is given when the reproduction is either (a) distorted, (b) incomplete but placed properly, or (c) complete but placed poorly; and 0.5 point is credited when the element is distorted or incomplete and placed poorly. A score of 0 is given when the element is absent or is not recognizable (Osterrieth, 1944).
Statistical analyses
Detailed statistical analyses used to generate the normative data for the ROCF copy and immediate recall (3 minutes) scores are described in Rivera and Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure. 1) First, the ROCF copy and immediate recall test scores were computed separately by means of the final multiple regression models. The full regression models included the following as predictors: age, age2, sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted
For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values areconsidered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).
Results
ROCF copy
The final multivariate linear regression models for the ten country-specific ROCF copy scores were significant (see Table 1). In all countries, the ROCF copy scores increased linearly as a function of age. The ROCF copy scores for all countries except for Puerto Rico were affected by a quadratic age effect. Children from Chile, Puerto Rico, and Spain whose parent(s) had a MLPE >12 years obtained higher ROCF copy scores than children whose parent(s) had an MLPE≤12 years. The child’s sex affected their ROCF copy scores in Chile and Puerto Rico, such that girls scored higher than boys. The amount of variance these predictors explained in the ROCF copy scores ranged from 25.5% (in Guatemala) to 63.6% (in Cuba).
Final multiple linear regression models for ROCF copy
Final multiple linear regression models for ROCF copy
Note. MLPE: Mean level of parental education.
The final multivariate linear regression models for the ten country-specific ROCF immediate recall scores were significant (see Table 2). In all countries, the ROCF immediate recall score increased linearly as a function of age. The ROCF immediate recall scores were affected by a quadratic age effect for all countries except, Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Children from Paraguay and Spain whose parent(s) had a MLPE >12 years obtained higher ROCF immediate recall scores than children whose parent(s) had a MLPE≤12 years. The child’s sex affected ROCF immediate recall score for Chile and Puerto Rico, such that girls scored higher than boys. The amount of variance these predictors explained in the ROCF immediate recall scores ranged from 24.8% (in Ecuador) to 53.4% (in Cuba).
Final multiple linear regression models for ROCF immediate recall (3 minutes)
Final multiple linear regression models for ROCF immediate recall (3 minutes)
Note. MLPE: Mean level of parental education.
The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF≤1.097; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.080 in a F (2,201) distribution which correspond to percentile 6).
Norms (e.g., a percentile score) for the different ROCF copy or immediate recall scores by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate the understanding of the procedure used to obtain the percentile associated with a score on this test. To find the percentile score for a 12-year-old Chilean boy who scored a 21 on the ROCF immediate recall (3 minutes) test and whose parent(s) have a mean of 14 years of education (MLPE). The steps to obtain the percentile for this score are: 1) Find Chile in Table 2, which provides the final regression models by country for the ROCF immediate recall score. Use the B weights to create an equation that will allow you to obtain the predicted ROCF immediate recall score for this child using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Chilean sample, which is equal to 11.5 years), and sex, which was coded as male = 1 and female = 0. Age2 and MLPE were not significant predictors, and therefore are not included in this model. See Rivera and Arango-Lasprilla (2017) to figure out the mean age of each country’s sample. Finally, the result is added to the constant generated by the model in order to calculate the predicted value.
In the case of the Chilean boy, the predicted ROCF immediate recall score would be calculated using the following equation:
User-friendly normative data
The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the ROCF copy or immediate recall test. However, this method can be prone to human error due to the number of hand calculations necessary. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 21 on the ROCF immediate recall test for a Chilean boy who is 12 years old whose parent(s) have 14 years of education) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and test (copy vs. immediate recall). In this case, the table for the ROCF immediate recall score for boys from Chile can be found in Table A13. (2) Find the appropriate age of the child, in this case, 12 years old. (3) Next, look in the 12 years’ age column to find the approximate location of the raw score obtained on the test. Within the 12 years’ column, the score of 21 obtained by this Chilean boy corresponds to an approximate percentile of 90.
The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.
Discussion
The ROCF is one of the most widely used neuropsychological tests in the world to evaluate visuospatial, visual-motor, and visual memory processes in both children and adults (Frisk et al., 2005). In Latin American countries and in Spain, this test is among the 10 most utilized neuropsychological tests by clinical neuropsychologists during their professional practice (Arango-Lasprilla, Stevens, Morlett-Paredes, Ardila, & Rivera, 2016; Olabarrieta-Landa et al., 2016). However, in spite of its great use, there are currently very few studies on validation and standardization of this test for Spanish-speakers. The vast majority of the studies have been done with adult populations, and to date very few studies have been conducted with pediatric populations. Therefore, neuropsychologists who use this test in Latin American countries or in Spain with pediatric populations usually perform their interpretation using norms from other countries (Arango-Lasprilla et al., 2016). Thus, there was an overwhelming need for normative data of the ROCF for pediatric populations in both Latin American countries and Spain (Arango-Lasprilla et al., 2016; Olabarrieta-Landaet al., 2016).
To fill this gap in the literature, the purpose of this study was to obtain normative data for the ROCF copy and immediate recall (3 minutes) scores for children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, México, Paraguay, Peru, and Puerto Rico) and Spain. The results of the study showed that there are different types of variables associated with the performance of the test such as age, quadratic age, sex, and MLPE. In general, it was found that the final regression models explained between 25.5% and 63.6% of the variance for the ROCF copy, and between 24.8% and 53.4% of the variance for the ROCF immediate recall.
Age was significantly related to the total score of both the ROCF copy and the ROCF immediate recall, so that the scores increase linearly as children become older. These results are similar to those reported in other studies in which ROCF scores were found to increase significantly with age (Frisk et al., 2005; Meyers & Meyers, 1996). Additionally, a curvilinear effect of age on the ROCF copy was shown in all countries, except Puerto Rico. Scores increased prominently between ages 6 and 13, while that increase slowly decreased after the ages 13–14 approximately, to later stabilize and resemble that of adults. Other studies, however, have found that the increase in scores begins to occur mainly between the ages of 12 and 16, being at age 17 when performance is matched to that of adults (Meyers & Meyers, 1996).
A curvilinear effect was also observed for the ROCF immediate recall for all countries, except for Chile, Guatemala, Honduras, Paraguay, and Puerto Rico. Score increased prominently between the ages 6 and 15, followed by a smaller increase between the ages 16 and 17 years of age. Past research has found that for the ROCF immediate and delay recall, there is a large increase in scores between the ages 6 to 12 followed by smaller increases between the ages of 12 and 17 (Anderson & Lajoie, 1996; Boone, Lesser, Hill-Gutierrez, Berman, & D’elia, 1993; Caffarra, Vezzadini, Dieci, Zonato, & Venneri, 2002; Chervinsky, Mitrushina, & Satz, 1992; Chiulli, Haaland, Larue, & Garry, 1995; Denman, 1984; Hartman & Potter, 1998; Kramer & Wells, 2004; Meyers & Meyers, 1995, 1996; Miatton, Wolters, Lannoo, & Vingerhoets, 2004; Pontón et al., 1996; Van Gorp, Satz, & Mitrushina, 1990).
On the other hand, the results of the present study contradict those found by Beltrán Dulcey and Solís-Uribe (2012). In their study, the ROCF copy and the ROCF immediate recall scores were not associated with age. One possible explanation might be the small sample size (141 children) used. Another potential explanation is that Beltrán Dulcey and Solís-Uribe (2012) divided the sample into two age ranges (9 to 12 years and 13 to 16) to compare the performance of these two groups using an independent measures t-test. In the present study, the variable age was analyzed as a continuous variable, and the sample sizes were symmetrical for each age. Subsequently, the effect of age on each test score was calculated by multiple linear regressions.
Sex was not associated with the test performance neither for the ROCF copy nor for the ROCF immediate recall in the vast majority of countries. These results are similar to those reported in other studies in which sex was not found to be associated with the performance of this test (Beltrán Dulcey & Solís-Uribe, 2012; Poulton, & Moffitt, 1995). However, sex was associated with copy and immediate recall scores in Chile and Puerto Rico, such that Chilean and Puerto Rican girls scored better than boys.
Despite the little attention paid to parents’ education levels in the standardization of neuropsychological tests for pediatric populations, research supports parents’ level of education as influential in children’s development of the cognitive functions (Schady, 2011). In this study, parents’ MLPE was associated with ROCF copy scores in Chile, Puerto Rico and Spain and with ROCF immediate recall scores in Paraguay and Spain. In both cases, children whose parent(s) had more than 12 years of education scored significantly higher than children whose parent(s) had an education below 12 years.
The results of the present study have notable clinical implications. Established norms for the ROCF for ten Spanish-speaking countries provide an excellent opportunity for clinical neuropsychologists to use this test as part of their neuropsychological evaluation protocols with the objective to evaluate alterations in visuospatial, visual-motor, and visual memory processes in pediatric populations between 6 and 17 years of age. Using these norms, the performance of each child can be assessed in a more exact and standardized manner according to their age, gender, and the education of their parent(s). The creation of standardized norms will improve neuropsychological evaluation and diagnosis both in normal and clinical populations, since deficits are usually present in children with learning disabilities (Kirkwood, Weiler, Bernstein, Forbes, & Waber, 2001), brain injury (Verger et al., 2000), autism (Czermainski, Riesgo, Guimarães, Salles, & Bosa, 2014), and attention and hyperactivity disorder (Rizzutti et al., 2008), among others. Furthermore, norms to value the neuropsychological performance of children with impairments will inform their prognostic course and facilitate the implementation of cognitive rehabilitation programs among populations who may benefit.
Limitations
Despite the strengths of the present study in filling a gap in the literature and providing the largest sample to validate and standardize the ROCF with Spanish-speaking populations, the results of the study should be interpreted in light of several limitations. This study presents normative data of the ROCF for nine countries in Latin America and Spain. For this reason, it is not advisable to use these norms to assess pediatric populations in Spanish-speaking countries not included in the present study. Future research should be conducted to standardize the ROCF in other Spanish-speaking countries.
Although the norms generated by the present study could be used by neuropsychologists in other countries to evaluate Spanish speaking immigrant children from any of the nine Latin American countries or Spain included in our sample, we caution this use due to the potential influence of other variables not assessed in the present study, such as level of acculturation, bilingualism, and number of years living in the country. In addition, the quality of education of both the child and the parent(s) is another aspect that may influence the cognitive performance of children.
The ROCF is one of the most used instruments in the world to measure visuospatial, visual-motor, and visual memory problems. However, it is essential to consider that no clinical diagnosis should be made based solely on the scores of this test. Scores should be integrated and interpreted as one part of a much larger battery that evaluates these processes in greater detail. Because there are a limited number of tests and norms in Latin America and Spain to evaluate these processes, standardization efforts should be made in future research for other similar assessments.
Although the size of the sample was adequate in each of the countries where the study was conducted, it is important to note that only the samples in Chile, Mexico, Paraguay, Puerto Rico, and Spain were obtained from several regions of the countries, whereas in the remaining countries the samples were collected from one geographical area. Future studies should expand data collection to other geographical areas of these countries to improve representativeness and generalizability.
Spanish was the first language for the children who participated in the present study. Despite Spanish being the first language for the majority of the population in Latin America and Spain, it is important to consider the cultural and linguistic richness of these countries. For example, the first language of many children may be completely different from Spanish (e.g., Portuguese, Euskera, Catalan, Guaraní, Maya, Quechua). For this reason, caution should be used when using these norms in children whose first language is not Spanish.
Finally, participants in the present study all represent a normal, healthy population. Future studies should be performed with clinical populations to establish the sensitivity and specificity of this test.
Conclusions
The ROCF is one of the most widely used neuropsychological tests for evaluating visuospatial, visual-motor, and visual memory processes in children and adolescents in Latin America and Spain. Present study findings established standardized norms for ten countries, and results indicated that age, sex, and parental education level influence ROCF scores. These variables ought to be taken into account when interpreting children’s scores. These norms provide neuropsychologists in these countries a valid assessment tool that can be used in everyday practice with pediatric populations.
Conflict of interest
None to report.
