Validity of The Evaluation System of Children and Adolescents to Measure Executive Functions: Convergent Validity

Abstract

Executive functions (EFs) refer to a set of cognitive skills used to achieve goals and adapt to novelty and social situations. These cognitive skills strongly impact people’s daily lives. Hence, assessing these skills is important for an early intervention to address their lack if need be. Nevertheless, relevant scientific literature reports difficulties in assessing EFs, as they are a set of processes that involve many dimensions. Further, there are differences between everyday situations wherein EFs are employed and the context of a typical performance-based task, involving the use of EFs. Questionnaires are an excellent way to address these difficulties. The objective of this study is to test SENA’s convergent validity in terms of EFs’ assessment. The SENA is the first Spanish standardized questionnaire based on DSM-5 (2013), it assesses emotional and behavioral problems in children aged 3–18 years. This experiment involved students from regular schools (N = 140; M age = 6.97 SD = 1.02). Statistical analysis was conducted that compared a validated EFs test (BRIEF-2) and the SENA test. The results of this study suggested the presence of convergent validity between SENA and BRIEF-2 vis-a-vis EFs. Thus, SENA is a valid assessment for EFs that are called for in school and family environments for those who speak Spanish. That said SENA is more comprehensive than BRIEF-2, as it not only assesses EFs but also identifies the problems currently associated with the failure in employing EFs, psychological vulnerabilities, and personal resources that are lacking. SENA provides a complete picture of children’s and adolescents' behaviors and emotions.

Keywords

executive functions convergent validity SENA

Validity of SENA to Measure Executive Functions: Convergent Validity

In recent years, there has been a great deal of interest in assessing executive functions (EFs).

EFs refer to a set of cognitive skills used to achieve goals and adapt to novelty and social situations (Diamond, 2013; Cristofori et al., 2019). EFs have been related to academic performance (Andersson, 2008; Zelazo et al., 2016), intelligence (Chen et al., 2019), behavior regulation, social adjustment, and even success in a professional field (Bryck & y Fisher, 2012). In some disorders, such as Attention-Deficit/Hyperactivity Disorder (ADHD), conduct behavior disorder, and autism, EFs seem to be particularly affected (Barkley, 1997; Ciesielski & y Harris, 1997; Graziano & Hart, 2016; Harvey, 2019). In fact, “the remarkable increase of interest in early EF has been driven by the study of clinical populations rather than normally developing children” (Hughes & Graham, 2002, p. 2). For all these reasons, assessing EFs accurately is important for detecting possible handicaps and for ensuring early intervention where necessary. It is also essential to consider EFs' development to establish how to assess EFs, considering developmental criteria. There are critical periods reflected in EFs’ development. Certain essential aspects of EFs such as attentional control, emerge in early childhood. Some of the complex aspects such as cognitive flexibility, goal setting, and information processing find relative maturation around 12 years of age (Diamond, 2013).

Additionally, other complex components such as planning, verbal fluency, or sequencing do not reach maturation until adulthood (Zelazo et al., 2016). Thus, assessing EFs in childhood, adolescence, and adulthood seems appropriate, and it is necessary to adapt these assessments to the different stages of EFs’ development.

The problems and difficulties in assessing EFs in children have been noted in the scientific literature.

Hughes and Graham (2002) remark that these difficulties are due to different factors. First, EFs mature in adolescence; as a result, there are different outcomes of prefrontal lesions during childhood compared to an adult’s lesions, as the child’s brain is still under development. Second, language skills contribute to EF’s performance (Manfra et al., 2014), and these skills are rather limited in children due to developmental factors. Historically, performance-based tasks have been given disproportionate importance, often leading to discrepancies between the results of EFs’ assessments and children’s daily life behaviors (Hughes & Graham, 2002). There is an inherent difference between everyday situations which require EFs and the context of a typical performance-based task (Fish & Wilson, 2021).

Moreover, many of these assessments were adaptations of tests meant for adults, because of which they lost the EFs’ developmental perspective. It is for these reasons that EFs' assessment in children must be as complete as possible, drawing on different sources of information (Gioia et al., 2017). There are other general difficulties too in EFs’ assessment, which affect both children and adults. First, a novel task’s performance gradually becomes an automatic response. So, the standardized tests for EFs may be less reliable. Secondly, “cognitive converge” means that almost every EF process involves different cognitive skills, making it difficult to isolate a particular EF (Harvey, 2019). For this reason, traditional assessments to measure EFs may be biased.

Last but not the least, EFs manifest in a wide variety of situations, resulting in low correspondence between their processes and natural behavior. Relevant questionnaires provide parents and teachers the opportunity to observe child behavior in multiple contexts and for durations that exceed the duration of performance-based tasks (Zelazo et al., 2016).

The Evaluation System of Children and Adolescents (SENA) (Fernández-Pinto et al., 2015) is a Spanish standardized questionnaire based on the standard classification of mental disorders (APA, 2013). It is the first Spanish questionnaire to assess a broad and complete spectrum of behaviors: emotional and behavioral problems (e.g., depression, anxiety, and learning disabilities), problems in the personal context (e.g., problems concerning parents or the school), vulnerability (e.g., isolation), and psychological resources (e.g., self-esteem and social competence). SENA includes three questionnaires (teacher’s questionnaire, parent’s questionnaire, and self-report) with different versions meant for those aged between 3 and 18 years: 3–6 years (early childhood education), 6–12 years (primary school), and 12–18 years (secondary school to post-secondary school).

Previous research shows convergent validity of SENA, showing a significant correlation with the Behavior Assessment System for Children (BASC) (Reynolds et al., 2004a), concretely positive correlation (moderate to very high) between attentional problems indexes of SENA and BASC, as well as between hyperactivity indexes of SENA and BASC from parent’s and teacher’s questionnaires (N ≥ 29) (Fernández-Pinto et al., 2015). Regarding EFs’ tests, SENA’s convergent validity has been tested in a pilot study with The Behavior Rating Inventory of Executive Function-children version (BRIEF-P; Gioia, Espy, et al., 2015) and The Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al., 2000). The results of the analyses of BRIEF-P and BRIEF showed their moderate to highly significant correlations with SENA EF’s variables, concretely between the indexes of attentional problems and hyperactivity-impulsivity from SENA and BRIEF P and BRIEF, inhibition from BRIEF P and BRIEF, and hyperactivity-impulsivity from SENA, rigidity from SENA and flexibility from BRIEF P and BRIEF, and EFs index from both tests (Fernández-Pinto et al., 2015). Nevertheless, due to the small sample size used (N ≥ 14), its replication was an issue

SENA and BRIEF-2 convergent validity study with a higher number of participants seems desirable.

It is vital to test the validity of new tests. Validity refers to the empirical data that supports a score’s interpretation in the sense in which it is determined. (A test reflects what it is supposed to measure.) By examining the relationship between a test that evaluates a similar construct, convergent validity evidence can be provided (Fernández-Pinto et al., 2015).

In the present study, the focus has been on EFs’ assessment in children while considering relevant cultural and ecological aspects. It is necessary to use different assessment sources (not only performance-based tasks) to avoid inaccuracy in EFs’ assessments (Carrasco et al., 2013). Questionnaires offer a balanced solution that facilitates information gathering about processes and children’s behaviors in their daily lives (Chaytor & Schmitter-Edgecombe, 2003; Zelazo et al., 2016). They can also help in finding a way around children’s language limitations and the difficulties of EFs’ traditional tasks that children may encounter.

This study aims to test the convergent validity between SENA (version 6–12 years old) and BRIEF-2 concerning EFs. The variables of SENA that were selected related to EFs as determined by SENA authors themselves. It was hypothesized that SENA is a valid test to measure EFs in daily life as far as the Spanish population is concerned. This study intended to provide new validity SENA.

Method

Ethics Statement

In adherence to the requirements of the Declaration of Helsinki, the parents of the participants signed a written informed parental consent, and the teachers also signed informed consent. All of the participants verbally agreed to take part in this research. The Ethics Committee approved this study of the San Carlos Hospital (n° 15/315-E) in June 2015.

Participants

The participants were recruited by the schools that agreed to participate in the study. Various schools in Madrid (Spain) were contacted via email to participate in the study. Finally, two of them agreed to participate. Once the schools agreed to participate, the teachers and families concerned were invited to participate in the study through posters, Email, and information meetings. In all 140 students consented to participate (N = 140; M age = 6.97 years SD = 1.02 years). Of these, 65 were girls (46.4%), and 75 were boys (53.6%). All of the participants were Spanish speakers. Both the schools were located in the same district to avoid differences in terms of sociocultural levels. The parents’ professional range displayed an Mdn. of 3.00 (IQR = 2.00) (0 = low, 1 = medium-low, 2 = medium, 3 = medium-high, and 4 = high) according to “National Institute of Professional Range” (Spain). No economic benefits were offered for participation in the study.

The children’s parents (parents’ questionnaires, N = 119) and their teachers (teachers’ questionnaires, N = 9) participated in this study. The teachers filled out the questionnaires for the child participants of their respective classes.

Design Overview

Data was obtained from teachers and parents (through their responses to the questionnaires), from October to April 2016. The parents and teachers received a packet of questionnaires for each participant, consisting of two questionnaires (40 minutes each), which they were to completed at home. They had 2 weeks to fill out the questionnaires. The researcher collected all of the completed questionnaires at school in closed envelopes.

Instruments

Questionnaires

SENA

SENA (Fernández-Pinto et al., 2015) is a standardized questionnaire based on DSM-5 (APA, 2013), with three sub-questionnaires (i.e., one for teachers, one for parents, and one facilitating self-reporting). It allows us to assess emotional or behavioral problems in children aged 3–18 years. It is a Likert-style questionnaire. Higher scores indicate problems in those constructs (T, typical scale M = 50, SD = 10), reliability is based on Cronbach’s alpha ≥ .70 for all of the scales.

The Comparative Fit Index (CFI) of the three-factor analysis ranged from .84 to .98. For this study, we selected those scales related to EFs that were prescribed by SENA authors, such as SENA EFs problems, SENA attentional problems, SENA hyperactivity-impulsivity, SENA emotional regulation problems, and SENA rigidity. Apart from those, SENA also measures other emotional and behavioral problems (e.g., depression, anxiety, and learning disabilities), problems in the personal context (e.g. those related to parents and school), vulnerability (e.g., isolation), and psychological resources (e.g., self-esteem and social competence). The norms developed for SENA pertain to a Spanish-speaking sample. SENA is a Spanish test that considers cultural diversity and acknowledges that EFs’ development is sensitive to cultural values and factors.

BRIEF-2

BRIEF-2 (Gioia, Isquith, et al., 2015) is a standardized test for 5–18 year-olds, focusing on the assessment of EFs with two questionnaires (one for teachers and one for parents). BRIEF-2 is an updated version of BRIEF (Gioia et al., 2000), the first scale that was used to measure self-regulation and EFs in children and adolescents. There have been more than 800 scientific publications that have used this test, constructing extensive knowledge about BRIEF’s reliability and validity in the clinical population and typically covering an individual’s lifespan (Gioia et al., 2017). BRIEF-2 is a Likert-style questionnaire where parents and/or teachers have to respond to 63 items in terms of the frequency of certain behaviors (never, sometimes, and frequently).

Three main indexes combine the different clinical scales: BRIEF behavior regulation index, BRIEF emotional regulation index, and BRIEF cognitive regulation index. The Global Executive Function composite summarizes these three scales. This test provides different index scores concerning EF, such as inhibition, flexibility, supervision, self-monitor, and working memory. Higher scores indicate that problems exist in these constructs (T, typical scale M = 50, SD = 10). We used the Spanish adaptation (Gioia et al., 2017). This test has fairly high internal consistency-related reliability (based on Cronbach’s alpha M = .86). The CFI of the three-factor analysis values ranged, in this case, from .95 to .99.

An Overview Of The Data Analytics Approach

This study adopts an ex post facto research design. The Spearman correlation test was conducted between both the questionnaires’ versions separately (teacher’s questionnaire and parent’s questionnaire) to test the convergent validity between SENA’s variables related to EFs (according to the authors) and BRIEF-2. T scores were used in the analysis to control age. Examining the relations between tests that evaluate similar constructs is a way of testing convergent validity (Fernández-Pinto et al., 2015). Pearson correlations ranged from −1 to 1. Negative correlations (< 0) indicated an inverse relationship between variables. In contrast, if the correlation is > 0, there is a positive relationship (i.e., the variables are directly related). The correlation coefficients were interpreted as follows: r ≤ 0.1 (very small, negligible), r = 0.1–0.3 (small, low), r = 0.3–0.5 (moderate), r = 0.5–0.7 (high), r = 0.7–0.9 (very high), and r ≥ 0.9 (almost perfect) (Hopkins, 2000).

Results

Descriptive Statistics

Table 1 presents each variable’s descriptive statistics (means and standard deviation) in the different questionnaires (teacher’s and parent’s versions). As seen in this table, all of the measures correspond to non-clinical criteria.

Table 1.

Descriptive statistics of participant’s scores. Parent’s version Teacher’s version.

	Mean	SD	Mean	SD
BRIEF inhibit	50.50	8.48	49.36	10.61
BRIEF self-monitor	49.89	9.16	50.93	11.73
BRIEF flexibility	50.00	10.58	50.71	12.07
BRIEF emotional control	47.73	9.34	48.34	9.87
BRIEF initiate	50.38	9.72	51.96	12.77
BRIEF working memory	51.23	10.53	51.04	12.44
BRIEF plan/organize	49.53	10.25	50.90	11.47
BRIEF task-monitor	49.21	9.30	49.25	11.07
BRIEF organization of materials	52.33	9.21	50.64	11.43
BRIEF behavior regulation index	50.30	8.44	49.93	11.44
BRIEF emotional regulation index	48.51	9.70	49.44	10.88
BRIEF cognitive regulation index	50.60	10.01	50.83	12.63
BRIEF global EF composite	49.92	9.32	50.44	12.46
SENA EF problems	49.34	9.33	49.16	11.39
SENA attentional problems	50.19	8.58	50.07	12.32
SENA hyperactivity-impulsivity	50.47	9.50	49.44	11.70
SENA emotional regulation problems	49.24	10.15	45.73	9.75

SD=standard deviation. SENA: system of evaluation for children and teenagers; BRIEF: behavior rating inventory of executive functions; N=119 for parent’s version. N=140 for teacher’s version. M age=7.00 SD=1.04.

Convergent Validity

Table 2 and Table 3 report the correlations between SENA’s variables of EFs and BRIEF-2. (Table 2 pertains to the parent’s version and Table 3 to the teacher’s version).

Table 2.

Spearman correlations between BRIEF-2 parents’ version and SENA EF domains of parents’ version.

	SENA EF problems	SENA Attention problems	SENA Hyperactivity- impulsivity	SENA Emotional Regulation problems	SENA rigidity
BRIEF inhibit	.57**	.44**	.62**	.50**	.08
BRIEF self-monitor	.46**	.43**	.29**	.42**	.19*
BRIEF flexibility	.60**	.33**	.13	.42**	.7**
BRIEF emotional control	.62**	.32**	.30**	.73**	.37**
BRIEF initiate	.38**	.52**	-.11	.20*	.44**
BRIEF working memory	.51**	.74**	.20*	.21*	.33**
BRIEF plan/organize	.51**	.64**	.15	.31**	.36**
BRIEF task-monitor	.34**	.57**	.13	.12	.17
BRIEF organization of materials	.25**	.41**	.17	.05	.01
BRIEF behavior regulation index	.59**	.49**	.56**	.53**	.14
BRIEF emotional regulation index	.70**	.37**	.26**	.68**	.61**
BRIEF cognitive regulation index	.51**	.72**	.15	.23*	.35**
BRIEF global EF composite	.69**	.70**	.31**	.49**	.46**

N=119.Note. ** p < .01. SENA: system of evaluation for children and teenagers BRIEF: Behavior rating inventory of executive functions.

Table 3.

Spearman correlations between BRIEF-2 teacher version and SENA EF domains of teacher version.

	SENA EF problems	SENA Attention problems	SENA Hyperactivity- impulsivity	SENA Emotional Regulation problems	SENA rigidity
BRIEF inhibit	.78**	.68**	.90**	.70**	.27**
BRIEF self-monitor	.77**	.67**	.82**	.67**	.37**
BRIEF flexibility	.62**	.51**	.35**	.47**	.73**
BRIEF emotional control	.68**	.46**	.67**	.79**	.35**
BRIEF initiate	.59**	.70**	.37**	.27**	.56**
BRIEF working memory	.69**	.83**	.53**	.34**	.51**
BRIEF plan/organize	.71**	.77**	.53**	.42**	.58**
BRIEF task-monitor	.76**	.86**	.63**	.46**	.51**
BRIEF organization of materials	.69**	.77**	.60**	.38**	.46**
BRIEF behavior regulation index	.81**	.70**	.90**	.71**	.32**
BRIEF emotional regulation index	.75**	.56**	.59**	.73**	.64**
BRIEF cognitive regulation index	.74**	.85**	.57**	.41**	.57**
BRIEF global EF composite	.86**	.86**	.74**	.62**	.60**

N=140. Note. ** p < .01. SENA: system of evaluation for children and teenagers BRIEF: Behavior rating inventory of executive functions.

Correlations ranging from low to high were found between BRIEF-2 and SENA EF variables of the family version. High correlations were found between BRIEF-2 and SENA in the following indexes: BRIEF inhibit and SENA EF problems (r = .57, p < .01), SENA hyperactivity-impulsivity (r = .62, p < .01), and SENA emotional regulation problems (r = .50, p < .01). BRIEF flexibility and SENA EF problems (r = .60, p < .01). BRIEF emotional control and SENA EF problems (r = .62, p < .01). BRIEF initiative and SENA attention problems (r = .52, p < .01). BRIEF working memory and SENA EF problems (r = .51, p < .01). BRIEF plan/organize and SENA EF problems (r = .51, p < .01), and SENA attention problems (r = .64, p < .01). BRIEF task-monitor and SENA attention problems (r = .57, p < .01). BRIEF behavior regulation index and SENA EF problems (r = .59, p < .01), SENA hyperactivity-impulsivity (r = .56, p < .01) and SENA regulation problems (r = .53, p < .01). BRIEF emotional regulation index and SENA emotional regulation (r = .68, p < .01) and SENA rigidity (r = .61, p < .01). BRIEF cognitive regulation index and SENA EF problems (r = .51, p < .01). BRIEF global EFs composite and SENA EF problems (r = .69, p < .01).

Very high correlations were found between BRIEF-2 and SENA EF variables in the following indexes: BRIEF flexibility and SENA rigidity (r = .7, p < .01). BRIEF emotional control and SENA emotional regulation problems (r = .73, p < .01). BRIEF working memory and SENA attention problems (r = .74, p < .01). BRIEF emotional regulation index and SENA EF problems (r = .70, p < .01). BRIEF cognitive regulation index and SENA attention problems (r = .72, p < .01). BRIEF global EF composite and SENA attention problems (r = .70, p < .01).

Small to very high correlations were found between BRIEF-2 and SENA EF variables of the teacher version. All of the correlations were statistically significant. High correlations were found between BRIEF-2 and SENA in the following indexes: BRIEF inhibit and SENA attention problems (r = .68, p < .01). BRIEF self-monitor and SENA attention problems (r = .67, p < .01) and SENA emotional regulation problems (r = .67, p < .01). BRIEF flexibility and SENA EF problems (r = .62, p < .01) and SENA attention problems (r = .51, p < .01). BRIEF emotional control and SENA EF problems (r = .68, p < .01) and SENA hyperactivity-impulsivity (r = .67, p < .01). BRIEF initiate and SENA EF problems (r = .59, p < .01) and SENA rigidity (r = .56, p < .01). BRIEF working memory and SENA EF problems (r = .69, p < .01), SENA hyperactivity-impulsivity (r = .53, p < .01), and SENA rigidity (r = .51, p < .01). BRIEF plan/organize and SENA hyperactivity-impulsivity (r = .53, p < .01) and SENA rigidity (r = .58, p < .01). BRIEF task-monitor and SENA hyperactivity-impulsivity (r = .63, p < .01). BRIEF organization of materials and SENA EF problems (r = .69, p < .01) and SENA hyperactivity-impulsivity (r = .60, p < .01). BRIEF emotional regulation index and SENA attention problems (r = .56, p < .01), SENA hyperactivity-impulsivity (r = .59, p < .01), and SENA rigidity (r = .64, p < .01). BRIEF cognitive regulation index and SENA hyperactivity-impulsivity (r = .57, p < .01) and SENA rigidity (r = .57, p < .01). BRIEF global EF composite and SENA emotional regulation problems (r = .62, p < .01) and SENA rigidity (r = .62, p < .01).

Very high correlations were found between BRIEF-2 and SENA in the following indexes: BRIEF inhibit and SENA EF problems (r = .78, p < .01), SENA hyperactivity-impulsivity (r = .90, p < .01) and SENA emotional regulation problems (r = .70, p < .01). BRIEF self-monitor and SENA EF problems (r =.77, p < .01) and SENA hyperactivity-impulsivity (r = .82, p < .01). BRIEF flexibility and SENA rigidity (r = .73, p < .01). BRIEF emotional control and SENA emotional regulation problems (r = .79, p < .01). BRIEF initiate and SENA attention problems (r = .70, p < .01). BRIEF working memory and SENA attention problems (r = .83, p < .01). BRIEF plan/organize and SENA EF problems (r = .71, p < .01) and SENA attention problems (r = .77, p < .01). BRIEF task-monitor and SENA EF problems (r = .76, p < .01) and SENA attention problems (r = .86, p < .01). BRIEF organization of materials and SENA attention problems (r = .77, p < .01). Brief behavior regulation index and SENA EF problems (r = .81, p < .01), SENA attention problems (r = .70, p < .01), SENA hyperactivity-impulsivity (r = .90, p < .01) and SENA emotional regulation problems (r = .71, p < .01). BRIEF emotional regulation index and SENA EF problems (r= .75, p < .01) and SENA emotional regulation problems (r = .73, p < .01). BRIEF cognitive regulation index and SENA EF problems (r = .74, p < .01) and SENA attention problems (r = .85, p < .01). BRIEF global EF composite and SENA EF problems (r = .86, p < .01), SENA attention problems (r = .86, p < .01) and SENA hyperactivity-impulsivity (r = .74, p < .01).

Discussion

This study provided data about the convergent validity between the SENA (variables of EF) and BRIEF-2. The results of the present study support the hypothesis about SENA’s validity to measure EFs. SENA’s validity to measure EFs is especially relevant, as the questionnaires concerned provide balanced solutions to overcome EFs' assessment-related difficulties, especially in childhood (Chaytor & Schmitter-Edgecombe, 2003; Hughes & Graham, 2002; Zelazo et al., 2016). It would be apposite to accept SENA’s validity for practical reasons. It must be considered that SENA is the first Spanish questionnaire that assesses a broad and complete spectrum of behaviors, psychological resources, and vulnerabilities. This is advantageous, as it gives the clinician or educational psychologist a general overview of the psychological status of the children or adolescents that is more rounded than the one offered by the BRIEF-2, which focuses only on EFs. Also, the norms of this assessment duly relate to Spanish speakers, which is relevant considering the cultural diversity of the target population, knowing that EFs' development is sensitive to cultural values and factors.

The key finding of the present research is the convergent validity test.

Previous studies have shown a significant correlation between SENA and other versions of BRIEF, such as BRIEF-P (Gioia, Espy, et al., 2015), which focuses on preschoolers and BRIEF (Gioia et al., 2000). Regarding the first analysis, although the sample was small (N = 19), moderately to highly significant correlations were found between the variables of EFs of the SENA and BRIEF-P.

These correlates tend to be higher in the teacher’s version. Considering the data from the second analysis (BRIEF), although the sample was small (N = 11), the results showed moderately to highly significant correlations between SENA and BRIEF’s EFs' variables (Fernández-Pinto et al., 2015). Considering the sample sizes of these studies, the results must be considered with caution. In this study, we have found a higher number of correlations in the parent’s version.

Both the analyses showed that the SENA Executive problems index correlates more than the different BRIEF-P and BRIEF indexes. These results also showed moderately to highly significant correlations between both the tests concerning EFs. We have found even more correlations between SENA EFs' variables and BRIEF-2. In the teacher’s versions, all of the correlations were found to be significant. In general terms, the previous analyses mentioned above showed a significant correlation between SENA EF problems and BRIEF-P global EF composite (r =.76, p < .05) for the parent’s version and (r = .92, p < .05) for the teacher’s version, and BRIEF global EF composite (r = .94, p < .05) for the parent’s version and (r = .43, p < .05) for the teacher’s version. In this study, SENA EF problems and BRIEF-2 global EF composite also correlated significantly (r = .69, p < .01) for the parent’s version and, (r = .86, p < .01) for the teacher’s version. This pattern of results is consistent with those of previous literature (Fernández-Pinto et al., 2015). At the same time, these data confirms the tendency found in previous research concerning showing convergent validity of SENA in terms of EFs' assessment in a robust sample (N = 140). This study also revealed moderately to highly significant correlations between EFs and attention, such as SENA attention problems and BRIEF initiate (parent’s and teacher’s versions), BRIEF plan/organize (parent’s and teacher’s versions), BRIEF working memory (parent’s and teacher’s versions), and BRIEF EF global Composite (parent’s and teacher’s versions). Also, in the teacher’s versions, between SENA attention problems and BRIEF inhibit, BRIEF self-monitor, BRIEF flexibility, BRIEF task-monitor, BRIEF organization of materials, and BRIEF cognitive regulation index.

The data generated by this study reinforces the idea of a close relation between EFs and attention that is claimed by scientific literature (Conway et al., 2003; Rueda et al., 2012). On the other hand, moderately to highly significant correlations between EFs and emotional/behavior regulation included SENA emotional regulation problems and BRIEF inhibit (parent’s and teacher’s versions), SENA EF problems, and BRIEF emotional control (parent’s and teacher’s versions). Also, in the teacher’s version between SENA emotional regulation problems and BRIEF global EF composite. This finding may be explained by the idea that EFs imply emotional and behavioral control (Stuss, 2011). Some situations in daily life require the employment of EFs, and emotions often interfere with these tasks (Zelazo et al., 2010). EFs are related to adaptation capability in a social context, inhibiting disruptive behaviors (Bryck & y Fisher, 2012).

Regarding EFs' assessment in childhood, we strongly recommend using different sources of assessment data. While questionnaires are helpful to assess EFs in daily life, it would be best to avoid traditional issues such as adapting EFs' performance-based tasks from adults to children who are still developing EFs (Hughes & Graham, 2002) or overlooking language limitations (Manfra et al., 2014), among others, for more often than not, these could cause discrepancies between assessments of performance-based tasks and children’s daily life behaviors.

Further research is needed to extend the current findings by examining SENA’s convergent validity vis-à-vis older participants (aged 9 years old to adolescence) and to Different sociocultural strata (low/high), as the sample of this study corresponded only to the medium-high sociocultural level. In other words, the age ranges covered in this study and the lack of other sociocultural strata are among its limitations. Despite these limitations, this is the first study that looks into SENA’s validity, using a robust number of participants. This study provides preliminary support to SENA as an appropriate test to measure EFs in childhood.

Conclusions

The results of this study indicate the presence of convergent validity between SENA and BRIEF-2 in terms of EFs. Thus, SENA is a valid assessment of EFs in school and family environments for those who speak Spanish.

SENA is more complete than BRIEF-2, as it also assesses problems currently associated with EFs’ failures, psychological vulnerabilities, and personal resources.

Footnotes

Acknowledgment

We thank I+D+i Ediciones Tea department for the material supply. We thank Enzade School and Teresa de Berganza School (Madrid, Spain) for their participation in this research. We thank Ada Brunstein for the English review of the manuscript.

Authors Contribution

TRP and EPH conceived and designed study;

TRP conducted data collection, data analysis and interpretation, and drafted the article.

EPH conceived and designed study, interpreted data and reviewed critically the article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Compliance with Ethical Standards

The Ethics Committee approved this study of the San Carlos Hospital (n° 15/315-E) in June 2015. and attests that the study was performed following the ethical standards set forth in the 1964 Declaration of Helsinki and its later amendments.

Informed Consent

The parent of the participants signed a written informed parental consent, and the teachers also signed informed consent.

ORCID iD

Teresa Rossignoli-Palomeque

References

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Andersson

(2008). Working memory as a predictor of written arithmetical skills in children: The importance of central executive functions. British Journal of Educational Psychology, 78(2), 181–203. https://doi.org/10.1348/000709907X209854

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65–94. https://doi.org/10.1037/0033-2909.121.1.65

Bryck

R. L.

y Fisher

P. A.

(2012). Training the brain: Practical applications of neural plasticity from the intersection of cognitive neuroscience, developmental psychology, and prevention science. The American Psychologist, 67(2), 87–100. https://doi.org/10.1037/a0024657

Carrasco

Ramirez

Del Barrio

(2013). Evaluación clínica. Diagnóstico, formulación Y contrastación de los trastornos psicológicos. Madrid: Editorial Sanz Y Torres.

Chaytor

Schmitter-Edgecombe

(2003). The ecological validityof neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13(4), 181–197. https://doi.org/10.1023/B:NERV.0000009483.91468.fb

Chen

Spagna

Kim

T. H.

Chen

Fan

(2019). Testing a cognitive control model of human intelligence. Scientific Reports, 9(1), 2898. https://doi.org/10.1038/s41598-019-39685-2

Ciesielski

y Harris

R. J.

(1997). Factors related to performance failure on executive tasks in autism. Child Neuropsychology, 3(1), 1–12. https://doi.org/10.1080/09297049708401364

Conway

A. R.

Kane

M. J.

Engle

R. W.

(2003). Working memory capacity and its relation to general intelligence. Trends Cogn. Sci, 7, 547–552. https://doi.org/10.1016/j.tics.2003.10.005

10.

Cristofori, I., Cohen-Zimerman, S., & Grafman, J. (2019). Executive functions. Handbook of clinical neurology, 163, 197–219. https://doi.org/10.1016/B978-0-12-804281-6.00011-2

11.

Diamond

(2013). Executive functions. Annual Review of Psychology, 64, 135–168. https://doi:10.1146/annurev-psych-113011-143750

12.

Fernández-Pinto

Santamaría

Sánchez-Sánchez

Carrasco

M. A.

Del Barrio

(2015). Sistema de Evaluación de Niños y adolescentes. SENA. Madrid: TEA Ediciones.

13.

Fish, J & Wilson, F.C (2021) Assessing Children's Executive Function: BADS-C Validity. Front. Psychol. 12:626291. https://doi.org/10.3389/fpsyg.2021.626291

14.

Gioia

G.A.

Espy

K.A.

Isquith

P.K

Basuela

Luque

(2015a). BRIEF-P. Evaluación Conductual de la Función ejecutiva-versión infantil. Madrid: TEA Ediciones.

15.

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

Kenworthy

(2015b). BRIEF-2: Behavior rating inventory of executive function. Lutz, FL: Psychological Assessment Resources.

16.

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

y Kenworthy

(2000). Behavior rating inventory of executive function: BRIEF. FL: Psychological Assessment Resources Odessa.

17.

Gioia

G. A.

Isquith

P. Q.

Guy

S. C.

Kenworthy

Roca

M. E. S.

Santamaría

(2017). BRIEF 2: Evaluación conductual de la función ejecutiva. TEA.

18.

Graziano

P. A.

Hart

(2016). Beyond behavior modification: Benefits of social-emotional/self-regulation training for preschoolers with behavior problems. Journal of School Psychology, 58, 91–111. https://doi.org/10.1016/j.jsp.2016.07.004

19.

Harvey

P. D.

(2019). Domains of cognition and their assessment. Dialogues in Clinical Neuroscience, 21(3), 227–237. https://doi:10.31887/DCNS.2019.21.3/pharvey

20.

Hopkins

W. G.

(2000). A new view of statistics. Internet Society for Sport Science http://www.sportsci.org/resource/stats/

21.

Hughes

Graham

(2002). Measuring executive functions in childhood: Problems and solutions? Child and Adolescent Mental Health, 7(3), 131–142. https://doi.org/10.1111/1475-3588.00024

22.

Manfra

Davis

K. D.

Ducenne

y Winsler

(2014). Preschoolers' motor and verbal self- control strategies during a resistance-to-temptation task. The Journal of Genetic Psychology, 175(4), 332–345. https://doi.org/10.1080/00221325.2014.917067

23.

Reynolds

C. R.

Kamphaus

R. W.

González

Fernández

Pérez

Santamaría

(2004a). BASC. Sistema de Evaluación de la Conducta de Niños y Adolescentes. Madrid: TEA Ediciones., adaptadores).

24.

Rueda

M. R.

Checa

y Cómbita

L. M.

(2012). Enhanced efficiency of the executive attention network after training in preschool children: Immediate changes and effects after two months. Developmental Cognitive Neuroscience, 2(Supplement 10), S192–S204. https://dx-doi-org.web.bisu.edu.cn/10.1016/j.dcn.2011.09.004

25.

Stuss

D. T.

(2011). Functions of the frontal lobes: Relation to executive functions. International Neuropsychological Society Journal, 17(5), 759–765. https://doi.org/10.1017/S1355617711000695

26.

Zelazo

P. D.

Blair

C. B.

Willoughby

M. T.

(2016). Executive function: Implications for educationNCER 2017-2000. National Center for Education Research.

27.

Zelazo

P. D.

y Kesek

A. C.

(2010). Hot executive function: Emotion and the development of cognitive control. In Calkins

S. D.

Bell

M. A.

(Eds), Human brain development. Child development at the intersection of emotion and cognition (pp. 97–111). American Psychological Association. https://doi.org/10.1037/12059-00