Abstract
Introduction
The purpose of the study was to examine the Weekly Calendar Performance Activity for students (WCPA-S), an executive function (EF) measure in everyday life, comparing between students with and without ADHD and across gender. This study aim is crucial to understand the difficulties encountered by the population of students with ADHD attending universities/colleges as it is steadily increasing (Beezhold, Johnston, & Nochta, 2014). Yet, the accumulated data concerning ADHD among college students are limited (Garnier-Dykstra, Pinchevsky, Caldeira, Vincent, & Arria, 2010). Based mainly on behavioral rating scales (O’Rourke, 2014), approximately 2% to 8% of all college students report having clinically significant levels of ADHD symptoms (DuPaul et al., 2001; DuPaul, Weyandt, O’Dell, & Varejao, 2009; McKee, 2008; Pryor, Hurtado, DeAngelo, Blake, & Tran, 2010).
In Israel, about 12% of the students attending higher education in 2012 were recognized as having cognitive/learning deficits, for example, learning disabilities (LD) and/or ADHD (Council for Higher Education in Israel, unpublished, 2012). Among the university students and applicants diagnosed with cognitive/learning deficits, 48% are diagnosed with ADHD, with or without co-existing LD (Ben-Simon, Inbar-Weiss, Barneron, & Polacheck, 2012). Studies suggest that students with ADHD are at an increased risk of academic problems, lower grade point average, and compromised academic coping skills compared with their non-ADHD peers (Frazier, Youngstrom, Glutting, & Watkins, 2007; Weyandt & DuPaul, 2006).
EF Deficits and Students With ADHD
EF abilities are crucial for daily function and participation (Katz & Hartman-Maeir, 2011; Weiner, Toglia, & Berg, 2012). Among higher educational students, EF processes and components are called upon in the performance of many academic tasks and play an important role in ensuring the quality of their academic performance (Preston, Heaton, McCann, Watson, & Selke, 2009). The International Classification of Functioning, Disability and Health (ICF), defines EFs as “higher-level cognitive functions . . . including complex goal-directed behaviors such as decision-making, abstract thinking, planning and carrying out plans, mental flexibility, and deciding which behaviors are appropriate under what circumstances” (World Health Organization, 2001, p. 57). In addition, executive functioning is needed to coordinate skills such as time management, organization, and planning that are vital for achieving academic success (Brown, 2009; Dahan, Hadas-Lidor, Meltzer, & Roitman, 2008; Ustun, 2007; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Specifically, the ICF describes time management as a mental function that includes ordering events in chronological sequence as well as allocating amounts of time to events and activities (2001). Langberg et al. (2011) described task planning behaviors as the ability to ascertain how to start work or projects and complete them on time, how to organize action behaviors using strategies such as the use of lists, schedules, calendars, and how to approach work systematically.
Barkley (1997) claims that ADHD is characterized by deficits in EF, expressed specifically in difficulties with time, self-organization and self-management (Barkley & Murphy, 2011), or planning (Willcutt et al., 2005). Adults with ADHD have problems in everyday situations which required multitasking (Gawrilow, Merkt, Goossens-Merkt, Bodenburg, & Wendt, 2011) and prospective memory (PM) abilities (Altgassen, Kretschmer, & Kliegel, 2014), both associated with EF (Gawrilow et al., 2011; Mahy, Moses, & Kliegel, 2014). Multitasking is the ability to plan, perform, and monitor while working on tasks or jobs concurrently (Burgess, 2000; Shallice & Burgess, 1991). PM refers to the ability to remember doing something in the future (Einstein & McDaniel, 1990; Talbot & Kerns, 2014), despite the presence of other ongoing activities that may interfere or capture attention (Mahy et al., 2014). Orientation to the future is vital for efficient academic performance (Mahy et al., 2014), as well as working on tasks concurrently. Studies have shown how EF deficits may negatively impact on academic functioning in adults with ADHD (Biederman et al., 2006). For example, Wolf (2001) claimed that the increased planning and organizational demands of college students, coupled with a more flexible daily schedule and the absence of a supervising parent, may severely tax the function of the student with ADHD in many daily activities such as preparing meals, managing finances, and/or performing errands (Kofman, Larson, & Mostofsky, 2008). Not surprisingly, regarding multitasking situation, it appears that adults with ADHD perform best in well-structured non-interleaving conditions (Gawrilow et al., 2011).
Students with ADHD have many academically related performance difficulties, they forget about assignments or recall them only at the last moment; have difficulty organizing study materials and tend to misplace or lose them; do not anticipate or plan for future events; and have problems initiating and following through on tasks, estimating the time needed to complete tasks, and distinguishing between relevant and non-relevant information (Abikoff et al., 2009; Langberg et al., 2011; Weyandt & DuPaul, 2006; Wolf, 2001). Furthermore, Dahan et al. (2008) describe how poor time organization affects the function and achievements of students with LD and ADHD by not getting to classes on time or not remembering where the class is located. Moreover, deficient time management often leads to procrastination and may hinder students from completing long-term assignments on time (Abikoff et al., 2009).
Gender Differences in Adults With ADHD
Retz-Junginger, Rosler, Jacob, Alm, and Retz (2010) claim that despite the growing interest in the study of ADHD in adults, most of the knowledge accumulated about ADHD is based on research performed among children and adolescents and primarily on male study samples (Dupuy, Clarke, & Barry, 2013; Valera et al., 2010). Concordantly, previous studies had found higher ADHD prevalence among boys (American Psychiatric Association, 2000; Bauermeister et al., 2007; Rucklidge, 2008), and suggest the possibility of under diagnosis in girls in childhood (Merkt & Gawrilow, 2014). However, in adulthood, equal prevalence rates of ADHD in men and women have been reported (Biederman et al., 1994; de Zwaan et al., 2012). In a review of gender differences among children with ADHD, Gershon (2002) found that girls with ADHD showed fewer primary symptoms and externalizing problems than boys, however fared worse than boys on related problems such as intellectual functioning.
Although many studies found gender differences in the severity of ADHD symptoms (Biederman, Faraone, Monuteaux, Bober, & Cadogen, 2004; Retz-Junginger et al., 2010; Rucklidge, 2008; Rucklidge, Brown, Crawford, & Kaplan, 2007), performance differences between men and women adults with ADHD have rarely been addressed (Fedele, Lefler, Hartung, & Canu, 2012). Research has started to focus on the characteristics of ADHD in adulthood only in recent years (Rucklidge et al., 2007). In a recent study, women with ADHD reported difficulties in organizing activities, problem-solving, and time management more often than men participants (Retz-Junginger et al., 2010). Other studies have shown greater impairment among women students with ADHD compared with their men counterparts (Fedele et al., 2012; Merkt & Gawrilow, 2014; O’Callaghan & Sharma, 2014). Fedele et al. (2012) found higher rates of inattention, hyperactivity, and impairment among women students with ADHD regarding social life, education, money management, and daily life activities. Merkt and Gawrilow (2014) found higher impairment regarding health-related quality of life and worse psychological functioning among women students. O’Callaghan and Sharma (2014) also found lower ADHD-related quality of life and psychological health in female students compared with male students with ADHD. Although those studies represent clear gender differences among college students with ADHD, their results are based on self-report measurements rather than using actual diagnostic performance tool. Retz-Junginger et al. (2010) stressed the need for further research to determine accurate diagnostic criteria and develop appropriate diagnostic tests.
Tools for Measuring EF
Educators and clinicians working with adult students with ADHD often rely on assessments that do not measure EFs in student-related performance measures. Barkley and Murphy (2011) indicate that scores on EF tests that do not have ecological validity are largely unrelated to EF performance ratings, which are strongly associated with impairments in major life activities. Therefore, ecological tools are preferred when evaluating EFs (Katz & Hartman-Maeir, 2011). Furthermore, ecological tools can be generalized to different daily life situations to reveal the individuals’ level of independence in his or her natural surroundings, enabling also the development of appropriate intervention programs (Burgess et al., 2006). Alongside the many benefits of ecological tools due to their higher external validity, Gawrilow et al. (2011) discuss possible lower internal validity regarding their findings. By testing multitasking abilities, using both self-report and ecological tools, they found other factors than EFs—for instance mood and motivation—to have an influence on performance in adults with ADHD. Commonly used EF self-report questionnaires include the Behavior Rating Inventory of Executive Function–Adult version (BRIEF-A; Giora, Isquith, Guy, & Kenworthy, 2000) and the Dysexecutive Questionnaire (DEX; Wilson, Alderman, Burgess, Emslie, & Evans, 1996) that are standardized to report EF in daily behaviors. However, these tools do not provide an opportunity to assess actual performance on complex daily tasks. In addition, Knouse, Bagwell, Barkley, and Murphy (2005) claim that adults with ADHD may not be accurate self-reporters and tend to overestimate their performance and overlook specific tasks. Therefore, to accurately assess EF abilities in adult students with ADHD, it is essential to rate their actual performance in a task which imitates a typical real-world activity that relies heavily on EF.
The Weekly Calendar Planning Activity (WCPA)
The WCPA was developed to examine how subtle EF difficulties influence one’s ability to perform multi-step activities in daily life (Toglia, 2012). It involves the scheduling of a list of daily life appointments into a weekly calendar while following given rules. The person must satisfy the task constraints while strategically scheduling the appointments with minimal errors. Thus, the WCPA provides a structured activity analysis and assessment of complex task performance, including errors, self-evaluation, and strategy use, which can then be used to guide intervention (Weiner et al., 2012). Discriminant validity was found to distinguish between at-risk and typical high school youths in following rules, entering appointments accurately, self-reporting of errors, and use of strategies. These findings suggest that the WCPA is suitable for adolescents and young adults as a performance-based assessment of EF (Toglia & Berg, 2013). Findings from the WCPA pilot study indicated that participants who performed better on the task used more strategies, took more time, followed more rules, and were more deliberate and accurate overall (Weiner et al., 2012). A version for students was developed and is reported in the current study.
The objectives of the current study were to examine the diagnostic efficacy of the WCPA-S in discriminating between students with and without ADHD and across gender.
Method
Participants
Participants include a convenience sample of 157 students enrolled in universities/colleges in Israel. Sixty-one participants were diagnosed with ADHD by standard diagnostic procedures and 96 students without ADHD served as the control group. The total study sample included 101 (64%) female and 56 (36%) male students, ranging in age from 20 to 34 years (M = 26.12 years, SD = 2.55). Students diagnosed with additional LD (n = 10) were excluded from the study. Most of the students (94%) were undergraduates. They included students majoring in the social sciences and in business studies (29%), health professions (21%), and education (14%), and the remainder were studying law, communication, or basic sciences.
To ensure that the general academic ability of the ADHD and non-ADHD groups were similar, a t-test analysis was applied to examine the difference between the mean score of the two groups on the standardized Israeli psychometric entrance test, which is required for admission into Israeli higher educational institutions. The difference between their mean scores (ADHD: M = 584.71, SD = 52.22; non-ADHD: M = 596.01, SD = 46.90) was not significant (t = −1.190, p > .05).
Instrument
The WCPA-S (Toglia, Katz, Inbar-Weiss, Ben-Simon, & Lahav, 2012) is an adapted version of the original WCPA described above (Toglia, 2012). It is a performance-based EF assessment that explores the relationships between task performance accuracy, time management, strategy use, error patterns, and the self-evaluation of performance. After receiving the author’s permission, the WCPA was translated into Hebrew and modified to better suit the Israeli students’ culture and daily routines (e.g., pick up the cellular phone from repair, buy sneakers at the mall, lunch with a friend, exercise at the gym), their ages, and educational levels. In addition, following the administration of a draft version of the WCPA-S to a small sample of ADHD and non-ADHD students, the wording of some of the instructions were changed to improve their clarity and part of the instrument’s formatting was modified (Toglia et al., 2012). The revised version (WCPA-S) was then administered to 94 healthy Israeli students to assess its psychometric properties. Moderate to high test–retest reliabilities were found for the various performance measures, with intra-class correlations ranging from 0.60 to 0.85.
The final version of the WCPA-S involves the scheduling of 17 daily life appointments into a weekly calendar while adhering to four constraint rules, such as “do not cross out appointments after entering them on the weekly calendar sheet.” The task is administered individually and students are instructed to finish the task within 30 min. However, to measure total performance time, the examiner does not stop the testing at that point. The examiner observes the student perform the task and interviews the student upon his or her completion. In addition, the student is required to monitor the time left for the completion of the task and inform the examiner when 20, and then 10 min remain.
The WCPA-S performance outcome measures are (a) the number of appointments correctly entered; (b) the number of appointments missing; (c) stating the time left for completion correctly at two points in time (10 and 20 min) on an ordinal 4-point scale (1 = stated within a minute of the correct time, 2 = between 1 and 2 min, 3 = more than 2 min from the correct time, and 4 = did not state at all); (d) using strategies (yes or no) during the scheduling process, for example, reads over the tasks briefly before beginning, checks oneself after inserting an appointment in the calendar, used a finger while reading, prepares a list or a diagram on the scratch sheet; (e) total performance time; and (f) self-report of the difficulty experienced in performing the task (on an ordinal 3-point scale, 1 = stated no difficulty, 2 = moderate difficulties, and 3 = many difficulties). A detailed manual was developed to outline the scoring components (Toglia et al., 2012).
Inter-rater reliability of the coding and scoring of performance was examined on 20 participants, and ranged from r = .937 to .989 for the first three performance measures.
Procedure
The study was approved by the Ono Academic College Institutional Ethics Committee. Each participant signed a consent form. Participants were individually tested by trained examiners in private testing rooms according to the WCPA-S protocol. The administration of the test took 20 min and another 15 min for rating the test, all rating were done after the test was completed.
Data Analysis
To test the difference in performance between the ADHD and non-ADHD students and between gender, the study applied a Two-Between (Group: ADHD vs. control) × Two-Between (Gender: female vs. male) design. Dependent variables were the WCPA-S performance scores and strategies use. All analyses were performed with IBM SPSS 20.0 software (IBM SPSS, Inc., Chicago, IL).
Results
The ANOVAs by Group (ADHD vs. control) and Gender (female vs. male) for WCPA-S performance scores and strategies are presented in Tables 1 and 2. The results for the WCPA-S scores are presented in Table 1 while the strategy use is shown in Table 2.
Means, Standard Deviations, and Results of ANOVAs for WCPA-S Performance Scores.
Note. WCPA-S = Weekly Calendar Performance Activity for students.
Means, Standard Deviations, and Results of ANOVAs for Strategies Use.
Significant differences between the ADHD and the control group were found in almost all performance scores on the WCPA-S. These includes the number of appointments correctly entered (F = 6.31, p = .013), number of missing appointments (p = .024), total performance time (F = 13.51, p = .000), stating 10 min left (F = 11.55, p = .001), and the total number of strategies in use (F = 9.83, p = .002). On all scores, the ADHD group were underperforming compared with the control group. Only one score was nearly significant, stating that 20 min have left (p = .057; see Table 1).
Gender differences were found on three scores only: correctly entered (F = 7.74, p = .006), missing appointments (F = 4.59, p = .034), and number of strategies in use (F = 7.40, p = .007). Female students performed better than male students on the above scores. No gender differences were found on all remaining scores.
For the interaction, one significant Group × Gender (F = 6.70, p = .011) was found in the number of correctly entered appointments. This result stems from the large difference between ADHD males (M = 11.18) and control males (M = 13.58) while female performance in both groups was similar (M = 13.70 for ADHD and M = 13.67 for the control group; see Table 1).
Significant differences regarding the use of strategies between the ADHD and control group are presented in Table 2. The control group participants used three strategies significantly more often than ADHD participants: read before began (F = 5.41, p = .021), uses finger (F = 4.99, p = .027), and self-talk (F = 11/65, p = .001). No significant group differences were found in the remaining six strategies.
Significant gender differences were found for three strategies that were used more frequently, mark key words (F = 8.28, p = .005), use lists (F = 5.11, p = .025), re-read instructions (F = 4.09, p = .045). No significant Group × Gender interaction was found. A near significant interaction was found in uses finger (p = .067). Within the ADHD group, females used it much more often (M = .82 vs. .61), whereas in the control group, females used it a little less often (M = .84 vs. .88).
Discussion
The aim of the current study was to examine the WCPA-S, an EF measure in everyday life, comparing between students with and without ADHD and across gender.
Group and Gender Differences
Performance
The findings of the current study indicate that students with ADHD underperformed significantly in most WCPA-S measures compared with control group students (without ADHD). Students with ADHD required more time to complete the task, missed more appointments, were less successful stating specific task remaining time, used fewer strategies, and expressed difficulty more often. These results are in line with the finding on EF that tend to be impaired among people with ADHD (Barkley & Murphy, 2011; Biederman et al., 2006). The WCPA-S assessment evaluates EF performance with respect to time management, organization abilities, and planning (Toglia et al., 2012). Barkley and Murphy (2011) found that EF deficits among adults with ADHD might be represented in daily life activities by five problem dimensions, including self-management to time and self-organization. College students with ADHD reported more difficulties compared with students without disability in time management and organizational skills (Barkley & Murphy, 2011; Reaser, Prevatt, Petscher, & Proctor, 2007).
Thus, the current study results support previous research and suggest that impairments in these abilities may explain the gaps between the students with and without ADHD on the performance of the WCPA-S. Experience of time is a specific mental function of a person’s subjective experiences relating to the length and passage of time (World Health Organization, 2001) and is a component of EF (Brown, 2009; Ustun, 2007) often impaired in people with ADHD and LD (Brown, 2009; Dahan et al., 2008; Grinblat & Rosenblum, 2012; Ustun, 2007). Grinblat and Rosenblum (2012) measured this ability among students with learning difficulties, by asking them to estimate how long it would take them to prepare a cup of coffee before and after the performance, and compared their estimate to the actual performance time as measured with a timer. The findings of their study indicate that students with LD have difficulties estimating the time needed to perform this activity. It can be assumed that students with ADHD also struggle with difficulty in experiencing time, such as when stating how much time was left to complete the WCPA-S task. Therefore, poor time management, experience of time, organization and planning skills contributed to the fact that students with ADHD needed more time to complete the assessment, were less accurate, missed more appointments, and had more difficulty in estimating the time remaining to complete the task. In addition, impaired working memory is one of the core deficits of people with ADHD (Barkley, 1997; Brown, 2009), which may reflect the difficulty of holding one or several things “on line” while attending to another tasks (Brown, 2009). This could explain why it was difficult for the students with ADHD to remember the instruction asking them to state when 10 min remained for the completion of the WCPA-S in their working memory. It is reasonable to assume that these impairments led the students with ADHD to express difficulty more often than those without ADHD.
A few significant gender differences were revealed in the current study. Female students were more accurate, missed fewer appointments, and used more strategies. This is consistent with Gershon (2002) results, in which women with ADHD were rated as being significantly less impaired than men with ADHD regarding hyperactivity, inattention, and impulsivity. However, it doesn’t fit previous findings suggesting higher impairment among female students with ADHD (Fedele et al., 2012; Merkt & Gawrilow, 2014; O’Callaghan & Sharma, 2014). As indicated earlier, those findings were based on self-report measurements, which are subject to the person degree of self-awareness and insight and the timing of diagnosis (Merkt & Gawrilow, 2014). In addition, Fedele et al. (2012) suggest that college women may have higher behavioral and functional standards for themselves, place more value than men on functioning in some area of performance and negatively influenced by societal expectations and stereotypes. Women with ADHD are more likely to internalize symptoms (Gershon, 2002; Quinn, 2005; Rucklidge, 2008), show poorer self-concept, and report having fewer assets and more problems than their male counterparts (Arcia & Conners, 1998, as cited in Quinn, 2005). However, there is no definite agreement about gender and subtype distribution among adults (Grevet et al., 2006; Retz-Junginger et al., 2010).
According to some researchers, women with ADHD are more often diagnosed as predominantly inattentive than men with ADHD (Nussbaum, 2012; Quinn, 2005; Rucklidge, 2008), and show low prevalence of co-morbid externalizing problems (Gershon, 2002). This may be the reason why they performed better in the current study. The group differences in the number of correctly entered appointments were qualified by Group × Gender interaction. Males with ADHD entered less accurate appointments compared with control group males, whereas females with ADHD performed similar to control group females. This is in line with Graetz, Sawyer, and Baghurst (2005), as well as Nussbaum (2012) findings, both indicating the tendency of men to be more impaired than women in combined and hyperactive-impulsive groups. Among children, both girls and boys aged 8 to 13 with ADHD have been shown to exhibit deficits in response inhibition, but the deficits among boys were consistently observed on tasks requiring the ability to consciously withhold an incorrect response (O’Brien, Dowell, Mostofsky, Denckla, & Mahone, 2010). Thus, difficulties in response inhibition and response preparation may also explain the tendency of the men with ADHD in our study to be less accurate and miss more appointments in the WCPA-S than those without ADHD.
Strategies use
Significant differences were found regarding the use of strategies between the ADHD and control group. The control group participants used more often the strategies read before began, uses finger, and self-talk than ADHD students. In accord with previous studies, students with ADHD have difficulties in focusing on details, identifying important points, understanding what is asked of them, and planning their work (Reaser et al., 2007). In addition, they are less organized, procrastinate considerably more, and employ fewer self-control behaviors (Turnock, Rosen, & Kaminski, 1998). According to Barkley (1997), developing effective coping strategies is a multifaceted and attention-demanding process that relies heavily on EFs and, therefore, present special difficulties for those with deficiencies in these areas as can be seen in our findings. Efficient performance of a multi-step activity such as the WCPA-S requires the use of coping strategies. This may help explain the limited use of strategies by students with ADHD in the current study, which resulted in the fact that their performance was less accurate and that they missed more appointments compared with students without ADHD. The use of strategies may also be influenced by the time of diagnosis. Adults who receive their diagnosis in adulthood reported higher rates of depression/anxiety, a more external locus of control, and comorbidity of health problems. They adopt more maladaptive coping strategies such as being overly cautious and invest more compensatory effort (Rucklidge et al., 2007; Rucklidge & Kaplan, 1997). Those strategies fail to be sufficiently when demands get higher in university (Merkt & Gawrilow, 2014).
Significant gender differences were found regarding the use of strategies between men and women. Women used more often the strategies mark key words, use lists, and re-read the instructions. J. Lawrence, Ashford, and Dent (2006) investigated gender differences in the coping strategies adopted by first-year students in higher education. They found significant gender differences in terms of engagement in coping strategies and academic attainment. Males exhibited greater ability to detach themselves from the emotions of a situation, were more inclined to demonstrate emotional inhibition, and reported higher self-esteem, whereas, females attained significantly higher level than males. Their results, revealing gender to be a key determinant in academic success, are in line with this current study suggesting higher strategies use and better performance among women.
Conclusion, Limitations, and Implication
The WCPA-S enabled the comparison of students with ADHD and without ADHD in the performance of everyday EFs, and contributes to our understanding of the specific factors relating to EFs that prove to be difficult within the ADHD student population. The advantage and main contribution of the WCPA-S assessment of EFs as compared with other self-report measures is that it is based on the performance of a task involving relevant real-world everyday activities that make sense to the individuals who execute it. As such, it is a useful and relevant measure of EF in daily performance that provides objective data and is easy to administer. The findings support the validity of the instrument in differentiating between groups of students with and without ADHD as well as gender differences within each group. However, the study was limited in that the WCPA-S was not validated against other known EF instruments. Thus, further research should try to relate the WCPA-S to other measurements of EF. Future testing should also try to assess the predictive value of the WCPA-S for college achievement. The WCPA-S may also be tested for its sensitivity for interventions and to discern post-intervention outcomes, to establish its use to help ADHD college students with time management and academic performance.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
