Assessing the Content Validity of the Investigator-Rated ADHD Rating Scale Version IV Instrument Inattention Subscale for Use in Adults With Phenylketonuria

Abstract

Content validity of the 18-item Investigator-Rated Attention-Deficit Hyperactivity Disorder (ADHD) Rating Scale IV (I-ADHD RS-IV) with adult prompts was investigated using qualitative interviews of US clinicians who had prior experience rating adults with phenylketonuria (PKU) using the I-ADHD RS-IV. Fourteen qualitative interviews were conducted to obtain key symptom experiences of adults with PKU and assessed the relevance, clarity, and administration of the I-ADHD RS-IV. Participants (n = 13, 92.9%) endorsed the inattention symptoms as key experiences by adults with PKU and endorsed the instrument as fit for purpose for adults with PKU. Participants generally reported low frequencies of occurrence for the 9 I-ADHD RS-IV hyperactivity/impulsivity items. Despite some clinicians’ concerns for the lack of patient self-awareness, the participants reported no difficulty selecting a rating on these items. This in-depth study of the content validity of the I-ADHD RS-IV provides evidence that this clinician-reported instrument captures the severity of important inattention symptoms in adults with PKU.

Keywords

phenylketonuria content validity ADHD RS-IV clinician-reported outcome inattention

Introduction

Phenylketonuria (PKU; Online Mendelian Inheritance in Man 261600 and 261630) is an autosomal recessive genetic disorder caused by a functional deficit in the phenylalanine (Phe) hydroxylase enzyme, which subsequently impairs the conversion of Phe into tyrosine.¹ With diagnosis and treatment established, early symptoms can be prevented, but if untreated Phe can accumulate in the body and cause mental retardation, microcephaly, delayed speech, delayed social skills, jerking movements or seizures, psychiatric symptoms, and behavioral abnormalities.^2,3 The PKU treatment consists of a Phe-restrictive diet supplemented with a Phe-free mixture of amino acids; however, the diet can be difficult to adhere to and adherence to the diet is often poor.⁴ Uncontrolled blood Phe levels in adulthood are associated with executive dysfunction, lack of concentration, anxiety, depression, and a variety of behavioral and psychiatric problems.^5

-8

The Investigator-Rated Attention-Deficit Hyperactivity Disorder (ADHD) Rating Scale IV (I-ADHD RS-IV) with adult prompts is a measure of attention-deficit and hyperactivity/impulsivity used in a number of ADHD studies.⁹ The psychometric properties of this clinician-reported instrument have been previously established in a population of adults with symptoms of ADHD¹⁰; however, the content validity of the I-ADHD RS-IV in an adult population with PKU has not been established. Nonetheless, this instrument could be useful for measuring key symptoms of adults with PKU.

Prior investigations of the Adult ADHD Self-Report Scale (ASRS) among adults with PKU demonstrated the content validity¹¹ and psychometric properties¹² of the inattention subscale of this instrument. However, among adults with PKU, full self-awareness of their symptoms and the effects of these symptoms is problematic. A recent survey of metabolic clinics in the United States noted that 50% of clinics report that patients’ cognitive deficits may impact their ability to obtain regular medical care and three-quarters cited related issues, such as patient apathy and treatment noncompliance toward a continued Phe-restricted diet.¹³ One metabolic clinic that implemented screening for psychiatric distress during routine clinic visits found that 53% of adults screened positive for psychiatric distress and 24% for executive function impairment.¹⁴ Therefore, the use of a self-reported instrument like the ASRS to measure inattention in adults with PKU may also be problematic.

Aims

This study investigated the content validity of the I ADHD RS-IV instrument with adult prompts, with a focus on the inattention subscale, through qualitative interviews among professionals with experience working with adults with PKU. Content validity is the extent to which the instrument measures the concept of interest; supported by evidence from qualitative interviews that the items and domains of an instrument are appropriate and comprehensive relative to its intended measurement concept, population, and use.¹⁵ The interviews were designed to achieve the following goals: (1) elicit the symptom experiences of adults with PKU from the clinician perspective and (2) assess the relevance and clarity of the inattention items in the I-ADHD RS-IV and any difficulties associated with administration in adults with PKU.

Methods

Study Design

Cross-sectional qualitative interviews were conducted with 14 professionals who have experience in rating adults with PKU using the I-ADHD RS-IV. Potential participants were identified as those with an educational background in genetics, psychology, psychiatry, or a related field and who satisfy the following inclusion criteria: master’s or PhD candidate or degree holder in psychology or related field, or a doctor of medicine (MD) with specialization in psychiatry; having at least 5 years of experience with research in clinical trials; experience with adult ADHD strongly recommended, with PKU-related experience preferred; at least 3 years of prior experience conducting structured or semi-structured interview research assessments; and at least 1 year of past experience evaluating ADHD-like symptoms in 18 years and older adults with PKU. Participants were selected using a sampling frame of clinicians and researchers provided by the study sponsor, all of whom had been trained to use the I-ADHD RS-IV as part of a clinical trial. The study team contacted potential participants by telephone and/or e-mail to assess eligibility and interest.

The content validity interviews followed a semi-structured guide that included a concept elicitation discussion that encouraged emergent descriptions of neuropsychiatric symptoms and impacts of PKU. Afterward, a cognitive debriefing portion asked questions about the I-ADHD RS-IV inattention items and prompts to address the clarity, comprehensiveness, relevance, recall, vocabulary, and other key aspects of the instrument. After discussion of the inattention items, the interview guide probed on the occurrence of hyperactivity/impulsivity symptoms included in the I-ADHD RS-IV. Interviews were conducted until saturation on PKU-related symptoms was achieved, as defined as the point when no new relevant or important information merges.^16,17 All interviews were conducted via telephone, in English, and were digitally recorded with the consent of each participant.

Investigator-Rated ADHD RS-IV With Adult Prompts

The I-ADHD RS-IV, which assesses adult ADHD symptoms, was designed to be completed by a trained clinical psychologist or psychiatrist. The scale includes 18 items assessing the frequency, severity, and impairment of ADHD symptoms and behaviors in adults over the past month. The scale was derived directly from the diagnostic criteria as delineated in the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition).¹⁸ The questions are divided into 2 subscales: 9 items measure symptoms of inattention and 9 items measure symptoms of hyperactivity–impulsivity. The 9 items in the inattention subscale include careless mistakes, difficulty sustaining attention on task, doesn’t listen, difficulty finishing and following instructions, difficulty organizing tasks, dislikes/avoids tasks requiring attention, loses things, easily distracted, and forgetful in daily activities. The 9 items in the hyperactivity and impulsivity subscale include fidget, difficulty remaining seated, restlessness, difficulty playing quietly, on the go/acts like driven by a motor, talks excessively, burst out answers, difficulty waiting turn, and interrupts or intrudes. The items are rated using a 4-point Likert-type frequency scale (0 = none, 1 = mild, 2 = moderate, and 3 = severe). The I-ADHD RS-IV inattention and hyperactivity subscale scores range from 0 to 27, and the instrument’s total score ranges from 0 to 54. Higher scores indicate a worse severity of symptoms.

The I-ADHD RS-IV was designed to be used as a semi-structured interview guide during the assessment. The interviewer starts by asking the patient about each item and follows up with the relevant prompts for each item prepared by the developers. Using the prompts, the interviewer can gain clarity on the interviewee’s circumstances and probe beyond a yes/no response. Ratings on the I-ADHD RS-IV are based on clinical judgment and not entirely on the patient’s report.

Data Analysis

Qualitative data were analyzed using ATLAS.ti qualitative data analysis software version 7.1.6.¹⁹ All interviews were digitally recorded, and the audio recordings were transcribed. Content analysis of the output was conducted with the following steps: (1) a coding dictionary was developed based on the semi-structured interview guide and relevant probes, (2) using the coding dictionary, 2 scientific team members independently coded the first structured interview transcript and then met with a senior scientific reviewer to discuss the coding dictionary and the coding process, and (3) if the coding was sufficient, then the coders independently coded the remaining transcripts with review from a senior scientific staff member.

The cognitive interview data were coded based on the participant feedback on (1) the clarity of the I-ADHD RS-IV inattention items and prompts, (2) the ease of completing the instrument, (3) the appropriateness of the format and recall period, and (4) the appropriateness of the scale in adults with PKU. Qualitative findings were summarized with exemplary quotes, frequencies, and percentages as appropriate to best summarize the results.

Results

Study Sample

Disposition of the study sample is provided in Figure 1. Overall, 49 (57%) participants responded to the invitation to participate in the cognitive interview. A total of 37 (43%) participants did not respond to the invitation, and 35 (41%) of the invitees who responded to the invitation were either not interested or did not meet all of the cognitive interview study inclusion/exclusion criteria. A total of 14 (16%) participants expressed interest with subsequent enrollment into the study.

Figure 1.

Disposition of the study sample.

Demographics and the clinical experience of the study population are provided in Table 1. Approximately half of the participants stated their educational background as doctor of philosophy (PhD; n = 6, 42.9%) and 2 participants indicated they were medical doctors (MD/MBBS; 14.3%). Other participants’ educational backgrounds included doctorate in psychology (n = 4, 28.6%), master’s in social work (n = 1, 7.1%), and master’s in nutrition (n = 1, 7.1%). The majority of the participants’ professional role with adult patients with PKU involved diagnostic and assessment work (n = 11, 78.6%), and most of them worked with adult patients with PKU in an outpatient setting (n = 9, 64.3%). The participant’s mean years of experience working with adults with PKU was 9.9 (SD = 9.2), while the mean years of experience working with adults with inattention and hyperactivity was 14.9 (SD = 7.9). The mean age of adult patients with PKU treated by participants ranged from 19.5 (SD = 2.6) to 47.9 (SD = 9.6). The average number of adult patients with PKU seen by participants each month was 6.3 (SD = 6.8), and each year was 52.7 (SD = 59.3). Only 1 participant prescribed medication for patients with psychosocial issues (7.1%), and 1 participant for symptoms of ADHD (7.1%).

Table 1.

Clinical Practice Characteristics.

Clinical Practice Characteristic	Total Sample (N = 14)
Educational background, n (%)
Doctor of philosophy (PhD)	6 (42.9%)
Medical doctor (MD or MBBS)	2 (14.3%)
Other^a	6 (42.9%)
Profession, n (%)
Psychologist	7 (50.0%)
Other^b	7 (50.0%)
Experience working with adult patients with PKU (years)
Mean (SD)	9.9 (9.2)
Range	1-30
Experience working with adult patients with inattention or hyperactivity (years)
Mean (SD)	14.9 (7.9)
Range	2-28
Age of adult patients with PKU seen by participants (years)
Minimum age (mean, SD)	19.5 (2.6)
Maximum age (mean, SD)	47.9 (9.6)
Average number of adult patients with PKU seen by participants
Each month (mean, SD)	6.3 (6.8)
Each year (mean, SD)	52.7 (59.3)
Prescribe medication for adult patients with PKU with psychosocial issues, n (%)
Yes	1 (7.1%)
Prescribe medication for adult patients with PKU with symptoms of ADHD, n (%)
Yes	1 (7.1%)
Professional role with adult patients with PKU, n (%)^c
Involved in diagnostic and assessment work	11 (78.6%)
Prescribe medications	2 (14.3%)
Other^d	4 (28.6%)
Type of setting with adult patients with PKU, n (%)^c
Inpatient	1 (7.1%)
Outpatient	9 (64.3%)
Other^e	6 (42.9%)

Abbreviations: ADHD, attention-deficit hyperactivity disorder; PKU, phenylketonuria; SD, standard deviation.

^aMaster’s in social work (n = 1); doctorate in psychology (n = 4); master’s in nutrition (n = 1).

^bClinical psychologist (n = 2); neuropsychologist (n = 1); physician (n = 2); research dietician (n = 1); research and educational coordinator in genetics (n = 1).

^cCategories are not mutually exclusive.

^dEducation (n = 1); maintain diet and study coordinator (n = 1); referrals (n = 1); researcher (n = 1).

^eClinical trial setting/research setting (n = 4); community (n = 1); university/medical school (n = 1).

The PKU Symptoms

Participants were asked to discuss the symptoms observed in adult patients with PKU. If not spontaneously mentioned, participants were probed on previously reported PKU-related cognitive difficulties (eg, difficulty paying attention, difficulty concentrating), difficulty in social situations, emotional difficulties (eg, problems with mood, specific mood problems), physical problems, and hyperactivity/impulsivity. The PKU symptoms endorsed by the participants were mapped to the I-ADHD-RS IV inattention items. Table 2 provides examples of PKU symptoms reported by participants for each I-ADHD RS-IV inattention item. The most common symptoms reported by participants included poor sustained attention (n = 8, 57.1%), difficulty with cognitive ability (n = 6, 42.6%), difficulty with planning and organizational skills (n = 6, 42.6%), forgetting things/memory (n = 6, 42.6%), and anxiety (n = 5, 35.7%). Other symptoms reported by the participants were grouped according to the following categories: cognitive difficulties (unable to problem solve, mental fogginess, difficulty concentrating, and difficulty making decision), difficulty in social situations (social difficulties and withdrawn), emotional difficulties (irritability, depression, anger, anxiety, and feel like they have a personality disorder), physical problems (fatigue or tired and hypersomnia), and hyperactivity/impulsivity (hyperactivity, interrupting others, fidgeting and moving around, impulsive responses, restless, impulsive behavior, and obsession).

Table 2.

The PKU Symptom Mapped to the I-ADHD RS-IV Inattention Subscale Items.

I-ADHD-RS IV Inattention Items	PKU Symptom Reported by Participant
Careless mistakes	Careless mistakes
Careless mistakes	Inattention to detail
Difficulty sustaining attention on tasks	Poor sustained attention
Difficulty sustaining attention on tasks	Unable to focus
Doesn’t listen	Missing pieces of conversations
Doesn’t listen	Drift off in conversations
Difficulty finishing, following instructions	Poor follow through on tasks
	Difficulty following instructions
	Difficulty completing tasks
Difficulty organizing tasks	Time management problems
Difficulty organizing tasks	Difficulty with planning and organizational skills
Dislikes/avoids tasks requiring attention	Things that people are interested or motivated in that they’ll continue to be able to pay attention to, but then when you get to things like the reading or lectures for some people are boring and repetitive tasks [unable to pay attention]
	Didn’t want to really do it or it seemed overwhelming, so they start a different task
	Avoid tasks like work, chores and reading
Loses things	Loses things
Easily distracted	Easily distracted
Forgetful in daily activities	Forgetting things/memory
	Difficulty keeping track of things
	Missed appointments
	Forget to finish chores

Abbreviations: I-ADHD RS-IV, Investigator-Rated Attention-Deficit Hyperactivity Disorder Rating Scale IV; PKU, phenylketonuria.

A total of 8 participants (57.1%) reported symptoms of hyperactivity or impulsivity, with 2 participants stating that hyperactivity/impulsivity symptoms are not really observed in adults, and 1 participant stated that hyperactivity/impulsively symptoms are probably less common. The hyperactivity symptoms included fidgeting (n = 4, 28.6%), moving around (n = 1, 7.1%), interrupting conversations (n = 1, 7.1%), difficulty remaining seated (n = 1, 7.1%), being on the go (n = 1, 7.1%), restless (n = 1, 7.1%), talking too much (n = 1, 7.1%), and hyperactivity with no specific details (n = 3, 21.4%).

Self-Awareness of Adults With PKU

As reported by several participants, many factors contribute to the challenge of self-awareness that adults with PKU have regarding their own symptoms. Five of these professionals felt that adults with PKU, in general, are aware of their symptoms, while another 5 reported that overall adults with PKU have problems with self-awareness. Several participants commented on the spectrum of self-awareness and the contributing factors that cause some adults with PKU to be more self-aware than others. Specifically, participants felt that those adults with PKU with more symptoms, lower cognitive ability, and Phe levels that are not controlled (definition of control not provided by participant) have less self-awareness, while those with higher cognitive functioning, employment, and Phe levels that are well controlled tend to have more self-awareness. The following are 3 exemplary quotes from study participants. “I don’t think they’re very aware, I mean some more so than others, there’s always a spectrum. I think the few who are more figuratively affected or more symptomatic have less self-awareness.” “I think their self-awareness increases as they get on the job. I think that they do not have a lot of self-awareness initially and a lot of times I find that when I do some of the initial ratings with them they say everything is fine and then when we go back once they on the drugs for a while then they really start saying oh, now I see that this was a problem before and now I see how much better it is.” “The well-controlled patients actually have quite good self-awareness. The patients with higher Phe levels will certainly be worse off, the lack of awareness is there, and paradoxically we get higher reading scores on the milder patients.”

Cognitive Interview Results: I-ADHD RS-IV Inattention Subscale

The raters in this study endorsed the I-ADHD RS-IV inattention subscale as an appropriate instrument to measure inattention symptoms in adults with PKU, with 71% (n = 10) of participants stating each item was applicable to adults with PKU (Figure 2). The majority of the participants who were asked (n = 11, 85%) stated a 1-month recall period was appropriate to assess the concepts. In addition to providing feedback on the items, participants were asked about the item prompts. The majority of the participants (n = 10, 71%) stated that the item-level prompts were sufficient for 7 of the 9 items, with just over half of the participants (n = 9) stating that item-level prompts were sufficient on the 2 items (difficulty finishing, following instructions, and dislikes/avoids tasks requiring attention; Figure 2). Some participants had recommendations for augmentations to the prompts (additional prompt(s)) but noted that the I-ADHD RS-IV training guide and instructions allow the rater to ask the patient additional questions/prompts to elicit greater details in the symptom experience when administering the instrument.

Figure 2.

I-ADHD RS-IV inattention subscale items: cognitive interview results. I-ADHD RS-IV indicates Investigator-Rated Attention-Deficit Hyperactivity Disorder Rating Scale IV.

At the item level, the majority of participants stated that a patient’s self-awareness of symptoms can make the rating of some items more challenging (ie, careless mistakes, doesn’t listen, dislikes/avoids tasks requiring attention, and easily distracted; Figure 2), but self-awareness was less likely to affect 3 of the 9 items in the inattention domain (ie, loses things, difficulty finishing, following instructions, and difficulty organizing tasks). For 1 inattention item (forgetful in daily activities), half of the sample stated that self-awareness posed a challenge, and half of the sample stated that self-awareness is not a problem for patients. While feedback from the participants on a patient’s ability to accurately reflect on their own symptom experience plays an important role in the administration and interpretation of the I-ADHD RS-IV, the participants fully understood that the rating is based on clinical judgment, and despite concern for the lack of patient self-awareness, the participants reported no difficulty selecting an appropriate rating on these items when assessing an adult with PKU.

Cognitive Interview Results: I-ADHD RS-IV Hyperactivity/Impulsivity Subscale

Items 10 to 18 in the I-ADHD RS-IV instrument contain hyperactivity and impulsivity symptoms. Given that participants stated that hyperactivity/impulsivity symptoms are less common in adults with PKU, participants were asked about these items in aggregate with a general question about which items were applicable or not applicable to adults with PKU and which items represent symptoms frequently occurring in adults with PKU. In general, participants felt that the hyperactivity/impulsivity items are appropriate for adults with PKU, with the exception of item 13 (difficulty playing quietly). While each of the 9 items were reported by at least 1 participant as occurring in adults with PKU, the frequency of occurrence was low for each item. Moreover, each of the hyperactivity/impulsivity item was mentioned as not occurring by at least 1 participant (Table 3).

Table 3.

The I-ADHD RS-IV Hyperactivity/Impulsivity Subscale Items.

I-ADHD RS-IV Hyperactivity/Impulsivity Subscale Item	Frequently Occur^a (N = 12)^b	Do Not Occur Frequently^a (N = 12)^b
10. Fidget	7 (58.3%)	1 (8.3%)
11. Difficulty remaining seated	3 (25.0%)	3 (25.0%)
12. Restlessness (runs or climbs excessively)	4 (33.3%)	2 (16.7%)
13. Difficulty playing quietly (leisure activities)	1 (8.3%)	8 (66.7%)
14. On the go/acts like driven by a motor	5 (41.7%)	3 (25.0%)
15. Talks excessively	6 (50.0%)	2 (16.7%)
16. Burst out answers	5 (41.7%)	3 (25.0%)
17. Difficulty waiting turn	7 (58.3%)	1 (8.3%)
18. Interrupts or intrudes	5 (41.7%)	2 (16.7%)

Abbreviation: I-ADHD RS-IV, Investigator-Rated Attention-Deficit Hyperactivity Disorder Rating Scale IV.

^aCategories are not mutually exclusive.

^bTwo participants did not provide a response; percentage was calculated based on 12 participants.

Discussion

Qualitative interviews were conducted in 14 participants with experience in rating adults with PKU using the I-ADHD RS-IV; the purpose of these interviews was to understand the PKU symptom experience and to assess the relevance of the concepts and clarity of the I-ADHD RS-IV inattention items for use in the adult population with PKU. Saturation, as defined as the point when no new relevant or important information merges and collecting additional data will not likely add to the understanding of the concepts of interest and the items in the questionnaire, of the PKU-related symptoms reported by the participants was obtained with the 14 cognitive interviews.^16,17 The most common spontaneously reported symptoms of adults with PKU by these professionals included poor sustained attention, difficulty with cognitive ability, difficulty with planning and organizing skills, forgetting things/memory, and anxiety. Symptoms probed during the course of the interview included cognitive difficulties, difficulty in social situations, emotional difficulties, physical problems, and hyperactivity/impulsivity. While symptoms within these clusters were reported as observed traits in adults with PKU, inattention was the major theme and heavily endorsed by these professionals as a hallmark symptom in adults with PKU. In addition, the PKU inattention symptoms reported by the participants mapped to all 9 inattention subscale items of the I-ADHD RS-IV, further supporting the content validity of the instrument for use in adults with PKU. The findings from the concept elicitation questions were consistent with the literature; executive functioning, including planning and prioritizing, task initiation, and organization are the major deficits in adults with PKU, and hyperactivity/impulsivity symptoms are more commonly reported in children with PKU versus adults.^20

-24

As part of the cognitive interview, participants were asked about symptom self-awareness in adults with PKU. Accurate reflecting and reporting of the symptom experience by the patient is a critical part of the I-ADHD RS-IV assessment. While the I-ADHD RS-IV allows the rater to select a response based on their clinical judgment, the rating is dependent on the patient interview and observation of the patient. Mixed responses from the participants were reported, with half of the participants endorsing self-awareness of patients’ own symptoms as problematic, and half of the participants responding that self-awareness is not a problem in eliciting the patient’s symptom experience. While a patient’s self-report on her or his symptom severity may differ from a rater’s clinical judgment and observation of the patient’s behavior, the patient is able to report on their experience and the I-ADHD RS-IV allows for the use of prompts (both scripted and not scripted), clinical judgment, and observation to make a rating for each of the instrument’s items despite lack of symptom self-awareness in adults with PKU.

In qualitative research methodology, there are some innate limitations. All data were self-reported and provided through one-on-one telephone interviews that lasted 60 minutes. The interview guide was semi-scripted and intended to facilitate a fluid discussion about PKU symptoms and the I-ADHD RS-IV. The interviewer was allowed to use judgment and adapt the guide in order to cover the questions during the interview time allotted. If a participant was more verbose in his or her responses or focused on a particular area of interest, it sometimes was not possible to ask all the questions in the guide within the interview time frame. Therefore, given the fluid nature of a semi-scripted qualitative interviews, the sample size for each question that was asked was not always 14 (the total sample enrolled), and the data were summarized in a reduced sample.

In conclusion, this in-depth study of the content validity of the I-ADHD RS-IV inattention subscale provides evidence that the clinician-reported instrument captures the severity of these important symptoms in adults with PKU and is appropriate for use in a PKU clinical trial to measure treatment efficacy on inattention symptoms. Additional research is required to assess the psychometric properties (ie, reliability, construct validity, measurement of change) in the context of the PKU population of interest and provide additional support for the usefulness of this measure for understanding key symptoms in adults with PKU.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Yinpu Chen, Rishabh Jain, Laurel Konkol, Markus Merilainen, and Haoling Weng are employees of BioMarin Pharmaceutical and receive financial support in terms of salary and stock.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Kathleen Wyrwich, Elizabeth Bacci, Katharine Gries, and Gemma Al-Jassar are employees of Evidera, which received financial support from BioMarin Pharmaceutical in connection with the study development and execution as well as the manuscript development.

References

Hoeks

den Heijer

Janssen

. Adult issues in phenylketonuria. Neth J Med. 2009;67(1):2–7.

Fitzgerald

Morgan

Keene

Rollinson

Hodgson

Dalrymple-Smith

. An investigation into diet treatment for adults with previously untreated phenylketonuria and severe intellectual disability. J Intellect Disabil Res. 2000;44(pt 1):53–59.

Yannicelli

Ryan

. Improvements in behaviour and physical manifestations in previously untreated adults with phenylketonuria using a phenylalanine-restricted diet: a national survey. J Inherit Metab Dis. 1995;18(2):131–134.

Santos

Magalhaes Mde

Januario

Aguiar

Carvalho

. The time has come: a new scene for PKU treatment. Genet Mol Res. 2006;5(1):33–44.

Moyle

Fox

Arthur

Bynevelt

Burnett

. Meta-analysis of neuropsychological symptoms of adolescents and adults with PKU. Neuropsychol Rev. 2007;17(2):91–101.

Pietz

Fatkenheuer

Burgard

Armbruster

Esser

Schmidt

. Psychiatric disorders in adult patients with early-treated phenylketonuria. Pediatrics. 1997;99(3):345–350.

Smith

Knowles

. Behaviour in early treated phenylketonuria: a systematic review. Eur J Pediatr. 2000;159(suppl 2):S89–S93.

Waisbren

. Phenylketonuria. In: Goldstein

Reynolds

, eds. Handbook of Neurodevelopmental and Genetic Disorders in Children. New York, NY: Guildford Publications; 1999:433–458.

Michelson

Adler

Spencer

. Atomoxetine in adults with ADHD: two randomized, placebo-controlled studies. Biol Psychiatry. 2003;53(2):112–120.

10.

Adler

Spencer

Biederman

. The internal consistency and validity of the attention-deficit/hyperactivity disorder rating scale (ADHD-RS) with adult ADHD prompts as assessed during a clinical treatment trial. J ADHD Relate Disord. 2009;1(1):14–24.

11.

Wyrwich

Shaffer

Gries

. Content validity of the ADHD Rating Scale (ADHD RS-IV) and Adult ADHD Self-Report Scale (ASRS) in phenylketonuria. J Inborn Errors Metab. 2016;4:1–9.

12.

Wyrwich

Auguste

. Evaluation of neuropsychiatric function in phenylketonuria: psychometric properties of the ADHD rating scale-IV and adult ADHD self-report scale inattention subscale in phenylketonuria. Value Health. 2015;18(4):404–412.

13.

Angelino

Bone

Kuehl

. A neuropsychiatric perspective of phenylketonuria II: needs assessment for a psychiatric presence. Psychosomatics. 2012;53(6):541–549.

14.

Burton

Leviton

Vespa

. A diversified approach for PKU treatment: routine screening yields high incidence of psychiatric distress in phenylketonuria clinics. Mol Genet Metab. 2013;108(1):8–12.

15.

Food and Drug Administration. Guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Fed Regist. 2009;74(235):65132–65133.

16.

Leidy

Vernon

. Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics. 2008;26(5):363–370.

17.

Willis

. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage; 2005.

18.

Goodman

. ADHD in adults: update for clinicians on diagnosis and assessment. Prim Psychiatry. 2009;16(11):21–30.

19.

Friese

Ringmayr

. ATLAS.ti 7 User Guide and Reference. Berlin, Germany: ATLAS.ti Scientific Software Development GmBH; 2013.

20.

Smith

Beasley

Wolff

Ades

. Behavior disturbance in 8-year-old children with early treated phenylketonuria. Report from the MRC/DHSS Phenylketonuria Register. J Pediatr. 1988;112(3):403–408.

21.

Karimzadeh

Tabarestani

. Promising medical treatment for childhood psycho-cognitive problems. Neural Regen Res. 2010;5(21):1663–1667.

22.

Sadek

Emam

lhaggagy

. The impacts of phenylketonuria (PKU) on children in Sohag University Hospital-Upper Egypt. J Am Sci. 2012;8(12):1326–1332.

23.

Cortese

. The neurobiology and genetics of Attention-Deficit/Hyperactivity Disorder (ADHD): what every clinician should know. Eur J Paediatr Neurol. 2012;16(5):422–433.

24.

Faraone

Biederman

Mick

. The age-dependent decline of attention deficit hyperactivity disorder: a meta-analysis of follow-up studies. Psychol Med. 2006;36(2):159–165.