Reliability and Validity of the HD-PRO-Triad TM,a Health-Related Quality of Life Measure Designed to Assess the Symptom Triad of Huntington’s Disease

Abstract

Background:

Huntington’s disease (HD), is a neurodegenerative disorder that is associated with cognitive, behavioral, and motor impairments that diminish health related quality of life (HRQOL). The HD-PRO-TRIAD^TM is a quality of life measure that assesses health concerns specific to individuals with HD. Preliminary psychometric characterization was limited to a convenience sample of HD participants who completed measures at home so clinician-ratings were unavailable.

Objectives:

The current study evaluates the reliability and validity of the HD-PRO-TRIAD^TM in a well-characterized sample of individuals with HD.

Methods:

Four-hundred and eighty-two individuals with HD (n = 192 prodromal, n = 193 early, and n = 97 late) completed the HD-PRO-TRIAD^TM questionnaire. Clinician-rated assessments from the Unified Huntington Disease Rating Scales, the short Problem Behaviors Assessment, and three generic measures of HRQOL (WHODAS 2.0, RAND-12, and EQ-5D) were also examined.

Results:

Internal reliability for all domains and the total HD-PRO-TRIAD^TM was excellent (all Cronbach’s α >0.93). Convergent and discriminant validity were supported by significant associations between the HD-PRO-TRIAD^TM domains, and other patient reported outcome measures as well as clinician-rated measures. Known groups validity was supported as the HD-PRO-TRIAD^TM differentiated between stages of the disease. Floor and ceiling effects were generally within acceptable limits. There were small effect sizes for 12-month change over time and moderate effect sizes for 24-month change over time.

Conclusions:

Findings support excellent internal reliability, convergent and discriminant validity, known groups validity, and responsiveness to change over time. The current study supports the clinical efficacy of the HD-PRO-TRIAD^TM. Future research is needed to assess the test-retest reliability of this measure.

Keywords

INTRODUCTION

Huntington’s disease (HD) is a neurodegenerative disorder that is estimated to affect between 3 and 10 of every 100,000 individuals worldwide [1, 2]. HD is an autosomal-dominant disorder that is caused by the presence of 36 or more CAG trinucleotide repeats in the gene coding for the huntingtin protein, which is located on the short arm of chromosome 4 [3]. HD is diagnosed on average around age 40 and symptoms progressively worsen until death (course is ∼20 years) [4]. The symptoms of HD commonly occur as a ‘triad’ of motor, cognitive, and behavioral deficits [5]. Behavioral and psychiatric symptoms often appear prior to the onset of motor symptoms [6, 7]. Behavioral symptoms include depression, anxiety, apathy and irritability [8]. Common motor complaints include uncontrollable movements such as chorea or dystonia, loss of balance, and incoordination [9]. Cognitive symptoms can include a decline in executive function, memory problems, and difficulty concentrating [7 , 10–12].

Not surprisingly, HD symptoms can have a detrimental effect on health related quality of life (HRQOL) [13], or the effect of a disease or disability on an individual’s physical, emotional, cognitive, and social well-being [14]. The first phase of HD-PRO-TRIAD^TM development was to identify a conceptual framework upon which to base the new measure. Specifically, previous work highlights the utilization of the World Health Organization framework for HRQOL, in conjunction with a panel of HD experts (i.e., practitioners and nurses) and patients to identify the physical, emotional, and cognitive aspects of HRQOL that are most relevant to individuals with HD, as well as the most common core triad symptoms of HD [15]. Although a handful of measures have been developed to examine HRQOL specific to HD [16 –21], psychometric properties to support their clinical utility is mixed (or not yet available). In some cases these measures are not designed to examine the full triad of symptoms in HD, or do not assess symptoms that are most important to HD patients (i.e., chorea, speech, swallowing) [15]. More generic measures have also been used in HD (in particular, the SF-36 and SF-12), but by design these measures do not evaluate HD-specific aspects of HRQOL (e.g., chorea), and data would suggest that these generic measures are typically not responsive to change over time [22, 23] or to treatment [24] which is a significant limitation given the progressive nature of the disease.

The purpose of the HD-PRO-TRIAD is to assess the HD symptom triad as they pertain to HRQOL. The HD-PRO-TRIAD^TM is unique because it examines the full triad of HD symptoms. The HD-PRO-TRIAD^TM was developed to mitigate the limitations of previous HD-specific and generic HRQOL measures by targeting the triad of symptom domains most relevant to HRQOL in HD [15, 25]. Additionally, the HD-PRO-TRIAD^TM was designed to be easily administered and publicly available for clinicians and researchers to use via an online domain [25]. Initial development utilized phone interviews with HD patients and caregivers to create a domain framework that determined areas of HRQOL important to individuals with HD [15]. The measure was created using a pool of items from other established PRO measures including Neuro-QoL (Executive Function, General Concerns, and Emotional/Behavioral Dyscontrol) [26], TBI-QOL [27], HDQLIFE (Chorea) [18], and the Functional Assessment of Chronic Illness Therapy (FACIT) questionnaire [28]. The finalized items were then selected by a team of experts in HD, including practitioners and nurses. The HD-PRO-TRIAD^TM is a 47-item questionnaire that evaluates the three domains that comprise the HD symptom triad, namely cognition, emotional and behavioral dyscontrol, and motor function. Each item is self-rated on a 5-point Likert scale [25]. Final scores for each symptom domain range from 1 to 5, with lower scores representing higher functioning and high scores representing worse functioning (details for how to convert raw scores to final scores for each domain are detailed elsewhere [25]). The total score is calculated by summing the final scores from each of the domains. Total scores range from 3 to 15 with higher scores associated with worse outcomes [25].

The HD-PRO-TRIAD^TM was previously validated in a cross-sectional study consisting of 132 patients with HD and 40 caregivers, and was found to have good internal consistency (Cronbach’s α >0.95) [25]. Additionally, the HD-PRO-TRIAD^TM had strong convergent and discriminant validity among the three domains, as each domain was strongly correlated with comparator measures (r > 0.50) and were not as strongly correlated with non-comparator measures [25]. However, the previous study noted that the survey was completed by participants at home, and therefore self-reported stage of disease, functional ability, and independence could not be verified by a clinician. Researchers were also unable to verify whether the individual with HD had completed the survey as instructed without input from their caregiver. The current study aims to replicate the reliability and validity findings for the HD-PRO-TRIAD^TM and to provide additional information about floor and ceiling effects, measurement error, known groups validity using clinician-rated disease status, and responsiveness to change over time.

MATERIALS AND METHODS

Data for the validation of the HD-PRO-TRIAD^TM were analyzed retrospectively from a longitudinal study that examines HRQOL at baseline, 12- and 24-months [16]; a detailed description of the broader study protocol is reported elsewhere [16]. Recruitment for this study also took place in collaboration with the PREDICT-HD study, a global cohort study with the purpose of assessing early symptoms of HD in prodromal individuals; ∼36% (n = 173) of participants were recruited in conjunction with the PREDICT-HD study [29]. In addition to completing several core assessments as a part of their PREDICT-HD study visit, these individuals also agreed to complete additional self-report measures that were specific to this study protocol. The remaining 64% (N = 309) of participants were recruited through other recruitment sources [16]. All data were collected in accordance with and approval of the local institutional review boards. To be eligible for the study, participants were required to have a positive gene test and/or a clinical diagnosis (made by a neurologist, physician, or other medical professional) of HD and be ≥18 years of age. Participants were also required to be capable of providing informed consent; cognitive status was assessed using a standard assessment [30].

Participant visits

Participants were recruited through eight established HD clinics (Los Angeles, CA; Iowa City, IA; Indianapolis, IN; Baltimore, MD; Ann Arbor, MI; Golden Valley, MN; St. Louis, MO; Piscataway, NJ), the National Research Roster for Huntington’s Disease, online medical record data capture systems [31], and through articles/advertisements in HD-specific newsletters and websites. Recruitment also included HD support groups and HD specialized nursing home units throughout the United States. Participants completed an in-person assessment, which was followed by a computer based survey regarding HRQOL. For baseline visits, participants were required to be seen and provide informed consent in-person in either a hospital, clinical, or home setting. For 12- and 24-month visits participants who were unable or unwilling to return to the clinic, but wanted to continue their participation, were given the option of being interviewed via telephone given that previous work does not find differences regarding mode of administration [32, 33]. Each study visit lasted approximately two hours (ninety minutes for phone visits, as motor and cognitive assessments that required in-person contact were not administered).

Measures

Demographic variables

Participants were asked to self-report demographic information (age, gender, marital status, race, and ethnicity) through Assessment Center^SM, an online data collection platform [34].

Medical record confirmation

After obtaining informed consent from the participants, researchers retrieved participant medical records to confirm their HD diagnosis and to collect data regarding the CAG repeats from the results of any previous genetic testing in the record.

HD-PRO-TRIAD^TM items [25]

All self-report measures were completed during the study visit on Assessment Center^SM within two weeks of the clinician interview. Participants were asked to recall their quality of life using questions from the previously developed and validated HD-PRO-TRIAD^TM questionnaire [25] at the baseline, 12- and 24-month visits. The HD-PRO-TRIAD^TM consists of three domains: Cognition, Emotional and Behavioral Dyscontrol, and Motor Functioning. The Cognition domain asks questions regarding self-reported memorization, concentration, and ability to learn new tasks. These items were drawn from Neuro-QoL measures (Executive Function, General Concerns) and TBI-QOL Cognition items [35 –37]. The Emotional and Behavioral Dyscontrol domain asks participants to answer questions about impulsivity, irritability, and violent or aggressive behavior. These items were taken from the Neuro-QoL measure of Emotional and Behavioral Dyscontrol [37]. The Motor domain asks for a self-report assessment of a participant’s movements, and how frequently these movements impact activities of daily living. These items were taken from the newly developed HDQLIFE Chorea item bank and from the Functional Assessment of Chronic Illness Therapy (FACIT) questionnaire [18, 28]. Some of the questions from the original HD-PRO-TRIAD^TM [25] were omitted from the current study because these items were drawn from measures that were not evaluated for the purpose of relieving participant burden (i.e., Functional Assessment of Chronic Illness Therapy or FACIT [28] and TBI-QoL [27]). The HD-PRO-TRIAD^TM scoring guide [25] allows for calculation despite missing values assuming each participant answers at least 8 of 14 cognitive items, 8 of 14 emotional and behavioral dyscontrol items, and 10 of 19 items from the Motor domain. In total, 9 questions were administered from the Cognitive domain, 13 questions from the Emotional and Behavioral Dyscontrol domain, and 13 questions from the Motor Functioning domain (see Appendix A for the complete list of items that were administered) thus exceeding the minimum criteria required for scoring. Participants who did not answer the minimum number of items in each domain were excluded from analyses (n = 54 were excluded from the baseline assessments, n = 32 from 12- and n = 177 from 24-months).

Quality of life in neurological disorders (Neuro-QoL) item banks [38, 39]

Neuro-QoL [38, 39], from which some HD-PRO-TRIAD^TM Emotional and Behavioral Dyscontrol and some Cognitive items were drawn, is a patient reported outcome measurement system designed to evaluate HRQOL in individuals with neurological conditions. Convergent and discriminant validity were examined using the Neuro-QoL measures of Upper Extremity Function and Lower Extremity Function from the baseline visit. Items were completed on a 5-point Likert scale. Neuro-QoL measures are scored using a T-score metric with a mean of 50 and a standard deviation (SD) of 10 [40]. Higher scores are representative of better functioning.

Patient Reported Outcomes Measurement Information System (PROMIS) item banks [41, 42]

PROMIS^® [41, 42] is a measurement system designed to evaluate HRQOL across a diverse range of health issues. Convergent and discriminant validity was examined by using PROMIS Depression, Anxiety, and Anger baseline data. These measures were self-reported on a 5-point Likert Scale that ranged from “Not at All” to “Very Much”. All measures are scored using a T-score metric with a mean of 50 and a standard deviation (SD) of 10 [41]. Higher scores indicate worse functioning.

EQ-5D [43]

The EQ5D [43] is a generic self-report measure of HRQOL that generates two different scores: the Index Scale score and a Health Scale score. This measure was administered at all three study visits. The EQ-5D Index Scale includes 5 items that assesse mobility, self-care, ability to perform usual activities, pain/discomfort, and anxiety/depression. Each item is rated from 0 (no problems) to 5 (severe problems or impairments), and then summed to create a total score. The EQ5D Health Scale includes a single item that assesses overall health status; this scale is rated from 0 to 100 with higher scores indicating better overall health. The baseline EQ5D Index Score was used to examine convergent and discriminant validity. Baseline scores on the EQ-5D Health Scale were subtracted from both 12-month and 24-month scores to determine change scores to examine responsiveness to change over time.

World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0.) [44]

The WHODAS 2.0 [44] is a 12-item self-report measure that examines cognition, mobility, self-care, life activities, and participation in community activities. Responses are rated from 0 (none) to 4 (extreme) and then are summed to create a total score. Scores from the baseline visit were used to determine convergent and discriminant validity.

RAND-12 [45]

The RAND-12 [45] is a 12-item measure that is used to assess self-reported mental and physical health. Physical and mental health composite scores (PHC and MHC) can be computed for the RAND-12 with scores ranging from 0 (low health) to 100 (highest level of health). We examined baseline data to determine convergent and discriminant validity.

Clinician rated measures

Clinician rated measures of behavior, cognitive function, motor ability, and functioning were utilized to further examine convergent and discriminant validity of the HD-PRO-TRIAD^TM. The Problem Behaviors Assessment-short form (PBA-s) [46] was administered to examine the severity and frequency of eleven [11] behaviors. These behaviors included depression, suicide ideation, anxiety, irritability, aggression, apathy, perseverative thinking, obsessive compulsive behaviors, delusion, hallucinations, and disorientation. PBA-s severity scores range from 0 (symptom absent) to 4 (severe). Frequency scores range from 0 (never) to 4 (daily). For each behavior, the respective severity score is multiplied by the frequency score to create a final score for that behavior. Data was examined from the baseline visit for the purposes of the proposed analyses.

The Unified Huntington Disease Rating Scales (UHDRS) [46] is a standardized tool used in evaluating motor functioning, cognition, functioning, and independence. For cognitive measures we examined scores from the baseline visit on the color naming and word reading versions of the Stroop [47] and the total score from the Symbol Digit Modalities Test [48], which both provide a measure of processing speed. The UHDRS Total Motor Scale (TMS) from the baseline visit was used to assess motor functioning. Scores on each item are rated from 0 (symptom absent) to 4 (severe symptom/could not complete); resulting in scores that range from 0 to 124, with higher scores indicating worse functioning. The final question on the TMS asks the clinician whether they can state with >99% certainty (based on a scale of 0 [Normal] to 4 [>99% confidence]) that the participant has motor symptoms that are unequivocal signs of manifest HD. If the rater did not feel with at least 99% confidence that the participant had manifest HD (i.e., score of 4) then they were rated as prodromal. Baseline data was used to differentiate prodromal verses manifest status. Finally, the Total Functional Capacity (TFC) scale was administered at baseline to determine HD staging for manifest participants. Specifically, participants with scores from 7 to 13 on the TFC were classified as early-stage HD (stages I and II), and those with scores less than 7 were classified as late-stage HD (stages III, IV & V) [49].

Statistical analysis

Statistical analysis was performed using SAS 9.4 software [50]. Reliability measures of the Cronbach’s α (minimal acceptable level ≥0.70) [51] were estimated for each of the HD-PRO-TRIAD^TM domains. Floor and ceiling effects defined by the percent of participants who answered either 1 (floor) or 5 (ceiling) were calculated for each domain and the total score of the HD-PRO-TRIAD^TM (minimal acceptable rates ≤ 20%) [52, 53]. Preliminary data analysis determined that distributions for each domain and the total HD-PRO-TRIAD^TM did not meet normality assumptions and therefore non-parametric analyses were used as appropriate. Convergent and discriminant validity were examined between each of the three HD-PRO-TRIAD^TM domains relative to the standardized clinician rated items and the already established patient reported outcome measures using Spearman-rho correlations. To support convergent validity, correlations between each HD-PRO-TRIAD^TM domain and its associated comparator (e.g., motor domain with other measures of motor function) should be moderate to high (i.e., 0.60–0.80), but should not exceed 0.80 (which would be indicative of too much overlap). Correlations between each HD-PRO-TRIAD^TM domain and the other comparator domains clinician-rated measures of function should be small to moderate (i.e., 0.30–0.59) [54]. As the items from the HD-PRO-TRIAD^TM were drawn from other existing measures, we excluded these measures from the reliability matrix used to evaluate convergent/discriminant validity (e.g., motor domain was not compared to HDQLIFE-Chorea, cognitive domain was not compared to Neuro-QoL-Executive Function).

Known groups validity was assessed using a one-way ANOVA (Kruskal-Wallis Test method) to examine whether each domain including the total HD-PRO-TRIAD^TM score could differentiate between prodromal, early, and late stage individuals. Prodromal participants should report better functioning than early stage participants, who should report better functioning than the late stage group. For domains with significant Kruskal-Wallis findings, Mann-Whitney U post-hoc tests were then conducted to determine where these differences were most prominent.

Next, measurement error was calculated for each domain of the HD-PRO-TRIAD^TM. This was done by calculating the Standard Error of Measurement (SEM), which uses the formula: $SEM = SD * \sqrt{1 - {Cronbach}^{'} s Alpha}$ [55, 56]; baseline standard deviations and Cronbach’s alphas were used in this calculation. The SEM is an index of absolute reliability that can be used to create a confidence interval around an individual’s observed score that estimates the true score for that individual [57]. Furthermore, the Minimal Detectable Change (MDC) was calculated as: $MDC = 1.96 * SEM * \sqrt{2}$ [58]. The MDC is a measure of the minimal change between assessments that is not due to variation in measurement [57 –59]. As suggested by Beckerman (2001), a 95% error band was calculated for change scores from baseline to 12-month follow-up to detect the amount of change that is real/relevant [58]. For easier interpretability, SEMs and MDCs are presented in percentages (SEM or MDC divided by the mean of all observations across assessments, times one-hundred) [59].

Finally, to examine responsiveness to change over time, participants were categorized into 2 groups using the EQ-5D Health Scale: those with significant health declines, and those with no changes or improvements. These classifications were made by subtracting the baseline EQ-5D Health Scale scores from either the 12-month or the 24-month scores, and then determining if the resulting change scores were ≥ 1SD below the mean for this sample (“health decline”) [60 –62]; sample means for the 12-month change scores were M = 0.36, SD = 15.54 and sample means for the 24-month change scores were M = 2.02, SD = 15.30. In order to be included in the 12-month “health decline” group, change scores must be ≥ 15.54. Similarly, in order to be included in the 24-month “health decline” group, change scores must be ≥ 15.30. HD-PRO-TRIAD^TM change scores for the group who reported health declines were compared to the group who reported no change/improvements using Mann-Whitney U tests. We hypothesized that participants who reported greater health declines on the EQ-5D would have higher HD-PRO-TRIAD^TM change scores (towards worse functioning) than those who stayed the same/improved.

Standardized Response Means (SRM) were then calculated by dividing the average change from baseline to follow-up (12-month and 24-month) and dividing by the standard deviation of the change for each of the HD-PRO-TRIAD^TM domains [63, 64]. Effect sizes between 0.00 and |0.19| were considered “negligible,” |0.20| to |0.49| were “small,” |0.50| to |0.79| were “medium,” and≥ |0.80| were “large” [63]. We hypothesized that participants who reported significant health declines would have SRM effect sizes < –0.20 and that those who reported staying the same/improving would have SRM effect sizes between –0.19 and 0.19 (i.e., negligible change).

RESULTS

Demographic data

Four hundred and eighty-two individuals with either prodromal (n = 192) or manifest HD (early stage n = 193 or late stage n = 97) completed enough items from the HD-PRO-TRIAD^TM to enable the calculations of a total score. Groups did not differ on gender (χ ² ₂ = 3.47, p = 0.17). Groups differed significantly in age (F[2, 479] = 43.97, p < 0.0001). The average age of the prodromal group ( $\bar{x}$ = 42.5) was nine years younger than the early group ( $\bar{x}$ s = 52.0) and twelve years younger than the late stage group ( $\bar{x}$ = 55.4). These age differences across groups were expected as HD is progressive [4]. The difference for race was also significant (χ ² ₆ = 30.59, p = 0.006); the late-HD group had a larger proportion of African Americans than either of the other two staging groups. Group did not differ significantly in ethnicity (χ ² ₄ = 6.35, p = 0.17). Education was significantly different (F[2, 463] = 15.85, p < 0.0001) among groups; the prodromal group had completed more years of education than either of the manifest groups. The late-stage group had significantly more CAG trinucleotide repeats than either of the other HD groups (F[2, 463] = 9.20; p = 0.0001). Full demographic information is provided in Table 1. A total of 118 participants (24.5%) completed all three study visits. Those who completed all three assessments did not differ on gender, race, ethnicity, or marital status. Not surprisingly, there were significantly fewer late-stage participants to complete all three assessments (χ ² ₂ = 18.04, p = 0.0001).

Table 1

Demographic Characteristics of Individuals with HD, stratified by stage of the disease

Variable	Prodromal-HD	Early-HD	Late-HD	Combined Sample
	(N = 192)	(N = 193)	(N = 97)	(N = 482)
Age (Years)
M (SD)	43.00 (12.10)	52.10 (12.20)	55.40 (11.70)	49.14 (13.12)
Gender (%)
Female	63.50	54.40	56.70	58.50
Male	36.50	45.60	43.30	41.50
Race (%)
White	97.40	97.40	92.80	96.10
African American	0.00	1.00	7.20	1.30
Other	2.10	1.60	0.00	1.40
Unknown	0.50	0.00	0.00	0.20
Ethnicity (%)
Not Hispanic or Latino	92.20	93.30	96.90	93.60
Hispanic or Latino	1.60	3.10	2.10	4.10
Not Provided	6.20	3.60	1.00	2.30
Education (# of years)
M (SD)	15.97 (2.81)	14.73 (2.78)	14.21 (2.51)	15.12 (2.83)
Marital Status (%)
Single, Never Married	16.10	14.60	11.30	14.60
Married	66.70	55.70	62.90	61.50
Separated/Divorced	14.10	22.40	22.70	19.10
Widowed	0.00	3.10	3.10	1.90
Living with Partner	3.10	4.20	0.00	2.90
CAG Repeats
M (SD)	42.18 (2.93)	42.93 (3.54)	44.72 (7.32)	42.78 (4.04)

Note. HD = Huntington disease.

Internal Consistency

The HD-PRO-TRIAD^TM and all three of the symptom domains and the total score showed excellent overall internal consistency (all Cronbach’s α >0.90; Table 2).

Table 2

Descriptive data for HD-PRO-TRIAD^TM

HD-PRO-TRIAD^TM Domain	N	# of Items	Cronbach’s α	% of Sample	% of sample	Median (IQR)
				with Floor	with Ceiling
				Effects	Effects
	BASELINE VISIT
Cognition	482	9	0.94	9.96	0.41	2.38 (1.83)
Emotional and Behavioral Dyscontrol	482	13	0.94	12.24	0.00	1.62 (1.08)
Motor	482	13	0.95	24.48	0.21	1.62 (1.38)
Total Score	482	35	0.96	2.07	0.00	5.64 (3.46)
	12-MONTH VISIT
Cognition	304	9	0.93	9.87	0.33	2.25 (1.81)
Emotional and Behavioral Dyscontrol	304	13	0.95	15.46	0.00	1.54 (1.08)
Motor	304	13	0.96	23.03	0.00	1.52 (1.32)
Total Score	304	35	0.96	3.95	0.00	5.75 (3.61)
	24-MONTH VISIT
Cognition	118	9	0.96	10.17	0.00	2.31 (1.88)
Emotional and Behavioral Dyscontrol	118	13	0.95	14.41	0.00	1.54 (1.08)
Motor	118	13	0.95	23.73	0.00	1.38 (1.08)
Total Score	118	35	0.97	2.54	0.00	5.25 (3.99)

Floor and ceiling effects

In general, floor and ceiling effects were acceptable for the three HD-PRO-TRIAD^TM domains (Table 2). The only exception was that the Motor domain exceeded the a priori cutoff (≤20%). Further examination indicated that the prodromal group was driving this floor effect (83% of the individuals exhibiting floor effects were in the prodromal group).

Convergent and discriminant validity

Table 3 highlights the Spearman-rho correlations among the HD-PRO-TRIAD^TM domains and total score and comparator measures. In general, the pattern of the correlations supported convergent and discriminant validity. Specifically, the HD-PRO-TRIAD^TM was moderately correlated (r≥0.45, with the exception of the RAND-12 MHC) with all clinician-rated measures and all PROMIS measures, and had the highest correlations with the general HRQOL measures. The Emotional/Behavioral Dyscontrol domain had the strongest relationships with other measures of mood, and less robust relationships with measures of cognition or motor functioning. Similar patterns were seen for the Motor domain. For the Cognitive domain, correlations were highest with the WHODAS and the Neuro-QoL measures of motor functioning, but were also strong between other measures of thinking and memory.

Table 3

Spearman Correlations between HD-PRO-TRIAD^TM and comparator measures

	MOOD				COGNITION			MOTOR			GENERAL
HD-PRO-	PROMIS	PROMIS	PROMIS	PBAs	Symbol	Stroop	Stroop	UHDRS	NeuroQoL	NeuroQoL	RAND-12	RAND-12	WHODAS	EQ5D^*
TRIAD^TM	Anger	Anxiety	Depression	Total	Digit	Color	Word	Motor	Lower	Upper	Physical	Mental
Domain				Score	Modalities	Naming^*	Reading^*	Sum	Extremity	Extremity	Health	Health
					Test^*				Function	Function	Score^*	Score^*
Cognition	0.38	0.43	0.44	0.46	0.47	0.43	0.42	0.48	0.69	0.64	0.56	0.33	0.79	0.62
Emotional	0.87	0.70	0.72	0.59	0.13^∧	0.13^∧	0.08^∧	0.09^∧	0.36	0.32	0.28	0.59	0.54	0.47
and Behavioral
Dyscontrol
Motor	0.26	0.30	0.32	0.40	0.63	0.58	0.58	0.68	0.77	0.75	0.67	0.16^∧	0.71	0.69
Total	0.55	0.54	0.56	0.57	0.52	0.48	0.46	0.53	0.75	0.70	0.62	0.39	0.85	0.72

Note. PBAs – short Problem Behaviors Assessment; WHODAS = World Health Organization Disability Assessment Scale; EQ5D = EuroQol5D; All correlations are p < 0.05 (2tailed) unless otherwise noted; ^∧ = p > 0.05; ^* = RAND-12 Physical and Mental Health Scales, Symbol Digit Modalities Test, Stroop Tests, EQ5D Index Scale, and NeuroQOL measures of mobility are reverse scored, where higher scores indicate better functioning. Negative signs were removed for consistency of presentation and to emphasize the strength of the relationship between variables.

Known groups validity

Known groups validity was supported for the total score of the HD-PRO-TRIAD^TM as well as each of the three domains (Table 4). Each of these domains was able to differentiate between the three HD groups. The exception to this was for the Emotional and Behavioral Dyscontrol domain, the measure was unable to differentiate between early and late-stage participants.

Table 4

Known groups validity for the HD-PRO-TRIAD^TM

HD-PRO-TRIAD^TM Domain	Prodromal HD (n = 192)	Early-HD (n = 193)	Late-HD (n = 97)	χ ²	p-value
	Median (Q1^, Q3^)	Median (Q1^, Q3^)	Median (Q1^, Q3^)
Cognition^a,b,c	1.63 (1.13, 2.38)	2.50 (1.88, 3.38)	3.50 (2.63, 4.25)	120.66	<0.0001
Emotional and Behavioral Dyscontrol^a,b	1.46 (1.15, 1.92)	1.69 (1.23, 2.46)	1.85 (1.23, 2.54)	10.00	0.0007
Motor^a,b,c	1.00 (1.00, 1.17)	2.00 (1.31, 2.62)	2.80 (2.00, 3.46)	222.32	<0.0001
Total^a,b,c	4.17 (3.52, 5.55)	6.26 (5.03, 7.69)	8.36 (6.78, 9.42)	145.97	<0.0001

^*Q1: Quantile 1; Q3: Quantile 3. 50% of data falls between this range. ^aIndicates significant differences between prodromal and early – HD. ^bIndicates significant differences between prodromal and late-HD. ^cIndicates significant differences between early-HD and late-HD.

Measurement error

All SEM% values were 11.19% or lower, indicating that the difference between observed scores and true scores is minimal (Table 5). The 95% MDC error band suggests that for each domain, less than a one-integer change in scores is sufficient in detecting real change. For the total HD-PRO-TRIAD^TM, less than a two-integer change in scores implies real change not due to variation in measurement.

Table 5

Measurement Error: Standard Error of Measurement (SEM) and Minimal Detectable Change (MDC)

HD-PRO-TRIAD Domain	d (95% CI)^*	SEM	SEM %	95% MDC^**	MDC %
Cognition	–0.02 (–1.29, 1.26)	0.27	10.92	–0.76–0.72	30.27
Emotional and Behavioral Dyscontrol	–0.02 (–1.23, 1.19)	0.18	10.04	–0.52–0.48	27.83
Motor	–0.04 (–1.03, 0.95)	0.21	11.19	–0.61–0.53	31.01
Total Score	–0.08 (–2.56, 2.40)	0.46	7.62	–1.36–1.20	21.12

Note: SEM = Standard error of measurement; MDC = Minimal Detectable Change; CI = Confidence Interval. *d: Average change score. From Baseline to 12-month. ^**95% MDC reflects error band around the change from baseline to 12-month visits.

Responsiveness to change over time

Table 6 highlights 12- and 24-month responsiveness data. From baseline to 12-months, scores for all HD-PRO-TRIAD^TM domains (except emotional and behavioral dyscontrol) and the total score were worse for participants who had significant declines in health; these effect sizes were small (except for emotional and behavioral dyscontrol which was negligible). There were no significant differences in 12-month change scores between participants who reported worsened health and those who reported same/improved health. From baseline to 24-months, scores for all HD-PRO-TRIAD^TM domains and the total score were worse for participants who had significant declines in health (again determined by EQ-5D Health Scale scores); all effect sizes were moderate. There were significant 24-month differences between those with health declines and those with no changes/improvements for all HD-PRO-TRIAD^TM domains (except motor) and the total score.

Table 6

Standardized Response Means for responsiveness (self-report change between assessments)

	Baseline to 12 Months				Baseline to 24 Months
HD-PRO-TRIAD^TM Domain	Sample	Significant	Same/Improved	P-value^*	Sample	Significant	Same/Improved	P-value^*
	Size	Health Decline	(n = 254)		Size	health Decline	(n = 104)
		(n = 42)				(n = 14)
Cognitive	296	–0.22	0.00	0.18	118	–0.72	–0.06	0.03
Emotional/Behavioral Dyscontrol	296	0.03	–0.04	0.93	118	–0.62	0.04	0.01
Motor	296	–0.27	–0.04	0.32	118	–0.65	–0.16	0.10
Total	296	–0.21	–0.03	0.40	118	–0.75	–0.06	0.01

^*P-value based on results of the Mann-Whitney U test comparing change scores between the 2 groups; bolding for p-values indicates significant group differences (i.e., p < 0.05).

DISCUSSION

The sensitive evaluation of PROs in HD has been limited by a lack of HD-specific measures of health-related quality of life. To address this need, the HD-PRO-TRIAD^TM was developed to evaluate the triad of symptoms characteristic of HD (cognition, emotional/behavioral dyscontrol, and motor function). Findings support the clinical utility of the HD-PRO-TRIAD^TM as a valid measure of HRQOL in individuals with HD.

Findings replicate previous work on the HD-PRO-TRIAD^TM as internal consistency was excellent and convergent/discriminant validity was supported. The exception to this was in regards to the discriminant validity of the Cognition domain, for which the highest correlations were with physical functioning measures. One possible explanation for this finding is that motor and cognitive declines occur at similar rates, or as an alternative, it is possible that individuals with HD have difficulty distinguishing motor difficulties form cognitive difficulties, as these two things are interrelated and can go hand in hand. On the other hand, the moderate relationships with the emotional functioning measures are consistent with previous findings on the HD-PRO-TRIAD^TM, and is not especially surprising given that self-reported cognition generally exhibits moderate relationships with measures of mood and objective measures of cognition, and sometimes reflects overall distress rather than objective cognitive dysfunction [65 –72]. Taken together, findings were generally consistent with our proposed hypotheses and suggest that the HD-PRO-TRIADTM is a valid measure of HRQOL for people with HD.

The current study also expands upon previous literature by examining floor effects, known groups validity, and responsiveness of the HD-PRO-TRIAD^TM. The total score for the HD-PRO-TRIADTM was free of both ceiling and floor effects. Furthermore, the domain scores were also free of floor and ceiling effects (with the exception of the Motor domain, which had some evidence for a floor effect secondary to a lack of motor manifestation in prodromal HD participants). Such findings are consistent with other measures of HRQOL in HD. Specifically, measures that are comprised of multiple HRQOL domains, typically do not exhibit floor or ceiling effects, but when the focus is on a single domain, there are often floor effects for prodromal participants (which is not surprising given that these measures are typically designed to evaluate dysfunction, and these individuals are not yet exhibiting problems in these areas) [73, 74].

In addition, known groups validity for the HD-PRO-TRIAD^TM scores was also supported. Specifically, individuals with late-HD reported worse functioning than those with early- or prodromal HD, and individuals with early-HD reported worse functioning that individuals with prodromal HD across nearly all domain and the total scale of the HD-PRO-TRIADTM. The only exception was a lack of difference between early-and late-HD for the Emotional and Behavioral Dyscontrol domain. These findings generally provide empirical support for the construct validity of HD-PRO-TRIAD^TM.

Additionally, measurement error and minimal detectable change scores were small, and generally were less than a 1-point difference for each domain and less than 2-points for the HD-PRO-TRIAD^TM total score. Therefore, clinicians and researchers can be confident that a participant’s score will reflect their true score, and that change scores larger than those in Table 4 reflect real change rather than error in measurement.

Finally, the HD-PRO-TRIAD^TM was also found to have acceptable responsiveness to self-reported change over time. There were small 12-month effect sizes for all HD-PRO-TRIAD^TM domains (except emotional and behavioral dyscontrol), and moderate 24-month effect sizes for all domains and the total score. The fact that effect sizes were evident and based solely on change over time (i.e., expected declines from disease progression) and not on changes due to clinical interventions, makes this finding especially promising. In addition, while there were not significant differences between the group with significant health declines and those with no change/improvements at 12-months, there were significant group differences in change at 24-months which provides additional support for responsiveness. Specifically, while the absence of significant group differences at 12-months may raise concerns regarding the sensitivity of the measure, previous research indicates that non-significant group differences do not necessarily indicate a trivial finding [75 –78]. Furthermore, the significant group differences at 24-months (despite the small sample size) mitigates these concerns. Thus, findings support responsiveness to change; findings that we anticipate will be more robust given a larger sample size and/or the introduction of an effective clinical intervention.

While this study provides important reliability and validity data for the HD-PRO-TRIAD^TM, we also recognize several limitations. First, as described above, we did not administer all of the items of the HD-PRO-TRIAD^TM and therefore, findings using the full form, as intended, may not be fully comparable. Additionally, test-retest reliability was not examined in this study and further work is needed to provide this information. Also, as the measurement error and minimal detectable change was calculated from just one aspect of the reliability coefficient (the Cronbach’s alpha), there may be minor variation in these findings should this study be replicated. Furthermore, the retention rates for the 24-month sample size were not ideal and dropout rates, especially for those with more severe HD may have influenced reported longitudinal findings. Finally, given that this sample was primarily female and Caucasian, generalizability to other groups (i.e., men, as well as racial and ethnic minorities with HD) may be less robust.

Despite these limitations, this study provides additional support for the reliability and validity of HD-PRO-TRIAD^TM. Findings supporting internal consistency and convergent and discriminant validity replicated previous published work. New analyses from this study, demonstrating the lack floor and ceiling effects, as well as analyses supporting known-groups validity, minimal measurement error, and responsiveness of the HD-PRO-TRIAD^TM, provide additional support for the clinical utility of this measure. Ultimately, findings suggest that the HD-PRO-TRIAD^TM is a reliable and valid assessment tool for evaluating HRQOL individuals with prodromal and manifest HD.

CONFLICTS OF INTEREST

N.R. Boileau declares no conflicts of interest.

J.C. Stout has received research funding in the past three years from the Australian National Health and Medical Research Council, University College London, the CHDI Foundation, Prana Biotechnology, and the University of California, Davis. She is a Director of Stout Neuropsych Pty Ltd, which has received funding from Omeros, Teva Pharmaceuticals, Vaccinex, and Isis. She has been a consultant to Prana Biotechnology and Roche. She receives compensation as a member of the Board of the Huntington’s Study Group.

J.S. Paulsen currently has research grants from the NIH; she is also supported by grant funding from NIH, NINDS, and CHDI; she declares no conflicts of interest.

D. Cella receives grant funding from the National Institutes of Health and reports that he has no conflicts of interest.

M.K. McCormack currently has grants from the NJ Department of Health; he declares no conflicts of interest.

M.A. Nance declares no conflicts of interest.

S. Frank receives salary support from the Huntington Study Group for a study sponsored by Auspex Pharmaceuticals he declares no conflicts of interest.

J.-S. Lai currently has research grants from the NIH; she declares no conflicts of interest.

N.E. Carlozzi currently has research grants from the NIH; she is also supported by grant funding from the NIH, NIDILRR, and CHDI; she declares no conflicts of interest.

Footnotes

Appendix A.

Questions from original HD-PRO-TRIAD^TM paper administered for the purposed of this study.

HD-PRO-TRIAD^TM Domains

Cognitive: Rated from 1 (Very often; Cannot Do) to 5 (Never; None)

In the past 7 days...I had trouble keeping track of what I was doing if I was interrupted

In the past 7 days...I had trouble concentrating

In the past 7 days...I had difficulty doing more than one things at a time

In the past 7 days...I had trouble planning out steps of a task

In the past 7 days...I had trouble remembering new information, like phone numbers or simple instructions

How much difficulty do you currently have... getting things organized?

How much difficulty do you currently have... keeping important personal papers such as bills, insurance documents and tax?

How much difficulty do you currently have... learning new tasks or instructions?

How much difficulty do you currently have... remembering a list of 4 or 5 errands without writing it down?

Emotional/Behavioral Dysfuntion: Rated from 1 (Never) to 5 (Always)

In the past 7 days...It was hard to control my behavior

In the past 7 days...I was irritable around other people

In the past 7 days...I was bothered by little things

In the past 7 days...I became easily upset

In the past 7 days...I said or did things without thinking

In the past 7 days...I got impatient with other people

In the past 7 days...I felt impulsive

In the past 7 days...It was hard to adjust to unexpected changes

In the past 7 days...I was in conflict with others

In the past 7 days...I said or did things that other people probably thought were inappropriate

In the past 7 days...I threatened violence toward people or property

In the past 7 days...I felt angry

In the past 7 days... I had trouble controlling my temper

Motor Function: Rated from 1 (Never) to 5 (Always)

In the past 7 days...How often did you feel unsteady when you were standing?

In the past 7 days...How often did you have movements (e.g., chorea)?

In the past 7 days...How often were you unable to stay still?

In the past 7 days...How often did your movements (e.g., chorea) impact your ability to hold things, like a glass or for

In the past 7 days...How often did you experience severe movements (e.g., chorea)?

How much difficulty do you currently have...speaking clearly?

In the past 7 days...How often did you have to speak slowly for other people to understand you?

In the past 7 days...How often did choking interfere with your ability to eat?

In the past 7 days...How often were you bothered by your choking?

In the past 7 days...How often were you unable to maintain a conversation?

In the past 7 days...How often were you unable to swallow?

In the past 7 days...How often did you have shakiness?

In the past 7 days...I needed help doing my usual activities.

ACKNOWLEDGMENTS

Work on this manuscript was supported by the National Institutes of Health (NIH), National Institute of Neurological Disorders and Stroke (R01NS077946) and the National Center for Advancing Translational Sciences (UL1TR000433). In addition, a portion of this study sample was collected in conjunction with the Predict-HD study. The Predict-HD study was supported by the NIH, National Institute of Neurological Disorders and Stroke (R01NS040068), the NIH, Center for Inherited Disease Research (provided supported for sample phenotyping), and the CHDI Foundation (award to the University of Iowa). We thank the University of Iowa, the Investigators and Coordinators of this study, the study participants, the National Research Roster for Huntington Disease Patients and Families, the Huntington Study Group, and the Huntington’s Disease Society of America. We acknowledge the assistance of Jeffrey D. Long, Hans J. Johnson, Jeremy H. Bockholt, and Roland Zschiegner. We also acknowledge Roger Albin, Kelvin Chou, and Henry Paulsen for the assistance with participant recruitment. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

HDQLIFE Site Investigators and Coordinators: Noelle Carlozzi, Praveen Dayalu, Stephen Schilling, Amy Austin, Matthew Canter, Siera Goodnight, Jennifer Miner, Nicholas Migliore (University of Michigan, Ann Arbor, MI); Jane Paulsen, Nancy Downing, Isabella DeSoriano, Courtney Shadrick, Amanda Miller (University of Iowa, Iowa City, IA); Kimberly Quaid, Melissa Wesson (Indiana University, Indianapolis, IN); Christopher Ross, Gregory Churchill, Mary Jane Ong (Johns Hopkins University, Baltimore, MD); Susan Perlman, Brian Clemente, Aaron Fisher, Gloria Obialisi, Michael Rosco (University of California Los Angeles, Los Angeles, CA); Michael McCormack, Humberto Marin, Allison Dicke, Judy Rokeach (Rutgers University, Piscataway, NJ); Joel Perlmutter, Stacey Barton, Shineeka Smith (Washington University, St. Louis, MO); Martha Nance, Pat Ede (Struthers Parkinson’s Center); Stephen Rao, Anwar Ahmed, Michael Lengen, Lyla Mourany, Christine Reece, (Cleveland Clinic Foundation, Cleveland, OH); Michael Geschwind, Joseph Winer (University of California – San Francisco, San Francisco, CA), David Cella, Richard Gershon, Elizabeth Hahn, Jin-Shei Lai (Northwestern University, Chicago, IL).

References

Pringsheim

, Wiltshire

, Day

, Dykeman

, Steeves

, Jette

. The incidence and prevalence of Huntington’s disease: A systematic review and meta-analysis. Mov Disord. 2012;27(9):1083–91. doi: 10.1002/mds.25075

Fisher

, Hayden

. Multisource ascertainment of Huntington disease in Canada: Prevalence and population at risk. Mov Disord. 2014;29(1):105–14. doi: 10.1002/mds.25717

Nance

, Seltzer

, Ashizawa

, Bennett

, McIntosh

, Myers

, et al. Laboratory guidelines for Huntington disease genetic testing. Am J Hum Genet. 1998;62(5):1243–7. doi: 10.1086/301846

Paulson

, Albin

. “Huntington’s disease: Clinical features and routes to therapy”. In: Lo

, Hughes

, editors. Neurobiology of Huntington’s disease: Applications to drug discovery Frontiers in Neuroscience. Boca Raton: CRC Press; 2011. p. PMID 21882418.

Singer

. Comprehensive treatment of Huntington disease and other choreic disorders. Cleve Clin J. 2012;79:S30–S4. doi: 10.3949/ccjm.79.s2a.06

Duff

, Paulsen

, Beglinger

, Langbehn

, Stout

, Predict HDIotHSG. Psychiatric symptoms in Huntington’s disease before diagnosis: The predict-HD study. Biol Psychiatry. 2007;62(12):1341–6. doi: 10.1016/j.biopsych.2006.11.034

Duff

, Paulsen

, Mills

, Beglinger

, Moser

, Smith

, et al. Mild cognitive impairment in prediagnosed Huntington disease. Neurology. 2010;75(6):500–7. doi: 10.1212/WNL.0b013e3181eccfa2

Roos

. Huntington’s disease: A clinical review. Orphanet J Rare Dis. 2010;5(1):40. doi: 10.1186/1750-1172-5-40

Long

, Paulsen

, Marder

, Zhang

, Kim

, Mills

, et al. Tracking motor impairments in the progression of Huntington’s disease. Mov Disord. 2014;29(3):311–9. doi: 10.1002/mds.25657

10.

Vaccarino

, Sills

, Anderson

, Borowsky

, Craufurd

, Giuliano

, et al. Assessment of cognitive symptoms in prodromal and early Huntington disease. PLoS Curr. 2011;3:RRN1250. doi: 10.1371/currents.RRN1250

11.

Peavy

, Jacobson

, Goldstein

, Hamilton

, Kane

, Gamst

, et al. Cognitive and functional decline in Huntington’s disease: Dementia criteria revisited. Mov Disord. 2010;25(9):1163–9. doi: 10.1002/mds.22953

12.

Kirkwood

, Siemers

, Bond

, Comically

, Christian

, Foroud

. Confirmation of subtle motor changes among presymptomatic carriers of the huntington disease gene. Arch Neurol. 2000;57(7):1040–4.

13.

Cohen

, Biesecker

. Quality of life in rare genetic conditions: A systematic review of the literature. Am J Med Genet. 2010;152(5):1136–56. doi: 10.1002/ajmg.a.33380

14.

Carlozzi

, Tulsky

. Identification of Health-Related Quality of Life (HRQOL) issues relevant to individuals with HD. J Health Psychol. 2013;18(2):212–25. doi: 10.1177/1359105312438109

15.

Victorson

, Carlozzi

, Frank

, Beaumont

, Cheng

, Gorin

, et al. Identifying motor, emotional-behavioral, and cognitive deficits that comprise the triad of HD symptoms from patient, caregiver, and provider perspectives. Tremor Other Hyperkinet Mov (N Y). 2014;4:224. doi: 10.7916/D8JW8BWS

16.

Carlozzi

, Schilling

, Lai

J-S

, Paulsen

, Hahn

, Perlmutter

, et al. HDQLIFE: Development and assessment of health-related quality of life in Huntington disease (HD). Qual Life Res. 2016;25(10):212–25. doi: 10.1007/s11136-016-1386-3

17.

Carlozzi

, Downing

, McCormack

, Schilling

, Perlmutter

, Hahn

, et al. New measures to capture end of life concerns in Huntington disease: Meaning and purpose and concern with death and dying from HDQLIFE (a patient-reported outcomes measurement system). Qual Life Res. 2016. doi: 10.1007/s11136-016-1354-y

18.

Carlozzi

, Downing

, Schilling

, Lai

, Goodnight

, Miner

, et al. The development of a new computer adaptive test to evaluate chorea in Huntington disease: HDQLIFE Chorea. Qual Life Res. 2016. doi: 10.1007/s11136-016-1307-5

19.

Carlozzi

, Schilling

, Lai

, Perlmutter

, Nance

, Waljee

, et al. HDQLIFE: The development of two new computer adaptive tests for use in Huntington disease, Speech Difficulties, and Swallowing Difficulties. Qual Life Res. 2016. doi: 10.1007/s11136-016-1273-y

20.

Hocaoglu

, Gaffan

, Ho

. The Huntington’s Disease health-related Quality of Life questionnaire (HDQoL): A disease-specific measure of health-related quality of life. Clin Genet. 2012;81(2):117–22. 10.1111/j.1399-0004.2011.01823.x

21.

Clay

, De Nicola

, Dorey

, Squitieri

, Aballea

, Martino

, et al. Validation of the first quality-of-life measurement for patients with Huntington’s disease: The Huntington Quality of Life Instrument. Int Clin Psychopharmacol. 2012;27(4):208–14. doi: 10.1097/YIC.0b013e3283534fa9

22.

Tabrizi

, Reilmann

, Roos

, Durr

, Leavitt

, Owen

, et al. Potential endpoints for clinical trials in premanifest and early Huntington’s disease in the TRACK-HD study: Analysis of 24 month observational data. Lancet Neurol. 2012;11(1):42–53. doi: 10.1016/s1474-4422(11)70263-0

23.

Tabrizi

, Scahill

, Durr

, Roos

, Leavitt

, Jones

, et al. Biological and clinical changes in premanifest and early stage Huntington’s disease in the TRACK-HD study: The 12-month longitudinal analysis. Lancet Neurol. 2011;10(1):31–42. doi: 10.1016/S1474-4422(10)70276-3

24.

Khalil

, Quinn

, van Deursen

, Dawes

, Playle

, Rosser

, et al. What effect does a structured home-based exercise programme have on people with Huntington’s disease? A randomized, controlled pilot study. Clin Rehabil. 2013;27(7):646–58. doi: 10.1177/0269215512473762

25.

Carlozzi

, Victorson

, Sung

, Beaumont

, Cheng

, Gorin

, et al. HD-PRO-TRIAD validation: A patient-reported instrument for the symptom triad of Huntington’s disease. Tremor Other Hyperkinet Mov (N Y). 2014;4:223. doi: 10.7916/D8PN93NZ

26.

Cella

, Nowinski

, Peterman

, Victorson

, Miller

, Lai

J-S

, et al. The Neurology Quality of Life Measurement (Neuro-QOL) Initiative. Arch Phys Med Rehabil. 2011;92(Suppl 1):S28–S36. doi: 10.1016/j.apmr.2011.01.025

27.

Tulsky

, Kisala

, Victorson

, Carlozzi

, Bushnik

, Sherer

, et al. TBI-QOL: Development and calibration of item banks to measure patient reported outcomes following traumatic brain injury. J Head Trauma Rehabil. 2016;31(1):40–51. doi: 10.1097/HTR.0000000000000131

28.

Webster

, Cella

, Yost

. The Functional Assessment of Chronic Illness Therapy (FACIT) measurement system: Properties, applications, and interpretation. Health Qual Life Outcomes. 2003;1(79). doi: 10.1186/1477-7525-1-79

29.

Paulsen

, Langbehn

, Stout

, Aylward

, Ross

, Nance

, et al. Detection of Huntington’s disease decades before diagnosis: The Predict-HD study. J Neurol Neurosurg Psychiatry. 2008;79(8):874–80. doi: 10.1136/jnn2007.128728

30.

Jackson

, Novack

. Effective serial measurement of cognitive orientation in rehabilitation: The Orientation Log. Arch Phys Med Rehabil. 1998;79(6):718–20.

31.

Hanauer

, Mei

, Law

, Khanna

, Zheng

. Supporting information retrieval from electronic health records: A report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE). J Biomed Inform. 2015;55:290–300. doi: 10.1016/j.jbi.2015.05.003

32.

Bjorner

, Rose

, Gandek

, Stone

, Junghaenel

, Ware

. Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. J Clin Epidemiol. 2014;67(1):108–13. doi: 10.1016/j.jclinepi.2013.07.016

33.

Magnus

, Liu

, He

, Quinn

, Thissen

, Gross

, et al. Mode effects between computer self-administration and telephone interviewer-administration of the PROMIS((R)) pediatric measures, self- and proxy report. Qual Life Res. 2016;25(7):1655–65. doi: 10.1007/s11136-015-1221-2

34.

Gershon

, Rothrock

, Hanrahan

, Bass

, Cella

. The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J Appl Meas. 2010;11(3):304–14.

35.

Carlozzi

, Tulsky

, Kisala

. Traumatic brain injury patient-reported outcome measure: Identification of health-related quality-of-life issues relevant to individuals with traumatic brain injury. Arch Phys Med Rehabil. 2011;92(10):S52–S60. doi: 10.1016/j.apmr.2010.12.046

36.

Tulsky

, Kisala

, Victorson

, Carlozzi

, Cella

. Development of tailored outcomes measures for traumatic brain injury and spinal cord injury. Neurology. 2009;72(11):A495-A.

37.

Gershon

, Lai

, Bode

, Choi

, Moy

, Bleck

, et al. Neuro-QOL: Quality of life item banks for adults with neurological disorders: Item development and calibrations based upon clinical and general population testing. Qual Life Res. 2012;21(3):475–86. 10.1007/s11136-011-9958-8

38.

Cella

, Lai

, Nowinski

, Victorson

, Peterman

, Miller

, et al. Neuro-QOL Brief measures of health-related quality of life for clinical research in neurology. Neurology. 2012;78(23):1860–7. doi: 10.1212/WNL.0b013e318258f744

39.

Cella

, Nowinski

, Peterman

, Victorson

, Miller

, Lai

, et al. The neurology quality-of-life measurement initiative. Arch Phys Med Rehabil. 2011;92(10 Suppl):S28–36. S0003-9993(11)00438-2 [pii] 10.1016/j.apmr.2011.01.025

40.

Gershon

, Lai

, Bode

, Choi

, Moy

, Bleck

41.

Cella

, Riley

, Stone

, Rothrock

, Reeve

, Yount

, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested in its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63:1179–94.

42.

Cella

, Yount

, Rothrock

, Gershon

, Cook

, Reeve

, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3–S11. doi: 10.1097/01.mlr.0000258615.42478.55

43.

Rabin

, de Charro

. EQ-5D: A measure of health status from the EuroQol Group. Ann Med. 2001;33(5):337–43.

44.

Ustun

, Chatterji

, Kostanjsek

, Rehm

, Kennedy

, Epping-Jordan

, et al. Developing the world health organization disability assessment schedule 2.0. Bull World Health Organ. 2010;88(11):815–23. doi: 10.2471/BLT.09.067231

45.

Ware

, Kosinski

, Turner-Bowker

, Gendek

. How to score version 2 of the SF-12 health survey (with a supplement documenting version 1); 2002.

46.

Unified Huntington’s Disease Rating Scale: Reliability and consistency. Huntington Study Group. Mov Disord. 1996;11(2):136–42. doi: 10.1002/mds.870110204

47.

Koss

, Ober

, Delis

, Friedland

. The Stroop color-word test: Indicator of dementia severity. Int J Neurosci. 1984;24(1):53–61.

48.

Smith

. Symbol digit modalities test: Manual. Los Angeles: Western Psychological Services; 1982.

49.

Marder

, Zhao

, Myers

, Cudkowicz

, Kayson

, Kieburtz

, et al. Rate of functional decline in Huntington’s disease. Huntington Study Group. Neurology. 2000;54(2):452–8.

50.

Institute S. SAS 9.4 language reference concepts. Cary, NC: SAS Institute; 2013.

51.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

52.

Cramer

, Howitt

. The Sage dictionary of statistics. Thousand Oaks, CA: Sage; 2004.

53.

Andresen

. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil. 2000;81(12 Suppl 2):S15–20. doi: 10.1053/apmr.2000.20619

54.

Campbell

, Fiske

. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56(2):81–105.

55.

Cohen

, Swerdlik

. Psychological testing and assessment. Burr Ridge, IL: McGraw-Hill; 2010.

56.

Tavakol

, Dennick

. Post-examination analysis of objective tests. Med Teacher. 2011;33(6):447–58. doi: 10.3109/0142159X.2011.564682

57.

Weir

. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231–40. doi: 10.1519/15184.1

58.

Beckerman

, Roebroeck

, Lankhorst

, Becher

, Bezemer

, Verbeek

. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10:571–8. doi: 10.1023/A:1013138911638

59.

Flansbjer

, Holmback

, Downham

, Patten

, Lexell

. Reliability of gait performance tests in men and women with hemiparesis after stroke. J Rehabil Med. 2005;37:75–82. doi: 10.1080/16501970410017215

60.

Goodwin

. Focus on quantitative methods: Determining cut-off scores. Res Nurs Health. 1996;19:249–56. doi: 10.1002/(SICI)1098-240X(199606)19:3<249::AID-NUR8>3.0.CO;2-K

61.

Flynn

, Reese

. Development and evaluation of classroom tests: A practical application. J Nurs Educ. 1988;27:61–5. doi: 10.3928/0148-4834-19880201-05

62.

Gray

, Payne

. An innovative method for evaluating means to award placement credit in a baccalaureate nursing program for RN students. J Nurs Educ. 1991;30:326–7. doi: 10.3928/0148-4834-19910901-09

63.

Cohen

. A power primer. Psychol Bull. 1992;112(1):155–9.

64.

Kopjar

. The SF-36 health survey: A valid measure of changes in health status after injury. Inj Prev. 1996;2:135–9.

65.

Derouesne

, Lacomblez

, Thibault

, LePoncin

. Memory complaints in young and elderly subjects. Int J Geriatr Psychiatry. 1999;14(4):291–301.

66.

Mostafaie

, Rossmanith

, Hombauer

, Dechat

, Raffelsberger

, Bauer

, et al. Mitochondrial genotype and risk for Alzheimer’s disease: Cross-sectional data from the Vienna-Transdanube-Aging “VITA” study. J Neural Transm (Vienna). 2004;111(9):1155–65. doi: 10.1007/s00702-004-0161-8

67.

Lautenschlager

, Flicker

, Vasikaran

, Leedman

, Almeida

. Subjective memory complaints with and without objective memory impairment: Relationship with risk factors for dementia. Am J Geriatr Psychiatry. 2005;13(8):731–4. doi: 10.1176/appi.ajg13.8.731

68.

Sawrie

, Martin

, Kuzniecky

, Faught

, Morawetz

, Jamil

, et al. Subjective versus objective memory change after temporal lobe epilepsy surgery. Neurology. 1999;53(7):1511–7.

69.

Schmidt

, Berg

, Deelman

. Relations between subjective evaluations of memory and objective memory performance. Percept Mot Skills. 2001;93(3):761–76. doi: 10.2466/pms.2001.93.3.761

70.

Smith

, Petersen

, Ivnik

, Malec

, Tangalos

. Subjective memory complaints, psychological distress, and longitudinal change in objective memory performance. Psychol Aging. 1996;11(2):272–9.

71.

Zandi

. Relationship between subjective memory complaints, objective memory performance, and depression among older adults. Am J Alzheimers Dis Other Demen. 2004;19(6):353–60. doi: 10.1177/153331750401900610

72.

Carlozzi

, Grech

, Tulsky

. Memory functioning in individuals with traumatic brain injury: An examination of the Wechsler Memory Scale-Fourth Edition (WMS-IV). J Clin Exp Neuropsychol. 2013;35(9):906–14. doi: 10.1080/13803395.2013.833178

73.

Nance

. Comprehensive care in Huntington’s disease: A physician’s perspective. Brain Res Bull. 2007;72(2-3):175–8. doi: 10.1016/j.brainresbull.2006.10.027

74.

Paulsen

, Wang

, Duff

, Barker

, Nance

, Beglinger

, et al. Challenges assessing clinical endpoints in early Huntington disease. Mov Disord. 2010;25(15):2595–603. doi: 10.1002/mds.23337

75.

Cohen

. The earth is round (p<0.05). Am Psychol. 1994;49(12):997–1003.

76.

Rosenthal

, Rubin

. The Counternull value of an effect size: A new statistic. Psychol Sci. 1994;5(6):329–34.

77.

Bartko

, Pulver

, Carpenter

. The power of analysis: Statistical perspectives. Part 2. Psychiatry Res. 1988;23:301–9. doi: 10.1016/0165-1781(88)90021-2

78.

Middel

, van Sonderen

. Statistical significant change versus relevant or important change in (quasi) experimental design: Some conceptual and methodological problems in estimating magnitude of intervention-related change in health services research. Int J Integr Care. 2002;2(e15).

Reliability and Validity of the HD-PRO-Triad TM,a Health-Related Quality of Life Measure Designed to Assess the Symptom Triad of Huntington’s Disease

Abstract

Background:

Objectives:

Methods:

Results:

Conclusions:

Keywords

INTRODUCTION

MATERIALS AND METHODS

Participant visits

Measures

Demographic variables

Medical record confirmation

HD-PRO-TRIADTM items [25]

Quality of life in neurological disorders (Neuro-QoL) item banks [38, 39]

Patient Reported Outcomes Measurement Information System (PROMIS) item banks [41, 42]

EQ-5D [43]

World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0.) [44]

RAND-12 [45]

Clinician rated measures

Statistical analysis

RESULTS

Demographic data

Internal Consistency

Floor and ceiling effects

Convergent and discriminant validity

Known groups validity

Measurement error

Responsiveness to change over time

DISCUSSION

CONFLICTS OF INTEREST

Footnotes

Appendix A.

ACKNOWLEDGMENTS

References

HD-PRO-TRIAD^TM items [25]