Nurturing Longitudinal Samples 2.0

Description of Longitudinal Studies

Longitudinal designs provide an excellent opportunity to add important knowledge and details to how persons adapt and live with chronic and disabling health conditions over time. The 2 studies described below have provided important insights to understanding the impact of health promotion as a self-management strategy for persons with MS and cognitive outcomes of persons treated for breast cancer.

Structure of Longitudinal Studies

Caruana and colleagues ¹⁰ note the need for longitudinal studies to have an appropriately robust infrastructure that will withstand the test of time. Longitudinal studies require consistency of methods in data collection necessitating careful and continuous training and supervision of staff (See Table 1). We provided education and training to research staff on the detailed protocol, the importance of enrolling and retaining the cohort for a longitudinal study, and how these data could contribute to valuable new understanding. The study protocol included careful tracking and follow-up of participants to maintain addresses and send study reminders. In the MS study, every participant was assigned a unique research ID number at the initiation of the study. While individual addresses were used to mail the survey, the survey itself had only the participant number and data are thus de-identified when the surveys are returned and entered. The unique patient ID allows the matching of responses across data collection periods. Both studies described above received National Institutes of Health (NIH) funding at their initiation, which helped support the establishment of appropriate research infrastructure for a longitudinal study.

OPEN IN VIEWER

Table 1.

Key Points for Project Management of a Longitudinal Study.

• Develop strong study protocols and infrastructure to support consistent tracking of participants, response rates, and data management. • Establish a toll-free telephone number, study email and website to support communication with participants. • Train staff to establish and maintain rapport with participants over time. • Establish strong protections for privacy of participants’ identity and data including staff access to data stored in the cloud. • Explore technology and digital methods to decrease burden in data collection, data entry and data management. • Carefully develop variable names to allow linking of responses over time • Work to minimize the direct and indirect costs of study participation. • Listen to participants and provide prompt and personal responses. • Develop a plan for interim data analysis and publications during the longitudinal study.

It is important that both participants and staff can strongly identify with the longitudinal study over time. Thus, the development and consistent use of a study name and logo is essential. Branding of the study with an institution or organization where the study is housed can also be helpful in establishing study credibility and connecting participants to a broader infrastructure. All study mailings from recruitment to data collection to follow-ups need to be well designed with large easy to read fonts so communications are not discarded as “junk mail” or spam for electronic messages. All communications included clear information about how to easily contact study staff including a study specific email and toll-free number. Participants’ questions and comments to the staff received prompt and personal responses. We also partnered with established patient advocacy groups (eg, Army of Women, National MS Society, local support groups) to promote the study and have greater credibility with research participants. Building rapport and trust with participants can be challenging when in-person interactions are limited.

At the initiation of the MS study, we established a digital database using commercially available software to track response rates, change of address, and other relevant information. Similarly, the breast cancer study utilized a HIPAA-compliant, web-based survey and database application. We also asked participants to provide a name and address for a secondary contact to use if we were unable to reach them. In addition to linking the participant’s name and unique study ID number in this database, we were able to easily track changes in address as well as changes in status—for example, when the family members notified us when a participant had died or been institutionalized. This database allowed us to easily determine who needed follow-up contact (eg, reminders to return the survey) and to summarize response rates and dropouts. These data were essential for our longitudinal analysis where we determined that our missing data from participants were not missing at random. The careful maintenance of this database has allowed us to not waste time and money sending surveys to those who were no longer interested or able to participate.

There are important privacy considerations related to both mailings/data collection and study tracking. The electronic database used commercially available software, was password-protected, and accessible to only the project director and principal investigator. Strong passwords were used and updated regularly. To protect privacy regarding their diagnosis, we were careful to not use any language on external envelopes or postcards that referred to a specific disease. As data sets and study documents were stored in cloud applications, we had to be sure digital access was kept up to date. As students or staff members came and went from the project, we needed to remove their access and remind them to delete any and all files they might have downloaded for preliminary analyses. It was important to ensure that cloud solutions were HIPAA-compliant and, in the case of the PROBC study, could be easily accessed by staff members from other study sites.

Careful consideration in the selection of data collection instruments is especially important since the core battery of study instruments will be used consistently and administered in the same format and order throughout the study. We also developed and implemented a systematic way to name variables in our database for statistical analysis where the key variable label remained the same, but the year changed. For example, we have collected data using the CES-D (Center for Epidemiological Studies Depression Scale) as a depression measure across many of years. Item 1 of the CES-D in year 22 would be labeled MS22CESD1 and in year 23 it would be labeled MS23CESD1. A clear protocol for naming and entering data is particularly important in a long-term study where time and changes in staff may make communication more challenging.

Minimizing Cost

Weinert and Burman ² urged researchers to give careful consideration to strategies that minimized the cost of participation to participants. We thought broadly about how we could make it easy for participants to respond and complete study measures and procedures while staying connected to participants (See Table 2).

OPEN IN VIEWER

Table 2.

Strategies to Enhance Data Collection Response in Longitudinal Studies.

• Develop and use a consistent “brand” for the study (name and logo). • Establish and maintain connection with an established institution or organization for credibility, and ensure that participants are aware of this relationship. • Carefully design surveys (paper and digital) for ease of handling readability, and understanding. • With online data collection, use branching logic when possible, to decrease participant burden. • Provide meaningful rewards for participation including updates of study findings when possible. • Obtain secondary contacts for participants in case data collection instruments are “non-deliverable.” • Use shorter scales when possible to decrease participant burden. • Use holidays (birthdays or national holidays) as opportunities to make personal connections with participants.

In the MS survey study, we sought to make the questionnaire brief, easy to complete, and interesting. As recommended by Weinert and Burman, ² this included providing the survey in booklet format (easier for participants to manipulate) and provided substantial white space, clip art “breaks,” and encouraging messages within the survey. We also included the shortest scales that were psychometrically sound. Recent advances in psychometric analyses have provided short, yet reliable and valid alternatives to many older instruments, which may be particularly important to decrease burden in longitudinal studies (eg, the Patient Reported Outcome Measurement Information System [https://commonfund.nih.gov]). We made sure that directions were clear and response sets repeated on each page of a multipage instrument. Each survey was mailed with a return postage-paid envelope. For the MS Performance testing substudy, we paid transportation costs for those who did not drive and provided free parking for those who were able to drive themselves. For in-person visits required in the PROBC study, careful attention was paid to staying on time so that participants would not have to spend additional time waiting and all participants were provided free parking passes for all tests and appointments for data collection. Participants who had moved out of the area were provided renumeration for travel and hotel costs when they returned for data collection visits.

The PROBC study additionally utilized online surveys and assessments when possible. These were administered during the in-person study visits but provided several advantages to study efficiency. Specifically, online surveys included branching logic so that participants were only required to answer questions that were relevant for that patient. Known as computerized adaptive testing (CAT), this method not only increases assessment efficiency but also improves precision medicine and measurement accuracy. ¹¹ Another advantage of online measurement tools is that they typically have internal logic to detect input errors and they tend to provide automated scoring, which further reduces errors. Many online surveys have Application Programming Interfaces (APIs) that make it possible to automatically populate study databases with participant data or even administer the survey from the study database structure.

Advances in data collection technology, such as online surveys and wearable data sensors, may also serve to decrease the data collection burden for participants and staff. However, this increase in convenience must be balanced against potential threats to privacy (and in some cases participant resistance) for these data collection measures. In addition, participants need consistent access to staff who can assist them with unexpected technological glitches. In the PROBC study, staff were available to guide participants through Zoom as needed. Furthermore, the validity of data collected from these technologies requires additional scrutiny given the higher probability of user errors, environmental distractions, and incidental noncompliance with protocols that can occur in unsupervised environments. In addition, some online databases and assessments involve fees that increase study costs.

Increasing Rewards

Both studies implemented a number of techniques, many as recommended by Weinert and Burman, ² to increase the rewards of participation. All participant contacts include statements of appreciation, acknowledgement of the importance of their involvement, and positive regard for their time and energy. Open-ended questions at the end of questionnaires allowed participants to share other issues that were important to them. ¹² Participants were provided with monetary incentives—$75 for each in person visit (MRI and cognitive surveys) for the PROBC study and $20 to $30 gift cards for completing the MS survey (until 2013). In addition, participants received various merchandise with the study name and logo (eg, study t-shirt, brain-shaped stress ball, cooling bandana, carry-bags).

Many persons choose to participate in studies to not only contribute to science but also to learn additional information about their condition and their current status. Individual feedback was particularly important for persons with MS due to its long and highly variable disease course. Thus, we provided specific individual-level feedback multiple times during the MS study. These reports, hand-signed by the principal investigator, included easy to read graphs of how their functional limitations, health behaviors, and other variables had changed over time. We provided succinct interpretations of each person’s graph and provided recommendations based on the data. With the annual questionnaire, we provided study updates and information on “group-level” findings with reference to recent publications. We also provided updates of study findings in person through various advocacy group meetings and workshops. The PROBC study staff updated participants about the study’s benchmarks, conducted 2 seminars with expert speakers on key topics of interest to participants, and provided all participants with a copy of the PI’s book Improving Cognitive Function After Cancer. ¹³

Discussion

While longitudinal studies provide exceptional opportunities for greater understanding of sequence and outcomes, there are several important issues that must also be considered. One limitation—particularly for very long-lasting studies such as the MS study above—is that researchers are limited to the measures of constructs available when the study started if they wish to track changes over time. In the case of the MS study, there have been refinements and improvements in measures of many of the important constructs, but we must continue with “state of the art” from 25 years ago for comparison purposes. Investigators must also make careful decisions about what analyses to publish and at what time intervals. While one would clearly not publish longitudinal findings for the same set of variables annually, it would be important to determine what time intervals make sense logically. One strategy used in the MS study to address both the changes in instruments and the desire to publish is that we have added new constructs/instruments each year to the questionnaire battery. Generally, this involves 1 to 3 “new” scales and these instruments are added at the end of the core battery of instruments. In subsequent years, these “additions” are removed and other instruments may be added. Given the length of follow-up in the study, we have been able to track changes in some of these related constructs over time. For example, we may track how self-efficacy (measured at year 5 and year 15) has changed over a 10-year period.

Response rates in the 2 studies presented here were consistent with those of other similar longitudinal studies including the ACTIVE multicenter study, ¹⁴ the Nurse’s Health Study, ¹⁵ the UK Biobank, ¹⁶ and the Thinking and Living with Cancer study. ¹⁷ In fact, our response rates surpassed those of some studies, including those with greater resources, fewer follow-ups, or shorter durations. These comparisons lend support to the recommendations presented here.

Further research is needed to better understand how to best tailor incentives for participation in these types of studies. While we did not specifically evaluate the impact of payments on participation in either study, anecdotal evidence suggests that the impact of financial incentives may differ for individuals, perhaps depending on their circumstances. For example, in the PROBC study, we noted that participants who moved away or had a longer commute were incentivized to participate when they were given compensation for travel and lodging. These accommodations and payment amendments were made in response to the participants’ requests. While the study was not able to reimburse the full expense of travel and lodging, many in the PROBC study found that being heard and providing compensation was responsive to their concerns. In the MS study, response rates ranged from 81% to 90% during the years we were able to provide the small financial incentive. When we were no longer able to provide the incentive (after 2013), annual participation rates dropped to 72% over the following 2 years, suggesting that the small incentive may have been a deciding factor for some respondents. While some participants in both the MS and PROBC studies were incentivized by a payment, others specifically asked to donate their payments back to the study to further the research or to an MS or breast cancer organization. Consistent with earlier research, ¹⁸ investigators in both studies found that having a personalized approach, communicating results, providing incentives, and making reasonable accommodations were synergistically helpful in keeping high motivation for the participation.

Analysis of longitudinal data is complex and must carefully consider the inevitable missing data. An important consideration is whether the data can be considered to be missing at random. If so, various statistical procedures may appropriately be used to substitute for missing data. Among persons with chronic and disabling conditions, it is likely that “dropouts” are those who were older, had fewer resources, or had greater illness impact. Thus, analyses may need to include reason for dropout in their predictive models.

Conclusion

Longitudinal studies can provide rich data that allow greater understanding of participant experiences over time. The sequential data collection is particularly useful in refining predictive algorithms and models by providing a more dynamic timeline of syndromes and their determinants. However, these longitudinal designs unquestionably make time and effort demands on participants and staff and financial demands for ongoing study support. One question investigators must consider carefully is how long a longitudinal study should continue? Researchers must ensure ethical practices in participant follow-up (privacy, transparency in communication, etc). Researchers must also consider if their remaining “survivors” in a longitudinal study are sufficiently representative of the original study population and if the balance of benefits continues to outweigh the costs of continuing the study. These steps will help mitigate some of the disadvantages and leverage the strengths of longitudinal research to produce reliable, insightful, and impactful outcomes.