Abstract
Response variability between individuals (interindividual variability) and within individuals (intraindividual variability) is an important issue in the transcranial magnetic stimulation (TMS) literature. This has raised questions of the validity of TMS to assess changes in corticospinal excitability (CSE) in a predictable and reliable manner. Several participant-specific factors contribute to this observed response variability with a current lack of consensus on the degree each factor contributes. This highlights a need for consistency and structure in reporting study designs and methodologies. Currently, there is no summarized review of the participant-specific factors that can be controlled and may contribute to response variability. This systematic review aimed to develop a checklist of methodological measures taken by previously published research to increase the homogeneity of participant selection criteria, preparation of participants before experimental testing, participant scheduling, and the instructions given to participants throughout experimental testing to minimize their effect on response variability. Seven databases were searched in full. Studies were included if CSE was measured via TMS and included methodological measures to increase the homogeneity of the participants. Eighty-four studies were included. Twenty-three included measures to increase participant selection homogeneity, 21 included measures to increase participant preparation homogeneity, while 61 included measures to increase participant scheduling and instructions during experimental testing homogeneity. These methodological measures were summarized into a user-friendly checklist with considerations, suggestions, and rationale/justification for their inclusion. This may provide the framework for further insights into ways to reduce response variability in TMS research.
Introduction
Response variability to noninvasive brain stimulation (NIBS) has become an important issue in recent years. Response variability serves as an umbrella term to both interindividual variability, the variability in response between individuals, and intraindividual variability, the variability in response within individuals both in and across different experimental testing sessions. An increased focus on response variability is due to previously published studies reporting subgroups of individuals who respond to NIBS as historically expected or not as historically expected (Chew et al., 2015; López-Alonso et al., 2014; Müller-Dahlhaus et al., 2015; Wiethoff et al., 2014).
A number of factors have been proposed to contribute to this previously reported response variability. Recent literature and systematic reviews have identified several anatomical, biological, and physiological factors that may differ between individuals that may, in part, explain the differences in responses to NIBS protocols (Li et al., 2015; Pellegrini et al., 2018a; Ridding and Ziemann, 2010). Systematically investigating these factors is an important step forward in NIBS research. If the NIBS technology is to progress to consistent large-scale use with standardized dosages and protocols, having the capacity to predict an individual's response is essential. This highlights the ongoing necessity for investigating response variability.
Differences in changes in corticospinal excitability (CSE) between individuals form the basis of response variability, and are typically assessed using transcranial magnetic stimulation (TMS) technology. Thought to assess and quantify the physiological state of corticospinal tract in real-time (Di Lazzaro, 2004; Ridding and Rothwell, 1997), TMS depolarizes pyramidal tract neurons evoking a subsequent motor response, the motor evoked potential (MEP) that can be quantified (Di Lazzaro, 2004; Ridding and Rothwell, 1997), the larger the MEP amplitude, the greater the excitability of the corticospinal tract (Nitsche and Paulus, 2000; Ridding and Rothwell, 1997). Previous reports have posed problems when determining whether changes in CSE following NIBS, as measured by TMS-evoked MEP amplitude, are true reflections of changes in CSE or whether they are just an indication of response variability between individuals or with individuals across multiple experimental testing sessions (Ammann et al., 2017).
Variability in changes in CSE between individuals and across experimental testing session, as measured via TMS-evoked MEPs, is thought to have several contributing factors (Li et al., 2015; Pellegrini et al., 2018a; Ridding and Ziemann, 2010). Referring to factors that differ between participating individuals, interindividual variability can include anatomical, biological, and physiological factors intrinsic to the participating individuals. For example, males versus females (Inghilleri et al., 2004; Zoghi et al., 2015), specific genetic variations (Antal et al., 2010; Hwang et al., 2015; Puri et al., 2015), older versus younger individuals (Fujiyama et al., 2014; Heise et al., 2014), or anatomical differences in skull and cortical morphology (Datta et al., 2012; Miranda et al., 2013; Opitz et al., 2015). Variations in physiological factors can also contribute to interindividual variability, for example, differences in responses across different phases of the menstrual cycle (Ansdell et al., 2019; Inghilleri et al., 2004; Smith et al., 2002), differences in caffeine intake between participating individuals (Cerqueira et al., 2006; Hanajima et al., 2019; Orth et al., 2005), or differences in cortisol levels between participants by conducting experimental sessions at different times-of-the-day (Sale et al., 2007, 2008). Intraindividual variability is important when repeated-measures study designs are utilized and can include factors both extrinsic and intrinsic to the participating individual. Extrinsic factors can include technical methodological measures such as the choice of TMS device, the reliability of the TMS operator, as well as the consistency in the experimental setup. Intrinsic factors can include the same physiological factors as listed above if they are not kept consistent within each participant across repeated-measures experimental testing sessions. While it is unlikely that one specific factor is the underlying cause of the previously observed response variability, it is important that these factors are systematically identified and consistently controlled to minimize the effect of response variability on subsequent results. Currently to these authors' best knowledge, a summary of the specific factors that may contribute to response variability, and that may be consistently controlled, has not been previously reported in the literature.
One recent attempt to improve consistency among researchers in the TMS literature was the development of a checklist by Chipchase and colleagues (2012). In response to a lack of consistency in methodological reporting in the TMS literature, a checklist was developed for future researchers to report on important technical and methodological factors that should be documented and controlled for in studies utilizing TMS-evoked MEPs as an index of CSE. For the checklist developed by Chipchase and colleagues (2012), experts in the field of TMS as an assessment tool were consulted to provide recommendations on important methodological factors that should be consistently controlled and documented in the wider literature. The Delphi technique (Hsu and Sandford, 2007) was used to achieve consensus among expert researchers on the items most essential to include in the checklist. The resultant checklist included factors relating to participant demographics, medical and medication history, participant muscle activity and level of attention, time-of-day of testing, and the technical TMS device characteristics, including coil size and orientation and TMS pulse characteristics (Chipchase et al., 2012).
The checklist developed by Chipchase and colleagues (2012) has since been cited in over 100 publications and has therefore been an important step forward in the consistent documentation and reporting of methodological factors when TMS is used as an assessment tool. Checklists are able to “democratize knowledge” (Winters et al., 2009) and standardize experimental processes to facilitate consistent task completion (Hales et al., 2007; Winters et al., 2009). With the issue of response variability remaining, there is an opportunity to develop an additional checklist that focuses on participant-specific components of response variability that may be controllable. One component that may be controllable is the actual individuals selected to participate in studies assessing changes in CSE via TMS. With the knowledge that individuals can vary in their changes in CSE, as measured by TMS, greater detail may be required to ensure that selected participating individuals are not too different from each other. Therefore, as a point of difference to the methodological checklist developed by Chipchase and colleagues (2012), there is an opportunity to develop a checklist focusing on maximizing the homogeneity of participant selection, participant preparation before experimental testing, and the instructions given to participants throughout the experimental testing. The rationale for this checklist being that, if as many aspects of the participating individuals are kept as constant as possible, differences in observed response rates (i.e., responders vs. nonresponders) following interventions such as NIBS between individuals or within individuals across multiple experimental testing sessions can be investigated while simultaneously minimizing potential confounding factors.
While the Delphi technique is effective at achieving consensus among experts, it is limited by its reliance on subjective opinion of a selected small number of researchers (Yousuf, 2007). This may therefore not provide a comprehensive and systematic review of the current practice in the wider literature, an approach that is required in the development of thorough checklists (Hales et al., 2007). Alternatively, systematic reviews of the literature can minimize the reliance on subjective opinion and are capable of reviewing and synthesizing a large scope of previously published research to gain additional insight and draw conclusions (Wright et al., 2007). Comprehensive systematic reviews have played a substantial role in recently developed checklists in the medical and scientific fields (Shankar et al., 2013; Sherren et al., 2014).
Given the significance of the checklist developed by Chipchase and colleagues (2012) in documenting technical and device measures to control, the aim of this study was to systematically review the TMS literature to develop an additional checklist based on previous literature that focused on methodological measures to increase the homogeneity in participant selection, preparation, and instructions during experimental testing. This is to accompany and complement the checklist developed by Chipchase and colleagues (2012) to minimize the effect of participant response variability on research findings.
Materials and Methods
This systematic review was conducted in alignment with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (Moher et al., 2009).
Literature search
Seven databases were searched in detail from January 2012 to December 2018: CINAHL Plus, Ovid Medline, ProQuest, PubMed, Scopus, SPORT Discuss, and Web of Science. This date range was chosen to concentrate included studies to recently published works to provide a checklist based on recently published methodologies. Key search terms included were as follows: Transcranial Magnetic Stimulation, Corticospinal Excitability, Motor Evoked Potential, and Primary Motor Cortex. Database yields were compiled, the duplicates were discarded, and the journal articles remaining were screened for suitability for inclusion. The reference lists of all of the retrieved journals were searched for additional journal articles that met the inclusion criteria.
Selection criteria
The lead author (M.P.) reviewed the titles and abstracts of all journal articles from the database search yield to determine their eligibility based on the following inclusion and exclusion criteria. Journal articles that met these criteria were retained for full-text examination. Criteria were kept broad to ensure a large pool of studies could be examined for suitability.
Inclusion criteria
Journal articles investigating CSE via TMS were included if they met the following criteria: (1) utilized TMS as an assessment tool, (2) measured CSE via single-pulse TMS-evoked MEPs, and (3) measured CSE of the primary motor cortex (M1), including but not limited to the hand, wrist, face, and lower limb areas as the specificity of the location on M1 will not change the measures taken to maximize homogeneity of participant selection, preparation, scheduling, or instructions during experimental testing.
Exclusion criteria
Journal articles were excluded if they met the following criteria: (1) did not use TMS, (2) measured CSE via outcome measures other than single-pulse TMS-evoked MEPs, (3) measured CSE of brain regions other than the M1 (e.g., dorsolateral prefrontal cortex or primary somatosensory cortex), (4) animal studies, (5) literature reviews, case reports, and oral or poster presentation abstracts.
Data extraction
For each of the included journal articles, the methodology sections were read in full. The methodology sections were examined for measures that were implemented to increase the homogeneity of participant selection, participant preparation before experimental testing, participant scheduling, or the instructions given to participants throughout the experimental testing. Data on these factors were extracted and grouped based on whether they focused on (1) participant recruitment/selection criteria; (2) preparation of participants before experimental testing; (3) participant scheduling; or (4) instructions given to participants throughout experimental testing. Factors were included if they were considered a potential contributor to the onset of interindividual variability between participants that would generate subgroups of individuals, or intraindividual variability within participants across multiple experimental testing sessions. That being, if there was enough justification and rationale to suggest that each factor may act as a potential source of response variability and cause different magnitudes in changes in CSE between individuals or within individuals across multiple testing sessions, it was included in this study and the resultant checklist. The factors that were implemented by previous journal articles were then synthesized to make considerations/suggestions for future studies with accompanying rationale/justification (Tables 1 –3). These recommendations were then summarized to a checklist (Fig. 1). If measures taken by previous journal articles were not justified or supported by reference to previously published research, they were not included in the summary, Tables 1 –3, or the resultant checklist. Examples of excluded measures are listed in Table 4.
Final checklist to address factors that may act as potential sources of response variability in TMS studies. TMS, transcranial magnetic stimulation.
Summary of Participant Selection Criteria That May Influence Response Variability and Considerations/Suggestions with Justification/Rationale
a-tDCS, anodal transcranial direct current stimulation; CSE, corticospinal excitability; ICF, intracortical facilitation; MEP, motor evoked potential; MMSE, mini-mental state examination; NIBS, noninvasive brain stimulation; PAS, paired associative stimulation; SICI, short interval intracortical inhibition; TMS, transcranial magnetic stimulation.
Summary of Participant Preparation Criteria That May Influence Response Variability and Considerations/Suggestions with Justification/Rationale
GABA, gamma-aminobutyric acid; M1, primary motor cortex.
Summary of Participant Scheduling Criteria and Instructions During Experimental Testing That May Influence Response Variability and Considerations/Suggestions with Justification/Rationale
EMG, electromyography; %MVC, percentage of maximal voluntary contraction.
Results
Literature search
All electronic database search yields were combined for 5654 journal articles. These were manually reviewed and 555 remained after removal of duplicates and journal articles that were not relevant based on analyzing the title and abstract. Journal articles were then excluded for not meeting the inclusion criteria (n = 327), leaving 228 journal articles remaining for qualitative synthesis. Journal articles were then read in full with 133 excluded based on not reporting methodological measures to homogenize participant selection, preparation, and instructions throughout the experimental testing that may be a potential source of response variability. A further 11 journal articles were excluded based on including methodological measures without sufficient rationale/justification. Therefore, 84 journal articles were included in this systematic review of the literature.
Study characteristics
The methodological measures, taken by these 84 included journal articles to reduce response variability, were divided into 3 subgroups.
Participant recruitment/selection criteria
Twenty-three of the 84 journal articles reported methodological measures to increase participant selection homogeneity. These included controlling for age (Fujiyama et al., 2014; Goodwill et al., 2015; Heise et al., 2014), athletes (Murase et al., 2015), cognitive state (Gedankien et al., 2017; Goodwill et al., 2013; Heise et al., 2014; Lee et al., 2014; Petitjean and Ko, 2013; Puri et al., 2015, 2016; Vernet et al., 2013), gender and the female menstrual cycle (Cambieri et al., 2017; Hashemirad et al., 2017b; Vaseghi et al., 2016), handedness (Liu and Au-Yeung, 2014; Matamala et al., 2018; Simis et al., 2013), medications and medical conditions, musicians, or people with specialized use of their hands (Cirillo et al., 2009; Goodwill et al., 2013, 2015; Hirano et al., 2017; Littmann et al., 2013; Murase et al., 2015), and smokers or drug users (Ammann et al., 2017; Batsikadze et al., 2015; Cappon et al., 2016; Hermsen et al., 2016; Horvath et al., 2016; O'Leary et al., 2015). A summary of these methodological considerations or suggestions, as well as rationale and justification, is listed below in Table 1.
Participant preparation before experimental testing
Twenty-one of the 84 journal articles included specific preparation instructions for participants before attending experimental testing to increase their homogeneity upon presentation to the testing session. These instructions included abstinence periods before experimental testing for the consumption of alcohol (Ammann et al., 2017; Cappon et al., 2016; Fresnoza et al., 2014; Fujiyama et al., 2017; Jamil et al., 2017; Littmann et al., 2013; Matamala et al., 2018; O'Leary et al., 2015, 2016, 2018; Peri et al., 2017), caffeine (Aboodarda et al., 2017; Ammann et al., 2017; Fujiyama et al., 2017; Hanajima et al., 2017; Hermsen et al., 2016; Jamil et al., 2017; Kluger et al., 2012; Littmann et al., 2013; Matamala et al., 2018; O'Leary et al., 2015, 2016, 2018; Rupp et al., 2012), and energy drinks (Chew et al., 2015). Further measures included restrictions on physical activity levels before experimental testing (Aboodarda et al., 2017; Jamil et al., 2017; Littmann et al., 2013; Matamala et al., 2018; O'Leary et al., 2015, 2016; Rupp et al., 2012) and instructions regarding number of hours of sleep before experimental testing (Badawy et al., 2013; Chew et al., 2015; Fecchio et al., 2017; Fujiyama et al., 2017; Hashemirad et al., 2017a; López-Alonso et al., 2015; Peri et al., 2017; Rupp et al., 2012). See below (Table 2) for a summary of these methodological measures.
Participant scheduling and instructions during experimental testing
Sixty-one of the 84 journal articles reported methodological measures that were taken to ensure consistent experimental testing between participants and across different experimental testing sessions. Scheduling measures included controlling the time of day experimental testing was conducted (Ammann et al., 2017; Bastani and Jaberzadeh, 2013a,b; Cash et al., 2016; Chew et al., 2015; Cirillo et al., 2012; Faber et al., 2017; Fecchio et al., 2017; Fujiyama et al., 2017; Goldsworthy et al., 2014; Greenhouse et al., 2017; Hashemirad et al., 2017a; Hinder et al., 2014; Lahr et al., 2016; Monda et al., 2017; Murakami et al., 2012; Opie et al., 2017; Pellicciari et al., 2013; Player et al., 2012; Puri et al., 2015, 2016; Sale et al., 2017; Sidhu et al., 2017; Strube et al., 2016; Vallence et al., 2013, 2015; Vaseghi et al., 2016) and controlling the number of days between testing sessions (Ambrus et al., 2016; Ammann et al., 2017; Cash et al., 2016; Chew et al., 2015; Heise et al., 2014; Hirano et al., 2017; López-Alonso et al., 2014; Müller-Dahlhaus et al., 2015; Player et al., 2012; Puri et al., 2015, 2016; Sidhu et al., 2017; Suzuki et al., 2012; Vallence et al., 2013, 2015; Vaseghi et al., 2016). Finally, instructional measures included controlling for level of muscle activity throughout experimental testing (Arai et al., 2012; Brownjohn et al., 2014; Dileone et al., 2018; Faber et al., 2017; Hordacre et al., 2017; Jelić et al., 2015; Mathew et al., 2016; McCambridge et al., 2015; Müller-Dahlhaus et al., 2015; Player et al., 2012; Schilberg et al., 2018; Shirota et al., 2017; Tsutsumi et al., 2014; Vallence et al., 2015; Vernet et al., 2013; Wu et al., 2012), controlling for head and neck posture (Bastani and Jaberzadeh, 2012; Davidson et al., 2016; Dyke et al., 2016; Jaberzadeh et al., 2015; Petitjean and Ko, 2013; Radel et al., 2016; Schintu et al., 2016; Vallence et al., 2014; Young-Bernier et al., 2014), and controlling for the participant's attentional focus throughout the experimental testing (Batsikadze et al., 2015; Cash et al., 2017; Cirillo et al., 2012; Conde et al., 2012; Delvendahl et al., 2012; Kumru et al., 2017; Lahr et al., 2016; López-Alonso et al., 2014; Müller-Dahlhaus et al., 2015; Murase et al., 2015; Player et al., 2012; Sidhu et al., 2017; Strube et al., 2015). Table 3 below provides a summary of these methodological measures.
Excluded measures
Excluded measures previously taken to maximize participant homogeneity or experimental testing consistency based on a lack of rationale or justification or it was unclear how the measure may be a potential source of response variability. These included controlling for handedness (Conforto et al., 2012; Nuzum et al., 2016), body mass index (BMI) (Azabou et al., 2013), level of cardiorespiratory fitness (Cirillo et al., 2012; Fujiyama et al., 2014; Goodwill et al., 2013, 2015; Hortobágyi et al., 2017; Puri et al., 2015; Williams et al., 2013), hair length and thickness (Srivanitchapoom et al., 2016; Tremblay et al., 2015), amount of food intake before experimental testing (O'Leary et al., 2015, 2016, 2018; Rupp et al., 2012), and restricting talking during experimental testing (Dileone et al., 2018; Koizume et al., 2017; Mordillo-Mateos et al., 2012; Tremblay et al., 2016). These are listed below in Table 4.
List of Criteria That Were Excluded from the Final Checklist for Reducing Response Variability, with the Measures Taken by Previous Literature and Their Reason for Exclusion from the Checklist
BMI, body mass index.
Discussion
The purpose of this systematic review was to investigate the methodological measures taken by previous studies to maximize homogeneity of participant selection, preparation, scheduling, and instructions during experimental testing sessions to facilitate the development of a checklist to address response variability. To these authors' knowledge, this is the first study to-date to systematically investigate the methodological measures taken by previous research and to summarize these measures into a user-friendly checklist. The results of this systematic review report a total of 84 previous studies implementing methodological measures to increase the homogeneity of the individuals participating in studies assessing CSE via TMS. Furthermore, this systematic review also provided justification or rationale for each item in the checklist (Tables 1–3). The checklist that resulted from this systematic review thus adds insight into the issue of response variability and may be a useful tool for future researchers addressing the impact of response variability in the TMS literature.
Checklist use
In alignment with the checklist developed by Chipchase and colleagues (2012), the resultant checklist from this systematic review can be used in several ways to address the issue of response variability. First, the checklist can be used as a tool during the research study design phase. Depending on the primary hypotheses of the study, future researchers can utilize the checklist to determine the measures focusing on the participating individual that should be considered and controlled for that may be a potential source of response variability. Second, the checklist can assist in critically appraising methodologies and interpreting the findings of previously published research. Depending on the research aims of previous studies, not considering response variability and controlling for the selection, preparation, and scheduling of participants, as well as the instructions given to the participants throughout experimental testing, sessions may assist in evaluating and challenging the previous methodologies and the significance of the researcher's findings. Finally, this checklist offers future researchers the opportunity to display their methodological rigor when submitting articles of completed research for peer review. It will provide the framework for researchers to demonstrate their consideration of methodological factors specific to the participating individuals that contribute to response variability and to aid the journal editors and reviewers when assessing the merit of their methodological procedures and overall study design.
To these authors' knowledge, this appears to be the first time considerations and suggestions to increase participant homogeneity have been compiled into a user-friendly checklist in the TMS literature. While this may be the case, these considerations or suggestions were deliberately nondefinitive and should act as a guide only, based on the rationale and justification provided (Tables 1–3). Depending on the particular hypotheses, future researchers can utilize this checklist and decide whether each listed consideration or suggestion is relevant to their study designs by checking “controlled,” “not-controlled,” or “N/A.” In addition, this checklist may serve as a tool for investigating the mechanisms underlying response variability. Future researchers can control for as many of these included factors as required, depending on the particular study design, and leave other factors uncontrolled to determine the role as a potential source of response variability. This is the ultimate endeavor to gain further insight into the factors that contribute to response variability.
Checklist significance
The significance of the checklist developed in this systematic review is the comprehensive review of the large scope of literature and the structured manner in which it brings to focus the factors that may contribute to the issue of response variability in the TMS literature. By systematically synthesizing all relevant previous methodological measures, the checklist developed by this systematic review has been able to evaluate a large scope of previous methodological measures and only includes those with clear rationale and justification. Systematic reviews are considered the highest form of evidence, while expert opinion is considered the lowest as defined by the Oxford Centre for Evidence-Based Medicine (Howick, 2009). The checklist developed by the current systematic review therefore provides an example of how systematic reviews allow targeted and comprehensive investigation of greater scopes of evidence-based factors when compared with expert consensus-based approaches such as the Delphi technique. While the expert consensus-based approaches do have the capacity to include factors that may not have been clearly reported in previous literature, it does call into question the relevance and validity of specific methodological measures that focus on the participating individuals if certain research groups assessing CSE via TMS deem them not significant enough to explicitly report in their published work. Developing a checklist via a systematic review of the literature therefore avoids the pitfalls of the opinion-based subjective nature of the expert consensus-based approach (Yousuf, 2007).
The checklist developed in the current systematic review highlights the importance of the participating individuals in research findings. Common within TMS literature is to recruit “healthy young” participants, which may overlook inherent differences between each individual who participates. As such, while important measures to consider, the current checklist was not intended to be an updated version of the checklist developed by Chipchase et al. (2012). It therefore does not include technical and device-specific measures such as the number of TMS-evoked MEPs required to record an accurate representation of CSE, the type of TMS device utilized, or the process by which the cortical hotspot is located. The checklist developed in the current study is instead intended to be used in conjunction with the checklist developed by Chipchase and colleagues (2012) and focuses on methodological measures surrounding the participants rather than the technical and device-specific measures. Utilizing the checklist developed in the current study will therefore guide future researchers in minimizing the effect participant-specific factors may have on results, thus ultimately reducing the effect of response variability on research findings. Consistently incorporating this checklist into study designs may, as previously reported, contribute to a greater understanding of the factors and mechanisms behind response variability and may ultimately result in increased reliability and predictability of changes in CSE following interventions such as NIBS (Hordacre et al., 2017; Li et al., 2015; Strube et al., 2016).
Included methodological measures
This systematic review identified 84 previous studies utilizing 15 methodological measures to increase the homogeneity of participants (Fig. 1 and Tables 1–3). Of these 84 studies, 22 implemented 5 different methodological measures to homogenize participant selection (Table 1), 21 studies implemented 5 measures to homogenize participant preparation (Table 2), and 61 studies implemented 5 measures to homogenize the scheduling of participants and instructions given throughout the experimental testing sessions (Table 3).
As listed in Figure 1 and Tables 1–3, the number of previously published studies that implemented each of these methodological measures varied, as did the extent of justification or rationale. For example, for participant selection criteria, the suggestion to consider controlling for history of musical instrument playing or specialized use of their hands was implemented by six previous studies, while the suggestion to consider controlling for high-performing athletes was only implemented by one previous study (Table 1). Similar discrepancies were reported for participant preparation criteria with 15 previous studies controlling for caffeine consumption before participating in experimental testing, while just 1 previous study controlled for energy drink consumption (Table 2).
While these discrepancies were present, as described in Tables 1–3 based on the justification or rationale from previous studies assessing CSE via TMS, the capacity for each of these measures to be a potential source of response variability validated their inclusion in the final checklist (Fig. 1). As an example, the suggestion to consider controlling for estrogen hormone fluctuations across the difference phases of menstrual cycle may act as a source of response variability. By selecting female participants to attend experimental testing sessions and not controlling for which phase of their menstrual cycle they were currently in, increases the possibility of observing interindividual variability by creating subgroups of individuals who may respond differently to interventions such as NIBS. In addition, by not controlling for this across different experimental testing sessions within participants, there may be an increased possibility of observing intraindividual variability for repeated-measures study designs. The same rationale can be applied for all included measures, for example, musicians versus nonmusicians, those who have consumed alcohol before experimental testing versus those who have not, or those who have their attentional focus controlled throughout experimental testing sessions versus those who do not. This rationale is expanded upon for each included methodological measure in Tables 1–3.
Excluded methodological measures
Table 4 outlines a number of factors that were not included in the final checklist. Identified initially in the data extraction phase, upon analysis and investigation of previous literature, it was unclear how these factors may act as a potential source of response variability. The basis for excluding these factors was for several reasons. Factors were excluded if it was unclear how they may directly influence CSE as with level of cardiorespiratory fitness and amount of food intake. In addition, factors were excluded if it was unclear how dividing participants based on the presence or absence of a specific factor (e.g., individuals with high BMI vs. low BMI) would act as a potential source of interindividual variability and subgroup individuals into responders or nonresponders to interventions such as NIBS. Finally, factors were excluded that primarily focused on ensuring the reliability of TMS-evoked MEPs rather than addressing response variability, such as controlling for hair length.
In the process of reviewing the previously published studies from the search yield, in addition to the participant-specific methodological measures identified, a number of technicalities were also identified. These included measures relating to locating the TMS coil hotspot, by consistently re-evaluating the TMS coil position throughout the experimental testing session (Kidgell et al., 2013), with the majority of included studies specifically marking the cortical hotspot with a permanent marker. Technical measures also included incorporating rest periods between determining the active and resting motor thresholds and commencing recording of TMS-evoked MEPs to ensure relaxation (Ambrus et al., 2016; Rivera-Urbina et al., 2015; Sasaki et al., 2018; Shirota et al., 2017), as well as varying the TMS intertrial interval to limit participant anticipation of TMS pulses (Faber et al., 2017). Technical measures to optimize muscle electromyography signal such as consistent skin preparation technique (Kim et al., 2012; Nakazono et al., 2016) were also utilized by previously published studies. As discussed above, these methodological measures were beyond the scope of the current systematic review. Considered measures to increase the overall reliability of single-pulse TMS-evoked MEPs rather than influencing the variability in response to NIBS, these technical measures were therefore not included in the final checklist. While also important, this does, however, opens the possibility of future studies to conduct a systematic review into the technical methodological measures to optimize the reliability of single-pulse TMS-evoked MEPs. This is to ensure that future NIBS studies reporting changes in CSE report true changes that are not influenced by reduced reliability of TMS as an assessment tool.
Limitations
A potential limitation to the checklist developed in this systematic review is that, as discussed above, the recommendations provided are not evenly weighted with regard to previous reporting in the literature. An additional limitation is the fact that despite summarizing a list of potential factors that may be a source of response variability, as well as suggestions/considerations on how to reduce their influence with accompanying justification/rationale, it does not offer quantifiable insight into the extent these factors may contribute to response variability, an insight that warrants future investigation.
Future directions
As response variability becomes a greater focus in the TMS and NIBS literature and subgrouping participants into responder or nonresponder categories is reported more often (Ammann et al., 2017; Chew et al., 2015; Pellegrini et al., 2018b; Wiethoff et al., 2014), updates or modifications to this checklist may be required. As new evidence arises regarding the intrinsic and extrinsic factors that may contribute to response variability, future researchers may consider controlling for these factors when designing their experimental protocols or when recruiting participants. It is at this stage that these factors may be added to an updated version of this checklist. For example, as more findings are reported regarding specific genetic variants that may influence response variability to NIBS or the history of neuronal activity effecting response to NIBS, considerations and suggestions with accompanying justification/rationale surrounding these factors should be added to an updated version of the checklist.
Furthermore, as stated above, this checklist can only identify and control for the factors that may influence response variability, not the extent to which they contribute. By investigating methods to quantify response variability, particularly intraindividual variability, such as those previously utilized (Fernandez et al., 2017; Iscan et al., 2016; Orth et al., 2003), the TMS literature may progress to a greater understanding of the extent to which each factor contributes and how to minimize the effect. Not only will this facilitate direct comparisons between experimental protocols regarding the extent to which response variability was minimized, it may additionally lead to systematic investigations into the extent to which specific factors contribute to response variability. For example, as discussed above, depending on the specific study design, by controlling for all factors listed in this checklist, the degree to which genetic variation or history of neuronal activity may influence interindividual variability could be quantified. To these authors' knowledge, such investigations are yet to be conducted. Developing robust methods of quantifying observed response variability is an integral component if the TMS and more broadly the NIBS field are to minimize response variability and progress to developing reliable and predictable protocols for consistent use.
Conclusions
The novel checklist developed in this systematic review provides a useful tool for future researchers to control for methodological participant-specific factors that may contribute to response variability and ultimately minimize their effect on research findings. It also has the added potential of being used in conjunction with the technical and device-specific checklist developed by Chipchase and colleagues (2012) to display methodological rigor when submitting articles for peer review. With the increasing need for predictable, reliable changes in CSE as measured via TMS, and consistency in reporting to the wider literature, this checklist may be a valuable resource in investigating the mechanisms and factors involved in observed response variability.
Authors' Contributions
Conceived and designed the review: M.P. and S.J. Performed the literature searches: M.P. Conducted the analysis: M.P. Interpreted the findings: M.P., S.J., and M.Z. Wrote the article: M.P. Writing and editing of drafts: M.P., S.J., and M.Z.
Footnotes
Author Disclosure Statement
No competing financial interests exist.
Funding Information
No funding was received for this article.