EEG-Neurofeedback and the Correction of Misleading Information: A Reply to Pigott and Colleagues

As scientists, we gladly welcome academic debate surrounding our research. However, when commentators Pigott and colleagues contact our universities demanding that we (R.T.T. and A.R.) be reprimanded and claiming that we are “contaminating the scientific literature with [our] animus-driven venom” and should “(re)take Learning 101 before publishing further,” their arguments cease to hold scholarly appeal. The three directors at McGill University in receipt of the accusations put forward by Pigott and colleagues discussed the complaint and all agreed that it did not merit a response. Chapman University launched an assessment and an administrator at the level of Dean performed the evaluation and dismissed the complaint. Pigott and colleagues are willfully tying-up academic resources in an attempt to stifle scientific research that challenges their opinions. Pigott informed us that the letter they sent our universities will soon appear in their own journal, NeuroRegulation (which is indexed neither by Thomson Reuters nor by Web of Science).

In their comment, Pigott et al. make four main points, all of which are misdirected and misleading.

First, they question the relevance of our pilot study, stating that “it is hard to see how this study provides anything more than anecdotal support for a new experimental treatment.” This study provides preliminary evidence and we do not argue otherwise. Our statement that the behavioral benefits of electroencephalogram-neurofeedback (EEG-nf) stem from psychosocial effects and general cognitive training mechanisms (Thibault & Raz, 2018) rely on published EEG-nf data, not our pilot experiment.

Second, they state “Thibault and colleagues ignore the evidence suggestive of [EEG-nf’s] specificity and effectiveness in treating the ‘neural signatures’ of attention deficit hyperactivity disorder (ADHD).” Not only is it unclear whether agreed upon EEG correlates of ADHD exist, this argument deflects our main point—our writing centers on behavioral outcomes, not EEG signatures.

Third, they state “[Thibault and colleagues] fail to acknowledge that in eight head-to-head comparisons with [stimulant medication] [EEG-nf] resulted in essentially equivalent improvement in treating ADHD.” We address this exact point in a previous publication (see the section titled “EEG-nf Treats ADHD Equivalent to Stimulant Medication” from Table 1 in Thibault & Raz, 2017).

Analyzing the data behind the fourth statement from Pigott and colleagues turns their crux argument against themselves: “Thibault and colleagues fail to acknowledge the extensive evidence from [EEG-nf] studies whose training methodology mirror the best practices of operant conditioning. These studies consistently find . . . improvements on a wide variety of ADHD outcome measures.” And yet, the effect sizes from the studies and meta-analysis that the commentators deem to “mirror the best practices” are comparable with those they criticize because they “violate established learning science.” The Cohen’s d values for the common primary measures in these experiments (reported severity of inattention and hyperactivity) range from 0.50 to 0.80 for the meta-analysis (van Doren et al., 2018) and 0.14 to 0.62 (Strehl et al., 2006), 0.40 to 0.57 (Strehl et al., 2017), and 0.34 to 1.02 (Leins et al., 2007) for the studies Pigott and colleagues support. Of the six studies they criticize, four document effect sizes for reported ADHD severity, or provided enough data to calculate them. These are similar between genuine and sham-nf groups and range from 0.52 to 1.91 (Lansbergen, van Dongen-Boomsma, Buitelaar, & Slaats-Willemse, 2011), “large” effects sizes, but no specific number reported (Arnold et al., 2013), 0.33 to 0.84 (Van Dongen-Boomsma, Vollebregt, Slaats-Willemse, & Buitelaar, 2013), and 1.00 to 1.51 (Schönenberg et al., 2017). Pigott and colleagues haphazardly use these studies to falsely argue that operant conditioning must drive the behavioral benefits of EEG-nf. A systematic survey of the empirical data alongside an appreciation for parsimonious explanations (cf., Occam’s Razor) reveals a different story—one that can explain the relevant behavioral results without ever invoking operant conditioning.

We were surprised to see that the commentators did not consider Rex Cannon’s position as the Director of Science And Operations for the Knoxville Neurofeedback Group, and Mark Trullinger’s position as Managing Director of NeuroThrive, LLC—both private clinics—as potential conflicts of interest.

The Editor in Chief of Journal of Attention Disorders accepted the comment by Pigott and colleagues under the pretense that providing them with a platform and responding to the erroneous basis for their opinions is better than allowing them to complain that mainstream science is ignoring what they believe to be good science (personal communication, May 22, 2018). In the future, we hope that academic journals heed the advice from the cognitive bias literature (e.g., Lewandowsky, Ecker, Seifert, Schwarz, & Cook, 2012) and refrain from publishing misleading information—even if they believe a future expository account will set the record straight.