Practicing What We Preach (and Sometimes Study): Methodological Issues in Experimental Laboratory Research

At the core of any causal claim is an experimental study, and laboratory research often provides the most valid arena for conducting experiments. Laboratory experiments allow researchers to provide internal validity to any phenomena they examine. However, researchers must consider and address certain methodological concerns when conducting lab research. Before participants ever agree to participate, forces are beginning to affect how they will respond to experimental manipulations. These and additional concerns continue during and after the completion of the experiment, yet many researchers do not fully understand their implications. As serious as these issues have always been, the introduction of social media and Internet data collection only highlight the importance of understanding and addressing methodological issues in laboratory experiments. In the following article, we discuss potential methodological flaws in laboratory research, examine research relevant to these problems, review evidence that suggests problems actually do contaminate our data, and suggest areas for future research. We conclude by providing recommendations for eliminating or alleviating these problems in future research, considering the role of replication studies as a post-hoc means of detecting these issues, and apply our conclusions to experimental research performed outside or a laboratory. In all, we believe that addressing these concerns in experimental research and practice will strengthen the validity of our conclusions, the replicability of our findings, and science as a whole.