The Use of Human Factors to Address Medical Research Replicability Through the Development of Software Based Solution

Abstract

The lack of research replicability is becoming more of a concern in many fields. The medical research community and specifically the National Institutes of Health are concerned about how this issue will affect the well-being of the field (Collins & Tabak, 2014). However, this situation represents an opportunity to showcase how human factors can act as a solution through a combination of usability and human systems integration. The current work details how the development of a software platform designed to address replicability and reproducibility concerns in drug dependency research was informed by human factors principles from early development.

References

Asendorpf

J. B.

Conner

De Fruyt

De Houwer

Denissen

J. J.

Fiedler

. . . Perugini

(2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27(2), 108-119.

Baker

(2016). 1,500 scientists lift the lid on reproducibility. Retrieved from http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970#/ref-link-2.

Begley

C. G.

Ellis

L. M.

(2012). Drug development: Raise standards for preclinical cancer research. Nature, 483(7391), 531-533.

Begg

C. B.

Berlin

J. A.

(1988). Publication bias: a problem in interpreting medical data. Journal of the Royal Statistical Society. Series A (Statistics in Society), 419-463.

Cannon-Bowers

J. A.

Tannenbaum

S. I.

Salas

Converse

S. A.

(1991). Toward an integration of training theory and technique. Human Factors, 33(3), 281-292.

Clegg

B. A.

Heggestad

E. D.

Blalock

L. D.

(2010, September). The influences of automation and trainee aptitude on training effectiveness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, No. 27, pp. 2329-2332). Sage CA: Los Angeles, CA: SAGE Publications.

Collins

F. S.

Tabak

L. A.

(2014). NIH plans to enhance reproducibility. Nature, 505(7485), 612.

Cruess

R. L.

Cruess

S. R.

(2008). Expectations and obligations: professionalism and medicine’s social contract with society. Perspectives in Biology and Medicine, 51(4), 579-598.

Dickersin

(1990). The existence of publication bias and risk factors for its occurrence. Jama, 263(10), 1385-1389

10.

Duval

Tweedie

(2000). Trim and fill: a simple funnel plot–based method of testing and adjusting for publication bias in meta analysis. Biometrics, 56(2), 455-463.

11.

Dwan

Gamble

Williamson

P. R.

Kirkham

J. J.

(2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PloS one, 8(7), e66844.

12.

Easterbrook

P. J.

Gopalan

Berlin

J. A.

Matthews

D. R.

(1991). Publication bias in clinical research. The Lancet, 337(8746), 867-872.

13.

Elam

Bertilsson

(2003). Consuming, engaging and confronting science the emerging dimensions of scientific citizenship. European Journal of Social Theory, 6(2), 233-251.

14.

Franco

Malhotra

Simonovits

(2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502-1505.

15.

Fowlkes

J. E.

Merket

D. C.

Oser

R. L.

(2000, July). Transitioning SA theory and research into practical training guidance: A case study. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 44, No. 11, pp. 419-422). Sage CA: Los Angeles, CA: SAGE Publications.

16.

Gibbons

(1999). Science's new social contract with society. Nature, 402, C81-C84.

17.

Gore

S. M.

Jones

I. G.

Rytter

E. C.

(1977). Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976. Br Med J, 1(6053), 85-87.

18.

Hartshorne

J. K.

Schachner

(2012). Tracking replicability as a method of post-publication open evaluation. Beyond open access: visions for open evaluation of scientific papers by post-publication peer review, 70.

19.

Ioannidis

J. P. A.

Greenland

Hlatky

M. A.

Khoury

M. J.

Macleod

M. R.

Moher

. (2014). Increasing value and reducing waste and research design, conduct and analysis. Lancet 383, 166–175.;

20.

Ioannidis

J. P.

(2005a). Contradicted and initially stronger effects in highly cited clinical research. Jama, 294(2), 218-228.

21.

Ioannidis

J. P.

(2005b). Why most published research findings are false. PLos med, 2(8), e124.

22.

Kilkenny

Browne

W. J.

Cuthill

I. C.

Emerson

Altman

D. G.

(2010). Improving bioscience research reporting: the ARRIVE guidelines for reporting animal re

23.

Kleiner

B. M.

Booher

H. R.

(2003). Human systems integration education and training. Handbook of human systems integration, 121-163.

24.

Lawton

J. S.

(2016). Reproducibility and replicability of science and thoracic surgery. The Journal of Thoracic and Cardiovascular Surgery, 152(6), 1489-1491.

25.

Louridas

Gousios

(2012). A note on rigour and replicability. ACM SIGSOFT Software Engineering Notes, 37(5),1-4.

26.

Macaskill

Walter

S. D.

Irwig

(2001). A comparison of methods to detect publication bias in meta analysis. Statistics in medicine, 20(4), 641-654.

27.

Mumaw

R. J.

Roth

E. M.

(1995, October). Training complex tasks in a functional context. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 39, No. 20, pp. 1253-1253). SAGE Publications.

28.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

29.

Pashler

Harris

C. R.

(2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531-536.

30.

Patterson

R. E.

Pierce

B. J.

Boydstun

A. S.

Ramsey

L. M.

Shannan

Tripp

Bell

(2013). Training intuitive decision Making in a simulated real-world environment. Human factors, 55(2), 333-345.

31.

Prasad

Vandross

Toomey

Cheung

Rho

Quinn

(2013, August). A decade of reversal: an analysis of 146 contradicted medical practices. In Mayo Clinic Proceedings (Vol. 88, No. 8, pp. 790-798). Elsevier.

32.

Prinz

Schlange

Asadullah

(2011). Believe it or not: how much can we rely on published data on potential drug targets?. Nature reviews Drug discovery, 10(9), 712-712.

33.

Sarewitz

(2004). How science makes environmental controversies worse. Environmental science & policy, 7(5), 385-403.

34.

Schmidt

(2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90.

35.

Straus

S. E.

McAlister

F. A.

(2000). Evidence-based medicine: a commentary on common criticisms. Canadian Medical Association Journal, 163(7), 837-841.

36.

Tversky

Kahneman

(1971). Belief in the law of small numbers. Psychological bulletin, 76(2), 105.

37.

Wager

Williams

(2013). “Hardly worth the effort”? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ, 347, f5248.