Increasing and sustaining the engagement of participants in clinical research studies is a goal for clinical investigators, especially for studies that require long-term or frequent involvement of participants. Technology can be used to reduce barriers to participation by providing multiple options for clinical data entry and form submission. However, electronic systems used in clinical research studies should be user-friendly while also ensuring data quality. Directly involving study participants in evaluating the effectiveness and usability of electronic tools may promote wider adoption, maintain involvement, and increase user satisfaction of the technology. While developers of healthcare applications have incorporated user-centered designs, these methods remain uncommon in the design of clinical study tools such as patient-reported outcome surveys or electronic data capture digital health tools.
Methods:
Our study evaluated whether the clinical research setting may benefit from implementing user-centered design principles. Study participants were recruited to test the web-based form for the Measurement to Understand the Reclassification of Disease of Cabarrus/Kannapolis (MURDOCK) Study Community Translational Population Health Registry and Biorepository that would enable them to complete their study forms electronically. The study enrollment form collects disease history, conditions, smoking status, medications, and other information. The system was initially evaluated by the data management team through traditional user-acceptance testing methods. During the tool evaluation phase, a decision was made to incorporate a small-scale usability study to directly test the system.
Results:
Results showed that a majority of participants found the system easy to use. Of the eight required tasks, 75% were completed successfully. Of the 72 heuristics violated, language was the most frequent violation.
Conclusion:
Our study showed that user-centered usability methods can identify important issues and capture information that can enhance the participant’s experience and may improve the quality of study tools.
CoonsSJEremencoSLundyJJ, et al. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient2015; 8: 301–309.
2.
OrriMLipsetCHJacobsBP, et al. Web-based trial to evaluate the efficacy and safety of tolterodine ER 4 mg in participants with overactive bladder: REMOTE trial. Contemp Clin Trial2014; 38(2): 190–197.
3.
O’ConnellSPalmerRWithersK, et al. Requirements for the collection of electronic PROMS either “in clinic” or “at home” as part of the PROMs, PREMs and effectiveness programme (PPEP) in Wales: a feasibility study using a generic PROM tool. Pilot Feasibility Stud2018; 4: 90.
4.
RothrockNEBassMBlumenthalA, et al. AO patient outcomes center: design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics. JMIR Form Res2019; 3: e10880.
5.
EverattGDMcLeodRJr. The software development life cycle. Software testing across the entire software development life cycle. Hoboken, New Jersey: John Wiley & Sons, 2007, pp.29-58.
6.
Society for Clinical Data Management. Good clinical data management practices (GCDMP) 2008, SCDM.org/gcdmp (accessed 2 June 2020).
7.
FlaviánCGuinalíuMGurreaR.The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manage2006; 43: 1–14.
8.
IvesBOlsonMHBaroudiJJ.The measurement of user information satisfaction. Commun ACM1983; 26: 785–793.
9.
NielsenJ. Usability inspection methods. In: Proceedings of the conference companion on human factors in computing systems, Boston, MA, 24–28 April 1994, pp. 413–414. New York: ACM.
10.
GraneheimUHLundmanB.Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today2004; 24(2): 105–112.
FrøkjærEHertzumMHornbækK.Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? In: Proceedings of the SIGCHI conference on human factors in computing systems, The Hague, 1–6 April 2000. New York: ACM.
13.
JohnsonCJohnstonDCrowleP.EHR usability toolkit: a background report on usability and electronic health records. Rockville, MD: Agency for Healthcare Research and Quality, 2011.
14.
TangPCPatelVL.Major issues in user interface design for health professional workstations: summary and recommendations. Int J Biomed Comput1994; 34(1–4): 139–148.
15.
BatesDWKupermanGJRittenbergE, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med1999; 106(2): 144–150.
16.
TierneyWMMcDonaldCJMartinDK, et al. Computerized display of past test results: effect on outpatient testing. Ann Intern Med1987; 107(4): 569–574.
17.
EricssonKASimonHA.Protocol analysis: verbal reports as data. Cambridge, MA: MIT Press, 1984.
FaulknerL.Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput2003; 35(3): 379–383.
23.
VirziRA.Refining the test phase of usability evaluation: how many subjects is enough?Hum Factors1992; 34: 457–468.
24.
KushnirukAWPatelVL.Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform2004; 37(1): 56–76.
25.
SuhSYTrabassoT.Inferences during reading: converging evidence from discourse analysis, talk-aloud protocols, and recognition priming. J Memory Lang1993; 32: 279–300.
26.
JohnsonCMShawRJ.A usability problem: conveying health risks to consumers on the Internet. AMIA Annu Symp Proc2012; 2012: 427–435.
27.
BrookeJ.SUS: a “quick and dirty” usability scale. In: JordanPWThomasBWeerdmeesterBA, et al. (eds) Usability evaluation in industry. London: Taylor & Francis, 1996, pp. 4–7.
28.
BangorAKortumPTMillerJT.An empirical evaluation of the system usability scale. Int J Hum Comp Interact2008; 24: 574–594.
29.
TullisTAlbertW.Measuring the user experience: collecting, analyzing, and presenting usability metrics. Amsterdam: Elsevier, 2008, p. 320.
30.
SauroJ.SUPR-Q: a comprehensive measure of the quality of the website user experience. J Usabil Stud2015; 10: 68–86.
31.
ZhangJWaljiMF.TURF: toward a unified framework of EHR usability. J Bomed Inform2011; 44: 1056–1067.
32.
NielsenJ.Usability engineering. Burlington, MA: Morgan Kaufmann, 1993, p. 358.
33.
ShneidermanBPlaisantC.Designing the user interface: strategies for effective human-computer interaction. 5th ed.New Delhi, India: Pearson, 2010.
34.
ZhangPCareyJTe’eniD, et al. Integrating human-computer interaction development into SDLC: a methodology. In: Proceedings of the 10th Americas conference on information systems, New York, NY, 6–8 August 2004.
35.
MiddletonBBloomrosenMDenteMA, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc2013; 20(e1): e2–e8.
36.
TangZJohnsonTRTindallRD, et al. Applying heuristic evaluation to improve the usability of a telemedicine system. Telemed J E Health2006; 12(1): 24–34.
37.
ZhangJJohnsonTRPatelVL, et al. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform2003; 36(1-2): 23–30.
38.
Paasche-OrlowMKParkerRMGazmararianJA, et al. The prevalence of limited health literacy. J Gener Inter Med2005; 20: 175–184.
39.
KutnerMGreenburgEJinY, et al. The health literacy of America’s adults: results from the 2003 national assessment of adult literacy (NCES 2006-483). National Center for Education Statistics, 2006, https://nces.ed.gov/pubs2006/2006483.pdf
40.
GoodmanMFinneganRMohadjerL, et al. Literacy, numeracy, and problem solving in technology-rich environments among US adults results from the program for the international assessment of adult competencies 2012: first look (NCES 2014-008). National Center for Education Statistics, 2013, https://nces.ed.gov/pubs2014/2014008.pdf
41.
BrittoMTJimisonHBMunafoJK, et al. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc2009; 16(5): 660–669.
42.
PyperCAmeryJWatsonM, et al. Patients’ experiences when accessing their on-line electronic patient records in primary care. Br J Gen Pract2004; 54(498): 38–43.
43.
JohnsonCMNahmMShawRJ, et al. Can prospective usability evaluation predict data errors?AMIA Annu Symp Proc2010; 2010: 346–350.
44.
SchmierJKKaneDWHalpernMT.Practical applications of usability theory to electronic data collection for clinical trials. Contemp Clin Trial2005; 26: 376–385.
45.
WaljiMFKalenderianETranD, et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform2013; 82(2): 128–138.
46.
FoxJEMockovakWFisherSK, et al. Usability issues associated with converting establishment surveys to web-based data collection. Stat Policy2004; 63, https://www.oecd.org/sdd/33630831.pdf
SivajiAAbdullahMRDowneAG, et al. Hybrid usability methodology: integrating heuristic evaluation with laboratory testing across the software development lifecycle. In: Proceedings of the 2013 10th international conference on information technology: new generations, Las Vegas, NV, 15–17 April 2013, pp. 375–383. New York: IEEE.
