Abstract
Abstract
Background:
Patients increasingly use mobile devices to send text messages and photographic data to surgeons. There is potential to harness this patient-generated health data (PGHD) for clinical and public health surveillance of surgical site infection (SSI). Leveraging PGHD collected via remote monitoring in the post-operative period has the potential to produce important benefits for patients, surgeons, care teams, and infection surveillance and prevention.
Methods:
We conducted a health technology assessment (HTA), drawing heavily on stakeholder engagement to better understand current and potential uses of PGHD in post-operative care. Stakeholder engagement activities included assembling an advisory board composed of stakeholder experts, interviewing key informants, and seeking out stakeholder guidance to synthesize evidence from interviews, literature review, and technical app review in order to develop recommendations on the use of PGHD in SSI surveillance.
Results:
We conducted a review of the published literature, a technical/market scan of available apps for capturing post-operative PGHD, and two rounds of key informant interviews with stakeholders. In addition, we held a day-long workshop to solicit stakeholder feedback on initial findings of the project and to guide additional work. These activities culminated in an HTA report that provides guidance and recommendations on the use of PGHD in SSI surveillance, including practice, research, and public health surveillance, and identifies open issues on post-operative use of PGHD for which additional evidence and experience are needed to optimize application of those data for clinical and public health purposes.
Conclusion:
Stakeholders, individuals with direct experience, or interest in a given topic are critical to the HTA process. They provide insight to guide the work conducted, ensure that the topics addressed are relevant and important, and that products of the work are accessible and meaningful to the individuals who will be most impacted.
The increasing use of mobile devices by patients to send images and text messages to surgeons and care teams after surgery presents new opportunities for both clinical and public health practice. Used wisely, this rapidly evolving application of information technology has the potential to produce important benefits for individual patients, ambulatory surgical centers and hospitals, practitioners, and infection surveillance and prevention. For this reason, the U.S. Centers for Disease Control and Prevention, through the Safety and Healthcare Epidemiology Prevention Research Development (SHEPheRD) program, supported a two-year project led by principal investigators at the University of Washington and Medical University of South Carolina, to evaluate current uses of patient-generated health data (PGHD) and mobile devices in post-operative surgical site infection (SSI) surveillance by conducting a health technology assessment (HTA).
Conducting a Stakeholder-Driven Health Technology Assessment
Stakeholders, individuals with direct interest in, or experience with a given topic are important throughout the HTA process. Their involvement helps shape and inform the work conducted, ensuring that relevant and important issues are addressed. Furthermore, stakeholder involvement encourages a sustained focus that the advice and reporting rendered are accessible, transparent, and that outcomes of interest reach those impacted by the decision [1]. The Assessing Surgical Site Infection Surveillance Technologies (ASSIST) HTA addressed the need for systematic study of current uses of PGHD and sought to define how PGHD should be leveraged for SSI monitoring and surveillance across clinical, research, and public health settings [2]. To conduct the HTA, we used an adapted framework that closely paralleled the National Institutes of Health “HTA 101” framework and the Center for Medical Technology Policy process for developing Effectiveness Guidance Documents [1,3,4]. This stakeholder-driven approach incorporated methods for engaging expertise, refining topic areas by available evidence and expert judgement, gathering and organizing evidence, developing recommendations, reaching agreement on guidance, incorporating public comment, and disseminating findings (Table 1).
ASSIST Health Technology Assessment Process
ASSIST = Assessing Surgical Site Infection Surveillance Technologies; HTA = health technology assessment.
Our HTA process called for the formation of a scientific board composed of expert stakeholders with experience in surgical infectious disease research, infection control, antimicrobial stewardship, clinical informatics, clinical protocol implementation, and quality improvement. The scientific board provided guidance on HTA activities, refined topic areas, and provided expert review and refinement of project outputs. Health technology assessment activities included gathering evidence and expert judgement through a landscape analysis (literature and technology/market reviews), key informant interviews, and targeted stakeholder engagement activities. The process was iterative, incorporating guidance from the scientific board and stakeholders at each step. As the work progressed, the project team synthesized learnings, drafted work products (e.g., evidence summaries, presentations, and workshop reports), and relayed these to the scientific board for feedback and refinement.
Assembling the Evidence
The topics identified by the scientific board guided identification, retrieval, and review of available evidence on the use of PGHD for SSI surveillance. This evidence appraisal started with a review of the peer-reviewed and gray literature. As technology and implementation efforts typically move faster than publications, we augmented the landscape analysis with technical and market scans of available post-operative apps. We identified commercial and other mobile apps that are offered or have been used to allow patients to collect post-operative images or other data, and developed briefs on the technology, extent of usage, and notable characteristics of those systems. We also sought evidence from expert judgement and experience through key informant interviews.
Literature review
The evidence appraisal started with a review of the literature. The purpose of this review was to gain an understanding of the current application of mobile health (mHealth) and PGHD in post-operative incision management and SSI surveillance. The literature search was conducted in partnership with a University of Washington biomedical and translational sciences librarian. The search yielded more than 4,000 references, 133 of which were included for abstract review. Thirty-four manuscripts, including 26 peer-reviewed manuscripts, and 8 research briefs were included in the evidence review; and an additional 18 were included as background material for the HTA although they were not included in the evidence review. We assessed the included studies for findings relevant to the application of PGHD in post-operative care including clinical/surgical specialty, length of follow-up period, type of mHealth application used, type of PGHD included, all outcomes measured, whether SSI was included as an outcome, and how SSI was defined.
Technical and market review
As technology and implementation efforts typically move faster than publications, we augmented the literature review with technical and market scans of available apps for post-operative remote management. We identified commercial and other mobile apps that allow patients to collect post-operative incision images or other data, and developed briefs on the technology, extent of usage, and notable characteristics of those systems. Apps used directly by patients or caregivers in the post-discharge setting for tracking surgical sites were included for review. Apps were identified through the published literature, colleagues of the ASSIST team and scientific board, at the Health Information and Management Systems Society (HIMSS) annual meeting, and through online searches for relevant mHealth products. Excluded from the review were apps intended only for provider use, apps used primarily for long-term wound tracking, and apps intended primarily for care coordination.
Key informant interviews
Our team conducted 21 interviews with key stakeholders in the field of mHealth and PGHD for SSI detection and monitoring to elicit information on their experience with, and perspectives on PGHD use in post-operative SSI diagnosis and surveillance. Areas of inquiry were informed by the project team's experience, and input from the scientific board. Interviews provided important stakeholder insights about how PGHD is captured and implemented across the health system and evidence gaps in the use of PGHD for SSI surveillance.
The interviews were conducted to uncover the experiences and views of stakeholders that typically remain unpublished, but that influence practice patterns, data reporting, and interpretation in the field. The initial round of interviewees was identified through purposive sampling of experts in the field of mHealth, PGHD, and SSI. Additional key informants were identified through recommendations by first-round interviewees, authors of published studies identified in the literature review, and creators of apps identified in the technical and market review. A list of the stakeholder perspectives captured in key informant interviews is provided in Table 2.
Stakeholder Perspectives of Key Informants
Key informants in these categories included one individual with expertise in both design and ethics.
Key informants in these categories were included in the second round of interviews.
Synthesizing and Refining the Evidence: PGHD Stakeholder Advisory Group Workshop
After the landscape analysis and key informant interviews, the evidence was synthesized and presented to stakeholders at a day-long workshop designed to elicit stakeholder feedback on the findings. These stakeholders comprised a PGHD Stakeholder Advisory Group (PSAG) whose members included engaged key informants as well as other leaders in the field of PGHD for SSI surveillance. Solicitation of involvement in the PSAG was intended to ensure broad stakeholder input from researchers, clinicians, industry, and healthcare administration. Importantly, this engagement framework addressed the system-level disconnect that exists between research, healthcare delivery, and technology design through an explicitly community-driven approach.
The purpose of the PSAG workshop was to inform and refine further the HTA process by providing feedback on ASSIST project work, initial results, and inform the development of recommendations. PGHD Stakeholder Advisory Group members provided additional insights into how stakeholders approach the design, assessment, and adoption of new technology, and highlighted gaps in current knowledge about PGHD integration into SSI diagnosis and surveillance efforts. This full-day workshop included a morning session presenting the preliminary results of the landscape analysis and key informant interviews. The afternoon session included topic-based breakout discussion sessions with PSAG members to refine evidence-based guidelines, identify evidence gaps, and establish priority areas for future research. The workshop was audio recorded to enable transcripts to be produced of the proceedings. Key learnings were synthesized in a workshop report that was circulated to PSAG members for additional feedback.
Additional key informant interviews
A critical outcome of the PSAG workshop was the consensus among stakeholders that a second round of key informant interviews focused on the patient experience should take place. PGHD Stakeholder Advisory Group members helped the ASSIST team identify additional key informants who were willing to be interviewed about the patient experience. We conducted a second round of targeted key informant interviews with a nurse case-manager who had direct experience of utilizing PGHD for SSI surveillance, and two patients with experience using PGHD for post-operative monitoring. These interviews were included in the final evidence synthesis.
Producing the HTA Findings and Recommendations
The project team and scientific board worked together to synthesize the information gathered from the landscape analysis, key informant interviews, and PSAG workshop in order to produce an HTA report, including a set of 10 recommendations for utilizing PGHD for SSI monitoring and surveillance. In areas in which there is sufficient evidence, the recommendations propose practice guidance. In areas in which there are gaps, the goal was to propose recommendations for a research agenda and to identify high-priority research questions based on both impact and feasibility.
This HTA report identifies guidance and recommendations in several areas, including practice, research, and public health surveillance, and identifies open issues on post-operative use of PGHD for which additional evidence and experience are needed to optimize application of those data for clinical and public health purposes. The HTA report and recommendations were refined through a series of editorial cycles with the ASSIST team and scientific board, and a public comment period during which broad input was solicited. The report and recommendations were refined further through feedback obtained from presentations to three national surgical organizations: the Surgical Infection Society (SIS), the National Surgical Quality Improvement Program (NSQIP), and the American College of Surgeons (ACS). Feedback was solicited directly after symposia presented at the annual meetings of each of these organizations and incorporated into the HTA report.
Conclusion
The ASSIST project was conducted using the framework of a health technology assessment, incorporating involvement from multiple stakeholders representing the groups invested in and affected by the use of mHealth to acquire patient-generated health data for surgical site infection monitoring. In this manuscript, we demonstrate that the HTA processes helped synthesize the current state of evidence addressing topics such as, does the technology work? For whom does it work? And, what is the benefit of the technology? In this context, we sought to understand the current evidence of mHealth in post-operative care to inform recommendations for future work. To our knowledge, this is the first HTA to assess mHealth for SSI surveillance and represents a snapshot in a rapidly changing environment. In conducting this assessment, we have not only generated foundational work in this domain but formed the beginnings of a community of practice comprising diverse stakeholder perspectives to inform future research on this important topic.
Footnotes
The ASSIST Investigators
Sheri Chernetsky Tejedor; E. Patchen Dellinger; Heather L. Evans; Traci Hedrick; Shuai Huang; Danielle C. Lavallee; Jenney R. Lee; William B. Lober; Aven Samareh; Robert Sawyer; John Semple; Jyotirmay Sharma; Elizabeth Wick
Acknowledgments
This work was supported by U.S. Centers for Disease Control and Prevention award #200-2016-91803 through the Safety and Healthcare Epidemiology Prevention Research Development (SHEPheRD) Program, which is managed by the Division of Healthcare Quality Promotion. The content is solely the responsibility of the authors and does not represent the official views of the CDC.
Author Disclosure Statement
No competing financial interests exist.
Heather L. Evans, Tetraphase Advisory Board, KCI Symposium Speaker.