Using structured data entry systems in the electronic medical record to collect clinical data for quality and research: Can we efficiently serve multiple needs for complex patients with spina bifida?

1. Introduction

Certain studies are difficult to interpret in the aggregate due to the huge amount of information that they contain. A videourodynamic study (VUDS) is an example of such a challenge, which becomes even more challenging if the formal interpretation of the tracing is left until after the urodynamics session is completed. As shown in several studies, and in an interactive presentation by Dr. Stacy Tanaka at the 2017 Spina Bifida Association meeting, the interpretation of the cystometrogram after the fact only increases the chances for greater inter-observer variability and possible intra-observer variability as well [1, 2, 3]. In addition, after one has performed hundreds of studies, how can one begin to categorize the subsets of patients without looking at each record? In this era of the electronic health record (EHR), we can only do better in terms of identifying these subsets if we can commit to standardizing the test conditions and ideally the means of recording the data during the actual encounter.

EHR platforms allow use of structured data entry systems (SDES) which allow data entry based on predefined categories and conditions. While SDES can be employed to generate patient history or physical examination findings, they can also be used in the development of specific flow sheets tailored to a certain disease process or procedure. SDES allow for uniformity and standardization of data entry and collection. This, in turn, allows for easier reporting of that data and use in templated notes for the EHR [4].

Developing and utilizing a SDES is critical for both optimizing delivery of top-quality patient care and collecting data for research endeavors. A properly designed SDES can decrease research costs, increase patient-oriented research, and facilitate medical advancements without compromising privacy, the completeness, or accuracy of the EHR [5]. Murray and Berberian have proposed four steps to creating a usable SDES: 1) institute a clinical advisory committee to develop and maintain standards for clinical protocols for clinical information within EHR, 2) identify the “deal breakers” for structured data entry, especially in regards to physician resistance, 3) identify the workflows to facilitate data entry capture, and 4) identify the technology platforms necessary for seamless integration, often with the help of information technology or services departments [6].

The International Children’s Continence Society (ICCS) recently published a guideline for the standardization of the measurement, quality control, and documentation of UDS in children [7]. With these recommendations in mind and with the understanding that medical record keeping must be focused around the patient at the time of the encounter and thus cannot be purely focused on the research questions, we set out with the goal of building a SDES that we could populate in real time during the videourodynamic encounter that would do the following:

Calculate expected bladder capacity and the volumes representing 25, 50, and 75% of the expected bladder capacity. These calculations are based on the patient’s weight in kg which we would enter at the start of the visit.

Our goal was to develop a SDES capable of serving a patient care function by allowing for a better standardization of the study, a safety function by allowing periodic assessment of the most hostile bladders, as well as a research function.

OPEN IN VIEWER

OPEN IN VIEWER

OPEN IN VIEWER

A working group of three pediatric urologists and two nurses who specialize in caring for children with spina bifida convened to determine what data should be recorded, after which a trial was undertaken for three months using a simple template modification in EPIC (Epic Systems Corp., Madison, Wisconsin), the EHR employed at our institution. The impact of the EPIC modification on workflow during the VUDS encounter was examined. Important parameters and data for entry were discussed among the group and included if consensus was reached by the majority of the group. During the three month trial period, as studies were being performed, additional parameters deemed important for inclusion were discussed with the group and added to the form if consensus was reached.

Once this was done, hospital informatics staff joined us to help design a template SDES form in a flow sheet format with drop down menus for descriptive variables (Fig. 1). This allowed for calculations of expected bladder capacity as well as the volumes at 25, 50, and 75% of estimated bladder capacity based on weight (Estimated bladder capacity [in cc] = patient weight in kg × 7). In addition, the flow sheet automatically tallied the voided (or leaked) volume (if any) as well as the residual urine determined by aspiration at the end of the study. This total volume is then subtracted from the infused volume to arrive at the volume of urine produced during the actual study. This value can be substantial depending upon the degree of diuresis a patient may be experiencing, either as a result of a post obstructive diuresis effect or because of the variabilities in the state of hydration.

All studies were conducted by the attending physician. Data was entered in real time as each check point was reached during the urodynamic study. Images captured during the study are stored in the GE fluoroscopic system and then transmitted to the department of radiology for final interpretation and storage. The flow sheet is then transferred into a prepopulated area of a clinic progress note upon completion of the VUDS study (Fig. 2). The progress note also contains areas for the physical examination as well as free text entries to describe the indications for the study, any current medical and social issues, as well as the final impression and plan. As a result of how the data is stored within the flow sheets, Information Systems can send us a weekly summary of all the VUDS completed in a CSV delimited format for easy use in programs such as Excel ( © and (P) 1998–2007 Microsoft Corporation, Redmond, Washington) (Fig. 3). This data can also be entered into REDCap (Research Electronic Data Capture) which then allows for sharing across multi-institutional trials. REDCap is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources. In addition, there is no information loss since the actual urodynamic tracing with associated images is also scanned into EPIC in a PDF file for permanent storage. This file can then be retrieved for further analysis as needed.

Data was analyzed using Stata 14.2 (StataCorp, College Station, TX, USA). Continuous variables were analyzed using the Wilcoxon rank-sum test and dichotomous variables using Pearson’s chi-square test. Significance was set at 0.05.

3. Results

During the 18 month period between September 2015 to February 2017, a total of 316 urodynamic studies on patients with spina bifida were completed. All studies were completed using a SDES form. Information was inputted into the form in real-time during the urodynamic study by the pediatric urology attending. At the end of each week, the data from each VUDS was delivered via email to one attending who compiled the data into one complete database. Overall, analysis of the columns of data revealed that entries were missing in 3% of cells. This data accrual rate was achieved by physician data entry alone as there was no research assistant who monitored the weekly data feed or went back over the data to fill in any missing values. Physicians were encouraged to leave cells empty if they could not be populated with certainty.

To demonstrate the ease at which the entered SDES data can be used for potential research purposes, several simple analyses were performed of the complied database of all 316 urodynamic studies in consecutive SB patients. Of these patients, 74 had undergone a prenatal closure and 242 had undergone a postnatal closure. A table was then generated to compare percentage of expected bladder capacity reached at end of study, presence of vesicoureteral reflux, and presence of detrusor overactivity during the study between the SB patients who underwent prenatal vs. postnatal closure (Table 1). The bladder capacity attained for the two groups and the presence of vesicoureteral reflux did not differ between the prenatal and postnatal closure cohorts (Table 1). Statistical analysis can then easily be performed on this collected data as shown in Table 1.

In addition to providing VUDS data in an easily sortable and analyzable fashion, the information compiled by the SDES allows for quick assessment of the reliability and precision of the data as well as inter-observer agreement. For instance, any parameter or data in the SDES form can be pulled, such as bladder neck status on fluoroscopic images or bladder pressure at 50% of expected bladder capacity. A second physician can review the original VUDS study to determine inter-observer agreement. This allows the SDES form and data to be used in quality control or improvement projects.

With any SDES form and implementation, there are potential limitations. First, at our institution, the attending pediatric urologist is present for all urodynamic studies and inputs all data into the SDES in real time during the study. This policy may not be the same at other institutions. The robustness and reliability of data entry by other health care providers was not tested in our study.

While adoption of EHR systems was already spreading by the late 2000s, the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009 as part of the American Recovery and Reinvestment Act accelerated that growth by offering incentives as well as penalties for failure to convert to an EHR by 2015 [8]. In fact, by 2014 97% of all hospitals and 75% of all physicians nationwide were using certified EHR technology [9]. As a consequence of this widespread adoption of EHR, the amount of electronically stored patient data available for quality improvement, research, and outcomes analysis has also increased dramatically. Unfortunately, many EHR systems were initially designed with billing and reimbursement purposes in mind and thus adaptation of these systems for clinical assessment and improvement has not always been smooth. Additionally, physician attitudes towards EHR, and the use of SDES within the EHR, vary across specialties and age groups and may form a barrier to successful implementation in clinical practice and research [5].

Given the complexity of important clinical information acquired from urodynamic studies (UDS), it is a model procedure for implementation of a SDES. VUDS and UDS without fluoroscopy are important tools in the pediatric urologist skillset to objectively evaluate the lower urinary tract function in children. UDS studies are often employed in various clinical scenarios and are helpful in both non-neurogenic and neurogenic bladder dysfunction [10, 11]. Retrieval of data from these and other complex studies can be accomplished if the study is standardized via an easy to use and accurate SDES and if the data is stored in the EHR in a flowsheet template that can be retrieved on a regular basis. This allows for the analysis of complex subsets of patients on an ongoing basis.

Recently, the International Children’s Continence Society (ICCS) published a guideline for the standardization of the measurement, quality control, and documentation of UDS in children [7]. With these recommendations and those of the International Continence Society (ICS)’s “Good Urodynamic Practices” in mind, we created a SDES in the form of a flow sheet to be filled out during the UDS in real-time. The design of this SDES flow sheet followed the four principles outlined by Murray and Berberian described above [6]. The important data points for collection and clinical protocol for UDS were defined by the spina bifida working group at our institution, a multidisciplinary group of physicians and nurses. Barriers to implementation including physician resistance were minimized by creating an easy to use flow sheet that can be filled out during the study so that no additional time is added to the patient encounter. The entire department (including both physicians and non-physician providers involved in UDS) was taught the proper workflows to obtain and capture the data for the SDES during the urodynamic study. Lastly, with collaboration from the information services department, the ability to integrate the SDES into the encounter/procedure note in the EHR was established as well as the automated system for collecting and disseminating the weekly reports via email.

In this study we have shown that a structured data entry system in real time for a complex diagnostic test like VUDS can yield high quality data with minimal disruption of patient care. The total time it took to analyze the datasets for this report was under 2 hours which is very short and efficient when one considers how long it would have taken to build the dataset from each individual patient chart and inputting the considerable amount of data one cystometrogram at a time. As an example analysis, we compiled specific parameters and characteristics of interest between spina bifida patients who had undergone postnatal versus prenatal closures. Statistical analyses of the data are shown in Table 1 as a proof of concept, not to draw conclusions about differences between the SB populations.

Analysis of the weekly compiled datasets showed that approximately 3% of all cells have missing values, often because there is no appropriate entry for the value. In some instances, the failure to enter data was an oversight at the time of the urodynamic study. Review of the missing cells on a weekly basis allows for these data points to be filled in accurately in the chart. Ideally, a research assistant could curate the data set on a weekly basis. If an assistant were to review the 6–8 studies per week that are done in the division, the accuracy of the data set would improve. This could be accomplished with a short time commitment making this an efficient investment of research funds. It is also critical to remember that all tracings and recordings from the VUDS system are stored separately and available for review, thus, there is no chance for information loss.

We have learned some lessons along the way that we would share with other groups seeking to enter such data. First and foremost is the need for all the providers involved in these studies to discuss what information they are committed to documenting in the EHR. The more information included in the SDES, the longer it will take to fill out and the higher the likelihood of increased provider burden. Some patient data and characteristics such as age at the time of the study should be easy to auto populate directly from the EHR and thus can be done without need for additional entry by the provider. One important note is that a hard stop was not built to prevent closure of an EPIC encounter if the urodynamics flowsheet was not filled out. This was to prevent rapidly filling in cells just to get an encounter closed to keep up with the workflow. It is better to leave a cell empty, and the data added later by a research assistant’s weekly review than to quickly put in data that is not accurate.

The essence of any successful SDES lies in its ability to standardize data collection across patients through an easy reporting system while allowing for improved decision support, real-time quality assessment, and opportunities for patient-oriented clinical research [4, 12]. Ideally, a well-designed SDES should facilitate quality improvement whether in a hospital setting or a clinical practice. Batalden and Davidoff define the goals of “quality improvement” as making changes that lead to “better patient outcomes (health), better system performance (care), and better professional development (learning)” [13]. The ease of both collecting and reviewing UDS data in our SDES flow sheet makes it the ideal model for implementing quality improvement projects via the “plan-do-study-act” (PDSA) approach [14]. From a quality improvement standpoint, our UDS SDES can allow for the identification of the patients at highest risk of renal damage. Furthermore, this systematic approach to data collection also allows us to bank urine samples from well characterized phenotypes in a search for biomarkers that would be predictive of a progressive loss of compliance. Future projects in a PDSA approach can focus on any of the numerous questions still surrounding pediatric UDS such as identifying risk factors at the time of study for post-UDS urinary tract infections or identifying predictors of which children with neurogenic bladder will require management changes based on UDS results.

5. Conclusion

The era of the EHR brings with it the ability to systematically collect and record innumerable amounts of data, especially for complex procedures such as VUDS. Designing and implementing a structured data entry system based on a flow sheet that can both allow the data to seamlessly be placed in the electronic medical note and be integrated into a searchable database for quality improvement and research purposes, allows one to harness the true potential of the EHR. Minimization of physician resistance by creating a data entry system that is easy to use without increasing time burden will improve compliance and use of the SDES.