The impact of automatic history-taking software on data quality in the cardiology outpatient clinic: Retrospective observational study

Experimental design

Our planned experimental intervention utilized two methods for gathering and storing medical history data. The first method involved physicians delivering routine cardiology care and inputting their observations into the EHR. The second method employed expert system software trained to conduct interviews with patients attending cardiology examinations.

Study setting and recruitment of patients

This study is a single-canter (Atlas University Medical Faculty Hospital, Istanbul) retrospective, observational cohort study and included non-randomized eligible patients who visited seven general cardiology outpatient clinics between September 18, 2023, and October 27, 2023. These clinics were staffed by the same physicians who were unaware of the content and details of the study. The patient's inclusion and exclusion criteria can be listed as follows:

Inclusion criteria:

Patients who applied only for interpretation of previously ordered test results and for various reports and official documents.

Ethical considerations

Our study was conducted in accordance with the Declaration of Helsinki. Ethical approval was obtained from Atlas University Medical Faculty Non-invasive Scientific Research Ethics Committee (E-22686390-050.99-21532 Date: 08.11.2023). Written informed consent was provided by all participants during the primary data collection. Due to the retrospective nature of our study and the use of deidentified health information, the Atlas University Medical Faculty Non-invasive Scientific Research Ethics Committee approved this study without requiring additional patient consent.

CAHTS software

The logic of CAHTS software has been described in detail elsewhere.^20–26 In brief, CAHTS is a dynamic expert system for history taking from adults with acute and long-term complications from chronic disease. The knowledge base of questions and answers was developed in Turkish so that a native Turkish speaker with a primary school education could understand it. Textual questions have answer sets of Yes/No or Yes/No/Uncertain answers. Each answer is coded to indicate the question asked and the answer entered. In these algorithms, questions about the patient's medical history, substance use, dietary habits, drug allergies and intolerances, mobility, stress and sleep status, questions evaluating genealogical characteristics, and the complaint that led to the current admission (1) chest pain, (2) shortness of breath, (3) palpitations and arrhythmia, (4) high blood pressure and instability, (5) dizziness and fainting, and (6) ankle edema. The patient selected which of the cardiovascular symptoms he/she was experiencing, and under these headings, questions questioning the character of the symptom were presented on the screen in such a way that written and read aloud simultaneously, and the patient was asked in a guiding character by marking one of the options including yes, no, and sometimes quantity. In addition, the information selected in the algorithm was obtained from the patients’ e-Nabız systems. Laboratory tests (glucose, HbA1c, urea, creatinine, sodium, potassium, total cholesterol, low-density lipoprotein, high-density lipoprotein, triglyceride, alanine aminotransferase, gamma-glutamyl transferase, creatine kinase, thyroid stimulating hormone, uric acid) selected from the e-Nabız system were recorded with the date of the most recent value. In addition, if available, the results of echocardiography, myocardial perfusion scanning, chest radiography, computed tomography, magnetic resonance imaging and ultrasonography of the thoracic and vascular structures were recorded with the date of the most recent examination. Finally, the medications that the patients were currently taking were questioned and recorded together with the dose and method of use. In patients who did not bring their medications with them and could not remember their names, all medications previously prescribed to the patient were taken from the e-Nabız system, the package picture of the medication was displayed on the screen and asked if they were currently taking the medication on the screen. Filling in the pre-examination algorithms and scanning the e-Nabız information in the study protocol and adding it to the form were done under the supervision and with the help of high school students who were given basic training on this subject. The high school students explained to the patients the purpose of the research program and showed them how to interact with the CAHTS program running on an iPad (Apple Inc, Cupertino, CA, USA).

Patients were told that agreeing or refusing to participate would not affect their care. Patients were informed that the data collected by computerized interviewing would only be used for clinical research. At the end of these procedures, the patients’ responses to the questions in the computer-assisted history-taking system were organized to create a form called the Automated Assessment of Cardiovascular Examination (AACE), which systematically collects information about the patient's medical history and current disease history. After the questions and inquiries were completed, the AACE form was organized into four parts. In the first part, in line with the answers given to the questions inquiring the medical history and surname, the results that negatively affected the risk of cardiovascular disease were marked in red, and the results that had a positive or neutral effect were marked in green and included in the form. In the second part, the patient was asked to select the complaints that led to the current application, and the answers given to the questions within the selected complaint title were marked in green if yes, and in red if no, and added to the form. If the patient could not fit the complaint under any of these six headings, none of them was selected and noted on the form without additional questioning. In the third part, the results of the most recent laboratory tests and imaging tests selected in the algorithm in the e-Nabız system were written with the date of the available ones. Finally, in the fourth section, all medications currently used by the patients were written on the form together with the dose and method of use (Figure 1 a,b,c,d and Figure 2).

OPEN IN VIEWER

Figure 1.

a,b,c,d: Automated Assessment of Cardiovascular Examination (AACE) form created from a sample patient. 1a; interrogation of resume and personal historical data in relation to cardiovascular risk profile. 1b; interrogation of complaints that are the reason for the patient's application. 1c; the results of the most recent laboratory tests and imaging tests selected in the algorithm. 1d; all medications currently used by the patients.

OPEN IN VIEWER

Figure 2.

Algorithm used to generate AACE form from CAHTS.

Time-relationship between physician history-taking and CAHTS interview

In the first week, the AACE form was completed by the patients included in the study, but the form was not given to the physician. Each patient leaving the examination was questioned by the experienced cardiologist in charge of the study and the number of patients who were found to have incorrect, misleading, and incomplete information was recorded under the subheadings of the AACE form organized in four sections. In addition, the duration of the examination, the number of tests ordered, and the number of new medications prescribed for each patient were questioned and recorded. At the end of each day, all information filled in by the physicians in the electronic health record system was taken for examination.

In the second week, patients who applied to the same outpatient clinics were subjected to the protocol applied in the first week before the examination, and the forms prepared for seven physicians who were not informed about the details of the study were given before the examination of the patient. The physicians were asked to record the number of patients for whom the information was filled in the four main sections and their subheadings in the AACE forms were incorrect, misleading, and incomplete. After the examination, the duration of the examination, the number of tests requested, and the number of new medications prescribed were recorded. At the end of the study, a 5-point Likert-type scale (1 = extremely poor; 5 = excellent) consisting of questions in six areas (1. richness of content; 2. utility; 3. usefulness; 4. organization; 5. accuracy; and 6. understandability) was applied to the blinded physicians who performed the examination to question the effect of AACE forms on examination quality.

Statistical methods

This study used SPSS version 26.0 to analyze data collected from patients undergoing cardiology examinations and compared traditional physician-led history-taking methods with computer-assisted history-taking systems. Before analysis, the collected data underwent cleaning procedures to ensure data validity and reliability. First, the data set was analyzed by frequency analysis and no missing values were found. Outliers were identified by Z-test and the data set was cleaned. Then, Cronbach's alpha (a) value was examined to test the validity and reliability of the data set. After calculating (a) 0.861 for the data collected in the first protocol and (a) 0.844 for the data collected in the second protocol, the normality distribution of the data was examined. Skewness and kurtosis values were examined to examine the normality distribution, and it was seen that the values were within the range of ±1.5 and met the normality assumption. ²⁷ As a result, the t-test and ANOVA test were used for statistical comparison and the chi-square test was used to compare categorical variables.

Principal findings

We compared the AACE generated by a computer algorithm collecting patients’ data in a cardiology examination with records created by physicians during usual care. We found that computer-generated AACE was of higher quality, more comprehensive, better organized, and more relevant than records documented by physicians. The mean overall impression score for computer-generated EHRs was higher versus physician EHRs. These results provide evidence that a computer can generate clinically meaningful and useful forms of anamnesis, and the findings are consistent with recent evidence that patients are comfortable disclosing their health information to “virtual human” interviewers as supportive and “safe” interaction partners. ²²

In contrast, these results do not indicate that computers could ever replace healthcare providers. The art of medicine requires physicians to connect with patients, interpret complex and often confusing data, diagnose despite incomplete information, and communicate effectively. While the number of visits to physicians per capita has increased over the years, the insufficient increase in the number of physicians has led to a decrease in the average examination time per capita. ²³ This is particularly evident in developing countries such as Turkey. ²⁴ Inadequate time allocation for outpatient diagnosis and treatment has been shown to have negative effects on the quality of care. Short consultation times have been shown to adversely affect public health by leading to poor management of disease screening and inappropriate diagnosis and treatment.^25,26 Therefore, this study suggests that computers can remove at least some of the burden by collecting and organizing data to help clinicians focus on what they do best: practicing the human art of medicine.

Turkey made significant advancements in health information technology (IT) as part of the Health Transformation Program from 2003 to 2017. ²⁸ The Personal Health Information Record Systems, known as e-Nabız, developed by the Turkish Ministry of Health, are widely utilized in Turkey. The e-Nabız system allows users to access prescription, report, analysis-test, and imaging information, along with features for uploading additional health-related materials such as allergies and emergency notes. ²⁹ In Turkey, there is an application in EHR systems that enables physicians to access the patient's e-Nabız information during every examination. However, data on the rate and efficiency with which the information in e-health applications is used by healthcare professionals both in Turkey and in the world are very limited. ³⁰ In addition, the content structure of the e-Nabız application used in Turkey, which is not summarized and is a large mass of information, restricts its use by making it impractical to access the desired information in short outpatient clinic examination times. With the aim of optimizing the physician's utilization of the e-Nabız system and minimizing time loss, we extracted pertinent information deemed significant in cardiology examinations from the e-Nabız system and added them to the form.

Studies show that the quality of physician-created EHRs is highly variable, with many EHRs providing inaccurate or incomplete. ³¹ In our study, it was shown that the information filled in EHRs by physicians in the examinations performed by the standard method was quite incomplete and inadequate. On the other hand, it was observed that AACE forms filled this deficiency quite successfully. It was observed that most of the physician EHRs did not make any notes on the patient's lifestyle characteristics (mobility in life, nutritional properties, stress level, sleep quality and disorders), which have been shown to play an important role in the risk profile of cardiovascular diseases. There was either no or incomplete information in the physician EHR notes on family history of cardiovascular disease and Socioeconomic determinants. Even information on patients’ history of cardiovascular disease and other chronic diseases was insufficient. It was observed that computer-generated EHRs could be filled in correctly and appropriately in most of the patients and the rate of missing or incorrect information in the generated AACE forms was below 5% in most topics. It was also observed that the program was successful in extracting the results of selected laboratory and cardiovascular diagnostic tests planned in the study protocol from the e-Nabız system and summarizing them in the AACE form in patients who had e-Nabız system and allowed access to records. In physician EHR records, almost all patients (91.9%) did not have any information notes about previous diagnostic tests. Of course, this does not mean that the physician did not access these results from the e-Nabız system during the examination. Nevertheless, summarizing the results, if any, of selected tests in the AACE form within a short examination period was found to be practical, useful, and beneficial by physicians. In our study, the CAHTS program was applied to patients under the supervision of two high school students who were given a simple training. This increased patient compliance with the program. The fact that these supervisors were high school students shows that patients who are illiterate or who have difficulty comprehending the program are more likely to have a relative with high school education in their homes, indicating the applicability of the program and that there is no need for an expert supervisor. It is very important that patients’ current regular medications are accurately conveyed to the physician during the examination to organize the treatment correctly and to avoid harmful drug interactions. However, in practice, many patients do not bring their medications with them and often do not remember their names and usage patterns. In Turkey, all medications and reports previously prescribed to the patient can be accessed through the e-Nabız system. However, when the patient is questioned about which medications, he/she is currently using among all the medications prescribed, this takes a lot of time during the examination period and the patient often cannot remember the name of the drug being questioned. In the program we created, for patients who could not remember the medications they were taking, the previously prescribed medications were taken from the e-Nabız system, the package picture of the medication was displayed on the screen and the patient was asked if they were currently taking the medication on the screen. Thus, the list of medications used in only 9.6% of the patients in the AACE form could not be created properly. However, this rate is significantly higher in physician EHRs with 48.2%.

Physicians question the complaint that led to the current admission during the examination but do not record the summary of this questioning in sufficient detail in EHRs, possibly due to lack of time. This leads to serious diagnostic and therapeutic inadequacy in controls and subsequent examinations due to lack of data. Indeed, in our study, in 24.6% of the patients, the primary complaint was only included as a title, and it was observed that no query note was made about this complaint. Data reveal that clinicians spend up to one third of the patient encounter looking at a computer and, as a result, missing important non-verbal cues from the patient.^12,32 In addition to strengthening the physician–patient relationship, adequate eye contact is extremely important in terms of increasing patient satisfaction, better identification of health problems and improving service quality and effectiveness. The AACE algorithms tested in this study provide a model to begin to address these issues, as they generate EHRs informed by patient self-report and e-Nabız query, thereby increasing the validity of the EHR and reducing data inconsistencies. Furthermore, the use of computer-generated EHRs can improve the patient–physician relationship by informing the clinician before the patient encounter and saving more time by reducing computational tasks. We found that there was a 21% discrepancy between the most bothersome symptom self-reported by the patient and the main complaint documented by the physician. In addition, the rate of patients ticking more than one symptom on the AACE forms of the patients recruited in the first week of the study (61%) was significantly higher than the rate in the physician record system (30%). It is possible that some patients did not mention their most bothersome symptom to their physicians, or perhaps felt more comfortable reporting it to the computer.

When the AACE forms completed by patients attending outpatient clinics in the second week were given to the physicians before the examination, we found that the examining physicians felt that the computer-generated AACE forms were of better quality, more comprehensive, better organized, and more relevant than their own records. Although we did not have a method to measure it in this study, physicians expressed that they thought that obtaining these forms before the examination would help them diagnose patients more accurately and prevent possible incorrect or unnecessary treatment. They also stated that these forms made the examination more efficient and enabled them to perform a longer and more comprehensive physical examination. It has been shown that short examination times negatively affect public health by leading to poor management of disease screening and inappropriate diagnosis and treatment.^28,29 It is also important to show these positive results despite the shorter duration of the examination with AACE forms.

It is indisputable that knowing patients’ previous examinations will improve the quality of examination and reduce the rate of unnecessary tests ordered. In Turkey, patients’ past tests can be accessed through the e-Nabız system. However, as in our program, we tested the additional effect of summarizing selected tests and sections of all tests in addition to the existing e-Nabız system. In the examination accompanied by AACE forms, we found that while the frequency of ordering general laboratory tests, exercise stress testing, myocardial perfusion scanning, and general imaging tests was lower, the frequency of ordering gold standard tests such as coronary angiography and coronary computed tomography angiography was higher. We attributed this to improved examination quality and the ability to reach a diagnosis in a shorter and more precise way without the need for preliminary tests. We also thought that the fact that AACE forms reduced the number of new drug prescriptions would have a beneficial effect on both public health and cost-effectiveness.