A Multidisciplinary Delphi Consensus-Based Checklist to Define Clinical Documentation Tools for Both Routine and Research Purposes

Abstract

Background:

To the best of our knowledge, a strategic approach to define the contents of structured clinical documentation tools for both clinical routine patient care and research purposes has not been reported so far, although electronic health record will become more and more structured and detailed in the future.

Objective:

To achieve an interdisciplinary consensus on a checklist to be considered for the preparation of disease- and situation-specific clinical documentation tools.

Methods:

A 2-round Delphi consensus-based process was conducted both with 19 physicians of different disciplines and 14 students from Austria, Switzerland, and Germany. Agreement was defined as 80% or more positive votes of the participants.

Results:

The participants agreed that a working group should be set up for the development of structured disease- or situation-specific documentation tools (97% agreement). The final checklist included 4 recommendations concerning the setup of the working group, 12 content-related recommendations, and 3 general and technical recommendations (mean agreement [standard deviation] = 97.4% [4.0%], ranging from 84.2% to 100.0%).

Discussion and Conclusion:

In the future, disease- and situation-specific structured documentation tools will provide an important bridge between registries and electronic health records. Clinical documentation tools defined according to this Delphi consensus-based checklist will provide data for registries while serving as high-quality data acquisition tools in routine clinical care.

Keywords

managerial epidemiology managed care disease management documentation clinical medical informatics health-care research health-care records

Introduction

Clinical documentation of systemic diseases with multiple organ manifestations should be shared between all family physicians and specialists to allow a universal view of the patient for decision-making. Especially for patients with complex systemic diseases, it can be postulated that therewith quality of health care will improve in the same way as research collaborations raise scientific productivity.^1,2

In our center, for example, a group of specialists from dermatology, neurology, ophthalmology, rheumatology, urology, and vascular surgery has taken a special interest in Adamantiades-Behcet disease (ABD). The ABD is a rare systemic, autoinflammatory disease, potentially involving all vascularized organs.³ Because of our interdisciplinary interest for a multi-organ disease, the necessity of a shared clinical documentation system specific for ABD is evident.

The European League Against Rheumatism (EULAR) has developed a “checklist for reporting longitudinal observational drug studies in rheumatology,” designed to assist investigators developing structured registries.^4,5 For clinical routine, additional aspects may be necessary in order to provide adequate support for a multidisciplinary approach (like in ABD) and to optimize administrative tasks such as reporting of safety data.⁶ The use of electronic health records (EHRs) incorporating structured documentation tools (SDTs) has the potential to improve quality of care and patient safety.⁷ Such an EHR provides several advantages to the documenting physicians: (1) completeness of relevant items: consideration and documentation of all organ systems, symptoms, and findings possibly relevant; (2) comparability: internationally accepted disease activity scores can be embedded so that patients’ scores can be directly compared to benchmarks; (3) interconnectivity: coding can be integrated into the SDT, so that administrative tasks can be performed without additional workload for the physicians. As a benefit for research, anonymized SDT data can easily be accessed for epidemiological studies. Leveraging patient data to optimize both clinical outcomes and research is a highly important issue for additional research.

To the best of our knowledge, a systematic approach to define SDTs for both clinical routine tasks and research purposes has not been published so far, although SDTs may be viewed as a component of EHRs. Here, we present an interdisciplinary Delphi-based consensus for a checklist to be considered for the definition of items for disease- and situation-specific SDTs.

Methods

In brief, a questionnaire was constructed and a Delphi process with 2 rounds was performed as outlined in Figure 1. The questionnaires of both rounds were distributed in an international, interdisciplinary group of potential users as participants (students, residents, general practitioners, and specialists). The level of agreement was defined with a limit of 80% for acceptance by the group.

Figure 1.

Flowchart of methods applied.

This study did not involve patients for research purposes. It was a methodological approach on how to perform such research in the future. This work did not include patients, was a pure physicians’ and students’ activity without patients’ contact, and therefore was exempted from approval by institutional review board.

Questionnaire and First Delphi Round

The EULAR checklist “for reporting longitudinal observational drug studies in rheumatology”⁵ assisted in proposing the items to the Delphi participants. The questionnaire for the first Delphi round covered 3 dimensions, deemed essential for the checklist: (1) invitation of a competent working group, (2) content—selection of specific items, and (3) general aspects. The first part was designed to ask for the number and qualification of members who should develop and decide about SDTs (professions involved, leadership). The second part of the questionnaire asked for answers to content-related questions, which were considered necessary for clinical practice. It included 26 characteristics, which could be categorized as “obligatory,” “helpful,” “open,” “rather unhelpful,” “rather not obligatory,” or “don’t want to answer.” The third part included general questions. At the end, participants could propose additional questions for the second Delphi round.

Provisional Checklist Distributed in the Second Delphi Round

All answers of the first Delphi round which reached a consensus level of higher than 80% together with additional questions posed by the participants were included in a provisional checklist for the second Delphi round. The provisional checklist was again distributed to the same international group of participants.

Finalization of the Strategy in the Form of a Checklist

The final strategy was selected according to percentages of agreement with a final cutoff again set at 80%.

Statistical Analyses

Statistical analyses were performed using IBM SPSS Statistics 23 (IBM Corporation, Armonk, New York). For comparisons between the subgroups of participants, the 2-tailed Fisher exact test or χ² test was used. A P value <.05 was considered significant.

Results

Results of the First Delphi Round

A total of 33 persons from Austria, Switzerland, and Germany, including 19 physicians from different disciplines and 14 medical students, participated in the first Delphi round. Participants’ characteristics are summarized in Table 1.

Table 1.

Characteristics of Students of the Last 2 Years of Medical School (After First Experiences in Clinical Work) and of Physicians Participating in the Delphi Rounds.

Characteristics	Students (n = 14)	Physicians
Characteristics	Students (n = 14)	Residents (n = 6)	Specialists (n = 13)
Female sex (%)	31%	66%	46%
Age (≥50 years)	0%	0%	53%
Hospital based		100%	77%
Specialists (n)
Dermatology		-	4
Family medicine		1	3
Internal medicine		3	3
Ophthalmology		-	1
Pediatrics		1	1
Surgery		1	-
Urology		-	1

Part 1: Invitation of a working group

There was a high consensus (97%) that a working group should be setup for the development of any disease- and situation-specific SDT. Structured documentation tools individualized for single physicians should not be favored. Instead (1) adaptation by each group of practices/hospitals and each practice/outpatient service or (2) adaptation on a national or (3) an international basis for each group of practices/hospitals fulfilled the limit of 80% agreement (with 94%, 97%, and 88%, respectively).

Within the SDT working groups, at least 1 clinically active specialist should participate (91% agreement). Other professionals such as general practitioners (64%), nursing professionals (58%), physiotherapists/ergotherapists (30%), dieticians/social workers (18%), students (12%), patients/patients’ representatives (18%), lawyers (15%), and representatives of administrations (9%) and of insurance companies (6%) were not recommended as obligatory participants of the working group.

Regarding the lead of the working group, a specialist was favored by 94%. The specialty of the leader should reflect the focus of the working group. As a prerequisite, it was recommended that specific additional training is necessary (97% agreement), but subgroups of additional training like training as a mediator (64% agreement), being a lecturer/professor of a medical school (with 30%), or others did not reach the agreement of 80%. Students tended to prefer a trained mediator more often than physicians (with 79% and 53%, retrospectively).

Taken together, small- to medium-sized working groups (2-10 participants) were endorsed by the majority of participants. Other sizes of working groups did not reach agreement for the consensus.

Part 2: Content-related questions

Of 18 options, the following questions found the necessary agreement of ≥80% in the first round (Table 2):

The primary list of items should be exhaustive to allow the selection of final items by the working group. Selection criteria should be current evidence, availability, and affordability in the clinical setting.

Specifically, items should relate to (locally) established clinical pathways (even without agreement of the students), national and international recommendations, and the most important differential diagnoses.

Items should define disease activity, disease-related organ damage, and typical disease complications as well as specify disease-specific acute and long-term treatments (for easier documentation). Whereas physicians agreed to recommend disease-specific quality of life, students did not (84% vs 43%, respectively), so with 67% agreement disease-specific quality of life was not integrated into the consensus. Also, physicians emphasized items to specifically assess side effects of used medications, but these were not supported by the students.

Table 2.

All Questions for the First Delphi Round Are Listed.^a,b,c

Questions of first Delphi round	Total (N = 33)	Students (n = 14)	Physicians (n = 19)	P ^d
Primary item collection as large as possible	85%	71%	95%	.13
Selection by working group (eg, using Delphi-process)	64%	71%	58%	NS
Items should…
Relate to (locally) established clinical pathways	85%	64%	100%	.008
Relate to national recommendations	88%	86%	89%	NS
Relate to international recommendations	91%	86%	95%	NS
Relate to diagnostic/classification criteria	79%	79%	79%	NS
Relate to most important differential diagnoses	88%	79%	95%	NS
Define disease activity	94%	86%	100%	.17
Define organ damage	91%	86%	95%	NS
Define typical possible disease complications	82%	71%	89%	NS
Assess general life quality	61%	57%	63%	NS
Assess disease-specific life quality	67%	43%	84%	.02
Specify disease-specific acute treatment	85%	79%	89%	NS
Specify disease-specific long-term therapies	82%	79%	84%	NS
Assess side effects of used medication	73%	57%	84%	.12
Be collected from technical suppliers (thermometer, ECG, etc)	64%	64%	63%	NS
Allow photo/film documentation for assessing complex findings (eg, of skin and joints)	73%	71%	74%	NS
Allow retrospective data entry from medical charts	76%	79%	74%	NS

Abbreviations: ECG, electrocardiography; NS, nonsignificant.

^aAnswers of the participants that reached the acceptance rate of at least 80% are marked in bold and were used as a provisional checklist for the second Delphi round.

^bFirst Delphi round, percentages ≥80% are marked in bold letters.

^c P values show the difference between students and physicians.

^d p values <.2 were considered as a trend, <.05 as significant and <0.01 as highly significant.

Part 3: General aspects

Of the 8 general proposals, only 1 was selected as a possible recommendation for an SDT (with 91%), which was to assign each item to a workflow as performed by the physicians. All others like reduction of items to a lower number (better than too many items), linkage of items in different user-specific tools (for physicians, nurses, etc), links to (updated) literature, and guidelines and recommendations were not agreed to in the consensus. Before release to the market, physicians wanted to know the time spent for assessing all items, whereas this feature of a new SDT did not reach priority for the students in the first round (with 84% to 58%, respectively, and an average agreement of 73%).

The list of items agreed to in the first Delphi round by at least 80% of the participants (marked in bold letters in Table 2) was then used as the provisional checklist for the second Delphi round. In the first Delphi round, no additional questions were proposed by the participants.

Results of the Second Delphi Round and Finalization

The provisional checklist included all questions of the first round agreed upon by more than 80%. Nineteen participants responded to the second round.

Results are summarized as the final checklist in Table 3. The final average agreement was 97.4% ± 4% (84.2%-100.0%). Definition of the group size and the recommendation of an additional training of the lead were not supported (both with 74% agreement), whereas all content-related and all general (and technical) recommendations reached the cutoff point of 80% and were included in the checklist.

Table 3.

Final Checklist: Disease- and Situation-Specific SDT should Be Set Up by a Working Group (97% Agreement).^a

Checklist—Recommendations	% Agreement
Recommendations for the working group
Specialists’ participation recommended in each working group specialization	100.0%
Specialist recommended as lead of the working group specialization	100.0%
Definition of the topic: Definition of the proposed setting for the new SDT as…office, hospital, national, international setting	100.0%
Choose items by majorities in the working group	94.7%
Content-related recommendations
Primary collection of items as complete as possible (literature review)	100.0%
Items should be considered which refer to
Diagnostic/classification criteria	100.0%
Disease activity	100.0%
Organ damage	100.0%
Most important differential diagnoses	100.0%
(Locally) Established clinical pathways	100.0%
Typical possible disease complications	100.0%
National management recommendations if available	89.5%
International management recommendations if available	94.7%
Options for disease-specific acute therapeutic interventions	100.0%
Options for disease-specific long-term therapies	94.7%
General and technical recommendations
Assign items to users’ workflow (history, exam, management, etc)	100.0%
Provide simplified retrospective data entry integrated in software	94.7%
Specify time spent for assessing all items before release of SDT	84.2%

Abbreviation: SDT, structured documentation tool.

^aAll points of the checklist should be considered and realized by the working group (as agreed by at least 80% of the participants).

Discussion

This study provides the first data for the consensual definition of disease- and situation-specific SDTs both for clinical routine work and research applications. The recommended final checklist reached a high degree of multidisciplinary agreement in the Delphi consensus. We propose to use the checklist specifically for multidisciplinary documentation in systemic diseases such as ABD, which involve multiple organ systems and require close cooperation between different disciplines. With ethical and legal approval, anonymized data can then be used as easily as registries for research purposes.

Registries support an interdisciplinary approach and collect important initial and follow-up data for further research purposes. Thus, SDTs based on the same items used by registries will meet the challenge of “interoperability between health care and clinical research” when applied in clinical routine—without demanding additional resources for separate research documentation. Structured documentation tools may also provide the knowledge of clinical specialists to physicians not specialized in the specific disease, enabling a broader collaborative medical network and higher standards of care also by physicians not specialized in the specific disease.

The process of item selection for SDTs must be performed on the basis of validated literature reviews, which is then adapted for use in clinical practice. As this approach has not been directly covered by recent guidelines, this checklist aims at a new bridging strategy to support future development and updates of similar SDTs. The EULAR checklist “for reporting longitudinal observational drug studies in rheumatology”⁵ assisted in proposing the items to the Delphi participants. The items were not uniformly integrated into the final version, as the new checklist covers the development of SDTs for clinical routine work.

A potential limitation of this consensus procedure may be the high number of medical students as participants, as younger people may be more open to accept modern technologies. However, we included students in the last 1 to 2 years of medical school, thus anticipating the attitude of “future” physicians as possible users of SDTs. Nevertheless 53% of the specialists were still aged older than 50 years. Interestingly, the students’ answers were congruent with those of physicians in most points, with only 2 questions answered divergently between students and physicians in the Delphi process: First, 100% of physicians agreed to “integrate items of (locally) established clinical pathways,” versus only 64% of students (P = .008). Second, 84% of physicians agreed to “disease-specific items to assess life quality,” while only 43% of students agreed (P = .02). It can be assumed that students were not aware of the possible value of clinical pathways and life quality assessment so far or they may consider clinical pathways as additional work and life quality as not directly relevant for clinical decision-making.^8,9 As a consequence, it may be appropriate to give more attention to these aspects in medical school. Another potential bias could be the high rate of hospital-based physicians. We think that SDTs will be first introduced in hospital-based work until young residents bring these tools into their office. We are not aware of any bias introduced by the wording of questions or other elements of the process.

Conclusions

This “checklist for generation and update of disease- and situation-specific SDTs” provides consensual definitions of SDTs, both for routine clinical tasks and research purposes. We anticipate that using this checklist will increase satisfaction of physicians with “their” SDTs, both in routine clinical work and for research like epidemiological studies in the future.

Footnotes

Authors’ Note

V.C., H.P., and S.M. contributed to conceptualization, methodology, and project management. V.C. and S.M. contributed to investigation, data curation, formal analysis, and validation. V.C. contributed to visualization and writing—original draft preparation. V.C., H.P., and S.M. contributed to writing—review and editing.

Acknowledgments

The authors thank all participants of the Delphi rounds for their efforts, both physicians and students.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Michael Schirmer

References

Lee

Bozeman

. The impact of research collaboration on scientific productivity. Soc Stud Sci. 2005;35(5):673–702. doi:10.1177/0306312705052359.

Miller

. The cognitive revolution: a historical perspective. Trends Cogn Sci. 2003;7(3):141–144. doi:10.1016/S1364-6613(03)00029-9.

Gül

. Pathogenesis of Behçet’s disease: autoinflammatory features and beyond. Semin Immunopathol. 2015;37(4):413–418. doi:10.1007/s00281-015-0502-8.

Vandenbroucke

von Elm

Altman

. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297. doi:10.1371/journal.pmed.0040297.

Zavada

Dixon

Askling

; EULAR Study Group on Longitudinal Observational Registers and Drug Studies. Launch of a checklist for reporting longitudinal observational drug studies in rheumatology: a EULAR extension of STROBE guidelines based on experience from biologics registries. Ann Rheum Dis. 2014;73(3):628–628. doi:10.1136/annrheumdis-2013-204102.

Dixon

Carmona

Finckh

. EULAR points to consider when establishing, analysing and reporting safety data of biologics registers in rheumatology. Ann Rheum Dis. 2010;69(9):1596–1602. doi:10.1136/ard.2009.125526.

Schmajuk

Yazdany

. Leveraging the electronic health record to improve quality and safety in rheumatology. Rheumatol Int. 2017;37(10):1603–1610. doi:10.1007/s00296-017-3804-4.

Kinsman

Rotter

James

Snow

Willis

. What is a clinical pathway? Development of a definition to inform the debate. BMC Med. 2010;8:31. doi:10.1186/1741-7015-8-31.

Burckhardt

Anderson

. The Quality of Life Scale (QOLS): reliability, validity, and utilization. Health Qual Life Outcomes. 2003;1:60. doi:10.1186/1477-7525-1-60.