Data quality in primary care,Scotland

Abstract

Background and aims

This project explores primary care data quality (DQ) across Scotland.

Methods and results

A survey was sent to primary care staff in winter 2019. National data regarding Quality and Outcomes Framework (QOF) performance indicators and the GP software system used was obtained, analysed with T-tests and Chi-squared tests. Overall QOF performance with non-financial incentives from 918 practices was 77%. There was no significant difference with overall QOF performance against GP system (p = 0.46) or if the practice had a coder (p = 0.06). From the survey, search systems that make it hard to search for particular codes was the most important barrier to DQ; 61% of respondents (n = 491) felt there was particular information GPs should code, 16% of respondents stated that hospital discharge letters generally include corresponding codes and 9% for outpatient correspondence; 43% stated their practice had undertaken steps to improve DQ, training was the most common initiative, followed by workflow optimisation, dedicated coder(s), audit, guidelines and using code lists; 80% (n = 475) of respondents had received training in using their GP system, an average of eight years ago.

Conclusion

Obtaining improved GP systems, training, agreeing what GPs should code and improving transfer of data should be explored.

Keywords

Data accuracy electronic health records Health Services Administration primary health care surveys and questionnaires

Introduction

Primary care records are important because they provide a comprehensive longitudinal account of the health status and the care received. High data quality (DQ) is essential to enable delivery of healthcare services and accurate records enable strategic use of population health data for healthcare planning and research. In Scotland high-quality coded primary care data is required to allow accurate information to be shared via key information summaries, which provide communication across services.¹ Evidence suggests that there is a wide range of variability in DQ and accuracy of coding in primary care records.^2–4

Coding issues affect many diseases, historically there were 100-fold differences in rates of recording osteoporosis,⁵ more recently widespread increasing under recording of osteoarthritis⁶ and patients with biochemically confirmed chronic kidney disease are not all coded.⁷ Coding issues are present internationally, for example a Dutch study⁴ revealed that 30% of cases of cancer are not adequately coded on the electronic health record (EHR).

A lack of time to adequately code has previously been identified as a barrier to DQ^2,8 and when insufficient time is available, time with patients would be prioritised over recording data.⁹ Consistent data entry is challenging, clinicians and non-clinicians enter data and may face challenges overcoming ambiguity.¹⁰ Inadequate training,¹¹ lack of perceived benefit² and practitioners not seeing the value of adding structured coded data¹¹ have been noted.

In 2004 there was a national drive to standardise the management of certain conditions called Quality and Outcomes Framework (QOF). The process provided financial payments to general practices for managing and coding certain conditions in a particular way.¹² QOF led to a short term improvement to incentivised quality, with minimal further improvements over time.¹³ This suggests that certain disorders were at some stage being coded consistently across the county. In 2016 payment for QOF was abolished in Scotland, it was subsequently noted that when financial incentives for certain QOF conditions were withdrawn this led to an immediate reduction in the documented quality of care.¹⁴ It is plausible that practices may have invested less time in ‘tidying up’ their data for QOF indicators. Scottish QOF performance data was collected for the last time in 2019, which allows for an assessment of this data that could be a marker for DQ with no financial assessment. This study sought to explore DQ in Primary Care, Scotland and assess whether QOF performance data is correlated with IT system.

Methods

Information Service Division (ISD) Data: Two Data Sets were obtained from ISD in 2019. One data set contained all relevant QOF performance data for April 2018 until March 2019. The second data set contained details of the GP software system practices use across Scotland.

Survey: An electronic survey was designed based on known existing barriers to DQ^{2,4,8–11,15} in conjunction with relevant experts in this field and disseminated via email to multidisciplinary primary care across Scotland. Links to the survey were open from 24 October 2019 until 18 December 2019.

Results from the survey data and GP software system were analysed using QOF performance data as a reference point to assess if there were correlations. Statistical analysis occurred using Software R.¹³ Welch Two Sample t-tests for binary variables and Chi-Squared two-sample tests for equality of proportions with continuity correction were performed, were relevant.

As the survey was a voluntary survey to healthcare professionals and ISD data is in the public domain no formal Ethics Committee review was required.

Results

ISD data

Data was obtained from 918 general practices across Scotland for the financial year 2018/19, these had 99.6% complete data for the 22 unique QOF performance indicators studied. The distribution of performance from each QOF indicator and overall QOF performance is anonymised and represented in Figure 1. Combining these indicators the overall QOF performance was normally distributed with a mean of 77%, standard deviation 6, median 79, Q1 68% and Q3 87%. Although this seems high exclusion coding is possible so the target of 100% is achievable on QOF. As a comparison in England, in which financial QOF incentives remain in place, with similar measurements the overall mean achievement was greater at 96.5%.¹⁶

Figure 1.

Practice performance for each QOF indicator.

QOF performance data versus survey, coder and GP system

Respondents provided practice identifiable data from 261 unique practices. This allows average QOF performance to be compared with practices that did or did not have a responder to the survey. These were compared, as shown in Table 1 to assess if there was a selection bias from practices that responded to the survey, no statistically difference noted. The majority of practices who responded had a coder (76%, n = 258), usually, the coder has a non-clinical role. As shown in Table 1, practices with a coder versus no coder and clinical coders versus non-clinical coders both had similar overall QOF performance, with no statistically significant difference noted. In Scotland 486 practices used EMIS¹⁷ and 415 practices used Vision,¹⁸ likewise there is no significant difference in average QOF performance noted when comparing against the system used.

Table 1.

Overall QOF performance against practices that responded, practices that had a coder, clinical versus non-clinical coder and GP system use.

	Practice had one or more who responded to survey	Practice didn't respond or declined to information	Coder	No Coder	Clinical Coder	Non-Clinical Coder	EMIS	Vision
Number	261	657	195	63	11	183	486	415
Overall QOF mean	78.0	77.5	77.7	79.1	77.2	77.8	77.8	77.5
Standard deviation	5.2	5.9	4.8	5.1	5.3	5.1	5.6	5.9
Median	78.4	77.6	78.2	79.2	76.6	78.3	78.3	77.5
Interquartile range	75.2 to 81.6	73.9 to 81.4	75.0 to 81.2	76.2 to 82.7	73.5 to 80.2	75.1 to 81.2	74.5 to 81.4	73.9 to 81.6
Statistical analysisWelch Two sample t-test	t = 1.2, df = 541, p-value = 0.236, 95% CI: −0.31 to 1.25		t = −1.9, df = 106, p-value = 0.061, 95% CI −2.86 to 0.07		t = −0.3, df = 11, p-value= 0.752, 95% CI: −4.15 to 3.09.		t = 0.7, df = 859, p-value = 0.458, 95% CI −0.47 to 1.04.

Survey data

The survey was completed by 508 respondents; practice managers (PMs) (n = 210), general practitioners (n = 182), administrators (n = 47), nurses (n = 27), other (n = 22) and receptionists (n = 20). All groups stated DQ was extremely important. The survey asked respondents to rank the importance of six barriers to DQ, these are shown in Table 2. Overall, search systems that make it hard to search for particular codes were ranked as the most important barrier. Little perceived benefit from end users was the least important barrier. PMs ranked the main barrier as limited training, GPs ranked the main barrier as lack of time whereas administrative staff ranked the main barrier as the complexity of clinical cases seen.

Table 2.

Perceived barriers to data quality.

Job roles, number	Search systems that make it hard to search for particular codes	Inadequate training	Limited time available to code	Lack of lists of recommended codes to select from	Complexity of clinical cases seen	Little perceived benefit from end users
Administrator	3=	2	5	3=	1	6
GP	2	4	1	5	3	6
Nurse	1	3	5	2	4	6
Other	4	3	1	2	6	5
PM	2	1	5	3	4	6
Receptionist	1	4	2	3	5	6
Overall	1	2	3	4	5	6
rank =1 highest barrier, 6 = lowest barrier

The survey respondents stated that only 16% of corresponding codes are generally attached to hospital discharge letters and only 9% stated corresponding codes are generally attached to outpatient correspondence, as shown in Table 3. Subgroups of each job categories were compared against all other respondents combined and analysed with Chi-squared tests.

Table 3.

Are codes attached to Hospital Discharge or Outpatient Letters and is there particular information GPs should record.

Job roles	Do hospital discharge letters have code attached % Yes (n)	Do outpatient clinic letters have code attached % Yes (n)	Are there particular types of information for GPs to record % Yes (n)
Administrator	26% (43)	12% (43)	73% (44)
GP	19% (180)	6% (180) P = 0.01	54% (181) P = 0.02
Nurse	17% (23)	22% (23)	74% (23)
Other	20% (20)	15% (20)	65% (20)
PM	9% (203) P > 0.001	7% (203)	61% (204)
Receptionist	21% (19)	16% (19)	79% (19)
Total	16% (488)	9% (488)	61% (491)

Overall 61% (n = 491) of respondents felt that there was information that GPs should code themselves, GPs were significantly less likely to state that this. Relevant respondents were asked to (n = 271) elaborate about the types of information that GPs should code. The most common theme (n = 105) was data from the consultations that GPs perform, this included observations from assessing patients, interventions that were carried out and outlining specific patient requests. Coding diagnoses (n = 51) was mentioned frequently along with new diagnosis (n = 35). Medications changes were also frequently mentioned (n = 35). In general complex, sensitive, important information, including anticipatory care plans, key information summaries and assisting administrative staff by providing expert detailed coding input was noted in responses about what GPs should code.

Training

Overall 80% (n = 475) of respondents had received GP system training and 50% (n = 482) in data entry. GP system training usually was performed by GP system providers and data entry training by a broad mix of providers, both around the time of software implementation. Subgroup analysis of GP systems training by job roles indicated that GPs received significantly less GP system training than all other job roles combined (72%, n = 172, p > 0.001), whereas PMs received more (89%, n = 194, p > 0.001). For data entry training GPs had received less training compared to all other groups (27%, n = 179, p ≥ 0.001). Whereas more training in data entry had been received by administrators (70%, n = 43, p = 0.01), PMs: (64%, n = 199, p > 0.001) and receptionists (89%, n = 19, p = 0.001). Over half of the respondents had the desire for more GP system (54%, n = 472) and data entry (55%, n = 475) training.

Suggestions to improve data quality

A wide range of suggestions to improve DQ were received. Training was by far the most common theme, followed by code list subsets, improved secondary care coding, improved GP search functionality along with generally better GP IT systems. Table 4 lists the main themes in order of frequency. When reviewing responses it was clear that in some practices it was the GPs who were the crucial individuals attempting to improve DQ. These GPs led in practice training, produced in house guidance, mentored non-clinical staff when coding queries arose and highlighted the importance of quality data entry. Whereas in other responses it was clear that non-clinical staff such as coders or PMs were the key team members improving DQ, performing the above roles and attempting to educate GPs not to simply enter everything as free text.

Table 4.

Suggestions to improve data quality.

Suggestions to improve data quality	Suggestion count
Training	91
Code list subset	61
Secondary Care better coding	35
Better IT search system	34
Better GP IT system	23
Time	20
Guidance/Promote Benefits	19
Cluster/Regional/National Standardisation	16
Interoperability	16
Pick list/Template to help data entry	12
Audit	11
Dedicated Coder	11
Money	11
Other	11
SNOMED	10
Staff	9
No theme	6
Voice Activated System	6
Single IT system between primary and secondary care	5
Hardware/Software (not GP system)	3
Total	410
280 responses analysed (1 response: 5 suggestions, 7 responses: 4 suggestions, 18 responses: 3 suggestions, 70 responses: 2 suggestions, 177 responses: 1 suggestion, 6 responses: no suggestion)

Initiatives undertaken to improve data quality

Respondents were asked if their practice has instigated initiatives to improve DQ. A total of 422 responses were received, with over half the respondents stating that no data improvement initiatives had been instigated (No = 57%, Yes = 43%). Of those that had undertaken steps to improve DQ, training was the most common initiative, followed by workflow optimisation, dedicated coder(s), audit, guidelines and using code lists.

Discussion

Summary

Search systems that make it hard to search for particular codes were perceived as the main barrier to DQ, overcoming this barrier was frequently mentioned as a suggestion for improvement. Addressing this could be the priority for improving DQ across Scotland as this would facilitate data entry by both clinicians and administrative staff members. It is clear that there is no ‘one size fits all solution’ to improve DQ and improving search systems is not the sole solution. The survey data suggests there is a desire for further training in both system use and data entry. Most training has been performed almost a decade ago. Training was also the most frequent suggestion about how to improve DQ. When planning training distinctions between how the job roles ranked the barriers are worth bearing in mind, as training may need to be tailored for particular job types.

There have been efforts put in place to improve DQ by 43% of the participants that responded, many practices had recently completed work surrounding workflow optimisation, a toolkit produced by the iHub assisted this.¹⁹ The type of information that people thought important for specifically the GPs to code was diverse. It is worth seeking agreement in individual practices teams about what information GPs should code. Code lists, audit tools and standardised DQ tools that assess for variation across clusters were mentioned.

When QOF performance was compared against if a practice had a coder, clinical or non-clinical coder or which GP system was in use no statistically significant differences were detected. This is surprising, particularly comparing GP systems as there is evidence that the quality of the software can have an impact on data entry⁸ and usability is important,²⁰ it maybe that overall QOF performance is not a sensitive marker of DQ.

Although the focus of this study is on DQ in primary care, this study suggests the solution doesn’t rely solely on primary care, given the perceived low rate of adequate coding data from incoming correspondence from hospital and outpatient clinics.

Strengths and limitations

The main strengths of this study are that it uses the latest and most comprehensive datasets relating to national QOF data and GP system used. Actual behaviour is highly likely to have been captured, rather than this behaviour artificially changed due to the participants knowing that they are being studied.²¹ In addition the nationally distributed survey used some principles to encourage completion of electronic surveys²² and captured a large number of responses allowing statistical analysis to occur.

The main limitations of the survey component of this study is that it is performed via a convenience sampling framework^23,24 and an element of self-selection bias will be present, which is difficult to overcome.²⁵ In addition, how the survey invitation was distributed within individual general practice teams maybe a potential source of bias.

The main limitation of the data component of this study is that the datasets have not been validated. In previous years, when there was payment based on QOF results visits occur to some practices to validate data, this process ceased when payment desisted.

Comparison with existing literature

Improving DQ is known to be a complex process.¹⁰ Search systems that make it hard to search has been reported previously in literature as a known barrier to DQ.^2,8 To the author’s knowledge this has not been reported as the main barrier in Scotland previously.

The NASSS framework considers influences on the adoption, non-adoption, abandonment, spread, scale-up, and sustainability of patient-facing health and care technologies.²⁶ This notes the importance of the perceived value and staff within the context of the wider system, it is therefore notable that little perceived benefit from end users was ranked as the lowest barrier to DQ.

Training is a prevalent theme in this study, both in terms of what has been performed and what could be done. A literature review published in 2006 revealed that almost all interventions feeding back data to primary care led to an improvement in DQ,²⁷ therefore there maybe potential to combine training with a feedback intervention.

Previously it has been shown in Scotland that some Read codes are frequently used, whereas others are never used.³ In this study practices both had used and produced recommended code lists in an attempt to improve DQ, the ability for code lists to be managed and updated nationally would allow for standardisation of data entry.

Some practices mentioned using pick lists and templates to assist data entry. Templates are generally good at standardising data entry although should be used with caution as templates can change the nature of work performed.²⁸

Implication for research and/or practice

Currently a re-provisioning process of improved GP software systems is occurring in Scotland.²⁹ Newer GP systems have the potential to include improved search functions, particularly if user centred design is provided to make these systems user friendly.

Timing of training should be given careful thought as currently, in Scottish Primary Care, the clinical terminology in use is Read. Read is a coded thesaurus of clinical terms which has stopped being updated³⁰ and is due to be replaced by SNOMED CT.³¹ Given the financial cost of training it maybe wise to strategically delay some training.

Collaborating with secondary care colleagues and perhaps agreeing on code subsets for secondary care specialities and standardised communications could be beneficial. At present in Scotland most hospitals do not have an EHR. The process of implementing EHRs widely into hospitals across Scotland represents an opportunity to improve secondary and indirectly primary care DQ. True interoperability³² between primary and secondary care systems would overcome the need to re-enter data and is something to aim towards.

Supplemental Material

sj-xlsx-1-scm-10.1177_0036933021995965 - Supplemental material for Data quality in primary care, Scotland

Supplemental material, sj-xlsx-1-scm-10.1177_0036933021995965 for Data quality in primary care, Scotland by Christopher J Weatherburn in Scottish Medical Journal

Supplemental Material

sj-pdf-2-scm-10.1177_0036933021995965 - Supplemental material for Data quality in primary care, Scotland

Supplemental material, sj-pdf-2-scm-10.1177_0036933021995965 for Data quality in primary care, Scotland by Christopher J Weatherburn in Scottish Medical Journal

Supplemental Material

sj-xlsx-3-scm-10.1177_0036933021995965 - Supplemental material for Data quality in primary care, Scotland

Supplemental material, sj-xlsx-3-scm-10.1177_0036933021995965 for Data quality in primary care, Scotland by Christopher J Weatherburn in Scottish Medical Journal

Footnotes

Acknowledgements

The author would like to acknowledge Dr Ruth Claire Black who assisted with devising this project. In addition the author would like to thank SCIMP, SNUG and SPIRE DQ members for their support with the survey design. The author would also like to thank Public Health Scotland, formerly known as ISD for kindly providing data and also all those who completed the survey.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project received no external funding, although the project did contribute toward an MSc in Digital Healthcare Leadership run by Imperial College London. Fees were funded by Scottish Government and NHS Tayside.

ORCID iD

Christopher J Weatherburn

Supplemental Material

Supplemental material for this article is available online.

References

Craig

Morris

Cameron

, et al. An evaluation of the impact of the key information summary on GPs and out-of-hours clinicians in NHS Scotland. Scott Med J 2015; 60: 126–131.

de Lusignan

The barriers to clinical coding in general practice: a literature review. Med Inform Internet Med 2005; 30: 89–97.

Mukherjee

Wyatt

Simpson

, et al. Usage of allergy codes in primary care electronic health records: a national evaluation in Scotland. Allergy 2016; 71: 1594–1602.

Sollie

Sijmons

Helsper

, et al. Reusability of coded data in the primary care electronic medical record: a dynamic cohort study concerning cancer diagnoses. Int J Med Inform 2017; 99: 45–52.

de Lusignan

Valentin

Chan

, et al. Problems with primary care data quality: osteoporosis as an exemplar. Inform Prim Care 2004; 12: 147–156.

Jordan

Peat

Underrecording of osteoarthritis in United Kingdom primary care electronic health record data. Clin Epidemiol 2018; 10: 1195–1201.

Nitsch

Caplin

Hull

, et al. First national CKD audit report on behalf of the National CKD Audit and Quality Improvement Programme in Primary Care, 2017.

Ghosh

McCarthy

Halcomb

Perceptions of primary care staff on a regional data quality intervention in Australian general practice: a qualitative study. BMC Fam Pract 2016; 17: 50.

St-Maurice

Burns

An exploratory case study to understand primary care users and their data quality tradeoffs. J Data and Information Quality 2017; 8: 1–24.

10.

Morrison

Fernando

Kalra

, et al. National evaluation of the benefits and risks of greater structuring and coding of the electronic health record: exploratory qualitative investigation. J Am Med Inform Assoc 2014; 21: 492–500.

11.

de Lusignan

Hague

Brown

, et al. An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. J Public Health (Oxf) 2004; 26: 34–37.

12.

Lester

Campbell

Developing quality and outcomes framework (QOF) indicators and the concept of “QOFability”. Qual Prim Care 2010; 18: 103–109.

13.

Guthrie

Tang

What did we learn from 12 years of QOF?

Scott Sch Prim Care Ser Lit Rev 2016; 12.

14.

Minchin

Roland

Richardson

, et al. Quality of care in the United Kingdom after removal of financial incentives. N Engl J Med 2018; 379: 948–957.

15.

Lucyk

Tang

Quan

Barriers to data quality resulting from the process of coding health information to administrative data: a qualitative study. BMC Health Serv Res 2017; 17: 766.

16.

Merrifield

More practices achieving maximum QOF points pulse. Today, www.pulsetoday.co.uk/pulse-intelligence/more-practices-achieving-maximum-qof-points/20039771.article (2019, accessed 16 June 2020).

17.

EMIS Health. PCS clinical software for Scotland, www.emishealth.com/products/emis-pcs/ (n.d., accessed 16 June 2020).

18.

Cocking

General practice software and solutions from vision, www.visionhealth.co.uk/general-practice/ (2020, accessed 16 June 2020).

19.

Workflow Optimisation Toolkit. iHub, https://ihub.scot/project-toolkits/workflow-optimisation-toolkit/workflow-optimisation-toolkit/ (2020, accessed 9 December 2020).

20.

Middleton

Bloomrosen

Dente

, et al.; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc JAMIA 2013; 20: e2–e8.

21.

McCambridge

Witton

Elbourne

DR.

Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 2014; 67: 267–277.

22.

Edwards

Roberts

Clarke

, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009: MR000008.

23.

Marshall

MN.

Sampling for qualitative research. Fam Pract 1996; 13: 522–526.

24.

Merriam

Tisdell

EJ.

Qualitative research: a guide to design and implementation. 4th ed. Hoboken, NJ: Wiley, 2015.

25.

Bethlehem

Selection bias in web surveys. Int Stat Rev 2010; 78: 161–188.

26.

Greenhalgh

Wherton

Papoutsi

, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the Scale-Up, spread, and sustainability of health and care technologies. J Med Internet Res 2017; 19: e367.

27.

Brouwer

Bindels

PJE

Weert

HC.

Data quality improvement in general practice. Fam Pract 2006; 23: 529–536.

28.

Swinglehurst

Greenhalgh

Roberts

Computer templates in chronic disease management: ethnographic case study in general practice. BMJ Open 2012; 2: e001754.

29.

Scottish Government. General Practice (GP) surgeries switching their IT systems framework: FOI release, www.gov.scot/publications/foi-19-01556/ (2019, accessed 16 June 2020).

30.

NHS Digital. Read Codes. NHS Digital, https://digital.nhs.uk/services/terminology-and-classifications/read-codes (2018, accessed 16 May 2020).

31.

NHS Digital.SNOMED CT. NHS Digital, https://digital.nhs.uk/services/terminology-and-classifications/snomed-ct (2020, accessed 16 May 2020).

32.

Pronovost

Johns

MME

Palmer

, et al. Procuring interoperability: achieving high-quality, connected, and person-centered care. Washington, DC: National Academy of Medicine (U.S.), 2018.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.15 MB

0.74 MB

0.11 MB