A multi-method quality improvement approach to systematically improve and promote the quality of national health and social care information

Abstract

Safe and reliable healthcare depends on access to health information that is accurate, valid, reliable, timely, relevant, legible and complete. National data collections are repositories of health and social care data and play a crucial role in healthcare planning and clinical decision-making. We describe the development of an evidence-informed multi-method quality improvement program aimed to improve the quality of health and social care data in Ireland. Specific components involved: development of guidance to support implementation of health information standards; review program to assess compliance with standards; and educating health information stakeholders about health data and information quality. Observations from implementation of the program indicate enhanced health information stakeholder awareness of, and increased adoption of information management standards. The methodology used in the review program has proved to be a robust approach to identify areas of good practice and opportunities for improvement in information management practices. There has been positive adoption of the program among organisations reviewed and acceptance of the proposed recommendations. Early indications are that this multi-method approach will drive improvements in information management practices, leading to an improvement in health and social care data quality in Ireland. Aspects of this approach may be adapted to meet the needs of other countries.

Keywords

quality improvement health information systems health policy information management data management health information management information management standards

Introduction

High-quality data and information is that which is relevant, accurate, reliable, timely, punctual, coherent, comparable, accessible and clear (Canadian Institute for Health Information (CIHI), 2009; Health Information and Quality Authority, 2018c; Liaw et al., 2013; Wang and Strong, 1996). Within health and social care, quality data and information are critical for the effective and efficient management of health and social care systems, including population health outcomes and health system costs (Cabitza and Batini, 2016; Kushniruk et al., 2013; Nabyonga-Orem, 2017). Driving improvements in the quality of health and social care data and information is an ongoing multifaceted challenge for many countries (Craig and Kodate, 2018; Kushniruk et al., 2013; Nabyonga-Orem, 2017). Challenges occur at many levels, including data collection, data processing and the national health system levels (Cabitza and Batini, 2016; Craig and Kodate, 2018; Kushniruk et al., 2013; Nabyonga-Orem, 2017). For instance, in Ireland, there are many different sources of health information, practices are not standardised and there is suboptimal use and sharing of health information (Craig and Kodate, 2018; Health Information and Quality Authority, 2011).

National data collections (NDCs) are national repositories of routinely collected health and social care data responsible for the collection, storage and dissemination of national health and social care information (Health Information and Quality Authority, 2017a). Many countries are working towards the integration of NDCs to address these challenges to improve health and social care data and information quality (Health Information and Quality Authority, 2014; Kushniruk et al., 2013). For instance, England has adopted a legislative route whereby one organisation, NHS Digital, carries out specific functions that are underpinned by the legal framework set out in the Health and Social Care Act 2012 (NHS Digital, 2018). NHS Digital’s functions include collecting, analysing and presenting national health and social care data; publishing guidelines and standards to shape how services are delivered; and designing, implementing and maintaining national data standards (NHS Digital, 2016). In contrast, Canadian Institute for Health Information (CIHI) is a voluntary and independent body whereby bilateral agreements between CIHI and individual provinces and territories support engagement with CIHI’s vision and mandate (Richards and White, 2013). Using this approach, CIHI engages in consultation, education and promotion with NDCs and other stakeholders to systematically assess, document and improve data quality for all data holdings (CIHI, 2003, 2009).

As outlined in Box 1, Ireland and England have a more fragmented health information system in place with more NDCs than in other countries. In addition, unlike other countries, NDCs in Ireland are housed within multiple managing organisations, as opposed to in one key organisation. They are also housed across various health sectors: public, private and charity (see Box 1) (Craig and Kodate, 2018; Health Information and Quality Authority, 2014). Therefore, a coordinated approach, led by a single organisation, the Health Information and Quality Authority (HIQA), was necessary to improve the health information quality within the fragmented Irish health information landscape. HIQA was established under the Health Act 2007 to improve health and social care services in Ireland (Health Act 2007, 2007). Similar to organisations responsible for improving national health and social care data and information, HIQA’s health information function is responsible for setting standards, developing guidance and recommendations and assessing compliance with standards for organisations under its remit (i.e. public sector organisations).¹

Box 1.

Summary of the similarities and differences in the approaches to NDC integration in selected countries: Ireland, Canada, New Zealand, Australia and England.^a

	Ireland	Canada	New Zealand	Australia	England
Population (million)	4.8	35.4	4.5	23.5	53.9
Health system	Mixed public and private	Universal, publicly funded	Mixed public and private	Mixed public and private	Universal, publicly funded
Number of NDCs	120	>28	15	>84	>121
National health information system structures	One or more NDCs held within one of 39 managing organisations. Managing organisations (and associated NDCs) are part of the public health sector, private health sector or other charities.	CIHI (n = 28) Additional NDCs held within Statistics Canada	Ministry of health (Information Group) (n = 15)	Australian Institute of health and Welfare (n = 84) Additional NDCs held within the Australian Bureau of Statistics and the Department of health.	NHS Digital (n = 121).^b Additional NDCs held within other charities and agencies, such as Public health England.
Organisation responsible for national health information quality improvement	HIQA (Health Information Function). Has a legislative remit for the public health sector.	CIHI	Ministry of Health	Australian Institute of Health and Welfare	NHS Digital

NDC: National Data Collection; CIHI: Canadian Institute for Health Information; HIQA: Health Information and Quality Authority.

^a The information presented in Box 1 is based on an international review of national health and social care data collections conducted by HIQA (Health Information and Quality Authority, 2016).

^b Previously Health and Social Care Information Centre (GOV.UK, 2016).

This article describes the development of an evidence-informed multi-method, novel quality improvement (QI) program that has been driving improvements in the quality of health and social care data and information in Ireland. We report key learnings to date and outline future work HIQA will conduct as part of the broader strategy to implement this program of work in Ireland.

Health information QI approach

The design of the QI program was informed by an international review of QI approaches to improve national health and social care data and information and by contextual factors specific to the Irish health information landscape (Craig and Kodate, 2018; Department of Health and Children, 2004). The international review found that multi-method approaches to QI may be effective at improving the quality of national health and social care data and information in Ireland (Health Information and Quality Authority, 2016). Individual components of the QI program aim to support implementation of HIQA information management standards (IMS) for NDCs (Health Information and Quality Authority, 2017). These include compliance monitoring, stakeholder engagement and education, and supporting tools and documentation compliance monitoring (see Figure 1). We found no evidence of the simultaneous use of these evidence-based QI strategies in other countries (Health Information and Quality Authority, 2016).

Figure 1.

HIQA’s approach to drive quality improvements in health information in Ireland. HIQA: Health Information and Quality Authority.

Review program to assess compliance with health information standards

HIQA developed a structured review program to assess compliance with the IMS (Health Information and Quality Authority, 2019b). The aim of the review program was threefold: to determine the information management practices of individual NDCs; to identify areas of good practice and areas where improvements are necessary and to work with NDCs to achieve compliance to the standards. In this way, the review program functions as a type of audit and feedback process, an effective and commonly used strategy to support guideline implementation and improve processes within organisations (Gagliardi et al., 2011; Powell et al., 2015). Program components were a prioritisation process for identifying the NDCs to be reviewed, use of an assessment and judgement framework to assist in the data collection and an analysis process using the triangulation of evidence (Health Information and Quality Authority, 2018b).

A review of NDCs within the Irish health and social care service is currently underway. Given the number of NDCs in Ireland after (see Box 1), the following prioritisation criteria were used to select key NDCs for review: (i) quality and safety impact, such as the contribution of the NDC to the quality and safety of health and social care services; (ii) policy impact, such as the potential of the NDC to inform planning and funding of services and to impact on the healthcare agenda; and (iii) other operational factors that may impact on the review program.

Review program process

The review program process is described in Figure 2. Each individual review within the program has four phases: data collection, data analysis, report preparation and QI plans. NDCs will be reviewed at a later date to assess compliance with review recommendations, to monitor changes in compliance with standards and continue to recommend areas for improvement. The assessment and judgement framework is kept updated at each stage of the review process. Review programme phases are detailed below.

Figure 2.

Process for reviewing national data collections.

Data collection

NDCs complete a self-assessment tool (SAT) of compliance with the IMS at the start of a review. The SAT is used to measure performance against the IMS, identify areas for improvement, inform the development of an improvement plan, review progress towards meeting the standards and prepare for an on-site assessment (Health Information and Quality Authority, 2019b). The assessment and judgement framework supports the evidence gathering process and guides judgments on compliance. It sets out the lines of enquiry the review team explore to assess compliance with the IMS (Health Information and Quality Authority, 2018a).

Once SAT data have been analysed, an information request is sent to the NDC under review (see Figure 2). The aim of the information request is to provide supporting evidence of the SAT data by showing the health data and information governance and processes in place in the organisation. Anonymised data sources requested include meeting minutes, organograms, risk registers, audits, key performance indicators, progress reports, policies and procedures and strategic and business plans. This allows for systematic identification of any gaps in the SAT, which can then be further explored at site-visits. Site-visits enhance HIQA’s understanding of high-level information governance strategies and day-to-day information management practices within the organisation. Visits involve a presentation of information management structures by the organisation, interviews with relevant staff and observation of data process activities.

Data analysis

The evidence and findings are analysed against the lines of enquiry as outlined in the assessment and judgement framework. Data are analysed using a triangulation approach to ensure findings are based on multiple, rather than on single sources of evidence (Flick, 2017). As well as validating findings, triangulation has been shown to increase the scope, depth and consistency in methodological proceedings (Flick, 2017).

Report preparation

HIQA collates results into a draft report outlining HIQA’s overall assessment, including a summary of key findings in relation to areas of practice that are working well and opportunities identified for improvement. HIQA provides each NDC the opportunity to provide feedback for factual accuracy of the draft report prior to publication. The assessment and judgement framework and any additional evidence supplied by the organisation guides changes to the report following the factual accuracy check. The final report is published on the HIQA website.

QI plans

Managing organisations are responsible for preparing and implementing QI plans. These provide assurance that recommendations from the review are prioritised and implemented. Follow-up reviews of the individual NDCs will assess implementation of QI plans.

Stakeholder education and engagement to improve health information quality

In 2017, HIQA held two national health information seminars entitled “Better data, better decisions.” The seminars were specifically targeted at the entire national health information community in Ireland, including those working in NDCs. Seminars were chosen as an appropriate educational method for the QI program as they offer attendees a wealth of information in one place, in a condensed period of time and also provide attendees with an opportunity to create networks with their peers.

The aim of the seminars was to provide, for the first time, a forum for representatives from NCDs in Ireland to meet and learn from each other. Keynote speakers included international and national information management experts that explained the benefits of data quality and discussed approaches to information management. In addition to the educational component of the seminars, they also provided a networking opportunity for attendees. To facilitate the continued education and engagement with seminar attendees, HIQA invited attendees to subscribe to HIQA’s newsletter, which has a dedicated section for information management.

QI tools and guidance documents to support the implementation of standards

In 2017, HIQA developed a SAT for national health and social care data collections based on its IMS. Engagement with the NDCs and also analysis of the findings of the SAT collected for the review program (as previously described) enabled HIQA to identify areas where guidance was required to support and enable NDCs to implement the IMS. Strategies to support implementation of the IMS included the publication of guidelines to support the development of a data quality framework (Health Information and Quality Authority, 2018c) and a Privacy Impact Assessment (Health Information and Quality Authority, 2017b). Informative documents and modules explaining the purpose and benefit of data quality (Health Information and Quality Authority, 2019a, 2019c) and information governance (Health Information and Quality Authority, 2012) have also been published.

Lessons learned

Early observations from the multi-method approach indicated enhanced health information stakeholder awareness of, and increased adoption of, the IMS. Tailoring a QI program to meet contextual needs is a key step in maximising its uptake (French et al., 2012; Kilbourne et al., 2007; May et al., 2007; Murray et al., 2010). The individual QI program components complement one another by meeting different contextual requirements; buy-in to the health information agenda across stakeholders and active support to implement the standards in practice. For instance, in the review program reports published to date, some organisations implemented components of the IMS prior to the review or planned to enact some or all of the recommendations made in the reports. In addition, consistent with the literature, targeted educational seminars and guidance documents enhanced stakeholders’ awareness of the IMS and encouraged adoption of them (Leeman et al., 2017). Taken together, this shows that as we work towards a more integrated health information landscape in Ireland, this QI approach serves to improve the information management practices of major NDCs in Ireland. Hence, there is early evidence that it is driving improvements in the overall quality of national health and social care data and information in Ireland.

Although the review program is in an early stage of implementation, HIQA is confident in the findings of the reviews published to date. The methodology used in the review program is proving robust in identifying areas of good practice and opportunities for improvement in information management practices of NDCs reviewed. Specifically, the four major NDCs reviewed to date vary in size, scope and organisational context. These factors, as well as organisational leadership, have influenced the time required to complete further reviews. Still, among the four NDCs reviewed, there has been a positive adoption of the program and acceptance of the proposed recommendations in relation to information management practices. Some key learnings have emerged in relation to the need for improvement in governance structures to support information management across those NDCs reviewed to date. The need for a strategic focus on information management has also been highlighted. Further research will evaluate the implementation and effectiveness of the QI program to improve information quality practices of NDCs in Ireland and the processes by which improvements are occurring (Craig et al., 2008; Oakley et al., 2006). Following completion of the four reviews, HIQA will undertake a composite review to determine the overall effectiveness of the program and to share system wide learning.

Conclusion

Assurance in relation to information management practices for NDCs is crucial to instil confidence in patients, clinicians and other stakeholders that healthcare decisions are based on high-quality information. In the absence of a coordinated approach to national health information, a QI program aimed at driving improvements in the quality of health and social care data and information has been implemented in Ireland. The approach targets common challenges to health information quality (Kuhn et al., 2007; Kushniruk et al., 2013), as well as those specific to the Irish context (Craig and Kodate, 2018; Department of Health and Children, 2004; Houses of the Oireachtas, 2017). Early outcomes indicate that this multi-method approach is driving improvements in information management practices, leading to an improvement in health and social care data and information quality in Ireland. Although the approach was designed to meet the needs of the health information landscape in Ireland, there will be learning in the approach for countries aiming to improve information management practices within their NDCs.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Niamh McGrath, MA

Barbara Foley, PhD

Rachel Flynn, MSc

Note

References

Cabitza

Batini

(2016) Information quality in healthcare. In: Batini

Scannapieco

(eds) Data and Information Quality: Dimensions, Principles and Techniques. Berlin: Springer, pp. 403–419. Available at: https://link-springer-com-443.web.bisu.edu.cn/chapter/10.1007/978-3-319-24106-7_13 (accessed 12 March 2020).

Canadian Institute for Health Information (2003) Earning Trust: Key Findings and Proposed Action Plan from the Data Quality Strategies Study. Ontario. Available at: https://wcmextprd.cihi.ca:8888/en/sites/default/files/document/dad_earning_trust_2003_en.pdf (accessed 12 March 2020).

Canadian Institute for Health Information (2009) The CIHI Data Quality Framework. Ontario. Available at: www.cihi.ca (accessed 12 March 2020).

Craig

Dieppe

Macintyre

, et al. (2008) Developing and evaluating complex interventions: the new medical research council guidance. BMJ 339: 1655. Available at: https://www.bmj.com/content/337/bmj.a1655.short (accessed 12 March 2020).

Craig

Kodate

(2018) Understanding the state of health information in Ireland: a qualitative study using a socio-technical approach. International Journal of Medical Informatics 114: 1–5. Available at: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S1386505618301734 (accessed 12 March 2020).

Department of Health and Children (2004) Health Information: A National Strategy. Dublin. Available at: www.reddog.ie (accessed 12 March 2020).

Flick

(2017) Mantras and myths: the disenchantment of mixed-methods research and revisiting triangulation as a perspective. Qualitative Inquiry 23(1): 46–57. Sage.

French

Green

O’Connor

, et al. (2012) Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implementation Science 7(1). DOI: 10.1186/1748-5908-7-38.

Gagliardi

Brouwers

Palda

, et al. (2011) How can we improve guideline use? A conceptual framework of implementability. Implementation Science 6(1). DOI: 10.1186/1748-5908-6-26.

10.

GOV.UK (2016) HSCIC changing its name to NHS Digital - GOV.UK. Available at: https://www.gov.uk/government/news/hscic-changing-its-name-to-nhs-digital (accessed 25 May 2020).

11.

Health Act 2007 (2007) Available at: http://www.irishstatutebook.ie/eli/2007/act/23/enacted/en/pdf (accessed 12 March 2020).

12.

Health Information and Quality Authority (2011) Identification of Themes for Standards for National Health Information Sources in Ireland. Available at: http://www.lenus.ie/hse (accessed 12 March 2020).

13.

Health Information and Quality Authority (2012) What You Should Know About Data Quality A Guide for Health and Social Care Staff. Available at: https://www.hiqa.ie/sites/default/files/2017-01/Data-Quality-Guide.pdf (accessed 12 March 2020).

14.

Health Information and Quality Authority (2014) International Review of Approaches Countries Have Taken to Integrate National Health and Social Care Data Collections. Available at: https://www.hiqa.ie/sites/default/files/2017-01/International-review-National-Health-Social-Care-Data-Collections.pdf (accessed 12 March 2020).

15.

Health Information and Quality Authority (2016) International Review of National Health and Social Care Data Collections. Available at: https://www.hiqa.ie/sites/default/files/2017-01/International-review-National-Health-Social-Care-Data-Collections-2016.pdf (accessed 12 March 2020).

16.

Health Information and Quality Authority (2017) Information Management Standards for National Health and Social Care Data Collections. Available at: https://www.hiqa.ie/sites/default/files/2017-02/Information-management-standards-for-national-health-and-social-care-data-collections.pdf (accessed 25 May 2020).

17.

Health Information and Quality Authority (2017a) Catalogue of National Health and Social Care Data Collections. October. Dublin. Available at: https://www.hiqa.ie/sites/default/files/2017-10/Catalogue-national-health-social-care-data-collections.pdf (accessed 12 March 2020).

18.

Health Information and Quality Authority (2017b) Privacy Impact Assessment Toolkit for Health and Social care. Dublin. Available at: https://www.hiqa.ie/sites/default/files/2017-10/Privacy-Impact-Assessment-toolkit-A5.pdf (accessed 12 March 2020).

19.

Health Information and Quality Authority (2018a) Assessment Judgment Framework for Designated Centers for Older People. Cork. Available at: https://www.hiqa.ie/sites/default/files/2018-02/Assessment-Judgment-Framework-DCOP_Guidance.pdf (accessed 12 March 2020).

20.

Health Information and Quality Authority (2018b) Enhanced Authority Monitoring Approach – Guidance. Cork. Available at: https://www.hiqa.ie/sites/default/files/2018-02/Enhanced-Authority-Monitoring-Approach_Guidance.pdf (accessed 12 March 2020).

21.

Health Information and Quality Authority (2018c) Guidance on a Data Quality Framework for Health and Social Care Health Information and Quality Authority. Dublin. Available at: https://www.hiqa.ie/sites/default/files/2018-10/Guidance-for-a-data-quality-framework.pdf (accessed 12 March 2020).

22.

Health Information and Quality Authority (2019a) Developing a data quality framework within health and social care – overview. Available at: https://rise.articulate.com/share/MJeQfRWJ1sLPYtXwJCbdhKzF17fy5N41#/ (accessed 12 March 2020).

23.

Health Information and Quality Authority (2019b) Guide to the health information and quality authority’s review of information management practices in national health and social care data collections. Dublin. Available at: https://www.hiqa.ie/sites/default/files/2017-03/Guide-HIQA-review-information-management-practices-in-national-health-and-social-care-data-collections.pdf (accessed 12 March 2020).

24.

Health Information and Quality Authority (2019c) Introduction to data quality – overview. Available at: https://rise.articulate.com/share/SOISyb4VnqIboZlA2R_cEJLnmhUKWr8l#/ (accessed 12 March 2020).

25.

Houses of the Oireachtas (2017) Houses of the Oireachtas Committee on the Future of Healthcare Sláintecare Report, May 2017. Dublin. Available at: https://assets.gov.ie/22609/e68786c13e1b4d7daca89b495c506bb8.pdf (accessed 12 March 2020).

26.

Kilbourne

Neumann

Pincus

, et al. (2007) Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implementation Science 2(1). DOI: 10.1186/1748-5908-2-42.

27.

Kuhn

Giuse

Lapão

, et al. (2007) Expanding the scope of health information systems: from hospitals to regional networks, to national infrastructures, and beyond. Methods of Information in Medicine 46: 500–502. DOI: 10.1160/ME9058.

28.

Kushniruk

Bates

Bainbridge

, et al. (2013) National efforts to improve health information system safety in Canada, the United States of America and England. International Journal of Medical Informatics 82(5): e149–e160. Available at: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S138650561200247X (accessed 12 March 2020).

29.

Leeman

Birken

Powell

, et al. (2017) Beyond ‘implementation strategies’: classifying the full range of strategies used in implementation science and practice. Implementation Science 12(1). BioMed Central Ltd. DOI: 10.1186/s13012-017-0657-x.

30.

Liaw

Rahimi

Ray

, et al. (2013) Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature. Elsevier Available at: https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S1386505612001931 (accessed 12 March 2020).

31.

May

Finch

Mair

, et al. (2007) Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Services Research 7. DOI: 10.1186/1472-6963-7-148.

32.

Murray

Treweek

Pope

, et al. (2010) Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Medicine 8. DOI: 10.1186/1741-7015-8-63.

33.

Nabyonga-Orem

(2017) Monitoring sustainable development goal 3: How ready are the health information systems in low-income and middle-income countries? BMJ Global Health 2(4). Available at: https://gh.bmj.com/content/2/4/e000433?cpetoc=&itm_campaign=bmjgh&itm_content=consumer&itm_medium=cpc&itm_source=trendmd&itm_term=0-A (accessed 12 March 2020).

34.

NHS Digital (2016) NHS Digital data and information strategy. Available at: https://digital.nhs.uk/data-and-information/nhs-digital-data-and-information-strategy (accessed 12 March 2020).

35.

NHS Digital (2018) Our role and remit in the health service – NHS Digital. Available at: https://digital.nhs.uk/about-nhs-digital/our-work/our-role-and-remit-in-the-health-service (accessed 12 March 2020).

36.

Oakley

Strange

Bonell

, et al. (2006) Process evaluation in randomised controlled trials of complex interventions. BMJ 332(7538): 413–416. Available at: https://www.bmj.com/content/332/7538/413.short (accessed 12 March 2020).

37.

Powell

Waltz

Chinman

, et al. (2015) A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implementation Science 10(1). BioMed Central Ltd. DOI: 10.1186/s13012-015-0209 -1.

38.

Richards

White

(2013) Ensuring the quality of health information: the Canadian experience. In: Sadiq

(ed) Handbook of Data Quality: Research and Practice. Berlin: Springer, pp. 321–346. DOI: 10.1007/978-3-642-36257-6_14.

39.

Wang

Strong

(1996) Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems 12(4): 5–34. DOI: 10.1080/07421222.1996.11518099.