Abstract
Object:
The purpose of this article is to analyse and compare the frameworks of performance measurement in primary health care in the world. The objective is to determine whether the frameworks of performance measurement in primary health care have an influence on the performance of health centres.
Method:
We conducted a systematic review of the literature to: (a) identify a conceptual framework for measuring quality management systems; and (b) assess the effects of the conceptual framework on quality improvement and quality of care outcomes. We chose frameworks highly cited in the literature and analysed and compared these frameworks.
Results:
Eight dimensions were identified for assessing performance in primary health care facilities (PHCFs) in more than 50% frameworks: effectiveness, safety, accessibility, equity, efficiency, acceptability, patient-centredness and timeliness.
Conclusion:
Even if the links are not established within the framework of a scientific research, quality approaches are generally recognised as an essential tool to help establishments improve their quality of care and the safety of their patients. Until now, the evaluation of the quality of care in the 'PHCF' is not yet in place, a blatant need for performance measurement tools, relevant information, coherence between the operational and strategic levels, integration of organizational objectives in the measurement of performance in order to direct the structures towards a true management by quality.
Introduction
Quality management has become an essential step in the industrial sector. Regarding social systems, experiences around the world are widely divergent, and there is no consensus on the appropriate manner and the effective and efficient tools. The field of health, recognised by its complexity, suffers from many problems. During the last few years, the improvement of quality of healthcare has become an absolute priority and a main objective for health systems all over the world (Berwick, 2004; OECD, 2011). However, the quality of care is a concept difficult to grasp seize because of the complexity of its evaluation (Piligrimienė & Bučiūnienė, 2008). The quality of health care has multiple dimensions (effectiveness, efficiency, ...), and the points of view may differ depending on the manager's perspective In addition, the healthcare quality dresses multiple dimensions and can be arrested according to various points of view (OECD, 2011). The organisation, configuration and delivery of healthcare services affect the performance of the overall health system (Veillard et al., 2005). The World Health Report identified three overall goals of a healthcare system: achieving good health for the population, ensuring that health services are responsive to the public and ensuring fair payment systems (Veillard et al., 2005; WHO, 2000). The measurement of quality of healthcare becomes essential to improve the quality, to reduce errors and to boost efficiency (Hogg, 2008). Quality of care has three main objectives: (a) empowerment of people receiving health care services; (b) development of more appropriate health care policies that; and (c) provide positive feedback to the recipients of services and other stakeholders (Veillard et al., 2010).
Now, considerable progress has been made in assessing the quality of care and implementing a wide range of quality improvement strategies, such as accreditation systems, organisational quality management programmes, clinical audit, patient safety systems, clinical practice guidelines, quality improvement collaboratives, performance indicators and systems for getting patient views (Groene et al, 2013). In spite of the efforts regarding research on the quality of care, a number of questions remain to be asked: Do quality approaches improve the performance of healthcare providers, and what is the appropriate method to measure performance regarding quality of care?
Concepts and Definitions
There are numerous definitions of performance (Franco-Santos et al., 2007) because of its complexity (Adair et al., 2003; Campbell et al., 2000) and multidimensional (Campbell et al., 2000; Sicotte et al., 1998) and paradoxical nature (Da Silva et al., 2011). In addition, there are several fundamental dimensions to evaluating the performance of health and social services (Smith & Papanicolas, 2013), as well as several ways to measure performance (Sicotte et al., 1998). Campbell identified seven themes as the key domains for the development of performance indicators: experience of the patient, clinical activity, development of services and innovation, access, promotion of health, cost-efficiency and the quality of life (Campbell et al., 2000). The health system tries to drive improvements in six domains of quality: (a) effectiveness; (b) efficiency; (c) accessibility; (d) patient-centredness; (e) equity; and (f) safety (IOM, 2001; WHO, 2006). The process-oriented conceptualization of performance shows the relationship of patient or professional values to quality, goals, and adaptation to the environment (Adair et al., 2003). The German Health Practice Collection considered the same elements for the evaluation of health care organizations: effectiveness, efficiency, accessibility, equity, and patient-centeredness. (GTZ, 2011). Therefore, most definitions of quality are similar, except that sometimes acceptability, sustainability and the experience of patients are added (Da Silva et al., 2011).
Performance Measures
Several models of performance measurement were developed, which are based on the integration of the following dimensions: experience of the patient, clinical activity, development of services and innovation, access, promotion of health, cost-efficiency and the quality of life (Da Silva et al., 2011). According to Arah (2006) and Grimmer (2014), Mmultiple domains are assigned to the quality of care: acceptability, accessibility, appropriateness, care environment and amenities, expenditure, governance, competency or capability, continuity, patient-centeredness, effectiveness, improved care, clinical focus, efficiency, safety, sustainability, and timeliness (Arah et al., 2006; Grimmer et al., 2014).
Methods
The Goal of the Study
The purpose of this article is to analyse and compare the frameworks of performance measurement in primary health care. Our primary research question is: Do the frameworks of performance measurement in primary health care systems have an influence on the performance of the health facilities? We chose frameworks highly utilised and cited in the literature. The conceptual framework must be based on a credible, relevant and transparent measurement system (McGlynn et al., 2003), and must take into account expected effectiveness and acceptance by health professionals.
Data Sources and Search Strategy
We conducted a systematic review of the literature to: (a) identify a conceptual framework for measuring quality management systems; and (b) assess the effects of the conceptual framework on quality improvement and quality of care outcomes. We opted for frameworks highly cited in the literature and analysed and compared these frameworks. We searched Medline and the Cochrane Library, including the Cochrane Database of Systematic Reviews (CDSR), for all studies on performance measurement of quality of primary health care facilities (PHCFs). The ranking is based on the number of citations in Google scholar, we chose the frameworks that are most cited in the literature, and then we proceeded to the analysis and comparison of these frameworks. The key search words are "conceptual framework", "performance measurement", "performance evaluation", "quality of health care" and "primary health care".
Inclusion Criteria
The studies included are those that address the measurement of quality of care performance and its impact. Outcomes included both clinical outcomes and process measures.
Exclusion Criteria
we excluded studies that deal with statistical analyses, and studies that deal with the financial aspect of quality of care such as cost-benefit, cost-effectiveness.
Eligible Papers
Eligible papers are described in terms of general characteristics (settings, type and level of respondents, mode of data collection), methodological properties (sampling strategy, item derivation, conceptualisation of quality management, assessment of reliability and validity, scoring) and application/implementation (accounting for context, organisational adaptations, sensitivity to change, deployment and effect size) (Groene et al, 2013).
Characteristics of Studies Included in the Scopus Review.
Results
Literature Base
The database search yielded 668 articles and 14 articles identified by other sources with a total of 682. About 447 were excluded due to duplication and selection criteria, 129 articles were excluded after reading abstracts and titles, 106 articles were fully reviewed of which 49 articles were excluded after full review of articles based on inclusion criteria, leaving 57 articles for inclusion in the review. The literature search chart is shown in Figure 1.
There are numerous challenges in carrying out international comparisons of health system performance, first among which is the limited availability of comparable data (Smith & Papanicolas, 2013). It advocates a multidimensional approach of hospital performance: All dimensions are considered interdependent and are to be assessed simultaneously (Veillard et al., 2005). A conceptual model of performance was elaborated to identify dimensions and sub-dimensions of performance (Veillard et al., 2005). Several dimensions have been cited in the literature: effectiveness, safety, responsiveness, accessibility, equity, efficiency, acceptability, appropriateness, competence or capability, continuity, timeliness and sustainability (Smith & Papanicolas, 2013; Table 2).
The ‘Donabedian model’ proposes four levels of evaluation: (a) the quality of care realised by the nursing staff; (b) the quality of patient care ; (c) the profit brought to the patients and to their families; and (d) the quality of service care at the level of a territory (accessibility, continuity, coordination) (Donabedian, 1988).
The Word Health Organization (WHO) model is based on efficiency, and ‘the results obtained according to the resources established the real measurement of performance of a health system’ (WHO, 2006). This approach was used more and more in the field of health when safety became a concern in the discussions about reforms of the health system (IOM, 2001). WHO’s strategic orientations encompass six interrelated dimensions: clinical effectiveness, safety, patient-centeredness, responsiveness of governance, staff orientation and efficiency.
‘The Kruk et al. model’ is based on the performance assessment of the structures of primary health care, through the analysis of four dimensions—effectiveness, efficiency, equity, and satisfaction—by taking into account the environment of an organisation (Kruk & Freedman, 2008).
The ‘Framework for Performance Assessment in Primary Health Care’ (FPA-PHC) is grounded in evaluation theory and explicitly identifies the processes of primary health care articulated by the WHO. It is based on Donabedian’s (1998) now-classic ‘structure’, ‘process’ and ‘outcome’ model for assessment of quality of care (Sibthorpe & Gardner, 2007). The FPA-PHC specifies the development of objectives that are focused on patients/families/communities and has four indicator levels relating to stewardship, organisational structures and processes, processes of care and intermediate outcomes (Sibthorpe & Gardner, 2007). As health systems strive to achieve improvements in quality, equity and efficiency, systems of performance assessment are being increasingly introduced. They have two principal uses: external accountability and internal quality improvement (Freeman, 2002).
The ‘EGIPSS’ model is a multidimensional model of performance affected by the efficiency and effectiveness of healthcare activities (Champagne, 2005). Performance is often understood as a concept that includes all the following notions: efficiency, effectiveness, responsiveness, productivity, quality, access and equity. The model is based on organisational performance (Champagne, 2005). The ‘EGIPSS’ model rests on the use of best-quality indicators to improve the performance of a system. These indicators have to do with the accessibility, continuity, global nature and productivity of the care. Each indicator is a reflection of the clinical and organisational practices, the availability of the resources and the particular environment. Hence, it must be analysed in this context.
The ‘Performance Assessment Tool for quality improvement in Hospitals’ (PATH) model has six dimensions, which were identified for assessing hospital performance: clinical effectiveness, safety, patient-centeredness, production efficiency, staff orientation and responsiveness of governance. The following outcomes were achieved: (a) definition of the concepts and identification of key dimensions of hospital performance; (b) design of the architecture of ‘PATH’ to enhance evidence-based management and quality improvement through performance assessment; (c) selection of a core and a tailored set of performance indicators with detailed operational definitions; (d) identification of trade-offs between indicators; (e) elaboration of descriptive sheets for each indicator to support hospitals in interpreting their results; (f) design of a balanced dashboard; and (g) strategies for the implementation of the ‘PATH’ framework (Veillard et al., 2005).
The ‘Health Care Quality Indicator’ (HCQI) is a project of the Organisation for Economic Co-operation and Development (OECD) which is aimed at developing a set of indicators for comparing the quality of healthcare across OECD member countries, requiring a balanced conceptual framework that outlines the main concepts and domains of performance that should be captured for the current and subsequent phases of the project (Arah et al., 2006). The ‘OECD’ model is divided into two main components: the service delivery of healthcare and the technical quality of clinical care (OECD, 2011). The measure of the yield in the primary care can serve two main objectives: improvement of quality and promotion of responsibility (OECD, 2011).
The ‘Institute of Medicine’ model conceptualises quality as having six dimensions: safety, timeliness, effectiveness, efficiency, equity and patient-centeredness (IOM, 2001). It leans strongly on the prospect of comparison for the measurement of performance. (Veillard et al., 2010).
Comparison of Performance Measurement Models of Quality of Care around the World According to Performance Dimensions.
Of the eight main quality performance measurement models in the international community in more than 50% are the dimensions of accessibility, effectiveness, efficiency, equity, patient-centeredness, and safety, and all frameworks use the dimensions of effectiveness and equity.
Comparison of Performance Measurement Models of Quality of care around the World According to Performance Criteria.
Discussion
The quality improvement process is not only beneficial to the health care system, but also remains a fundamental element for improving provider practices (Bradley, 2004). According to Starfield, measuring the quality of prevention strategies helps: (a) better encircle their operating mechanisms, as well as their advantages and potential risks; (b) measure their impact and their degree of adequacy; and (c) estimate their utility with regard to the reduction of disparities in health (Starfield., 2009). Bradley and Yuan (2012) reported that while certain countries failed to reach their goals, others were successful (Bradley & Yuan, 2012). Health systems in low-income countries are weak in terms of performance due to multiple challenges that can be grouped into five main themes: leadership and governance, organization, health information, financing, and human and material resources (de Savigny et al., 2009). Valuing effort is a motivating factor that affects the sustainability of performance among healthcare professionals (Dieleman et al., 2006). Inter-hospital comparison of performance improves care procedures. (Merle, 2009). However, participation in competitions with the aim of winning a prize can represent an incentive for professionals to improve their procedures and services (Milakovich, 2004).
In a developing country, measuring the health sector’s performance can be difficult, costly and controversial (WHO, 2000). The value of performance measures is diminished by questions of motive, intention and agenda (Solon et al., 2009). Evaluations of health system performance have only recently begun to include a critical measurement of system performance: the quality of healthcare (Arah et al., 2003; Solon et al., 2009). According to Arah et al. (2006), a good conceptual framework is particularly essential when there are societal requirements for fairness, transparency, accountability, performance attribution and rewarding of excellence (Arah et al., 2006).
Most of the frameworks reviewed in this chapter were constructed with different aims. For example, the EGIPSS (integrated performance model for the healthcare system) framework was constructed to provide conceptually sound performance models based on theory. The WHO (2000) and OECD (2011) frameworks were created in order to facilitate performance measurement and evaluation efforts. The ‘control knobs’ (2003), ‘systems thinking’ (2008) and IHP (2008) frameworks were constructed to evaluate specific health system reforms. While the purposes of these frameworks may differ, all frameworks go about achieving their ends by attempting to provide conceptual clarity in analytical, technical and operational thinking for the different stakeholders involved (Smith & Papanicolas, 2013). More than a package of services, Health centres have four core functions: comprehensiveness of promotive, preventive, curative and palliative care services; continuity across the life cycle; coordination across service providers and levels of the healthcare system; and access to a point of first contact for the majority of patients’ health needs (Veillard, 2017). These core functions, first articulated by Starfield, have been broadly accepted by and included in all ‘HCS’ frameworks. The influential work on quality of care by Donabedian, which emphasises interactions among patients, providers and communities as fundamental to quality, is also reflected in many models of health system performance (Veillard, 2017). Research into the effectiveness of certification and/or accreditation has become of great interest to many researchers in the field of quality of care (Greenfield & Braithwaite et al., 2008; Ovretveit, 2005; Pomey et al., 2004; Shaw, 2003; Walshe & Rundall, 2001). Some studies explored the impact of these programmes on the accreditation bodies (Devkaran, 2014; Pomey et al., 2004), and others have attempted to analyze certification performance by examining their structure and process (Devkaran, 2014; Greenfield & Braithwaite et al., 2008). Further complicating the problem is the failure to unify research methods on health care quality and performance measurement.
Performance management consists of the formulation of performance objectives, establishment and enhancement strategies, evaluations with regard to the infringement of these objectives, individual performance measures and a feedback system (Bititci et al., 2011). The implementation of a transparent performance measurement at all levels of the health system, with a close correspondence between the objectives of an intervention and the expected results, appropriate indicators, the impact measurement which takes into account the elements of performance likely to be managed by management (Figueras et al., 2008).
There are divergent and contradictory opinions as for the incidence of quality approaches for customer satisfaction. The existing studies do not contain rigorous and thorough analysis of the process of the conceptual frameworks and its relation with the performance results, quality improvement and safety of the patients (Greenfield & Braithwaite, 2008). Ovretveit and Gustafson (2002) found that the existing literature contained relatively little evidence regarding the overall effectiveness of quality interventions and quality standards in health care.
Several accreditation / certification bodies have introduced, developed, integrated and verified clinical quality indicators in health establishments. improvements were observed in the results of these organizations on the control performance of nosocomial infections (Devkaran, 2014).
Outcomes relevant to primary health care systems are linked to equity, patient and beneficiary satisfaction, costs and efficiency, links are revealed with governance, access, continuity, the coordination, efficiency and strength of primary care (Kringos et al., 2010). In the context of primary health care, the same advantages have been reported: 1) better quality control by reaching a minimum level of quality of service. 2) Better regulation while respecting legal and safety rules. 3) Continuous and lasting improvement of quality. 4) Providing information to patients, which enables them to make informed decisions. 5) insists on service standards to establish a positive image (Bueton & Wellingham, 2003).
Limits
The limitations of this study can be represented by the following aspects: (a) insufficient conceptual framework for assessing the reliability and validity of qualitative studies; (b) most of the PHCF assessment tools are not yet validated; and (c) the difference in economic and performance levels between countries limits the comparability of results.
Conclusion
The quality improvement approach is an essential tool for improving quality of care, however, there is no consensus on the most appropriate measurement framework.
Even if the links are not established yet within the framework of a scientific research, quality approaches are generally recognised as essential in helping establishments improve their quality of care and the safety of their patients. The conceptual frameworks proposed in quality approaches remain an essential lever for the promotion of quality approaches; however, lack of a valid framework complicates this action.
The current approach regarding measurement of performance of quality is far from establishment of an effective system of performance measurement and production of reports, due to the following reasons. First, the objectives are not well defined and are not accessible. Second, the existing approaches produce measures that are inconsistent, complex and unstable, imposing on the persons receiving benefits of care the uncertainty and burden of the contradictory measures. Third, healthcare facilities face certain important challenges, including the epidemiological transition in chronic diseases, the decentralisation of the health system and the erosion of the financing of the health sector.
Footnotes
Declaration of Conflicting Interests
Funding
The authors received no financial support for the research, authorship and/or publication of this article.