Staff perceptions of organisational performance measurement implementation in a health charity

Abstract

The non-acute health charity sector forms part of the global health services industry. Organisational Performance Measurement (OPM) is fundamental for modern business to achieve sustained excellence yet is under-utilised by non-acute health charities. The Non-Acute Health Charity Performance Implementation Framework (NCPI Framework) was developed to support non-acute health charities undertaking OPM. A non-acute health charity case study undertook a 12-month OPM implementation process using the NCPI Framework. This study had three aims. Firstly, to understand the perceptions of staff employed by the case study organisation regarding their experience of the 12-month NCPI Framework implementation process, secondly to identify whether an interrelationship existed between the NCPI Framework’s implementation elements and the staff’s perspectives, and thirdly, identify areas of strength and improvement which could enhance the NCPI Framework’s effectiveness. Data were collected post implementation and analysed thematically from open-ended question responses from an anonymous survey and semi-structured interviews. Participants supported the introduction of OPM and valued the NCPI Framework’s structured implementation guidelines. Six interrelated themes were identified: clarity, adaptability, alignment, transparent communication, capability and accountability. Participants suggested adaptations to enhance the NCPI Framework’s impact. A culture aligned with organisational learning was identified as potentially having a positive impact on the NCPI Framework’s implementation. The NCPI Framework may also have relevance and application to the non-profit industry beyond non-acute health care and for use in program development and service quality initiatives.

Keywords

Charity organisational performance performance measurement qualitative research

Introduction

Groups of people form organisations to achieve common goals under the premise that more can be achieved together than acting separately.¹ Performance is key to achieving sustainable competitive advantage which occurs when an organisation “attains and maintains the factors that contribute to outperforming competitors over long-periods”.² Organisational Learning Theory (OLT) enables the complex interweaving of organisational environments, cultures, business management and leadership necessary to build the performance characteristics for sustainable competitive advantage.^1,3,4

Organisational Performance Measurement (OPM) is an approach that is well aligned to OLT. Through formal processes, “OPM supports organisations attain and maintain the factors that contribute to higher performance and service excellence”.² This in turn supports the realisation of sustainable competitive advantage.^5–9 Whereas program evaluations assess singular, point in time, activities within organisations, OPM enables assessment of the full spectrum of capability to achieve their potential and organisational purpose.^6,8

As described in recent publications, non-acute health charities are both not-for-profit organisations and health service providers,^2,10 and despite broad service scopes, share a number of common governance and service similarities.¹¹ They traditionally provide initial assessments and concentrate on non-hospital maintenance care health services over long periods as opposed to complex stabilisation.^2,10–13 Such organisations are well recognised in Western nations and developing countries and form part of the World Health Organisation’s Civil Society Initiative’s non-government health industry.^14–16 Colbran at al.² suggest this sector is growing in relevance and importance as it increasingly relieves governments and for-profit providers of workload.^3,15 They also posit that due to diverse stakeholder groups and multiple revenue streams; such organisations require astute management.^17–19 In the Australian context, The Australian Charities Report 2014²⁰ suggests there could be thousands of non-acute health charities such as Royal Flying Doctors, Cerebral Palsy Alliance, NSW Rural Doctors Network and Royal Far West, with a combined annual turnover in excess of $3 billion.

OPM is widely used by for-profit and government organisations.²¹ Health care organisations are also known to use OPM, yet charity and not-for-profit organisations have been slower to adopt it.^2,21 In their PRISMA informed systematic review of OPM in non-acute health charities, Colbran et al.¹¹ found that while OPM is used by some charities, OPM is under-utilised in the non-acute health charity sector.¹¹ The low level of OPM utilisation by non-acute health charities is surprising considering not-for-profit sectors face increasing demands for governance, accountability and service standards to be in-line with for-profit and government entities.^22–24 Research has considered barriers or factors that have limited OPM uptake by not-for-profits.^15,16,25 Some scholars suggest there are deeply ingrained cultural beliefs that restrict modern practices in not-for-profit organisations and cite beliefs that not-for-profit organisations will risk losing their special virtues if they adopt corporate models.^3,25,26 Others are concerned that corporate behaviours will undermine the need for collaboration with social and welfare services.^23,27

Successful OPM implementation requires tailored approaches to respond to the unique nuances of industry and sector types whether they are in for-profit, government or not-for-profit organisations.^8,28 Tailoring is necessary because OPM requires flexibility and adaptability to take into account the specificities of individual sectors and organisations and replicating OPM measures or implementation strategies does not guarantee success.^8,28–31 The lack of OPM by non-acute health charities may in part be as a result of the fact an evidence-informed model of implementing OPM in non-acute health charities, tailored to the unique nuances of the sector, has not existed in published literature.¹¹ In response, Colbran et al.¹⁰ developed the Non-Acute Health Charity Performance Implementation Framework (NCPI Framework, Table 1). The NCPI Framework was developed using OLT and action implementation methodology.¹⁰ These approaches fundamentally look to build and embed whole-of-organisation experience, and the culture and learning processes necessary to develop systemic commitment, investment, and renewal in knowledge and behaviour.^10,11 The NCPI Framework recommends 30 operating elements that are grouped into five implementation factors – Implementation Plan (9 elements); Commitment (5 elements); Organisation Understanding and Learning (8 elements); Alignment, Integration and Resourcing (5 elements); and Measures and Indicators (3 elements).^2,10 These step-by-step elements can be used as a checklist to inform the tailored design of an OPM plan within an individual non-acute health charity.

Table 1.

Non-acute health charity performance implementation Framework (NCPI Framework).¹⁰

FACTOR 1: Implementation plan

1.1 Existence of a formal organisation strategy

1.2 Development and endorsement of a formal implementation plan which utilises evidence

1.3 Translate organisational vision and strategy into tangible objectives and measures

1.4 Articulation of organisational strategy and implementation plan in a strategy Map

1.5 Go beyond short-term agendas and pay attention to medium and long-term objectives

1.6 Acknowledge and plan for deployment over extended time period

1.7 Ensure model is adapted to meet unique organisational realities and demands

1.8 Confirm who is responsible for which actions and activities

1.9 Undertake a test experience to improve the balanced scorecard measures and processes

FACTOR 2: Commitment

2.1 Secure full organisational support for implementation

2.2 Demonstrated and continued Executive and Senior management support

2.3 Organisational culture is appropriate and receptive to implementation

2.4 Creation of an implementation Steering Committee – potentially with mixed representation across staff

2.5 Appointment of an implementation Coordinator as a sole role or within existing role

FACTOR 3: Organisation Understanding & learning

3.1 Address any conceptual barriers within the organisation

3.2 Communicate organisational strategy, purpose, the implementation plan and its status and ensure staff are aware OPM IS for strategic management not just performance measurement

3.3 participation of staff in design and revisions

3.4 Facilitate periodic and systematic review, adjustment and improvement

3.5 provide feedback to learn about and improve organisational strategy and implementation plan

3.6 identify and support champions across the organisation

3.7 skills and tools in data analysis and management, and implementing feedback and learning systems in-place

3.8 Use team-based collaborative approaches among disciplines that done regularly work together

FACTOR 4: Alignment, integration & resourcing

4.1 identify and align strategic initiatives

4.2 Designate implementation within the organisation’s Business and Operational plans

4.3 Acknowledge and prepare for deployment resourcing investment

4.4 Integrate within existing management processes, governance mechanisms, policies and reporting systems

4.5 Cascading accountability in departmental and personal goals throughout organisation

FACTOR 5: Measures & Indicators

5.1 measures and indicators customised to the organisation to represent all dimensions of the organisation

5.2 Ensure shared vision amongst staff cohort (esp. Clinicians and managers) regarding priority measures and indicators

5.3 Ensure major objectives are set for each measure and measures and indicators are meaningful

An Australian non-acute health charity undertook a 12-month implementation process to introduce OPM. A tailored OPM implementation plan was informed by the NCPI Framework. A pre and post quantitative evaluation utilised the Gervais Program Evaluation Model³² to assess the effectiveness of the NCPI Framework. Results from Board members, staff and salaried clinical advisors feedback confirmed the NCPI Framework positively impacted the introduction of OPM into the case study organisation and significant organisational change was reported in both usability (functionality/ease of operation) and utility (satisfaction) of OPM in the case study organisation.^2,11

To complement the quantitative evaluation and further understand the NCPI Framework’s impact on the introduction of OPM in the case study organisation a qualitative study was undertaken to explore staff perceptions and lived experience. In relation to the process of the 12-month NCPI Framework implementation period, the aims of this study were to:

1. Identify staff perspectives of the key implementation factors for successful OPM implementation;

2. Identify if an interrelationship existed between the NCPI Framework’s implementation elements and participant’s perspectives of the key factors that contributed to successful implementation of the OPM process;

3. Identify areas of strength and improvement which could enhance the NCPI Framework’s effectiveness.

Method

This study forms part of a pragmatist evaluation study³³ that used a convergent parallel mixed methods design.^33,34 The evaluation, reported by Colbran et al.,² assessed the 12-month implementation of an organisational performance process using the NCPI Framework in a case study non-acute health charity. The pragmatist paradigm was deemed useful because it encourages a practical solution mindset pragmatism and draws information from diverse sources.³³ As cited by Colbran et al.,² this aligns with the dynamic nature of organisations, the need to consider their multidimensional reality along with the need to overcome OPM’s existing evidence gap. The case study organisation operates in the geographical territory of New South Wales (Australia) working in the area of rural and remote health. Due to the dynamic nature of organisations and, as described by OLT,¹ the complex factors that can enable sustainable competitive advantage, the study utilised a qualitative evaluation using phenomenological research design. This was deemed valuable in this study’s context so that nuanced insight from lived experience and contextual understanding could be attained and staff motivations, actions and understanding could be identified and arranged as learnings to inform future implementation.^35–38

Instruments

Following the case study organisation’s 12-month OPM implementation period using the NCPI Framework, data was collected anonymously from Board members, staff and salaried clinical advisors through semi-structured interviews and open-ended questions included within a quantitative survey. The use of multiple data collection methods promotes more accurate descriptions of lived experience and is consistent with qualitative phenomenology³⁹ however as suggested by Serry and Liamputtong⁴⁰ and Creswell et al.³³ rigorous and consistently applied data analysis techniques are important for sound interrogation of such qualitative tools. Semi-structured interviews are used regularly in descriptive phenomenology, and more recent evidence is suggesting that the age of electronic mediums that open-ended survey questions can also be of value especially where digital surveys are utilised.^41,42

Triangulation, a recognised strategy within qualitative research, was used to develop a comprehensive understanding of the data by testing the validity of the different data sources and the data sets alignment and variations.⁴⁰ Further interpretation of data patterns was undertaken so that themes could be documented. This was achieved by using the inductive reasoning framework suggested by Serry and Liamputtong.⁴⁰ Both sources of information were conducted in-line with the case study organisation’s Research, Data and Privacy policy protocols overseen by the General Manager. Ethics approval was obtained from Deakin University, Australia (approval number HEAG-H 89_2017). Staff were invited to participate on a voluntary basis and anonymity of all responses was assured in-line with the study approved ethics methodology.

Semi-structured interviews

Semi-structured interviews encourage more natural discussion between the participant and the researcher and can draw out deeper insight and perspectives.^43–45 Qualitative studies can be compromised by the experience and skillset of researchers.⁴⁶ To manage potential risks, an interview guide was developed by the research group. The guide included a plain language statement, further background on the study and its purpose, the four main questions noted below:

(i) What were your perceptions of organisational performance measurement (OPM)?

(ii) What were the important factors for successfully implementing OPM?

(iii) Were there any challenges/gaps that impacted on implementation?

(iv) If you observed any changes in the organisations as a result of implementation of OPM, what were they?

The guide also included suggested techniques for follow-up questions, clarification and probing.⁴⁰ This was planned to gain depth to recounted experiences and more detailed perceptions. As previously reported by Colbran et al.,² the research inquiry concentrated on the introduction of the process, the approach taken, the participant’s observations about the ease of implementation and the strengths and challenges associated with the process. Other aspects of interest included the level of communication and information provided, the participant’s views about the key success factors, their observations of any impacts or outcomes as a result of implementing the organisational performance process and suggestions as to next steps.² 24% of staff (15 of 62) participated in 30–45-minute interviews. The interviews were audio-taped using a digital App and then transcribed verbatim by a professional transcription service and de-identified. Three participants accepted the offer the review their transcript for accuracy, edit or add to their contribution.

Qualitative survey responses

The survey was administered post the 12-month implementation period as a voluntary and anonymous electronic survey to staff. The questionnaire, informed by the Gervais Program Evaluation Model,³² consisted of a set of 53 5-point Likert scale questions and five open-ended questions. The survey which was completed by 43 staff members (64% of the organisation’s 73 staff). Only staff responses to three open-ended questions were considered in this paper because they aligned with the interview questions and therefore added further depth to the analysis. These questions were designed to capture staff members views and insights about implementation of the OPM process:

(i) Are the any comments you would like to make about OPM at [case study organisation]?

(ii) Do you have any comments about the impact and value of the NCPI Framework and its effectiveness in supporting OPM at [case study organisation]?

(iii) Do you have any advice for [case study organisation] as it sets about determining how best to measure the organisation’s overall performance?

Participant Recruitment

An introduction to the study and invitation to participate was emailed from the CEO to all staff with a direct weblink to the electronic survey. The weblink site included a welcome note from the second author as independent research lead and a plain language statement which confirmed respondent anonymity and a consent form which needed to be completed before the survey could be commenced. The CEO’s correspondence also included an invitation to participate in the semi-structured interviews and a request for those interested to contact the second author directly in her capacity as an independent researcher. The study’s Plain Language Statement and research questions were emailed to those interested in participating and they were asked to provide written consent to being interviewed and for their interview to be digitally recorded.

Data Analysis

As noted earlier, an inductive thematic approach informed by Serry and Liamputtong⁴⁰ was used for data analysis of the interview transcripts and open-ended question responses. This thematic analysis enabled the authors to identify, analyse and document themes emerging from the data in order to make sense of participants narratives and written comments. These themes were then collated to increase understanding and to generate knowledge about the phenomenon,^47,48 and to report patterns within the data.⁴³

The thematic analysis process involved five steps. The first step of data analysis consisted of reading and re-reading each interview transcript and survey response thoroughly to select units of analysis. As suggested by Elo & Kyngas⁴⁸ a process of open coding was undertaken, whereby relevant quotes and recurring topics were highlighted, and any interesting ideas or thoughts were noted in the page margins by the second author. To ensure anonymity each data source was numbered and then checked to replace names with codes and any personal information was removed. As this study was focused on the implementation process, data relating to participant perspectives of the outcomes of the introduction of OPM to the case study organisation was recorded separately in preparation for a future outcomes analysis study proposed by the authors. The second step involved reviewing these highlighted topics, ideas and notes and developing them into codes, or higher order categories.⁴⁹ Codes were used to label and organise data from interview transcripts and comprised of either a word or a short phrase. As a checking mechanism, and to ensure credibility of final results the first author independently coded four interviews and four qualitative post survey respondents and discussed the developing coding structure in an iterative fashion with the second author until agreement was reached. The analysis generated 163 categories or units of text.

In the third step, all codes were entered into a table and organised into themes and sub-themes. Codes with shared meaning and addressing similar topics were combined to form sub-themes. These sub-themes were then grouped together to determine an overall theme.

Step four involved compiling data relevant to each theme and sub-theme, ensuring that they could be reported and defined with clarity. In the fifth step themes were revised to confirm that they were an accurate interpretation of the data and had been assigned clear names and definitions.

Results

The results of the qualitative phenomenology data analysis suggested that organisational participants demonstrated strong awareness of OPM and the OPM implementation process. The participants were complementary of the NCPI Framework’s usability (functionality/ease of operation). They also reported “a high level of utility (satisfaction) within the implementation process”.² These results validated the earlier quantitative study of the case study organisation’s OPM implementation process. Importantly, as reported in the following section, the results demonstrate the study successfully achieved its three aims. Firstly, features that enabled the OPM implementation process were identified by staff, secondly factors for successful OPM implementation in the case study organisation and NCPI Framework elements were shown to have an interrelationship, and thirdly, participant feedback suggested areas of strength and improvement for the NCPI Framework.

“I think it was needed for lots of different reasons … we had nothing like it and … so I thought it was a really good idea.” (Female, respondent 05, lines 10–13)

“I think that it’s really necessary in any organisation to really be able to understand what you’re doing as an individual and how that aligns with the you know the overall plan of the organisation.” (Female, respondent 03, lines 11–13)

As described earlier, the NCPI Framework OPM implementation elements align with Organisational Learning Theory (OLT) and the value of creating organisational environments that promote experience, positive culture, learning and psychological safety. The findings of this study have shown an interrelationship between the NCPI Framework’s OPM implementation elements and the six primary themes for OPM implementation success as identified by the participants of the case study organisation. The six themes were clarity, adaptability, alignment, transparent communication, capability and accountability.

Key Implementation Themes

Clarity

The theme clarity summarised the focus for work effort, creation of an organisational compass to guide decision-making and investment and reinforced the importance of ensuring there was no confusion about the process within the organisation. One staff member considered having a clear focus to be important in planning to best meet the targets developed by the organisation:

“You know when done well and you’re using those plans to guide the work you do then it really does keep you accountable and help you to have sort of short and longer-term goal planning” (Female, respondent 03, 11–15).

Comments by staff explained how a greater understanding of organisational objectives not only gave them guidance for making decisions about their work, but also a clearer focus on the actions they needed to take to achieve success, and a greater sense of accountability for delivering on their targets.

“Yeah, I feel like that’s all really exciting and that helps us focus, focus our resources, focus our money, focus our energies … you become very invested” (Female, respondent 14, lines 236–8)

Some staff felt appreciation for the opportunity to be more engaged and aware of what has happening across the whole of the business.

“I now have line of sight across the organisation” (interview; male, respondent 13, lines 291–2)

Adaptability

This theme captured staff views about accepting implementation as a journey, an opportunity to reflect, making mistakes as learning opportunities, and building trust. Many of the participants highlighted and appreciated the creation and reinforcement of a cultural philosophy which accepted the implementation process was not perfect, acknowledged the need for adaptability and valued mistakes for their learning experience.

“this is new – we’re not going to get it right straight away – don’t be afraid to do it”. (Female, respondent 02, lines 88–9)

This in turn appeared to build a level of trust in the process and management’s overall approach. Dedicated and regular opportunities to reflect and make clear next steps before proceeding was cited as being particularly valued and contributed to the learning experience.

“There have been times to ask questions or put your hand up and say look I can’t or I don’t understand this or how does that tie in... Also, I don’t think it’s actually been rushed and it was conveyed slowly and it was implemented really well.” (Female, respondent 05, lines 47–50)

In-line with this sentiment, the majority of staff also liked the fact that this sense of organisational adaptability conveyed that there would be an opportunity to review and refine organisation, unit and individual targets if they needed to be changed because of new learnings during the year.

Alignment

The theme alignment captured participants desire for, and appreciation of, a structured approach to OPM which connected and engaged with all levels of the organisation and utilised common templates and tools.

“Tying the organisational goals with the team and individual goals has bought a synergy to the organisation that may have been missing in the past.” (Female, respondent 19, post survey)

The majority of participants saw the engagement of all people in the organisation in the implementation process, no matter their position, as critical to support implementation as too the development and use of consistent templates and tools.

However not all participants found the approach or alignment trouble-free.

“I felt quite confused by the processes involved throughout some of the process involved. There appeared to be a lack of clarity of the requirements in developing team and individual goals which I found frustrating.” (Female, respondent 46, post survey)

Transparent communication

Transparent communication described the need for regular information throughout the process. Effective and regular information and dialogue to all levels of the organisation was frequently cited in participant feedback. Communication is clearly important for success of the OPM process.

An important aspect of participant feedback was a desire for clarity around processes for documenting progress towards targets and feedback regarding progress and outcomes from organisational, unit, team and individual perspectives. At the organisation level this was thought to be done well during the implementation period with almost all participants remembering receiving information via emails, meetings and participating in a workshop about the process and how to set a good target.

“It was mentioned repeatedly and we’ve been kept up to date with how things are going… that’s more important for me than kind of keeping to a plan or keeping to a deadline.” (Male, respondent 01, lines 193–6)

The ability of managers to consistently and clearly communicate the OPM process and engage their staff in the process was cited regularly by staff.

Such feedback correlated with staff observations that manager engagement with OPM and proficiency in enabling the OLT environment was important. For example, prior to NCPI Framework implementation one staff member noted in the pre survey:

“Do not assume that every unit, manager, team and individual has the capability to set their own targets. Most need strong leadership and strong guidance to land on the right thing.” (Male, respondent 26, pre survey)

Capability

This theme captures the supportive culture and responsiveness of managers and staff. Efforts to build personnel capability in a supportive environment was consistently recognised in participant feedback. Participants commonly acknowledged the need for dedicated investment to support learning and improvement in competency and proficiency of managers and staff to enable the effective implementation of the process and its various elements.

Nearly all participants also noted the OPM process had enhanced their knowledge of the organisation and the work of other Units. This also encompassed references to the fact that a more positive culture and clearer values were developing which enhanced the organisational performance process.

“working on the values and the objectives of the organisation was really good and that was where I actually had a better idea of what it was and then from that felt a lot more confident about it” (Female, respondent 07, lines 30–32)

Feedback relating to communication from line managers suggested depth of discussion and clarity in unit meetings was variable. This appeared to impact on their overall confidence in the process, however not their commitment to OPM’s potential value to the organisation.

Accountability

The accountability theme includes evidence-informed targets, embracing accountability, and completion of performance processes for all. Participant feedback also suggested a shift on the organisation’s overall perceptions of the benefits of being accountable and an increased willingness to be accountable against the stated organisational objectives.

“It’s good to finally have something staff performance can be measured against. It’s great for accountability.” (Female, respondent 14, post survey)

Discussion

OPM implementation in non-acute health charities is a new phenomenon and this study’s most significant contribution is to provide the unique perspectives of the OPM process by participants in an organisational environment that has not previously been tested or evaluated. The analysis of the qualitative participant data offers researchers, policy makers and those practising OPM new evidence when considering existing OPM implementation literature. The six participant themes interrelate with several elements of the NCPI Framework as they demonstrate that staff within the case study organisation that implemented the NCPI Framework reported an organisational experience that enabled the systemic whole of organisation commitment and environment of psychological safety necessary or organisational learning.^{1,4,6,7,50,51} These themes could act as guiding principles, or a checklist, to further incorporate participant experience within a more comprehensive OPM implementation model.

In achieving the study’s aims, four underlying patterns central to the experience of staff of the case study organisation were identified. Firstly, staff had a strong desire and appreciation for structure, engagement of all people within the organisation and common templates and tools as identified within the alignment theme. This corroborates evidence suggesting structural aspiration in an organisation is associated with well-articulated organisational visions, missions and business performance.^29,30,52,53 In terms of the NCPI Framework, this is predominately enabled through Factor 1 – Implementation Plan and Factor 4 – Alignment, integration and resourcing, which promote the use of cascading targets for units, teams and individuals in response to the organisation performance objectives (NCPI Framework elements 1.3 and 4.5).

Secondly, the implementation themes of clarity, alignment, transparent communication and accountability correlate with literature that encourages embedding of OPM into standard ways of doing business.⁵³ Authors such as Bisbe and Barrubés²⁹ and Martello et al.⁵⁴ suggest these important factors occur through supportive organisational cultures, governance mechanisms, planning processes, staff appraisal systems and continual management reinforcement and staff engagement in the implementation design process. Such factors are encouraged within the NCPI Framework including the internal publishing of the Implementation Plan (Operating Element 1.2), confirmation of actions and who was responsible for those actions (Operating Element 1.8), creation of an internal project committee (Operating Element 2.4), participation of staff in Implementation Plan design and revisions (Operating Element 3.3), cascading goals and measures (Operating Element 4.5), and shared vision amongst staff cohort on priority measures and indicators (Operating Element 5.2).

Thirdly, participant feedback relating to the role of infrastructure, tools, systems, information technology, information synthesis and progress reporting mechanisms in supporting organisational learning were prominent in the clarity, transparency, capability and accountability feedback themes. Such themes are consistent with published literature which suggests learning organisations require open, unrestricted and free flow of information and knowledge at all levels to support practice as opposed to being used for control purposes.^50,52,55 Such aspects are recommended within NCPI Framework’s Operating Elements 3.8 (team-based collaborative approaches among disciplines that done regularly work together) and 4.4 (Integrate OPM within existing management processes, governance mechanisms, policies and reporting systems).

Finally, staff interest in dedicated learning support and skill development was identified within the capability theme. Organisational Learning Theory (OLT) promotes learning within organisational contexts as being fundamental for organisational performance, strengthening competitive advantage and enabling company survival.^4,56 Learning and skill development requires action through lived experience and not just theoretical thinking.^4,50 This is enhanced by creating trusting and safe team environments^4,52 and embedding active participation, knowledge acquisition, and cycles of continued quality improvement.^4,50,53–57 Elements within the NCPI Framework’s Factor 3: Organisation Understanding and Learning and Factor 4: Alignment, integration and resourcing, pay attention to these learning and skill development areas.

Aspects that Could Enhance the Impact and Value of the NCPI Framework

In addition to the underlying patterns noted above, participant feedback has also suggested aspects which should not be underestimated, or could strengthen, the NCPI Framework’s impact. Firstly, the inclusion of an NCPI Framework operating element that recommends that staff performance appraisal processes should be linked to an organisation’s OPM processes. This is implied within NCPI Framework Element 4.5 which refers to accountability of departmental and personal goals throughout the organisation. Regardless, participant feedback that the staff appraisal process must be linked to OPM is consistent with evidence that suggests it is possible to increase and support employee engagement through stronger performance management tools and techniques.⁵⁸

Secondly, many of the participants identified line-manager commitment and support being central to each of the six implementation themes. Further, it was perceived that a lack of manager commitment, understanding or capability may compromise NCPI Framework’s potential. This would be consistent with common management practices as leaders and line managers are ultimately responsible for building and maintaining cultures that promote learning and change, and ultimately turning organisational vision into concrete and successful operations^51,52,59 As is the case with organisational learning, the important role of leaders and line managers could be explicitly identified in an introductory statement to the NCPI Framework.

Thirdly, to reinforce the importance of the cultural aspects enabling OPM implementation^{50,51,55,60–62} an explanation of organisational learning and the link of this, and innovation, to enhanced organisational performance, competitive advantage and survival could be included as a precursor or introductory statement to the NCPI Framework.

Finally, it may be necessary to consider the NCPI Framework’s inclusion within a broader and more comprehensive OPM implementation model. Tactical frameworks have been criticised as being linear checklists and not able to drive the whole-of-organisation experience, culture and learning processes necessary to develop systemic commitment, investment necessary, and renewal in knowledge and behaviour.^1,4,35 Such environments require multi-layered implementation models which bring to life the relationship between theoretical reasoning and action through theoretical explanations, guiding principles, tactical frameworks and monitoring processes.^4,35,56

Potential for Broader Application

As OPM is still in its infancy within non-profit industry,²¹ the NCPI Framework, or a broader implementation model, may potentially have relevance and application for the non-profit industry. The six implementation themes identified in this paper could also be considered as guiding principles in frameworks developed to enable organisation change, or growth of organisational services and quality.

Limitations and Opportunities for Further Study

The findings of the study are promising however only relate to a single organisation within the Australian context. Exploration of the impact of provincial or national legislative and governance variations is necessary to strengthen the NCPI Framework’s global validity. Future studies may be strengthened by comparative assessment of participant perception post implementation to that of participant perspectives before, and during, the implementation period. Comparison between the qualitative data of participant perceptions with that of a quantitative evaluation data, using an evidence-based organisational change or program evaluation tool, would complement and strengthen the overall understanding of the NCPI Framework’s effectiveness.

Conclusion

Organisational success is demonstrated by performance and OPM is an under-utilised or under-reported phenomenon in this non-acute health charity sector. The NCPI Framework was developed to support OPM implementation by the sector. This study investigated staff perceptions of the implementation process of OPM using the NCPI Framework. It has reinforced the value of qualitative phenomenological research within convergent parallel mixed methods evaluation as it has offered more in-depth understanding of participant’s lived experience and enabled consideration of interrelationship between implementation framework and the perspective of participants.

The study’s three aims were achieved. Firstly, staff feedback was supportive of their experience with OPM implementation. Secondly, an interrelationship between the operating elements of the NCPI Framework and the staff’s perspectives of six key implementation factors for OPM implementation was determined. Thirdly, aspects that may improve or strengthen the utility of the NCPI Framework were identified.

The six implementation themes - clarity, adaptability, alignment, transparent communication, capability and accountability; are aligned to experience, culture and learning process factors of Organisational Learning Theory. They reinforce existing evidence and offer contextual and real-world perspectives of staff involved in the implementation of OPM in the non-acute health charities. In relation to improving the utility of the NCPI Framework, it needs to be considered within a comprehensive implementation model which promotes cultures that encourage organisational learning in OPM implementation. In this context, the NCPI Framework may also have relevance and application for the non-profit industry more broadly and for use within more tactical program development and service quality initiatives.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Richard William Colbran

References

Jones

. Organizational theory, design, and change. Upper Saddle River, NJ: Pearson, 2013.

Colbran

Ramsden

Pepin

, et al. Case study evaluation of organisational performance measurement implementation in a non-acute health charity. J Public Pol Adm 2022; 6(3): 139–150. DOI: 10.11648/j.jppa.20220603.14.

Mueller

. When doing good is just the start to being good: a possible tool to improve the organizational effectiveness of non-profit health care organizations. J Hosp Mark Public Relations 2007; 17(2): 45–60. DOI: 10.1300/j375v17n02_05.

Argote

. Organizational learning: creating, retaining and transferring knowledge. Berlin, Germany: Springer Science & Business Media, 2012.

Cacciatore

Kannengiesser

Carini

, et al. Balanced Scorecard for performance assessment in healthcare settings: a review of literature. Eur J Public Health 2019; 29(Supplement_4): ckz186.396. DOI: 10.1093/eurpub/ckz186.396.

Nalwoga

van Dijk

. Organisational performance measurement models, also for poverty alleviation. Int J Water 2016; 10122(2/3): 138. DOI: 10.1504/IJW.2016.075564.

Carneiro-da-Cunha

Hourneaux

Jr Corrêa

. Evolution and chronology of the organisational performance measurement field. Int J Business Perform Manage 2016; 17(2): 223–240. DOI: 10.1504/ijbpm.2016.075553.

Bititci

. Managing business performance: the science and the art. New York, NY: John Wiley & Sons, 2016.

Griseri

. An introduction to the philosophy of management. Thousand Oaks, CA: Sage, 2013.

10.

Colbran

Ramsden

Stagnitti

, et al. A framework to implement organisational performance measurement in health charities. J Public Pol Adm 2021; 5(1): 13. DOI: 10.11648/j.jppa.20210501.13.

11.

Colbran

Ramsden

Stagnitti

, et al. Advancing towards contemporary practice: a systematic review of organisational performance measures for non-acute health charities. BMC Health Serv Res 2019; 19(1): 1–2. DOI: 10.1186/s12913-019-3952-1.

12.

Department of Health and Human Services, State Government of Victoria [DHS] . Maintenance care, 2020. Retrieved March 15, 2020, from https://www2.health.vic.gov.au/hospitals-and-health-services/patient-care/rehabilitation-complex-care/maintenance-care/

13.

Myburgh

Brandborg-Olsen

Albert

, et al. The Nordic maintenance care program: what is maintenance care? interview based survey of Danish chiropractors. Chiropr Man Ther 2013; 21(1): 1–9. DOI: 10.1186/2045-709X-21-27.

14.

Dickerson

Grills

Henwood

et al. The world health organization engaging with civil society networks to promote primary health care: a case study. Glob Health Governance 2012; 6(1): 1–13.

15.

Soysa

Jayamaha

Grigg

. Operationalising performance measurement dimensions for the Australasian nonprofit healthcare sector. TQM J 2016; 28(6): 954–973. DOI: 10.1108/tqm-08-2015-0109.

16.

Hardwick

Anderson

Cooper

. How do third sector organisations use research and other knowledge? a systematic scoping review. Implement Sci 2015; 10(1): 84. DOI: 10.1186/s13012-015-0265-6.

17.

Park

Peng

. Advancing public health through tax-exempt hospitals: nonprofits’ revenue streams and provision of collective goods. Nonprofit Volunt Sector Q 2020; 49(2): 357–379. DOI: 10.1177/0899764019872007.

18.

Hung

Hager

. The impact of revenue diversification on nonprofit financial health: a meta-analysis. Nonprofit Volunt Sector Q 2019; 48(1): 5–27. DOI: 10.1177/0899764018807080.

19.

Hunter

. A systems approach to revenue fluctuations in non-profit human services organizations. Digital Commons @ ACU, Electronic Theses and Dissertations Paper 61, 2007. https://digitalcommons.acu.edu/cgi/viewcontent.cgi?article=1061&context=etd

20.

Cortis

Young

Powell

, et al. Australian charities report 2015.

21.

Aboramadan

Borgonovi

. Strategic management practices as a key determinant of superior non-governmental organizations performance. Probl Manage 21st Century 2016; 11(2): 71. DOI: 10.33225/pmc/16.11.71.

22.

Young

(ed). Handbook of research on nonprofit economics and management. Cheltenham, UK: Edward Elgar Publishing, 2018. DOI: 10.4337/9781785363528.

23.

Hyndman

McConville

. Trust and accountability in UK charities: exploring the virtuous circle. Br Account Rev 2018; 50(2): 227–237. DOI: 10.1016/j.bar.2017.09.004.

24.

Clancey

Westcott

. “This rabid fight for survival”: small NGO manager’s experiences of funding reform. Aust J Soc Iss 2017; 52(2): 163–179. DOI: 10.1002/ajs4.9.

25.

Boateng

Akamavi

Ndoro

. Measuring performance of non‐profit organisations: evidence from large charities. Bus Ethics A Eur Rev 2016; 25(1): 59–74. DOI: 10.1111/beer.12108.

26.

Newcomer

Baradei

Garcia

. Expectations and capacity of performance measurement in NGOs in the development context. Public Adm Develop 2013; 33(1): 62–79. DOI: 10.1002/pad.1633.

27.

Schmitz

. Is charitable giving a zero-sum game? the effect of competition between charities on giving behavior. Manage Sci 2021; 67(10): 5969–6627. DOI: 10.2139/ssrn.2862479.

28.

Gomes

Romão

. Sustainable competitive advantage with the balanced scorecard approach. In: Advanced Methodologies and Technologies in Business Operations and Management. Pennsylvania, United States: IGI Global.IGI Globa, 2019, pp. 1415–1428. DOI: 10.4018/978-1-5225-7362-3.ch106.

29.

Bisbe

Barrubés

. The balanced scorecard as a management tool for assessing and monitoring strategy implementation in health care organizations. Revista Española de Cardiología (English Edition) 2012; 65(10): 919–927. DOI: 10.1016/j.rec.2012.05.011.

30.

Niven

. Balanced scorecard: step-by-step for government and nonprofit agencies. New York, NY: John Wiley & Sons, 2008.

31.

Mossialos

Papanicolas

Leatherman

(eds). Performance measurement for health system improvement: experiences, challenges and prospects. Cambridge, UK: Cambridge University Press, 2009. http://www.who.int/management/district/performance/PerformanceMeasurementHealthSystemImprovement2.pdf

32.

Gervais

. A journey through five evaluation projects with the same analysis framework. Can J Program Eval 2008; 23(2): 165.

33.

Creswell

Klassen

Plano Clark

, et al. Best Practices for Mixed Methods Research in the Health Sciences. Bethesda (Maryland): National Institutes of Health, 2011, pp. 541–545.

34.

Creswell

. Steps in conducting a scholarly mixed methods study. DBER Speaker Series. 48, 2013. http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=dberspeakers

35.

Ricciardi

Cantino

Rossignoli

. Organisational learning for the common good: an emerging model. Knowl Manag Res Pract 2020; 19(3): 277–290. DOI: 10.1080/14778238.2019.1673676.

36.

Graham

McAleer

. An overview of realist evaluation for simulation-based education. Adv Simul 2018; 3: 13. DOI: 10.1186/s41077-018-0073-6.

37.

Yin

. Case study research and applications: design and methods. Thousand Oaks, CA: Sage publications, 2017.

38.

Matua

Van Der Wal

. Differentiating between descriptive and interpretive phenomenological research approaches. Nurse Res 2015; 6: 22. DOI: 10.7748/nr.22.6.22.e1344.

39.

Giorgi

. The descriptive phenomenological method in psychology: a modified husserlian approach. Pittsburgh, PA: Duquesne University Press, 2009.

40.

Serry

Liamputtong

. The in-depth interviewing method in health. In: Research methods in health: foundations for evidence-based practice. Oxford, UK: Oxford University Press, 2013, pp. 39–53.

41.

Groenewald

. A phenomenological research design illustrated. Int J Qualit Methods 2004; 3(1): 42–55. DOI: 10.1177/160940690400300104.

42.

Bowden

Galindo-Gonzalez

. Interviewing when you’re not face-to-face: the use of email interviews in a phenomenological study. Int J Doc Stud 2015; 10(12): 79–92. DOI: 10.28945/2104.

43.

Austin

Sutton

. Qualitative research: Getting started. Can J Hosp Pharm 2014; 67(6): 436. DOI: 10.4212/cjhp.v67i6.1406.

44.

Coffey

Atkinson

. Making sense of qualitative data: complementary research strategies. Thousand Oaks, CA: Sage Publications, Inc, 1996.

45.

Hycner

. Some guidelines for the phenomenological analysis of interview data. Hum Studies 1985; 8(3): 279–303. DOI: 10.1007/bf00142995.

46.

Yüksel

Yıldırım

. Theoretical frameworks, methods, and procedures for conducting phenomenological studies in educational settings. Turkish Online J Qualit Inq 2015; 6(1): 1–20. DOI: 10.17569/tojqi.59813.

47.

Cavanagh

. Content analysis: concepts, methods and applications. Nurse Res 1997; 4(3): 5–16. DOI: 10.7748/nr.4.3.5.s2.

48.

Elo

Kyngäs

. The qualitative content analysis process. J Advanc Nurs 2008; 62(1): 107–115. DOI: 10.1111/j.1365-2648.2007.04569.x.

49.

Burnard

. A method of analysing interview transcripts in qualitative research. Nurse Educ Today 1991; 11(6): 461–466. DOI: 10.1016/0260-6917(91)90009-y.

50.

Odor

. A literature review on organisational learning and learning organisations. Int J Inf Business Manag 2019; 11(3): 281–295.

51.

Guta

. An analysis of factors that influence organisational learning: the case of higher education institutions. Rome: 15th International Academic Conference, 2915. DOI: 10.20472/IAC.2015.015.067, https://www.iises.net/proceedings/international-academic-conference-rome/table-of-content?cid=10&id=067&rid=3387

52.

Som

Saludin

Shuib

, et al. Learning organization elements as determinants of organizational performance of non-profit organizations (NPOs) in Singapore. Int NGO J 2010; 5(5): 116–127.

53.

Schalm

. Implementing a balanced scorecard as a strategic management tool in a long-term care organization. J Health Serv Res Pol 2008; 13(1_suppl): 8–14. DOI: 10.1258/jhsrp.2007.007013.

54.

Martello

Watson

Fischer

. Implementing a balanced scorecard in a not-for-profit organization. J Business Econ Res (Jber) 2008; 66(99): 67–80.

55.

Andjelkovic

Boolaky

. Organizational learning: a case study of an international non-profit organization. Int J Business Adm 2015; 6(2): 124. DOI: 10.5430/ijba.v6n2p124.

56.

García-Morales

Jiménez-Barrionuevo

Gutiérrez-Gutiérrez

. Transformational leadership influence on organizational performance through organizational learning and innovation. J Business Res 2012; 65(7): 1040–1050. DOI: 10.1016/j.jbusres.2011.03.005.

57.

Carroll

Edmondson

. Leading organisational learning in health care. BMJ Qual Saf 2002; 11(1): 51–56. DOI: 10.1136/qhc.11.1.51.

58.

Mone

London

Mone

. Employee engagement through effective performance management: a practical guide for managers. London, UK: Routledge, 2018.

59.

Lipshitz

Friedman

Popper

. Demystifying organizational learning. Thousand Oaks, CA: Sage Publications, 2006.

60.

Jones

Harrison

Felps

. How applying instrumental stakeholder theory can provide sustainable competitive advantage. Acad Manage Rev 2018; 43(3): 371–391. DOI: 10.5465/amr.2016.0111.

61.

León

Bousquet

. A Performance Measurement Framework for NPOs. 2018 IISE Annual Conference, 2018. https://www.researchgate.net/profile/Hilda_Martinez_Leon/publication/325581735_A_Performance_Measurement_Framework_for_NPOs/links/5bc67a50299bf17a1c55d563/A-Performance-Measurement-Framework-for-NPOs.pdf

62.

Sen

Bingol

Vayvay

. Strategic enterprise management for innovative companies: The last decade of the balanced scorecard. Int J Asian Soc Sci 2017; 7(1): 97–109. DOI: 10.18488/journal.1/2017.7.1/1.1.97.109.