Using non-participant observation to uncover mechanisms: Insights from a realist evaluation

Abstract

This article outlines how a realist evaluation of dementia care in hospitals used non-participant observation to support the refinement and testing of mechanisms likely to lead to the use of person-centred care. We found that comments and explanations of their actions from hospital staff during observation periods provided insights into the reasoning that generated their actions for care in real time. This informed subsequent data collection and analysis. Two worked examples of mechanisms first identified during non-participant observation demonstrate (1) how they were uncovered, and (2) how this informed research activities for theory refinement. Early, iterative engagement with the analytic process, primarily involving reflection and debate with the research team, maximised the potential of observation data to support surfacing underlying mechanisms, linking them to specific contexts and outcomes.

Keywords

dementia hospitals mechanisms non-participant observation realist evaluation

Introduction

Realist evaluation offers a theory driven approach to evaluation and is not defined by particular data collection methods. To quote Westhorp, ‘It begins by clarifying the ‘programme theory’: the mechanisms that are likely to operate, the contexts in which they might operate and the outcomes that will be observed if they operate as expected’ (Westhorp, 2014: 1). The assumption is that context is central to understanding how and why a programme works (or not) for different individuals and in different circumstances. Observation is a commonly used data collection method (Greenhalgh et al., 2009; McGaughey et al., 2017; Rycroft-Malone et al., 2009). Data from observations record naturally occurring events and interactions, complement evidence from other sources, such as interview data (Wye et al., 2014) and can highlight disparities between reported practice and actual practice (Marchal et al., 2010; Mays and Pope, 1995). Recording contextual factors, such as work routines, environmental, personal and social features, build understanding of the influence of structures and processes on the actions of healthcare staff and outcomes for patients (Walshe et al., 2012). Less clear is how observational data contribute to understanding the ‘reasoning’ of those who are making the decisions as a result of the programme or intervention, for example, patients and service providers. While the mechanisms generating responses during interactions are not visible (Westhorp, 2018), those involved in the interaction can provide insights into ‘why’ they respond in a particular way as the action takes place.

Our study of dementia care in hospitals aimed to explain how hospital staff were able to provide dementia sensitive care that improved patient and family outcomes, such as comfort, confidence in their care, reduced distress, length of stay and symptom management (Handley et al., 2018). Non-participant observation was one of several data collection methods employed during the realist evaluation. This article considers how observations of staff actions and their comments on what was happening were key to understanding how the programme worked and its success, or otherwise, was interpreted.

Realist evaluation

Realist evaluation is a methodological approach to research which is grounded in a realist philosophy of science (Bhaskar, 1978; Pawson and Tilley, 1997). Key to realist evaluation is that causation is understood as generative. Thus, outcomes are not the result of the implementation of an intervention, but rather explained through the context-mechanism relationships associated with the intervention. Realist evaluations of interventions start with initial theories that are then tested and refined through empirical data. Data collection and analysis in realist evaluations are theory-driven and adhere to realist principles (Emmel et al., 2018).

Interventions are conceptualised to work in a particular way to produce a desired outcome, but how staff respond to them in practice will vary (Pawson and Tilley, 1997). Realist evaluation seeks to explain how and why interventions lead to a range of outcomes by theorising underlying mechanisms and their relationship to context (Astbury and Leeuw, 2010). Mechanisms are understood to have three key characteristics:

They are often hidden;

They are context-dependent;

They generate outcomes (Astbury and Leeuw, 2010; Dalkin et al., 2015).

Researchers using realist evaluation have highlighted the difficulties of identifying mechanisms: such as interpreting them as a context or defining intervention activities as mechanisms (Greenhalgh et al., 2009; Marchal et al., 2012; Rycroft-Malone et al., 2010). To address these issues, Dalkin et al. (2015) proposed a model that builds on Pawson and Tilley’s (1997) original conceptualisation of mechanisms that links yet separates mechanisms into their component parts: (1) the resources an intervention provides and (2) the reasoning or responses that people using the intervention have. In a study of what supports hospital patients with dementia to receive care that can address both the consequences of their dementia and their reason for being in hospital, we used this model to develop, test and refine context-mechanism-outcome configurations (CMOCs). Our use of non-participant observation, in combination with other data collection methods, not only addressed questions about context and outcome but also recorded real-time explanations from staff that ‘revealed’ the mechanisms driving their actions.

Method

Our study objective was to explain what supports (contexts and mechanisms) hospital staff to provide dementia sensitive care and what the outcomes are for people living with dementia and their family carers. Initial CMOCs were developed using a realist review of the literature and interviews with 15 stakeholders (Handley et al., 2017).

The realist evaluation was conducted at two hospital trusts in the East of England. These sites were purposively selected as they had developed different approaches that aimed to enhance care of people with dementia. Site 1 had a dedicated ward for people with dementia. The ward had a higher staff to patient ratio than other hospital wards admitting patients who were comparable in terms of age and reason for admission (1:2 compared with 1:5) and there was daily scheduled input from mental health professionals. Site 2 had a team of healthcare assistants who were supported by dementia specialist nurses to provide care for people with dementia across the hospital. The study was conducted between October 2016 and March 2017 for a period of 7 weeks in each site.

Data were gathered using non-participant observations in patient bay areas, interviews with staff, patients with dementia and family carers of patients with dementia, medical notes review, neuropsychiatric inventory and documentary review. Only non-participant observations and interviews with staff were relevant to this article. Observations took place on a total of five hospitals wards: one ward at Site 1 (ward for people with dementia) and four wards at Site 2 (mix of older people wards and surgical wards). This reflected the organisation of services and resources for people with dementia at each site.

We used a semi-structured approach to non-participant observation to record an in-depth account of the context of care, events, staff actions for care and patient outcomes. A topic guide, developed from a realist review (Handley et al., 2017), informed, but did not restrict, the focus of note-taking. Based on the review findings, we focused on six CMOCs of interest: (1) understanding behaviour as communication and responding to patient needs, (2) training that generates empathy and encourages reflection on care practices, (3) clinical experts who legitimise the priorities for care, (4) staff confidence to adapt working practices and individualise care, (5) staff time to focus on psychosocial needs and (6) confidence to provide person-centred risk management.

Observations on the ward operated an opt-out process for recruitment (Caswell et al., 2015). Posters about the research were displayed prominently on participating wards. Verbal and written information about the study were given to all ward staff and eligible patients in participating wards at least 24 hours prior to an observation period. This information included an opt-out form and all requests to not participate in the study, whether received verbally or in writing, were respected. A total of four written opt-out forms were received from staff. On the day prior to the start of data collection, observation sessions were negotiated with ward managers, staff and patients. During these discussions, involvement from staff and patients was checked and no verbal objections from staff or patients were raised. Patients were reminded of the role of the researcher and the purpose of the research during the observation period as necessary, for example, if the patient appeared confused by the presence of the researcher. Ongoing consent was used (Dewing, 2007); if a patient exhibited distress due to the presence of the researcher or the research process, the observation ceased. However, general distress and annoyance was not a reason for halting data collection as a level of distress was to be expected due to the setting and study population (Goldberg et al., 2014). Decisions about where the researcher (MH) would conduct observations were based on their potential to contribute to theory testing. For example, discussions with ward staff included whether there were plans for activities and with which patients, or if additional staff were expected to support patients with dementia identified at risk of falling. Observation periods ranged from 2 to 6 hours (average 3.6 hours) to minimise the researcher effect on events (Clissett et al., 2013; Mulhall, 2003). A total of 80 hours of non-participant observation were conducted across the two sites over 22 observation periods.

Observations were performed by one researcher (MH) who took handwritten field notes at the time of the observation period and transcribed notes to an electronic format. The researcher does not have a clinical background and on this basis would be considered a naïve observer. This meant she could clarify understanding during observations, for example, in relation to procedures and care decisions.

Data from non-participant observation were testing, confirming and refuting theories and used to corroborate understanding with data collected via other methods. In this article, we report how data pertaining to potential mechanisms surfaced during non-participant observation to inform discussions during subsequent interviews with staff. A total of 42 participants were interviewed across the sites, including 36 staff, 4 people with dementia and 2 family carers. Interviews were audio recorded and transcribed verbatim except for two staff interviews where handwritten notes were taken during the interview. The average duration of interviews was 22 minutes (range 11–64 minutes).

To support theory refinement, a responsive and iterative approach to data analysis was undertaken, commencing at the start of data collection (Manzano, 2016). This approach allowed for the incorporation of emerging findings to inform and be tested in subsequent data collection. Active reflection on data collected during observation sessions was part of the analytic process and included sharing the impromptu explanations offered by staff with the research team. This supported debates around if data were consistent with or added to the preliminary CMOCs, or suggested rival explanations. During interviews, the theories were discussed with participants to see if they resonated or prompted alternative interpretations or accounts (Figure 1).

Figure 1.

How non-participant observation informed subsequent data collection and theory refinement.

Ethical approval

Ethics and research governance approval was obtained prior to data collection (Ethics reference East of England – Essex Research Ethics Committee 16/EE/0263).

Findings

Framework for data collection and analysis

The initial CMOCs from the realist review were refined drawing on the data collected during the realist evaluation. The refined CMOCs suggested that for hospital staff to use best practice approaches in dementia care, they needed to recognise dementia care practices as fundamental to their role. Training and support from clinical experts in dementia care and the use of care planning tools were important. However, if dementia care was not an integral aspect of clinicians’ work and within their place of work, the resources from these interventions would not change how staff provided care for people with dementia. Instead, change in care practices would be limited to the actions of individual staff with extensive experience of working with people living with dementia. Table 1 shows how non-participant observation contributed to refinement of the six original CMOCs. The refinement of two CMOCs is detailed below. All CMOCs are described in full elsewhere (Handley et al., 2017; Handley et al., 2018).

Table 1.

Refinement of the context-mechanism-outcome configurations.

Initial context-mechanism-outcome configurations (realist review)	Revised context-mechanism-outcome configurations (realist evaluation)
1. Understanding behaviour as communication improves staffs’ ability to respond When staff understand behaviours as communication of an unmet need (context), and consider they have the capacity and capability to make a difference (mechanism), they will be more likely take action to identify and address the patient’s need (outcome).	1. Knowledge and authority to respond to an unmet need When staff understand behaviours as communication of an unmet need (context), and consider they have the capacity, capability and authority to share information about patient care and are knowledgeable in a range of strategies (mechanism), they will be more likely to identify and address the need (outcome).
2. Experiential learning and empathy encourages reflection on responsibilities of care Access to training (context) that provides an experiential understanding of the impact of dementia and promotes empathy helps staff reflect on the deficiencies of current working practices (mechanism), leading staff to adapt care practices (outcome).	2. Role relevant training and opportunities for reflection Access to training and support from experts and colleagues with experience in dementia care (context) develops skills and techniques relevant to staffs’ work (mechanism) which staff are likely to apply in their work, and reduces patient episodes of distress (outcome).
3. Clinical experts promote good care practices When experts with clinical and organisational authority legitimise best-practice dementia care (context) and staff to feel confident they understand the expectations and responsibilities for patient care (mechanism), this helps to embed good care practices (outcome).	3. Clinical experts and senior staff promoting practices that are patient-focused Where standards for dementia care are defined, monitored and reinforced by experts and senior staff (context), ward staff will understand what is expected of them and how care is to be prioritised, and recognise the benefit of new practices (mechanism) supporting staff decisions for care practices (outcome).
4. Staff with confidence to adapt working practices and routines to individualise care Where staff are supported to be flexible and autonomous (context), understand their responsibilities for patient care and have confidence to adapt care provision (mechanism), they will respond in a timely, individualised manner (outcome).	4. Engaging with opportunities to spend time with patients Staff with a clear understanding of their responsibilities and the priorities for patient care (context) have knowledge of the patient they can use in their interactions, resist competing demands on their time (mechanism) and can provide care that priorities patient experience and maintains their identity (outcome).
5. Staff with responsibility to focus on psychosocial needs Provision of activities and therapies for patients with dementia (context) through the allocation of staff to support patients psychosocial needs (mechanism) maintain for patients’ functional and cognitive abilities (outcome).
6. Building staff confidence to provide person-centred risk management Where risk management procedures and expectations are defined through the use of person-centred approaches (context), and senior staff encourage and reinforce these practices, staff may feel confident they are supported to address risk proportionately (mechanism) and they may support safety in ways which help maintain abilities and accept patient choices (outcome).	5. Risk management as an opportunity for dementia care Encouragement and permission to manage risk in a person-centred way (context), and staff knowledge for addressing risk and concerns for the potential consequences (mechanism) informed whether staff used more or less restrictive practices (outcome).
	6. Valuing dementia care as skilled workWhere staff understand the complexity of working with patients with dementia (context), they can define their expertise in dementia care and recognise dementia care as skilled work (mechanism reasoning). Increasing the likelihood that they will remain working in dementia care (outcome)

Bold text indicates refinements to the context-mechanism-outcome configurations from the realist evaluation.

Conduct of non-participant observation

The initial aim was for the researcher (MH) to record events in patient bay areas without influencing events, interactions and care processes. There were also opportunities as part of the observation to clarify with staff what was happening and why, for example, information related to medication administration or clinical procedures. In practice, participants regularly initiated interactions with the researcher providing an unsolicited commentary to explain (and possibly justify) their actions. These comments, made during observation periods by both staff and patients, were documented in field notes at the time of data collection and transferred to electronic observation transcripts. Such comments from staff and patients offered insights for possible underlying mechanisms that, for example, led staff to engage with patients. It also provided insight into the outcomes from those encounters.

Using observation to identify mechanisms

In the following section, we provide two examples of how data from non-participant observations contributed to refining the mechanisms set out in two CMOCs developed from the literature review. We have used bold font to highlight key staff comments in the quotes that supported theory refinement.

Example 1: Concerns around the consequences of an adverse incident

Person-centred risk management strategies are approaches that address a particular issue that could lead to patient harm, for example, falling because of being disoriented. It focuses on strategies that are likely to reflect the individual’s abilities, interests and choices. Findings from the review suggested a key mechanism for hospital staff to adopt this approach was ‘confidence the approach was supported by senior staff’. Initial reflections on early data collection at Site 1 appeared to confirm the review findings. At this site, clinical experts in dementia care were observed to provide advice to ward staff around the day-to-day care and support for patients. Most patients on the ward were assessed as being a high risk to themselves or others, and staff were encouraged to promote mobility, address patient preferences for personal care and join patients in activities and conversations outside of task-related interactions. However, as the study progressed, data gathered during non-participant observations suggested that despite this involvement, ward staff, particularly healthcare assistants, were inclined to use more risk averse, restrictive practices that separated people with dementia from ward activities. In the following extract, the healthcare assistant’s dilemma is demonstrated. She acknowledges that the nurse has the authority to respond to the patient’s preference not to be restricted but that does not override her concern about preventing harm:

(FN: Bob is resting in bed close to the edge. The HCA puts up Bob’s bedrail and then attends to another patient. After a few minutes Bob becomes frustrated with the bedrail and a nurse lowers them.) A few minutes later the HCA walks back over to Bob and notices the bedrails are down. She turns to me and asks who put the rail down. I explain it was the nurse. The HCA goes ‘oh’ and has a furrowed brow. She explains ‘I was worried he was going to fall out of bed. I thought it was better to have them up than for him to hurt himself’. She turns to Bob and then back to me and says, ‘oh well, if it was a nurse’. (Site 1, OB0101)

The healthcare assistant’s reasoning for using the bedrails was her dominant concern about Bob’s risk of falling. Although seniority influenced the actions of individual staff on the ward, it had not necessarily changed how other staff interpreted patient actions and needs. During observation sessions at both sites, healthcare assistants talked through what they considered important when maintaining patient safety. This was often related to the use of bedrails or supporting mobility with patients at high risk of falling. The data provided evidence of how clinician endorsement and modelling good practice for less restrictive approaches from senior colleagues (mechanism) were insufficient when this conflicted with an organisational ethos that stressed falls prevention (context) but did not support shared decision-making and discussion about patients who, because of their dementia, needed a range of strategies.

Commentary from healthcare assistants during observations helped refine an understanding of the reasoning that informed their approach to risk management. The seniority of staff could support less restrictive practises by sharing the decisions around care. However, when unqualified members of staff had not been permitted by a senior member of staff to use less restrictive practices, they would revert to more restrictive practices, which they understood to be ‘safer’. While healthcare assistants spoke of the conflicting issues around patient choice and maintaining safety, access to advice on how to balance both was not readily available. Observational data provided access to the mechanisms in action as the following examples demonstrates:

(FN: Flo is trying to get out of bed, however the HCA is about to support another patient with personal care). The healthcare assistant puts up Flo’s bedrails and explains that is to keep her safe while he cannot see her. He then continues that if she was to have a fall the first thing the Trust would ask is ‘where were you?’ (Site 2, OB0210)

This challenged the initial theory that staff were motivated to align risk management strategies with patient’s preferences through a mechanism of ‘confidence in senior staff support’. Instead, data suggested staff approaches to risk management were motivated by ‘concern for the consequences of an adverse incident’ to themselves and that it could not be assumed they had the knowledge or resources to be able to accommodate patient preferences. This mechanism was explored and tested further in interviews with ward staff and clinical experts about how risk was managed. It highlighted, that despite access to senior clinicians, healthcare assistants felt vulnerable and exposed when supporting patients with dementia at risk of harm:

. . . [where there are multiple patients’ needs to address] you feel like you are put in the situation ‘who do I go to?’ because you are leaving the other one vulnerable. (Site 2, ST0203, Healthcare assistant)

They [ward staff] were too scared do anything different even though it might make it better. Just changing where the patient sits, if they haven’t done that before they might not think about it. They are just worried about the implications of them falling and getting aggressive with them. (Site 2, ST0201, Senior nurse)

Observations suggested the mechanism ‘concern for the consequences of an adverse incident’ appeared to cause staff to adopt one of two main strategies during risk management activities: (1) the use of practices which recognised patient choice and personhood, and (2) a strict adherence to maintaining safety, for example, by restricting patient movement. These strategies are demonstrated in the following examples:

Alan is standing at the middle table. A healthcare assistant goes over to him and tries to encourage him to sit down, stating ‘you seem a bit wobbly, I’m worried about your legs’. There is a puzzle on the table and the healthcare assistant asks Alan if he would like to do the puzzle with her. He shows an interest and they sit down and complete the puzzle together. (summary of Site 1, OB0102)

The healthcare assistant explains, ‘I need to stay with this lady as she is not to stand’. As she is saying this, the patient tries to stand and the healthcare assistant puts her hand out to encourage her to sit down. The patient bats the hand away and complains about not being allowed to get up. (Site 2, OB0213)

In the first example, the healthcare assistant had the awareness and skills to respond to the person by offering an alternative to walking. The second example illustrates how the healthcare assistant focused on what the patient was ‘allowed’ to do rather than knowing how to respond to what the patient wanted to do. Both responses took up the same amount of healthcare assistant time but achieved different outcomes in terms of meaningful patient/staff interactions. This reinforces the interpretation that how micro encounters take place between staff and patients are linked to staff concerns and their knowledge, as well as the culture and support on the ward and from senior staff.

Example 2: Recognising dementia care as skilled work

While the importance of valuing dementia care across care settings has been widely written about, the realist review (Handley et al., 2017) did not identify evidence to articulate how valuing dementia care changed patient outcomes. However, during the realist evaluation it was found to be a key overarching element that operated at personal, service and organisational level and linked to a mechanism defined as ‘staff recognising dementia care as skilled work’. The following extract, from observation field notes, triggered a line of thinking about dementia care not being seen as clinically valued work, which in turn developed and refined the CMOC:

[conversation with senior ward member relating to nursing staff leaving the ward] Some [nurses] were moving to jobs in A&E. She [senior staff member] accepted that for nurses who had recently qualified they wanted to be in a more clinically focused area so they would not lose their clinical skills. (Site 1, OB0104)

Concerns of some nursing staff were that working predominantly in dementia care would lead to a loss of clinical skills and ultimately hinder future career prospects. The extract suggested that most ward nurses conceptualised skilled work as clinical activities, related to intensive, continual professional development and training. This was observed in the two wards such that even patients with behaviours that staff found challenging, or whose dementia complicated their treatment regimes, were delegated to the care of unqualified staff. Dementia care skills were often developed through personal experiences, role modelling from experienced colleagues or attendance at a short course, and were referred to as ‘common sense’. This idea of ‘common sense’ was linked to actions for care which could enhance interactions with patients with dementia, such as ensuring sensory aids were fitted, using verbal and non-verbal communication appropriate for the person and the situation, and supporting nutrition by knowing what patients liked to eat. Clinical experts in dementia care identified this as skilled work, validating why actions, such as using someone’s favoured name, were important and needed to be taught:

So these things you could say they are basic common sense, and you’re right, but at the same time they are things that you need to teach people and reinforce as part of good practice. Because otherwise they can get missed because they are so easy, so basic in some ways that they can be easily overlooked. (Site 1, ST0116, Medical staff)

Observations recorded evidence of basic actions being overlooked and what the impact of this was for the patient in terms of their ability to engage with care decisions, communicate their choices and understand what was happening. Opportunities to gain insights into the motivations of staff when they did attend to patients’ non-clinical needs arose during observations. On occasions, staff offered a commentary for what they hoped their actions would achieve for the patient. The example below demonstrates how a spontaneous comment from a member of staff led the researcher to instigate a more in-depth conversation that explored how they had come to recognise these areas of care as important and something they could influence:

(FN: The one-to-one HCA has just changed the bedside chair of Gary [a patient] to one that is better suited to a tall person, the patient comments how that is much more comfortable). The 1:1 then turns to me and explains that that will stop the pressure on his back or pain from sitting in a bad position. She goes on to tell me about how pain is always one of the first things she considers when a person is agitated or aggressive. I ask her what made her become aware that pain could be a problem. She tells me that she went on a pain in dementia course and since that training she has been ‘on it’ looking out for evidence of pain. She then gives the example that Brian has been complaining of headaches since she has been working with him and suggests this is possibly because he is used to wearing glasses but he does not have them with him. (Site 2, OB0212)

This data helped us theorise that for staff who prioritised these actions for care and understood them as fundamental to their role, the mechanism ‘recognising dementia care as skilled work’ was triggered. To test our theory, staff were asked in interviews about the skills they used when working with patients with dementia. This revealed perceptions that dementia care was not real work and was often considered a low priority in comparison to work that observably contributed to the workload on the ward:

They think they should be on the shop floor working and doing all the things and then obviously being involved in [dementia focused] activities can be seen as being a bit of fun rather than actually work. (Site 1, ST0111, Occupational Therapist)

Further probing during observations and interviews revealed an awareness among staff working predominantly with patients with dementia of how other staff understood their work. This was generally considered a low priority, undemanding and something that was not the responsibility of all hospital staff:

They [ward staff] don’t quite understand what the role [one-to-one] is and why aren’t you doing as much as I’m doing and why are you just sitting with that one patient rather than helping out on the whole ward. (Site 2, ST0204, Senior Nurse)

These views of dementia care were held despite nurses and healthcare assistants reporting in interviews the observed difficulty of attending to the multiple psychological, social, physical and medical needs of many patients with dementia.

Discussion

In this article, we have used examples from our study to demonstrate how observation methods can contribute to the testing and refinement of mid-level theory, going beyond evidence of contexts and outcomes to make explicit the mechanisms that led to different or reoccurring outcomes. We have used examples from our data to demonstrate how non-participant observations on hospital wards gave an insight into staff’s motivations and the resources they drew on when providing care for patients with dementia. During observations, comments from staff before, during or after care interactions provided real-time access to how staff were interpreting their role and responsibilities and potential causal mechanisms. These comments complemented the realist interview (Manzano, 2016) and were a resource for reflection and debate across the team. Our two detailed examples demonstrate how staff explanations during observations can inform subsequent data collection through early engagement with the data and an iterative approach to analysis.

The limitations of interviews in relation to what people say they do compared with what people actually do are widely recognised and have been used as a rationale for undertaking observations (Holloway and Galvin, 2016; Mays and Pope, 1995). Observations also provided us with real-time access to staff’s expressed motivations and reasoning, allowing us to locate mechanisms within specific contexts and witness the accompanying action taken (Hammersley and Atkinson, 2007). While we aimed to minimise researcher influence on participants and their actions during observation sessions, commentary provided to researchers indicates that staff were aware of the researcher. Hospital wards are busy places and it is unlikely that observations influenced how staff provided care (Mulhall, 2003). Staff were keen to provide detail related to their reasoning for different care actions, perhaps due to engaging with the research process or through a desire to share their reasoning. As part of maintaining rigour in the analysis process, it was important not to rely solely on this commentary, but to acknowledge these remarks as evidence of potential mechanisms and search for alternative explanations and negative instances (Pawson and Tilley, 1997).

Defining mechanisms and identifying them as a component separate to context is a widely recognised problem for realist researchers (Dalkin et al., 2015; Emmel et al., 2018; Lacouture et al., 2015; Marchal et al., 2012; Shaw et al., 2018). Our observation data focused on staff interactions with patients. Dalkin et al. (2015) definition of mechanisms as the resources inherent in an intervention and the response to those resources of those using the intervention were applied in our study to understand how staff did (or did not) engage with dementia sensitive care. This could be challenged as an unnecessarily narrow understanding of mechanisms where reasoning is conceptualised as the conscious choices of those studied (Shaw et al., 2018). We nevertheless found observation a useful method for helping to surface mechanisms located at an individual level. Identifying structural mechanisms, where the level of analysis is located at organisational change, may rely more on interview data and documentary review.

The six CMOCs developed from the realist review provided the framework for the evaluation, including data collection and analysis. While decisions during fieldwork were based on these propositions, such as who and what activities to observe, we applied an iterative process to data collection to incorporate previously undefined factors to test our assumptions. This was facilitated through regular, spirited discussions to challenge interpretations and provided the basis for testing in subsequent data collection and analytic activities. As demonstrated in our second example, ‘recognising dementia care as skilled work’, our deliberations supported a responsive approach to data collection which could incorporate evolving theorising and progress our explanations beyond the original CMOCs.

Strengths and limitations

Non-participant observation was an important method for understand the experience of providing dementia care in hospital wards. We collected rich data for testing and refining our CMOCs. The method did not interfere with participants work, but for the researcher it was time intensive and an ethically sensitive activity. Negotiating access to the wards, introducing the research to potential participants and checking acceptability of those affected by the observation activity throughout the day were also time consuming. This could add between 2 and 4 hours per observation period. Post-observation session work included expanding handwritten notes, transferring notes to electronic formats, reflecting on the observations, making memos related to support for or challenges to theories and team meetings to discuss the data.

It is recognised that researchers can become overwhelmed during observation sessions by numerous activities taking place within the environment (Hammersley and Atkinson, 2007; Silverman, 2013). In our study, the ward environment was often busy and multiple interactions between staff and patients could occur at once. The CMOCs developed in the realist review (Handley et al., 2017) enabled the researcher to focus on situations that might contribute to theory testing, for example, observing how staff worked with people with dementia who were distressed and disorientated. By observing, recording, reflecting on and discussing with the research team staff comments explaining their actions for care, we were able to consider their potential contribution and devise strategies for further testing, that is, what would be useful to observe and what would be useful to follow up in interviews. This required the researcher to maintain regular contact with staff to establish and maintain trust and hold additional meetings with the research team to discuss the data.

Conclusion

Manzano (2016) explained techniques researchers could use during interviews to ‘ask questions like a realist’ (p. 343). In this article, we presented the contribution of non-participant observation for theory testing with examples of how the method could be used to observe like a realist. We have demonstrated that as part of a multi-method approach to data collection, observation has the potential to uncover mechanisms as they are triggered. Early engagement with analytic work through reflection and debate helped identify emerging theories which could be further tested with participants. This can improve our understanding of what works, for whom, and in what circumstances.

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by funding from the Alzheimer’s Society (grant no. ALZSOC-PhD-2013-025). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

ORCID iD

Melanie Handley

Melanie Handley is a Research Fellow at the University of Hertfordshire. She uses realist methodology to investigate the use of primary and secondary care services by older people with dementia.

Frances Bunn is a Professor in Health and Complex Conditions and Deputy Director of Centre for Research in Public Health and Community Care (CRIPACC). Her research focuses on older people with complex needs including those with dementia and comorbid conditions.

Jennifer Lynch is an NIHR Knowledge Mobilisation Research Fellow. Her research explores the use of technology in community health and social care provision for people with dementia and their carers.

Claire Goodman is a Professor of Health Care Research. Her research focuses on the health and social care needs of people affected by dementia and people living in long-term care settings.

References

Astbury

Leeuw

(2010) Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation 31: 363–81.

Bhaskar

(1978) A Realist Theory of Science. Hassocks: Harvester Press.

Caswell

Pollock

Harwood

, et al. (2015) Communication between family carers and health professionals about end-of-life care for older people in the acute hospital setting: A qualitative study. BMC Palliative Care 14: 35.

Clissett

Porock

Harwood

, et al. (2013) The challenges of achieving person-centred care in acute hospitals: A qualitative study of people with dementia and their families. International Journal of Nursing Studies 50: 1495–503.

Dalkin

Greenhalgh

Jones

, et al. (2015) What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science 10: 49.

Dewing

(2007) Participatory research: A method for process consent with persons who have dementia. Dementia 6: 11–25.

Emmel

Greenhalgh

Manzano

, et al. (2018) Doing Realist Research. London: SAGE.

Goldberg

Whittamore

Pollock

, et al. (2014) Caring for cognitively impaired older patients in the general hospital: A qualitative analysis of similarities and differences between a specialist Medical and Mental Health Unit and standard care wards. International Journal of Nursing Studies 51: 1332–43.

Greenhalgh

Humphrey

Hughes

, et al. (2009) How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Quarterly 87: 391–416.

10.

Hammersley

Atkinson

(2007) Ethnography: Principles in Practice. London: Routledge.

11.

Handley

Bunn

Goodman

(2017) Dementia-friendly interventions to improve the care of people living with dementia admitted to hospitals: a realist review. BMJ Open 7: e015257.

12.

Handley

Bunn

Goodman

(2018) Supporting general hospital staff to provide dementia sensitive care: A realist evaluation. International Journal of Nursing Studies. DOI: 10.1016/j.ijnurstu.2018.10.004.

13.

Holloway

Galvin

(2016) Qualitative Research in Nursing and Healthcare. Newark, NJ: Wiley.

14.

Lacouture

Breton

Guichard

, et al. (2015) The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science 10: 153.

15.

McGaughey

O’Halloran

Porter

, et al. (2017) Early warning systems and rapid response to the deteriorating patient in hospital: A realist evaluation. Journal of Advanced Nursing 73: 3119–32.

16.

Manzano

(2016) The craft of interviewing in realist evaluation. Evaluation 22: 342–60.

17.

Marchal

Belle

Olmen

, et al. (2012) Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation 18: 192–212.

18.

Marchal

Dedzo

Kegels

(2010) A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Services Research 10: 1–14.

19.

Mays

Pope

(1995) Qualitative research: Observational methods in health care settings. BMJ: British Medical Journal 311: 182–4.

20.

Mulhall

(2003) In the field: Notes on observation in qualitative research. Journal of Advanced Nursing 41: 306–13.

21.

Pawson

Tilley

(1997) Realistic Evaluation. London: SAGE.

22.

Rycroft-Malone

Fontenla

Bick

, et al. (2010) A realistic evaluation: The case of protocol-based care. Implementation Science 5: 38.

23.

Rycroft-Malone

Fontenla

Seers

, et al. (2009) Protocol-based care: The standardisation of decision-making? Journal of Clinical Nursing 18: 1490–500.

24.

Shaw

Gray

Baker

, et al. (2018) Mechanisms, contexts and points of contention: Operationalizing realist-informed research for complex health interventions. BMC Medical Research Methodology 18: 178.

25.

Silverman

(2013) Doing Qualitative Research: A Practical Handbook. London: SAGE.

26.

Walshe

Ewing

Griffiths

(2012) Using observation as a data collection method to help understand patient and professional roles and actions in palliative care settings. Palliative Medicine 26: 1048–54.

27.

Westhorp

(2014) Realist Impact Evaluation: An Introduction. London: Overseas Development Institute, pp. 1–12.

28.

Westhorp

(2018) Understanding mechanisms in realist evaluation and research. In: Emmel

Greenhalgh

Manzano

, et al. (eds) Doing Realist Research. London: SAGE, pp. 42–46.

29.

Wye

Lasseter

Percival

, et al. (2014) What works in ‘real life’ to facilitate home deaths and fewer hospital admissions for those at end of life? Results from a realist evaluation of new palliative care services in two English counties. BMC Palliative Care 13: 37.