Complex realism in social research

Realism

Through the 20th century, realism has offered an alternative paradigm in scientific thinking to that of idealism and positivism, and their variants (Harré, 1986). It differs from these (which would include phenomenology, logical positivism, instrumentalism, etc.) by equally privileging the issue of what exists, alongside our warrant to know what exists. That is, it reinstates ontology, which has been denied or postponed in rival philosophies of science. To its critics, this is a leap of faith for it requires the theorisation of non-observables and their treatment as entities, or potentially so. In much of the earlier discussions of realism (see, for example, Potter, 1996), entities were seen as simplified physical things or forces, capable of measurement. Relativist critics of realism, in social science, referred to this as the ‘death and furniture’ argument, suggesting that social processes were dynamic, complex and, most of all, socially constructed and inevitably interpreted (Potter, 1996). In the natural sciences, this was already a bit of a straw person; where realist interpretations of wave–particle duality, in quantum physics, posited an interaction between observer and the collapse of the wave function. In other words, the issues raised by unobservables in the physical world were meat and drink to realist philosophers of natural science. It has been a long while since the physicist, in the laboratory, was able to rely on his or her own senses for empirical confirmation or falsification!

However, Potter and other relativists do have a point about the social world, in that not only is it endlessly dynamic but its existence and its development depend on human interaction and, at root, individual action born of interpretation. As Norman Denzin (1983) claimed, there is an ‘inherent indeterminateness in the lifeworld’ (p. 133). This is a seductive claim and one often deployed against realism. But, on the other hand, a moment’s reflection shows this to be a claim too far. To begin with, supporters of social indeterminism must explain how it is that social order exists and is maintained and, perhaps paradoxically, how sometimes it breaks down or changes quite dramatically. Added to this, if the world was so indetermined and simply a product of consciousness (as Denzin goes on to claim), then the constraints that prevent us from doing just as we wish (laws, rules, sanctions and physical constraints created by humans) should be routinely capable of transcendence, but they are not.

Social science variants of realism do, however, seem to go the other way, ¹ with what might be described as a ‘strong’ version of causality that depends on natural necessity. Critical realism, the dominant variety in social science, relies on the idea of ‘causal powers’ (elsewhere often referred to as ‘dispositions’; Mumford, 2007). These can be either passive or active, depending on context. Take the example of ‘labour power’, used by Andrew Sayer (1992) – in his words,

The nature or constitution of an object and its causal powers are internally or necessarily related: a plane can fly by virtue of its aerodynamic form, engines etc; gunpowder can explode by virtue of its unstable chemical structure; multinational firms can sell their products dear and buy their labour cheap by virtue of operating in several countries with different levels of development … (p. 105)

However, gunpowder and multinational firms have quite different kinds of dispositional properties. The unstable chemical structure of gunpowder can be explained by a relatively small number of environmental contingencies, themselves grounded in physical laws, whereas multinational firms are time-specific evolving and their dispositions explained only in relation to other evolving structures, themselves having a dispositional character.

In order to work, people must sell their labour, or make a product that they can sell, and there has to be a demand for their labour, or product. A person may possess labour power, but the circumstances under which this may be activated or sustained can vary enormously. At no point can we specify a set of conditions necessary to the exercise of labour power, though of course we might specify sufficient conditions. Powers must be realised under particular circumstances, but their necessity is ever elusive.

Critical realism and its variants face both an ontological paradox and an epistemological one. The ontological one is that the strong version of causality they propose is (at least apparently) deterministic, yet the causal powers responsible are indeterministic. The ‘real’ world is ontologically indeterministic, yet structured. It is complex.

Complexity

Some systems in the world are deterministic and most of these are more closely bound by fundamental physical laws, such as gravity and thermodynamics. They will never be entirely deterministic because local variations of (say) air pressure or material construction will intervene. Some phenomena are wholly stochastic, for example, particles suspended in a fluid (described as Brownian motion), some are apparently stochastic, but through time some patterning can be discerned, and others, though exhibiting some degree of randomness, nevertheless exhibit order and form over time. Indeed, all phenomena that might be described probabilistically exhibit some degree of stochastic behaviour. Karl Popper (1979) provided us with a useful analogy (p. 296), when he contrasted the behaviour of clouds with that of clocks. The former have measurable properties, but these can only be approximately known and measured through time, whereas clocks (and he meant old-fashioned clockwork clocks) have relatively precise mechanisms that behave in a reliable and predictable way. Some systems, he goes on to say, are more like clouds and some more like clocks.

This is a good analogy for the social world, in which its components are mostly more cloud than clock like, yet some parts of it will veer towards ‘clockness’, while others hardly achieve the recognisable form of clouds.

Clocks are complicated systems, so are mobile phones, motorbikes and space shuttles, but they are not complex – their behaviour and properties are not subject to influence of random variables (at least when they are working properly) (Cilliers, 1998). Clouds are, and so are most phenomena in the social world. This distinction is at the heart of the development of ‘chaos theory’, and complexity, in the last 50 years. The discovery of ‘chaos’ was in the behaviour of weather systems. Well into the 20th century, meteorologists subscribed to the view that if we knew enough about initial conditions, we would be able to predict and even control the weather. But in the 1960s, scientists modelling weather found that very slight changes in initial conditions could bring about huge changes in weather patterns later (Gleick, 1987: 14–15). The discovery of chaos heralded a whole new way of thinking about the natural (and later social) world. Researchers noted how relatively stable systems in weather, oceans and biological systems could change suddenly, or descend into apparent disorder, and conversely, how apparently stochastic systems could become ordered. Complex systems, exhibiting such characteristics, exist throughout nature in the development of galaxies to cell division (Kauffman, 1995). There is much to say about chaos and complexity, but for our purposes, it is enough to make a few observations about the social world that suggest that it and its components are a quintessentially complex system.

The first thing to note is that despite what the social constructionists would say, it is possible to explain and predict the social world, partly because it is rule based and partly because agents in it act inductively on past experience, which mostly exhibits a large degree of predictability. In other words, in our everyday lives, we are aware of and (mostly) obey social rules, and those people with whom we interact also do this and exhibit relatively predictable behaviour. This permits good enough lay explanations and predictions. It is mostly not stochastic. Social scientists, who wish to produce explanations and predictions with a greater precision, see a world that exhibits both continuity and change, but is also characterised by similarity and difference across social strata and time. It is fashionable to say that social scientists study social change, but in order to do this, they must also explain social stability. Stability is not stasis and will be disturbed eventually to one extent or another. As Harvey and Reed (cited in Byrne, 1998: 23) put it, in the absence of significant perturbations, systems will follow a normal trajectory, but if disrupted, random fluctuations may arise. Our clock analogy has one further use: a mechanical clock, with a pendulum, is a relatively precise and predictable instrument, but if you disturb the pendulum, it will fluctuate randomly and, depending how much you disturbed it, that degree of randomness persists for a greater or lesser period, until it returns to equilibrium. Whack it hard enough and the ‘randomness’ may preclude a return to equilibrium, because you have broken it!

Complexity theorists employ a vocabulary of ‘Strange Attractors’ and ‘Phase Space’ to describe this. Strange Attractors, in complexity theory, refers to time-ordered patterns towards which other trajectories converge. Strange Attractors can be understood to exist within ‘Phase Space’, the space of the possible where anything can happen but not everything will happen given the structured, rule-driven nature of the social:

The geometry of dynamical systems takes place in a mental space, known as phase space. It’s very different from ordinary physical space. Phase space contains not just what happens but what might happen under different circumstances. It’s the space of the possible. (Cohen and Stewart, 1994: 200)

The social world can therefore be thought of as overlapping, or multi-dimensional, ‘phase spaces’, with different probabilities and trajectories of change.

For example, some social institutions, or practices, may exist for centuries, changing only slightly, because they have not been ‘disturbed’. Others exist so briefly they are hardly named or recognised because they are greatly disturbed. Compare, for example, the institutions of a country in flux, such as Somalia, with the stability of Switzerland.

These kinds of comparisons are popular with those who write about social complexity (and rightly so), but this works better for historians or political scientists than it does for social researchers attempting to describe and explain social life at the micro- or meso-level. In particular, how do the specific changes and interactions at individual level create, maintain, change or destroy specific local systems? We use the term ‘systems’ as a portmanteau term to describe ways of doing things, mores, customs, rules – the reality of social life as experienced and acted upon by agents. Although the explanation of continuity is crucial (why does the social trajectory of an institution or practice remain undisturbed?), to do this, we need to know what is going on at the point of change and how that point of change is (seemingly) related to further change. Unlike Hume, who said that the striking of one billiard ball against another is no warrant to claim the cause of the second moving was the striking of the first, as realists, we want to theorise a system (or mechanism) that will provide an explanation. Thus, measurement, in complex systems, is an important aspect of realist ‘closure’.

Probability

We noted that stochastic systems are described by probability. Objective probability (as opposed to subjective, or Bayesian probability) takes both an epistemological and an ontological form. The first is familiar to us, through the sample survey, and is known as the frequency theory of probability. A probabilistic sample of a given population can statistically describe that population, within certain known and calculable tolerances. Those tolerances are described in the error terms of a sample and are subject to the law of large numbers. Roughly speaking, the larger the sample, the smaller the error term. That is, the bigger the sample, the better it will resemble the population. However, a sample can tell us nothing about the individual units (often people) because they are an ‘ideal’ unit, that is, we do not have data on all members of a population and we could substitute one random sample for another, within which those in the second may or may not have been in the first.

There is another version of objective probability that is ontological in character and better mirrors what is meant by ‘probability’ in relation to complex systems (Gillies, 2000: 113–168; Williams and Dyer, 2009). ² While our knowledge of complex systems may be probabilistic (in the first, epistemological, sense), it is the very character of the systems and components of systems that are themselves probabilistic in character. To fully grasp this, we need to consider what we mean by probability, in relation to certainty. The probability of an event always lies on a continuum between 0 and 1. Zero is the impossibility of an event and one is its necessity. All probabilities eventually resolve themselves into zeros or ones. So, for example, an accused person will have a probability of conviction, which may change as a trial progresses, but eventually will evolve into a ‘One’ when the judge presents his or her verdict. That One may be ‘guilty’ or ‘not guilty’. To continue the hypothetical story, the One of ‘guilty’ or ’not guilty’ will then set the probability of future events, such as going to prison, re-offending (if guilty), personal stigma and so on. Also, the verdict is not neutral in respect of the other agents involved, such as family members, police officers, co-accused, victims and so on. Each of their life trajectories will be ‘perturbed’ and the probabilities of future events initially set, or changed. In other words, the probabilities are ‘nested’ and changed at each bifurcation (such as the judgement of guilty or ‘not guilty’).

The story so far

Before introducing the empirical example, let us summarise the theoretical and methodological case we are making. The reality of the social world is that it is indeterminate. As one of us has argued elsewhere (Williams, 2011), there are no necessary conditions in the social world, but rather there are different levels of sufficiency, enough to produce outcomes. In the physical world, such grounding exists in fundamental laws such as gravity or thermodynamics (Mumford, 2007: 74). However, in the social world, most outcomes exist within relatively stable rule-bound social structures, but change comes about as a result of perturbations – the greater the perturbation, the greater the change. It follows that probability is not just an epistemological matter of describing or predicting change, the social world itself is probabilistic, with some things very much more probable than others. Within ‘phase space’, all outcomes are possible, but some are very much more probable than others. Yet it is often the ‘improbable’ outcome that brings about greater change. The challenge is to develop a method that can follow these changes, or map these trajectories, in actual systems over time.

Background

The data presented in this example were originally collected as part of a National Institute of Health Research (NIHR)-funded research project one of us was involved with which aimed to identify a set of characteristics to assess future risk of self-harm in adult male and female prisoners (Horton et al., 2014). Prisoners are at increased risk of self-harm (Fazel et al., 2011; Hawton et al., 2014; Ministry of Justice, 2015). However, not all prisoners experience the same level of risk; in particular, there are a small number of prisoners who are responsible for a large number of self-harm events. Much of the research and theories in the area of self-harm broadly derive from the ‘Cry of Pain’ model (Williams, 1997) which suggests self-harm is a behavioural response to a stressful situation with three components which interact to increase risk, namely, (1) the presence of defeat, (2) no escape or entrapment, and (3) no rescue. At a general level, prisoners share similar sociodemographic characteristics and all experience the ‘pains of imprisonment’ (Sykes, 2007 [1958]). Why then are only some rather than all prisoners at ‘ultra-high risk’ of repeated self-harm?

As we noted above, the social world is neither simply determinate nor chaotically indeterminate or stochastic. Cases (in this case prisoners) will experience periods of relative stability and periods of change where, dependent upon perturbations or ‘disturbances’ in their current conditions, their lives will bifurcate along different pathways (in this case to repeat self-harm or not). Prisoners are not one single, homogeneous group but instead consist of different groups or types of people, some of whom share common patterns of experiences and individual characteristics. Rather than one ‘cry of pain’ model, how different groups of prisoners cope with perturbations caused by the ‘pains of life’, including imprisonment, will be different, but within groups, their shared experiences give rise to the same or similar outcomes. In other words, while we can identify the immediate perturbations that cause the self-harm, as in a Markov chain, these initially appear to be independent of patterns of prior events. The aim of this research was to identify different groups or types of prisoner where membership (described by shared patterns of experiences and personal characteristics) meant they were at increased or decreased probability of repeated self-harm in prison.

Methodology and method

This search for prisoner groups or typologies that are more or less likely to repeatedly self-harm is akin to the search for ‘Strange Attractors’ described earlier in this article. Such ‘prisoner self-harm attractors’ (in a metaphorical sense) emerge over time as shared individual characteristics such as resilience and coping (the hidden mechanism in Realism), which develop iteratively as a consequence of common experiences (the contexts in Realism), mean that pathways or personal trajectories converge. Strange Attractors describe a series of bifurcations in systems, for example, in turbulence flow, which produces patterns, shapes or likenesses. In this case, it is people who are more like one another than they are like people in other attractors. These underlying patterns or ‘prisoner self-harm attractors’ exist within Phase Space – the macro-level conception of context within which are micro multiple overlapping contexts or ‘state spaces’. These contexts include prisoners’ previous experiences which impact on the development of individual characteristics, including individual vulnerabilities, resilience and coping, and how they interact with the current prison environment (Griller, 2014; Liebling, 2005; Slade et al., 2014), as described in Figure 1 (a conceptual model or abstraction of the complex-realist ‘space’ within which prisoner self-harm pathways can be understood).

OPEN IN VIEWER

Figure 1.

A conceptual model of the Phase Space of prisoner self-harm attractors.

This framework informed the development of the research study:

Results

The results described four male and four female prisoner self-harm attractors defined by a number of bifurcation points (including history, current and future), where key experiences and personal characteristics – such as a history of self-harm outside of prison and/or within prison, a history of contact with a psychiatrist, current feelings of self-blame and inability to concentrate – are responsible for perturbations in the trajectory which led prisoners down one pathway which included repeat self-harm or another other with no repeat self-harm.

The first male trajectory (see Figure 2) describes the shared experience and characteristics of those prisoners all of whom repeated self-harm during the 6-month follow-up. This is a key Attractor which gives clues to the increased probability of repeat self-harm in this high-risk prisoner population when compared with the other Attractors, which map the trajectories of those who do not repeat self-harm during the follow-up. The key differences or points of perturbation between the two most similar male prisoner Attractors 1 (where everyone went on to repeat self-harm) and 4 (where no one went on to repeat self-harm) are the presence or absence of a previous experience of prison ACCT monitoring, and the presence or absence of a number of key experiences and personal characteristics labelled as ‘risk and protective factors’ in the original study.

OPEN IN VIEWER

Figure 2.

Adult male prisoner self-harm trajectories (pathways between clusters).

Findings from the study of male prisoners at ultra-high risk of repeat self-harm suggest the probability of a negative outcome is greatly reduced for those in Attractors 2 and 3 who have no previous experience of self-harm or ACCT monitoring in prison and who report more positive thoughts, feelings, actions and experiences, and fewer negative ones. A number of factors identified as greatly increasing the probability of a negative outcome are shared by both Attractors 1 and 4 including a history of previous self-harm, contact with a Psychiatrist and current feelings of self-blame and inability to concentrate. However, those in Attractor 4 are much less likely to repeat self-harm because they also report a number of features in addition to no previous experience of prison ACCT monitoring which ‘protect’ them from this negative outcome including having obtained academic or vocational qualifications and current access to support from prison healthcare.

Interpreting the female attractors is a more difficult undertaking, not least because the risk and protective factors identified by the original logistic regression do not map as clearly onto the female attractors, again uncovered using a time-ordered cluster method, as they did for the male attractors. The first female trajectory (see Figure 3) describes the shared experience and characteristics of those who repeated self-harm during the 6-month follow-up. The key points of perturbation between the two most similar female prisoner Attractors 1 (where everyone went on to repeat self-harm) and 4 (where no one went on to repeat self-harm) which increase the probability of repeat self-harm include

OPEN IN VIEWER

Figure 3.

Adult female prisoner self-harm trajectories (pathways between clusters).

There is one risk factor shared by Attractors 1 and 4 which increases the probability of repeat self-harm and this involves the receipt of any form of outside communication, that is, family visits, phone calls and letters. Attractor 4 also experiences much more consistent negative ‘symptoms’ compared with attractor 1, but which includes negative thoughts and feelings (including ‘over the last week talking to people has felt too much for me’, ‘anxiety has prevented me doing important things’, ‘I have felt panic’, ‘made plans to end my life’, and ‘felt overwhelmed by my problems’) rather than actions (as in attractor 1, including ‘during the last week I had episodes of binge eating’, ‘over the last week I have been physically violent to others’ and ‘I have scratched myself on purpose’), and which common sense might argue should put them at greatly increased probability of a negative outcome. However, female prisoners in Attractor 4 did not go on to repeat self-harm during follow-up, which suggests that the key features which ‘protect’ them from this negative outcome include no previous experience of prison ACCT monitoring and that reporting negative thoughts and feelings is not the same as actually acting on them.

This empirical example demonstrates how the application of a complex-realist framework and associated methodology and techniques can uncover patterns of similarity and difference at the level of individual cases which, rather than suggesting the social world is either deterministic or stochastic, demonstrates that common or shared contextual experiences and the hidden mechanism of personal characteristics lead to common or shared outcomes. This being the case, rather than applying the ‘cry of pain’ model at a macro- or generalised level, we could use data for individual prisoners with similar experiences to calculate the likely probability of future repeat self-harm events.