Abstract
This article describes the psychometric properties and factor structure of the Working Alliance for Mandated Clients Inventory (WAMC-I). First, we explain how, in contrast to other European jurisdictions such as England and Wales, community supervision in Belgium remains a specific form of social work practice, which is referred to as “social work under judicial mandate” (Devos, 2009: 18). Just as in general social work practice, the professional relationship between practitioners and clients1 is considered to be of paramount importance in community supervision practice. To capture the essence of this professional relationship, we draw on the pan-theoretical concept of the Working Alliance (WA) (Bordin, 1979) and a theoretical adaptation of this concept for the field of community supervision (Menger, 2018). Building on this theoretical adaptation, an instrument to measure the WA was developed for the Dutch context of community supervision: the WAMC-I. The objective of the present study is to assess the psychometric properties and factor structure of the WAMC-I with a sample of justice assistants2 and clients in the Flemish Houses of Justice. This study offers an elaborated conceptualization and operationalization of the concept of the WA. Preliminary tests on the psychometric properties and factor structure of the WAMC-I show that three factors on the WAMC-I for professionals proved valid and reliable: trust, clarity of rules and regulations and reactance. However, tests on the WAMC-I for clients showed no factor solution. Based on the theoretical framework of the WA and scientific-methodological arguments we express our reservations about the use of the WAMC-I and offer suggestions for improvement.
Introduction
Intertwined with broader societal evolutions, penal policy and practice in general and community supervision in particular have been subject to constant processes of change. Certainly, in the UK, these changes have been extensively discussed by scholars. An important theme that emerges from this growing body of literature is that the traditional rehabilitative ideals of community supervision which are rooted in social work values, have been eroded against the background of a developing ‘culture of control’ (Garland, 2001). Deering and Feilzer (2019) have argued that community supervision in England and Wales has been ‘hollowed out’ over the past 40 years. By this they mean that the impact of neoliberal policy, new public management and managerialism has diverted community supervision from its overriding rehabilitative purpose towards managing risks, reducing reoffending, punishment and public protection. From the traditional adage of ‘advise, assist and befriend’, community supervision in England and Wales took a punitive turn and has shifted to ‘confront, challenge and change’ (Barry, 2007). However, research has shown that these shifts in policy and official discourse have not necessarily led practitioners in the field of community supervision to denounce their social work values and belief in the possibility of social reintegration (Deering, 2011; Deering and Feilzer, 2015). It has been argued that probation might have moved away from traditional social work, but meanwhile has shifted halfway back (Raynor and Vanstone, 2016). However, it should be noted that the situation of the probation service in England and Wales remains in constant flux. The most recent episode in the saga of constant change was written only recently (on 11 June 2020 to be precise), when Justice Secretary Buckland has announced the resocialization of probation.
In Belgium, the criminal justice system has equally been in a state of constant reform over the last decades and this has also impacted community supervision within the so-called Houses of Justice (HoJ). However, where the fundamental changes in the probation service in England and Wales can be described as ‘transformational’, Bauwens (2011) considers the reforms within the Belgian HoJ as ‘incremental’ change. A change process leads to transformational change when organizational strategies, systems, structures and values are radically changed, which causes changes in the distribution of power across the whole organization. Change is considered as incremental when management processes are being adapted without fundamental changes to organizational strategies, structures and values (Bauwens, 2011; Burke and Litwin, 1992; Dunphy and Stace, 1993). Although the HoJ have recently been defederalized as a consequence of the sixth State reform 3 , their organizational strategy (reducing re-offending and supporting reintegration), structure (a public sector organization with a central head office and local houses) and values (social work related) have been untouched. Moreover, the traditional allegiance with social work methods and values has been emphasized by reconfirming community supervision as “social work under judicial mandate” in official policy documents (Bauwens and Devos, 2015: 12). The mandate refers to the court rulings that lead to and shape the coerced supervision of offenders. In this specific form of social work practice, community supervision primarily takes place within one-on-one meetings between justice assistants and clients. This suggests that community supervision practice remains to a large extent an interpersonal and relational phenomenon. Therefore, drawing on research regarding common factors of effective practice in the field of psychotherapy, social work and counseling, the paramount importance of the professional relationship between justice assistants and clients should be highlighted (Blay and Boxstaens, 2018).
The Working Alliance (WA) between justice assistants and their clients
The professional relationship between social workers and clients has been described as the core of social work practice by several scholars (Alexander and Charles, 2009; Blom, 2002; Knei-Paz, 2009; Payne, 2014; Perlinski et al., 2012). Professional relationships are considered to be a cornerstone for practice in the sense that they can provide the fundamental conditions in which processes of human change primarily take place (Blom, 2009). Perlinski et al. (2012) point out that social work researchers and practitioners who are interested in the professional relationship have primarily drawn on research and practice experience from the field of psychotherapy in which an abundant body of literature on this topic is present. The concept of the WA has originated from this field but also found its way into social work research and practice. To conceptualize the professional relationship between justice assistants and their clients, we draw on Bordin’s (1979) pan-theoretical concept of the WA. In brief, three important elements shape this WA: an agreement on the goals of the change process, consensus on the tasks that must be completed to achieve these goals and a bond between professional and client that facilitates collaboration (Bordin, 1979). The pan-theoretical concept of the WA claims to be applicable in all human change processes, hence also in community supervision. Research on community supervision practice has shown that it is necessary to invest in strong, supportive professional relationships with mandated clients in order to promote the process of desistance from crime (Burnett and McNeill, 2005). It can be argued that goals, tasks and bond, the three elements of the WA distinguished by Bordin (1979), are also present in community supervision (DeLude, Mitchell and Barber, 2012; Hart and Collins, 2014; Wild, 2011). Nevertheless, there might also be difficulties in applying the concept to the specific situation of community supervision (Skeem et al., 2007). First of all, there is a legal mandate in all forms of community supervision, given by a third party (e.g. a court) that legitimizes the interventions of practitioners, which is different from a therapeutic relationship formed on the basis of a client’s demand for help and framed by mutual agreement. In other words, whereas Bordin (1979) stresses the importance of collaboratively determining mutual goals and tasks to reach those goals, in the context of community supervision goals and tasks are often predetermined by a legal mandate (Bourgon and Gutierrez, 2013; Hart and Collins, 2014; Kozar and Day, 2012). Therefore, it could be argued that the professional relationship in community supervision is actually a triad between practitioners, clients and the mandating party; for example, a sentence implementation court (Klockars, 1972; Skeem and Manchak, 2008). The decisions of the latter can obviously influence the process of community supervision in general and the relationship between practitioners and clients in particular. For example, clients often show a form of reactance against the mandatory character of community supervision (Rooney, 1992) and practitioners have to try and bend this reactance to, at least minimal, compliance with the imposed conditions. This is different from a professional relationship with voluntary clients where there are no imposed conditions and no external mandate. A second element that makes the concept of the WA more difficult to apply to the context of community supervision concerns the dual role that practitioners need to develop. On the one hand, they need to monitor the clients’ compliance with imposed conditions and report to a mandating authority, on the other hand practitioners are expected to support clients in their personal process of desistance from crime. The element of control leads to a disproportionate power structure in the relationship with the client and can make it more difficult to establish a collaborative relationship, let alone a relationship based on trust (Skeem et al., 2007). Effectively balancing the dual role is challenging, but is considered to be a key factor in achieving a good professional relationship in community supervision (Andrews and Bonta, 2010; Trotter, 2015).
Drawing on these critiques and building on earlier work by Ross et al. (2008) and Skeem et al. (2007), research by Menger (2018) has concluded that the three traditional elements of the WA (goals, tasks and bond) are important, yet insufficient to capture the complexity of the professional relationship between practitioners and clients in community supervision. Therefore, the theoretical model of the WA was adapted and expanded. The core of the new theoretical model consists of one negative and three positive characteristics of the WA in a mandated context. The negative characteristic is reactance, the positive characteristics are shared goals and tasks, bond and trust and the clarity of rules and regulations (Menger, 2018).
Shared goals and tasks
Just as voluntary change processes, community supervision is also aiming to achieve certain goals and in function of these goals, certain tasks have to be performed. However, there is a difference since in community supervision goals and tasks are primarily shaped by a legal mandate. Because this mandate is actually the raison d’être for the professional relationship between practitioners and clients in community supervision, the importance of clear, shared goals and specific tasks is paramount (Menger, 2018).
Bond and trust
In the pan-theoretical concept of the WA, bond stands for a collaboration between client and practitioner that is characterized by mutual respect, appreciation and trust (Bordin, 1979). Menger (2018) retains this interpretation in her new theoretical model, but emphasizes the importance of trust by explicitly naming it as a characteristic of the WA in a mandated context. Bond refers to the extent to which clients and practitioners commit themselves to the supervision process and to each other as actors in that process. In order to establish a bond, clients have to have sufficient trust and willingness to openly discuss their needs and problems. On the other hand, a realistic amount of trust of practitioners in the positive intentions of their clients can foster a successful working collaboration. Bond and trust are both characteristics of the WA in a mandated context that serve the achievement of shared goals. This entails that, in order to have a good quality WA in a mandated context, bond and trust have to be strong enough to enable both parties to achieve the goals that are mostly predefined by a legal mandate. Based on the work of Skeem et al. (2007), Menger (2018) argues that in order to develop a bond, a realistic amount of trust should be established between practitioner and client. In other words, the quality of the bond dimension of the WA correlates with the establishment of a realistic amount of trust between worker and client and therefore with sufficient willingness to openly discuss problems and needs (Menger, 2018). To clarify what we understand as a realistic amount of trust, we refer to Smith (2004) who differentiates between ‘confidence’ and ‘trust’ as two distinct aspects of service provision and how this is perceived by clients. The concept of confidence applies to how we perceive abstract systems, while the concept of trust “characterizes our attitude to social exchanges that rely on moral motivation and that have moral consequences” (Smith, 2004: 7). In order to understand the realistic amount of trust between worker and client (Menger, 2018) that is necessary to form a bond, both confidence and trust have to be taken into account. It has been argued that in community supervision practice, the WA between justice assistants and their clients should actually be considered as a triad in which the mandating authority (e.g. probation commission) is the third party (Blay and Boxstaens, 2018; Klockars, 1972). The concept of confidence can be used to characterize how the abstract system of community supervision, embodied by the mandating authority, is perceived by clients. The concept of trust relates to the WA between practitioner and client and arises from the direct interaction between both. Whether a client develops confidence in the system of community supervision depends on indications that community supervision is well designed for its purpose and that practitioners are seen as technically competent for the job. However, whether a client develops trust in the practitioner is related to the latter’s moral competences, which entails that practitioners show respect, honesty, optimism and care. This is supported by recent research that shows that – specifically in a criminal justice setting – people are more likely to trust the involved practitioners and the system as a whole, when they feel they are treated fairly (Kaiser and Reisig, 2019). Smith (2004) argues that both confidence and trust are necessary to adequately address client’s need for effective practice and moral competence. These findings support the paramount importance of trust as a feature of the WA in a mandated context (Menger, 2018). However, it should be noted that – while it is indeed built within a specific WA – trust remains a complex concept that is influenced by many factors; for example, personal experiences or the experiences or significant others (Fagan and Tyler, 2005) and cultural background (Boateng and Kaiser, 2019).
Clarity of rules and regulations
Clarity of rules and regulations is a fourth characteristic in Menger’s (2018) theoretical model of the WA in a mandated context. It is derived from the concept of relational fairness that is considered to have a large impact on the quality of the WA in a mandated context (Skeem et al., 2007). Relational fairness refers to how clear the framework of supervision, mutual obligations and the limits of the mandate are expressed and to which extent the rules and regulations of supervision are consistently, but also flexibly and respectfully enforced throughout the supervision process. Upholding relational fairness avoids unrealistic expectations from clients and possibly keeps practitioners from developing an authoritarian style that would have a negative impact on the quality of the WA (Menger, 2018). While Bordin (1979) stresses the importance of collaboratively determining mutual goals and tasks, in the context of community supervision goals and tasks are often predetermined by a legal mandate (Bourgon and Gutierrez, 2013; Hart and Collins, 2014; Kozar and Day, 2012). Therefore, providing clarity about the rules and regulations that are constituted in this legal mandate is imperative. Nevertheless, community supervision practice also entails taking into account clients’ personal expectations and needs: not all goals can be achieved at the same time, hence the need for mutual agreement on which objectives should be prioritized remains important. Therefore, also in community supervision practice the importance of reflective listening is highlighted (Trotter, 2015; Vanstone and Raynor, 2012).
Reactance
In community supervision practice, clients are forced to engage with practitioners by a legal mandate. Rooney (1992) has defined interactions in a mandated context as an involuntary transaction. This entails that clients can develop a natural response against the feeling of being restricted in their autonomy. This natural response is called reactance and is considered to be a characteristic of involuntary clients (Rooney, 1992). However, in Menger’s (2018) theoretical model, reactance is considered as a characteristic of the interaction between client and practitioner that reflects negatively on the WA. Reactance can be influenced by both parties. In community supervision, clients can show reactance for example by actively or passively challenging the involuntary character of supervision in general or one or more conditions that are imposed in particular. Especially when client reactance is directed towards the practitioner, this can have a negative impact on the quality of the WA. However, also practitioners can have a negative impact on the WA by not balancing their dual role and rigidly focusing on their controlling tasks or surveillance role (Skeem and Manchak, 2008). In doing so, they can elicit reactance of clients which again can lead to a more one-sided punitive stance of the practitioner.
Measuring the quality of the WA: Development of the Working Alliance in a Mandated Context Inventory (WAMC-I)
In a recent meta-analytical review, Flückiger et al. (2018) have estimated that there are currently more than 70 different instruments being used in empirical research on the WA. The lion’s share of research papers published on alliance measures refer to Bordin’s (1979) pan-theoretical concept of the WA as an underlying theoretical construct and measure the quality of the alliance in correlation with therapy outcome. Depending on the context, researchers use an adapted measure to operationalize what they all define as the WA. There seems to be a fairly stable and broad consensus as to the most important features that need to be measured, but nevertheless differences between measures persist (Elvins and Green, 2008). Hence, the lack of a persuasive and consensual definition has led to a situation where researchers from different fields and theoretical backgrounds could embrace Bordin’s (1979) concept of the WA and creatively adapt it to fit their specific research context.
A review of the most common alliance measures available in literature in the field of adult psychotherapy summarizes the most important differences between measures (Ardito and Rabellino, 2011). In brief, instruments vary in terms of the operational definition of the alliance, the assessment method (self-report scales, observation scales, rating scales, etc.), the source of information (perception of client, therapist or observer); the number of items in the scale (between 6 and 145 items) and psychometric properties (Ardito and Rabellino, 2011).
Amongst the large variety in measuring instruments, the Working Alliance Inventory (WAI; Horvath and Greenberg, 1986) is clearly the most widely used measure of the WA (Doran, 2016). To be precise, the most recent meta-analytical review showed that 69% of the articles included in the study used an instrument that incorporated WAI-items (Flückiger et al., 2018).
Based on the aforementioned theoretical model of the WA in a mandated context (see Figure 1), a new instrument – the WAMC-I – was developed to measure the quality of the WA between practitioners and clients in the Dutch context of community supervision. This instrument is the result of a longitudinal study in the Dutch probation service, conducted by independent researchers 4 (Menger, 2018; Menger and Donker, 2013). The researchers used previously developed questionnaires to create the aforementioned theoretical model of characteristics that determine the quality of the WA (Menger, 2018; Menger et al., 2013). The instruments that were used in the Dutch research are: (1) the WAI (Horvath and Greenberg, 1986); (2) the WAI Short-Form (WAI-S) (Tracey and Kokotovic, 1989); (3) the Offender-Version of the WAI Short-Form (WAI-S Offender-Version) (Tatman and Love, 2010); and (4) the Dual-Role Relationships Inventory-Revised (DRI-R) (Skeem et al., 2007). The WAMC-I measures the aforementioned characteristics of the WA in a mandated context that are most predictive for its quality. These characteristics are operationalized in 19 statements that can be answered on a 7-point Likert scale ranging from strongly disagree to strongly agree and – in the middle – a neutral position. In brief, 14 out of the 19 items are adopted from the WAI-S Offender Version (Tatman and Love, 2010) and the DRI-R (Skeem et al., 2007). Five items are added by the Dutch researchers (Menger, 2018). The 19 items that operationalize the WA in a mandated context are assigned to four subscales: trust, bond, clarity of rules and regulations and reactance 5 (Menger, 2018; own translation). There are two versions of the WAMC-I, one for professionals and one for clients. The two versions mirror the same statements.

Frequency distribution total WA scores (TOTSCRWP).
The present study
As argued in the introduction, community supervision practice in Flanders is primarily relationship-based. Up until now, justice assistants do not use standardized instruments in their professional practice in general and certainly not to measure the quality of their WA with clients in particular. Therefore, the present study aims to pilot the use of the WAMC-I in the HoJ using Principal Factor Analysis. This article focusses on one element of a larger, mixed-methods study in which the use of the WAMC-I is not merely explored as a measure for the quality of the WA, but also as a tool for practice that facilitates joint reflection between justice assistants and clients (Boxstaens, 2019).
Method
Participants
In this study, the participants are dyads of justice assistants and clients within all of the 14 Flemish HoJ. Dyads were initially selected on the basis of three criteria: (1) within the broader field of community supervision, the focus lies on probation as it is – next to fines – the most used alternative to imprisonment that is applied in Flanders. Due to chronic prison overcrowding the political support for community supervision in general and probation in particular has grown in the last two decades (Bauwens and Devos, 2015). Hence, all dyads in this study consist of clients under a probation mandate and justice assistants who are following up on this mandate; (2) all selected dyads are formed recently, which means that in all cases justice assistants and their clients had minimum one and maximum four one-on-one meetings. In other words, in all cases justice assistants and clients were in the phase of establishing or developing their WA; (3) in an attempt to obtain a representative research sample, dyads from all 14 Flemish HoJ were included. Based on these three criteria, the HoJ’s central administration provided us with a database containing 618 dyads of potential respondents. The minimum sample size for our study was set according to the rule of thumb of at least seven participants per item in the WAMC-I to perform factor analysis. Hence, the minimum number of respondents required was 133 (7 X 19) (Terwee et al., 2007). In order to avoid bias and in an effort to minimize the impact of our research for daily practice, justice assistants could participate in maximum two dyads. In total, 300 dyads were randomly selected from the aforementioned database. Taking into account the criteria of a maximum of two dyads per justice assistant and the representation of all 14 HoJ in the sample, 248 dyads were invited to participate in the study on a voluntary basis. This procedure resulted in the voluntary participation of 118 justice assistants and 105 clients. The initial non-response can primarily be explained by the refusal of clients to participate in the research (+/- 35%), the selection of case files that were in the provided database, but in reality were inactive (e.g. due to the fact that the case file was put on a waiting list; +/- 25%) or other, various reasons (e.g. client was referred to another justice assistant, after selection client was imprisoned or admitted to a psychiatric hospital or the justice assistant argued that the client was not fit to participate due to mental health issues; +/- 40%).
Eighty-five percent of all clients that participated in our study were male, 15% were female. The justice assistants in our sample were predominantly female (92%), only 8% was male. These numbers are consistent with the overall situation regarding probation in the Flemish HoJ. On average, the justice assistants in our sample had 9.9 years of professional experience in probation. 6 Our sample shows a slight underrepresentation of justice assistants with a professional experience over 10 years. In a previous study conducted in the Flemish HoJ, a representative sample of justice assistants had on average 13.56 years of professional experience (Uzieblo et al., 2019).
Measure
The instrument used in this study comprises a custom-built scale: the WAMC-I (Menger, 2018). In this scale, four components of the WA, namely reactance, shared goals and tasks, bond and trust and the clarity of rules and regulations, are operationalized in 19 items. The response scales were 5-point Likert scales with the following options: strongly disagree, disagree, neither agree nor disagree, agree, strongly agree. We retained the option of a middle or neutral choice, as all questions in the survey were obligatory and could not be skipped. The original Dutch version of the WAMC-I used a 7-point Likert scale, which is advocated to increase the probability of meeting the objective reality of people (Joshi et al., 2015). However, the experience of the Dutch research team showed that the added value of using a 7-point Likert scale proved to be limited in their research. In a first phase of our study, we extensively discussed the original Dutch version of the WAMC-I with an expert group consisting of five directors of local HoJ and five focus groups with justice assistants from all Flemish HoJ. In these focus groups, we have worked on content validity by providing clear descriptions of the measurement aim, the target population, the concepts that are being measured, and the item selection. The target population and experts were involved in item selection. Based on findings of the focus groups and the recommendations of the expert group, several adaptations were made to the original Dutch version to ‘translate’ the 19 items and make the instrument suitable for use in the Flemish context of community supervision. This was necessary because, although Dutch is a common language in the Netherlands and Flanders, the original Dutch version contained words and phrases that are not used in Flanders. Next to translating the WAMC-I, we also opted to use a 5-point Likert scale instead of the original 7-point scale because the latter was considered as potentially confusing by respondents in the focus groups. 7 By investing in the involvement of practitioners and local directors in the process of adapting the WAMC-I to the Flemish context of community supervision, we sought to create support for the instrument and increase justice assistants’ willingness to engage in our study. This process resulted in two Flemish versions of the WAMC-I, one for professionals (WAMCI-Ip) and one for clients (WAMC-Ic). The two versions mirror the same statements, which are shown and linked to the respective subscales of the WAMC-I in Table 1. To test the psychometric properties of a scale, quality criteria for content validity, internal consistency, criterion validity, construct validity, reproducibility, longitudinal validity, responsiveness, floor and ceiling effects, interpretability and model fit, need to be met (Singer, 1998; Terwee et al., 2007). Moreover, cooperative knowledge production in various phases of the research creates a relevant instrument that could be applied, accepted and valued in the practice of social work (Depauw and Driessens, 2017; van Dop et al., 2016). In this study, we performed some preliminary tests on content validity, internal consistency and construct validity by performing a principal factor analysis and adapted the WAMC-I through the use of focus groups.
The WA in a mandated context inventory.
Adapted from Menger (2018: 128); own translation. 3
X = justice assistant (practitioner).
Y = client.
Obviously, the Flemish version of the WAMC-I is in Dutch, but for the purpose of this article, the different items were translated to English by the researchers.
Procedure
All requested and necessary permissions to carry out the research were acquired. After obtaining authorization, all of the justice assistants received information about the aim of the research. They were guaranteed anonymity by working with codes and not adding names or other features that would make identification possible. Sample collection took place in February–March 2017. Due to privacy reasons, we were provided with a database containing a limited number of variables. Most importantly, the HoJ could only provide us with a case file number and the name of the justice assistant involved. Therefore, the involved justice assistants were contacted by mail or phone and provided with a brochure containing information on the study and an informed consent form, which they could pass on to the involved client(s). When clients were informed and indicated their willingness to cooperate, the informed consent form was filled out and sent back. We acknowledge this procedure might have led to some form of selection mechanism. The justice assistants that agreed to participate in our study received an e-mail containing a link to an online version of the WAMC-Ip. Clients were provided with a paper version of the WAMC-Ic by their justice assistant. On average, it took clients and practitioners 10 to 15 minutes to complete their version of the WAMC-I. After filling out the instrument, clients could put the form in a sealed envelope that was then sent to the researchers who then entered the data in SPSS.
Data analysis
The aforementioned research procedure resulted in the collection of 118 completed WAMC-Ip and 105 WAMC-Ic. We started our analysis by conducting Kaiser-Meyer-Olkin (KMO) and Bartlett’s tests on the WAMC-Ip. Results for KMO (0.87) and the significant Bartlett’s test (p< 0.001) showed that the variables masked common variation that would suggest the need to conduct factor analysis on this group of variables.
The next step in the data analysis consisted of evaluating the various dimensions of the WA. Cronbach’s alpha scores were calculated in order to assess the internal consistency of the measurement scales, which were composed of the items that had previously been assigned to the theoretical components of the WA.
We then conducted exploratory factor analysis. More specifically, we conducted principal factor analysis (PFA) using SPSS 25. Exploratory factor analysis offers the advantage of identifying unexpected factors and ambiguous items that load on more than one factor. We checked to determine whether the scale was unidimensional or multidimensional using a scree test and calculating the proportion of explained variance (Mortelmans, 2009). The analysis of the WAMC-I for professionals revealed four factors thus indicating multidimensionality.
We ran the same procedures for the WAMC-Ic where the KMO measure of sampling adequacy (0.84) and the significant Bartlett’s test (p< 0.001) also suggested the need to conduct factor analysis. However, the exploratory factor analysis of the WAMC-I for clients revealed no factor-solution.
Cronbach’s alpha scores of the WAMC-Ip were calculated in order to assess the internal consistency of the measurement scale. Both statistical and theoretical arguments were used in the interpretation of the factors and corresponding items. Ambiguous items were eliminated from the analysis.
According to Costello and Osborne (2005), strong data in factor analysis means uniformly high communalities without cross loadings, plus several variables loading strongly on each factor. In order to be retained, indicators and factors must meet certain criteria. Firstly, an indicator should have a factor loading above 0.4. Secondly, an indicator should not load on more than one factor. And, last but not least, an appropriate number of indicators for each factor is between three and six (Costello and Osborne, 2005; Devellis, 2012; Hatcher, 1994).
Results
An exploratory factor analysis or – to be more precise – a principal factor analysis (PFA) was conducted using SPSS 25 starting with all 19 variables. To determine whether the scale was unidimensional or multidimensional, a scree test was used to calculate the proportion of explained variance. These analyses revealed four factors, thus indicating multidimensionality. The KMO (0.87) and Bartlett’s test (p< 0.001) also indicated common variation which suggests the need to conduct factor analysis on this group of variables. The rotated factor matrix showing the factor loadings is presented in Table 2. Items with lower loadings than 0.4 have been deleted from the table. Three indicators that loaded on more than one factor were also deleted (item 12. I think Y cares for me as a person; item 10. Y and I agree on the goals we are working on during the probation period; and item 16. I think Y finds it hard to comply with the imposed conditions).
Rotated factor matrix. 1
Extraction method: Principal Axis Factoring; Rotation Method: Varimax with Kaiser Normalization: Rotation converged in six iterations.
As demonstrated in Table 2, only three factors meet these criteria and have an appropriate number of indicators for each factor. The subsequent step entailed the possible theoretical and content-based groupings for the indicators that loaded on the same factor.
Cronbach’s alpha (α) scores were calculated to assess the reliability of the subscales, which were based on theory.
In general, scales with Cronbach’s α scores above 0.70 are considered reliable, although some authors consider values above 0.60 as indicating reliable scales (Bryman, 2004; Devellis, 2012). With regard to internal consistency, we identified one scale with good (factor 1: α = 0.88), and two scales with acceptable reliability (factor 2: α = 0.63; factor 3: α = 0.66).
The first factor combines eight items that were originally categorised under four dimensions in the Dutch version of the WAMC-I (Menger, 2018): the first four items (4, 5, 2 and 3) are part of the trust-scale, items 15 and 14 belong to the bond-scale and item 17 is one of the items of the reactance-scale. According to the initial factor structure presented by Menger (2018: 127) item number 12 loads on both the bond-scale and the clarity of rules and regulations-scale. The internal consistency measures show that deleting item number 12 from our first factor can slightly increase its internal consistency (α = .887 > .882). The seven remaining items all refer to the concepts of trust or bond. In Bordin’s pan-theoretical framework (1979) on the WA, no distinction is made between trust and bond. Both concepts are captured by the bond-dimension. Going back to the results of our PFA of the WAMC-Ip, all items in the first factor relate to what is referred to by Smith (2004) as ‘trust’. All items address the direct interaction between practitioner and client in general and more specifically the moral competences that are necessary to build trust. We therefore decided to designate this first factor as ‘Trust’.
The second factor combines five items that were initially placed under two dimensions in the Dutch version of the WAMC-I (Menger, 2018): the first item (13) is a part of the bond-scale, all other items (6, 9, 8 and 7) belong to the clarity on rules and regulations-scale. In the pan-theoretical concept of the WA, Bordin (1979) refers to a mutual agreement on the goals of an intervention and the tasks that need to be carried out to achieve those goals. The original Dutch version of the WAMC-I tries to capture this mutual agreement on goals and tasks in the specific context of community supervision under the ‘clarity of rules and regulations’-scale. This is also reflected in the results of our PFA of the WAMC-Ip. Item number 7 refers directly to the specificity of a mandated context in which non-compliance with imposed conditions or agreements can lead to ruptures in the WA. Moreover, in the context of community supervision, breach of imposed conditions might even lead to revocation of the probation measure, provoking modification of conditions or even imprisonment. Items 6, 9 and 8 refer to the goals and tasks in community supervision. The importance of reflective listening in community supervision practice (Vanstone and Raynor, 2012; Trotter, 2015) provides an explanation for the fact that, in our model, item number 13 also loads on this factor: in order to establish a WA, practitioners need to combine providing clarity on the legal mandate that determines the imposed conditions with taking into account the personal expectations of the client. In order to do so, they need to display good listening skills. Based on its content and the aforementioned theoretical arguments, we designated this factor as ‘clarity on rules and regulations’.
The third factor combines three items that initially were placed under two dimensions in the Dutch version of the WAMC-I (Menger, 2018): the first (19) and last (18) item belong to the ‘reactance’-scale, the item in the middle (1) is a part of the ‘trust’-scale. As mentioned in the theoretical framework, ‘reactance’ is considered as a characteristic of the interaction between client and practitioner that reflects negatively on the WA. For example, item number 19 refers to the fact that after a meeting, a client quickly forgets what is discussed. However, ‘quickly forgetting’ is not merely considered as a client characteristic, it is explicitly attributed to the interaction between worker and client. Item number 1 refers to a client withholding important information from the practitioner. In the Dutch version of the WAMC-I, this item is considered as a part of the ‘trust’-scale. We argue that withholding important information could also reflect negatively on the WA and therefore should also be seen as a form of reactance. Given this interpretation, all items in factor 3 relate to the concept of ‘reactance’ and therefore we decided to designate this factor as such.
Based on the three aforementioned subscales, we calculated the sum scales and a total score on 100% for the quality of the WA. Table 3 contains basic statistical measures for the total WA score, Figure 1 shows that the WA scores approximate a normal distribution just as many other psychological and educational variables. This means that many kinds of statistical tests can be derived from this distribution, which makes it a useful variable for statistical research.
Basic statistical measures for the total WA score.
Next to assessing the quality of the WA with their client by completing the WAMC-Ip, justice assistants were also asked how satisfied they were with the quality of the WA with the involved client at that particular moment in time. Again, a 5-point Likert scale was used ranging from very satisfied to very dissatisfied. Statistical analysis shows a strong and positive correlation between the total WA score and general satisfaction about the WA reported by justice assistants (r = .735; p< .000 2-tailed). This indicates that the WAMC-Ip is able to capture practitioners’ general (dis)satisfaction about the WA with clients.
Also, for the WAMC-Ic a PFA was performed starting with all 19 variables. A PCA suggested five factors. However, the results emerging from the SPSS output indicated no factors-solution. Seventeen out of 19 indicators showed double loadings. We also calculated a total WA score for clients and correlated that with the total WA score for professionals. Also, this effort did not show any significant relationship (τ = .171; p > .05 2-tailed; r = .242; p > .05 2-tailed).
Discussion
In this discussion, we address a number of methodological and conceptual issues regarding our study. We critically consider the limitations of the present study and offer suggestions for further research.
Methodological issues
Limitations of this study include the sample size and method through which the participants were selected. Taking into account that the largest part of non-response in this study can be explained by the refusal of clients to participate in the research, we can assume self-selection through non-participation. This leads us to the hypothesis that primarily docile clients participated in the study and presumably produced socially desirable responses on the different WAMC-Ic items. Combined with the responses of other clients who did not answer in a socially desirable way, this may have led to the no-factors-solution in the PFA on the WAMC-Ic. Moreover, social desirability might also be an issue in the answers produced by practitioners. In this perspective it should be noted that research has shown discrepancies between the information that practitioners provide about their practice in surveys and interviews, and what they actually do (Bauwens, 2011).
Furthermore, the target group of clients is characterized by specific needs with regard to the context of social work under judicial mandate, their language level and various cultural aspects of the terminology used in the WAMC-I. Although we have tried to adapt the instrument to relatively low language skills, foreign language backgrounds and/or cultural differences, the WAMC-I might require further improvement.
Conceptual issues
The professional relationship between practitioners and clients in community supervision practice is conceptualized as a WA in a mandated context which is operationalized in four subscales: goals and tasks, bond and trust, clarity of rules and regulations and reactance (Menger, 2018). The importance of the WA in the fields of psychotherapy, counselling and social work has been advocated by many scholars (e.g. Horvath, 2018; Menger, 2018; Perlinski et al., 2012). Nevertheless, others have argued that the WA only accounts for about 7.5% of ‘what works’ in effective practice (Norcross, 2011). Wampold and Imei (2015) have found that context factors and client characteristics actually represent a much larger part of the explained variance (86.7%). We therefore suggest that it might be a good idea to integrate context variables and client characteristics in the WA measure or the analyses of data derived from the WAMC-I.
A specific feature of community supervision practice is that a third party (e.g. a court) mandates practitioners to engage with clients. Decisions from this mandating authority can influence the process of community supervision in general and the WA in particular. This might be an issue for the validity of the measure. For example, the WAMC-I claims to measure reactance as one of the core characteristics of the WA. However, the question remains what form of reactance is actually measured. Does the client version of the WAMC-I measure reactance against the strict nature of the imposed conditions (and thus reactance against the mandating authority)? Or does it measure reactance towards the way the imposed conditions are presented and translated by the justice assistant? Drawing on this validity issue, we propose to broaden the scope of the WAMC-I. To do right by the complex reality of community supervision practice in general and the WA in a mandated context in particular, it might be necessary to include more variables; for example, motivation and other contextual or client characteristics. In that case, a multilevel analysis would be preferable, which means that the WAMC-I should be complemented with other measures. Inspiration for this can be found in the Basic Psychological Need Theory (BPNT). Within BPNT, the psychological needs for autonomy, competence, and relatedness are considered critical for the prediction of individuals’ growth and well-being, problem behavior and psychopathology (Ryan and Deci, 2000; Vansteenkiste and Ryan, 2013). To capture both the satisfaction and the frustration on this, the Basic Psychological Need Satisfaction and Frustration Scale (Chen et al., 2014), was developed, which included a balanced combination of satisfaction and frustration items. The scale has been successfully used in a number of publications (Campbell et al., 2015; Chen et al., 2014; Chen et al., 2015, Haerens et al., 2015). Contextual variables and sociodemographical variables can be taken into account as control variables.
A third conceptual issue targets the incomplete conceptualization of the concept of the WA in a mandated context. As described, we have ‘translated’ the initial Dutch version of the WAMC-I to the Flemish context of community supervision practice. This has led to divergent results that suggest the practitioner version of the WAMC-I may be a reliable instrument to measure the WA in Flemish community supervision practice, but questions the reliability and validity of the client version. A solution might lie in the adaptation of the concept of the WA in a mandated context by integrating variables that operationalize characteristics of Flemish community supervision practice that are explicitly mentioned in policy documents: respect, authenticity, empathy, an appreciative and supporting attitude, a strengths-based approach, and so forth (Bauwens and Roose, 2017). However, as research has already shown that there might be a discrepancy between what is articulated in policy documents and what is actually happening in practice (Bauwens, 2010, 2011), it seems appropriate to empirically study whether the aforementioned characteristics are actually reflected in contemporary practice. In addition, it might be necessary to differentiate between the practitioner version and the client version of the WAMC-I. In their current state, both versions mirror the same items, which is in line with other WA measures (e.g. WAI; Horvath and Greenberg, 1986). In order to develop a client version of the WAMC-I, it might be necessary to abandon this mirrored way of assessing the WA and is entirely adapted to the situation and point of view of clients (Keijsers, 2014). For future research, it will be important to develop this adapted client version since earlier studies have already shown that the correlation between outcome and the quality of the WA is stronger when assessed by the client (Horvath, 2018; Wampold and Imei, 2015).
Conclusion
In this article, we contextualized community supervision in Flanders as “social work under judicial mandate” (Devos, 2009: 18). This traditional allegiance with social work methods and values has recently been reemphasized in Flanders, while in other jurisdictions (e.g. England and Wales), community supervision practice has moved away from social work. Within this social work, the paramount importance of the professional relationship between worker and client has been firmly established (Alexander and Charles, 2009; Blom, 2002; Knei-Paz, 2009; Payne, 2014; Perlinski et al., 2012). In this article, we conceptualized the professional relationship between justice assistants and clients in the Flemish context of community supervision as a WA. We piloted the WAMC-I, a measuring instrument for the quality of the WA in the Dutch context of community supervision (Menger, 2018), in the Flemish HoJ. Our study showed that the practitioner version of the WAMC-I, adapted for the Flemish context, seems a reliable instrument to measure the quality of the WA as it is perceived by professionals. PFA revealed a three-factorial model instead of the four-factor model of the initial Dutch version. Further research on validity and reliability is needed, for instance through a confirmatory factor analysis (CFA) (Devellis, 2012; Singer, 1998; Terwee et al., 2007). PFA on the client version of the WAMC-I indicated no factors-solution. This leads us to the conclusion that the current version of the WAMC-Ic is not a valid instrument to assess the quality of the WA in Flemish community supervision practice. However, we advocate further research to adapt the instrument integrating other important variables as motivation and reactance. In doing so, it might be necessary to abandon the usual strategy of mirroring client and practitioner versions of measurement instruments on the WA (Keijsers, 2014).
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Research Foundation Flanders (FWO) through a Special PhD Fellowship.
