Abstract
Conducting a public consultation is a popular way to draw on wider expertise in framing legislation in the UK. In Wales, low scrutinising capacity of a relatively small legislative chamber and limited civil service resources to prepare legislation may contribute to the popularity of consultations. Public consultations may also resonate with themes of inclusion and participation in Welsh governance. The Social Services and Well-being (Wales) Bill was the first large legislative project of the Welsh Government since gaining primary law making powers in 2011. This case study investigated the public consultation (conducted at Stage 1) for this bill in 2013. It used a coding matrix to analyse the submissions to the consultation. The findings reveal that individuals and organisations may struggle to effectively influence legislation. Using van Damme and Brans’ interpretative framework, the article locates the findings within the context of citizen participation, consultation techniques and discusses the usefulness of analysing submissions as part of consultation evaluations. In addition, the article makes a case for triangulating a documentary analysis of submissions with conventional qualitative evidence in future consultation research.
Background
Consultations have become a popular vehicle for policy makers to involve the wider public in a range of decision-making processes. Consultations are characterised by a plurality of means, aims and outcomes. There is now a burgeoning empirical literature in this field which has prompted the production of a sizable body of theorising on public consultations (Culver and Howe, 2004; Fishkin, 2009; LSE GV314 Group, 2012). Most theories are located within the fields of public administration or policy making and cover parliamentary consultations, consultations for policy making by central and devolved governments or consultations by public sector organisations such as the National Health Service. However, with some exceptions (Murray et al., 2009), there has been little empirical work about the consultations in devolved governments such as Wales, Northern Ireland and Scotland. Nor, to our knowledge, have there been any studies that systematically examine submission evidence to consultations.
Investigating public consultations organised by devolved executives or legislatures may increase our knowledge about the process of devolution itself, but more importantly, it may tell us more about how devolved legislatures and governments utilise external expertise to inform legislation under conditions of limited internal civil service capacity (McAllister and Stirbu, 2007). This article reports the findings of a case study of the public consultation for the Social Services and Well-being (Wales) Bill (henceforth: Social Services Bill). The consultation was part of Stage 1 of legislative scrutiny by the Welsh Assembly. The bill became law in April 2014.
The article outlines the Welsh background of devolution following the 2011 referendum and the theoretical framework that has been developed around legislative capacity in devolved government as a central concern for public engagement. It then sketches the existing literature on public consultation relating to legislative matters. This allows for the contextualisation of the aims and objectives of the study. In the third section, the article reports the method and findings of the study to be rounded off by a discussion and recommendations for future research.
Welsh devolution and Welsh referendum of 2011
As this case study about a consultation in Wales potentially offers particular insights into consultation techniques in a devolved context, it seems important to provide some political context. Welsh politics is shaped by the predominance of the Welsh Labour Party which has continuously been in power since devolution in 1999, even though at times in coalition with a junior partner. The dominance of Labour in Wales throughout devolution means that the party’s governing philosophy largely informs governance structures and mechanisms of participatory democracy. The principles of public consultation would appear to neatly align in Wales with Labour’s claim about inclusive politics, defined in contradistinction from the (allegedly) adversarial Westminster model of governing underpinned by competition of interests and lack of consensus building between commercial and public sectors. This particular Welsh context of supposed participatory and inclusive politics thus provides the backdrop for this case study. Public consultations would presumably play an important role in realising this aspiration.
Richard Rawlings (2003) notes that the National Assembly for Wales (hence: the Assembly) was ‘formally empowered’ in July 1999 which followed the narrowly won referendum for devolution in September 1997. By July 2000, the first full session of proceedings of the Assembly had been completed. The devolution settlement in Wales is complex and has seen gradual development rather than a clear delineation of powers at the outset. The formal separation of the executive and legislature occurred in 2006 which laid the ground for the 2011 referendum on further devolution of powers. Today, the Welsh Government can make primary legislation in 20 devolved areas of policy. In general, the devolution settlement is one of conferred powers. It means that all legislation is still subject to the question of whether it is within the competence of the Welsh Government. Challenges to the competence of the Welsh Government have repeatedly occurred.
In the first 15 years of devolution, the Welsh Labour Party has always been in power either as a minority government or within a coalition. Because of the small number of Assembly Members (AM), questions are frequently raised about the capacity of the Assembly to scrutinise the Welsh Government (Cole, 2014; Cole and McAllister, 2014). This issue has become particularly urgent following the adoption of new legislative powers in 2011. Scrutenising capacity is predominantly seen as a function of the availability of AMs not occupying governmental roles. Currently, there is a Welsh Labour minority government supported by 30 Welsh Labour AMs. Plaid Cymru have 11 AMs, the Welsh Liberal Democrats have 5 AMs and the Welsh Conservatives have 14 AMs. Additionally, 12 of the Welsh Labour AMs are Ministers or Deputy Ministers which again raises questions of capacity for scrutiny. With these concerns as to chamber size and scrutiny capacity, external expertise gained through public consultations is likely to play a large role in the legislative process in Wales both in terms of drafting and revising proposed legislation.
The Social Services Bill was the first major legislative project of the Welsh Government since the 2011 referendum. The Bill aimed to enshrine a ‘person-centred’ approach in social services but questions were raised throughout the scrutiny process on how this could be achieved. The Bill was subject to Stages 1 to 3 of scrutiny, with an extra Stage 3 report stage for additional amendments. The public consultation on the Social Services Bill was conducted at Stage 1 of the parliamentary process.
The Welsh context is characterised by the prevalent government discourse of partnership work, consensus building and inclusive politics in contradistinction from adversarial, competition-based Westminster (or English) model of governance (Welsh Assembly Government (WAG), 2007). The Welsh Government has consistently used language that emphasises participation, inclusion and consensus over competition, special interests and hierarchical relationships. Consultations can be a testing ground for those claims. Arnstein (1969) and in his wake most students of public consultations note that consultations are one step on the ladder of participatory politics. Analysing the inclusiveness of consultations may thus provide supporting or contrary evidence for the Welsh Government’s claim about inclusive politics.
Whilst inclusion and participatory governance is a key theme of the Welsh Labour Party, and thus of the Welsh Government (WAG, 2007; Welsh Local Government Association, 2012), observers noticed that the frequent and public rejection of the principle of choice and competition in Welsh political discourse feeds into a de-legitimisation of market forces and special interest articulation (Mitchell, 2006; Moffatt et al., 2012). The First Minister of Wales, Carwin Jones, repeatedly in chamber expressed satisfaction that private companies had no place in health-care provision in Wales. This furnishes an important aspect of the atmospheric context in which the Welsh Government conducts public consultations in social care services (our case) which is closely linked to the health-care sector by virtue of complex disease management requiring cross-sectoral collaboration, care provision in the community and co-operation across provider networks to implement care pathways for patients and service users (Guarneros-Meza et al., 2014; Kaehne, 2010, 2014a).
In particular, the Welsh Government claims about the preponderance of participatory forms of governance and inclusion speak to an increased role of consultation in the legislative process within a radically altered governance landscape (Cole, 2012).
Another aspect of Welsh governance is the slow but steady re-emergence of centralistic forces in the Welsh political field. With the introduction of the National Assembly for Wales and the transfer of power to Cardiff in 1999, the National Assembly for Wales (and subsequently the Welsh Government) arrogated executive powers to itself previously held by local authorities. In a paradigmatic way, the Social Services and Well-being (Wales) Bill contained provisions for ministers in Cardiff to set service requirements for local authorities that were previously left to them to define. The rationale publicly articulated in defense of this incremental centralisation is based on the need for standardisation of service quality, the removal of post code lottery effects across the principality and the right to equitable service quality throughout the home nation. Commentators, however, note that it also resembles government by diktat and an impetus to implement changes by utilising hierarchical structures that the Welsh Government publicly disavows (Guarneros-Meza et al., 2014; Moffatt et al., 2012; Rawlings, 2003).
The literature on public consultations
Arnstein (1969) formulated the paradigmatic interpretative framework for public consultations and argued that their key characteristic lies in the role they play in bringing about participation. Arnstein indicated that there are various steps on the ladder to a perfectly inclusive and participatory democracy and she located public consultations firmly within the tokenistic section of the ladder (Arnstein, 1969). Whilst she focused on consultations as a means to improve participatory politics, shared decision making is not the only context in which public consultation can be placed. As mentioned, public consultations have also received attention as vehicles to access external expertise for organisations (Cole, 2014; Mitchell, 2006; Rawlings, 2003). The dual role of consultations as participatory means and access vehicle to external expertise thus neatly circumscribes the space between political science and public management, outlining the boundaries of either field and their related yet distinct realms of theoretical and conceptual work.
Arnstein’s critical typology of participation apart, there is a surprising overlap between early conceptualisations (such as Hogwood, 1986) and later conceptual work. Theories of consultations commonly differentiate between various aims and objectives of consultation or types of consultation distinguished in terms of participants and techniques. They are often predicated on the assumption that governments and parliamentary assemblies are genuinely interested in the involvement of some or all sections of the public in policy making, an assumption criticised implicitly by Arnstein, met with some skepticism by others (Hogwood, 1986) and not always borne out by empirical analysis (Cheeseman and Smith, 2001).
More recently, the tension between tokenistic and genuine participation in the legislative process has become a key concern in the literature and emerged as a central evaluative criterion in empirical studies. This concern appears to provide a potential link between the literature on shared decision making in public services (Hudson, 2015; Jones and Einsiedel, 2011; LSE GV314 Group, 2012; Montpetit, 2003; OECD, 2003; Thunus and Schoenaers, 2012) and the role of professionals in public sector organisations (Cook, Crase et al., 2005). So far, however, with a few exceptions (Cole, 2004), empirical studies examining the role of consultation in the legislative process remain strangely dissociated from this wider debate of shared decision making in public services.
There are also studies highlighting that consultations remain subject to the overall strictures of the policy-making process. Johnston’s article gives an example of a consultation that did not happen because the legislation itself was scrapped (Johnston et al., 2013). Few studies utilise a theoretical framework for investigating consultations as part of the policy process. Thunus and Schoenaers’ (2012) article is an exception given that they use a quasi-experimental design allowing the authors to examine the effects of different consultation techniques. Their detailed analysis of actors, their viewpoints and the evidence of interaction between the actors highlight different ‘logics of action for actors’, supporting the claim that consultations encompass actors with different aims and objectives.
In an earlier contribution, Hogwood (1986) formulated a conceptual framework that still informs much of the discussion on public consultation. He approached the topic from the policy emergence angle, questioning whether consultation is a useful term given the diverse motivations and aspirations of policy makers. Throughout his paper, Hogwood deconstructed consultation as a product of various agents in the consultation process and differentiated between cosmetic ritual, imposition of policy on stakeholders, practice guided policy making and genuine negotiation. The article lacked empirical grounding but his conceptual framework clearly influenced subsequent theoretical work.
Most studies written from a public administration perspective articulate their main concern as one of effectiveness. Van Damme and Brans’ (2012) article is one of only a few that utilises a sophisticated theoretical framework to explore effectiveness. Their analysis points to three criteria that define the interpretative framework: (1) conditions of openness of the consultation which determine access of participants, (2) the scope of the consultation determining the content of participants’ submissions and (3) a model to understand the interaction between respondents that may impact on the quality of the deliberation. The last point has particular relevance to consultations that allow for deliberative processes between participants and highlights the need to examine pre-submission interaction. It informs an understanding of consultations as an ongoing deliberative process rather than as a one-way operation in which external expertise and knowledge is obtained by policy makers.
Whilst van Damme and Brans’ interpretative framework embeds consultations in the wider context of citizen engagement and democratic steering of policy networks, their study finds little evidence that consultations are actually enhancing the participatory ability of citizens to influence decision making in policy formulation. In other words, their framework articulates a normative account of public consultation within a democratic polity that contrasts sharply with actual practice. Their analysis focuses on the correlation between the process and management, ultimately offering the following interpretative model:
Shipley and Utz (2012) explore a similar vein of ideas in their study on citizen motivation and citizen engagement. Their article contextualises the empirical material within the debate on direct democracy and the control over public goods. Consultations are thus seen as a key component of the deliberative process that ascertains public interests in relation to certain questions over power or resource distributions. Their conclusion, that techniques may have a significant impact on whether a consultation is tokenistic or manipulative, is well argued and highlights the need for so-called deliberate polling (Fishkin, 2009; Fishkin et al., 2000) where consultations allow for interest and opinion formation as part of the consultative process.
Other commentators have highlighted the shortcomings of current interpretative models by pointing out the lack of research that investigates the link between consultation outcomes and policy formulation (Jones and Einsiedel, 2011). However, to consider consultation within the narrow administrative confines of functionalist analysis may fail to take into account the actual role consultation is thought to play in enhancing citizen involvement as a prerequisite for a healthy democracy, a concern that is articulated within the network governance literature as well (Jordan et al., 2005; Klijn and Skelcher, 2007; Marsh and Rhodes, 2002; Marsh and Smith, 2000; Sorensen and Torfing, 2007). Embedding consultations in the wider democratic process appears to develop some track with researchers. Murray in an analysis of consultations in Northern Ireland between 2000 and 2004 also noted a tension between general enthusiasm for public consultations and consultation fatigue felt by citizens (Murray et al., 2009). The authors are skeptical that increased numbers of consultations contribute automatically to improved access to policy making or improved horizontal governance which is ideally correlated with high levels of citizen engagement and participation in decision making. Their study highlights the need to explore further any possible link between various techniques of consultation, their effects on participation rates and policy-making outcomes.
Method
The study examined the nature and content of consultation submissions to the Social Services and Well-being (Wales) Bill (2013) consultation at Stage 1. The analysis of submissions was to produce empirical evidence located at the intersection of scope, technique (determining access and participation levels), and nature and extent of submissions (defined or open) and to provide an insight into the effectiveness of the consultation to citizen engagement in a newly devolved polity.
A literature review identified several key themes that are relevant to an analysis of public consultation submissions, and we defined three categories of classification: origin of response with respect to sector, organisation type and location; use of consultation template; and size and content of submission. With respect to the latter, we were interested if participants used more or less formulaic templates for submissions and/or shared them across participating organisations. Following on from these themes, we designed a coding matrix to be used for the documentary analysis (see Appendix).
The matrix allowed the raters to collect evidence on: Who participated in the consultation? What was the nature and content of the submissions? To operationalise these questions, a set of indicators were developed collecting evidence about the origin of submission (such as the type of organisation; Welsh or UK/England based), the size of individual submissions, the compliance/non-compliance of respondents with consultation brief.
The publicly available data of the Social Services and Well-being (Wales) Bill consultation (Stage 1) were then fed into the matrix. Descriptive statistics were conducted, and the content of a sub-sample of purposively selected submissions was analysed in more detail. The matrix contained an additional comment box to note anything that was not captured by the matrix but may have relevance for further investigation. Data entered were subjected to simple descriptive statistical analysis, including cross-tabulations, in SPSS. Since submissions including the consultation brief were published by the Welsh Assembly on its website, 1 and all relevant data were publicly available, the study required no ethical approval. Coding was done by two independent raters, and results were compared. Where discrepancies in rating occurred, the raters discussed the issue in detail on the basis of the submitted evidence to reach a consensus.
A documentary approach was utilised for several reasons. First, the study explicitly aimed to explore the nature and content of the submissions as opposed to obtain the views and opinions of the organisers or participants of the public consultation. It is striking that despite the enormous amount of documentary evidence in the public domain in the UK on public consultations, the documentary approach is still under-utilised. In a sense, this pilot thus acts as a proof of concept study which should encourage others to investigate the potential contribution of analysing documentary evidence to achieving a balance interpretative approach in public consultation studies.
Results and discussion
The analysis produced descriptive statistics on the characteristics of the participants, the content and extent of their submissions and information relating to pre-consultation activities. The publicly available consultation document contained 84 submissions from individuals and organisations (n = 84). About a fifth of all submissions (n = 16; 19%) originated with non-Welsh organisations. That is a substantial number of submissions, yet cross-referencing their content revealed that most of those from non-Welsh organisations related to the single issue campaign of smacking (or parental chastisement), which had no relevance to the Social Services Bill (Wales).
The majority of submissions were between one and seven pages long, and it was notable that longer structured submissions came exclusively from large public sector organisations (Figure 1). This may reflect their additional capacity and staff resources to engage in policy work and policy discussions and may create differential impact between consultation participants. Similar imbalances have previously been noticed in network governance analyses (Davies, 2005, 2007; Kaehne, 2010, 2014a). Of the 16 submissions on smacking or parental chastisement (19%), one originated with a commercial organisation, two came from professional associations and 13 from voluntary organisations.
Number of submissions by size (number of pages).
Table 1 lists the number submissions by sector. It shows a clear predominance of the voluntary sector in the number of submissions (n = 51; 61%) and a notable absence of commercial and health-care providers. Local government organisations made several submissions of substantial size, and some of them contained identical phrasing, which indicated pre-consultation interaction.
Self-advocacy was defined as submissions containing a substantive and clearly demarcated section dedicated to the work of the organisation. The analysis indicated that about a third of all submissions (n = 28; 33%) contained self-advocacy statements. These often went hand in hand with requests to be invited to give further evidence. There is some question as to whether self-advocacy in the consultation context contributes anything to the submission itself. The fact that some organisations, however, chose to add these sections may reflect their concern that recipients are not familiar with their work. Alternatively, it may simply be an administrative technique to boost the size of the submission. Most of these self-advocacy sections seemed to be lifted from mission statements. Self-advocacy was most often used by voluntary organisations (Figure 2).
Number of self-advocacy statements in submission documents by type of organisation.
Compliance with the set response framework was measured by whether respondents structured their submission documents according to the series of questions asked in the consultation brief. Responding to some but not all questions of the consultation brief was counted as full compliance where individual sections were enumerated in accordance with the consultation brief. Analysis of the data shows that marginally more than half of all submissions (n = 45; 54%) utilised the response framework, while the other half did not (n = 39; 46%). Analysing the utilisation of the response template revealed some fascinating differences by sector. Compliance with response formats may ease analysis of submissions and, consequently, failure to use it may mitigate against effective and robust interpretation of submission content. It is therefore surprising that most voluntary organisations chose not to use it. Their submissions were often lengthy texts without clear structure. This may be a result of insufficient resources, or inadequate familiarity with consultation techniques and the ‘rules of the game’. It is instructive that all submissions originating with local authorities and National Health Service with large research or policy-making capacity utilised the response template (Figure 3).
Utilisation of response template by type of organisation.
Submissions by sector category
Public consultation outcomes
Source: van Damme JV and Brans M (2012: 1050).
Van Damme and Brans identify two components of the content/objective domain: policy enrichment and policy impact. Social learning and conflict reduction are part of the process domain. The results of this study are mainly located in the policy enrichment and policy impact category, which theoretically includes the potential impact of public consultation participants on resultant policy or legislation.
Social learning processes, however, could still be detected through a close analysis of submissions, and the discussion will indicate below where this is the case. As such, a detailed analysis of submissions can yield some information in several domains, even though most evidence was arguably located in the contents/objective domain.
The following section will differentiate between three aspects that allow the findings to be clustered within this domain. Policy enrichment, as van Damme and Brans articulated it, is operationalised here as a function of nature of respondent, as well as scope and motivation of submission. In other words, the findings distinguished between the who, the how and the why of participation in this public consultation as evidenced by submission data. As Arnstein (1969) noted in her seminal article on public consultation, one of the key questions about the effectiveness of public consultations is who responds. Arnstein framed this question as an issue about the intentions, motivations and calculations about efficacy of participation. In other words, potential participants ask themselves whether, if they would respond to a public consultation, they would exert influence on an issue, whether the issue is of relevance to them and whether they would like to exercise some influence in the first place. It’s the questions of who, why and how.
Who participated
The coding matrix captured information in submissions with respect to origins (Welsh/English) and sector. An analysis of the origin of submission clearly showed a predominance of Welsh organisations responding to the consultation call. Interestingly, however, there was a substantial number of submissions from English organisations as well. This may reflect the increased importance of the Welsh Assembly and its recently acquired legislative ability. It may also demonstrate that Welsh policy making and influencing Welsh policy is seen as an important part of policy work by organisations outside Wales. This aspect links with the wider debate about multi-level governance in Wales (Entwistle et al., 2014) as well as the question as to who populates the policy space, and to what extent, in devolved nations. The analysis of submissions to public consultations may provide some useful empirical material for a subsequent mapping of the policy arena.
The analysis of submissions in terms of sector revealed some considerable asymmetry of participation with respect to policy actors. Examining the authorship of submissions by sector gives insights into depth and scope of the policy debate in Wales. There was a near complete absence of commercial and individual contributors to the consultation. A cross-comparison with other legislative consultations may reveal this to be normal for these types of consultation, but the predominance of voluntary organisations and charities is notable insofar as Wales operates a mixed economy in care services. The lack of responses from the commercial sector may indicate either that commercial organisations may not have the policy capacity to participate in consultations, or that they genuinely felt that the Social Services Bill was of little concern to their core business strategy. As serious concerns about the brokering role of local authorities and the effect on small care providers in the mixed economy of care services emerge in public debate, this lack of involvement of the main provider category or their professional associations in the consultation is worrying. Alternatively, a lack of engagement may indicate the increasingly agnostic attitude by small commercial care providers in the face of anti-business political rhetoric. Questioning whether private care providers should have a role in care provision in Wales has been a constant feature of the Welsh Government’s political rationality (Guarneros-Meza et al., 2014; Kaehne, 2014b).
Examining the length and structure of submissions, there was a weak association between the length of submissions and the type of organisation. Third sector organisations were more likely to submit smaller and un-structured responses, whilst statutory organisations would almost exclusively utilise the response template and submit wordy responses. This may have significant implications for differential impact of responses. As staff need to transpose responses through a detailed analysis within a tight schedule, lengthy un-structured responses may resist easy translation into politically useful messages.
As previously mentioned, health-care organisations were also noted by their relative absence. This may reflect the fact that the Welsh Government has so far failed to formulate a coherent strategy to integrate health and social care and deliberately presented distinct bills in health and social services, respectively. The Welsh Government’s approach to differentiate legislation in health and social services may thus simply reflect the continuing fragmentation of care services, which is a longstanding concern in both England and Wales (Darzi and Howitt, 2012; Hudson and Henwood, 2002; Rummery and Coleman, 2003; Snape, 2003; Snape and Taylor, 2003, WAG, 2006, Welsh Local Government Association, 2012).
How they responded
The main aspect of the content and structure of responses concerned compliance with the response format that was formulated by the Welsh Government. There is substantial discussion in the literature about the merits and disadvantages of a structured public consultation, mainly discussed under the auspices of consultation technique in public administration (Shipley and Utz, 2012). Previous research demonstrates that the choice of technique strongly influences the range of potential respondents, as well as the extent to which they may or may not exert influence over legislation, a main measure of their impact on policy making. It also appears to have an effect on the satisfaction of participants and commissioners of consultations.
In this study, there was a high level of non-compliance with the consultation response format. Almost half of all submissions (46%) did not utilise the consultation template. This may mitigate against their potential impact in the consultation process. Those submissions that failed to use the response template were usually presented as a free flowing text which appeared difficult to summarise. These submissions would require significant efforts from staff to operationalise them for consultation summary.
As mentioned, it was mainly charities and voluntary organisations that utilised non-compliant response formats. Arguably, this may have decreased their impact on policy making. The low number of template compliant responses also raises the question as to whether the selected technique was in fact suitable for this type of consultation. This is a question that can only be answered in view of the allocated resources to summarise the consultation responses. It may be that the consultation in this instance was simply too tightly framed and structured. Tightly structuring consultation formats may have the advantage to allow quick analysis of responses yet also run the danger of lowering the potential usefulness of those responses that do not comply with the given format. Where the majority of responses would not utilise the response template, an inordinate amount of steering may lead to a narrowing of the range of useful responses in the first place, a strategy that raises the issue of tokenism in public consultations again.
There was some evidence that some actors were consulting each other prior to their submissions. This took two forms. First, some participants would submit identically phrased responses. This may indicate pre-submission interaction between participants and co-ordination of responses. In some cases, however, submissions were not co-ordinated as such but participants used a pre-formulated template for submissions without evidence of prior direct interaction. The largest contingent of these responses concerned the issue of smacking. Almost a fifth of all submissions focused on smacking (lobbying for a total ban of smacking) which was clearly outside the remit of the consultation. This indicates that there was a co-ordinated campaign to submit responses to this consultation although, ultimately, those organising the campaign ostensibly acted on the basis of a misunderstood the consultation remit.
Why they participated
An analysis of the content of submissions reveals a surprising amount about the motivation of participants in the consultation. First, there is the large number of organisations (including some individuals) whose submissions lobbied for the ban of smacking. Given the difficulty of galvanising support for campaigns, these submissions may indicate higher than usual levels of motivation on this issue. Unfortunately, given that smacking (and parental chastisement legislation) is currently not a prerogative for the Welsh Assembly, the campaign was based on a misunderstanding of current legislative competence of the Assembly. Consequently, the Welsh Government dismissed calls to regulate or ban smacking.
There was also a significant amount of self-advocacy content in the submissions by third sector organisations, which may reflect concerns about insufficient access to the policy-making process. Where wording in the submissions hinted at evidence of prior attendance of committees and the like, it occurred exclusively in submissions from statutory organisations. The absence of similar evidence in the submissions by voluntary sector organisations may indicate a lack of routine access by them to Welsh Assembly committee work and/or the Welsh Government. Again, this would have implications about the ability of both devolved institutions to maximise external expertise in preparing legislation. Enhancing the policy network capacity and use of external expertise would depend on drawing a sufficient number of participants from a broad range of professional backgrounds into the policy-making process. Admittedly, consultations are only one part of a wider policy and legislative process, and the effectiveness of policy making and the nature, scope and range of policy networks can only be adequately measured across all components of this process. The absence of a whole segment of service providers in public consultation responses and any mention of committee attendance by them, however, may indicate either problems of access in the wider context of policy formulation or withdrawal of some actors from the process altogether.
Besides the issue of who, why and how respondents participated, the study also raised questions about public consultation as a mechanism of participatory democracy. This relates to debates about network governance and citizen engagement vis-à-vis policy making (Callaghan and Wistow, 2006; Culver and Howe, 2004; Guarneros-Meza et al., 2014; Lawton and Macaulay, 2014; Redley, 2009).
Network governance and citizen engagement
The evidence showed that the public consultation for the Social Services Bill (Wales) achieved good participation from organisations in the voluntary sector. There are, however, some doubts as to whether these organisations used the consultation in the most effective way to make their voices heard. Much of this has to do with the way in which the consultation technique was designed to gear responses to specific issues, whilst many third sector organisations were interested in raising issues outside the remit of the consultation. The response template unduly favoured structured responses, and future research should investigate whether or not non-compliance with the response template diminished the impact these responses had on the final legislation or the policy discussions within the legislative chamber.
There was also clear evidence about a lack of engagement with the consultation by a range of stakeholders in the social services field. In particular, commercial organisations and individuals failed to respond in significant numbers. This may indicate the predominance of organised interests, echoing the criticism about the organised character of modern participatory democracy (Klijn and Skelcher, 2007), and has implications for the role of individual stakeholders in policy-making processes. It may be due to differential access to resources by organisations (specifically public sector organisations) to policy response capacity. It may, however, also reflect a chronic level of deliberate disengagement by some stakeholders from the policy-making process. At this stage, it can only be a matter of speculation as to why this may be the case, although there is some anecdotal evidence in the public domain about the negative effects of the more ideological announcements of prominent members of the Welsh Government about the desire to reduce or eliminate altogether private enterprise from care provision in Wales (Downe et al., 2011; Guarneros-Meza et al., 2014).
The predominant anti-private enterprise rhetoric of the Welsh Government in the public domain in this respect may generate disengagement with public consultation exercises by some stakeholders. Whilst technically, social service provision is operating in a mixed economy, and individual budgets and direct payments are taking root (Roulstone et al., 2013; Stainton and Boyce, 2004), individual service users or commercial providers of care appear to be significantly under-represented in the policy-making process which the present study investigated. The Welsh Government has long discouraged some of the more flexible individualised care provision through commercial organisations and personal allocation of social care budgets and the absence of relevant stakeholders in the policy-making process may be a legacy of this strategy. Future research should explore if the Welsh Government’s engagement strategy through public consultations is adversely influenced by this wider climate.
Public administration and consultation techniques
The study showed that the chosen technique of this public consultation may have depressed the number and quality of responses at this early stage in the legislative process. There was a significant number of submissions that were unstructured and did not use the response template. The issue of selecting the most appropriate technique has implications for participation rates but also for motivational issues and harnessing external expertise within devolved governments with relatively small capacity. In the public administration literature, this aspect is discussed as one of ‘tokenism’ but the phrasing suggests more complex issues around governance of policy-making communities and the effective involvement of stakeholders in the process. As the Welsh Government repeatedly makes strong claims to govern through and for the Welsh public, stakeholder inclusion through participation in the legislative process has particular relevance and becomes a key benchmark to assess the validity of the Welsh Government’s claims with regard to inclusive governance and accountability.
The study also raises questions about the nature and scope of the Welsh policy context in general. Future research may usefully examine whether Welsh devolution has instigated the formation of new policy networks and policy-making arenas around the Welsh Assembly, and whether or not these new arenas have solidified in the wake of the 2011 referendum. The impact of the referendum on policy networks may be particularly significant in the Welsh context, given that Wales has a relatively small community of policy makers and politicians and hence may lack serious network capacity that ensures sufficient preparation and scrutiny of legislation proposed.
On the other hand, a narrowing of participation for Welsh legislation consultation along national lines may mean that consultations forfeit essential expertise and knowledge that is located in the other home nations. Alternatively, it may make questions about the quality and effectiveness of contributions to consultations more relevant. These are issues that could be explored through a detailed policy network analysis (Blanco et al., 2011; Montpetit, 2003). Parallel to such an approach, an analysis of submissions to Welsh Assembly consultations may provide further the empirical evidence that helps define the predominant vectors of policy influence and policy formation in the newly devolved polity.
There are some important implications of our case study for future research. Public consultations are a useful mechanism to improve legislation prior to parliamentary debate through canvassing stakeholders, and our study showed that this particular consultation provoked a range of responses from a wide variety of stakeholders. There is however, some serious doubt as to whether the chosen technique matched the aims and objective of the consultation. Echoing Arnstein’s argument about public consultations as falling short of genuine means of participatory democracy (Arnstein, 1969), this raises questions about tokenism, the role of external expertise and the ability of devolved governments to use consultations as a core mechanism for citizen and stakeholder engagement.
The results demonstrate that consultation responses came from a wide range of actors, that there was some limited amount of interaction prior to submission between respondents, and a significant amount of ‘white noise’; submissions that are not related to the topic. The analysis also revealed that a significant number of participants used the consultation as an opportunity to engage in self-advocacy, presumably hoping to gain further access to policy making. Organisational analysis of participants further revealed some imbalance of submissions by sector, manifested by a significant absence of commercial organisations amongst respondents.
Our analysis of submissions produced information on the range of organisations participating, the type, extent and content of their submissions, as well as the probability of them effectively influencing the legislative process. Employing the coding matrix to a large number of cases should therefore yield patterns of consultation engagement which have not yet been systematically investigated. Our study also produced valuable evidence how consultation techniques influence potential impact by stakeholders. An organisational snapshot of submissions by sector threw some light on the extent to which the selected consultation mechanism was utilised by a variety of organisations. This may have implications for the scope of democratic participation and citizen involvement, a theme that still awaits detailed exploration in the context of public consultation. This is particularly important to further evidence the Welsh Government’s claim of participatory and inclusive politics.
Since the present study examined only one case of consultation in a devolved policy context, it also raises the question as to the generalisability of our findings. As political scientists noted, ‘a sample of one is worse than a sample of none because it provides the false sense of understanding’ (Peters, 2014: 289). Our main aim in this study, however, was not to evidence particular associations between consultation techniques and participation yet to demonstrate that an analysis of consultation submissions, using the developed coding matrix, would produce data positively informing the debate on the relationship between consultation techniques, participation and policy making. We believe to have succeeded in highlighting the methodological potential of approaching the complex interaction between these themes by employing this novel data identification and analysis method.
Footnotes
Note
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.