Mini-publics and Public Opinion: Two Survey-Based Experiments

Abstract

In intense forms of public consultations, select groups of citizens, called mini-publics, are given a large amount of information and then asked to deliberate on policy directions and make recommendations. Government officials may refuse to act upon these recommendations, unless they are convinced that the recommendations have wider support in the populace. This article presents the results of two survey-based experiments that assess the impact of mini-publics on the opinions expressed by random digit dialing samples of the general public. The survey-based experiments were conducted in 2013 (n = 400) and in 2014 (n = 400). Being informed about the mini-publics affected support for some policies, but not others. In both studies, respondents who were informed about the mini-publics reported higher levels of political efficacy compared to the condition where respondents were not informed about the mini-public. Hearing about these mini-publics helps to generate a sense of legitimacy in the political system.

Keywords

deliberation experiment public opinion political trust political efficacy

Governments are facing a crisis of legitimacy as public confidence declines (Fournier et al., 2011; Gastil, 2000). To address these trends, government officials host public consultation initiatives around diverse topics. These initiatives demonstrate respect for the democratic process and may increase the perceived legitimacy of the democratic system and its outcomes. In some cases, these public consultations are used to guide public policy. The theory is that these policies will have greater legitimacy if they are derived from citizens and/or are supported by citizen groups (see discussion in Nabatchi, 2012). Beyond the legitimation of policy decisions, these citizen engagement initiatives are expected to have broad pay-offs in terms of generating confidence in the political system (Gastil, 2000). However, little research has been done on how awareness of these initiatives affects the broader public’s policy views and political attitudes.

This article assesses whether a specific type of public consultation, a mini-public, has any effects on the opinions expressed by a random digit dialing (RDD) sample of the general public. The research draws upon the large body of research on information cues and endorsement effects on voting and policy preferences (see reviews in Dewan et al., 2014 and Mutz, 2011). Like other studies of information cues (see Mutz, 2011), participants are randomly assigned to hear or not hear a paragraph describing a mini-public. The two groups are compared in terms of level of support for proposed policies, whether they render an opinion about proposed policies, the degree to which they report trusting government decisions in the policy domain and the extent to which they feel they can influence government decisions. The study is unique in presenting truthful descriptions of actual mini-publics, instead of fictitious vignettes, reinforcing external validity (Mutz, 2011; Sniderman and Grob, 1996). Unlike studies of endorsement effects, the endorsers are not political elites, but rather a select group of citizens who have deliberated on the proposed policy initiatives. This project goes beyond endorsement effects on policy views. This project examines the effects of hearing about mini-publics on citizens’ sense of political efficacy and trust.

This project uses two survey-based experiments to examine the impact of mini-publics on policy views: political trust and political efficacy. The replication of findings is important in the current academic climate (Baker, 2015; Van Noorden, 2015). Some argue that ‘scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence’ (Open Science Collaboration, 2015: 1). The first study demonstrated that being informed about a mini-public had an impact on some policy preferences, the likelihood of expressing a policy view (versus stating ‘don’t know’) as well as political trust and efficacy. The study was repeated with some modifications to assess the robustness of the findings. The two experiments varied the information about the mini-public, which produced differences in findings about political trust, support for proposed policies and the likelihood of expressing a policy view. However, the two experiments acquiesce on the finding that being informed about a mini-public increases respondents’ sense of political efficacy. The replication of this finding demonstrates the reliability of the findings, which is a condition of validity, as well as demonstrates the robustness of the finding across different conditions, which also helps establish validity.

Literature Review

Effects of Mini-publics on Public Opinion

Mini-publics are select groups of citizens who are given a large amount of information and then asked to deliberate on policy directions and make recommendations. Much of the research on mini-publics focuses on how the participating citizens are transformed by the experience (Fishkin and Luskin, 1999; Gastil et al., 2010; Grönlund et al., 2010; Morrell, 2005; Strandberg and Grönlund, 2012). This article does not address the impact of the deliberative process on participants. Instead, the article examines how mini-publics influence the broader public. Gastil et al. (2012) suggest that the pay-off of mini-publics could extend beyond the participating citizens. They write:

in the long term, deliberative civic engagement efforts could transform not only their participants but also the larger public. Those participating in, engaged with, or captivated by such efforts should report stable (or rising) levels of public trust and signs of reduced civic neglect (Gastil et al., 2012: 214–215).

In short, mini-publics may be a ‘remedy’ for political distrust (Bächtiger et al., 2014). However, there is little research examining the effects of mini-publics on public perceptions of political efficacy and trust.

The theory is that when citizens hear about other citizens engaged in public policy decisions, they will internalise this experience and believe that they, themselves, could influence public policy. In other words, those who are ‘captivated’ by such efforts may feel a genuine increase in their own ability to affect the direction of government. In terms of trust, the theory is that hearing about the mini-public could increase the perceived legitimacy of the political system. The two ideas are related in that feeling as though the government is listening to average citizens, that is, through mini-publics, may improve perceptions of the governments’ trustworthiness. However, there is little research to substantiate this theory.

Another way that mini-publics might influence public opinion is through policy preferences. Knowledge of mini-publics and their policy recommendations might serve as a heuristic or information shortcut that citizens use to determine their opinion on policy matters (Fournier et al., 2011; Warren, 2009). Many researchers have documented citizens’ low levels of knowledge about different policy initiatives (see discussion in Bullock, 2011 and Delli Carpini and Keeter, 1996), which leads citizens to express non-opinions, that is, say they don’t know their policy preferences, or to hold policy preferences that contradict their own interests (Kuklinski et al., 2000). The information cues provided as part of a survey interview or a political campaign could help respondents form opinions on different policy initiatives, reducing the reporting of ‘don’t know’ (Dewan et al., 2014). However, there is little research to confirm this expectation in relation to mini-publics (Dewan et al., 2014).

A related stream of research discusses how impersonal influence, that is, the opinions of an unknown collection of others, shapes people’s policy and voting preferences (Mutz, 1992). The research offers mixed support for impersonal influence as it relates to expertise-related cues presented through endorsements by public officials, law enforcement and scientists (Mutz, 1992). This article is distinctive in assessing whether a deliberating body of citizens has influence on an individual’s policy preferences. Information about these unknown others and their level of support for policies might provide a cue for citizens in trying to determine their own policy preferences (Mutz, 1992).

Moreover, the information cues provided during a survey interview could influence the level of support for a policy initiative (Sniderman and Grob, 1996). Specifically, ‘knowledge that a group of randomly selected ordinary citizens came up with the proposal after extensive work independently of government and political parties could be enough to support the recommendation’ (Fournier et al., 2011: 131–132). Alternatively, being presented with information about a mini-public’s lack of support for a proposed policy could decrease support for the policy, as observed in the Oregon Citizens’ Review Initiative 2010 (Gastil et al., 2014). When forming an opinion about a policy, citizens may use information about the origins of the policy recommendation when judging whether to support or reject a policy recommendation. The theory reflects how voting decisions are made (Gastil, 2000). Citizens often use information shortcuts, such as party affiliations and friends’ recommendations, when deciding how to cast their ballot (Gastil, 2000). The information provided by the mini-public may encourage the public to engage in their own vicarious deliberation (Gastil et al., 2014) or ‘deliberation within’ (Goodin and Niemeyer, 2003). This internal deliberation could produce an opinion change as people think about reasons for their friends’ or others’ recommendations (Mutz, 1992).

Citizens may ‘defer to more enlightened peers’ recognising the greater knowledge possessed by participants in a mini-public (Fournier et al., 2011: 127). In the study of third-party persuasion, Lupia and McCubbins (1998) argue that the ability to persuade depends, in part, on whether the third party has specialised knowledge that is relevant to the persuasion attempt (also see Mutz, 1992). In this case, the mini-public may be perceived as having specialised knowledge because of their participation in the deliberative event (Warren, 2009). Indeed, perhaps the degree of specialised knowledge could be assessed by the length of participation in the mini-public, with mini-publics extending days or months having greater knowledge than those meeting for several hours (Warren, 2009).

The theory draws on the large body of research on endorsement effects (see literature review in Dewan et al., 2014). The research on endorsement effects focuses on celebrities, members of the political elite, interest groups or political parties (Arceneaux and Kolodny, 2009; Bullock, 2011; Dewan et al., 2014; Druckman et al., 2010; Smets and Isernia, 2014). Some studies find significant effects of endorsement (vs no endorsement) on policy preferences, but no effect on the reporting of non-opinions (Dewan et al., 2014). Druckman et al. (2010) find that endorsement cues have effects on hard issues that are not salient and are relatively technical (for a counter-argument, see Bullock, 2011). Bullock (2011) finds effects of political party endorsement on policy preferences, but the findings are not clear on whether political party endorsement or policy information matters more to respondents’ opinions. Other studies suggest that endorsement effects interact with political awareness or ideology in shaping voting and policy preferences (Arceneaux and Kolodny, 2009; Bullock, 2011; Dewan et al., 2014; Gastil, 2000; Mutz, 1992; Smets and Isernia, 2014; Sniderman and Grob, 1996). Zaller (1992) attributes the success of endorsement effects to a citizenry with relatively undeveloped and unsophisticated political views. None of these studies have examined endorsement by a randomly selected group of citizens who engaged in a deliberative exercise, for example, a mini-public.

Effects of Mini-Publics on Governments

The role of mini-publics in shaping policy preferences is critically important because they could determine whether or not a proposed policy is adopted by government. For example, in the Oregon Citizens’ Review Initiative (CRI), the deliberating body produced an information pamphlet to help guide voters in casting their ballot in favour or against different policies (Gastil and Knobloch, 2010). The CRI failed to have broad impacts on the ballot because of low awareness (Gastil and Knobloch, 2010). In the Citizen Assemblies on Electoral Reform in Canada, policymakers required a minimum of 60% support in provincial referenda before they would commit to the recommendations (Cutler et al., 2008; Fournier et al., 2011). The proposed changes failed to reach this threshold. As observed in the Oregon CRI, the failures were attributed to a lack of public awareness and knowledge (Fournier et al., 2011).

In both cases, the supporting data are limited for assessing the causal relationship between knowledge of the mini-publics and policy preferences. In the Citizen Assemblies cases, cross-sectional surveys were used to assess awareness of the mini-public and policy preferences. These types of surveys are weak for assessing causality. These types of surveys cannot untangle the temporal order of variables, which is a critical component of proving causality. Furthermore, cross-sectional surveys are weak for assessing the role of other variables in influencing the relationship between the independent and dependent variables (Mutz, 2011). Does political interest, education or political ideology explain both variables?

Survey-Based Experiments

This study attempts to decipher the impact of being informed about mini-publics on citizens’ policy preferences. In other words, to what degree do citizens rely on mini-publics to inform their policy preferences? Policy preferences are examined in terms of support for various policy solutions as well as in terms of rendering an opinion about the policy under study, as opposed to reporting ‘don’t know’. In addition, this study will explore the effects of mini-publics on public perceptions that the government is trustworthy, responsive and is listening to the concerns of citizens. This article will present the results of two survey-based experiments that assess the impact of mini-publics on the public opinion expressed by RDD samples of the general public. A survey-based experiment was used for several reasons. First, a survey-based experiment can help assess the temporal order of two related variables, awareness of mini-publics and policy preferences (Mutz, 2011). Because information is the subject of the manipulation, I can examine information as precursor to opinions about different policy options. Second, a survey-based experiment can control what people know about the mini-publics, addressing problems of misinformation. Third, a survey-based experiment is better able to address confounding variables, which may create spurious effects between the independent and dependent variables (Mutz, 2011).

Experiments are widely criticised for being artificial. Most survey-based experiments employ fictitious vignettes to examine attitude differences (Mutz, 2011; Sniderman and Grob, 1996). Both mini-publics were indeed formed and did engage in the activities described as part of the experiment. As such, the manipulation engages a ‘real-life’ situation which addresses criticisms that experiments tend to use overly strong and unrealistic (as well as fictitious) experimental manipulations (Barabas and Jerit, 2010). For these reasons, this project has a stronger claim to validity compared to other types of survey-based experiments. In addition, the replication of results provides assurances of reliability, which is a precondition of validity.

The hypotheses are as follows:

Hypothesis 1: Respondents who are informed about the mini-publics are more likely to support policy recommendations compared to respondents who are not informed about the mini-publics.

Hypothesis 2: Respondents who are informed about the mini-publics are more likely to report an opinion on the policy recommendations compared to respondents who are not informed about the mini-publics.

Hypothesis 3: Respondents who are informed about the mini-publics report higher levels of trust in government decisions, on average, compared to respondents who are not informed about the mini-publics.

Hypothesis 4: Respondents who are informed about the mini-publics report higher levels of political efficacy, on average, compared to respondents who are not informed about the mini-publics.

Methods

Participants

The experiments were embedded in RDD surveys conducted by the Population Research Lab at the University of Alberta. The results focus on Edmonton residents. Edmonton is the provincial capital. For Study 1, data collection started on 18 June 2013 and ended on 23 July 2013. American Association for Public Opinion Research (AAPOR) Response Rate 1 is 10.26%. For Study 2, data collection started on 24 July 2014 and ended on 8 September 2014. AAPOR Response Rate 1 is 9.84%. The 2014 data set is available for download from the University of Alberta website (https://dataverse.library.ualberta.ca/dvn/). The 2013 data set contained an oversample of City of Edmonton residents. The oversample was not included in the publicly released data set, but is available by contacting the researcher.

Design and Manipulation

A two-group post-test design was used, as is typical (Mutz, 2011). Participants were randomly assigned to one of two conditions: being informed about the mini-publics versus being uninformed about the mini-publics. In the uninformed conditions, respondents were simply asked about the extent to which they agree or disagree with the following policies. In the informed condition, prior to asking about the policies, half of respondents were informed that these policy options were recommended by a randomly selected group of citizens who met to discuss different policy options (a mini-public). Both scenarios are truthful representations of actual mini-publics. In both cases, these mini-publics received no media coverage during their work and little coverage after their work concluded. As such, the public would not have prior awareness or knowledge of these mini-publics. The use of actual mini-publics distinguishes this study from typical vignette-based research (Mutz, 2011; Sniderman and Grob, 1996). The exact wording is as follows:

Study 1: In fall of 2012, the City of Edmonton randomly selected 57 Edmontonians to form the Citizens’ Panel on Edmonton’s Energy and Climate Challenges. This very diverse group got balanced information about these challenges and met for six Saturdays to talk about what the City should do. The following is a list of their recommendations. I would like to know your level of support for their recommendations. (INTERVIEWER: Please emphasise ‘their recommendations’ in reading the above sentences.)

Study 2: In fall of 2013, Albertans were randomly selected to participate in a two hour discussion about energy efficiency in Alberta. The project was led by the Alberta Energy Efficiency Alliance, along with researchers at the University of Alberta and Athabasca University. A diverse group of 165 Albertans participated in the online discussion. The participants were supportive of energy efficiency standards for building materials and the operation of industrial facilities. What about you?

The manipulations differ on a number of characteristics. First, the level of government and its role differ in both manipulations. In Study 1, the municipal government is targeted and is identified as the sponsor of the initiative. In Study 2, the provincial government is targeted, but the sponsors are a community group and academics. Second, the size of the mini-public and the length of its activities differ in both manipulations. In Study 1, the mini-public is smaller (57 participants) compared to the mini-public in Study 2 (165 participants). The mini-publics differed in how long they spent learning about and debating different policy initiatives (six Saturdays vs 2 hours). More subtle differences in the two studies are the references to ‘balanced information’ and the description of the ‘consensus’ in policy recommendations.

While some would argue that the differences in the manipulations affect the comparability of results, the value of these differences is that both descriptions were truthful descriptions of mini-publics. In other words, the manipulations were not artificial. Furthermore, the differences in manipulations permit an analysis of the robustness of findings and allow for an analysis of what factors may influence the effect of mini-publics on public opinion (level of government, involvement of government, size of mini-public and length of time that the mini-public met).

Measures

For the policy questions, respondents were asked to state their agreement or disagreement with different policy options. In Study 1, respondents were offered the following response options: strongly disagree, disagree, agree, strongly agree or don’t know. In Study 2, respondents were offered the following response options: strongly disagree, somewhat disagree, slightly disagree, slightly agree, somewhat agree, strongly agree or don’t know. However, given the highly skewed nature of the distribution (very few people expressed any level of disagreement with the policy options), the responses were dichotomised into agreement (1) or disagreement (0) with the proposed policy option. The recoding also allows for comparisons between the two studies, which used a different number of options in the response scale.

Studies show that political trust depends on the level of government referenced as well as different policy domains (Hetherington and Husser, 2012; Pickup et al., 2004). As such, to assess trust, respondents were asked about trust in the government’s decision-making around the policy domain being studied. In Study 1, the policy domain is climate change, and in Study 2, it is energy efficiency. For Study 1, participants were asked, ‘How much do you trust the municipal government to make good decisions about climate change?’ ‘Would you say: not at all, a little, some or a lot?’ For Study 2, participants were asked, ‘How much do you trust the provincial government to make good decisions about energy efficiency?’ ‘Would you say: not at all, a little, some, a lot or a great deal?’ The response options differ slightly in Study 1 and Study 2. Study 2 provides a fifth category – ‘a great deal’ – which affects the direct comparison of coefficients.

The first efficacy measure is a direct replication of a question used in the American National Election Study 2008. Respondents were asked in both surveys: How much can people like you affect what the government does? Would you say: not at all, a little, a moderate amount, a lot or a great deal? The second efficacy measure was modified from its original wording in the American National Election Study 2008. Instead of asking ‘How much do public officials care what people like you think?’, the question asked what portion of politicians care about what people like you think. The change was meant to reflect that there is no typical ‘politician’. Instead, the question asks about the collection of politicians and whether most care or few care about what citizens think. In both surveys, respondents were asked the following:

For the following question, please answer on a 1–5 scale, where 1 means ‘hardly any do’, 3 means ‘some do’ and 5 means ‘most care’. In general, do you think that politicians care what people like you think? (PROMPT IF NECESSARY: 1 means ‘hardly any do’, 3 means ‘some do’ and 5 means ‘most care’).

Analysis

The data are examined as a series of t-test of group differences. Table 1 verifies that the random assignment process produced two comparable groups in terms of demographic variables that may affect policy views and political attitudes. The data are analysed for each individual survey question and then as a composite measure, for example, combining two measures of political efficacy. The focus on single survey questions helps to understand the nuances in the effects of the manipulations on particular policy domains. The addition of the composite measure helps address measurement error (Ansolabehere et al., 2008). The composite measures are then used in multivariate analysis to replicate the manipulation’s effects after controlling for key demographic variables that may impact policy preferences and political attitudes. The bivariate and multivariate findings produce identical patterns of findings.

Table 1.

Random Assignment Check.

	Not informed	Informed	t-test	p-value
Study 1
Age in years	51.94 (SD = 17.54)	50.98 (SD = 17.09)	0.55	0.585
Education in years	15.76 (SD = 3.44)	15.51 (SD = 3.25)	0.75	0.455
Portion of females	50.47%	49.74%	0.15	0.883
Income group	25.37 (CAN$70,000–CAN$74,999)	26.46 (CAN$75,000–CAN$79,999)	1.19	0.235
Portion of home owners	77.62%	77.66%	0.01	0.992
Study 2
Age in years	54.29 (SD = 15.06)	54.88 (SD = 15.39)	0.38	0.703
Education in years	15.53 (SD = 3.89)	15.52 (SD = 3.45)	0.03	0.974
Portion of females	48.97%	50.73%	0.35	0.726
Income group	26.13 (CAN$75,000–CAN$79,999)	26.52 (CAN$80,000–CAN$ 84,999)	0.44	0.658
Portion of home owners	78.87%	84.24%	1.38	0.168

SD: standard deviation.

Approximately 200 respondents were assigned to each group. Equal variance assumed. Data are unweighted.

Results

Support for the different policy options was extremely high among both groups of respondents. In the first study, being informed about the mini-public’s support for the policies affected opinions about the policies (Table 2). Those respondents who were informed about the mini-public were more likely to agree with the policies compared to those who were not informed about the mini-public. The difference was statistically significant for three of the six policy options in Study 1. Approximately 98.35% of respondents who were informed about the mini-public agreed with policies to promote energy-efficient buildings compared to 93.65% of respondents who were not informed about the mini-public (t = 2.31, p = 0.022). Support for this policy option was extremely high among both groups.

Table 2.

Knowledge of the Mini-publics and Policy Preferences.

	Not informed	Informed	t-test	p-value
Study 1: Agreement with policies that…
(1a) Promote energy-efficient buildings	93.65%	98.35%	2.31	0.022
(1b) Promote renewable energy	89.64%	95.48%	2.13	0.034
(1c) Promote energy-efficient travel options, such as public transit, cycling and walking	92.20%	92.51%	0.12	0.906
(1d) Promote greater urban density and less expansion	71.66%	72.39%	0.15	0.879
(1e) Green the provincial electricity grid	82.11%	89.20%	1.93	0.054
(1f) Reduce greenhouse gas emissions	87.50%	89.19%	0.51	0.611
Combined measure of policy support. Averages and standard deviations reported	5.20 (1.22)	5.33 (1.14)	0.94	0.350
Study 2: Agreement with energy efficiency standards for…
(2a) Building materials, such as windows and doors	91.06%	91.58%	0.18	0.860
(2b) The operation of industrial facilities	87.64%	90.53%	0.89	0.375
Combined measure of policy support. Averages and standard deviations reported	1.78 (0.52)	1.83 (0.51)	0.79	0.429

Approximately 200 respondents were assigned to each group. Sample sizes for specific questions vary because ‘don’t know’ responses were excluded from analysis. Equal variance assumed. Data are unweighted.

A similar pattern occurs with policies to promote renewable energy. Approximately 95.48% of respondents who were informed about the mini-public agreed with policies to promote renewable energy compared to 89.64% of respondents who were not informed about the mini-public (t = 2.13, p = 0.034). One of the last policy options presented to respondents was a policy around greening the provincial electricity grid. Approximately 89.20% of respondents who were informed about the mini-public agreed with policies to green the provincial electricity grid compared to 82.11% of respondents who were not informed about the mini-public (t = 1.93, p = 0.054). For the three policies where there are significant differences, the effect sizes range from 5 to 7 percentage points.

For the other policy options, the differences between the two groups of respondents were minimal. Aggregating the policy questions into a single measure suggests that the effects of being informed about the mini-public are minimal (t = 0.94, p = 0.350). Instead, the effects are domain-specific. Overall, the findings from Study 1 suggest mini-publics may affect policy preferences in some policy domains.

In the second study, support for the two policy options was also very high. For this study, being informed about the mini-public’s support for the policies did not affect opinions about either policy. Aggregating the policy questions into a single measure suggests that the effects of being informed about the mini-public are minimal (t = 0.79, p = 0.429). In sum, while there was some support for the effect of mini-publics on policy preferences in Study 1, these findings were not replicated in Study 2.

In terms of registering a ‘don’t know’ response, some questions had a higher rate of ‘don’t knows’ than others. For example, in terms of policies to promote urban density which was one of the least supported policies, approximately 12.41% of respondents provided a response of ‘don’t know’. In terms of energy-efficient travel options, respondents were the least likely, compared to other policy options, to report they did not know their opinions about this policy issue. Only 2.23% of respondents reported they did not know their opinions about promoting energy-efficient travel options.

In general, being informed about the mini-public’s support for the policies decreased ‘don’t know’ responses to the policy questions compared to respondents who were not informed about the mini-public in Study 1 (Table 3). Aggregating the ‘don’t know’ responses across the policy questions suggests that being informed about the mini-public reduced the reporting of ‘don’t know’ responses (t = 1.94, p = 0.053). The largest differences are related to policies about energy-efficient buildings and greenhouse gas emissions. Approximately 4.71% of respondents who were informed about the mini-public reported that they did not know their policy preferences about energy-efficient buildings compared to 10.85% of respondents who were not informed about the mini-public (t = 2.28, p = 0.023). In terms of policies to reduce greenhouse gas emissions, approximately 2.62% of respondents who were informed about the mini-public reported that they did not know their policy preferences compared to 8.96% of respondents who were not informed about the mini-public (t = 2.71, p = 0.007). In both cases, the change in reporting ‘don’t know’ is 6 percentage points.

Table 3.

Knowledge of the Mini-publics and Don’t Know.

	Percentage reporting don’t know	Percentage reporting don’t know
	Not informed	Informed	t-test	p-value
Study 1: Agreement with policies that…
(1a) Promote energy-efficient buildings	10.85%	4.71%	2.28	0.023
(1b) Promote renewable energy	8.96%	7.33%	0.60	0.552
(1c) Promote energy-efficient travel options, such as public transit, cycling and walking	2.83%	1.57%	0.85	0.394
(1d) Promote greater urban density and less expansion	11.79%	13.09%	0.39	0.694
(1e) Green the provincial electricity grid	10.38%	7.33%	1.07	0.285
(1f) Reduce greenhouse gas emissions	8.96%	2.62%	2.71	0.007
Combined measure of don’t know responses. Averages and standard deviations reported	0.54 (1.02)	0.36 (0.70)	1.94	0.053
Study 2: Agreement with energy efficiency standards for…
(2a) Building materials, such as windows and doors	7.73%	6.76%	0.37	0.709
(2b) The operation of industrial facilities	7.73%	6.28%	0.57	0.570
Combined measure of don’t know responses. Averages and standard deviations reported	0.15 (0.47)	0.13 (0.40)	0.55	0.581

Approximately 200 respondents were assigned to each group. Equal variance assumed. Data are unweighted.

In Study 2, the portion of ‘don’t know’ responses was equivalent for both policy options. Being informed about the mini-public had minimal impact on the reporting of ‘don’t know’ as a response to those policy questions. In sum, while there was some support for the effect of mini-publics on reducing ‘don’t know’ responses in Study 1, these findings were not replicated in Study 2.

As mentioned, political trust was measured in terms of trust in decision-making around a specific policy domain. The survey question ties political trust to the policies being asked about. In Study 1, respondents who were informed about the mini-public reported higher levels of trust, on average, compared to respondents who were not informed about the mini-public (t = 2.33, p = 0.021). In Study 2, the informed respondents also reported higher levels of trust, on average, compared to respondents who were not informed, but the difference was small and failed to reach statistical significance (Table 4). In terms of trust, the two studies differ in the magnitude of the effect of being informed about the mini-public.

Table 4.

Knowledge of Mini-publics and Political Trust.

	Average (SD)	Average (SD)
	Not informed	Informed	t-test	p-value
Study 1
How much do you trust the municipal government to make good decisions about climate change? Range: 1–4	2.44 (0.83)	2.63 (0.81)	2.33	0.021
Study 2
How much do you trust the provincial government to make good decisions about energy efficiency? Range: 1–5	2.35 (1.03)	2.46 (0.93)	1.07	0.286

SD: standard deviation.

Approximately 200 respondents were assigned to each group. Equal variance assumed. Data are unweighted.

In terms of political efficacy, being informed about the mini-public positively affects respondents’ perceptions that they could influence government and the government cares about their opinions (Table 5). The magnitude of the effect varies slightly depending on the study and the measure of political efficacy, but the patterns are quite consistent across the studies. In Study 2, respondents who were informed about the mini-public reported higher levels of perceived influence on government, on average, compared to respondents who were not informed about the mini-public (t = 2.60, p = 0.010). In Study 1, the same pattern occurs. When asked how much people like you can affect government action, those informed of the mini-public reported high levels, on average, than those not informed about the mini-public (t = 1.70, p = 0.089). Furthermore, in Study 1, when respondents were asked whether politicians care about the public’s opinion, those informed about the mini-public reported higher levels, on average, than those not informed about the mini-public (t = 1.84, p = 0.066). Both the studies find that hearing about a mini-public affects respondents’ sense of political efficacy. When the two measures of political efficacy are combined, there is a clear and consistent finding that hearing about a mini-public influences political efficacy in both studies.

Table 5.

Knowledge of Mini-publics and Political Efficacy.

	Average (SD)	Average (SD)
	Not informed	Informed	t-test	p-value
Study 1
How much can people like you affect what the government does? Range: 1–5	2.59 (1.07)	2.77 (1.10)	1.70	0.089
In general, do you think that politicians care what people like you think? Range: 1–5	2.58 (1.12)	2.80 (1.20)	1.84	0.066
Combined measure of political efficacy	5.18 (1.75)	5.59 (1.95)	2.20	0.029
Study 2
How much can people like you affect what the government does? Range: 1–5	2.49 (1.04)	2.76 (1.04)	2.60	0.010
In general, do you think that politicians care what people like you think? Range: 1–5	2.53 (1.21)	2.65 (1.19)	1.04	0.298
Combined measure of political efficacy	5.02 (1.89)	5.42 (1.84)	2.14	0.033

SD: standard deviation.

Approximately 200 respondents were assigned to each group. Equal variance assumed. Data are unweighted.

In summary, the experimental manipulation – being informed about the mini-public – affected the likelihood of reporting opinions (vs don’t know) on the proposed policies as well as having an impact on political trust and efficacy in Study 1. For Study 2, being informed about the mini-public affected levels of political efficacy. These findings are replicated once key demographics are controlled for (Table 6). Education has an impact on policy support, political trust and political efficacy in Study 1. An interaction term was introduced composed of education and the experimental manipulation, but none of the interaction terms were statistically significant. For Study 1, the interaction terms were not significant for policy support (t = 0.32, p = 0.748), expressing an opinion (t = 0.52, p = 0.602), political trust (t = 0.80, p = 0.425) or efficacy (t = 1.11, p = .267). For Study 2, the main effects of the experimental manipulation, as well as education, were not significant, and neither were any of the interaction terms.

Table 6.

Multivariate Analysis of Experimental Effects.

	Policy support (combined)			Don’t know (combined)			Political trust			Political efficacy (combined)
	b (SE)	t-test	p-value	b (SE)	t-test	p-value	b (SE)	t-test	p-value	b (SE)	t-test	p-value
Study 1
Being informed = 1	0.17 (0.15)	1.13	0.261	0.21 (0.10)	2.22	0.027	0.25 (0.09)	2.69	0.008	0.51 (0.20)	2.52	0.012
Age	−0.00 (0.01)	−0.66	0.508	−0.01 (0.00)	−2.20	0.028	0.00 (0.00)	1.39	0.166	0.00 (0.01)	0.34	0.735
Education	0.07 (0.02)	2.95	0.003	−0.01 (0.02)	−0.55	0.584	0.06 (0.01)	4.09	<0.001	0.07 (0.03)	2.24	0.026
Females = 1	0.12 (0.15)	0.80	0.424	0.25 (0.10)	2.56	0.011	0.10 (0.09)	1.04	0.299	0.01 (0.20)	0.04	0.968
Income group	0.00 (0.01)	0.07	0.946	−0.01 (0.01)	−1.51	0.132	−0.01 (0.01)	−0.75	0.456	−0.00 (0.02)	−0.14	0.892
Own home = 1	−0.14 (0.20)	−0.71	0.478	−0.19 (0.12)	−1.52	0.130	0.10 (0.12)	0.81	0.418	0.07 (0.26)	0.26	0.794
Study 2
Being informed = 1	0.09 (0.06)	1.54	0.126	−0.01 (0.04)	−0.35	0.729	0.15 (0.11)	1.33	0.183	0.32 (0.21)	1.52	0.131
Age	−0.00 (0.00)	−1.24	0.215	0.00 (0.00)	1.19	0.237	−0.01 (0.00)	−1.24	0.217	−0.01 (0.01)	−1.07	0.286
Education	0.01 (0.01)	1.32	0.187	0.01 (0.01)	0.99	0.323	−0.01 (0.02)	−0.77	0.444	0.05 (0.03)	1.49	0.137
Females = 1	−0.06 (0.06)	−1.03	0.305	−0.06 (0.04)	−1.61	0.108	−0.07 (0.12)	−0.58	0.564	−0.22 (0.21)	−1.26	0.209
Income group	−0.01 (0.01)	−1.06	0.292	−0.00 (0.00)	−1.48	0.141	−0.01 (0.01)	1.46	0.145	0.00 (0.02)	−0.12	0.990
Own home = 1	0.05 (0.09)	0.63	0.532	0.03 (0.05)	0.51	0.631	0.18 (0.16)	1.11	0.269	0.22 (0.30)	0.73	0.464

Discussion and Conclusion

Why are there differences between Study 1 and Study 2? Are these differences related to the specific policies being proposed, question order effects or random chance? Each of these explanations, as well as differences in the characteristics of the mini-publics, is explored. While some scholars might argue that the different characteristics of the mini-publics are a design weakness, the value of the research design is (1) using truthful descriptions of actual mini-publics, (2) replicating findings about political efficacy across these two studies and (3) finally pointing to possible variables impacting the endorsement effects of mini-publics, which helps guide further research.

Zaller (1992) talks about the public’s dependence on elites for their policy preferences. This project reveals instances where the public could be influenced by non-elites. Gastil (2000) recommends evaluating the quality of mini-publics’ recommendations by whether citizens use the mini-publics’ judgements when deciding how to vote. In this project, the effects of mini-publics on policy preferences were mixed in Study 1 and non-existent in Study 2. Of the eight different policies examined, being informed about the mini-public affected policy support for three policies. The differences between the informed and uninformed groups were 5–7 percentage points for these three policy areas. Why these policies? One explanation is that mini-publics may affect contentious versus consensus issues differently (Warren, 2009). All policies had strong support, and the findings do not differ for those with the highest and lowest levels of support. For example, approximately 92.35% of respondents supported policies around energy-efficient travel options. Being informed about the mini-publics’ recommendations did not affect level of support for this policy option. In contrast, only 72.00% of respondents supported urban densification. Being informed about the mini-publics’ recommendations did not affect level of support for this policy option either. As such, the effect of being informed about the mini-public does not depend on whether or not the policy is contentious.

A second explanation for the mixed findings is that information about mini-public fades over the course of the survey, as additional questions are posed. However, for Study 1, the effect of the manipulation does not diminish as the survey continues since differences were observed with the first two policy options and the fifth policy option. In terms of reducing ‘don’t know’ responses, the manipulation impacted the first and the sixth policy options. The impacts were decreases in ‘don’t know’ by 6 percentage points in these two policy areas. Finally, differences re-emerged for the questions about political efficacy and trust questions (the seventh, eighth and ninth questions after the manipulation). Respondents recall the information about the mini-public, and this information has a continued effect on responses far into the survey interview. As such, the pattern of findings cannot be explained by question order effects and the diminishing role of the manipulation, that is, information about the mini-public, as the survey interview continues.

A third explanation relates to the ambiguity of the policies. For example, perhaps mini-publics affect policy preferences when the policy is more ambiguous, requiring respondents to draw upon cues within the survey to make a choice about the policy (Druckman et al., 2010). However, again, the findings do not offer support for this explanation. The policy domain with the most ‘don’t know’ responses was urban densification. This was also the policy with the lowest level of policy support. The effect of the mini-public was not more pronounced for this policy domain where 12.41% of respondents reported they did not know their policy preferences.

A last possible explanation would be to reduce Study 1’s findings to random chance. However, that conclusion would be too hasty. Two of the three policy domains affected by the mini-public’s recommendations are closely related to policy proposals. Being informed about the mini-public affected policy preferences around promoting renewable energy (the second question) and greening the provincial electricity grid (the fifth question). The consistency in findings for the two related policy areas suggests that the findings are not due to chance, but rather the effects are domain-specific. In other words, citizens do not take the mini-public’s advice indiscriminately. Instead, they seem to consider each proposed policy carefully and then decide whether they will support a mini-public’s recommendation on this policy domain. The findings offer evidence of a ‘deliberation within’ (Goodin and Niemeyer, 2003) or perhaps a ‘vicarious deliberation’ (Gastil et al., 2014) where respondents carefully consider a variety of inputs when determining their policy preferences. The specificity of the effects should not be viewed as a weakness of the study, but rather as an indication that citizens are discerning about what endorsements they follow.

Why did one study produce effects and the other study not produce effects on policy preferences? Study 1 found an effect of being informed about the mini-public on promoting energy-efficient buildings. Study 2 asked about policies related to the construction of buildings, but did not find similar effects. The most plausible explanation of this difference in findings relates to the characteristics of the mini-publics described. The two mini-publics differ on several dimensions: level of government, involvement of government, size of mini-public and length of time that the mini-public met as well as other subtle differences, such as framing of the information as balanced and description of the ‘consensus’ in policy recommendations. Size of the mini-public and framing of the consensus should favour, theoretically speaking, effects in Study 2, but the effects of mini-publics are observed in Study 1, which was longer and involved government in the sponsorship of the mini-public. The role of government in the deliberation is also important, but this factor would be expected to have an impact on political efficacy and trust, more so than policy views. The length of the deliberation signals a variety of important features about the deliberative process, including how informed participants were and the degree to which they deliberated. These features are likely to impact the broader public’s decision to follow the mini-public’s recommendations.

The difference that seems most relevant for explaining the findings is the length of time that the mini-public met. These differences could explain why Study 1 produced effects when Study 2 did not. In Study 1, members of the mini-public were engaged for a greater length of time than Study 2 (six Saturdays as opposed to 2 hours). Perhaps this greater length of time creates a perception of more highly specialised knowledge among the Study 1 mini-public compared to the Study 2 mini-public. As mentioned, Lupia and McCubbins (1998) argue that the ability to persuade depends, in part, on whether the third party has specialised knowledge that is relevant to the persuasion attempt. Warren (2009) suggests that the degree of specialised knowledge could be assessed by the length of participation in the mini-public, with mini-publics extending days or months having greater knowledge than those meeting for several hours. As such, Study 1 finds some effects of mini-publics on public opinion because of this more extensive engagement in the deliberative process, which produces more specialised knowledge. The most promising line of inquiry is about the length of time spent deliberating policies. Further research should test the effects of being informed about mini-publics by alternating the description of the time spent deliberating policies. This line of research is important since most government policy consultations take the form of 2-hour meetings. Study 2 suggests, but cannot definitely prove, that these 2-hour consultations will have minimal impact on the broader public’s policy preferences, the expression of opinions (vs ‘don’t know’) and building political trust.

The two studies also offer different findings in terms of political trust. The first study offers greater support for the effect of mini-publics on political trust compared to the second study. The political trust question was very specific in referencing the level of government being trusted and the policy domain in which this trust is granted. The political trust question is closely tied to the policy-making process. As such, it makes sense that the findings for political trust follow the findings around policy preferences. In Study 2, there were no effects of mini-publics on policy preferences and, likewise, no significant differences in level of trust reported around government decisions on this policy issue. In contrast, Study 1 offers some effects of mini-publics on policy preferences as well as differences in trust tied to the policy-making process. The difference in findings may be explained by differences in the sponsors of the mini-publics. Study 1’s mini-public was clearly identified as an initiative involving the government, whereas Study 2’s mini-public was completely independent of the government. This difference could explain why Study 1 observed differences in political trust, whereas Study 2 did not observe differences in political trust. Study 1’s mini-public offers greater legitimacy to government decision-making, because the government was front and centre in the organisation of the mini-public. Perhaps only under these circumstances do the effects of mini-publics translate in to increased political trust. Further research should test the impact of mini-public sponsorship to clarify how this impacts the broader public’s views about the mini-public.

The findings about political efficacy are more conclusive in that both studies replicated the finding that being informed about the mini-public affects sense of political efficacy. The effect differs slightly by study and by the specific political efficacy question, but the findings affirm that being informed about the mini-public leads to a greater sense of political efficacy than not being informed about the mini-public. The replication of the finding is a surprise since the mini-publics differ on a number of characteristics which may influence responses about political efficacy. First, the level of government and its role differ for the two studies. In Study 1, the municipal government is targeted and a sponsor of the mini-public. In Study 2, the provincial government is targeted, but the organisers are a community group and academics. However, the level of government and its role in the mini-public did not produce differential effects on political efficacy. The findings offer clear policy implications for governments. Publicising the results of these types of public consultations can have clear benefits on citizens’ sense that they can influence government and that politicians care. This positive effect translates across levels of government and across mini-publics with different characteristics.

Overall, the findings suggest that mini-publics help generate legitimacy in the political system, not necessarily through policy legitimation but through increasing political efficacy and possibly building political trust between citizens and government. The increase in trust may depend on the role of government in sponsoring the mini-public. Further research might explore the effects of mini-publics on political efficacy and trust more fully. While the political science literature focuses on individual political efficacy, the issue of collective efficacy seems more relevant in this situation. Mini-publics reflect the potential of a collective group of citizens to influence governments, rather than an individual citizen. As such, being informed about a mini-public might also affect collective efficacy. Furthermore, further research could explore how regular use of mini-publics (rather than a case study of a mini-public) affects perceptions about legitimacy. A single case study may not have enduring effects on citizens’ psyche, but a continual pattern of involving average citizens in generating policy might lead to broader pay-offs in generating legitimacy in the political system (Gastil, 2000). The broader impact of these mini-publics depends on awareness. The current research offers insights into how mini-publics and their work should be communicated to fellow citizens through press releases and media coverage.

Finally, this study points to the factors that may influence the impact of mini-publics on public opinion. These factors include the size of the mini-public, length of time spent on deliberation, framing of information received and decisions made, sponsorship of the deliberative event and level of government targeted with the deliberative event. The study is distinctive in highlighting two real-life deliberative events, but the trade-off of this approach is that many factors differed across the experimental conditions. Further research might use vignettes to examine the various factors in a multi-factorial design.

Footnotes

Acknowledgements

The author would like to thank David Kahane and the Alberta Climate Dialogue Steering Committee for supporting this research.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by a Community–University Research Alliance grant from the Social Sciences and Humanities Research Council to the Alberta Climate Dialogue group (#833-2009-4011).

Author Biography

Shelley Boulianne is an Associate Professor of Sociology at MacEwan University. She completed her PhD in Sociology at the University of Wisconsin–Madison. Her research focuses on media use, civic and political engagement and survey research methodology.

References

Ansolabehere

Rodden

Synder

(2008) The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting. American Political Science Review 102 (2): 215–232.

Arceneaux

Kolodny

(2009) Educating the Least Informed: Group Endorsements in a Grassroots Campaign. American Journal of Political Science 53 (4): 755–770.

Bächtiger

Setälä

Grönlund

(2014) Towards a New Era of Deliberative Mini-publics. In: Grönlund

Bächtiger

Setälä

(eds) Deliberative Mini-publics: Involving Citizens in the Democratic Process. Colchester: European Consortium for Political Research Press, pp.225–245.

Baker

(2015) First Results from Psychology’s Largest Reproducibility Test. Nature. Available at: http://www.nature.com/news/first-results-from-psychology-s-largest-reproducibility-test-1.17433?WT.mc_id=TWT_NatureNews (accessed 9 May 2016).

Barabas

Jerit

(2010) Are Survey Experiments Externally Valid? American Political Science Review 104 (2): 226–242.

Bullock

(2011) Elite Influence on Public Opinion in an Informed Electorate. American Political Science Review 105 (3): 496–515.

Cutler

Johnston

Carty

et al . (2008) Deliberation, Information, and Trust: the British Columbia Citizens’ Assembly as Agenda Setter. In: Warren

Pearse

(eds) Designing Deliberative Democracy: The British Columbia Citizens’ Assembly. Cambridge: Cambridge University Press, pp.166–190.

Delli Carpini

Keeter

(1996) What Americans Know about Politics and Why It Matters. New Haven, CT: Yale University Press.

Dewan

Humphreys

Rubenson

(2014) The Elements of Political Persuasion: Content, Charisma and Cue. Economic Journal 124 (574): 257–292.

10.

Druckman

Hennessy

St Charles

et al . (2010) Competing Rhetoric over Time: Frames versus Cues. Journal of Politics 72 (1): 136–148.

11.

Fishkin

Luskin

(1999) Bringing Deliberation to the Democratic Dialogue. In: McCombs

Reynolds

(eds) The Poll with a Human Face: The National Issues Convention Experiment in Political Communication. Mahwah, NJ: Lawrence Erlbaum, pp.3–38.

12.

Fournier

van der Kolk

Carty

et al . (2011) When Citizens Decide: Lessons from Citizen Assemblies on Electoral Reform. Oxford: Oxford University Press.

13.

Gastil

(2000) By Popular Demand: Revitalizing Representative Democracy through Deliberative Elections. Los Angeles, CA: University of California Press.

14.

Gastil

Knobloch

(2010) Evaluation Report to the Oregon State Legislature on the 2010 Oregon Citizens’ Initiative Review. Seattle, WA: University of Washington. Available at: http://healthydemocracy.org/wp-content/uploads/2012/07/OregonLegislativeReportCIR.pdf (accessed 9 May 2016).

15.

Gastil

Deess

Weiser

et al . (2010) The Jury and Democracy: How Jury Deliberation Promotes Civic Engagement and Political Participation. Oxford: Oxford University Press.

16.

Gastil

Knobloch

Kelly

(2012) Evaluating Deliberative Public Events and Projects. In: Nabatchi

Gastil

Weiksner

et al . (eds) Democracy in Motion: Evaluating the Practice and Impact of Deliberative Civic Engagement. Oxford: Oxford University Press, pp.205–230.

17.

Gastil

Richards

Knobloch

(2014) Vicarious Deliberation: How the Oregon Citizens’ Initiative Review Influenced Deliberation in Mass Elections. International Journal of Communication 8: 62–89.

18.

Goodin

Niemeyer

(2003) When Does Deliberation Begin? Internal Reflection versus Public Discussion in Deliberative Democracy. Political Studies 51 (4): 627–649.

19.

Grönlund

Setälä

Herne

(2010) Deliberation and Civic Virtue Lessons from a Citizen Deliberation Experiment. European Political Science Review 2 (1): 95–117.

20.

Hetherington

Husser

(2012) How Trust Matters: The Changing Political Relevance of Political Trust. American Journal of Political Science 56 (2): 312–325.

21.

Kuklinski

Quirk

Jerit

et al . (2000) Misinformation and the Currency of Democratic Citizenship. Journal of Politics 62 (3): 790–816.

22.

Lupia

McCubbins

(1998) The Democratic Dilemma: Can Citizens Learn What They Need to Know? New York: Cambridge University Press.

23.

Morrell

(2005) Deliberation, Democratic Decision-making and Internal Political Efficacy. Political Behavior 27 (1): 49–69.

24.

Mutz

(1992) Impersonal Influence: Effects of Representations of Public Opinion on Political Attitudes. Political Behavior 14 (2): 89–122.

25.

Mutz

(2011) Population-Based Survey Experiments. Princeton, NJ: Princeton University Press.

26.

Nabatchi

(2012) An Introduction to Deliberative Civic Engagement. In: Nabatchi

Gastil

Weiksner

et al . (eds) Democracy in Motion: Evaluating the Practice and Impact of Deliberative Civic Engagement. Oxford: Oxford University Press, pp.3–39.

27.

Open Science Collaboration (2015) Estimating the Reproducibility of Psychological Science. Science 349 (6251): 1–8.

28.

Pickup

Sayers

Knopff

et al . (2004) Social Capital and Civic Community in Alberta. Canadian Journal of Political Science 37 (3): 617–645.

29.

Smets

Isernia

(2014) The Role of Deliberation in Attitude Change: An Empirical Assessment of Three Theoretical Mechanisms. European Union Politics 15 (3): 389–409.

30.

Sniderman

Grob

(1996) Innovations in Experimental Design in Attitude Surveys. Annual Review of Sociology 22: 377–399.

31.

Strandberg

Grönlund

(2012) Online Deliberation and Its Outcome – Evidence from the Virtual Polity Experiment. Journal of Information Technology and Politics 9 (2): 167–184.

32.

Van Noorden

(2015) Political Science’s Problem with Research Ethics. Nature. Available at: http://www.nature.com/news/political-science-s-problem-with-research-ethics-1.17866 (accessed 9 May 2016).

33.

Warren

(2009) Two Trust-based Uses of Minipublics in Democracy. Paper Presented at the American Political Science Association Annual Meeting, Toronto, ON, 3–6 September.

34.

Zaller

(1992) The Nature and Origins of Mass Opinion. Cambridge: Cambridge University Press.