Abstract
Civic technology is used to improve not only policies but to reinforce politics and has the potential to strengthen democracy. A search for new ways of involving citizens in decision-making processes combined with a growing smartphone penetration rate has generated expectations around smartphones as democratic tools. However, if civic applications do not meet citizens’ expectations and function poorly, they might remain unused and fail to increase interest in public issues. Therefore, there is a need to apply a citizen’s perspective on civic technology. The aim of this study is to gain knowledge about how citizens’ wishes and needs can be included in the design and evaluation process of a civic application. The study has an explorative approach and uses mixed methods. We analyze which democratic criteria citizens emphasize in a user-centered design process of a civic application by conducting focus groups and interviews. Moreover, a laboratory usability study measures how well two democratic criteria, inclusiveness and publicity, are met in an application. The results show that citizens do emphasize democratic criteria when participating in the design of a civic application. A user-centered design process will increase the likelihood of a usable application and can help fulfill the democratic criteria designers aim for.
Introduction
Democratic innovations that take place face-to-face are a well-researched area (see e.g., Elstub & Escobar, 2019), but there is a need for a more systematic take on digital forms of democratic innovations both as a concept and in the form of empirical studies (Smith, 2019b). Democratic innovations are defined as “institutions designed specifically to increase and deepen the participation of citizens in the political decisions that affect their lives” (Smith, 2019a, p. 6) and their main aim is to increase the quality of democratic governance (Geissel, 2012). Some examples of democratic innovations are deliberative mini-publics, and online applications (Jäske & Ertiö, 2019). Smith (2019b, p. 578) points out that “although a great deal of sophisticated innovation has emerged in the digital realm, it has been accompanied by little sophisticated reflection on its democratic qualities. Digital innovations are likely to disrupt our categories of analysis, developed primarily through familiarity with face-to-face forms of engagement.”
There is a growing interest in academia surrounding the term civic technology, which can be defined as “technology…that facilitates democratic governance among citizens” (Saldivar et al., 2019, p. 170). This notion also highlights the use of relatively similar concepts in literature, civic technology and digital democratic innovations, and we will further develop this discussion in the theoretical section. Moreover, little is known about citizens’ perspectives on democratic innovations and “Citizens may…show complex democratic preferences, articulating participation, citizen deliberation and representation” (Jacquet, 2018, p. 16). Research on applications that make use of open data in the interest of the public is scarce, although repositories making open data available is on the rise (Saldivar et al., 2019, p. 193). Yet, constant technological advancements and an open data approach by governments open up new prospects for treating the ills of democracy. Hence, the field of democratic innovations must “reach out to colleagues working in areas such as ‘civic tech’ to better understand the range and dynamics of these novel forms of participation” (Smith, 2019b, p. 578).
We argue for a need to apply a citizen’s perspective on the design of digital democratic innovations. One solution is a user-centered design (UCD) process that focuses on an understanding of the users, their needs, and the context in all stages of design and development. UCD is a broad term to describe design processes in which end-users influence how a design takes shape (Abras et al., 2004, p. 1). UCD aims at improving the usability of software and technical applications, and “rather than describing different usability methods, it describes usability at a level of principles, planning and activities” (Jokela et al., 2003, p. 53). The international standard, ISO 9241-11 (ISO 9241, 2018), defines usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. In the design of civic technology, functionality and usability are often only being considered after applications are created (McDowell & Chinchilla, 2016, p. 471). This paper reports on a project called Pocket Democracy. The idea of the project is to create a mobile application that uses open data to help citizens learn about the activities of their municipality.
The main aim of this study is to gain knowledge about how citizens’ wishes and needs can be included in the design and evaluation process of a civic application. Hopefully, the lessons learned in this study can help future designers to design civic applications that both strive to fulfill visions from democratic theorists as well as fulfilling the wishes of citizens that are supposed to use these applications. This paper addresses the following research questions:
Which democratic criteria do citizens emphasize in a UCD-process of a civic application? How can a UCD-process help measure if civic applications meet the democratic criteria they aim for?
The paper is structured as follows: in the literature review, we introduce the concept of civic technology and discuss six democratic criteria for evaluating the democratic potential of civic applications. Thereafter, we describe how and why we use focus groups and usability tests to answer the research questions. In the result and analysis section, we present findings from our two sub-studies. Lastly, we revisit the research questions and conduct a concluding discussion accompanied by recommendations for designing digital democratic applications.
In this section, we first review the concept of civic technology. Secondly, we discuss how previous research has defined democratic criteria and how the dimensions relate to civic applications.
Civic technology
Some scholars define civic technology broadly as “the use of technology for public good” (Sifry et al., 2016, in Graeff, 2018, p. 24), while others more narrowly view it as “an emerging field that typically leverages open data and sometimes open-source software to address challenges that may be invisible or neglected by government in a collaborative, problem-oriented way” (Wilson & Charkraborty, 2019). According to Gilman and Peixoto (2019, p. 106), the academic literature has not been able to keep up the pace of democratic initiatives related to civic technology. Similarly to Gilman and Peixoto (2019, p. 106), we focus on the civic dimension of the term and define civic technology as “technology that is explicitly designed and leveraged to increase and deepen democratic participation.” In other words, we view certain civic technology as a form of digital democratic innovation, which encompasses new tools for connecting citizens with local-level decision-making. Not all democratic innovations are explicitly designed to engage citizens in decision-making (Smith, 2019b, p. 93). Civic technology with a democratic purpose, however, can be regarded as digital democratic innovation. Smith (2019b, p. 95) acknowledges that there is limited knowledge about digital and hybrid (online/offline) forms of democratic innovations.
Examples of civic technology include FixMyTransport – a platform for citizen feedback on public transportation (May & Ross, 2018), the social news site Menéame, and platforms for participatory democracy, such as Decide Madrid and Decidim Barcelona (Aragon, 2019), participatory budgeting including digital elements (Gilman & Peixoto, p. 108), the e-petition sites change.org (Gilman & Peixoto, 2019, p. 109) and We the People (Graeff, 2018, 24), mobile applications such as Citizens Connect (Crawford & Walters, 2013 in Gilman & Peixoto, 2019, p. 111), reporting platforms (e.g., SeeClickFix.com) (Gilman & Peixoto, 2019, p. 113) or parliamentarian activity monitoring sites such as TheyWorkForYou.com (mySociety, 2020).
Civic technology has been categorized as either government-centric or citizen-centric. However, the common theme in both perspectives is enabling participation in democratic governance (Saldivar et al., 2019, p. 170). For instance, the concept of the co-design process (Norman 2013; Blomkamp 2018) addresses social challenges for the public sector in discussions with citizens. Including citizens in the design process may “generate more innovative ideas, achieve economic efficiencies by improving responsiveness, foster cooperation between different groups, reinvigorate trust between citizens and public servants, and have transformative effects on participants’ agency and wellbeing” (Blomkamp 2018, p. 739). Another challenge for civic applications is the digital divide (Norris, 2001), where some groups may encounter challenges with using technology. Hence, even when civic technology is designed by a powerful government institution (e.g., the We the People petition platform in the US), it can fail to reach its democratic goals (Graeff, 2018, p. 19). Researchers should avoid cherry-picking exemplary successful cases when assessing the impact of democratic innovations to not distort the view on the impact of various innovations (Spada & Ryan, 2017; Smith, 2019b).
Democratic criteria in relation to civic applications
The internet is known to have a vulnerable democratic potential. Digital tools can both improve democracy by enabling cross-cutting discussions and undermine it by helping the spread of disinformation (Anderson & Raine, 2020; Coleman & Blumler, 2009; Morozov, 2013). Lately, civic applications that aim to solve public problems by leveraging government-provided and user-generated data (Desouza & Bhagwatwar, 2012), have gained attention among scholars (May & Ross, 2018; Jäske & Ertiö, 2019). To measure whether civic applications contribute to democracy, one can turn to democratic theory.
A study by Jäske and Ertiö (2019) introduces a framework for evaluating the democratic potential of civic applications. Rather than merely basing the evaluation of civic application on technical criteria, they propose a framework based on democratic theory (e.g., Young, 2000) and previous research (e.g., Desouza & Bhagwatwar, 2012). Their framework is part of a wider research agenda of democratic innovations, focusing on testing whether ICT-tools live up to normative criteria, “democratic goods” (Smith, 2009, p. 162), linked with democratic institutions. Smith (2009; 2019b) identifies inclusiveness, popular control, considered judgment, transparency, efficiency, and transferability as important democratic goods. Scholars have adapted these democratic criteria for evaluation of democratic innovations, although naming them differently (Geissel, 2012; Jäske & Ertiö, 2019). For the sake of clarity, we use the same dimensions and refer to the same literature regarding their definitions as Jäske and Ertiö (2019, p. 9). The six dimensions (inclusiveness, deliberation, influence, publicity, mobilization, knowledge production) are presented in more detail below and discussed according to how they can be implemented in civic technology.
Inclusiveness
The democratic core value of inclusiveness entails that those affected by political decisions should be included in the process of decision-making. Generally, democratic theorists distinguish between external inclusion and internal inclusion. The former means citizens should have equal possibilities to participate, whereas the latter refers to equality of voice within the participatory process (Young, 2000).
A civic application that embraces inclusiveness ensures equal participation of different groups within the app, pays attention to the representativeness of users, and makes special efforts to enhance inclusion of marginalized groups. In a user-centered perspective, inclusion entails that the design process takes the diversity of end-users (e.g., varying levels of digital know-how) into account.
Deliberation
Ideally, collective decisions are preceded by deliberation which means giving and listening to reasons for different viewpoints. Deliberation is thought to have several positive effects among participants: increased empathy and mutual understanding through perspective-taking, better awareness of one’s preferences, reduced cognitive biases, and increased efficacy (Myers & Mendelberg, 2013; Dryzek et al., 2019).
Civic applications can contribute to deliberation in the public sphere by enabling political discussion regardless of physical borders and creating arenas for discussion across opinion enclaves. However, to fulfill the ideals of deliberation, civic applications can include features that enable discussion among citizens as well as two-way communication between citizens and government.
Influence
A lack of influence on actual policy has concerned supporters of democratic innovations (Mikaelsson & Wihlborg, 2011). Ideally, citizens should be able to participate in collective decisions and influence these. The degree of influence can vary between direct authority (e.g., binding referendums) over decisions to an advisory role (e.g., government feedback on petitions, municipal surveys, or hearings).
To meet the criterion of influence, civic applications should link app activity to administrative or political processes, provide evidence of the uptake of input via the application, and provide users with feedback from administrative or political authorities. Thus, some kind of external impact should be possible if civic applications are to influence decision-making.
Publicity
A central requirement for representative democracy is that citizens ought to have access to correct information about government activity to hold decision-makers accountable for their actions and assess the democratic system (Gilman & Peixoto, 2019). Publicity refers to the manner information about government activity is made accessible to the public, whereas transparency is the extent to which this information is available. However, governmental information is often difficult to access for citizens (Kosec & Wantchekon, 2020).
Research has shown that civic applications are often developed to make the task of accessing government data less daunting (Desouza & Bhagwatwar, 2012). Accordingly, civic applications that reveal facts, visualize open data in a pedagogical manner, may contribute to publicity. In practice, to fulfill the criterion of publicity, a civic application could share or afford users with easy access to information on government processes and policies, or the political rights of citizens.
Mobilization
Motivating citizens to mobilize into collective action is often a challenge, and citizens are not always aware of their options to participate (Peixoto & Sifry, 2017). Citizens can increase their influence in the political process by joining associations or social movements that are created around specific political issues.
Civic applications can be designed with features that allow citizens to learn about or organize collective action, create groups, establish movements, or arranging street protests. For example, features that help citizens to organize themselves for collective action (Conroy et al., 2012), or provide information about rights concerning collective action, might contribute to mobilization (Jäske & Ertiö, 2019). Civic technology can also help us to understand who participates in different types of initiatives and who chooses not to.
Knowledge production
Both individual (e.g., contacting a politician) and collective (e.g., demonstrations) forms of political participation produce knowledge to decision-makers. Knowledge in this sense might mean that politicians know the public opinion on a certain issue or that a citizen has identified a problem relating to a publicly funded road, for example. Using “the wisdom of crowds”, crowdsourcing can produce knowledge and bring attention to both local and national problems relating to policy implementation (Jäske & Ertiö, 2019, p. 16).
In the context of civic applications, knowledge production can help to improve democratic governance by enabling opinion-voicing and problem identification by citizens. This feature can take the form of a problem-reporting tool or user-generated data, which represent the individual or collective opinions of citizens. In other words, a civic application allowing users to voice their opinions or observations produces a new channel for input to policymakers. Civic technology often produces vast amounts of data that can be turned into knowledge in the public interest (Baack, 2017).
Methods
The UCD process and the methods used in the project are illustrated in Fig. 1. After an initial mapping of the field to identify similar applications, the needs and wishes of citizens were collected during focus group discussions. Based on this information, a prototype of the application was created and tested in a laboratory setting. The prototype was a semi-interactive prototype of the app, meaning that some features were clickable and responsive, but other features of the app were static. After the usability tests in the lab, the application was improved and beta-tested before being released in the municipality. In this article, we report on data from the focus group discussions and the laboratory usability tests.
The UCD process in the Pocket Democracy project.
We used focus group methodology to increase our understanding of how end-users relate to democratic criteria. Focus groups are a qualitative methodology, where data is collected through small group discussions (i.e., six to eight people) involving a moderator that facilitates a discussion that centers on given themes in the form of open-ended questions (Esaiasson et al., 2007, p. 307; Cyr, 2017, p. 1038). The goal is a free discussion with minimal involvement from the moderator to create an open permissive atmosphere (Cyr, 2017; Gustafsson, 2008). Focus groups can be particularly helpful when studying people’s motives, experiences, and thought processes on a more in-depth level (Gustafsson, 2008).
The purpose of focus groups is not to generalize to the general population, but to map the existence of different thought categories and reasoning. Focus group methodology has two main weaknesses. They typically come together using nonprobability sampling, which makes it difficult for them to be representative of a population. Moreover, focus groups constitute an artificial setting where behavior does not necessarily reflect what an individual might say or do in a “real” setting. Additionally, focus groups may create group pressure to conform to a specific opinion (Cyr, 2017). However, when applying a user-centered design process, focus groups are a common method for conducting user research, trying to understand the end-users’ visions, purposes, goals, and constraints (Gulliksen et al., 2003).
The Pocket Democracy application prototype has been tested in a small (population circa 5000) bilingual municipality in Finland. At the start of the design process in November 2018, citizens were invited to the focus groups via the municipality’s Facebook page. Moreover, the Youth Council, the Council for issues concerning people with disabilities, and the Council for Elders were invited. In total, 18 participants (61% male, mean age 42 years, Mdn age
The focus group occasion started with a brief presentation of the Pocket Democracy project and the basics of focus group methodology. Participants were instructed to brainstorm freely about which features one could include in an app of this kind. After this, the groups and moderators moved to separate rooms. Initially, the moderator informed the participants of research ethics relating to the study and participants’ rights. The moderators then initiated a discussion around four broader themes: communication with the municipality, the groups’ wishes for app functionality, feedback on the project idea, and finally a summarizing discussion about the most important features of the app. The audio was recorded for transcription and each group discussion lasted for about one hour. After the discussion, participants filled out a short survey asking about background variables such as age, education, work affiliation, and media consumption habits. The discussions were later coded by the researchers to see if any of the democratic criteria in Jäske and Ertiö’s framework were present in the discussions using deductive qualitative text analysis.
Study two: Usability study
For the usability study, a prototype of the application was developed by a company. The main idea behind the tested application is to visualize municipal data and make it easier to follow the development of issues in the political process. The application prototype used a sample of real municipal data and some of the features were interactive and clickable while others were static. The features included in the prototype were a feed of personalized content, a feed of the latest news, the option to follow interest areas as well as specific errands, a list of politicians with information on their representations in municipal councils, executives and committees. The application contained contact details to politicians as well as a search function that searched the municipal documents (e.g., meeting agendas and protocols) visualized in the application. Since the app tested in the laboratory was a first semi-interactive prototype of the app that did not yet contain all features, two of the democratic criteria were tested: inclusiveness and publicity. These were also the criteria best related to the aim of the application: to make it easier for citizens to follow local politics.
The usability study in a laboratory setting was done in March 2019. In this study, 15 people participated (35% females, age 21–70, M age
When arriving at the laboratory, a short background interview was conducted, and participants’ rights were presented. To ensure that all participants had an equal understanding of the tested application, a short description of the aim of the application was presented.
During the second stage of the test, participants were seated in front of a computer, showing the prototype on a phone screen emulator, mimicking the appearance of the application on a smartphone. The prototype was shown on a computer to enable screen sharing with the control room and eye-tracking.1
Eye-tracking data was analyzed in another study.
Thirdly, a semi-structured interview was conducted with the participants. The application was shown on a mobile device, and the participant could click around and give feedback on the different features. Participants were asked to rank the six democratic criteria in relation to the application. Participants were introduced to the criteria by reading a paragraph explaining each criterion in short, based on definitions in Jäske and Ertiö (2019). Next, they were told to place each of the six criteria into three categories: 1. “Must-have” features, 2. “Could-have” features, or 3. “Not needed” features. Participants could choose to place all the six criteria in the same category (e.g., if they thought all were “must-have” features). This was done to approximate which democratic criteria were the most important for users in the development of an app for following politics at the local level.
Study one: Focus groups
The criteria mentioned in all three focus group discussions were inclusiveness, deliberation, publicity, and knowledge production. Moreover, the two groups that focused on influence were the municipal employees and the middle-aged/elderly people.
Democratic criteria mentioned in the focus groups
Democratic criteria mentioned in the focus groups
Inclusiveness was discussed in terms of the problems that might occur for certain user groups. The young people emphasized that some login features (such as bank code login) are a problem for young people, and the participants in the elderly group discussed that not all elderly people have access to a smartphone and that tablets are better for people with bad vision.
Deliberation, in the form of two-way communication between citizens and the municipality, was both seen as a feature that would encourage users to find the app, as well as a good way for politicians to learn about the opinions of the citizens. The municipal employees said that the application could help create a dialogue between the citizens and the municipality, something that was viewed as a challenge in today’s political process.
Publicity was seen as a problem, since finding information in the municipal documents can be a challenge. Several participants in the focus groups said that it would be nice to follow issues you are specifically interested in and to get a notification when new information is available.
Knowledge production was discussed as polls about different issues, where the results would be presented to the politicians or the municipality. This could be a way of communicating citizen’s opinions to decision-makers. However, the groups also discussed the problems related to asking questions, and what it means if only a small percentage of the citizens answer a poll.
Influence was discussed in terms of getting information out to the public before a decision is made. Knowing about what is going on in the local community and being able to voice opinions before an issue is decided was seen as desirable, but a challenge with today’s tools.
The overall experience of the application prototype was studied using the System Usability Scale (SUS). Participants’ responses were processed according to the SUS calculations and the average score for the application is 82, an A-rating (Sauro, 2011). Anything over 68 is a good rating over the average, hence, the application performed well. The strongest sentiments were that participants thought other people would learn to use the app very quickly and that it was very easy to use.
Test of inclusiveness
Inclusiveness seeks equal participation of different groups within the app. The app was developed with the help of citizens of different ages as well as participants from marginal groups in the form of the Council for people with disabilities. In the usability test, participants from different age groups were invited (21–70 years, mean 40 years), and this usability test of the inclusiveness criterion investigates if different age groups faced different usability problems in the app.
The results are presented in Table 2 and show no significant association between age and task completion
Task completion for different age groups
Task completion for different age groups
The criterion of publicity pursues easy access to information on municipal processes and policies. Publicity is one of the main aims of the tested app, visualizing public information in a more user-friendly way. The information in the app can be divided into three stages: municipal issues (containing information on specific issues), interest areas (containing information on several issues, e.g., culture and leisure time), and latest news (all new issues from the municipality).
The usability test contained tasks that test the usability of all three levels of information. The test person was either asked about the function on a broader level to see if they understand the idea behind the function (e.g., “What do you think will happen when you click on the feature interest areas?”) or asked to perform a task (e.g. “Add a new interest area to your feed”).
The results from the usability test are presented in Table 3. When presented with the functions, before being able to try them out and click around in the app, some participants struggled with understanding their idea. Especially the function Municipal Issues caused some trouble for more than half of the participants. Several participants said that it is good that the documents open directly in the app, and that you do not need any other tools to read a PDF-document. However, since the app is used on mobile devices, the participants would have preferred to get the text directly in the app, not having to zoom in a PDF-document. Many participants also called for a summary of the document.
Task completion concerning the function of publicity
Task completion concerning the function of publicity
Over 70% of the participants understood the idea of Interest Areas. The function was seen as a helpful tool for visualizing all the information from the municipality that otherwise can be a handful. However, some participants thought that the areas were too broad and wanted to be able to select interest areas in more detail (e.g., only leisure instead of leisure and culture). The idea behind the function Latest News was also easy to understand for most participants (67%), and all participants passed the task connected to Latest News. The comments concerning this feature were all positive, that it is a clear and simple way to get the latest information from the municipality.
When performing tasks concerning all three levels of publicity, most of the participants (93–100%) passed the tasks without any trouble. Hence, the app fulfills the criterion of publicity.
After the testing in the laboratory, participants were interviewed and asked to rank how important the democratic criteria presented by Jäske and Ertiö (2019) are concerning the Pocket Democracy app. The result of this ranking is presented in Table 4.
Participants’ ranking of criteria in order of importance
Participants’ ranking of criteria in order of importance
Note: the numbers represent the number of times a criterion was placed in a specific category. 14 participants ranked six criteria. The numbers do not always add up to 14 due to some participants not placing all criteria into a specific category.
When participants were asked to rank the six democratic criteria used for evaluating civic applications (Jäske & Ertiö, 2019), the criteria of publicity and inclusiveness were regarded as the most important (see Table 4 above). Publicity was a feature that every participant viewed as an essential feature in the app. The possibility of receiving information on local politics would increase knowledge and create more trust were some of the comments from the participants. Concerning inclusiveness, participants wanted the app to be free of charge and available for both Android and IOS. Their biggest concern was how to get elderly people to be able to use the app and several emphasized the usability aspect of the application. Influence and knowledge production were regarded as semi-important, mostly categorized as could-have features.
Participants ranked deliberation and mobilization as the least important features. Deliberation within the app was not regarded as a key function. Some participants argued that this would take away the focus from the main aim of the app (following local politics) and feared that some discussions would easily derail. Participants thought that contacting politicians directly would be better than discussing issues in the app. Likewise, participants thought the mobilization of groups regarding political issues could be done more effectively elsewhere.
In the focus group discussions, participants were asked to brainstorm with other citizens concerning the features they see a need for in the application. Several of the democratic criteria in the framework were present in the wishes and needs of the citizens: inclusiveness, publicity, deliberation, and knowledge production. Using focus groups was beneficial in identifying challenges for the different user groups. For example, the elderly discussed that eyesight and access to smartphones could pose a challenge for the inclusion of older people, and young people were afraid that using strict registration procedures would exclude the young. Taking these aspects into consideration when designing a feature will increase the likelihood of creating an inclusive application.
The wish for receiving information about municipal activity and decisions relates to the criterion of publicity (i.e., easy access to government processes/policies). In fact, when being asked to rank the criteria, participants in the laboratory study regarded publicity as the most important feature. This feature is arguably the most common goal, and easiest feature to design in civic applications (Desouza & Bhagwatwar, 2012). Nevertheless, this finding seems to support the notion that governmental information is difficult to access (Kosec & Wantchekon, 2020).
Some of the criteria mentioned in the focus group discussions were not ranked as “must-haves” in the interviews following the usability test. Users both understood some of the challenges with particular features and commented that not all features were equally important. Voicing one’s opinion or being able to comment on data/decisions/news provided by the municipality can be translated into the democratic criterion of deliberation (i.e., civic apps should enable dialogue between users and authorities). Clearly, citizens desired two-way communication. In contrast, public officials seemed reluctant towards the idea of establishing and maintaining online discussions with citizens. However, in the ranking of the six criteria, deliberation was ranked as one of the least important. This finding might have to do with citizens understanding the demanding task of upholding quality deliberation online and realizing that moderation might be required to do so (Jäske & Ertiö, 2019, p. 2). Similarly, the focus groups asked for a polling/survey feature to be able to provide input to politicians. This request can be characterized as knowledge production (i.e., producing expressions of “collective will” to policymakers) in terms of democratic criteria. Thus, the participants seemed to value the opportunity to produce input to the municipality with the hope of influencing decisions. Still, the representativeness of such “public opinion” was questioned (Jäske & Ertiö, 2019, p. 17). Technically, a polling feature would probably be easy to design, yet the quality of such user-generated data would certainly be debated. Nevertheless, if users of the civic application would be representative of the general population in the municipality, this could be a viable channel for citizen feedback. The problematic issue of skewed public opinion was reflected in the ranking of the criterion knowledge production as mostly a “could-have” feature. In other words, knowledge production is a feasible feature as long as the data is representative. In cases where the municipality would like other types of citizen feedback, for example, reporting of holes in roads, representativeness is not an issue.
Discussion and conclusions
The main aim of this study was to gain knowledge about how citizens’ wishes and needs can be included in the design and evaluation process of a civic application. We stated the following research questions: (1) Which democratic criteria do citizens emphasize in a UCD-process of a civic application? (2) How can a UCD-process help measure if civic applications meet the democratic criteria they aim for? Using focus groups, usability tests and interviews in a laboratory setting, we sought to gain insight into citizens’ opinions on relevant features in a civic application such as Pocket Democracy.
To summarize, the features put forward by the focus groups and the ranking of the democratic criteria by individual participants suggest that the criterion of publicity is the most important, followed by inclusiveness (e.g., Desouza & Bhagwatwar, 2012). In the context of designing a civic application for following local politics on the municipal level, the criteria of influence and knowledge production were seen as “could have” features, and mobilization and deliberation were deemed as not important in this application. We noted that elderly participants were worried about whether young people would use the app and vice versa. Participants did not seem to expect a civic app to have a large influence on decision-making. The apparent lack of need for an app helping people to mobilize citizens into collective action might be due to the variety of existing tools for this cause (e.g., social media: Facebook, Whatsapp).
In answering the second research question, we emphasize how understanding the users’ needs and wishes will facilitate the quest for citizen involvement and avoid unnecessary and frustrating usability problems once apps are released (McDowell & Chinchilla, 2016, p. 471). For instance, participants emphasized that reaching inclusiveness is also about the language used in political processes. The bureaucratic language in municipal documents was not deemed user-friendly. Another key challenge is how to present long documents with a complicated language on a phone or tablet in a user-friendly manner. Moreover, one way of explaining the features of an app is to provide a tutorial when users use it for the first time.
The framework for evaluating civic applications by Jäske and Ertiö (2019) was used as a normative “yardstick” to contrast the wishes of citizens against the visions provided by democratic theorists. The six democratic criteria of interest in this study were: inclusiveness, deliberation, influence, publicity, mobilization, and knowledge production (Jäske & Ertiö, 2019). It is worth noting that the criteria presented in Jäske and Ertiö’s framework are normative and can be used as evaluation criteria rather than setting expectations that every civic application should live up to these highly held standards. Applications might emphasize different criteria, and as Smith (2009) and Gilman and Peixoto (2019, p. 108) note, the promotion of one criterion can conflict with another. For example, promoting deliberation might have a negative effect on inclusiveness, since deliberation can be a demanding form of participation (Friess & Eilders, 2015, p. 322).
However, we argue that only determining whether applications live up to these criteria on a theoretical level is not enough when evaluating civic technology. We identify a lack of focus on citizens/users in the design of democratic innovations and civic technology. If we want citizens to use democratic innovations, these need to match the needs of the end-users. By a) asking what citizens would like civic technology to do, and b) test whether the technology manages to fulfill the needs of citizens, we can increase the chance of successful adoption of civic technology. In other words, we might make it more likely that digital democratic innovations reach their goals: activating the citizenry.
Several challenges remain when implementing democratic criteria in civic technology. Concerning deliberation, to live up to the ideal of deliberation, civic applications should allow discussion and dialogue among users, between users and authorities, and ensure deliberative quality through in-app moderation. Many civic apps are not focused on deliberation, partly because deliberation is demanding and might reinforce existing inequalities in political participation (e.g., undermining inclusiveness) (Gilman & Peixoto, 2019, p. 112).
Moreover, the possibility of actual influence is bound by the linkage between the civic application and actual decision-making bodies. A caveat of democratic innovations is that this linkage is often weak or non-existent (May & Ross, 2018; Spada & Ryan, 2017). Internet-based tools have reduced the costs of communication and mobilization of citizens into political groups and lowered the bar for political donations. Therefore, groups can more easily than before mobilize their supporters into collective action (e.g., demonstrations) using social media (Conroy et al., 2012). Civic online applications can help in mobilizing people, which in turn might strengthen offline political participation for the benefit of democracy. Nevertheless, the digital divide (Norris, 2001) can hinder inclusiveness, as some groups are more likely to be practically excluded due to, for example, not owning nor knowing how to use a smartphone. Thus, although civic applications can mobilize politically passive groups of citizens (e.g., younger people) it might simultaneously marginalize others (e.g., senior citizens). However, civic technology such as internet voting and e-petitions can lower the threshold for marginalized groups to enter the democratic process (Smith, 2019a, p. 108). This discussion also relates to how the usability of an app is connected to the implementation context. The UCD process emphasizes the identification of user scenarios: in what type of situations will the user utilize the tool and what tasks are necessary to reach the goals of the user? These questions need to be addressed to understand the context of app use.
One recommendation for designing digital democratic innovations and measuring their impact is that no application should try to fulfill all the six democratic criteria identified in previous research (Jäske & Ertiö, 2019). The usability of a product is better if an application has fewer functions that work well rather than many functions that do not work well. Hence, when measuring the effects of applications, researchers must consider the specified aims of the applications, what is the application supposed to do? Moreover, having a function present in an application does not automatically mean it is user-friendly. Instead, if the users cannot use the features correctly, the overall experience of the application could disturb its intended effects. Therefore, we would like to emphasize the need of involving users in the design process of digital democratic innovations. Previous studies (e.g., Gilman & Peixoto, 2019; Smith, 2019b), focus more on the will from political leaders, legal framework, or political culture, but forget to acknowledge that if citizens do not acknowledge a need for a new tool, they will not use it. Understanding the need and wishes of citizens, as well as designing well-functioning and user-friendly products will improve the impact of civic technology. Apart from the six criteria in Jäske and Ertiö’s framework, usability could be a criterion in itself. Even if a civic application provides features that technically promote inclusiveness, deliberation, influence, publicity, mobilization, and knowledge production, only a small group of people will use it if it is too complicated. Moreover, the analysis of the (causal) mechanisms that can be included in the design to improve the app’s use and the users’ satisfaction is of relevance for future research (Schmitt, 2020).
To our knowledge, this study is a first, exploratory attempt to combine a user-centered design process with the theory of digital democratic innovations. In usability studies, a low n is sufficient to find the most common usability problems (Macefield, 2009), but of course, to apply the results to a more general context, more tests and participants need to be included. Future research should include usability studies at later stages of the process, for instance, concerning the use of digital democratic innovations in real-life scenarios rather than in laboratory settings.
Footnotes
Acknowledgments
This work was supported by the Högskolestiftelsen Foundation.