Social participation in science: Perspectives of Spanish civil society organizations

Abstract

There is a general trend toward more active, broader, and more inclusive participation of different stakeholders in science. Civil society organizations’ inclusion in the scientific process is being promoted. However, there are few attempts to understand the role of civil society organizations in research. This study is based on the analysis of 31 semi-structured interviews with Spanish civil society organization managers and representatives. Our main results regarding the current relationship between civil society organizations and the research system are (a) civil society organizations mainly participate in science within one single research moment and they are unaware of their potential. (b) We identify a lack of resources, mutual knowledge (among civil society organizations and academia), and capabilities as barriers for civil society organizations’ participation. (c) There is a need to strengthen links between civil society organizations and research by training in collaborative methodologies and communication skills, promoting participatory research, increasing mutual understanding about the research system and the third sector, and aligning research and civil society organizations’ objectives.

Keywords

co-responsibility participation and science governance public engagement with science and technology social inclusion

1. Introduction

Over the past 20 years, new science and innovation spaces have been established promoting inclusion of the general public. Debate about the science–society relationship emphasizes the “democratization of science,” which is closely related to the idea of “co-responsibility” on which the “responsible research and innovation” (RRI) movement is based (Stilgoe et al., 2013).

There is a trend toward the more active, inclusive and broader participation of different stakeholders in scientific–technical projects in a wide variety of research and coordination activities throughout the world (Hockfield, 2018; Mejlgaard et al., 2018). For example, the quadruple helix model of innovation recognizes society as one of the actors in the innovation system (Schütz et al., 2019), together with science, policy, and industry. Similarly, “open science” can be understood as “transparent and accessible knowledge that is shared and developed through collaborative networks” (Vicente-Saez and Martinez-Fuentes, 2018: 23). Greater public involvement in science has also been promoted by the European Commission program Horizon 2020 (2014) and many governments, such as those of Australia (Halpin, 2012), Latin American countries (Smith et al., 2014), and India (Chakraborty and Giuffredi, 2020).

Public attitudes toward science and technology seem to have affected science-related areas of public policies, such as global warming, the price of antiviral drugs in developing countries, wireless smart meters, or the trade of genetically modified organisms (Buiatti et al., 2013; Doh and Guay, 2006; Hess and Coley, 2012). During the last decade, some social opinions on scientific–technical issues have shaped decision making regarding scientific policies, for example, in the case of genetically modified food (Berg and Lidskog, 2018). Or the Greta Thunberg and Fridays for Future phenomena, which has prompted unprecedented numbers of youth to join the climate movement worldwide (Fisher, 2019). This influence is important beyond its potential impact on climate policy because it is creating a cohort of active participants in democracy (Fisher, 2019).

These social-inclusion trends hold that the responsibility in science and innovation is shared and that scientists, funders, innovators, and other societal actors have a collective responsibility in scientific knowledge production and governance (Stilgoe et al., 2013). René von Schomberg also suggested that the condition of co-responsibility provides the ground of responsiveness among and between actors in research (Von Schomberg, 2013). Thus, co-responsibility implies explicit commitments and mutual obligations of all actors (Stilgoe et al., 2013) and can only be reached with broader stakeholder engagement in scientific–technical projects (Hockfield, 2018; Mejlgaard et al., 2018).

Society–science relationships encompass any form of active nonscientists’ (individual citizens, nongovernmental organizations (NGOs), groups of patients, etc.) participation in the research process, from setting the research agenda (Rowe and Frewer, 2004) by asking research questions, to collecting data, analyzing results, and/or contributing to decision making (Bonney et al., 2009; Lewenstein, 2004; Lidskog, 2008). Participative practices might be classified along a spectrum from minimal engagement, to total commitment as proposed by Arnsteins (1969). Thus, social participation in science goes from crowdsourced programs for collecting data to co-construction of knowledge and of research questions through deliberative processes (Haklay, 2013; Schrögel and Kolleck, 2018).

Civil society organizations as key stakeholders for public engagement

There have been numerous attempts to stimulate participation in research and to embed it in research governance (European Commission, 2014). Stakeholder inclusion in science improves the feeling of belonging to the research of all members (Calder, 2002). For example, Bernauer and Gampfer reported that popular legitimacy of global climate governance decreases when civil society is excluded and increases when civil society is added (Bernauer and Gampfer, 2013).

Doh and Guay (2006) reported similar results regarding European and United States’ environmental institutions combining the activism of NGOs with their corporate strategies to meet their social responsibilities. According to Frickel et al. (2010), these strategies can highlight the “undone sciences”–worthwhile investigations identified by civil society organizations (CSOs) that are not being carried out. Indeed, the greatest challenge of these initiatives lies in developing collective and integrated knowledge among scientists and the other stakeholders included in research (Cornell et al., 2013; Jacobi et al., 2017).

CSOs can act as platforms that collect social opinions and as speakers of society’s concerns. According to stakeholder theory (Hojnacki, 1999), CSOs can develop power, urgency, and legitimacy. Thus, CSOs are important actors who can realize the promise of participative research and respond to the real world (Smismans, 2008). Active participation of individuals with different forms of knowledge and from different sectors in finding solutions to a problem helps to promote the vision of a shared future (Jacobi et al., 2017), better understanding of scientific–technical processes (Stilgoe et al., 2014) and fulfillment of the reported will of society to acquire more responsibility in decision making (Rogers, 2006).

CSOs are seen as “alliance-brokers between public and policymakers” (Mercer and Green, 2013: 6), implying that these organizations may use their position to influence resolution of public policy issues, including those related to science and technology (Rainey et al., 2017). Due to this ability to participate in agenda setting, if CSOs’ expertise were included at the research project level (European Science Foundation, 2013) it could fulfill CSOs’ traditional dissemination role to the public and societal groups (Mercer and Green, 2013) but also give the sectoral knowledge and oversight that CSOs can provide in research (Ahrweiler et al., 2019; Bernauer and Gampfer, 2013). Moreover, establishing scientific dialogues with scientists and social actors (e.g. representatives of companies, NGOs and politicians) offers the opportunity to understand and jointly analyze global problems and make decisions accordingly (Malyska et al., 2016; Welp et al., 2006).

Despite being central to the co-responsibility approach, the term “civil society organization” is not well defined (Rainey, Wakunuma and Stahl, 2017; Ahrweiler et al., 2019). Terms such as “voluntary,” “third sector,” “charities,” and “non-governmental organizations” have overlapping characteristics and may be regarded as CSOs (Nugroho, 2011).

Furthermore, there is no consensus on the specific roles that CSOs should perform in research (Steen-Johnsen et al., 2011). However, CSOs’ role in research and innovation projects is reported to be more multifaceted (data providers, providers of access to the research field, providers of specific domain expertise, etc.) than is currently assumed (Ahrweiler et al., 2019).

Conversely, some authors have questioned the importance of CSOs in science. For example, Krabbenborg and Mulder (2015) considered CSOs as a beacon for potential risks and that this role of “scientists’ opponents” and “nego-technology developers” may be problematic in open early-stage debates. De Saille (2015) calls attention to power imbalances among CSOs. Similarly, Taminiau (2006) and Elzen et al. (2011) maintained that CSOs are only influential when aligned with other actors such as policymakers, companies, or regulators (Elzen et al., 2011).

Nevertheless, given the potential contribution of CSOs to research, it is necessary to determine the current relationship between science and technology system and these organizations to make recommendations for encouraging their effective participation.

The Spanish third sector at a glance

The third sector can be understood as the set of “organizations which are not part of the traditional private sector nor of the public sector” (Defourny, 2013: 12). In Spain, this mostly includes social cooperatives, charities, mutual societies, associations, foundations, and other labor-oriented enterprises (Monzón and Chaves, 2012). The other two major terms used in Spain to refer to those entities have been “nonprofit organizations” (Corral-Lage et al., 2019) and “social economy” (Chaves-Avila et al., 2016), terms also used by European Union institutions and in some European countries such as Portugal, Greece, Belgium, and France (Monzón and Chaves, 2012). This study focuses on Spain because it is more accessible to the authors but also due to the need for a better understanding of the third-sector reality in this country.

The third sector in Spain comprises 27.962 entities (Gómez Crespo and Cobo, 2019), has 1.3 million volunteers, and employs 644,979 persons (Ruiz et al., 2015). The Spanish third sector is composed of a large variety of organizations grouped in four categories (Ruiz et al., 2015):

First-level entities: associations and foundations and base entities that do not group with others;

Second-level entities: federations and entities that group with other entities (associations and foundations);

Third-level entities: confederations and entities grouping together several federations;

Singular entities: entities at any level that have their own characteristics in terms of legal nature, degree of implementation, historical presence, and recognition and social notoriety.

The Spanish third sector is formed mainly by entities within the first level (92%) such as Asociación Española Contra el Cáncer. Second-level (e.g. Federación Española de Enfermedades Raras) and third-level (e.g. Plataforma del Voluntariado Español) organizations are a minority within the sector (7.7%; Fundación Luis Vives, 2012). Of all these organizations, three are considered “singular entities” (ONCE, Cruz Roja and Caritas) that represent the majority of the third-sector volunteers and employed paid staff (Price Waterhouse Coopers Foundation, 2018; Ruiz et. al 2015). Singular organizations are the oldest (almost 100 years), but these are outlier entities (Fundación Luis Vives, 2012).

Moreover, the Spanish third sector is heterogeneous. Collaboration between organizations is almost nonexistent, although they sometimes build alliances regarding lobbying activities or cooperate in specific areas (Chaves-Avila et al., 2016).

Currently, the policy environment in Spain is highly decentralized and complex; public policies, funds, and services come from national, regional, and local governments (Sánchez-Hernández and Glückler, 2019). The third sector has difficulties building long-term partnerships with these governments (Chaves-Avila et al., 2016). European guidelines, lines of action, and inclusion of society in the scientific process can easily reach large international CSOs, platforms, or networks of organizations (Ahrweiler et al., 2019). However, most Spanish CSOs are small entities that belong to the first level (Fundación Luis Vives, 2012), meaning their level of action is local, they have less staff, and, generally, less purchasing power (Corral-Lage et al., 2019; Ruiz et. al 2015).

A central problem affecting the understanding of CSO participation in research stems from lack of clarity regarding definitions and the nature of their participation (Ahrweiler et al., 2019; Rainey et al., 2017). Due to their potential contribution to science, there is a need to understand CSOs’ current participatory nature, limitations, and expectations to propose effective engagement strategies.

This study therefore seeks to answer the following research questions:

RQ 1. What is the current state of CSO participation throughout the R&D&I process?

RQ 2. What are the barriers or limitations to enhancing such participation?

RQ 3. What kind of relationship do CSOs wish to have with the science and technology system (if any)?

2. Methodology

We conducted 31 semi-structured interviews with CSO managers and delegated representatives based in Spain. Selection criteria were developed considering Spain’s diverse CSOs. We set up an advisory board specifically for this project (details can be found in Supplemental material).

Sampling

Intentional sampling was used to select the interviewees; we selected a sample of strategic subjects based on their position (managers or delegated representatives with decision-making capacity) and the distribution of Spanish third-sector entities (Fundación Luis Vives, 2012), aiming to include all the diverse social points of view as grouped by the following organizations: animalists; environmentalists; patients; parents; consumers; lesbian, gay, bisexual, transgender, and intersexual (LGBTI+), and so on. We wanted to include managers of the three singular entities that represent the majority of the third-sector volunteers in Spain (Ruiz et. al 2015) in our sampling, but one declined. All organizations included are nonprofit.

Once the CSO representatives were identified, we e-mailed describing the project and invited them to be interviewed. Up to three follow-up emails and phone calls were made to solicit participation from those who did not respond. Additional effort was made to identify representatives working with traditionally underrepresented groups; 62 CSO representatives were contacted. Of these, 33 responded, two declined due to lack of time or interest, and interviews were completed with 31 people. In Table 1, we summarize the final sample. More information regarding the sample can be found in Supplemental material.

Table 1.

Sampling description.

Type of CSO	Number of entities
First-level entities	19
Second-level entities	7
Third-level entities	3
Singular entities	2
Total	31
Scope of action	Number of entities
Social	16
Environmentalist	7
Patients	8
Total	31

CSO: civil society organizations.

The average age of the interviewed representatives was 49 years (SD = 14), and there were more male (n = 18) than female (n = 13) interviewees. Most interviewees had completed a higher education degree (n = 26), and most of them had been linked to the CSO for more than 5 years (n = 25). Before participating, all were informed of the nature of the study and the data processing policies, and freely gave consent.

Data collection and processing

The research team developed a semi-structured interview protocol following the guidelines of Silvia Rabionet (2011). Interview questions can be found in Supplemental material. Two interviewers (the first author and a research team member) conducted face-to-face or Skype interviews according to interviewee preference. The first interviewer conducted 22 interviews, the second, 9 interviews. Interviews were conducted between June and September 2017. The average interview took 54 minutes to complete, with the range spanning 46 to 78 minutes.

Data analysis and interpretation

A sequential analysis of the interviews was carried out, and observational notes were included in the interview transcriptions. Inductive qualitative content analysis was used to analyze the data with the support of the research software Atlas.ti (version 7.5). This research method represents the systematic and objective means of describing and quantifying phenomena (Schreier, 2012). To do so, we reduced the data from interviews to codes or concepts that describe the research phenomena by creating categories. The specific categories in the “Results” section have been inductively determined from the data, but our interpretation echoes previous categorizations such as those regarding different levels of public participation in science (Arnstein, 1969; Bonney et al., 2009; Haklay, 2013; Lewenstein, 2004; Lidskog, 2008; Rowe and Frewer, 2004; Schrögel and Kolleck, 2018).

Investigator triangulation and peer debriefing were used to ensure reliability. Two interviewers worked to analyze and code the interview transcripts, first jointly coding a sample of interviews as a means of calibration and then working independently achieving a high degree of reliability. Discrepancies were discussed and resolved between coders and the other two authors.

Peer debriefing was used to ensure the collection of valid information. The first author did the first round of analysis of the code compilation, and the second and third authors reviewed the analysis and added their interpretations. The resulting joint categorization was shared with the advisory board and presented in two internal seminars with the research team. Their input and feedback were also integrated.

3. Results

In Tables 2 to 4, the main categories are described, along with the frequency of interviewees for whom this category was observed at least once. Frequencies are used only as a support for the results descriptions, since with qualitative research the meaning of these concepts (detailed in each subsection) is of more value.

Table 2.

Current participation in R&D&I.

Category	Findings	Frequency
Unspecific collaborations	Types of collaborations with universities, companies or other entities linked to R&D&I (agreements, internships, etc.).	17/31
To be subject of study	The representative, members of the CSO or the CSO itself participate as subjects of research studies (interviews, clinical trials, etc.).	11/31
To collect data	The CSO or its members participate in the process of field work or data collection for an investigation carried out by an external institution.	9/31
To carry out research	Research carried out directly by the CSO.	9/31
To fund research	The CSO funds research related to its area of interest by seeking funds, awarding prizes or subcontracting other institutions to conduct research.	7/31
To advise	The representative, CSO or some of its members have been consulted or are part of groups of people who advise in some way researchers or research projects that are related to the area of interest of the organization.	7/31
To train	The representative, CSO or its members participate in training activities at the university or in the research environment (conferences, master classes, mentoring, courses, etc.).	7/31
To not participate in research	The representative of the CSO considers that the organization does not participate in any way in the research system.	10/31

CSO: civil society organizations.

Table 3.

Limitations to participating in R&D&I.

Category	Findings	Frequency
Lack of resources	Refers to the limited resources of the CSO (financial, personnel, time . . .) as a major constraint for not participating in R&D&I.	11/31
Lack of mutual knowledge	Refers to the ignorance about the activity of the CSO by the research entities or the lack of knowledge by the CSO about the possibility of participating in R&D&I issues and/or the lack of communication between the two actors.	10/31
Lack of capabilities	Refers to the lack of skills (of the representative or members of the CSO) needed to conduct research.	2/31

CSO: civil society organizations.

Table 4.

Ideal relationship between the CSO and the R&D&I system.

Category	Findings	Frequency
To strengthen links	Strengthening the links between CSOs and the R&D&I system by stronger communication, formalized relationships and/or aligned objectives.	11/31
To request more research	The need for more research carried out by agents external to the CSO either at a generic level or in the specific field in which their CSO works.	5/31
To be consulted	The need for CSOs to be consulted on research topics related to their area of interest, either by forming part of a formalized advisory committee or through more informal consultations.	4/31
To be involved in R&D&I	Willingness of the CSO to participate in the R&D&I process by getting involved in the research, whether by carrying out field work, collecting data or as a “guinea pig” or subject of study.	3/31
To develop their own research	Willingness to carry out research activity within the CSO itself, including becoming a research center	1/31

CSO: civil society organizations.

Current participation in R&D&I

We grouped all interviewees’ mentions of different kinds of participation in science into eight categories, as summarized in Table 2.

Unspecific collaborations

In 13 of the 31 interviews, we found references to some type of connection with universities, hospitals, research centers, companies or another entity linked to research. However, these references were nonspecific. For example, there were references to internships for scholars who work or develop research at the CSO itself (e.g. Interview 19) or students doing their PhD (e.g. Interview 5). In this case, a formal link with the university or research center would normally be required, but the interviewee has given no information about it.

Other examples include mentions of being in contact (e.g. Interviews 23, 27, and 29) with different entities or of collaborating (e.g. Interviews 6, 12, and 21), connecting (e.g. Interviews 17 and 29), or participating (e.g. Interview 5) with them. Some interviewees also mentioned some agreements with universities or other research entities (e.g. Interviews 8, 16, 25, and 31), which also usually require a formal connection. However, in none of these mentions is the nature of the CSOs’ participation in research specified:

In fact, right now, at this moment, we are collaborating with the University and the Ministry of Health and Social Services in research; that is, in the preparation of the next state volunteering strategy. (Interview 21)

We are in contact with a research center, we are in contact with the Ministry of Economy and Competitiveness, we are in contact with the Ministry of Agriculture, in its different lines, and also, well, we are in contact with universities. Well, I insist that our approach is to avoid the suffering of these animals as far as possible, where possible, and taking into account who is doing it, this forces us to be in contact with researchers. (Interview 27)

During the interviews, four CSO representatives made some reference to the usefulness of research or innovation in their organization. These references may have been general about research and science (e.g. Interview 7) or about research in the specific field of action of the organization they represent (e.g. Interviews 13 and 14):

From our point of view, science can do much to improve the death of human beings by applying all kinds of technology, care and comfort at the end of life at a chemical level, not only at the orthopedic level . . . (Interview 13)

In an organization of patients closely linked to the research, the manager made specific reference to the inclusion of research in the action plan or strategic plan of the CSO:

Yes, it is included in our strategic plan; therefore, the research is in all the documents: in our annual operative plan, in the web page, in the memories, in the dossiers . . . It is a key aspect of our association. (Interview 15)

To be subject of study

Reference to participation in the science and technology system as a subject of study, for both the CSO itself and the associates or volunteers, was quite common. This is especially true regarding patient associations (e.g. Interviews 7, 12, and 15), some of which even reported specific knowledge of the technical terminology, as we can see in the following quotes:

We have also collaborated on European projects at the research level by volunteering ourselves for post-clinical trials. (Interview 15)

If you need celiac people, we participate in that way. We do not have a laboratory, but we can contribute with other resources that they do not have. (Interview 7)

We found it interesting that one of the interviewed managers indicated that people from the CSO are used as tools during investigations (Interview 5). However, there were managers who referred to their participation in the testing of certain products to know the effect that they have on a specific group of people:

All that is part of a macro project; it is a research project in which the part we collaborate with is the testing and searching for solutions; we manage the incorporation of users with different profiles and then the testing. We verify the effect that each product or technology development has on disabled people. (Interview 2)

There were also several CSO managers who described their participation in the research process as giving information about the institution (e.g. Interviews 14, 18, and 22) or being interviewed by researchers (e.g. Interviews 1 and 10). One of the interviewees commented about allowing researchers to stay in the same place as the residents so they can know their needs, or another type of observational work (Interview 16).

To collect data

All the interviewees who made some reference to their connection with science and technology by referring to field work spoke of collecting data or information (e.g. Interviews 1, 8, 18, and 29), recording interviews (e.g. Interview 19), passing information (e.g. Interviews 11, 22, and 28), or developing the project (e.g. Interview 16). Many of these cases are from environmental organizations, but this type of reference also appeared in relation to other kinds of organizations linked to more social investigations via questionnaires or interviews conducted by the CSO itself for an external entity (generally a university) that demands this information:

We have a type of collaboration in which we collect a lot of important data about partner organizations. Data of all kinds; financial, human resources, activities . . . and we make this information available to researchers of the university so that they can make their analyzes, extract data . . . (Interview 8)

To carry out research

Different types of investigations carried out by CSOs were identified; they are grouped in one category because each instance referred to the direct involvement of these organizations in a complete investigation, that is, from establishing the objectives of the analysis to the publication of results. Another interesting point that emerged from the data is the emphasis made by many of the interviewees that their research is purely social (e.g. Interview 13), even those CSOs who later comment about solid collaboration with research structures such as universities, hospitals or research centers (e.g. Interviews 9, 15, and 24).

The majority of CSO directors who referred to the research carried out by their own organization described research within the scope of their organization or aiming to determine the status of their associates within a particular aspect of their scope:

Right now, we are conducting a study among our associates that seeks to determine the real picture of people with rare diseases: access to diagnosis, access to treatment, knowledge, coordination formulas; that is, social research. (Interview 15)

Other entities, notably third-level entities, have performed their own research for a long period of time; some of them also referred to the fact that the results of this research have not yet been published. This indicates, in part, knowledge of how the research process works:

We have been collecting data for 10 years, and we have volunteers who are analyzing this information so that in the future, we can publish the results. But it is a purely social investigation. (Interview 13)

There are also larger associations with a greater research trajectory, for example, research carried out by the CSO in association with other research entities such universities:

We have been doing research at the academic level here in Spain and at the European level with other entities; this first approach has allowed us to open a line of work that has been financed with public and private funds. (Interview 30)

To fund research

Throughout the interviews, we also saw CSOs referred to as “charities,” that is, organizations set up to provide help and to raise money for those in need. However, seven respondents referred to specific research funding that generally implies either giving money to a research entity (e.g. Interviews 12, 15, and 16), hiring qualified personnel to carry out their own investigation (e.g. Interviews 8 and 17), and even in some cases, specific calls for awards or grants for researchers (e.g. Interviews 2 and 7). We found this type of link with research more among patient associations than other CSOs:

At this foundation, what we do at the moment is look for funds to endow research projects with resources. (Interview 15)

No, the foundation does not give money to science; we hire or give some economic help to a person who has the talent to get into the project. (Interview 17)

We give some research awards to support them economically so that they can continue researching. (Interview 7)

To advise

CSO representatives made reference to either themselves, the CSO, or some of its members who have been consulted or are part of groups who advise researchers or research projects.

Many times, an advisory board was not specifically mentioned, but the interviewees referred to a consortium of people (e.g. Interview 2), to giving their opinion or information (e.g. Interviews 12, 19, and 22), or to giving support (e.g. Interview 7) to research projects or research teams. There were references to a more formalized collaboration that involve the writing of or, at least, collaboration about the drafting of terms of reference by which future research will be governed (e.g. Interviews 8):

Different groups of research on celiac disease in Spain have asked us to support their projects. They give us the research proposal, and we give them an opinion. So, yes, we are hand in hand with the research projects that are being carried out. (Interview 7)

References to this type of collaboration with research are diverse, yet what they have in common is that the views or perceptions of the CSOs are integrated or, at least, taken into account during the research process.

To train

In reference to the question “Do you participate in research?,” seven of the 31 interviewees mentioned their participation in training. This suggests that, for them, training activities focused on researchers (in a university or other entities such enterprises or research centers) are part of their relationship with the science and technology system.

As seen in the following quotes, we found references to formal relationships with universities in which the CSO itself designs a whole course within a degree (e.g. Interviews 9, 12, 13, and 18) or teaches voluntary training (e.g. Interview 21); there were more nonspecific references to masters classes (Interview 15) or conferences (Interview 14):

We are doing training for volunteer trainers in competitions. But we are doing it with a university and with specialized volunteer centers in other countries. (Interview 21)

Group 9 is an association of nine public universities of autonomous communities where there is only one university [. . .] and they proposed for us to design an entire course for which these universities will offer optative and free-elective credits. (Interview 9)

To not participate in research

When we asked specifically about respondents’ collaboration with research, 10 of the 31 CSO managers indicated that they did not participate in any way. After more questions, some mentioned or explained some kind of collaboration or connection with research (e.g. Interviews 5, 6, and 8), generally social research, yet we included them in this category because they shared the feeling of not being involved in the research process:

We, as an organization, are not involved in the research; It’s too pretentious to call what we do “research.” That is, we do policy analysis, budget analysis, but to call it research would be too pretentious, I think. (Interview 8)

No. We have collaborated with other associations to do some kind of project, but not research. (Interview 4)

Limitations to participating in R&D&I

We identified three categories regarding the limitations of CSOs regarding participating in R&D&I, as summarized in Table 3.

Lack of resources

Limited resources, such as financial (e.g. Interviews 5, 8, 13, 20, and 21), personnel (e.g. Interviews 11 and 29), or lack of time (e.g. Interviews 2, 8. 10, 11, 18, and 23), seemed to be a major constraint for not participating in research:

The problem is the limitation of resources, basically, of economic resources. All the economic resources are for human resources . . . availability of time . . . Well, the more resources, the more things we do. (Interview 20)

We are very few people! This limits us a lot. (Interview 11)

Lack of mutual knowledge

We grouped interviewees’ mentions of the following: lack of communication between the CSOs and research entities (e.g. Interviews 1, 22, and 27), lack of knowledge of mechanisms needed to participate in research (Interviews 4, 6, 14, and 28), ignorance about investigations being carried out (e.g. Interview 27), or lack of knowledge about activity of CSOs by research entities (e.g. Interviews 5, 13, and 15). All types of example relate to a lack of awareness that could be improved or reduced with better communication.

For example, some CSO managers claimed that they “are not able to transfer well the potential that CSOs have for research” (Interview 22) or that “research and NGOs are two worlds that live separately and speak different languages” (Interview 27). Others referred to their own ignorance of how to participate in science, or admitted that until now, they have overlooked it:

It is due to ignorance, not knowing where to go or how to offer our participation. (Interview 4)

Our organization is more focused on activism and, perhaps, we have not thought about that possible collaboration with the world of research. (Interview 14)

This conception of two isolated worlds was also found when interviewees talked about the “lack of knowing or lack of interest of trained professionals to do research in this area” (Interview 13). One interviewee even said that they do not participate in research because the sector to which their association is dedicated is “unlikely to be interesting for an investigation” (Interview 5). Notably, one CSO representative blamed a lack of research on the social reluctance associated with the scope of action of their organization (child sexual abuse):

We work under a taboo area [. . .] This means that there are no investigations. The taboo is present even in those places [hospitals, universities] where they should investigate; this is curious, but real. (Interview 1)

Lack of capabilities

Only two interviewees mentioned lack of knowledge or of specific abilities needed to carry out research as a limitation to participation, seen in the following:

The problem is who among all of us is prepared to participate. Because here, we’re not researchers. (Interview 12)

I do not consider myself qualified for research, nor am I of the right age to start all this. (Interview 6)

The fact that only 2 of the 31 interviewees refers to the lack of skills of CSO members to carry out an investigation may reflect an overestimation of their capabilities.

Ideal relationship between the CSO and R&D&I system

We identified five categories referring to the relationship that CSO managers would like to have with the science and technology system (Table 4).

To strengthen links

We identified different needs that should be solved to strengthen the links between CSOs and the R&D&I system. For example, the need to improve communication (e.g. Interviews 7, 9, 11, 23, and 27), to have a fluid relationship (e.g. Interview 5), to strengthen relationships (e.g. Interview 25), or a greater proximity (e.g. Interview 9) between scientists and CSOs were mentioned:

I think there should be a more fluid relationship with these entities, either those that finance, or those that execute research, which would be more logical. If there are people specialized in research, let them do it, but with our demands. And there is no fluid communication right now. (Interview 5)

Some CSO managers mentioned the need to establish formalized relationships (e.g. Interviews 24 and 28) and synergies or institutional relations (e.g. Interview 15) or even the need to reduce or solve the difference of the objectives between CSOs and research entities (Interview 20) or a greater continuity of the research projects (Interview 24). Each need pertains to the will to improve these relationships.

To request more research

From analysis of the interviews, we can say that there is a need for more research carried out by agents who are external to the CSO, such as researchers at universities, hospitals or other research structures. Some interviewees mentioned research as a generic need (e.g. Interview 12), but most requested more research in the specific field of the CSO (e.g. Interviews 1, 5, 13, and 14):

There are practically no neuroscience or neuroimaging studies on what happens in a person’s body when they are dying because it is only studied from the point of view of the absence of health and, therefore, it is a failure. (Interview 13)

To be consulted

Some interviewees mentioned their intention to be part of a formalized advisory board (e.g. Interview 15) or, at least, to be consulted (e.g. Interviews 1, 3, and 12) on research topics related to their area of interest. Moreover, they considered that this could make and effective change (e.g. Interview 1) and that it is a need for the future (e.g. Interview 12):

We want to be part of an advisory committee at the research level that also allows us to start future concrete lines of work within the association to open research projects. (Interview 15)

I think that all the people who are fighting against that can also be an active part of these changes, of that investigation, so that these professional people have a whole series of ideas and proposals and so that we can make an effective change. It is very necessary. (Interview 1)

To be involved in R&D&I

Three interviewees expressed the desire to participate directly in the research by providing data as the subjects of study or through field work. They mentioned being a guinea pig (e.g. Interviews 12), being used (e.g. Interview 26) or contributing with concrete data (e.g. Interview 14) as the relationship that they want to establish with research.

Can we participate in an active way, as a federation, in research? . . . . Other than as being guinea pigs . . . I do not see another possibility. (Interview 12)

I would love for companies to see us as an option, for example, for testing their R&D&I . . . that is, we represent many women who have many problems, from sclerosis to maternity . . . therefore, anyone who developed a product and wanted to use us as a guinea pig, it would be fantastic, and I think that should be the relationship we should have. (Interview 26)

To develop their own research

One of the managers talked about the willingness to carry out research activity within the CSO itself:

I would love to go one step further and become a technology center. Of course, we would continue to be a foundation with the same statutes and activity but take that step further as a technological center, with a stable team of professionals dedicated to the line of research that we would establish. Now, we are working with researchers, managers, coordinators at universities or companies. But they are the ones who develop it; we want to be owners of the research. (Interview 2)

It is important to bear in mind that this reflection was made by a representative of a large CSO, with many affiliates and volunteers and with great economic potential.

4. Discussion

As far as the authors know, this is the first time that an exploratory study of CSO-science relations has been carried out in Spain. This allows us to gain a better understanding of the reality of Spain, a country very active in scientific research and which has recently promoted the inclusion of society in research. It also enables comparisons with investigations in countries with similar societies (Italy, Portugal, Greece, etc.). However, this is a qualitative study with an exploratory intention: our findings cannot be considered as representative of all CSOs in Spain.

Current participation in R&D&I

The results suggest that CSOs are actually participating in science, mainly through unspecific collaborations such as agreements and internships, as the subject of study or in data collection. This also includes advising research or research projects, funding research, participating in training activities and even carrying out their own research activities. However, many of the nonspecific relationships (e.g. research stays or students doing their PhD) mentioned in the interviews require an agreement, or some kind of formal relationship between the CSO and the academy. More research is needed to fully understand CSOs’ participation in science and how this participation is distributed.

Ahrweiler et al. reported that CSOs mainly participate in European research projects “to provide expert knowledge” and that project coordinators seemed to assign to CSOs a more passive role than to other partners (Ahrweiler et al., 2019: 11). The interview analysis suggests that CSOs participate only in a research moment (with funding, as a subject of study, data collection, etc.), but are not involved throughout the process; it seems that they are even less involved in defining objectives. Co-responsibility can only be reached with broader stakeholder engagement in technical and scientific projects (Mejlgaard et al., 2018). Thus, more efforts are needed to promote CSO inclusion in the entire research process to be aligned with effective collaborative strategies.

We consider it interesting to highlight that the CSOs’ participation in training activities was focused on research personnel. The CSO representatives understand this collaboration as a way of participating in research. The sectoral knowledge and oversight that CSOs can provide in research (Bernauer and Gampfer, 2013) could be a key issue in these training sessions. Of course, more research is needed to determine their nature and content.

Conversely, many interviewees stated that they are not participating in research; even those who mentioned some kind of collaboration with research. This last condition is more widespread among CSOs that collaborate with universities in social research (e.g. gathering data, answering questionnaires, being interviewed etc.). It seems that many of the interviewed managers overlook the value of their knowledge and do not consider these collaborations to be important or as participating in research.

Therefore, it is crucial for CSOs to become aware of their own potential for research. Only if both parties are aware of the mutual benefits of working together can we align efforts to build a more socially responsible science through transdisciplinary research: efforts must be made to promote CSO knowledge on science-related processes as well as collaborative methodologies (Värmland County Administrative Board, 2018).

Limitations to participating in R&D&I

The need for funding and the lack of time, training, CSO purposes and communication have been identified as barriers to participating in research among interviewees. These issues should be addressed if CSO participation in science and technology is to become an integral part of scientific practice, as co-responsibility movements currently advocate.

The majority of Spanish CSOs are small local entities, understaffed, with low economic capacity (Ruiz et al., 2015). Thus, lack of personal and economic resources are intrinsic limitations of the third sector in Spain. Moreover, attracting funding from big funding bodies (e.g. the European Commission) often requires adopting their policies, thereby limiting the freedom of CSOs to maneuver (Ahrweiler et al., 2019). It is difficult to think of a solution for reducing this barrier.

Lack of knowledge between the scientific community and CSOs reported in the interviews is a barrier that could more easily be reduced. Although communication, public engagement and collaborative strategies have a fundamental role in the professional life of scientists and significant social implications, most universities in Spain omit these skills in science degrees (Revuelta, 2018). Therefore, greater efforts are needed for better training for future scientists regarding communication skills and how to establish such relationships.

Another possible solution could be to create formal communication channels so that CSOs know how they can participate in research, that such collaborations are wanted and, especially, that they are useful and beneficial for both parties. We believe that this type of effort should be encouraged from government (local and national) and from scientific institutions (universities or research centers).

Ideal relationship with the R&D&I system

CSO representatives referred to formal associations with research entities through agreements to develop research work (undergraduate, masters or doctorate projects) or as part of advisory boards. However, to strengthen links between CSOs and the science system, a lack of communication, the divergence of objectives, research timing and the lack of continuity need to be addressed.

The inclusion of CSO representatives as part of research advisory boards is closely related to the concepts of co-responsibility and the idea of democratization of science (Hockfield, 2018). This formal civil society inclusion in science production implies CSOs empowerment in decision making but at the same time they acquire responsibility; broader stakeholder engagement in scientific-technical projects also relies on the willingness of CSO representatives to promote research within their focus.

Community-based research has increasingly been seen as having the potential to overcome some challenges of more standard research approaches, increasing the rigor and utility of science (Lucero et al., 2018). We believe that a good way to try to strengthen relations between the scientific community and CSOs is to promote this research approach. For example, universities can encourage it in final degree projects and masters’ theses or even in doctoral programs. This kind of collaboration already exists, but could be boosted.

5. Conclusions and future research

Due to the potential contribution of CSOs to science, there is a need to understand their current participatory nature, limitations and expectations to propose effective engagement strategies. In this article we have addressed the Spanish context by a qualitative approach to explore the CSO-science relationship. As such, our conclusions have limitations: further research is needed to fully understand Spanish CSOs’ participation in science and our suggestions should not be taken as definitive but rather as a starting point for discussion.

Our findings suggest that the CSOs analyzed mainly participate in science in a singular research moment. Thus, to promote public participation in science, it could be beneficial for CSOs to acknowledge the importance of their potential sectoral knowledge. Moreover, movements devoted to foster science-society relationships require including society from the beginning of the research to build more socially responsible science through transdisciplinary teams. Specific training programs regarding collaborative methodologies to accomplish such research approaches, both for CSO staff and researchers, could play a role.

Lack of funding, time, training, mutual knowledge and communication have been reported as barriers for CSO participation in research. Training future scientists regarding communication skills, public engagement methodologies and how to establish collaborative relationships more aligned with co-responsibility could help to overcome some of these barriers, as could establishing formal channels for CSO inclusion in research, and promoting among CSOs a better knowledge of the nature of the science and research system.

Our findings suggest that the divergence of objectives and lack of communication, research time and continuity should be addressed to strengthen links between CSOs and the science and technology system. Community-based research approaches could be promoted in universities, for example, to foster CSOs–science relations.

Supplemental Material

Llorente_Revuelta_Carrio_Supplemental_material – Supplemental material for Social participation in science: Perspectives of Spanish civil society organizations

Supplemental material, Llorente_Revuelta_Carrio_Supplemental_material for Social participation in science: Perspectives of Spanish civil society organizations by Carolina Llorente, Gema Revuelta and Mar Carrió in Public Understanding of Science

Footnotes

Acknowledgements

The authors would like to thank to the advisory board formed by Carolina Alonso from AZTI Technallia, Javier Alonso of the Carlos III University of Madrid, Ricardo Chacartegui of the University of Seville, Pilar Fernández of the Technical University of Madrid, Carlos Fernández of the National University of La Rioja, Elena Lázaro of the University of Córdoba, Chitina Moreno of the National Center for Research on Human Evolution, Reme Pérez of the Seneca Foundation, Carmina Puyod of the University of Zaragoza, and Rosana Sanahuja of the Jaume I University for their contributions to the conceptual issues of this research. In addition, the authors would also like to especially thank Maria Paulina Naranjo and Aina Crosas for their help during the field work and interview transcription.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was made possible by a grant from the Fundación Española para la Ciencia y la Tecnología (FECYT) of the Spanish Ministry of Science, Industry and Universities (FCT-16-11278) “Estudio de percepción y participación social en el proceso de la I + D + i: el rol de las organizaciones representantes de la sociedad civil.”

ORCID iDs

Carolina Llorente

Gema Revuelta

Mar Carrió

Supplemental material

Supplemental material for this article is available online.

Author biographies

Carolina Llorente Associate professor and researcher on Science Communication and Responsible Research and Innovation (RRI) at the Department of Experimental and Health Sciences at Pompeu Fabra University (UPF). She is a graduate in Biology from the University of Barcelona, a Master’s Degree in Scientific Communication from the UPF-Barcelona School of Management. Her research focuses on citizen participation in science, the perception of researchers on the public and Responsible Research and Innovation (RRI).

Gema Revuelta PhD in Science Communication, associate professor and researcher at the Department of Experimental and Health Sciences at Pompeu Fabra University (UPF), where she has been teaching science communication since 1998. She is also the director of the Science, Communication and Society Studies Center (CCS-UPF) and the director of the Master’s Degree in Scientific, Medical and Environmental Communication at UPF-Barcelona School of Management. Her lines of research include: Social perception of science, Science and health communication and journalism and Responsible Research and Innovation (RRI).

Mar Carrió PhD in biology, professor and researcher in the Department of Experimental and Health Sciences at Pompeu Fabra University (DCEXS-UPF). Head of the educational research group in health sciences (GRECS). Director of the UPF-UOC Master’s Degree in Secondary Education Teacher Training. Since 2005, she has been dedicated to teaching innovation and research in science education. Her current lines of research focus on problem-based learning methods and inquiry and pedagogical strategies to foster scientific creativity.

References

Ahrweiler

Gilbert

Schrempf

Grimpe

Jirotka

(2019) The role of civil society organisations in European responsible research and innovation. Journal of Responsible Innovation 6(1): 25–49.

Arnstein

(1969) A ladder of citizen participation. Journal of the American Institute of Planners 35(4): 216–224.

Berg

Lidskog

(2018) Deliberative democracy meets democratised science: A deliberative systems approach to global environmental governance. Environmental Politics 27(1): 1–20.

Bernauer

Gampfer

(2013) Effects of civil society involvement on popular legitimacy of global environmental governance. Global Environmental Change 23(2): 439–449.

Bonney

Cooper

Dickinson

Kelling

Phillips

Rosenberg

, et al. (2009) Citizen science: A developing tool for expanding science knowledge and scientific literacy. BioScience 59(11): 977–984.

Buiatti

Christou

Pastore

(2013) The application of GMOs in agriculture and in food production for a better nutrition: Two different scientific points of view. Genes & Nutrition 8(3): 255–270.

Calder

(2002) Forests and hydrological services: Reconciling public and science perceptions. Land Use and Water Resources Research 2(2): 1–12.

Chakraborty

Giuffredi

(2020) Science and technology for the people? On the framing ofinnovation in policy discourses in India and in EU. Journal of Science Communication 2011(2865): 1–9.

Chaves-Avila

Alguacil-Mari

Fajardo-Garcia

Savall-Morera

(2016) National report on third sector barriers in Spain. TSI National Report Series No. 8, Seventh Framework Programme (Grant Agreement 613034). Brussels: European Union.

10.

Cornell

Berkhout

Tuinstra

Tàbara

Jäger

Chabay

, et al. (2013) Opening up knowledge systems for better responses to global environmental change. Environmental Science and Policy 28: 60–70.

11.

Corral-Lage

Maguregui-Urionabarrenechea

Elechiguerra-Arrizabalaga

(2019) Estudio empírico del Tercer Sector en España: hacia una re-conceptualización unificada [Empirical study of the Third Sector in Spain: towards a unified re-conceptualization]. Revista de Contabilidad 22(2): 145–155.

12.

De Saille

(2015) Innovating innovation policy: The emergence of “responsible research and innovation.” Journal of Responsible Innovation 2(2): 152–168.

13.

Defourny

(2013) The third sector. In: Bruni

Zamagni

(eds) Handbook on the Economics of Reciprocity and Social Enterprise. Cheltenham: Edward Elgar Publishing, pp. 400–410.

14.

Doh

Guay

(2006) Corporate social responsibility, public policy, and NGO activism in Europe and the United States: An institutional-stakeholder perspective. Journal of Management Studies 43(1): 47–74.

15.

Elzen

Geels

Leeuwis

van Mierlo

(2011) Normative contestation in transitions “in the making”: Animal welfare concerns and system innovation in pig husbandry. Research Policy 40(2): 263–275.

16.

European Commission (2014) Horizon 2020 “science with and for society” providing advice on potential priorities for research and innovation in the work programme 2016–2017 consultation paper. Available at: http://ec.europa.eu/research/consultations/swafs/background.pdf

17.

European Science Foundation (2013) Science in society: Caring for our futures in turbulent times. Science Policy Briefing No. 50. Strasbourg: European Science Foundation.

18.

Fisher

(2019) The broader importance of #FridaysForFuture. Nature Climate Change 9(6): 430–431.

19.

Frickel

Gibbon

Howard

Kempner

Ottinger

Hess

(2010) Undone science: Charting social movement and civil society challenges to research agenda setting. Science, Technology, & Human Values 35(4): 444–473.

20.

Fundación Luis Vives (2012) Anuario del Tercer Sector de Acción Social en España, Fundación Luis Vives, Madrid [Yearbook of the Third Sector of Social Action in Spain].

21.

Gómez Crespo

Cobo

(2019) Informe de elaboración del Directorio del Tercer Sector de Acción Social [Report on the preparation of the Directory of the Third Sector of Social Action]. Plataforma ONG de Acción Social. Madrid.

22.

Haklay

(2013) Citizen science and volunteered geographic information overview and typology of participation. In: Sui

Elwood

Goodchild

(eds) Crowdsourcing Geographic Knowledge. Dordrecht: Springer, pp. 105–122.

23.

Halpin

(2012) Pressure groups and lobbying. In: Smith

Vromen

Cook

(eds) Contemporary Politics in Australia: Theories, Practices and Issues. Melbourne, VIC, Australia: Cambridge University Press, pp. 177–187.

24.

Hess

Coley

(2012) Wireless smart meters and public acceptance: The environment, limited choices, and precautionary politics. Public Understanding of Science 23(6): 688–702.

25.

Hockfield

(2018) Our science, our society. Science 359(6375): 499–499.

26.

Hojnacki

(1999) The conspicuous corporation: Business, public policy, and representative democracy by Neil J. Mitchell. The Journal of Politics 61(1): 269–271.

27.

Jacobi

Mathez-Stiefel

S-L

Gambon

Rist

Altieri

(2017) Whose knowledge, whose development? Use and role of local and external knowledge in agroforestry projects in Bolivia. Environmental Management 59(3): 464–476.

28.

Krabbenborg

Mulder

HAJ

(2015) Upstream public engagement in nanotechnology: Constraints and opportunities. Science Communication 37: 452–484.

29.

Lewenstein

(2004) What does citizen science accomplish? In: Prepared for meeting on citizen science, Paris, 8 June, pp. 1–8. Ithaca, NY: Cornell University.

30.

Lidskog

(2008) Scientised citizens and democratised science. Re-assessing the expert-lay divide. Journal of Risk Research 11(1): 69–86.

31.

Lucero

Wallerstein

Duran

Alegria

Greene-Moton

Israel

, et al. (2018) Development of a mixed methods investigation of process and outcomes of community-based participatory research. Journal of Mixed Methods Research 12(1): 55–74.

32.

Malyska

Bolla

Twardowski

(2016) The role of public opinion in shaping trajectories of agricultural biotechnology. Trends in Biotechnology 34(7): 530–534.

33.

Mejlgaard

Christensen

Strand

Buljan

Carrió

Giralt

MCI

, et al. (2018) Teaching responsible research and innovation: A phronetic perspective. Science and Engineering Ethics. Epub ahead of print 7 February. DOI: 10.1007/s11948-018-0029-1.

34.

Mercer

Green

(2013) Making civil society work: Contracting, cosmopolitanism and community development in Tanzania. Geoforum 45: 106–115.

35.

Monzón

Chaves

(2012) The Social Economy in the European Union. Brussels: European Social and Economic Committee.

36.

Nugroho

(2011) Opening the black box: The adoption of innovations in the voluntary sector—The case of Indonesian civil society organisations. Research Policy 40(5): 761–777.

37.

Price Waterhouse Coopers Foundation (2018) Radiografía del Tercer Sector Social en España: retos y oportunidades en un entorno cambiante [Radiography of the Third Social Sector in Spain: challenges and opportunities in a changing environment]. Price Waterhous Coopers Fundation. Madrid: Fundación PwC.

38.

Rabionet

(2011) How I learned to design and conduct semi-structured interviews: An ongoing and continuous journey. The Qualitative Report 16(2): 563–566.

39.

Rainey

Wakunuma

Stahl

(2017) Civil society organisations in research: A literature-based typology. VOLUNTAS 28(5): 1988–2010.

40.

Revuelta

(2018) Formación en comunicación en los estudios de grado Análisis en las áreas de ciencias de la salud y la vida, ciencias ambientales y ciencias naturales [Communication training in undergraduate studies Analysis in the areas of health and life sciences, environmental sciences and natural sciences]. In Mediaciones de la Comunicación 13(2): 159–182.

41.

Rogers

(2006) The real river management challenge: Integrating scientists, stakeholders and service agencies. River Research and Applications 22(2): 269–280.

42.

Rowe

Frewer

(2004) Evaluating public-participation exercises: A research agenda. Science, Technology, & Human Values 29(4): 512–556.

43.

Ruiz Vilafranca

Ramírez del Río

Delgado

García

Pérez Yruela

iménez Marín

(2015) El tercer sector de acción social en 2015: Impacto de la crisis [The third sector of social action in 2015: Impact of the crisis]. Madrid: Plataforma de Organizaciones de Acción.

44.

Sánchez-Hernández

Glückler

(2019) Alternative economic practices in Spanish cities: From grassroots movements to urban policies? An institutional perspective. European Planning Studies. Epub ahead of print 17 July. DOI: 10.1080/09654313.2019.1644295.

45.

Schreier

(2012) Qualitative Content Analysis in Practice. Thousand Oaks, CA: SAGE.

46.

Schrögel

Kolleck

(2018) The many faces of participation in science. Science & Technology Studies 32: 77–99.

47.

Schütz

Heidingsfelder

Schraudner

(2019) Co-shaping the future in Quadruple Helix Innovation Systems: Uncovering public preferences toward participatory research and innovation. She Ji: The Journal of Design, Economics, and Innovation 5(2): 128–146.

48.

Smismans

(2008) New modes of governance and the participatory myth. West European Politics 31(5): 874–895.

49.

Smith

Fressoli

Thomas

(2014) Grassroots innovation movements: Challenges and contributions. Journal of Cleaner Production 63: 114–124.

50.

Steen-Johnsen

Eynaud

Wijkström

(2011) On civil society governance: An emergent research field. VOLUNTAS 22(4): 555–565.

51.

Stilgoe

Lock

Wilsdon

(2014) Why should we promote public engagement with science? Public Understanding of Science 23(1): 4–15.

52.

Stilgoe

Owen

Macnaghten

(2013) Developing a framework for responsible innovation. Research Policy 42(9): 1568–1580.

53.

Taminiau

(2006) Beyond known uncertainties: Interventions at the fuel-engine interface. Research Policy 35(2): 247–265.

54.

Värmland County Administrative Board (2018) A Quadruple Helix Guide for Innovations. Värmland County Administrative Board. Sweden.

55.

Vicente-Saez

Martinez-Fuentes

(2018) Open Science now: A systematic literature review for an integrated definition. Journal of Business Research 88: 428–436.

56.

Von Schomberg

(2013) A vision of responsible research, responsible innovation. In: Managing the Responsible Emergence of Science and Innovation in Society. John Wiley. Available at: http://philpapers.org/archive/VONAVO.pdf (accessed 21 September 2016).

57.

Welp

de la Vega-Leinert

Stoll-Kleemann

Jaeger

(2006) Science-based stakeholder dialogues: Theories and tools. Global Environmental Change 16(2): 170–181.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB