Deliberative Dialogues Between Policy Makers and Researchers in Canada and Australia

Abstract

Knowledge translation (KT) and implementation science are growing fields in Canada, Australia, and worldwide. Many audiences are targeted as KT knowledge users—policy makers represent one key knowledge user in the health care field. The need for policy makers to understand research and for researchers to understand policy processes is commonly recognized. There is also increasing interest in health policy that focuses on KT as a framework for understanding the use of evidence and, in particular, describing the influence of research on policy along with concepts of coproduction and user involvement. With relationship building central to successful evidence-informed policy, this article explores deliberative dialogue as a potential approach to enhancing KT. It describes two examples of researcher efforts to cultivate relationships and contacts with policy and decision makers via such dialogues and illustrates the inherent opportunities and challenges of doing so.

Keywords

deliberative dialogue knowledge translation researcher–policy maker relationships

Context of Knowledge Translation (KT) in Canada and Australia

The field of KT and implementation science is growing in Canada, Australia, and worldwide. In Canada, funding agency mandates require researchers to engage with stakeholders, including policy and decision makers, practitioners, and consumers of health services (Goering, Boydell, & Pignatiello, 2008). For example, the Canadian Institutes of Health Research (CIHR) is a worldwide pioneer in promoting and supporting KT initiatives aimed at the meaningful involvement of stakeholder communities in research funding and translation for the purposes of effective and innovative changes in health, whether it be at the policy, practice, or product level (CIHR, 2012). CIHR has a strong KT component including dedicated funding streams and strategic initiatives, training programs, and awards. The CIHR definition of KT is taken up globally, along with the mandate of many federal health care funding bodies to fund KT activities and research.

CHIR (2012) defines KT as “a dynamic and iterative process that includes the synthesis, dissemination, exchange and ethically sound application of knowledge to improve health status, provide more effective health services and products, and strengthen the healthcare system.” (p. 1).

In Australia, KT is at an earlier stage and the structures and mandates present in Canada are not yet implemented. For example, there is no funding or specific provision made for KT strategies that move beyond dissemination of research results. Unlike Canada, there is no requirement from funders for a detailed KT plan as integral to grant proposals. Similar to Canada, however, the translation of research findings with an emphasis on influencing policy is a priority (albeit recently) of two major health research funding bodies: the Australian Research Council (ARC) and the National Health and Medical Research Council (NHMRC). The ARC Act (2001) states a clear aim to deliver analysis and evidence-based advice to government as well as contributing to policy on national research and innovation. Also, contributing to the advancement of research translation is a priority action of the NHMRC Strategic Plan (2013–2015). The NHMRC recognizes that researchers are embedded in positions where they are able to directly influence practice and policy and the training of future practitioners and policy makers and consequently, target calls for research in this area.

KT and Policy Makers

The intricacy of the policy-making process, and the need to engage with stakeholders removed from the research sphere, results in a gap between research and policy within health systems (Moat, Lavis, & Abelson, 2013). As a result, attention has turned to encouraging the effective translation of research knowledge outside of academia to enrich and support decision making. In particular, is a call for stronger links between research and policy to increase the chances of research being included as policies are developed, refined, and implemented, ultimately having an impact on the efficacy of practice (Lawrence, 2006; Mitchell, Pirkis, Hall, & Haas, 2009).

There is also increasing interest in health policy that focuses on KT as a framework for understanding the use of evidence (Vingilis et al., 2003) and, specifically, describing the influence of research on policy alongside concepts of coproduction and user involvement. Evidence-based policy and practice, KT and related concepts have become standards across disciplines (Oliver, Lorenc, & Innvaer, 2014), and there is growing evidence-based policy making in Canada and in Australia (Marston & Watts, 2003a, 2003b).

Deliberative Dialogues

Deliberative dialogues are a topic of recent health policy research (Boyko, Lavis, Abelson, Dobbins, & Carter, 2012; Culyer & Lomas, 2006). For example, The Romanow Commission on the Future of Health Care in Canada uses a deliberative process to involve stakeholders in discussions regarding health reform (Maxwell, Rosell, & Forest, 2003). In Australia, the Australian Health Care Reform Alliance uses a wide-ranging deliberative process to identify health reform priorities (Gregory, Hartz-Karp, & Watson, 2008). Policy dialogues allow research evidence to be considered together with the perspectives, experiences, and tacit knowledge of those involved in, or affected by, future decisions about a high-priority issue (Lavis, Boyko, Oxman, Lewin, & Freitheim, 2009). Increased interest in using policy dialogues include recognition that policy and decision makers benefit from locally contextualized decision support; research evidence is only one element of the decision-making processes of policy makers and other stakeholders; and a variety of stakeholders can add significant value to decision-making processes. Culyer and Lomas (2006) referred to deliberative processes as “a way of not only eliciting, legitimizing and incorporating stakeholder input, but also of usefully combining this with other evidentiary inputs for decision making” (p. 359).

Systematic reviews on evidence-based policy (Oliver, Innvaer, Lorenc, Woodman, & Thomas, 2014; Orton, Lloyd-Williams, Taylor-Robinson, O’Flaherty, & Capewell, 2011) demonstrate the importance of personal relationships and ongoing contact between decision makers and researchers. Published work in this area focuses on enhancing the use of research rather than examining processes of engaging with policy makers (Oliver, Lorenc, & Innvaer, 2014). With relationship building central to successful evidence-informed policy alongside the potential of deliberative dialogue, this article describes two examples of researcher efforts to cultivate relationships and contacts with policy and decision makers via such dialogue. We focus on answering the following question: What are the inherent opportunities and challenges of engaging in deliberative dialogues?

Deliberative Dialogue Exemplars

Our respective teams in Canada and Australia are engaged in KT work; in this case, facilitating researcher–policy dialogues in health and disability policy. Health and disability policy is typically formulated and implemented by bureaucrats based in large cities (Althaus, Bridgman, & Davis, 2007). With this metro-centric view come inherent challenges when policies are developed, refined, and implemented in rural/remote contexts (Dew et al., 2014; Peters, 2006). This urban bias is evident in KT research; thus, KT strategies are needed to bridge the metro–rural and research–policy gaps (Boydell, Stasiulis, Barwick, Greenberg, & Pong, 2008).

Use of research to influence health and disability care delivery and health outcomes is particularly important for research in rural areas (Boydell et al., 2008). Our exemplars focus on access to mental health care in rural communities in the Canadian context and to disability services in the Australian context (Dew et al., 2014). The term disability includes intellectual, physical and sensory impairments, developmental disabilities (e.g., cerebral palsy), and acquired conditions (e.g., brain injury). These exemplars describe a deliberative dialogue as a mode of KT with policy makers to improve the likelihood of research findings influencing government policy.

Our research teams in Canada and Australia draw on the work of Lavis and his colleagues (2009), who developed a tool intended to facilitate the organization of deliberative dialogues between researchers and policy makers based on the best available evidence, as well as their collective experience. Table 1 summarizes the central criteria for organizing such dialogues and the ways in which we applied them in each setting. The examples are provided using the deliberative dialogue criteria.

Table 1.

Deliberative Dialogues.

Canadian example	Features of deliberative dialogues	Australian example
Use of technologies to support child/youth mental health in rural Ontario. Identified as high-priority issue by Centre of Excellence for Child and Youth Mental Health.	1. High-priority issue	Access to therapy services for people with disability living in rural Western New South Wales. Identified as high-priority issue by regional director of government disability services.
Documents sent to all participants 1 week prior to forum: - background information package reuse of technology to deliver child/youth mental health services; - agenda for deliberative dialogues	2. Pre-circulated policy brief	Documents sent to all participants 1 week prior to forum: - project brief - policy-related publication - agenda for deliberative dialogues
One 4-hr forum held to share research evidence and best experience, tackle common myths about technology and service delivery, discuss barriers and enablers, and identify priorities and policy implications.	3. Opportunities to discuss the problem	Two 3-hr forums held to share research evidence, discuss myths about rural access, and identify the most appropriate knowledge translation approaches related to rural and remote access issues.
Careful recruitment and mapping of participants.	4. Fair representation	Careful recruitment and mapping of participants.
Forum participants selectively invited to represent Ontario Ministries of Health, Child and Youth, and Education. All invitees attended.		Forum participants selectively invited to represent the following: Forum 1—Senior policy advisors in Federal and New South Wales government departments with responsibility for disability services. Forum 2—CEOs and senior policy advisors of nongovernment disability organizations.
Professional facilitator with experience in leading guided discussions with health and social care practitioners, policy makers, and researchers. Formal presentation and guided discussion of facilitators/barriers.	5. Facilitator	Visiting Canadian knowledge translation specialist experienced in deliberative dialogue, rural service delivery, and policy work. Formal presentation followed by guided discussion of facilitators/barriers.
Forum was not recorded and notes taken did not identify specific individuals.	6. De-identified comments	Forums were not recorded and notes taken did not identify specific individuals.
Policy-ready paper on using technology to deliver child/youth mental health services in rural Ontario, publication, conference presentations.	7. Outputs produced	Summary of key discussion points, student thesis, peer-reviewed publications, conference presentations.
Policy-ready paper on Centre of Excellence for Child and Youth Mental Health website.	8. Follow-up activities	Summary of the forums posted on project website.

Example of a deliberative dialogue in Canada

Setting

Ontario is Canada’s second-largest province, with 1 million km² and a population of just over 12.8 million (2012). Ontario is the most populated province in the country, with one third of Canadians residing there. Almost 20% of the population resides in rural and remote communities. Children and their families in these areas may face more obstacles obtaining mental health services and supports than urban counterparts (Boydell et al. 2006). Problems of service access often result from specific geographic, economic, and cultural factors typical of rural communities (Pong, 2000).

Identification of a high-priority issue

As one way of addressing the rural challenges in delivering and accessing mental health services, deliberative policy dialogues were used in Ontario to develop a policy-ready paper on the use of technology to deliver child and youth mental health (CYMH) services. The need for this policy-ready paper, specifically evidence on the effectiveness of using technology to deliver mental health services, came from policy makers themselves. The policy-ready paper was identified as a priority for policy development and the Ontario Centre of Excellence for CYMH was engaged to commission this work.

By linking current research and relevant policy implications, the intent was to provide a comprehensive picture of the potential role that technology (existing and emerging) can have in improving CYMH services in Ontario. There are promising implications if technology can be used to provide more accessible mental health services, including reduced wait lists and enhanced service access in underserved communities, reduced barriers related to stigma, improved local resources, and cost-effective and clinically effective service delivery.

Pre-circulated policy brief

To prepare participants for the forum and make sure everyone was “on the same page,” a background information package summarizing best evidence about the use of technology to deliver CYMH services, and a forum agenda, were circulated 1 week prior to the forum. This evidence brief was used as primary input for the deliberative dialogue to facilitate interactions between researchers, policy makers, and stakeholders—defined as administrators in mental health institutions and organizations.

Opportunities to discuss the problem

A face-to-face meeting using a deliberative dialogue approach was convened with policy and decision makers across relevant sectors in Ontario, including the Ministry of Children and Youth Services, Ministry of Education, Ministry of Health and Long-Term Care, and Youth Justice. Stakeholders from the Ontario Centre of Excellence for Child and Youth Mental Health, Centre for Addiction and Mental Health, and Lawson Research Institute were also included given their provincial roles in supporting the CYMH sector. Together, this group discussed the relevance of proven and promising uses of technology in mental health service provision for children and youth to current policy considerations. The consultation process involved several activities that followed from our key questions, which included, “What are the best experiences you have had (or that you know about) using technology?,” “What are the central barriers to using technology in service delivery?,” and “What are the main priorities in the field?” The ensuing activities involved sharing research evidence and best experiences on the use of technology in CYMH service delivery, discussing barriers and enablers to using technology, establishing priorities regarding the use of technology in mental health service delivery, and identifying policy implications.

Fair representation

After careful recruitment and mapping of participants, invitations were sent to 16 people representing Ontario Ministries of Health, Child and Youth, and Education. All respondents confirmed and attended a half-day forum. We recognized the difficulties with respect to policy and decision makers’ ability to meet face-to-face, and offered alternate options to engage stakeholders and ensure an opportunity to contribute. We held teleconference calls and interviews, and gathered electronic feedback from 10 additional policy and decision makers, resulting in a total of 26 individuals.

Facilitator

The deliberative dialogue forum was moderated by a facilitator who had spent decades working with funders and policy makers in the health care field. A facilitator guide was used to ensure that background context was described; brainstorming of issues, concerns, and areas of agreement were covered; and priority areas were surfaced. The facilitator’s role was to ensure that deliberations focused on the issue at hand and that all dialogue participants had a voice in the deliberations, and to challenge constructively any possible misinterpretations of the issue under discussion. It was important for our facilitator to guard against the possibility that perceptions about the relative status of participants (whether based on position in an organization, educational background or other factors) did not privilege some participants in the dialogue over others.

De-identified comments

Forum discussions were not recorded; however, extensive notes were taken by the facilitator to document the discussion. Comments were not attributed to specific participants, allowing for the maintenance of confidentiality.

Outputs produced

A paper on the use of technology in rural mental health service provision for children and youth was produced that incorporated feedback from the forum, information from a scoping review of the current literature, and a scan of the use of technologies in Ontario. The paper provided policy-ready recommendations founded on the best available research evidence and informed by current practice. Recommendations included engaging in provincial policy development, establishing an advisory group to develop guidelines for using technology in CYMH service delivery, planning strategies to ensure knowledge mobilization and access to technology, and working with researchers to build the necessary evidence base. The paper also informed peer-reviewed publications (Dew et al., 2014) and conference presentations.

Follow-up activities

Once the policy paper and executive summary were completed, deliberative dialogue participants were emailed copies and informed that the documents were available on the Centre’s website. Participants were also invited to attend a stakeholder conference where recommendations from the paper were presented in a participatory workshop format.

Example of deliberative dialogue in Australia

Setting

New South Wales (NSW) is Australia’s most populated state, with 7.29 million people spread across a geographical area of 80,642 km² equaling 10% of Australia’s total land area (Australian Bureau of Statistics, 2012b).

Western NSW accounts for 72% of the geographical area of the State but only 9% of the population. The estimated resident population in 2011 was nearly 570,000 (Australian Bureau of Statistics, 2011, 2012a), distributed among large regional towns (population 20,000–40,000), smaller towns (1,000–5,000), and isolated rural communities (less than 1,000). The large land area and dispersed population create significant challenges for providing supports to people with a disability and their caregivers living in rural and remote areas (Dew et al., 2013).

Identification of a high-priority issue

The Wobbly Hub and Double Spokes project emerged from discussions in 2008 between western region senior staff of the then NSW Department of Ageing, Disability and Home Care (ADHC) and researchers at the Faculty of Health Sciences, University of Sydney. Significant problems were identified by the regional staff with recruiting and retaining therapists to work in rural and remote areas, resulting in difficulties for people with a disability and their carers in accessing timely and appropriate therapy services. Funded for 4 years through an NHMRC “Partnerships for Better Health” grant, the project involved clinicians, clients, and carers as primary knowledge-holders about service delivery and use; policy makers who possessed an understanding of policy imperatives and policy development; and academic researchers who brought expertise and techniques to rigorously collect, analyze, and interpret information. The goal of the overall project was to develop, implement, and evaluate evidence-based policies to promote timely and effective therapy service delivery to individuals with disability(s) living in rural and remote communities.

ADHC policy makers were engaged throughout the project via direct consultations, attendance at project meetings, and as recipients of project-related materials. Consultations with policy makers, practitioners, and service users, and analysis of policy documents, identified a mismatch between centrally formulated policy and on-the-ground therapy service provision. In addition, the research team struggled to keep policy makers engaged in the project in large part due to major political and departmental changes that occurred over the 4 years of the project. As the project drew to an end in 2014, and in recognition of the challenges encountered implementing policy in rural areas and in maintaining engagement with policy makers, the project team planned deliberative dialogue forums with policy makers and senior managers. The forums’ aims were to understand how policy makers and managers accessed research to inform policy, and together develop strategies for bridging the research–policy gap related to therapy service provision and access in rural and remote areas.

Pre-circulated policy brief

One week prior to each forum, confirmed participants were sent a project brief and policy-related publication (Dew et al., 2014) along with a forum agenda. These materials were designed to orient participants to the substantive work done by the project team in the area of rural and remote service delivery and policy. Four days prior to each forum, an online pre-forum survey consisting of questions related to participants’ experience of using research was sent to each participant.

The responses to the pre-forum surveys were summarized by one member of the research team and distributed to all other team members prior to each forum. The responses provided valuable information about participants’ current use of research on which to draw throughout the forum discussions.

Opportunities to discuss the problem

Two 3-hr forums were held: the first with government policy makers and the second with senior managers and policy staff in disability NGOs. The forums were structured to build on the results of the Wobbly Hub research project sent to participants prior to the forum. This was done via presentation of myths related to rural service access, which were then discussed in relation to project research results. A variety of KT strategies that team members had used across multiple projects were presented and discussed with participants followed by a collaborative brainstorming session in which participants identified strategies they thought would be most useful in bridging the research–policy gap related to rural and remote access issues.

Fair representation

Thirty-six invitations to participate in the first forum were sent to NSW State and Federal members of parliament and shadow members of parliament who had portfolio responsibility for disability and/or rural and remote issues.

Ministers and Shadow Ministers were asked to nominate as attendees senior bureaucrats within their offices with responsibility for policy making. Eight acceptances were received with six attendees. An additional person was subsequently interviewed individually. Nineteen email invitations for the second forum were sent to chief executive officers of NGOs involved in delivering services to people with disability in rural and remote areas. Ten acceptances were received with nine attendees. In total, 16 people participated in the two forums. It is important to note here the difficulties encountered in recruitment of forum participants; we “pitched” the forum to participants as many had not heard of our research project or, if their portfolio was not in disability, may have struggled to know why they were being invited. Recruitment is clearly a key challenge (and opportunity) for deliberative dialogue with policy and decision makers.

Facilitator

Visiting Canadian scholar and coauthor, Katherine Boydell, facilitated the forums. Katherine drew on her experience in similar policy maker engagement and KT work in Canada (Canadian example). She has research expertise in rural and remote service delivery in Canada and her knowledge and credibility in this area along with her KT and policy expertise was extremely valuable in engaging participants. In addition to Katherine, each forum included a project chief investigator and the project manager, whose attendance facilitated discussion of specific project-related areas.

De-identified comments

The forums were not recorded; however, two additional team members using laptops acted as note takers including an undergraduate student conducting research on the topic. Prior to the forums, the research team developed in Word a note-taking proforma using the headings of the broad areas to be covered in the forums. One note-taker focused on getting as much detail as possible on the content of the discussion, including direct quotes from participants. The other person made notes on the interaction between forum participants, group dynamics, and body language. As in the Canadian example, comments were not attributed to specific participants meaning that confidentiality was maintained. Nonetheless, the inclusion of direct quotes provided rich description to the write-up of the forum outputs.

Outputs produced

At the end of each forum, participants were asked to complete a pen-and-paper post-forum survey to find out which aspects of the forum were most useful to them. A summary of the key discussion points from each forum was produced and circulated to participants. The undergraduate student completed a thesis describing the forum findings. The project team prepared peer-reviewed publications and presented on the forum results at conferences.

Follow-up activities

The summary of the forums was posted on the Wobbly Hub project website.

Reflections on Deliberative Dialogues in Both Countries

Opportunities to Engage Policy Makers Through Deliberative Dialogue

The forums provided policy makers and senior managers with an opportunity to meet with researchers interested in mental health and/or disability research. Key individuals in government and nongovernment agencies related to the content area were represented. They were selected to reflect people with portfolios for decision making across areas relevant to the mental health and disability issues. Our targeted recruitment approach and careful stakeholder mapping meant that we brought together the many parties involved in or affected by future decisions related to our respective policy issues. The topic of enhanced access to care, specifically to rural and remote communities, was identified as a priority for policy makers in both countries; they expressed a need for evidence in this field. The face-to-face forums not only offered an opportunity for researchers and policy makers to communicate but also provided a space for policy makers to connect across different Ministries and organizations. The conversations enabled participants to understand how existing research evidence aligns with beliefs, interests, or political goals of key stakeholders, and therefore where opportunities for synergy can be capitalized upon and tensions can be addressed more openly.

Participants indicated that they valued the dialogues and expressed that they would appreciate participating in future similar sessions. One Canadian participant noted the “helpful activities for brainstorming that allowed for a lot of insight and ideas” and another stated, “I’ve come out with a different perspective . . . it was helpful to see how this process played out and hear firsthand from [other] policy makers.” The Australian forums generated similar comments, such as “I have a better understanding of both areas [research and policy] and how they interact” and “networking with others and participating in a discussion of how research findings can be made more accessible in a timely manner.” The pre-forum surveys in both contexts provided information about the extent to which attendees valued and used research in making policy, which we then used to tailor examples to the interests and skills of attendees. The interactive nature of the forums provided many opportunities for attendees to describe their own research experiences and engage in dialogue about their needs with respect to innovative KT strategies. This work is described in a paper currently under preparation for journal submission.

Challenges to Deliberative Dialogue

The dialogues were not without their challenges, however. In both case exemplars, we found that researchers needed a better understanding of the policy-making process. This process is complex and involves decision making, implementation, and evaluation as well as the rearranging of service. Even though it was not initially an aim of the forums, more time should have been given to gaining an in-depth understanding of how such policy decisions are made. Alternatively, these gaps may have resulted from an inability or reluctance of participants to adequately describe the policy development process. In addition, the organization and administration of the dialogues required a substantial investment of time and effort to plan, execute, and follow up. Contacting desired stakeholders was extremely challenging due to the careful mapping of targeted policy makers and subsequent difficulties locating individual emails and phone numbers, with generic emails or phone numbers for entire organizations and ministries. For future dialogues, organizational structures and resources are required to support such deliberative dialogues.

The lack of time available to access research or translate research into actionable strategies was a major challenge mentioned by all forum attendees. Coupled with a lack of time was the cost of accessing online journal databases and inaccessibility of research findings. Due to a lack of time and access, attendees indicated that they wanted short and concise research summaries with “policy-ready” key messages. Policy makers also indicated that creative strategies as a mode of sharing empirical research findings was attractive to them as they were powerful and emotional and had the capacity to pull at their heartstrings. Attendees commented on the often reactive rather than proactive use of research to validate rather than inform policy—a “knee jerk reaction”—in response to a crisis or development necessitating immediate access to research evidence.

A changing political and hence policy context was identified as contributing to this dilemma. These changes make it problematic for researchers to ensure that research topics and findings are relevant and timely in influencing policy. We also noted the conflict between policy cycles/timelines and those of research. In addition, because of movement within job positions, some participants were relatively new to their respective positions and as such took on the role of listener in the forum to a greater extent.

Implications arising from our deliberative dialogues suggest that we may need to adapt our role as researchers, focusing on establishing ongoing relationships with policy makers, ensuring that our written texts contain main messages that include practical research implications, and more story-based, creative KT strategies. Given the lack of intimate knowledge of the context of the policy-making process, researchers could enhance their knowledge base in this area, leading to more relevant useful research.

Discussion

This article describes the use of deliberative dialogues with policy makers as a system-level KT strategy. Such deliberative dialogues are ideal when applied to the health and disability arenas as they can meet the broad objectives of encouraging debate, improving awareness and understanding of complex issues, stimulating conversations regarding health and disability service needs and priorities, and ultimately increasing the use of research evidence for both researchers and policy makers. The dialogues were similar in that both forums were state- or province-wide, took place within the context of a particular research project, focused on priority issues or brought together diverse policy makers involved in future decisions related to the issue, and involved a facilitator. The dialogues in the two examples differed in that in Canada, the deliberative dialogue was featured at the beginning of the research study and in Australia, at the end of the study.

Our work supports previous research that illustrates the success of such dialogues; for example, Lavis’s (2009) foundational work in the area and his resulting toolkit have been used internationally to inform health decision-making processes. Also, experiences in Western Australia (Gregory et al., 2008) indicate that deliberative engagement processes can be successfully implemented by government and used to guide policy. Bringing together and jointly creating knowledge from the research, policy, and practice world presents an alternative to the idea of the academic expert, which relates to coproduction and cocreation of knowledge (Davies, Nutley, & Walter, 2008). The techniques can be adapted to suit particular contexts and issues, and the skills required to conduct deliberative processes can be fostered. To our knowledge, the only other study describing such dialogues occurred in a setting that differs from ours; as unlike our exemplars, the health topic was not regarded as a high priority (Boyko et al., 2012).

Given feedback from both exemplars, policy makers desire such forums where they can discuss research studies or research results with researchers and other policy makers. Creating regular and formal processes whereby these forums occur allows for building stronger relationships and provides opportunities for policy makers to describe how they want research messages to be delivered and what they consider important within particular research realms. Joint workshops, regular forums, and interactive conferences have also been suggested as strategies (Hyder et al., 2011). Forum participants felt it was critical to involve them early on in the research process. We found that policy makers in both countries regularly follow academic social science research and scholarship, contrary to typical assumptions that they do not access these. Several policy makers suggested that increased commissioned research be considered to meet their needs for evidence-based research. As Australia is further behind Canada in terms of funding and supporting KT generally, Australian funding applications should include the KT activities mentioned above.

With respect to the identified need to continue with these types of dialogues, there is recognition that the time-intensive nature of the activity and the opportunity cost for researchers in terms of academic currency (more valuable to use time toward peer-reviewed publications) may pose a drawback. Researcher perceptions that their efforts to affect policy are not valued by universities or funding agencies are pervasive (Campbell et al., 2009). In a recent survey, academic promotion committees regarded peer-review publications as significantly more important than work with policy makers (Phaneuf, Lomas, McCutcheon, Church, & Wilson, 2007). There is a demonstrated need to continue to advocate for KT activities as counting in academia. Advocacy is required for these activities to be included in consideration of applications for promotion or in assessment of research track record for funding applications. For example, the Netherlands Council for Medical Sciences (2002) developed a methodology and indicators for evaluating societal impacts of applied health research and University of Toronto’s Department of Psychiatry developed a Creative Professional Activities dossier that includes KT and collaborative activities with key knowledge users (Boydell, 2004).

Conceptually, further research is required to explore a variety of KT strategies with policy and decision makers. More rigorous evaluation of the impact of these dialogues would add to our knowledge base about what works and with whom. We currently lack a comprehensive understanding of the effect of deliberative dialogues on actual policy change. The few cases of such dialogues are of limited duration and fail to follow up formally over the long term. Ethnographic longitudinal work is needed to study the policy-making process as it evolves and attend to the lived experience of policy makers across diverse policy areas. Consequently, there would be an opportunity to address in a deeply textured manner the complexity of their work (Oliver, Lorenc, et al., 2014). Furthermore, policy makers in our projects indicated that they would like to have a sense of who is conducting research in particular areas of interest which points to the possibility of creating a central database of researchers by area of expertise. We note that universities often have this type of resource for media.

Conclusion

Better understanding of the nature of researcher–policy relations can assist in improving ways in which scientific evidence and advice is commissioned, constructed, and shared when developing evidence-based policy. There is potential for adoption of shared learning strategies that link researchers and policy makers to allow for research and policy to work together. These strategies include promotion and identification of policy and research priorities, membership of policy makers on research grant review panels, researcher placement in policy settings, research projects that are funded jointly, active participation of policy makers in research projects, and shared collaboration regarding planning for research dissemination (Lawrence, 2006). Whereas the policy-making process is complex, there have been many attempts to provide frameworks that help to understand and describe its complexity. Walt and Gilson’s (1994) policy analysis triangle distills the components into four categories of context, content, process, and actors. Also widely cited is Kingdon’s (1984) policy streams model identifying three independent “streams”: problems, political environments, and policy, and the small window of decision making presented when the streams converge. It may be beneficial for researchers to explore these frameworks and other research on policy making to get a sense of the process and the varying factors that influence policy making in general.

Deliberative policy dialogues allow for diverse lived experience and consequently a diversity of stakeholder “evidence.” They are congruent with the recognition that there is no one evidence base. The three types of knowledge identified by Head (2008) are particularly notable here—scientific research, program management experience (practice), and political judgment. We argue for a need to include in the dialogue people most affected by policy: those with lived experience of the issue under consideration. In our research, this means the inclusion of rural people with mental health problems, those with physical, intellectual, and sensory impairments and their carers. As a result of such dialogues, the entire group has an opportunity to learn from the varied views and experience at the table. This collaboration can uncover unique understandings of an underlying issue, and stimulate insights for potential solutions and key implementation possibilities that can only occur when all involved in or affected by future decisions related to an issue tackle it together.

Footnotes

Acknowledgements

The authors acknowledge the contributions of deliberative dialogue policy and decision-makers who attended the workshop sessions, as well as our deliberative dialogue facilitator.

Authors’ Note

Katherine M. Boydell, Angela Dew, and Michael Hodgins contributed equally to this work.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by funded from the Ontario Centre of Excellence for Child and Youth Mental Health, the TeleLink Mental Health Program at Hospital for Sick Children, an International Collaborative Research Award from University of Sydney, and a ‘Partnership for Better Health’ grant (# 571466) from the Australian National Health and Medical Research Council and the NSW Department of Family and Community Services, Ageing, Disability and Home Care (ADHC), Western Region.

References

Althaus

Bridgman

Davis

(2007). The Australian policy handbook (4th ed.). Sydney, Australia: Allen & Unwin.

Australian Bureau of Statistics. (2011). Census of population and housing, persons and relationships classifications, person’s place of usual residence, 2011, data cube: Table builder. Retrieved from https://www.censusdata.abs.gov.au

Australian Bureau of Statistics. (2012a). 1301.0-Year book Australia, 2012: Geography and climate, geography of Australia. Retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/by%20Subject/1301.0~2012~Main%20Features~Geography%20of%20Australia~12

Australian Bureau of Statistics. (2012b). Population by age and sex, regions of Australia. Retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/mf/3235.0

Boydell

K. M.

(2004). What counts? Towards a new definition of scholarship. Toronto, Ontario, Canada: The Hospital for Sick Children, Population Health Sciences.

Boydell

K. M.

Pong

Volpe

Tilleczek

Wilson

Lemieux

(2006). Family perspectives on pathways to mental health care for children and youth in rural communities. Journal of Rural Health, 22, 182–188.

Boydell

K. M.

Stasiulis

Barwick

Greenberg

Pong

(2008). Challenges of knowledge translation in rural communities: The case of rural children’s mental health. Canadian Journal of Community Mental Health, 27, 49–63.

Boyko

J. A.

Lavis

J. A.

Abelson

Dobbins

Carter

(2012). Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Social Science & Medicine, 75, 1938–1945.

Campbell

D. M.

Redman

Jorm

Cooke

Zwi

A. B.

Rychetnik

(2009). Increasing the use of evidence in health policy: Practice and views of policy makers and researchers. Australia & New Zealand Health Policy, 6, Article 21. doi:10.1186/1743-8462-6-21

10.

Canadian Institutes of Health Research. (2012). Guide to knowledge translation planning at CIHR: Integrated and end-of-grant approaches. Ottawa, Ontario: Author.

11.

Council for Medical Sciences. (2002). The societal impact of applied health research: Towards a quality assessment system. Amsterdam: Royal Netherlands Academy of Arts and Sciences.

12.

Culyer

Lomas

(2006). Deliberative processes and evidence-informed decision making in healthcare: Do they work and how might we know? Evidence & Policy: A Journal of Research, Debate and Practice, 2, 357–371.

13.

Davies

Nutley

Walter

(2008). Why “knowledge transfer” is misconceived for applied social research. Journal of Health Services Research and Policy, 13, 188–190.

14.

Dew

Bulkeley

Veitch

Bundy

Gallego

Lincoln

. . . Griffiths

(2013). Addressing the barriers to accessing therapy services in rural and remote areas. Disability and Rehabilitation, 35, 1564–1570. doi:10.3109/09638288.2012.720346

15.

Dew

Gallego

Bulkeley

Veitch

Brentnall

Lincoln

. . . Griffiths

(2014). Policy development and implementation for disability services in rural New South Wales, Australia. Journal of Policy and Practice in Intellectual Disabilities, 11, 200–209.

16.

Goering

Boydell

K. M.

Pignatiello

(2008). The relevance of qualitative research for clinical programs in psychiatry. Canadian Journal of Psychiatry, 53, 145–151.

17.

Gregory

Hartz-Karp

Watson

(2008). Using deliberative techniques to engage the community in policy development. Australia & New Zealand Health Policy, 5, Article 16.

18.

Head

B. W.

(2008). Three lenses of evidence-based policy. The Australian Journal of Public Health Administration, 67, 1–11.

19.

Hyder

A. A.

Corluka

Winch

P. J.

El-Shinnawy

Ghassany

Malekafzali

. . . Ghaffar

(2011). National policy-makers speak out: Are researchers giving them what they need? Health Policy and Planning, 26, 73–82.

20.

Kingdon

J. W.

(1984). Agendas, alternatives, and public policies. Boston, MA: Little, Brown.

21.

Lavis

J. N.

Boyko

J. A.

Oxman

A. D.

Lewin

Freitheim

(2009). SUPPORT tools for evidence-informed health Policymaking (STP) 14: Organising and using policy dialogues to support evidence-informed policymaking. Health Research Policy and Systems, 7(Suppl. 1), S14.

22.

Lawrence

(2006). Research dissemination: Actively bringing the research and policy worlds together. Evidence & Policy: A Journal of Research, Debate and Practice, 2, 373–384.

23.

Marston

Watts

(2003a). Research and advice giving: A functional view of evidence-informed policy advice in a Canadian Ministry of Health. The Milbank Quarterly, 87, 903–926.

24.

Marston

Watts

(2003b). Tampering with the evidence: A critical appraisal of evidence-based policy making. The Drawing Board: An Australian Review of Public Affairs, 3, 143–163.

25.

Maxwell

Rosell

Forest

P. G.

(2003). Giving citizens a voice in health-care policy in Canada. British Medical Journal, 326, 1031–1033.

26.

Mitchell

Pirkis

Hall

Haas

(2009). Partnerships for knowledge exchange in health services research, policy and practice. Journal of Health Services Research & Policy, 14, 104–111.

27.

Moat

K. A.

Lavis

Abelson

(2013). How contexts and issues influence the use of policy-relevant research syntheses: A critical interpretive synthesis. The Milbank Quarterly, 91, 604–648.

28.

Oliver

Innvaer

Lorenc

Woodman

Thomas

(2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Services Research, 14, Article 2.

29.

Oliver

Lorenc

Innvaer

(2014). New directions in evidence-based policy research: A critical analysis of the literature. Health Research Policy and Systems, 12, Article 34. doi:10.1186/1478-4505-12-34

30.

Orton

Lloyd-Williams

Taylor-Robinson

O’Flaherty

Capewell

(2011). The use of research evidence in public health decision making processes: Systematic review. PLoS ONE, 6, e21704.

31.

Peters

(2006). Towards a wider debate on federal and regional governance: The rural dimension. In Brown

Bellamy

(Eds.), Federalism and regionalism in Australia: New approaches, new institutions? (pp. 57-69) Canberra: Australian National University.

32.

Phaneuf

M.-R.

Lomas

McCutcheon

Church

Wilson

(2007). Square pegs in round holes: The relative importance of traditional and nontraditional scholarship in Canadian universities. Science Communication, 28, 501–518.

33.

Pong

R. W.

(2000). Rural health research in Canada: At the crossroads. Australian Journal of Rural Health, 8, 261–265.

34.

Sutherland

W. J.

Bellingan

Bellingham

J. R.

Blackstock

J. J.

Bloomfield

R. M.

Bravo

(2012). A collaboratively-derived science-policy research agenda. PLoS ONE, 7(3), e31824.

35.

Vingilis

Hartford

Schrecker

Mitchell

Lent

Bishop

(2003). Integrating knowledge generation with knowledge diffusion and utilization: A case study analysis of the consortium for Applied Research and Evaluation in Mental Health. Canadian Journal of Public Health, 94, 468–471.

36.

Walt

Gilson

(1994). Reforming the health sector in developing countries: The central role of policy analysis. Health Policy and Planning, 9, 353–370.