Abstract
Information monitoring and keeping abreast of research trends is essential for researchers. However, as the volume of information and the number of tools for staying up to date continue to grow, researchers struggle to continuously monitor and filter scholarly articles. This is particularly true for researchers working in complex multidisciplinary fields like Patient Oriented Research, who need to cast their nets wide to identify relevant high-quality studies. The proposed multiple case study will explore and analyze the use and user perceptions regarding a collaborative research trend monitoring system, called eSRAP. This research will advance knowledge on processes and outcomes of collaborative monitoring of research publications. The findings will be significant to those providing monitoring services, studying collaborative information behaviour, training future researchers and information professionals, as well as to system designers.
Keywords
Introduction
The purpose of our study is to assess the use of a collaborative system used to monitor research publications. Keeping up to date with scientific literature is fundamental across research disciplines and at all stages of the research process, yet it remains a challenge (Pain, 2016; Pontis et al., 2017). The most common barriers are information overload, time constraints, and insufficient evaluation skills (Blummer & Kenton, 2014; Pontis et al., 2017). Services and tools exist to monitor scientific literature (e.g., alerting services), but the growing number of options may further contribute to information overload (Barr, 2006). Many publications describe monitoring tools and services provided by information specialists, but more knowledge is needed on researchers’ awareness and actual use of such tools (McKee et al., 2009). Overall, scholarly literature in Library and Information Science (LIS) recognizes the importance of information monitoring (i.e., passive information behaviour), but most research continues to focus on active searching (Attfield et al., 2010).
Monitoring multidisciplinary fields such as Patient Oriented Research (POR) is especially challenging, as researchers need to scan several fields, for example, setting up alerts in many different databases and journals to identify few potentially relevant publications (Stacey et al., 2010). POR refers to research that either involves researchers, patients, health professionals, and decision-makers as partners (hereinafter referred to as POR stakeholders), or aims to address patient-identified priorities, or has direct impact on people’s health, health services, professional practice, health care system and policy (Canadian Institutes of Health Reasearch, 2017; Kaur & Pluye, 2019). Given that POR is meant to engage different types of stakeholders as research partners, their diverse levels of research expertise and evaluation skills may act as additional barriers in keeping up to date with scientific publications (Bates, 1996; Pontis et al., 2017).
A solution may be in using collaboration to monitor and filter scientific literature (Adams et al., 2005). In research, collaboration is intrinsic and is believed to have the potential to solve complex problems and lead to new knowledge discovery (Hara et al., 2003; Karamuftuoglu, 1998; Shah, 2014; Sonnenwald, 2007). Peers and social connections are known to be valuable information sources and facilitators for keeping up to date, also referred to as the ‘invisible college’ (Al Shboul & Abrizah, 2016; Bates, 2002; Case & Given, 2016; Choo, 2001; Foster, 2004; Meho & Tibbo, 2003; Pontis et al., 2017; Talja, 2002; Wang et al., 2007). However, more knowledge is needed to see if, how, and why collaboration could help overcome the challenges of keeping up to date.
In information seeking, people increasingly look for information collaboratively due to the proliferation of internet access, online information, and networked environments (Francq, 2011). Although there is no consensus on one definition of collaborative information seeking (CIS) (Shah, 2014), at its core it is a process of more than one person looking for information in collaboration with others and having a shared goal (Morris & Teevan, 2010). CIS may be influenced by collaborators’ individual characteristics (e.g., personal beliefs, values, attitudes), their roles and relationships, group size, length of involvement, motivation, location, time, group structure, system support, cost and benefits of participation (Beamish, 2010; Morris & Teevan, 2010).
In general, the majority of empirical studies and conceptual models in LIS focus on individual rather than group or collaborative information behaviour (Foster, 2010; Hyldegård, 2006; Karunakaran et al., 2013; Reddy et al., 2010; Shah, 2014). This focus is shifting, but more knowledge is needed to define CIS, its conceptual field, and how it should be studied (McNeese & Reddy, 2017). Moreover, the factors influencing CIS need to be looked at more closely (Hyldegård, 2006; Shah, 2010). Unanswered questions include: how do participants engage in CIS, how to encourage and facilitate participation, and finally, how to evaluate the impact and innovation resulting from collaboration (Morris & Teevan, 2010). There is a growing interest in CIS, but the understanding of its mechanisms is still limited and does not extend to monitoring behaviour (Shah, 2010).
In 2016, eSRAP, a collaborative research trend monitoring system (i.e., supporting a type of CIS), was developed and implemented by the Quebec SPOR-SUPPORT Unit. Based on a search strategy, scholarly publications automatically populate the system as they become indexed in bibliographic databases (see Fig. 1). System users then share the ongoing tasks of reading and filtering new publications using shared relevance criteria. Potentially, users benefit from a crowdsourced identification and peer-appraisal of research publications and a reduced individual workload (Brown & Allison, 2014).
Workflow of eSRAP, a research trend monitoring system.
This study addresses the knowledge gaps identified in the literature and answers the following research questions: (RQ1) How do users engage with the eSRAP system? (RQ2) What factors influence collaborative research trend monitoring from the perspective of eSRAP users? (RQ3) What are the outcomes of collaborative research trend monitoring from the perspective of eSRAP users? The eSRAP system is innovative and unique but has not been evaluated yet. The study focusing on eSRAP is justified by the researcher’s access to the system and user groups. This work will contribute to knowledge, methods, and practice in LIS, specifically related to information monitoring and collaborative information behaviour.
This research study is situated within human information behaviour research, looking at information seeking (monitoring and filtering), use, and potential outcomes linked to information use (Pettigrew et al., 2000; Wilson, 1997). The specific conceptual framework guiding this work is that of environmental scanning (Choo, 1999) (see Fig. 2).
Conceptual framework of environmental scanning by Chun Wei Choo (1999).
Choo’s framework identifies three types of factors influencing environmental scanning but does not include factors related to collaboration. A systematic mixed studies review with a framework synthesis will be conducted to adapt Choo’s framework to collaborative monitoring (Carroll et al., 2011, 2013; Hong & Pluye, 2018; Pluye et al., 2016).
Overview
Multiple case study overview (adapted from Yin, 2013).
The proposed methodological approach is that of a multiple case study (Yin, 2013) (see Fig. 3). Each interest-based group of eSRAP users (i.e., monitoring the same research topic) will constitute a “case”. Current eSRAP users will be invited to participate in semi-structured interviews. Specifically, critical incident and journey mapping techniques (Flanagan, 1954; Samson et al., 2017; Westbrook et al., 2007) will be used to understand users’ experience, system use, perceived factors and outcomes related to use. Data will be collected and analyzed for each case (i.e., intra-case analysis); subsequently, the cases will be compared to draw cross-case conclusions (i.e., inter-case analysis) (Yin, 2013). The current study combines a pragmatist and a social constructivist worldviews by being “problem centered” and interested in “multiple participant meanings” (Creswell & Plano Clark, 2011, p. 40).
The case study approach is recommended for investigating a contemporary phenomenon in-depth and in a real-life setting, especially when the boundaries between phenomenon and context are unclear (Yin, 2013). It is thus appropriate for the study of information seeking behavior that takes place in a specific context, from which it cannot be separated (Courtright, 2007; Pettigrew et al., 2000). This is particularly true in monitoring behavior, which is highly dependent on the information environment (i.e., context) (Choo, 1998). The multiple case study approach will facilitate developing a more comprehensive and in-depth understanding of collaborative monitoring that is contextualized and transferable. Ethical approval will be obtained from the Institutional Review Board (IRB) of McGill University prior to starting the data collection.
According to Yin (2013), cases should be carefully selected to replicate or contrast data. The present study will employ maximum variation purposeful sampling, selecting heterogeneous information-rich cases for in-depth study of their uniqueness and commonalities (Patton, 2002). Maximum variation sample is appropriate as this study aims to gain an understanding and produce detailed descriptions of each case and shared patterns across cases. Moreover, purposefully seeking heterogeneity allows to overcome the problems related to small samples (Patton, 2002).
Matrix sample of eSRAP user groups
Matrix sample of eSRAP user groups
Each eSRAP group monitoring a topic will constitute a case. Each case will be selected based on the following characteristics: group age (under a year/over a year), group size (small: two members or less/big: more than two members), group membership (voluntary/required) (see Table 1). All eSRAP users within a group, regardless of their role (e.g., researcher, patient partner, clinician, student) will be included in a case and will be invited to participate in the interview. Each one of the selected cases will offer a different situation for contrasting results and theoretical replication (Yin, 2013).
Data will be collected with semi-structured interviews to explore participants’ views of and experiences with the eSRAP system, the perceived factors influencing their experience, and outcomes associated with system use (negative, positive, or lack of). The interview guide will be developed based on the conceptual model and will be tested with the researcher’s supervisors. The researcher will conduct the interviews in person, or on Skype when necessary. The interviews will last between 60 and 90 minutes and will ideally take place at participants’ workplace (if unavailable, at a meeting room in the researcher’s department).
Each interview will be organized in three parts. The first part will include open questions on participants’ experience with collaborative monitoring in general, guided by the conceptual model adapted with the literature review. The second part will involve journey mapping to understand and visualize how eSRAP users interact with the system, identifying their perceived needs and pain points. Participants will be asked open questions about their actions, thoughts and emotions related to system use (Nielsen Norman Group, 2018).
The final part of the interview will be guided by the critical incident technique (Flanagan, 1954). A critical incident will be conceptualized as a recent session of using eSRAP, which can be recalled in sufficient detail and has a sufficiently clear effect (positive or negative) on the participant (Choo, 1999). A critical incident will be operationalized by the following question: “Please try to remember a recent instance of using the eSRAP system, when eSRAP was helpful (or unhelpful). Would you please describe that session to me in as much detail as you can?”. All interviews will be digitally recorded, with a back-up recording, and transcribed by the researcher. In addition, the interviewer will collect personal observation notes from the interviews.
Data analysis
All textual data will be analyzed using inductive and deductive thematic analysis (Fereday & Muir-Cochrane, 2006) with NVivo 12 qualitative data analysis software. The six stages of data coding and identification of themes outlined by Fereday and Muir-Cochrane (2006) will be followed: (1) developing the code manual, (2) testing the reliability of codes, (3) summarizing data and identifying initial themes, (4) applying templates of codes and additional coding, (5) connecting the codes and identifying themes, (6) corroborating and legitimizing coded themes. Data will be coded using existing concepts from the conceptual model (deductive coding), while being open and alert to emerging themes (inductive). Analyzed data (i.e., transcribed individual interviews) will be converged for each case (i.e., intra-case analysis) to produce in-depth case reports.
As a secondary analysis for each case, interview verbatim will be used to develop profiles of typical users (i.e., personas) and their respective journey maps. In other words, for each eSRAP user type, the experience, goals, actions, thoughts and emotions will be mapped chronologically, creating a narrative and a visualization of how users interact with eSRAP.
Following intra-case analysis, case reports, typical user profiles, and journey maps will be compared to identify emergent themes and draw cross-case conclusions (i.e., inter-case analysis) (Yin, 2013). Moreover, the initial conceptual model will be revised to incorporate the themes that emerged from data analysis.
Expected limitations
As with all studies, potential challenges and limitation face the proposed research. First, the generalizability of results may be questioned, as this investigation focuses on one system. There are no similar systems, which makes eSRAP unique and innovative. Moreover, the goal of qualitative research is not to generalize beyond the case (i.e., sample to population), but to understand the case in depth and in its complexity (Creswell, 2007). Generalizations such as case-to-case transferability or analytical generalization to theoretical propositions are more appropriate for case studies and qualitative research in general (Polit & Beck, 2010; Yin, 2013).
The study of outcomes of information faces its own challenges. For example, it is difficult to control for secondary effects acting as confounding variables and outcomes may be intangible and long-term, meaning that there is a time lag for them to appear (Gainor & Bouthillier, 2014; Poll, 2012; Poll & Payne, 2006; Tenopir, 2011). To overcome these challenges, the current study uses a multiple case study approach with maximum variation sampling to purposefully include heterogeneity of cases.
Finally, all qualitative data collection and analysis will be carried out by the researcher, who is familiar with the eSRAP system. She has also been working with researchers for over a decade and is very familiar with the challenges they experience associated with keeping up to date. Therefore, her preconceptions may introduce bias. To limit investigator bias, the researcher will maintain a reflexive research diary, which will be analyzed for potential sources of bias and subjectivity. In addition, member checking will be used to add credibility to the findings (Pickard, 2013).
Significance of expected results
The proposed research will address the gaps identified in the literature and contribute to knowledge, theory, methods, and practice in the field of LIS. More knowledge is needed on CIS and specifically on collaborative monitoring. This research will contribute needed knowledge on the factors and mechanisms influencing collaboration during information seeking, going beyond designing technology, and evaluate the impact and specific processes involved in CIS (e.g., making relevance judgements by a group) (Foster, 2006; Hyldegård, 2006; Reddy et al., 2010; Shah, 2010, 2014). The expected findings will shed light on how to encourage participation in CIS and how to evaluate the impact and innovation resulting from it (Morris & Teevan, 2010). Furthermore, the findings will be used to propose a revised conceptual model, as well as in-depth contextualized understanding of the factors and outcomes linked to collaborative monitoring of research publications, to be studied in future research.
Some CIS studies have been conducted in a laboratory (McNeese & Reddy, 2017; Shah, 2013). However, real-life settings are deemed more appropriate for the study of human information behaviour (Case & Given, 2016; Pickard, 2013). The research presented here will investigate collaborative monitoring and filtering behaviours in a real-life setting.
This research will benefit and guide information professionals who support researchers and scholars and provide current awareness services, thus, contributing to evidence-based library and information practice (Eldredge, 2014). The findings will also be relevant to system developers working on collaborative monitoring systems. For example, in addition to gaining a better understanding of the user experience, journey mapping may help developers to optimize the experience of system users. Finally, collaborative monitoring has the potential to save time and enable multidisciplinary teams of POR stakeholders to keep up to date with research publications, contributing to the advancement of POR, evidence-informed health practices and health systems.
Footnotes
Acknowledgments
The initial version of eSRAP has been sponsored by the Quebec SPOR SUPPORT Unit, the National Research Council of Canada (NRC), and other professional and philanthropic organizations. The first author holds a Doctoral Fellowship Award from the ‘Fonds de recherche du Québec – Société et Culture’ (FRQ-SC) and the third author holds a Senior Research Scholarship from the ‘Fonds de recherche du Québec – Santé’ (FRQ-S).