Tracing data for systematic reviews and meta-analyses in the advanced age of digital and social media

What this study added to our knowledge?

Digital/social media platforms can be successfully utilized to obtain unreported data

Introduction

Systematic reviews and meta-analyses provide pooled estimates of outcomes by combining and analyzing data from different studies on a similar research topic. Hence, meta-analysts depend on the data available from previous primary studies. ¹ However, substantial variations in the presentation of information between studies make it a considerable challenge to identify and locate all required data for an accurate meta-analysis. Synthesizing outcomes of under-reported primary studies can pose a serious threat to the validity of outcomes and conclusions of systematic reviews. ² Hence, it is essential to take all possible measures to contact the authors of eligible studies and request additional information on unreported outcome data. However, obtaining such information by contacting corresponding authors via traditional methods of correspondence, such as email/postage has become increasingly challenging. ³ A recent meta-analysis project which attempted to request missing outcome data from corresponding authors reported that only 30% of authors responded when they were contacted through emails. ¹ The poor response rate occasionally results in the exclusion of such studies from meta-analyses, or researchers having to make assumptions and imputations based on other included studies. However, methodologists believe that missing summary data should not be considered as a reason to exclude a study from a systematic review. ⁴

Media are platforms which can be utilized to obtain and share information across the world through the communication. ⁵ Digital media usage including social and interactive media has shown a drastic rise over the past few decades, providing means for social interaction and networking. ⁶ Researchers are encouraged to become active on social media platforms to promote their work and enhance the dissemination of their research. ⁷ We recently conducted a systematic review and meta-analysis to investigate the prevalence and trends of diabetes which included 1.7 million adults. ⁸ During analysis, it was noted that a substantial amount of vital information, including years of the survey, was not reported in the primary studies. Subsequently, we attempted to obtain this unreported information via traditional methods of correspondence, without much success, needing to improvise strategies, extending into social and other media platforms. Although some meta-analysts have previously attempted to gather unreported/missing data from the respective authors using traditional methods such as emails, to our knowledge, this is the first study which reports the usage of social media platforms for missing data acquisition. Our aim here is to highlight these multiple methods, which will be beneficial to many authors around the world who are involved in the conduct of systematic reviews and meta-analyses.

Materials and methods

The current study was based on the systematic literature search which we performed for the systematic review and meta-analysis on the prevalence and trends of diabetes which was conducted using two medical databases (PubMed and SciVerse Scopus). ⁸ The search consisted of studies up to 31 March 2020, and the search limits of the systematic review included language (“English”), subject (“humans”) and age (“adults: age ≥18 years”). The literature search identified a total of 2904 articles. The inclusions/exclusion criteria which were utilized for screening are stated elsewhere. ⁸ The systematic review included data from 69 full-text articles, with data being available from surveys conducted from 1972 onwards.

During the retrieval of data from included studies, two authors (PR and NG) identified and categorized all unreported data. Subsequently, the first step to obtain the missing information included the following traditional methods; (a) contacting corresponding authors via email and/or telephone by SMS (short message service) or call, (b) searching for forward citations of the article to locate others studies which may quote the missing information, (c) review of other publications from the same project or any related publications of the corresponding author and research team and (d) perusal of author’s institutional profile and related websites to locate abstracts, conference proceedings, non-indexed publications or any other published materials related to the survey that are not captured in the above medical databases as journal articles. In the first stage, we used traditional methods to communicate with corresponding authors or any other members of the study team.

In the second stage, we used digital and social media to identify and communicate with corresponding authors or other members of the study team. These included Facebook, ResearchGate, WhatsApp, Viber and LinkedIn. The authors who did not respond to SMS/telephone calls were contacted via WhatsApp and Viber, with current numbers being located from institutional profiles, personal and related websites or in a few instances by a simple google search of the author’s name. Communication was extended to all members of the research team via multiple digital and social media channels (described above) when the initial contact of the corresponding author via traditional methods failed. Data were also searched in the online Global Health Data Exchange (GHDx) database, a comprehensive catalogue of surveys, censuses, vital statistics, and other health-related data created by the Institute for Health Metrics and Evaluation (IHME), University of Washington. ⁹

Results

During data retrieval, it was identified that 41 individual data items were missing (year of the survey: 19, study setting: 2, gender distribution: 11, urban/rural distribution: 6 and prevalence data: 3). Of the 41 missing data items, we were able to identify 36 (87.8%) during data tracing, using both traditional (n = 10, 27.8%) and digital and social media-based (n = 26, 72.2%) methods. The retrieved data were as follows; year of survey: 17 (89.5%), study setting: 2 (100%), gender distribution: 8 (72.7%), urban/rural distribution: 6 (100%) and prevalence data: 3 (100%). 10 data items were identified through traditional methods, which included 5 year of the survey, 3 gender distribution, and 1 data item each on urban/rural distribution and prevalence. These 10 data items were identified via the following traditional methods: (a) email and/or telephone by SMS (n = 4), (b) searching of forward citations of the article (n = 3) and (c) review of other publications from the same project (n = 3). The remaining 26 data items were identified via digital and social media-based methods (year of the survey: 12, study setting: 2, gender distribution: 5, urban/rural distribution: 5 and prevalence data: 2). These 26 data items were identified through the following methods, (a) Facebook (n = 6), (b) ResearchGate (n = 3), (c) WhatsApp (n = 3), (d) Viber (n = 1), (e) LinkedIn (n = 1) and Global Health Data Exchange database (n = 12). The details of data retrieved are summarized in Table 1.

OPEN IN VIEWER

Table 1.

Data traced and methods.

Type of information	Number of data items (%)
	Missing/Unreported items	Retrieved	Traditional methods a	Via digital and social media b
Year of survey	19	17 (89.5)	5 (29.4)	12 (70.6)
Study setting	2	2 (100)	0 (0)	2 (100)
Gender distribution	11	8 (72.7)	3 (37.5)	5 (62.5)
Urban/rural distribution	6	6 (100)	1 (16.7)	5 (83.3)
Prevalence data	3	3 (100)	1 (33.3)	2 (66.7)
Total	41	38 (87.8)	10 (27.8)	26 (72.2)

Discussion

The quality of systematic reviews and meta-analyses is significantly determined by the accuracy, quality and completeness of the data reported in the entitled primary studies. Hence, contacting the authors of the respective studies to obtain unreported data or additional information is an important step in the conduct of most systematic reviews. ² The corresponding author holds a unique position in this regard and is generally considered responsible for queries related to the published research. ³ The usual means of contacting authors is via the details provided under the correspondence of the published article. However, as highlighted in the present article obtaining missing/unreported data via such traditional methods can be a challenge. Although we were able to successfully trace more than 85% of the missing data items, only about one-fourth (27.8%) were obtained via such traditional methods of data tracing. This was similar to a previous study finding which reported only a 30% acquisition of missing data through emailing authors. ¹ Additionally, previous studies have also shown that success rates of gathering missing/unreported data through emails/fax/letters are low.^10,11

Poor response rates associated with traditional communication methods could be attributed to multiple reasons. Changes in author affiliations over time and retirement are likely reasons for changes in email addresses and contact numbers, resulting in poor response to communications initiated via such channels. ⁵ We found digital/social media platforms were more successful to obtain unreported data compared to traditional methods. In accordance with previous literature suggesting social media presence promotes conversation and exchange of ideas, ⁷ we found a combination of both traditional and digital/social media methods is likely to yield the best results in tracing missing data for systematic reviews. Therefore, academic social network sites need to be considered as a method to promote information sharing and to promote networking. ¹²

The current study was based on our previous systematic review and meta-analysis which was limited to two medical databases (PubMed and SciVerse Scopus), which were felt to be the most relevant for the purpose of the same. ⁸ However, we did not include a few other social media platforms in the search for missing data (e.g., Twitter). Additionally, the age of the corresponding authors could not be considered in the current study. Future studies should consider including more databases to investigate the methods to obtain unreported data for systematic reviews and meta-analyses.

The current study findings could be beneficial in constructing guidelines for the procedure of contacting corresponding primary studies with suboptimal reporting which could be beneficial in forthcoming systematic reviews and meta-analyses. Journals should strongly consider the inclusion of social media details/links and non-institutional research profiles (e.g., ResearchGate) in future studies for correspondence in addition to the traditional information (e.g., Email addresses/phone numbers), since such profiles of professionals are unlikely to be changed with time compared to institutional profiles. The ORCID (Open Researcher and Contributor ID) a nonproprietary alphanumeric code to unique authors, can be considered one such initiative, that is currently being employed by some major publishers. However, it is important to note that although these methods were successful in tracing study context information (e.g., year of study, setting), it is unclear whether authors would respond via these methods for requests of more complex data required for statistical analysis.

Conclusions

Digital/social media platforms were found to be more successful to obtain unreported data compared to traditional methods. The current study indicated that a combination of both traditional and digital/social media approaches is likely to yield the best results in acquiring missing/unreported data for systematic reviews and meta-analyses. The current study findings need to be considered in developing guidelines on strategies which could be implemented in missing data acquisition as these will enhance the validity of future meta-analyses. Additionally, journals should consider including social media links and non-institutional research profiles in future studies, in addition to traditional methods for correspondence.