Rapid,Efficient Crowdsourcing Using Social Media for the Surgical Management of Nephrolithiasis

Introduction

In the surgical management of nephrolithiasis, multiple potential approaches are often feasible, clinically appropriate, and consistent with the evidence-based clinical guideline recommendations of multiple professional societies, including the American Urological Association/Endourological Society, the European Association of Urology, and the National Institute for Health and Care Excellence. ^{1 –4} Choosing among these surgical options for a particular patient may depend on a surgeon's experience and stone- and patient-related factors. For more complex stones, patients with atypical anatomy, or other situations where published guidelines are not applicable, surgeons may rely on the advice and experiences of their colleagues to develop and seek feedback on their surgical plans.

Crowdsourcing has emerged as an efficient and effective way of rapidly aggregating feedback about a specific question. In endourology, online crowdsourcing has been extensively studied for technical performance assessment ⁵ by having a “crowd” of reviewers rate individual surgeons' skills against standardized metrics. Notably, the increasing use of social media in urology in general ⁶ and endourology in particular ⁷ has enabled new platforms for engaging an international audience of skilled experts for feedback. The social media platform Twitter, for instance, has been used by urologists to crowdsource a broad range of advice, including surgical planning, solutions to clinical problems, and feedback about proposed actions. ⁸

The breadth of stone surgery is a high-yield focus for online crowdsourcing. The use, extent, and potential applications of this technology to assist endourologists in clinical decision making, however, are unclear. We aimed to characterize the patterns and contexts of social media crowdsourcing related to the surgical management of urinary stone disease.

Methods

Twitter was queried with 22 stone-related keywords (Supplementary Table S1) during a 60-day study period to identify all English-language, publicly accessible posts relating to the surgical management of nephrolithiasis. Crowdsourcing posts were defined as those requesting clinical advice, input, or response. ⁸ The solicitation of response, such as directly stating a question or requesting feedback, was a required element to be considered crowdsourcing. We defined “original posts” (OPs) as those originally authored by the user requesting feedback. The responses to OPs, or “reply posts,” were defined as those linked on Twitter to advice-seeking OPs. When a crowdsourcing OP was identified, any linked reply posts were tracked and included in the analysis. Similarly, when a reply post was identified, the associated crowdsourcing OP was included, along with any other replies to that OP. Posts using the Twitter poll function, in which respondents choose from a predefined number of responses instead of generating their own, were excluded to capture the broadest set of crowdsourcing responses.

For users whose posts were included in the analysis, their public Twitter profiles were reviewed to determine the user's country and Twitter “demographics” (e.g., number of Twitter accounts followed by the user and following the user). Two independent reviewers conducted the content analysis, which included the nature of the OPs, whether the post pertained to a specific patient, whether and how quickly responses to the OP were received, and whether patient data or imaging were included. Differences were resolved by consensus.

Institutional review board approval was not required for this analysis of public data.

Results

A total of 1209 public posts were identified based on the keyword query. Among these, 672 posts related to surgical stone management were reviewed, and 399 crowdsourcing posts were identified and included in the analysis. The final sample consisted of 29 crowdsourcing OPs linked to 370 reply posts. The 29 OPs were authored by 27 urologists, 2 of whom each posted 2 OPs. Characteristics of the 27 OP authors are summarized in Table 1. All were men, and most (25 users, 93%) were outside the United States. Posts originated from 12 unique countries. The median number of Twitter followers of each user was 525 (interquartile range [IQR] 214–1016.5), while the median number of Twitter accounts followed by each user was 360 (IQR 250–683.5).

Among the 29 crowdsourcing threads (i.e., a single OP and all of its associated reply posts), a total of 28 threads (97%) contained at least one reply to the OP, and the majority (16 threads, 55%) had >10 replies. In the 28 threads that contained at least one reply, the OP author frequently posted a follow-up reply (18 threads, 64%) to offer clarification, respond to a reply, or pose another question. Responses to OPs were typically rapid; among the 28 OPs with replies, the time to the first reply was <24 hours in all threads. Crowdsourcing threads also concluded quickly; among the 28 threads with replies, the median time to the final reply was 2 days (IQR 1–3). Finally, crowdsourcing discussions typically involved international crowd participants; in each of the 28 threads that received at least one reply author was outside the home country of the OP author. Table 2 summarizes the crowdsourcing posts.

In the content analysis of the 399 crowdsourcing posts, 4 categories of reasons for crowdsourcing emerged: developing a surgical plan for a patient-specific stone or condition (206 posts, 52%); requesting colleagues' experiences with a device, technique (e.g., laser settings for dusting lithotripsy), or approach (e.g., prone vs supine percutaneous nephrolithotomy; 86 posts, 22%); requesting advice or opinions about general concerns (57 posts, 14%); and soliciting ideas or solutions to a clinical dilemma or complication (50 posts, 12%). Representative examples of each category are provided in Supplementary Table S2.

Among the 12 crowdsourcing requests regarding a patient-specific surgical plan, 3 OP authors confirmed their decisions with the crowd. In one case, the crowdsourcing discussion had affirmed the author's original plan to treat an asymptomatic, nonobstructing stone in a solitary kidney. In the other two cases, the authors changed their minds based on the crowdsourcing replies and decided to proceed with surgery using different approaches. One OP author who had planned a laparoscopic ureterolithotomy decided to perform ureteroscopic lithotripsy instead. In the second case, the OP author was considering a percutaneous approach to multiple stones in an anticoagulated patient, but ultimately agreed with the crowd to perform staged ureteroscopy.

An interesting use of crowdsourcing was for quality improvement. Two OPs detailed operative complications that had occurred, and requested feedback about how others would manage the complication or change their technique to avoid it. In both threads, the OP authors received numerous replies (mean 23 replies and 12 replying participants) and responded to requests for clarification with additional details about the cases. Notably, the breadth of both discussions also expanded to include topics beyond the OP, such as preoperative patient counseling, postoperative communication, and systems-based factors that would impact intraoperative decision making.

Finally, the majority of crowdsourcing requests (17 posts, 59%) were initiated about a specific patient or case (Table 2). Most of these patient-specific OPs (15 posts, 88%) contained operative photographs, radiographs, or videos directly related to the crowdsourcing request, but none of these posts mentioned having obtained the patient's consent to share the media or discuss the case online.

Discussion

We examined urologists' use of social media for online crowdsourcing about the surgical management of stone disease. Among 29 crowdsourcing requests linked to 370 replies on Twitter, urologists used crowdsourcing to answer a diverse range of general and patient-specific questions. Crowdsourcing requests were answered rapidly, involved bidirectional exchanges between the authors of the original and reply posts, and engaged an international crowd of participants (Table 2). Of note, while many authors posted clinical details relevant to their patient-specific queries, none stated whether patient consent had been obtained to do so.

This is the first study of social media crowdsourcing for clinical feedback in the endourological literature. Online crowdsourcing has been proposed to provide feedback to individual surgeons about their operative performance. ⁵ The advantages of using decentralized observers to aggregate independent feedback include obtaining advice from a broad audience, reducing reviewer bias, ⁹ and receiving feedback rapidly when using an online platform. ^10,11 In addition to performance assessment, crowdsourcing has been employed to facilitate quality improvement initiatives. For instance, when videos of different surgeons performing robotic prostatectomy were assessed using scoring rubrics by expert surgeons and untrained “crowdworkers,” the scores were strongly correlated between the surgeons and crowdworkers, and both groups identified the same lowest performing surgeons. ¹² The same techniques are now being applied to endoscopy, such as the evaluation of ureteral injury after ureteroscopy, with replication of crowd-based efficiency. ¹³

In contrast to these retrospective performance assessments, this study extends the scope of prior work by characterizing urologists' crowdsourcing requests and replies before surgery, either to improve the certainty of surgical decision making or to solicit feedback about clinical dilemmas. Our findings suggest that, even when carried out on an international scale, online crowdsourcing retains the advantages of being efficiently conducted, rapidly concluded, and targeted to individual urologists' concerns (Table 2). Social media crowdsourcing also appears to be actionable; OP authors sometimes changed their minds on the basis of crowdsourcing and selected the surgical approach recommended by the crowd. This suggests that, rather than only seeking affirmation, urologists who engaged the crowd were willing to respond to compelling feedback, supporting implications from the surgical assessment literature that crowdsourced advice has the potential to change surgeon behavior. ¹⁴ As the literature on crowdsourcing expands, the potential to leverage social media as a strategy to improve clinical care merits attention. Concerted approaches by professional societies to incorporate these concepts, such as the 2016 Scientific Statement on social media crowdsourcing by the American Heart Association, ¹⁵ may help accelerate novel use and validation of these digital strategies.

This study has several limitations. First, the findings may not represent all potential patterns of crowdsourcing among stone surgeons. The focus of this exploratory study was to characterize the use of Twitter for crowdsourcing surgical stone management, but our search strategy and study period may not have comprehensively captured the feedback that may be solicited on social media. Similarly, the findings may not be generalizable to the way crowdsourcing is used in other disciplines within urology, although our data are largely consistent with prior work assessing U.S. urologists' use of Twitter for crowdsourcing. ⁸ We were not able to determine whether and how the authors who received replies to their crowdsourcing requests incorporated these data into their decision making or changed their practice as a result of the feedback, except in the cases we noted where the OP authors subsequently shared their decisions. Examining how urologists prioritize peer feedback they receive through social media, compared with other conventional sources of clinical guidance, merits further study.

Despite these limitations, the study has several important implications. First, optimal crowdsourcing relies on an engaged crowd. For social media crowdsourcing, the crowd consists of the OP author's network, such other users who “follow” the OP author on Twitter. Although the size and scope of the crowd may be augmented through networking tools (e.g., “retweeting” a Twitter post), users who are not connected to a critical mass of other users may not receive as many responses, or receive them as quickly. While nearly all crowdsourcing requests in this study were answered, OP authors were connected to an average of 525 other users in their networks, indicating that the size of the crowd was sufficient to support rapid crowdsourcing. Prior work has suggested, however, that novice users of Twitter do not necessarily need networks of that magnitude to initiate crowdsourcing if the users in the crowd are highly engaged or influential participants. ⁸ Whether a minimum network size is required to achieve responses of sufficient quality for decision making warrants further study.

Second, the potential medicolegal ramifications of online crowdsourcing are unclear. The use of telehealth for “e-consultation” is becoming more widespread in U.S. hospitals, particularly among geographically diverse hospital networks and health systems. Telehealth encounters are governed by structured clinical and legal frameworks that influence consultants' recommendations and reimbursement for services. ^16,17 The landscape for online crowdsourcing, in contrast, is largely in its nascent stages. For instance, if a urologist makes a clinical decision based on replies to a crowdsourcing request, and that decision is implicated in an adverse outcome, it is uncertain whether crowdsourcing participants could be held liable. In a recent malpractice case, the Minnesota Supreme Court held that a patient–physician relationship, such as evaluating the patient in person, is not necessary to sue a physician for malpractice. ¹⁸ Furthermore, without access to complete clinical data, which are generally too cumbersome to provide in an online forum like Twitter, urologists may wish to exercise care in publicly posting attributable clinical opinions. Similarly, while this study has suggested that crowdsourced clinical feedback may be readily obtained, the advice should be considered unvetted expert opinion by the receiving urologists, who may wish to contextualize the advice with other sources of guidance, such as clinical practice guidelines. The medicolegal context surrounding online crowdsourcing for clinical decision making will require further refinement.

Third, the findings suggest that the protection of confidential patient information and clinical data online remains a concern. While none of the posts contained explicitly protected health information, such as name, medical record number, or full-face images, most of the patient-specific crowdsourcing requests did include operative photographs or radiographs. At our institution, posting patient information on social media is generally prohibited, because even anonymized data are potentially identifiable by the posting date or clinical context. Recent analyses of the social media accounts of urology trainees and young urologists have suggested that a small but considerable proportion post patient-specific comments or protected health information on their public profiles. ^19,20 Although local policies may vary among urologists worldwide who participate in crowdsourcing, three international urological societies have issued guidelines for responsible social media use, all of which prioritize the protection of patient privacy and confidentiality. ^{21 –23} Prescriptive guidelines, however, should not preclude urologists from responsibly leveraging the potential of social media crowdsourcing. While low levels of awareness of these guidelines among urologists suggest that greater education is needed, ²⁴ individual users can encourage more widespread adoption by obtaining and acknowledging patients' permission to post, even for academic discussion. This may be done formally, such as through informed consent for an online second opinion, ²⁵ or by stating permission in Twitter posts (Supplementary Fig. S1). These steps can help address low rates of reported consent in online crowdsourcing discussions, and prioritize patients' autonomy and right to privacy.

Finally, the scenarios described in crowdsourcing requests may serve purposes beyond clinical feedback and decision making. For instance, replies that incorporate scientific data or evidence-based guidelines may have an educational benefit for urology trainees preparing for examinations or young urologists seeking to broaden their experience. Surgical device manufacturers could also leverage crowdsourced feedback about their products for quality improvement. Continued study of urologists' social media use will help characterize novel applications of crowdsourced data.

Conclusions

Social media crowdsourcing facilitates a range of general, patient-specific, and quality improvement challenges in surgical stone management. Most crowdsourcing requests involve bidirectional exchanges that are rapidly answered by an international crowd of participants. Urologists who engage in social media crowdsourcing should consider privacy concerns when sharing potentially identifiable patient data; approaches to informed patient consent for online crowdsourcing merit further attention.