Abstract
Background:
The use of text messaging is nearly ubiquitous and represents a promising method of collecting data from diverse populations.
Introduction:
The purpose of this study was to assess the feasibility and acceptability of text message surveys in a clinical setting and to describe key lessons to minimize attrition.
Methods:
We obtained a convenience sample of individuals who entered the waiting room of a low-income, primary care clinic. Participants were asked to answer between 17 and 30 survey questions on a variety of health-related topics, including both open- and closed-ended questions. Descriptive statistics were used to characterize the participants and determine the response rates. Bivariate analyses were used to identify predictors of incomplete surveys.
Results:
Our convenience sample consisted of 461 individuals. Of those who attempted the survey, 80% (370/461) completed it in full. The mean age of respondents was 35.4 years (standard deviation = 12.4). Respondents were predominantly non-Hispanic black (42%) or non-Hispanic white (41%), female (75%), and with at least some college education (70%). Of those who completed the survey, 84% (312/370) reported willingness to do another text message survey. Those with incomplete surveys answered a median of nine questions before stopping. Smartphone users were less likely to leave the survey incomplete compared with non-smartphone users (p = 0.004).
Discussion:
Text-message surveys are a feasible and acceptable method to collect real-time data among low-income, clinic-based populations. Offering participants a setting for immediate survey completion, minimizing survey length, simplifying questions, and allowing “free text” responses for all questions may optimize response rates.
Introduction
Text messaging is widely used across nearly all age and income categories. As of March 2015, 92% of all US adults own a cell phone, and at least 81% of all cell phone owners use their device for texting. 1,2 Thus far, text messaging has been successfully used in healthcare to promote behavioral change and improve disease management (e.g., by encouraging smoking cessation, improving medication compliance, increasing vaccination rates, disseminating accurate health information, and facilitating weight loss). 3 –14
In addition to supporting health-related behavioral change, text messaging is a promising means of eliciting patient perspectives. Patients could potentially use text messaging to “speak” to researchers about their healthcare knowledge, opinions, and beliefs. 15,16 Such a patient-centered data collection method shows promise for several reasons.
Text message surveys have been found to be cost-effective and easy to administer in low-income, community-based settings. 15 In addition, they offer anonymity and the convenience of asynchronous communication (so patients can respond to questions when it is convenient for them). Texting is a familiar and ubiquitous mode of communication, and may be particularly advantageous for reaching vulnerable populations such as minorities and adolescents. 17,18 In fact, a greater proportion of blacks and Hispanics report that they use text messaging compared with whites (85% and 87%, compared with 79%, respectively). 17 Moreover, they offer the advantage of collecting data in real-time, enabling assessment of evolving patient needs and preferences, and facilitating the translation of collected data into timely healthcare and policy change. When compared with Web-based surveys that can be completed through a smartphone (i.e., through mobile apps), text message surveys offer the advantage of not requiring Internet access and thus including people who do not have a smartphone or are conscious of their mobile data usage.
The use of text messaging as a two-way channel of communication between researcher and participant has thus far ranged from simple, numerical data collection (i.e., asking patients to report their blood glucose levels) 19 –21 to more complex clinical interaction (i.e., patients conversing back and forth with pharmacists regarding their medication questions). 22 A few studies 23 –27 have specifically used text message surveys, but largely to collect data before enrollment into clinical trials 23 –26 or via unsolicited “text blasts” to large groups, which has yielded very low (3.6%) response rates. 27 However, text messaging has yet to be used to collect real-time patient opinions and perspectives to improve clinical care and inform policy issues. The aim of this study was to assess the feasibility and acceptability of administering a large-scale text message survey and to describe lessons learned that may inform future text message surveys among clinical populations.
Materials and Methods
Survey Development
The survey instrument included 43 unique questions, which were first piloted using cognitive interviewing techniques among a convenience sample of 7 community members of varying ages and racial backgrounds. 28 The survey used branching logic, so respondents were asked between 17 and 30 questions. The survey began with questions regarding participants' age, race, education level, phone type, and gender. Questions were created that could specifically inform clinical practice or health policy. Survey domains included health insurance coverage, weight-related issues, and pregnancy-related issues. Question types included the following: multiple choice, yes/no, true/false, numerical, or “free text.” “Free text” questions accepted any answer as valid. Respondents could skip any question by typing the word “skip.” Please see Appendix 1 for a complete list of the questions, including branching logic. Responses to survey questions are reported in a separate publication. 29 The study was IRB exempt as the information collected is nonidentifiable [EXEMPTION #2 of the 45 CFR 46.101.(b)].
Recruitment
Research staff recruited participants in the clinic waiting room over the course of 32 days, for an average of 6 h per day. After checking in for their appointments, individuals entering the waiting room were directed toward a table that displayed thank-you tokens (frisbees, cookie-cutters, and tubes of lip balm) and instructional flyers. When a potential participant approached the table, the recruiter explained that a waiting room survey was being conducted, that it takes place on their phone through text messaging, and that if they would like to participate, they can choose any one of the displayed thank-you tokens. The recruiter then offered a flyer and gave simple instructions on how to begin. All adults who passed by the recruitment table were given the opportunity to participate, including friends and family of the patients. Participants were asked to complete the survey at their convenience, but to avoid texting while interacting with their healthcare provider.
Flyer Invitations
Paper flyer invitations included the following text: “We want to provide better care for you and we need your help. Helping out is easy and voluntary! If you are 18 or older, just text the phrase UMATTER to 66746 to complete a one-time, short survey. Your answers are completely anonymous!”
Survey Administration
Survey responses were stored and transmitted by
Outcomes and Analyses
The primary measure of feasibility includes the total number of those who initiated the survey. The primary measure of acceptability was the percentage of those respondents who completed the survey. Other process-related measures included the proportion of participants who responded affirmatively to the question “Would you be interested in doing another text message survey at a different time for a gift card?,” the median number of questions answered by respondents with incomplete surveys, the difference in proportion of respondents that did not complete the full survey based on the type of phone used (analyzed with a Chi-squared test), and the median duration of time spent interfacing with the survey.
Results
Our convenience sample consisted of 461 individuals who attempted the survey. Of those individuals, 80% (370/461) completed it in full. Respondents were predominantly female, non-Hispanic black or non-Hispanic white, and had at least some college education (Table 1).
Demographics of Respondents (N = 461)
The clinic from which we sampled is located in a community where 33% of the population is in poverty. 30 The demographics of the community are comparable with the demographics of our study, with greater representation from females and black community members (Community vs. Clinic Sample): 50% versus 75% female, 29% versus 42% black, 59% versus 41% white, 4% versus 4% Hispanic, 33% versus 39% Bachelor degree or higher.
Of those who completed the survey, 84% (312/370) reported willingness to participate in a hypothetical future texting survey for a gift card (with no monetary amount specified). People who attempted but did not complete the survey (91/461) answered a median of 9 (a mean of 9.3) questions before they stopped responding.
Smartphone users were less likely to leave the survey unfinished compared with flip phone (or “other” type of phone) users (15% vs. 30%, respectively; p = 0.004). In comparing these two populations, there was no significant difference between their age or gender. However, smartphone users were more likely to be non-Hispanic white (p = 0.032) and more educated (p = 0 02). Among those who completed the survey (n = 370), the median time from initiation to completion was 14 min (range: 3 min to 22 h), with 71% completing the survey within 30 min. Among those who attempted but did not complete the survey (n = 91), the median time spent interacting with the survey was 8 min (range: 0 min to 18 h 33 min).
In each of the 91 instances where a user started but did not complete the survey, investigators analyzed the text message exchange between the two parties. The content and duration of time between each message was noted. In 73% (66/91) of these exchanges, no clear reason for leaving the survey could be extrapolated from the text history. In 4% (4/91) of the exchanges, the user submitted a response that contained the word “stop,” as in “she should stop smoking.” By doing so, the user inadvertently opted out of all further survey questions. In 23% (21/91) of the exchanges, frustration with error messages may have led to the abandonment of the survey. One demonstrative excerpt from a real text exchange is presented in Figure 1. Additional examples can be found in Appendices 2–4.

Example of text message exchange that lead to respondent attrition.
Discussion
Consistent with previous community-based samples, 15 our findings demonstrate that both quantitative and qualitative data can be effectively collected via text message from a diverse clinic-based sample. Over 450 respondents were recruited over a 32-day period for only a nominal incentive, which represents a significantly lower financial burden than other forms of survey data collection. 31,32 In addition, the large proportion of respondents that would complete another survey demonstrates the appeal of this method of data collection among our sample.
Despite their ease of implementation, text message surveys also represent unique challenges compared with other forms of survey data collection. Several important lessons emerged from our experiences that may inform the future use of this simple technology for clinical research: (1) Recruit participants in a setting where there is an opportunity to complete the survey immediately. Although respondents could answer the survey at any time, the vast majority of those that completed the survey did so within the first few minutes of being in the waiting room. The waiting room of a medical clinic was ideal in this regard, as patients often had unoccupied “down time.” (2) Be judicious with the number of questions asked, and use branching/skip logic. In comparison to an in-person survey, for example, text message surveys are easier to leave incomplete due to frustration or boredom. Despite the use of skip logic, a common sentiment of those who said that they would not do another (hypothetical) survey was that the survey was too long. Therefore, administering fewer questions per survey may yield a higher completion rate. In addition, to establish users' expectations, survey authors can consider an introductory text message that states the total number of survey questions. Furthermore, eliminating all questions that do not pertain directly to that respondent (i.e., a question that elicits a male's perspective on healthy gestational weight gain) may decrease attrition. (3) Requiring a response from the user after every text message sent may prevent confusion, frustration, and attrition. Because text messages are limited to 140 characters, there were a few instances when prefacing the survey question required sending an additional message (In Figure 1, e.g., an introductory text message prefaces Question 19). Some users, upon attempting to respond to the prefacing text, would get trapped in a cycle of miscommunication that resulted in user frustration. A simple way to avoid this confusion would be to add the sentence, “Text the word ‘Ok’ to continue” at the end of the prefacing text. (4) Allowing “free text” responses for all questions, including multiple choice and true/false questions, may reduce user frustration. When a user texted a response that fell outside the parameters of a valid answer (i.e., texting letters when a question only accepted numbers as valid), the survey responded with: “Sorry, we do not recognize that answer. Please try again.” Users demonstrated evidence of frustration when the survey was unable to recognize a response due to preset conditions that only allowed certain answer types. If every question allowed respondents to answer in a free text format, this frustration is eliminated. Alternatively, the survey should use more informative error messages that specify what type of answer would be accepted (e.g., “Sorry, this question only accepts numerical answers such as ’25.’”)
Limitations
Our study is subject to limitations. Because we used a convenience sample of respondents from one clinic, the demographics and total number of those who elected not to participate are unknown. Our study sample included a large proportion of women and individuals who completed at least some college, which may not be representative of other low-income populations. Furthermore, various cell phone models and service providers have different speeds and performance levels that may have influenced respondents' survey experience, but are not captured in this study. Despite these limitations, text message surveys may provide a unique sense of anonymity or privacy, which may evoke more honest and accurate answers.
Conclusions
Text-message surveys are a feasible and acceptable method to collect real-time data among low-income, clinic-based populations. Furthermore, our findings provide valuable insights into this novel survey modality and demonstrate its broad applicability in collecting patient-centered data to improve the efficacy and impact of clinical interventions and policies.
Footnotes
Acknowledgment
The authors thank the Ypsilanti Health Center Staff for their assistance and cooperation in this study.
Disclosure Statement
No competing financial interests exist.
Appendix 1
An Elderly Hispanic Female Interacts with the Survey Using a Non-Smartphone
| SENDER | CONTENT OF MESSAGE | INVESTIGATOR'S INTERPRETATION |
|---|---|---|
| Survey | This next section is about today's visit. First, what insurance do you have? (a) Medicaid (b) Healthy Michigan Plan (c) Other (d) Unsure | |
| User | b | |
| Survey | When you first got insurance, did you have an option of choosing which clinic to go to? Yes/No | |
| User | Yes | |
| Survey | [Q28] Why did you pick this clinic over others? | |
| User | Close home | No issues thus far. |
| Survey | [Q29] Since being on your current insurance, have you had a preventative visit (check-up, annual exam)? Yes/No | |
| User | Short driving | User does not realize that the survey has asked a new question. She is elaborating on her answer to Question 28. |
| Survey | Sorry, we do not recognize that answer. Please try again. | The survey is looking for a Yes or No answer to Question 29. It does not recognize “Short driving” as a valid answer. |
| User | Somebody recommended | Again, she tries to answer Question 28. |
| Survey | Sorry, we do not recognize that answer. Please try again. | Again, the survey thinks she is trying to answer Question 29. |
| User | Friend tell me about | Same issue as above |
| Survey | Sorry, we do not recognize that answer. Please try again. | Same issue as above. |
| User | What you want from me | User expresses frustration. |
| Survey | Sorry, we do not recognize that answer. Please try again. | User gives up |
A Middle Aged Black Female Interacts with the Survey Using a Non-Smartphone
| SENDER | CONTENT OF MESSAGE | INVESTIGATOR'S INTERPRETATION |
|---|---|---|
| Survey | [Q20] Have you ever spoken with a pregnant woman about her SMOKING during pregnancy? Yes/No | |
| User | Yes | |
| Survey | [Q21] What did you say to her about smoking? | |
| User | Its okay to smoke but not a lot | No issues thus far. |
| Survey | [Q23] Have you ever spoken with a pregnant woman about her WEIGHT GAIN during pregnancy? Yes/No | |
| User | It's okay to smoke but not a lot | User thinks survey has not logged her answer for Q21, so she tries again. |
| Survey | Sorry, we do not recognize that answer. Please try again. | Q23 only accepts a yes or no answer, so survey rejects her answer. |
| User | Yes | User successfully answers Q23. |
| Survey | [Q24] What did you say to her about weight gain during pregnancy? | Q24 |
| User | Yes | User thinks survey has not logged her answer for Q23, so she tries again. Meanwhile, Q24 was a free text question which accepts any answer as valid. So, the survey thinks her answer to Q24 is “Yes.” |
| Survey | [Q26] This next section is about today's visit. First, what insurance do you have? (a) Medicaid (b) Healthy Michigan Plan (c) Other (d) Unsure | Q26 |
| User | Yes | User tries again to answer Q23. |
| Survey | Sorry, we do not recognize that answer. Please try again. | Q26 was looking for a multiple choice answer, and rejects her answer. |
| User | U7u l'll@ | User expresses frustration. |
| Survey | Sorry, we do not recognize that answer. Please try again. | Q26 was looking for a multiple choice answer and rejects her answer. |
| User | xXxXuiwqq y$ Ears AnA Aww daw w | User expresses frustration. |
| Survey | Sorry, we do not recognize that answer. Please try again. | User gives up. |
A Middle Aged Black User Interacts with the Survey Using a Non-Smartphone (Gender Unknown)
| SENDER | CONTENT OF MESSAGE | INVESTIGATOR'S INTERPRETATION |
|---|---|---|
| Survey | What is your race/ethnicity? (a) Non-Hispanic BLACK (b) Non-Hispanic WHITE (c) HISPANIC (d) ASIAN (e) OTHER Race-including Mixed Race | |
| User | a | |
| Survey | What type of phone are you using? (a) Smartphone like Apple iPhone or Samsung Galaxy (b) Flip-phone (c) Other | |
| User | c | |
| Survey | What is the highest level of school you have completed? (a) Less than High School (b) High School Graduate (c) Some College (d) College Graduate or above | No issues thus far. |
| User | High school graduate with professional degree in business in my country | User tries to answer with a free text response. Survey only accepts a, b, c, or d. |
| Survey | Sorry, we do not recognize that answer. Please try again. | |
| User | High school graduate | |
| Survey | Sorry, we do not recognize that answer. Please try again. | |
| User | No here | |
| Survey | Sorry, we do not recognize that answer. Please try again. | |
| User | No thank you | |
| Survey | Sorry, we do not recognize that answer. Please try again. | |
| User | I did high school in [Location] a no here in USA | |
| Survey | Sorry, we do not recognize that answer. Please try again. | User gives up. |
Location was redacted to protect patient privacy.
