Are e-Health Web Users Looking for Different Symptom Information Than Callers to Triage Centers?

Abstract

Introduction: Telemedicine options for symptom assessment include both telephonic means (call centers) and Internet sites. Although symptom assessment call centers have been available for decades, symptom assessment over the Internet is relatively new. It is not well known what types of symptoms Internet users are seeking assessment for and whether extant telephone triage algorithms would be applicable to Internet users. Methods: Symptom assessments on a heavily used Internet site (MayoClinic.com) were compared with symptom calls to a U.S. call center (Ask Mayo Clinic). Results: Internet users sought symptom assessments about adult symptoms 13 times more often than children's symptoms. In contrast, over the telephone, adult symptoms were addressed 2.1 times more frequently than children's symptoms. Despite the differences in frequencies of adult and child symptom assessments, users of the Internet and telephone callers asked about specific symptoms with similar relative frequencies. Analysis of 20 adult symptom types shared by the Web and call center revealed that by excluding only 2 assessments (nasal symptoms and leg pain) the remaining 18 showed a significant correlation in counts of use (r² = 0.68, p < 0.001 for linear trend). Conclusions: Internet users have symptoms assessed in similar proportions to callers, with a few exceptions. Compared with callers, Web users are much more likely looking for information about adult symptoms. Callers are proportionally asking more about acute symptoms, whereas Internet users appear more interested in symptom assessment of chronic conditions.

Introduction

Advice given over the telephone concerning medical symptoms is not new. Physicians' offices have been doing this for decades, primarily during office hours. When the office was closed, telephone advice was frequently given by the doctor himself. More recently, telephone triage centers have been commonly used to assist with these symptom-related calls after hours. These telephone triage centers are usually staffed by nurses or doctors called triagists and are often supported by computerized decision support tools in the form of symptom-related algorithms.¹ In the United States, a call center in Colorado is handling the after-hours calls for up to 95% of pediatric practices in that state.^2,3 Outside the United States, several countries have national call centers that take symptom-related calls from any of their citizens.⁴ The largest and best studied national call center is National Health Service (NHS) Direct in the United Kingdom, which gives symptom-related advice on 5 million calls yearly.⁵

There are other telemedicine options for acquiring information about symptoms. The Mayo Clinic, for example, has a symptom checker available online.⁶ WebMD^® also evaluates symptoms online and the NHS in the United Kingdom has an Internet version of NHS Direct that includes an online tool to evaluate symptoms.^7,8 These online symptom assessment tools are generating a great deal of interest among Internet users. The NHS Direct Internet site, for example, reported 30 million visits in 2008.⁹ Although there is a large amount of literature concerning evaluation of symptoms via telephone triage centers, very little has been published about online approaches to evaluation of symptom concerns. We also know little about what symptoms these online users are inquiring about. Are online users asking about similar symptoms to telephone callers? The answer to this question is important for healthcare organizations who may want to consider adding a Web-based symptom assessment tool to their telephonic system for assessing symptom calls.

To answer the basic question of what people are looking for in telemedicine symptom assessments, we retrospectively compared symptom assessments done online at MayoClinic.com, a health information Internet site, with telephonic symptom assessments done at Ask Mayo Clinic, a telephone triage center.

Methods

Setting

Ask Mayo Clinic is a 24/7 telephone triage call center located in the United States. Callers to the service speak with registered nurses who use computerized algorithms and guidelines to guide their assessments. The triage software, ExpertRN, has 50 symptom-driven algorithms and 87 guidelines that supply decision support for the triage nurses and captures relevant data concerning the calls. Access to Ask Mayo Clinic is by subscription only. Subscribers are generally medium to large corporations located in the United States. These corporations purchase the triage service on behalf of their employees and dependents. Ask Mayo Clinic had a total of 127,654 calls in 2009 and symptom calls accounted for 70,370 (55%) of those calls with the rest primarily devoted to medical information. Our analysis used only the 70,370 symptom calls.

MayoClinic.com is reported to be one of the most popular U.S. health-related Web sites.¹⁰ MayoClinic.com had nearly 180 million Web page visits in 2009. The site has a Web page called “symptom checker,” where the user can click from a menu of 28 different symptoms for adults or 17 symptoms for children.⁶ Clicking a symptom on the menu page transfers the user to a symptom-specific Web page where the user is asked to supply more information pertinent to the symptom.

Although both the Web symptom checker and the nurse triage software are designed to collect and interpret information about similar symptoms, the focus of the software differs. For the Web symptom checker, the software provides a list of possible diagnoses or conditions associated with symptoms supplied by the user. For the telephone triage software, the focus is on directing the caller to the most appropriate care. The telephone caller is advised about the seriousness of the symptoms and when and where to get further care (including advice for care at home if appropriate). To illustrate this difference, a Web user clicking on symptoms of sudden, squeezing chest pain with diaphoresis receives a list of possible diagnoses, including angina, heart attack, and aortic aneurysm. For the same symptoms, the triage software concludes the caller's chest pain may be life-threatening and recommends an immediate call to 911 for an ambulance.

Data Collection

We used the calendar year 2009 as our data collection interval for both Web site counts and symptom assessment call counts. Web site counts were determined by Google Analytics.¹¹ Specifically, we used the site overlay feature, which provides a visual display showing the numbers of times that standard links embedded on a particular Web page are clicked by visitors to that page. This method removes any search engine influence, which could potentially distort the relative frequencies and allowed us to examine the true proportion of use from a symptom menu. The method we used also negated influence of links directly to those symptom pages from other sources. On the telephonic side, all completed calls were counted and put into symptom categories based on which algorithm or guideline was used during the call.

For the telephonic symptom assessments, we used all 70,370 symptom assessments completed by Ask Mayo Clinic for 2009. These were categorized by age and symptom. We used all 2,059,299 symptom checker clicks occurring for 2009, categorized by symptom and dichotomously to pediatric (newborn through 17) or adult (age 18 and over).

Statistical Methods and Analysis

There were fewer symptom categories on the Web site than available for nurses to choose from at the telephone call center. The symptoms on the Web site were matched as closely as possible to the symptom description in the call center triage software. Matching was accomplished a priori and not modified after analysis.

Of the 28 symptoms in the adult symptom checker Web site, we were able to match 20 with symptoms from callers; 14 of the symptoms matched identically. Seven of the 28 Web site symptoms were collapsed down into three symptom categories to match single symptoms on the telephonic side. For example, we combined Web site categories of knee, foot, and hip pain into a single category that we could match with leg pain in the telephonic database. The other three Web symptom categories had more than one telephone category such as the Web site nasal symptoms category, which in the telephonic data was represented as cold symptoms and blocked and runny nose. Of the 17 pediatric symptoms available on the Web site, 13 were matched with equivalent pediatric symptoms in the telephone data; 10 of them matched identically.

To evaluate on a symptom-by-symptom basis whether calls were similar to Internet use, we used odds ratios of Internet to telephonic use. To assess overall match of symptom assessments on the Web site with telephonic use, counts of calls were correlated with counts of Internet clicks for each of the matched symptoms to get a Pearson correlation coefficient. We used SAS 9.1.3 (SAS Institute, Cary, NC) for statistical analysis.

This study was approved by the Mayo Clinic Institutional Review Board.

Results

Of the 70,370 symptom assessment calls in 2009, calls about children (birth to age 18) accounted for 32% (22,386 of 70,370). Adult symptom calls for ages 18–44 were 44% of the calls, adults age 45–64 accounted for 17% of the calls, and symptoms of those 65 and over were 7% of the calls. For calls concerning those under 18, there were 10,848 (49%) about females. For adults, 32,739 (70%) of the calls concerned females.

In 2009, there were 2,059,299 clicks to specific symptoms on the MayoClinic.com symptom checker page, or slightly over 1% of the total MayoClinic.com Web page visits. A voluntary online survey of 1,130 MayoClinic.com users in 2009 found no self-identified users under age 18. Users aged 18–44 accounted for 21%, whereas 54% were aged 45–64, and 25% were 65 and over. The survey also showed that Web users were predominantly female (77%). Only 7.0% (144,140 of 2,059,299) of the MayoClinic.com symptom checker clicks concerned children.

Using 20 adult symptoms, we were able to match 90% (1,715,707 of 1,915,159) of the Web use to 47% (22,592 of 47,984) of the telephone calls. The 13 children symptoms used in the analysis matched 89% (127,798 of 144,140) of the Web site traffic on children's symptoms with 61% (13,581 of 22,386) of the telephone calls about children.

MayoClinic.com online symptom checker users sought assessments about adult symptoms 13 times more often than children's symptoms. In contrast, at Ask Mayo Clinic, the telephone triage users consulted about adult symptoms 2.1 times more frequently than children's symptoms. There was a 6.2 (95% confidence interval; 6.1–6.3) odds for symptom assessments being about adults rather than children when comparing online symptom checker use to telephone triage use.

Odds ratios of Internet counts to call counts on a symptom-by-symptom basis are shown in Tables 1 and 2. The inverse (odds ratios of calls to Internet use) is also included to give a better perspective on how these two forms of symptom assessments differ. Figure 1 shows the scatter plot of Web use to telephonic use for each of the 20 matched adult symptoms. The correlation was modest with adjusted r ² = 0.25 and linear trend p = 0.01 (Fig. 1). Exclusion of only two symptoms (nasal symptoms and leg pain) out of the 20 resulted in a marked increase in the correlation coefficient to an adjusted r ² = 0.68 and a more significant linear trend (p < 0.001). For the 13 symptoms of children, the correlation of Internet to call counts was less, giving an adjusted r ² = 0.10 and no significant linear trend (p = 0.15). However, exclusion of only 1 symptom (nasal symptoms) out of the 13 increased the correlation coefficient to an adjusted r ² = 0.57 and produced a significant linear trend (p = 0.003).

Fig. 1.

Correlation of adult symptom Web use to telephonic use (fitted least squares linear regression line with 95% confidence band).

Table 1.

Comparison of Online Adult Symptom Assessment Use to Adult Telephone Triage Assessment Calls (Age 18 and Up)

Symptom	Online Symptom Checker Use (2009 Click Count)	Telephone Triage Center Calls (2009 Call Count)	Odds Ratio Online Use to Telephone Use (CI 95%)	Odds Ratio Telephone Use to Online Use
Difficulty swallowing	33,928	102	4.5 (3.6, 5.4)	0.22
Neck pain	63,957	228	3.8 (3.3, 4.3)	0.26
Leg pain (foot, knee, or hip pain)	213,090	1,053	2.9 (2.7, 3.1)	0.34
Constipation	50,154	304	2.21 (2.0, 2.5)	0.45
Numbness or tingling in hands	94,611	723	1.8 (1.6, 1.9)	0.57
Foot or leg swelling	60,329	535	1.5 (1.4, 1.6)	0.67
Cough	80,131	766	1.4 (1.3, 1.5)	0.72
Heart palpitations	47,489	461	1.37 (1.25, 1.5)	0.73
Low back pain	104,745	1,108	1.26 (1.19, 1.34)	0.79
Headaches	102,663	1,142	1.2 (1.13, 1.27)	0.84
Shortness of breath	58,311	647	1.19 (1.10, 1.29)	0.84
Dizziness	101,525	1,305	1.03 (0.97, 1.09)	0.97
Abdominal pain	253,097	3,291	1.01 (0.98, 1.05)	0.99
Diarrhea	68,331	1,204	0.74 (0.69, 0.78)	1.36
Sore throat	48,181	888	0.71 (0.66, 0.76)	1.42
Chest pain	98,205	1,791	0.71 (0.67, 0.74)	1.42
Eye or vision concern	74,766	1,458	0.66 (0.63, 0.70)	1.51
Urinary problems	82,012	1,813	0.58 (0.55, 0.60)	1.74
Nausea or vomiting	52,937	1,443	0.47 (0.44, 0.49)	2.14
Nasal symptoms	27,245	2,330	0.14 (0.13, 0.15)	7.13

CI, confidence interval.

Table 2.

Comparison of Pediatric Online Symptom Assessment Use to Pediatric Telephone Triage Assessment Calls (Age Newborn Through 17)

Symptom	Online Symptom Checker Use (2009 Click Count)	Telephone Triage Center Calls (2009 Call Count)	Odds Ratio Online Use to Telephone Use (CI 95%)	Odds Ratio Telephone Use to Online Use
Headaches	9,446	336	3.15 (2.8, 3.5)	0.32
Urinary problems	6,437	299	2.36 (2.1, 2.7)	0.42
Abdominal pain	16,011	881	2.06 (1.9, 2.2)	0.48
Earache	7,426	472	1.71 (1.6, 1.9)	0.58
Skin rashes	31,239	2,328	1.56 (1.5, 1.6)	0.64
Sore throat	6,009	491	1.32 (1.2, 1.4)	0.76
Fever	15,562	1,322	1.29 (1.2, 1.4)	0.78
Constipation	3,806	437	0.92 (0.80, 1.0)	1.08
Cough	11,617	1,361	0.90 (0.85, 0.95)	1.11
Eye or vision concern	5,630	754	0.78 (0.73, 0.95)	1.28
Diarrhea	5,122	753	0.71 (0.66, 0.77)	1.41
Nausea or vomiting	6,718	1,571	0.42 (0.40, 0.45)	2.36
Nasal symptoms	2,775	2,576	0.09 (0.09, 0.10)	10.6

Discussion

This retrospective study compared symptom checker use on the Internet to telephone triage center symptom calls. To our knowledge, this is the first large-scale study comparing the usage patterns of a telephone triage center to a Web-based symptom checker. A striking finding was how much more likely the Web users inquired about adult symptoms. The odds ratio was 6 to 1 in favor of Internet symptoms being about adults, compared with symptoms addressed telephonically. Conversely, children are 6 to 1 more likely the subjects of telephone triage compared with the Web symptom checker. This result may be indicative of a fundamental difference in delivery of information between these options. We know that callers greatly appreciate the reassurance delivered by call centers.¹² Even though Web sites may have similar content to the algorithms used by call centers, the human contact of a nurse reassuring a mother calling about a sick child is hard to capture on a Web page. It may be that people are more willing to explore their own symptoms through computer-driven algorithms, but they want the reassurance of a nurse when dealing with the symptoms of their children.

For adults, Internet counts compared with caller counts on a symptom-by-symptom basis showed modest correlation. This indicates that online symptom assessment users are proportionally interested in the same symptoms as the users of telephone triage centers; exceptions among the adults were leg pain and nasal symptoms. Telephone triage centers have higher odds of receiving calls related to adult nasal symptoms as compared with the online symptom assessment tool. On the other hand, more adults with leg symptoms used the online symptom assessment tool as compared with the telephone triage center. Leg pain is one of the symptoms that increase dramatically with age in telephonic calls, so an older Web audience could explain why leg pain symptoms were relatively higher on the Web site.¹³ For pediatric symptoms, after excluding nasal symptoms, there was a significant linear correlation between the Web and telephone counts.

We examined the possibility that the 2009 H1N1 influenza pandemic influenced our findings. Perhaps a surge of nasal symptoms caused by the H1N1 influenza was a cause for nasal symptoms predominating in calls as compared with Web use. To test this possibility we recalculated the correlation between calls and the Web symptom checker by just using the months January through August 2009. This excluded the time frame of September through December 2009 when the H1N1 pandemic occurred in the United States.¹⁴ The symptom count correlation coefficients after excluding the peak H1N1 months were only slightly different from for all of 2009. The adjusted r ² after excluding the H1N1 pandemic months was 0.27 for adults (compared with 0.25 for all 2009) and was 0.20 for children (compared with 0.10 for all 2009). On the basis of these calculations, the H1N1 pandemic appears not to have had much of an overall effect on the results.

The symptom-by-symptom odds ratios of calls to Internet use (Tables 1 and 2) suggest a pattern. The Internet users of the symptom checker appear to be looking more for chronic illness conditions. Symptoms with high odds ratios showing more Internet interest are associated mostly with chronic conditions such as leg pain, neck pain, constipation, and numbness or tingling. Difficulty swallowing may be comparatively high on the Internet symptom list because it can be associated with some chronic gastrointestinal reflux symptoms. On the other hand, symptoms associated with telephone triage tend to be more acute or noticeable so that immediate relief of the symptom is desired. These symptoms include nausea and vomiting, nasal symptoms, chest pain, and eye problems. When taken in this context, our results portray an overall tendency for the Internet users to seek information about chronic conditions, whereas callers to telephone triage are asking proportionally more about acute symptoms.

This study has limitations. We had only limited demographic information on the Web users to match with the callers. Although we matched Web and telephone use by age, the separation into only two groups (adult and child) limits some of our interpretation of the data. For example, we know that in a call center, frequencies of adult symptoms are significantly affected by age.¹³ Also, matching by other demographic variables such as gender, race, and socioeconomic status was not possible given the limitations of Internet count data. We also know that adult callers are frequently calling about someone else, and that these surrogate calls account for 14% of all adult symptom calls and 40% of calls concerning those 80 and over.¹³ Surrogate calls have some significant differences compared with self-calls.¹⁵ Internet users were not identified by surrogate or self, so we were also unable to match the Internet users to callers by this potentially important variable.

With the data available, it was difficult to know just how much of the Web site traffic was simply browsing. It is also possible that users could be clicking on the same symptom multiple times for the same problem. We did not have the ability to identify either of these types of symptom checker use. However, the nonrandom correlation of the symptom-driven Web traffic with the telephonic symptoms suggested that much of the Internet use of the symptom checker was purposefully directed.

Future studies should compare the outcomes of users of Internet symptom checkers to telephone triage callers. We know that adults with abdominal pain who called the Ask Mayo Clinic triage call center were hospitalized at a rate of 13% within 72 h of the call, a significantly higher rate of hospitalization than following regular office visits.¹⁶ Unfortunately, we do not have similar hospitalization data or other outcomes for comparison with the Internet users. Without this information it will be difficult to know how to customize tried and tested telephone triage algorithms for use on the Internet. For example, if Internet users with abdominal pain had a rate of hospitalization far different from telephone callers, then a time-tested telephone triage abdominal pain algorithm may not meet the needs of Internet users and would require major modification.

Our study suggests that users of online symptom assessment tools have a clear preference for assessing adult symptoms compared with symptoms of children. Healthcare organizations desiring to set up a symptom assessment Web site to shift some of their telephonic triage calls to a potentially less costly Web-based assessment may want to focus on their adult patients first. Viewed overall, symptom assessments by Internet and by telephone are similar enough to be significantly correlated. However, viewed on a symptom-by-symptom basis, there are important differences. These differences are confirmed by odds ratios, suggesting that Internet users are more likely looking for chronic disease assessment, whereas telephone callers are more interested in acute symptoms.

There is a plethora of knowledge about symptom assessment by telephone, but information about Internet symptom assessment is limited. Although our study shows some similarity between the use of telephone triage and a Web-based symptom checker, important differences exist. On the basis of our results, practices wishing to expand telemedicine symptom assessment from telephone to the Internet may want to focus on adult symptoms first. The finding of high Internet interest in certain conditions such as leg pain and abdominal pain should be helpful for those interested in building Web-based symptom assessments. Our study shows enough similarities between the use of Internet symptom assessment and telephone triage to suggest that some of the current knowledge of telephone triage may be transferable to Web-based symptom assessment. However, additional research is needed to determine whether tried and tested telephone triage algorithms can meet the needs of Internet users.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

O'Cathain

, Sampson

, Munro

et al. Nurses' views of using computerized decision support software in NHS Direct. J Adv Nurs, 2004; 45:280–286.

Kempe

, Dempsey

, Whitefield

et al. Appropriateness of urgent referrals by nurses at a hospital-based pediatric call center. Arch Pediatr Adolesc Med, 2000; 154:355–360.

Bunik

, Glazner

, Chandramouli

et al.

Pediatric telephone call centers: How do they affect health care use and costs?

Pediatrics, 2007; 119:e305–e313.

Grol

, Giesen

, van Uden

. After-hours care in the United Kingdom, Denmark, and the Netherlands: New models. Health Affairs, 2006; 25:1733–1737.

NHS Direct Annual Report 2008/2009. www.nhsdirect.nhs.uk/article.aspx?name = AnnualReport2009. 2010 December 1.

Mayo Clinic Symptom Checker. www.mayoclinic.com/health/symptom-checker/DS00671. 2010 December 1.

WebMD Symptom Checker. http://symptoms.webmd.com/. 2010 December 1.

NHS Direct Internet Symptom Assessment. www.nhsdirect.nhs.uk/sat/InitialAssessment/. 2010 December 1.

NHS Direct Annual report 2007–2008. London: The Stationary Office, 2008.

10.

Top 20 Most Popular Health Websites July 2010. www.ebizmba.com/articles/health-websites. 2010 July 21.

11.

Google Analytics. www.google.com/analytics/. 2010 December 1.

12.

O'Cathain

, Munro

, Nicholl

, Knowles

. How helpful is NHS direct? Postal survey of callers. BMJ, 2000; 320:1035.

13.

North

, Varkey

. A retrospective study of adult telephone triage calls in a US call centre. J Telemed Telecare, 2009; 15:165–170.

14.

Centers for Disease Control and Prevention (CDC). Update: Influenza activity—United States, August 30, 2009–January 9, 2010. MMWR Morb Mortal Wkly Rep, 2010; 59:38–42.

15.

North

, Muthu

, Varkey

. Differences between surrogate telephone triage calls in an adult population and self calls. J Telemed Telecare, 2011(in press).

16.

North

, Varkey

. How serious are the symptoms of callers to a telephone triage call centre? J Telemed Telecare, 2010; 16:383–388.