What Consumers Say About Hospices in Online Reviews

Abstract

Background:

Until recently, consumers have had limited resources to assess quality of hospices agencies, contributing to growing numbers of consumers turning to online review sites, such as Yelp. However, little is known about the content of hospice Yelp reviews and how these relate to recently available Center for Medicare and Medicaid Services' Hospice Compare (HC) site data.

Objective:

To better understand what consumers report on Yelp about hospice care and explore how these areas relate to HC data.

Design:

We examined 692 consumer Yelp reviews of 67 hospices in California and compared identified themes with quality measures presented on the HC site.

Setting/Subjects:

We used a purposive sample of California Hospice's with Yelp reviews.

Measurements:

Qualitative consumer comments about their experience with hospice care were analyzed by using a grounded theory approach.

Results:

We found that overall Yelp comments were positive, however Yelp themes were more extensive and diverse than those on HC.

Conclusion:

We recommend that consumers consider both HC and online review sites such as Yelp when evaluating a hospice.

Background

Over the past two decades, the hospice industry has experienced tremendous growth, yet hospice is one of the least scrutinized health care services in the United States.¹ Only recently have hospices been required to report quality metrics and undergo standardized review. In 2017, Center for Medicare and Medicaid Services (CMS) launched its Hospice Compare (HC) website, so that consumers can compare hospices on nine hospice-agency reported quality metrics.² In 2018, CMS added eight caregiver survey items to HC.^3,4 For each hospice- and caregiver-reported indicator, HC reports the percentage of respondents giving the “top-box” response, which is the percentage of all individual ratings that are the highest one or two possible ratings.⁵

The HC review system has limitations. Recent research found that less than one-third of California hospices reported quality measures on HC and even fewer had caregiver-survey measures.⁶ Studies also show that many consumers are unaware of government-run health care review sites,^7,8 contributing to the growing interest in online consumer reviews of health care services.⁹ One of the most popular review sites, Yelp, averages 63 million unique desktop visits each month, with 8% of all reviews related to health providers, a percentage that has increased steadily in recent years.^10,11 On Yelp, consumers post descriptive reviews of businesses and then rate them on a scale of 1–5 stars (5 is best).

Despite growing use of Yelp to rate health care services, there has been no research investigating the content of Yelp hospice reviews. To better understand consumers' experience of hospice care, we analyzed Yelp reviews of hospices in California and compared these with caregiver-reported HC quality measures.

Methods

We conducted a qualitative analysis of hospice reviews on Yelp.com, using 2016 data from the California Office of Statewide Health Planning and Development (OSHPD) to identify the hospice sample. We searched Yelp for 1057 California hospices and collected the reviews and overall star ratings (1 [lowest]–5 [highest]). We found reviews for 246 (23%) hospices. We eliminated 16 agencies that had closed and 54 agencies that provided both home health and hospice. Of the remaining 176 hospices, we limited the sample to hospice with 3+ reviews, an approach consistent with other studies.¹¹ We sorted these hospices by number of reviews and selected every other agency to obtain a purposeful sample of 67.

Analysis

Using the grounded theory, we identified themes and categories within the reviews.¹² Two teams of two trained research assistants each open-coded the reviews. Using a constant comparison approach, team members compared and discussed the identified themes after each team member had coded reviews. Discrepancies were discussed until 100% consensus was reached. After reconciling all coding, researchers met to conduct axial coding to identify categories among the individual themes by using inductive reasoning. Finally, we compared our themes with the caregiver-survey measures on HC. Table 1 identifies these eight measures.

Table 1.

Frequency of Yelp Codes and Associated Hospice Compare Categories

Category/code	Definition	Example quotations (individual identifier for quotation, which refers to a location on an Excel sheet)	No. (%) of Yelp reviews containing the code (n = 646)	Overall prevalence in Yelp reviews	HC measure or survey item addressed
Patient/caregiver–provider relationship
Compassionate, caring staff	Any reference to hospice staff being compassionate or caring with the patient/family, either present of or lack of	“[Hospice] was a blessing to our mother and family. Mom called them her “guardian angels”. They kept her comfortable, attending to every small detail of her care on a daily basis. Compassionate and caring giving her support at anxious time when the family could not always be by her side. They keep in contact with us everyday guiding us through to the end, and then helped us handle the final arrangements. I cannot imagine going through this difficult time without them. Thank you [hospice].” (107)	299 (46.28)	1	N/A
Patient and family grateful to hospice	General praise for the agency or staff; any comment related to feeling grateful to the agency at a time of need	“I'm very thankful for [hospice] because they were constantly by our side during one of the most difficult time in our lives… Words can't explain how thankful I am for all you have done.” (620)	289 (44.74)	2	N/A
Patient/caregiver–provider communication^*	Any reference to poor/clear communication between hospice providers and the patient/family; lack of understanding or miscommunication	“If I could leave zero stars I would… I'm not [quite] sure why they are even a business or how they even get paid when there is no communication and there are lies being told to us about my loved ones care… These people are getting paid and NOTHING is being done.”(265)	151 (23.37)	4	CAHPS^® Hospice Survey: Communication with family
Staff is respectful^*	Any reference to the staff showing respect (or lack of) for the patient/family	“All I can say is these people are real life angels. They helped me through the most difficult time of my life, and even when we disagreed, they demonstrated total respect toward me and my family. Lord, I can only pray that the rest of the health care field catches on to what these compassionate people are doing day by day.” (178)	51 (7.89)	13	CAHPS Hospice Survey: Treating patient with respect
Clinical care
Quality of patient care	Any comment pertaining to the quality or lack of quality of care, including references to the patient's comfort or cleanliness	“My father was diagnosed with colon cancer which spread and became stage 4 lung cancer within a 5 year period. We found [hospice] as an option because my father and our family needed further support. We were fortunate enough to have exceptional services provided to us in our home which is where my father wanted to be. The [hospice] team is amazing and tailored his plan of care for not only his needs but also per his wishes. I would not recommend any other Hospice to a friend or family member that is in need …” (8)	136 (21.05)	5	N/A
		“During the time of my 93-year-old mother's passing, she was unable to swallow, yet [the hospice] did not provide liquid morphine. Instead, my mother's health care providers were left to grind up morphine tablets in water which caused my mother to cough violently, as the morphine granules irritated her throat. It was horrific and I did not realize at the time this was entirely unnecessary if [the hospice] had not been negligent about providing good care during my mother's last days.” (579)
Medication management	Any reference to accessing medication, medication education, medication prescribing, or medication problems	“…Where to start? At our first meeting, we discussed medications and which ones should be slowly withdrawn and which ones to continue. The RN promised me to provide a plan for the medication changes. This never happened. Instead, when I complained to a manager, I was told it was MY responsibility to come up with the medication withdrawal plan! Then I found that the [hospice] doctor had removed my mother from a number of medications at once, without ramping down. Was there any negative effect on my mother? Difficult to say when a person is dying, but she did get more agitated and restless…”(96)	125 (19.35)	6	N/A
Pain and symptom management^*	Any reference to pain and symptom management or lack thereof	“…His pain was not managed and remained at a level 8 for many days…The whole point of going on hospice is to make the patient comfortable and to control his suffering. Sadly, none of this was accomplished…” (139)	90 (13.93)	10	CAHPS Hospice Survey: Help for pain and symptoms Hospice and Palliative Care- Pain Screening- NQF #1634Hospice and Palliative Care- Pain Assessment- NQF #1637Hospice and Palliative Care- Dyspnea Screening- NQF #1639Hospice and Palliative Care- Dyspnea Treatment- NQF #1638Hospice and Palliative Care- Patients treated with opioids who are given a bowel regimen- NQF #1617
		“…kept my father very comfortable when he was in extreme pain from cancer, until the day he passed. They helped us keep him clean, cozy, and happy until the end.” (548)
Provision of grief or bereavement counseling or support	Any comment related to support (or lack of support) for the caregiver preceding or after the patient's death	“I wouldn't have known how to survive the grief and continue to go through the grief of my father without the complimentary counseling services provided by this wonderful organization.” (119)	79 (12.23)	11	N/A
		“…My mother had just passed away and I was grieving… No one at [the hospice] that were counselors were calling me back to see someone… How can they treat people like this that are grieving and need help! …” (132)
Agency competency
Timeliness of care or responsiveness of staff^*	Any reference to the responsiveness of care delivery, or the lack thereof; can also be referenced as staff helpfulness or attentiveness to patient/family	“…They say they will have a team of caregivers, nurses and aids here to help. My dad has not had the same person come in to help my mother. We call for a nurse to check my mother. 24 hours later still waiting. Someone will call and say they will be there within 30 min. 2 hours later no one has called or arrived…” (559)	256 (39.63)	3	CAHPS Hospice Survey: getting timely help
		“…They are true angels. When you call in the middle of the night you are connected to either the director of nursing or to [the doctor] herself… not an answering service. You get a living breathing soul who is dedicated to their calling…” (383)
Care comprehensiveness	Presence (or lack) of embracing, all-inclusive care and support for the patient/family	“The comprehensive approach that [hospice] provides is beyond what my family and I could have imagined, and we will be forever grateful to the entire team!” (106)	63 (9.75)	12	N/A
Agency management	Any reference to the agency's overall management, administration, or structure	“Hospice can be such a gift and they were a very well run, professional service, that didn't ask us for anything other than feedback as our loved one passed on, to ensure his needs were met. Forever grateful.” (121)	43 (6.66)	14	N/A
Staff professionalism
Staff professionalism	Reference to staff as professional or unprofessional; also, staff behavior that is inferred to be either professional or not	“…Needless to say, they definitely took wonderful care of my grandfather, and made his last valuable months a lot easier as he was approaching his time. They were very professional and always assisted my grandfather in an excellent manner.” (585)	124 (19.20)	7	N/A
		“The Chaplain … was beyond unprofessional and incompetent, reading private texts messages over family members shoulders, interfering in my husband's medical care—things completely outside of her purview and training.” (650)
Knowledgeable, skilled staff	Any reference to the staff having (or lacking) the knowledge and skills necessary for caring for the patient	“{Hospice staff] are simply the best—caring and compassionate nurses, social workers, hospice aides, chaplains, volunteers, physicians who are highly skilled and knowledgeable and always go above and beyond to make sure you receive the care you want and need based on your wishes and preferences.” (606)	97 (15.02)	9	N/A
Safety	Care that promoted or impaired the safety of patients; reports of poor outcomes related to poor care, clinical decision making, or treatment	“A young nurse came the 3rd day and was scared to death and was second guessing if she was giving medication correct. She took the oxygen off my sister to give her meds. I came in the room to notice the nurse forgot to put her oxygen back on and as the camera was recording all this my sister passed away struggling to breath. So sad…” (222)	8 (1.24)	15	N/A
Supplies and equipment
Clinical/medical supplies and equipment	Any reference to clinical or medical supplies or equipment. Whether provided or not; any reference to supplies/equipment being ordered/received on time, or not	“They had dumped some useless equipment in his room that he never needed but did not supply or order the pain medication CADD pump they had told us he would have. (650)”	104 (16.10)	8	N/A
		Total number of codes for all themes	1915

This code is directly addressed by items in the HC caregiver survey, which is administered by independent evaluators after the death of a loved one. Not coded in this study, but included in the HC survey, are: (1) family training and (2) spiritual/emotional support.

Source: https://www.medicare.gov/nursinghomecompare/search.html?

HC, Hospice Compare; N/A, not applicable.

Results

We coded a total of 692 individual hospice Yelp reviews. The average number of Yelp reviews per hospice was 10.03 (median = 7; mode = 3). The average of the 67 overall Yelp ratings for the hospices was 4 stars (median = 4; mode = 5).

Yelp themes and overarching categories

Our team identified 15 themes across the Yelp reviews, which we grouped into five overarching categories. Table 1 shows each category, its associated themes (with definitions), example Yelp reviews, and the frequency of each theme mentioned in the Yelp reviews. In the last column, the table shows that four of our identified Yelp themes were directly associated with four of the eight caregiver-survey measures published on HC.

Category: Patient/caregiver–provider relationship

Four themes comprised the patient–provider relationship category. Across all themes, the most frequently mentioned theme was compassionate, caring staff, identified in nearly half (46.28%) of the reviews. This theme encompassed the type of care patients received and whether staff acted compassionately with patients and family members. Although most reviews in this theme were positive, some also mentioned uncaring staff.

Patient/family gratitude was the second-most frequently mentioned theme, identified in 44.74% of the reviews. This theme pertained to comments (all positive) thanking staff members and sometimes recommending the hospice to others.

Patient/caregiver–provider communication (also an HC measure) was identified in nearly a quarter (23.37%) of all Yelp reviews. Comments ranged from constant communication with management, lack of communication about specific procedures, to follow through (or lack of) with specific requests.

Respectfulness (also an HC measure) was one of the least mentioned themes in Yelp reviews (7.89% of all reviews). Most comments reflected a positive experience in hospice staff respectfulness.

Category: Clinical care

This category comprised four related themes. The theme of quality of patient care provided by the hospice agency included comments on topics such as cleanliness and comfort. Positive comments pertained to how great the care received was, whereas negative comments expressed disdain for how a loved one was treated.

The medication management theme included any reference to accessing medication, medication education, prescribing, and medication problems. Both negative and positive comments were coded under this theme.

Pain and symptom management (also an HC measure) related to both negative and positive comments on whether the patient experienced pain or other symptoms and whether these were resolved or managed.

The theme of provision of grief and bereavement counseling/support largely focused on family and caregiver needs and included both positive and negative comments.

Category: Agency competency

Three themes comprised this category. The third-most frequently mentioned theme in this study, timeliness or responsiveness of staff members was identified in 39.63% of the Yelp reviews. This theme (also an HC measure) included both positive and negative comments, primarily around agency responses to family needs for care or support. Among the negative comments, problems largely focused on lack of response to calls or no-shows for promised visits.

Comprehensiveness of services and care was identified as a theme in a relatively small portion of comments (9.75%). This theme pertained to whether the care provided included the complete wrap-around-services expected of hospice agencies.

Agency management was the second-least identified (6.66%) theme in this study. The consumer comments largely related to how well or poorly the agency seemed to have been managed. Some comments (all negative) pertained to problems with billing.

Category: Staff professionalism

This category comprised three themes. Most of the comments in this category focused on staff professionalism and appropriate care during home visits. Negative comments pointed out family members' frustration with how some agencies handled caregiver feedback in unprofessional ways.

Knowledgeable/skilled staff theme reflected hospice staff members' competence, including their education, training, knowledge, and skill. Positive feedback under this theme always aligned with good care, whereas negative comments expressed dissatisfaction with poor care.

Although safety was the least identified (1.24%) theme in our study, the negative reviews all reported serious outcomes from mismanaged care, which resulted in either injuries or unexpected acceleration of the dying process.

Category: Medical equipment and supplies

This category included just one theme, in reference to whether the hospice provided medical equipment or supplies. This theme also included any reference to supplies or equipment being ordered or received, whether on time or not.

Discussion

This study is the first to examine Yelp hospice reviews and compare the themes identified with the topics addressed by CMS's HC measures. We found considerable overlap between the themes captured in HC caregiver-survey items and Yelp. Yelp reviews, which were largely positive, with agencies receiving a rating of 4.0 on average, addressed four of the eight items in the HC caregiver survey. Two additional HC caregiver survey items—the caregiver's willingness to recommend the hospice and the caregiver's overall hospice rating—were reflected in the Yelp reviews, although not specifically identified as themes in our analysis. For example, throughout the reviews, whether negative and positive, consumers frequently mentioned whether they recommended the hospice. Only two HC caregiver survey items did not emerge in our analyses: family training, and provision of emotional and spiritual support. These results echo those from a previous study, which found that Yelp reviews of hospitals addressed most of the topics covered in a patient experience survey that Medicare-participating hospitals sponsor, with results published on CMS's Hospital Compare website.¹³

We found that Yelp reviews addressed a wider range of themes than did the HC caregiver survey. We identified eight themes that were not addressed by the HC caregiver items, including the most frequently mentioned theme, compassionate, caring staff, cited in almost half the reviews. This finding is consistent with findings from hospital and nursing home Yelp studies.^13,14

Our study coincides with growing concern about the quality of hospice care in the United States. In recent investigations, the federal Office of the Inspector General found serious deficiencies at 87% of hospices, with 20% of hospices having deficiencies that could endanger patients.¹⁵ Nearly one-third of hospices had a complaint filed, of which almost half were severe. These deficiencies and complaints are not published on HC or easily accessible to the public.

Online consumer review sites may mitigate these problems and help improve hospice care, because they give voice to hospice users whose stories may help others make more informed decisions. Their reviews may reveal flaws—and strengths—among hospices that otherwise might not come to light. Online review sites offer an unstructured format for consumers to share their experiences more broadly without the constraint of a limited response set from surveys, such as those used with caregivers on HC.

In addition, given both the range and specificity of feedback provided by Yelp reviews, hospice agencies may find these reviews valuable for improving services as well. Agencies may be able to identify patterns among the comments that point to a particular problem to address. For example, a hospice with several reviews related to hospice staff failing to show up for home visits as scheduled may want to consider strengthening staff accountability or changing visit schedules to ensure staff members can meet them. Yelp also allows businesses to comment on and respond to consumer reviews, which may allow hospice agencies to demonstrate to potential consumers that it values and is responsive to customer feedback.

All rating and review systems have weaknesses as well as strengths. Yelp, for instance, has been criticized because some reviews, an estimated 25%, are likely fake.¹⁶ To mitigate this problem, Yelp screens reviews with an algorithm designed to spot and remove reviews deemed fake. Consumers also can report suspect reviews to Yelp, which then screens these reviews. Yelp reviews also are not collected systematically. Consequently, not all hospice agencies are represented on Yelp, and the volunteers who post their reviews and ratings are not necessarily representative of other consumers.

On balance, however, we recommend that consumers be encouraged to both post hospice reviews to Yelp and consult Yelp reviews, along with HC ratings, when evaluating hospices. Hospice care is centered on eliciting and honoring patient preferences at the end of life. Given this, it seems particularly important to encourage consumers to review hospices in their own words (as opposed, or in addition, to answering a structured survey) and share them with others. Moreover, with more reviews comes more information to help hospice directors improve quality of care.

Limitations

The small sample and inclusion of only California hospices limits the study's generalizability. Yelp reviewers are not randomly selected, but are volunteers, and so this study's sample may be biased in some way.

Footnotes

Acknowledgment

The authors wish to thank (blinded) for help with data collection.

Funding Information

The authors received no financial support for the research, authorship, and/or publication of the article.

Author Disclosure Statement

No competing financial interests exist.

References

Whoriskey

: Is that hospice safe? Infrequent

Yelp mean it may be impossible to know

. The Washington

Post

. https://www.washingtonpost.com/business/economy/is-that-hospice-safe-infrequent-inspections-means-it-may-be-impossible-to-know/2014/06/26/e258e880-eaa4-11e3-b98c-72cef4a00499_story.html?noredirect=on (Last accessed April 4, 2020).

Current Measures: 2019. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Hospice-Quality-Reporting/Current-Measures.html (Last accessed April 4, 2020).

Fact Sheet: The Hospice Item Set (HIS). www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Hospice-Quality-Reporting/Downloads/HIS-Fact-Sheet.pdf (Last accessed April 4, 2020).

Dena

: A New Tool For Caregiving: Hospice

Compare

. https://www.aarp.org/caregiving/basics/info-2017/hospice-care-online-tool-fd.html (Last accessed April 4, 2020).

CAHPS Hospice Survey Project Team: CAHPS Hospice Survey. https://www.hospicecahpssurvey.org/ (Last accessed April 4, 2020).

Enguidanos

, Rahman

: The ratings game: An analysis of hospice compare and yelp ratings (In review). BMJ Qual Saf 2019.

Schapira

, Shea

, Duey

, et al.: The nursing home compare report card: Perceptions of residents and caregivers regarding quality ratings and nursing home choice. Health Serv Res, 2016; 51:1212–1228.

Shugarman

, Brown

: NURSING HOME SELECTION: HOW DO CONSUMERS CHOOSE? Volume I: Findings from Focus Groups of Consumers and Information Intermediaries. http://aspe.hhs.gov/_/office_specific/daltcp.cfmor. 2006.

Factsheet Yelp. https://www.yelp.com/factsheet (Last accessed April 4, 2020).

10.

Hedges

: How Patients Use Online

Reviews

. https://www.softwareadvice.com/resources/how-patients-use-online-reviews/ (Last accessed April 4, 2020).

11.

Yelp's Approach to Healthcare Quality Improvement | Pyzdek Institute. https://www.pyzdekinstitute.com/blog/healthcare-quality/yelps-novel-approach-to-healthcare-quality-improvement-patients-review-doctors.html (Last accessed April 4, 2020).

12.

Strauss

, Corbin

: Basics of Qualitative Research—Techniques and Procedures for Developing Grounded Theory, 2nd ed. London: Sage Publications, 1998.

13.

Ranard

, Werner

, Antanavicius

, et al.: Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Aff, 2016; 35:697–705.

14.

Johari

, Kellogg

, Vazquez

, et al.: Ratings game: An analysis of nursing home compare and Yelp ratings. BMJ Qual Saf, 2018; 27:619–624.

15.

Chiedi

: Hospice Deficiencies Pose Risks to Medicare Beneficiaries Hospice Deficiencies Pose Risks to Medicare Beneficiaries What OIG

Found

. https://oig.hhs.gov/oei/reports/oei-02-17-00020.pdf?utm_source=summary-page&utm_medium=web&utm_campaign=OEI-02-17-00020-PDF (Last accessed April 4, 2020).

16.

Yelp: Does Yelp recommend every review? https://www.yelp-support.com/article/Does-Yelp-recommend-every-review?l=en_US (Last accessed April 4, 2020).