Development of a validated patient satisfaction survey for HIV clinic attendees

Introduction

Patient experience of HIV care has changed significantly over the past 15 years primarily as a result of effective medical treatment which has led to longer life-expectancy and an enhanced quality of life. HIV patients’ priorities for how their health care is delivered is also likely to have changed to reflect these improved outcomes. Health-care quality is multidimensional and includes patient experience, safety and the effectiveness of care. ¹ Systematic evaluation of care to inform improvements in service quality is essential to reduce variation, improve outcomes and set benchmarks for good practice. ² Historically, the clinician's professional judgement was used as the principal indicator of high-quality care but robust measurement requires a more objective view centred on the patient's personal perspective.^3,4 Patient reported outcome measures (PROMS) are therefore required to complement the measurement of economic and organizational effectiveness.

Stigma remains an important reason why people do not present for HIV testing and it is estimated that 27% of the world's population with HIV are not aware of their status. ⁵ In the USA people who perceive high levels of stigma are more likely to report poor access to care or to not engage with healthcare services at all. ⁶ The benefits of early engagement with HIV services include improved individual patient survival and wider public health benefits associated with reduced infectivity following treatment. ³ A positive physician/patient relationship is also linked to better adherence to treatment and improved clinical outcomes. ⁶ It is therefore essential to obtain patients’ input regarding their care and use this feedback to improve services while recognizing that the characteristics and lifestyles of people with HIV vary significantly and that their care needs need to reflect this.

The objectives of the study were:

•

To design a validated tool to measure patients experience and satisfaction with their HIV outpatient care;

Methods

The questionnaire development and testing comprised the following stages, illustrated in Figure 1. OPEN IN VIEWER

Results

The 32 studies identified in the systematic review ³ used three main approaches to assess patient satisfaction; questionnaires; focus groups and interviews. No two studies used the same instrument and few studies met robust standards for validity and reliability. ³ Three potential questionnaires were identified^6–8 but were rejected as not addressing HIV-specific issues, being limited in scope, lacking meaningful service user involvement and/or not reflecting developments in HIV medical care. Data extraction and thematic analysis identified key areas of importance – doctors ⁷ and nurses ⁷ knowledge and expertise; perceived respect, dignity and autonomy; clear communication and information giving; and practical aspects of delivery of care such as the organization of the clinic, privacy and waiting times. Discussions from the four focus groups confirmed the relevance of these themes. The participants identified two additional issues:

•

The role of their primary care practitioner in providing care;

Ten participants were interviewed about their views on the questionnaire and what would encourage or discourage them from completing it. The majority (9/10) reported that completing a questionnaire was boring, a chore and a waste of time, but they were more likely to complete questionnaires if they believed that it would improve the clinic service or if their input would help other patients. Two patients had reading and writing difficulties. It was also reported that a person's mood and the length of waiting time at the clinic made a difference to whether they would be willing to accept a questionnaire to complete. Opinion was equally divided regarding completing a questionnaire in the waiting area or in a private room, because of concern over their privacy but anxiety about losing their place in the queue to see the doctor. Patients were willing to disclose sensitive and personal information providing they could be guaranteed anonymity, but agreed that if they were unsure about this then they might omit a question or even refuse to fill the questionnaire in at all.

All patients were in favour of short ‘tick box’ layouts, including questions requiring ‘yes’ and ‘no’ answers or short sentence replies to open questions. Rating scales and open-ended questions were disliked, but the opportunity to add comments at the end of the survey was welcomed. A questionnaire covering two sides of A4 or including 20 questions was considered an optimal length. A paper-based questionnaire was the most favoured format. The idea of computer touch-screen technology appealed to a few and the opportunity to take home or be mailed a questionnaire was also appealing to some patients but they admitted that they might lack motivation to complete and return it after leaving the clinic. Their reaction to completing a questionnaire sent via email or online was mixed. The convenience and privacy of completing an online form at home was balanced by concerns around Internet security and not wanting any HlV-related information to be linked to their computer. The majority of patients agreed that a financial or other incentive would not influence their decision to participate in a survey.

The data from the focus groups and interviews informed the design of a draft questionnaire. This followed the style and format of a previous satisfaction questionnaire developed for attendees at a sexual health clinic. ² Cognitive testing of the draft questionnaire was undertaken with a further 10 patients. Interviewees thought that the questionnaire was quite long, but acceptable. They found that the questions were easily understood, relevant and that the choice of response was appropriate. Some modifications were made to the wording of the questionnaire and re-routing of questions simplified. The refined questionnaire comprised seven subsections: before the appointment, arriving at the clinic, waiting for your appointment, during your appointment, tests, about the clinic and about you. In total 50 questions (including six demographic questions) were included using a tick box format with options to give further feedback if desired.

The questionnaire was piloted over a three-week period from December to January 2011–2012 in an outpatient HIV clinic in Birmingham, UK. During this time period 21 clinics were held with 490 patients attendances. One hundred survey packs were handed out and 80 were returned, giving a response rate of 80%. Ninety-one percent (n = 73) of the questionnaires were completed at the end of the patient's appointment and the remaining 9% (n = 7) of replies were returned by post. The characteristics of the respondents are shown in Table 1.

OPEN IN VIEWER

Table 1

Characteristics of questionnaire respondents

Characteristics	Questionnaire respondents N = 80 (%)
Gender
Male	24 (31 %)
Female	54 (69%)
No answer	2
Age
18–30 years	15(19%)
31–40 years	21 (27%)
41–50	31 (40%)
51–60	6 (8%)
61 years +	5 (6%)
No answer	2
Ethnic group
White/White British/other White	42 (55%)
Mixed	1 (1%)
Asian/Asian British	1 (1%)
Black/Black Caribbean	8(10%)
Black/Black African	24 (32%)
Other ethnic group	1 (1%)
No answer	3
Length of time attending clinic
First visit	13(17%)
Under six months	7 (9%)
Between 6 months and 2 years	10(13%)
Between 2 years and 5 years	20 (26%)
Over 5 years	27 (35%)
No answer	3

Excluding questions that were bypassed because of ‘go to’ instructions, four questions had an omission rate above 5%. Following review, routing instructions were clarified and additional answer options were inserted for three questions. The ‘question reached’ analysis showed that 96% of patients completed the full questionnaire, while one respondent reached question 21 and a further two reached question 44 suggesting that fatigue was not a major issue for those returning the questionnaire. It was not possible to conduct a factor analysis of the individual items due to the sample size, but pairwise correlations demonstrated a high correlation between arrival and waiting experience and the confidence placed in doctors and the experience of nurses. To minimize question redundancy, two questions were removed to create the final questionnaire (Appendix).

Discussion

Major themes that patients consider as important when attending an HIV clinic were identified following a systematic review of the relevant literature. ³ These themes were used to facilitate a discussion with current patients in a series of four focus group interviews, which informed the design of a draft patient satisfaction questionnaire. Concurrent work included interviews with patients about their views on completing a questionnaire which informed the format, layout and length of the instrument. Cognitive testing confirmed that the questionnaire was easy to complete and that the questions were relevant and understandable, but some additional routing instructions were required. The pilot survey conducted on a proportion of HIV clinic attendees demonstrated that there was a high completion rate. In response to the pilot two questions were removed. Pairwise correlations reflected the thematic structure of the questionnaire and supported good criterion validity. The combination of a systematic analysis of previous patient survey tools, focus group discussions and cognitive testing of the questionnaire was used to ensure high content validity. The questionnaire was found to be acceptable to patients and yielded a high completion rate.

Previous approaches to measure satisfaction among attendees at an HIV clinic have included the use of questionnaires, interviews and focus groups. Interviews and focus groups can have limited scope and relatively high cost. Three previously reported questionnaires were analysed for potential use, ³ but were not truly patient derived or did not reflect current practice. The first was a self-completion questionnaire ⁸ evaluating care in HIV outpatient clinics, but did not assess the effects of highly active antiretroviral therapy (HAART) on patient satisfaction. There is no published record of further use or subsequent development of this questionnaire. Secondly, a patient satisfaction survey for HIV ambulatory care ⁹ was developed by the New York State Department of Health AIDS Institute following an extensive literature review and a series of focus groups including both patients and staff. The resulting questionnaire consists of a core survey of 52 questions with additional modules covering different aspects of care such as Medicaid managed care and women's health services. It is therefore focused on a single model of health care and has not been assessed in a British or European setting. A third approach was the ‘Quote-HIV instrument ¹⁰ which was an HIV-specific version of the QUOTE (quality of care through the patient's eyes) questionnaire. A number of QUOTE instruments have been developed as part of the ‘Quality of care from the Patients Perspective’ project including those for asthma and rheumatoid disease. Although the questionnaire was rigorously developed and tested, a number of limitations were acknowledged by the authors including the self-selection of the patient sample and lack of representation from minority ethnic groups including migrants. A strength of the questionnaire was that patients were not only asked to rate the quality of various aspects of care, but also to rank their relative importance, which helps to prioritize subsequent interventions.

The HIV population is heterogeneous and generic standardized measures in the form of PROMs may lack precision. ¹¹ Various patient factors can affect which aspects of HIV care are identified as high priority, including gender, ethnicity, sexual behaviour, poverty and stigma. Different patient groups are likely to place variable importance on particular aspects of care and it is important to identify and quantify this heterogeneity to explore possible underlying causes, guide changes in service delivery, monitor changes in quality and allow a fair comparison of quality measures between different services. Patient involvement in developing these refined outcome measures is central to this process.

The development of this questionnaire benefited from robust methodological development, together with substantial input from a heterogeneous group of HIV clinic patients. Iteratively developed and appropriately piloted, it is likely that this survey tool provides a robust assessment of patient satisfaction that could be used as a patient reported outcome measure.

The questionnaire was developed in one HIV clinic and although many of the issues will be relevant to other HIV clinic attendees, variation in the population demographics, their beliefs and behaviours, together with different models of service delivery would suggest that further validation of the survey tool is needed. Reliability was not assessed by repeating the survey at different time intervals in the same patients as the questionnaires were anonymized, but acceptable internal consistency provided some assessment of this parameter.

Identifying areas of patient concern helps clinics prioritize new developments, and provides an opportunity to measure the quality of a service and its ability to meet benchmarks locally and nationally, in addition to supporting the commissioning of services. ¹² This locally validated questionnaire provides an initial step in HIV PROM development as a component of HIV clinical service evaluation.