Abstract
Professional bodies and healthcare commissioners recommend the provision of care for children and young people with sight impairment (SI) as a topic for quality improvement and service evaluation, but no unified evaluation tool is available. We aimed to develop and implement a tool to evaluate the quality of children’s low-vision services (LVS). We identified and reviewed available standards for LVS from government documents and surveys carried out by patient support organisations, developed a clinic proforma and six survey tools for families and professionals, and used these tools to carry out a first service evaluation of multidisciplinary LVS provided by children’s eye clinics, social services, and education teams for SI students in Bedfordshire and London, United Kingdom. The first service evaluation round showed that services provided are of high standard, but also identified specific areas for improvement such as provision of information in preferred format, formation of parent and young people support groups, and an unmet need for emotional support.
Introduction
In the United Kingdom, there are an estimated 25,000 children with sight impairment (SI) or severe sight impairment/blindness (SSI/BL; Morris & Smith, 2008). About 950 new cases are diagnosed each year (Bodeau-Livinec et al., 2007). SI can result in developmental delay by reducing the range of experiences the child is exposed to, and reduces vision-related and health-related quality of life, particularly when additional impairments are present (Chadha & Subramanian, 2011; Boulton, Haines, Smyth, & Fielder, 2006).
Children with SI may show difficulties with social interaction from infancy (Troster & Brambring, 1992) and are at risk of social communication difficulties and clinical autism spectrum disorders (Parr, Dale, Shaffer, & Salt, 2010). In addition, children/young people with SI face many challenges to participation in areas such as mobility, life at home, interpersonal interactions and relationships in later child- and teenagehood, major aspects of life, and leisure activities (Salminen & Karhula, 2014). Children with SI have lower numbers of activities, lower participation intensity, and more activities performed at home and with someone else (Engel-Yeger & Hamed-Daher, 2013). SI affects the quality of participation in one-to-one and larger group interactions (Caballo & Verdugo, 2007). Depression is common in adults with SI (Margrain et al., 2012), and adolescents with SI may have higher levels of anxiety than their peers (Bolat, Dogangun, Yavuz, Demir, & Kayaalp, 2011), lower self-esteem, and higher degrees of psychopathological symptoms (Garaigordobil & Bernaras, 2009). Systematic interviews with children/young people with SI have confirmed the impact on social relationships, participation and acceptance, independence and autonomy, psychological and emotional well-being, aspirations for the future, and their functioning at home, school, and during leisure activities (Rahi, Tadic, Keeley, & Lewando-Hundt, 2011) (Tadic, Hundt, Keeley, Rahi, & Vision-related Quality of Life, 2014).
Early assessment with provision of and training in the use of low-vision aids (LVAs) is essential to improve functional vision and adaptation to SI, so allowing most children to enter and remain in mainstream schools (Ducrey, Mathis, Goldschmidt, & Figueiredo, 1998; Silver & Gould, 1976) (Massof, 1998) (Labib, El Sada, Mohamed, Sabra, & Abdel Aleem, 2009; Metsiou, Papadopoulos, & Agaliotis, 2011).
Low-vision services (LVS) for children and young people and close co-operation between health, education, and social care professionals are vital to identify and support visually impaired children. The Royal College of Ophthalmologists recommends ‘the provision of care for children and young people with visual impairment’ as a topic for quality improvement and audit (Royal College of Ophthalmologists, n.d.). ‘Building on the QUIPP Agenda’ (UK Vision Strategy, n.d.) lists ‘Supporting children with sight loss’ as an important area in the commissioning of Eye Care and Sight Loss Services (UK Vision Strategy, n.d.). However, we were not aware of any suitable tool to audit these services, while evaluation of effectiveness and user-friendliness of LVS for adults are well established (Pearce, Crossland, & Rubin, 2011; Gillespie-Gallery, Conway, & Subramanian, 2012; Ryan, Khadka, Bunce, & Court, 2013).
The overall aim of this project was to develop a tool for multisource evaluation of multidisciplinary children’s LVS. Specific objectives were (1) to identify available tools, standards, and quality indicators; (2) to develop tools for multisource feedback, that is, the collection of information from clients and providers; and (3) to use these tools in a service evaluation, with a view to improving the tools for future rounds. To identify standards, we searched PubMed (search terms: Blindness, Vision/Low, Child Health Services, Adolescent Health Services, Health Services Evaluation) and the websites of the Royal College of Ophthalmologists, Department of Health, and Royal National Institute for Blind People (RNIB). We identified four sources of standards:
‘Certificate of Vision Impairment: Explanatory Notes for Consultant Ophthalmologists and Hospital Eye Clinic Staff’ (Department_of_Health, 2007; Table 1).
‘Recognising the needs of children and young people and their families’ (Children’s Subcommittee of the Low Vision Services Group (Vision 2020 UK), 2010);
‘Seeing it my way: A universal quality and outcomes framework for blind and partially sighted people’ (RNIB, 2012), the result of a wide public consultation on outcomes relevant to people with sight impairment (Table 2).
‘RNIB LVS Assessment framework’ (RNIB, 2009). This tool was commissioned by the RNIB and developed by the Low-Vision Services Model Evaluation (LOVESME) collaborative. It was developed from recommendations of the Low Vision Services Consensus Group, the Low-Vision Working group, a review of the peer reviewed literature (Dickinson et al., 2011), and visits to a variety of LVS.
Certification standards. From Certificate of Vision Impairment: Explanatory Notes for Consultant Ophthalmologists and Hospital Eye Clinic Staff, Department of Health, 2007.
Summary of outcomes defined by clients. From ‘Seeing it my way: A universal quality and outcomes framework for blind and partially sighted people’. RNIB/Vision2020 UK; published 2012.
In addition, the document ‘Ophthalmic services for children’ published by the Paediatric Subcommittee of the Royal College of Ophthalmologists (Royal College of Ophthalmologists, Paediatric Subcommittee, 2012) offers child-specific guidance and includes several of the standards listed in Table 1: All eligible children and young people should be offered certification as sight-impaired or severely sight-impaired where indicated; they should received timely, co-ordinated, and family-centred services based on assessed needs and promoting social inclusion; visual function and co-existing disabilities should be assessed and their educational impact determined; counselling should be available to families who want it; and parents should receive information in a language that they can understand.
Methods
We divided this project into two stages: tool development and service evaluation. We developed six tools based on the standards identified (Table 1): two for parents, three for children and young people, and one for professionals.
We then used these tools to carry out a service evaluation at Moorfields Eye Hospital in London and Bedford, addressing the following questions:
Is SI certification offered to all eligible children and young people?
Do our LVS meet relevant standards described in ‘Seeing it my way’ (RNIB, 2012), as determined by service users and parents?
Do our LVS meet relevant standards described by the Children’s subcommittee of the LVS Group (Vision 2020 UK; Children’s Subcommittee of the Low Vision Services Group (Vision 2020 UK), 2010) and the ‘RNIB LVS Assessment framework’ (RNIB, 2009), as determined by professionals?
Service evaluation/question 1: SI Certification
We carried out a prospective service evaluation of all children attending our children’s eye clinics between 01 May 2013 and 31 July 2013. A paediatric nurse recorded best-corrected visual acuity on a service evaluation proforma, if it fell below the certification threshold of 6/18 (0.48 logMar) recommended by the Department of Health (DH). At the end of the clinic, she ascertained whether certification had been offered.
Service evaluation/question 2: Service user and parent/carer survey
Over the same period, we carried out a prospective survey of children/young people with SI and their parents/carers. We also asked about the usability of our tools, specifically whether any of the statements were redundant.
Service evaluation/question 3: Professional view of service provision
In May 2013, we invited all teams of professionals involved in the care of children with SI in Bedfordshire and at Moorfields Eye Hospital in London to complete a questionnaire, either as a team or as individuals. This survey asked about compliance to the standards described by the Children’s subcommittee of Vision 2020 UK and RNIB LVS assessment framework.
Results
Tool development
Tools for families
We developed five tools based on the outcomes listed in ‘Seeing it my way’ (Supplementary Material 1). We designed two tools for parents/carers, one for those with a child newly diagnosed as SI, and one for those whose child had been diagnosed in the past, and who therefore had longer experience with the LVS. For children and young people, we devised three tools, for age groups 5–9, 10–14, and 15–19 years. Each tool consisted of a series of statements to which the respondents were invited to choose ‘yes/no’. The number of statements differed between tools, as not all outcomes were relevant for the younger age groups. The tools were presented in English, in black print on white paper, using Arial font size 14.
Tool for professionals
The tool we developed from the standards set by the Children’s subcommittee of the LVS Group (Vision 2020 UK; 2010) contained 28 items (Supplementary Material 2). The RNIB LVS Assessment framework consists of 12 sections and includes a total of 83 questions. We used the proforma as downloaded from the internet, without any modifications.
Feedback on tools
Service users and parents/carers
A total of 15 proformas were completed: (a) parents of newly diagnosed children: 4; (b) parents of previously diagnosed children: 6; (c) young people age 15–19 years: 1; (d) young people age 10–14 years: 3; and (e) children age 5–9 years: 1. This constituted a response rate of 100%. None of the parents/carers were themselves sight impaired.
Service users felt that none of the statements should be removed. The proportion of statements left blank or marked as ‘not applicable’ were (a) 4%, (b) 39%, (c) 9%, (d) 0, and (e) 10%.
Professionals
Twenty-five tools were returned. These were completed by the children’s eye care team at one hospital (orthoptists, optometrist, paediatric nurse, play specialist, and ophthalmologist), 10 individual paediatric ophthalmologists, 4 optometrists, 5 paediatric nurses, 2 orthoptists, 2 education teams, and 1 social service team for children with SI. Every team and paediatric optometrist we approached completed the questionnaire; the response rate for individual paediatric ophthalmologists was 71%.
Service evaluation: Does the current service meet the standards?
Are all eligible children/young people offered certification? Proformas were completed for 14 children/young people with SI
Of these, 7 had previously been certified as SI and 4 were certified during their visit. Three parents declined certification.
Survey of service users and parents/carers
When asked about how well professionals had communicated with them about their child’s problem, 93% of statements selected by parents of newly diagnosed children were positive; only one statement of ‘not applicable’ was selected (4%), as the development of the care plan was still ongoing and the parents felt that they did not yet understand the full impact of the diagnosis. In tools completed by parents of previously diagnosed children, 38% of answers were positive, 23% negative, and 39% of items either not answered at all or selected as ‘not applicable’. Examples of questions not answered were ‘help to access emotional support, information about local/national services, access to life skills and technologies, information about financial benefits, understanding of the child’s school/nursery of the eye condition and provision of information in appropriate formats’. The young person completing the tool for 15- to 19-year-olds gave 48% positive answers, 43% negative, and 9% ‘not applicable’. The three young people age 10–14 years gave 64% positive and 36% negative answers; no items were selected as ‘not applicable’. The one child who completed the tool for 5- to 9-year-olds selected positive statements in 70%, negative in 20%, and gave no answer or chose ‘not applicable’ in 10% of items.
Parents made the following suggestions for service improvement: providing information in their preferred format (17%), providing help to access services when feeling low (17%), starting a parent support group (50%), providing information on financial benefits (33%), providing a cohesive LVA service for children (33%), advice on free and low cost technology solutions (67%), and life skills and mobility training (50%). Young people made the following suggestions: providing more information about eyes, emotional support, peer support groups, transport, LVS, technologies, Braille, and school transitions.
Survey of professionals
The staff survey based on the LVS standards showed 53% positive answers, 8% negative; 35% of items were either not answered or selected to be ‘not applicable’ to the respondent. Respondents felt that no item should be eliminated. Not answered statements varied depending on the respondent’s setting and role. The statements which were most frequently marked in the affirmative concerned the provision of a child-friendly environment, documentation of refusal of certification or referral to other members of the LV team, provision of safeguarding measures in cases of refusal of referral to LVS, shared information when consent was given, respect for the child’s view, and maintenance of confidentiality. Areas for improvement were referral triage within 10 days of receipt, provision of information in accessible format, and development of a shared care record.
Four teams completed the survey of compliance with the RNIB LVS standards: a hospital children’s eye care team, two teams of VI teachers, and an optometrists’ LVA clinic team. No items were recommended for removal or addition, but 38% of items were not responded to (6–42, depending on team), as not all statements are relevant to all teams. In the staff survey using the RNIB LVS audit tool, 48% of responses were positive and 14% negative. Gaps in service provision identified were the absence of a General Ophthalmic Services contract for a paediatric LVA clinic, poor signposting of services within their respective buildings across all sites, and lack of counselling services.
Discussion
Tool development
The development of tools to evaluate patient-reported outcomes should begin with interviews with service users and/or focus groups, interviews with professionals, and a search of the published literature (Pesudovs, Burr, Harley, & Elliott, 2007). However, we feel that available documents reflect the opinions of a large number of vision-impaired people and were sufficient for our purpose. The content validity of the instruments we have developed based on these standards can be expected to be high, as all items are derived from discussions with those affected by low vision. A limiting factor is that no interviews specifically with younger children were carried out. Future developments of the tools should ensure that adequate numbers of children of each age group are consulted. Work on the development of tools to assess vision-related quality of life and functional vision has demonstrated that young children have important contributions to make (Khadka, Ryan, Margrain, Court, & Woodhouse, 2010; Rahi et al., 2011; Tadic et al., 2014). Families and carers completing our questionnaires did not feel that other items needed to be added, and also did not recommend items for removal. This may support the validity of our tools. However, the fact that up to 39% of statements were not answered indicates that they were not relevant to the respective respondents, which may reflect limited validity. As different respondents felt that different statements were not applicable, it was at this point not possible to determine which items could be removed. Item reduction based on stringent criteria or Rasch analysis would help deciding which items to keep and would increase the validity of the remaining tool (Khadka et al., 2010) (Cochrane, Marella, Keeffe, & Lamoureux, 2011; Tadic, Cooper, Cumberland, Lewando-Hundt, & Rahi, 2013). It would be desirable to develop one single tool for multisource feedback, rather than a multitude of tools for different groups. The voice of service users, that is, children and young people, should be placed at the heart of such a tool, and an evaluation by focus groups of children and young people would be the most appropriate continuation of the work described here.
As for professionals evaluating the service they provide, it is possible that they may have an unrealistically positive impression. However, a high level of concordance was seen between the different professionals completing the survey.
The tool should also be evaluated at multiple sites across the United Kingdom, as findings reported here are based on two sites in England only. All standards we identified should be valid across the United Kingdom, but healthcare and educational services and their interaction differ between regions.
Service evaluation
Our service evaluation demonstrated that the services provided are of high standard, but also identified areas for improvement. All eligible young people and children had been offered certification as sight impaired/severely sight impaired as appropriate. The analysis of user satisfaction compared to the standards was hampered by the large proportion of items left blank or selected as ‘not applicable’. Negative items, that is, lack of information or dissatisfaction with services were selected by 4% and 23% of parents and carers of newly diagnosed and previously diagnosed children, respectively. The proportion of negative items chosen was far higher in children and young people of 15–19 years and 10–14 years of age: 43% and 36%, respectively. This may reflect a neglected need to inform children and young people directly and appropriately about their diagnosis, management, and available support for education, career, and social life, as well as a lack of resources targeted at these age groups, such as age-specific counselling. These findings have led to a number of changes in our own services: We now ask for the families’ preferred format for information at each point of contact, and a parent support group has started at a local low-vision charity, which also provides information about optical LVA and computer-based assistive technologies. Emotional support remains a problem due to funding limitations. Development of a shared care record remains difficult, as National Health Service (NHS), education, and social services teams use different electronic patient/client records and software packages.
Footnotes
Acknowledgements
The research was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. We wish to thank all members of staff who contributed to the successful implementation of the service evaluation.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. We acknowledge the support of the NIHR Comprehensive Research Network Central and East London.
