Abstract
This is the first systematic review of assessments of cortical or cerebral visual impairment (CVI) feasible for use by occupational therapy practitioners.
Cortical or cerebral visual impairment (CVI), which cannot be explained by a problem with the eyes and is caused by damage to the parts of the brain that process vision, is thought to be the leading cause of visual impairment among children in developing and developed countries (Chang & Borchert, 2020). It is known to result in detrimental health effects and has often been overlooked (National Eye Institute, 2021). CVI may include characteristics that cannot be explained by an eye exam, such as difficulty making eye contact, absence of visually guided reach, and tripping over objects on the floor. These characteristics frequently affect daily occupations, such as recognizing friends on a playground, finding the way to the cafeteria in school, or matching shapes in a puzzle. In early intervention and school settings, children with CVI may exhibit delays in the domains of gross and fine motor skills, mobility, socialization, and other areas that affect participation in daily routines and occupations. Furthermore, visual function can change on the basis of temporal or environmental factors, such as time of day or crowded spaces.
Although survival of traumatic birth events and prematurity has increased, associated brain injuries can result in CVI (Chang & Borchert, 2020). Therefore, CVI has garnered more attention from the international scientific, optometric, and ophthalmic communities, particularly because the traditional defining characteristics used by eye care providers to diagnose low vision and blindness do not align with the visual dysfunctions typically exhibited by those with CVI (Kran et al., 2019). Several instruments have been developed to identify possible CVI in children (Chang & Borchert, 2020; McConnell et al., 2021). These instruments include clinician-reported outcome measures (ClinROMs), in which professional judgment is used to rate clients’ verbal responses or behaviors; patient-reported outcome measures (PROMs), in which clients provide direct responses via questionnaires or interviews; and performance-based outcome measures (PerfOMs), in which clients perform defined and objective tasks, and clinicians record client performance (Walton et al., 2015). Many, but not all, of the currently available instruments include items that pertain to the child’s engagement in daily occupations; however, to our knowledge, none are grounded in an occupational therapy theory, framework, or model of practice.
To promote optimal development of vision among children with complex visual conditions, including CVI, it is essential that occupational therapy practitioners use evidence-based assessment and intervention techniques. Evidence-based assessment requires the appropriate interpretation and use of assessment data gathered via CVI outcome measures, which, in turn, requires understanding of the measurement properties of these instruments. Measurement properties include various forms of validity (i.e., the degree to which an instrument appropriately, comprehensively, and accurately measures a particular construct) and reliability (i.e., the consistency of an instrument). Responsiveness, which refers to “the ability to detect change over time in the construct to be measured” (Mokkink, Terwee, Gibbons, et al., 2010, p. 5), is also an important measurement property. Studies of validity, reliability, and responsiveness can inform clinicians as they choose assessments for particular purposes; however, the quality of the measurement studies themselves must also be considered.
The purpose of this systematic review was to appraise the measurement properties of CVI assessments that can be used in occupational therapy and make recommendations for practice and research. Although occupational therapists use a wide variety of assessments to evaluate aspects of occupational performance and functional vision in children, this review was limited to outcome measures that were specifically designed to identify the presence or absence of behaviors linked to CVI. Our specific research questions were as follows: (1) What is the evidence for the measurement properties of currently available CVI instruments that are applicable to occupational therapy practice and (2) what is the clinical utility of these instruments?
Method
A comprehensive search for assessments and interventions that are feasible for an occupational therapy practitioner to perform when working with children who have or are suspected of having CVI was carried out using MEDLINE via PubMed, Scopus, and Web of Science from the date of inception to October 7, 2020. Reference lists of identified articles were also searched. The search strategy included “cerebral visual impairment” OR “cortical visual impairment” OR “cortical vision impairment.”
Inclusion and Exclusion Criteria
Articles were included if they met the following criteria: (1) reported original research published in English, (2) included pediatric participants, and (3) evaluated a CVI assessment or intervention applicable to occupational therapy practice. Articles were excluded if they (1) reported conference proceedings, retrospective studies, observational studies, or prevalence and incidence; (2) reported on assessments of visual or brain functions that would be outside the scope of occupational therapy practice (e.g., preferential looking, electrophysiological responses); or (3) did not report measurement properties of the assessment tool.
Although the original search included both intervention and assessment articles, in this systematic review we report only on the assessment articles.
Procedure
The first and third authors (Orli Weisser-Pike, an occupational therapist, and Leah Cordova, a health librarian) screened the titles and abstracts of retrieved articles. After a blind review, they met to discuss any discrepancies and reach a consensus. Consensus was obtained through the combined review and discussion of individual articles, taking into consideration the inclusion and exclusion criteria determined before study initiation. Weisser-Pike and Cordova completed a blinded full-text review of the remaining articles. Once again, they met to discuss any discrepancies to reach consensus. The second author (Anita Witt Mitchell, an occupational therapist, who did not participate in the original full-text review) assisted in triangulation by reviewing approximately one-third of the full-text articles to help reach consensus. The articles were then categorized as assessment or intervention articles.
Data Extraction and Quality Appraisal
Weisser-Pike and Mitchell used a structured form to extract the citation information, name of each assessment studied, assessment development data (if included), and details of the measurement data gathered in the study (e.g., sample size, comparison tests used, statistics used, statistical results). We independently used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) Risk of Bias checklist (Mokkink et al., 2020, 2021; Mokkink, de Vet, et al., 2018; Mokkink, Prinsen, et al., 2018; Terwee et al., 2018) to assess the methodological quality of the studies. The COSMIN Risk of Bias checklist provides specific standards for rating aspects of validity and reliability studies, as well as studies of test development and responsiveness. The checklist specifies criteria for rating each standard as “very good,” “adequate,” “doubtful,” “inadequate,” or “N/A” (not applicable). The lowest rating on any standard is considered the overall quality of the study. The COSMIN Risk of Bias checklist has been recommended for use in systematic reviews of measurement studies to assist practitioners in choosing the best instrument (Mokkink, Terwee, Gibbons, et al., 2010). Mokkink, Terwee, Gibbons, et al. (2010) studied the interrater agreement and reliability of the COSMIN Risk of Bias checklist and found that the overall percentage agreement was acceptable (68% of the ratings were above 80% agreement), although 61% of the κ coefficients were below .40. They recommended that the checklist be completed by two independent raters and that the raters reach consensus on the final rating, as we did in this study. After independent review, Weisser-Pike and Mitchell met to review each of their COSMIN Risk of Bias checklist ratings, discuss discrepancies, and reach consensus.
After the risk-of-bias appraisal, Weisser-Pike and Mitchell independently rated the quality of the studies using the COSMIN Guidelines for Systematic Reviews of Patient-Reported Outcome Measures (Prinsen et al., 2018). These guidelines outline a methodology for making evidence-based recommendations based on the quality of PROMs. ClinROMs and PerfOMs can also be evaluated using the same methodology (Mokkink et al., 2021). In addition, for the ClinROMs and PerfOMs, Weisser-Pike and Mitchell used the COSMIN Risk of Bias tool to assess the quality of reliability and measurement error studies, as recommended (Mokkink et al., 2021). The studies of each measurement property were independently rated as “sufficient,” “insufficient,” or “indeterminate,” according to the updated criteria for good measurement properties (Prinsen et al., 2018). Using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, modified for studies of measurement properties, we graded the quality of each study as “high,” “moderate,” “low,” or “very low,” according to the following factors: risk of bias, inconsistency, imprecision, and indirectness (Mokkink, Prinsen, et al., 2018; Prinsen et al., 2018). Inconsistencies were resolved by discussion until consensus was reached.
Results
We found 15 studies (published in 14 articles) that reported the measurement properties of 12 outcome measures appropriate for use by occupational therapy practitioners. One publication was included in the systematic review although it was published as correspondence with the salutation Sir (Macintyre-Béon et al., 2012). This letter to the editor reported the measurement properties of an instrument and was used as a basis for the development of four other instruments (Gorrie et al., 2019; Hellgren et al., 2020; Macintyre-Béon et al., 2013; Tsirka et al., 2020) that were examined in articles that did meet our inclusion criteria. Because of the number of times the letter was cited as the foundation for the development of subsequent outcome measures, we decided to include it in our review, even though we could not ascertain that it was published as a peer-reviewed manuscript. Figure 1 represents the results of the search strategy and screening process.
PRISMA flow diagram of the systematic review process.
Using the COSMIN guidelines, we categorized 4 of the outcome measures as ClinROMs, PerfOMs, or both and 8 as PROMs, expanding the definition of patient to include a parent or caregiver. The instruments ranged in number of test items from 5 (Five Questions) to 52 (Insight Inventory). Table A.1 in the Supplemental Material (available online with this article at https://research.aota.org/ajot) provides an overview of the outcome measures included in the systematic review.
Table A.2 in the Supplemental Material summarizes the COSMIN Risk of Bias checklist ratings for the reported measurement properties of each outcome measure, listed in reverse chronological order. Table A.3 reports a summary of the overall appraisal of the evidence for each outcome measure’s measurement properties using the COSMIN and modified GRADE approaches.
As noted earlier, a PROM described in a 2012 letter to the editor by Macintyre-Béon et al. served as the basis for instruments described in other articles, published between 2013 and 2020, included in this systematic review (Gorrie et al., 2019; Hellgren et al., 2020; Macintyre-Béon et al., 2013; Tsirka et al., 2020). Each outcome measure was made up of a different number of items, ranging from 5 to 48. A substantial number of the items in these PROMs were identical across the outcome measures, but omissions and additions were noted for each instrument (R. Bowman, personal communication, November 21, 2022; Gorrie et al., 2019; Hellgren et al., 2020; Macintyre-Béon et al., 2013; Tsirka et al., 2020). Currently, no published articles provide a description of the development or content validity of these instruments; thus, we could not assume that they were updated editions of each other. Two of the four PROMs were referred to by formal names, the Five Questions (Gorrie et al., 2019) and the Insight Inventory (Tsirka et al., 2020), but the original instrument and two other PROMs were not referred to by specific names. For clarity, in this article we refer to the original 51-item outcome measure as the Structured History Taking Inventory (Macintyre-Béon et al., 2012), the 48-item instrument described in Macintyre-Béon et al. (2013) as the CVI Inventory, and the 39-item PROM as the EXPRESS CVI Inventory (Hellgren et al., 2020).
Patient-Reported Outcome Measures
Flemish CVI Questionnaire–CVI Questionnaire
The Flemish CVI Questionnaire–CVI Questionnaire (Ben Itzhak et al., 2020; Gorrie et al., 2019; Ortibus et al., 2011) is a tool that has been studied since 2011, with the most recent analysis published in 2020. No published studies to date have provided evidence of its content validity. Two studies rated as adequate provided evidence of its construct validity via principal-components analysis (Gorrie et al., 2019) and exploratory factor analysis (Ben Itzhak et al., 2020), but the COSMIN guidelines for systematic review require confirmatory factor analysis for structural validity. High-quality evidence supports the tool’s internal consistency reliability (Gorrie et al., 2019), with the exception of the Parallel Processing in Multi-Tasking subscale, which has an internal consistency reliability below .70. A study rated as very good (Gorrie et al., 2019) showed convergent validity of the Flemish CVI Questionnaire with the Five Questions. A study (Ortibus et al., 2011) investigating its convergent validity with the L94 Visual Perceptual Battery, the Test of Visual Perceptual Skills—Revised (a nonmotor test of visual perception), and the Visual Perception subtask of the Beery–Buktenica Developmental Test of Visual– Motor Integration (Beery™ VMI) also demonstrated acceptable convergent validity; however, the study was rated as inadequate, due to lack of reporting of the measurement properties of the comparator instruments. Both Gorrie et al. (2019) and Ortibus et al. (2011) conducted studies of criterion validity rated as very good, but this evidence could not be appraised because it did not meet the COSMIN and modified GRADE criteria. To date, no studies of the test–retest reliability of the Flemish CVI Questionnaire–CVI Questionnaire have been published.
Insight Inventory
Studies of the Insight Inventory (Tsirka et al., 2020) have tested its convergent validity with ophthalmic features such as visual fields and visual inattention; the Benton Facial Recognition Test and the Benton Judgment of Line Orientation; domain scores of the Beery VMI; the Processing Speed and Perceptual Reasoning Index tests of the Wechsler Intelligence Scale for Children, Fourth Edition; and the Wechsler Preschool and Primary School Scale of Intelligence, Fourth Edition, subtests and full-scale IQ score. Of the 22 hypotheses, 4 were confirmed and 18 were not confirmed, yielding an overall rating of insufficient evidence. The evidence quality was rated as low because of a serious risk of bias and imprecision. Test–retest and internal consistency reliability evidence or factor analysis evidence of its construct validity are lacking.
EXPRESS CVI Inventory
The EXPRESS CVI Inventory (Hellgren et al., 2020) was modified from the Structured History Taking Inventory (Macintyre-Béon et al., 2012), which we report on later. In a study rated as very good (Hellgren et al., 2020), the EXPRESS CVI Inventory demonstrated the ability to predict stereoacuity, motion coherence, and form coherence accuracy, as well as scores on the Beery VMI (6th ed.). However, on the basis of COSMIN standards (Mokkink, Terwee, Knol, et al., 2010), these comparator tests are not considered gold standards for measuring CVI behaviors, and therefore this study was not appraised using the COSMIN and modified GRADE criteria (Mokkink, Terwee, Knol, et al., 2010). Hellgren et al. (2020) also investigated construct validity by comparing scores of children who were born extremely preterm with scores of children who were full term. This aspect of the study was rated as adequate; however, it did not include a statistical comparison of the two groups’ scores, and therefore it was not appraised using the COSMIN and modified GRADE criteria. The EXPRESS CVI Inventory also lacks confirmatory factor analysis evidence of construct validity, internal consistency reliability evidence, and test–retest reliability evidence.
Five Questions
The Five Questions (Gorrie et al., 2019) is composed of items that were taken from the Structured History Taking Inventory (Macintyre-Béon et al., 2012). Gorrie et al. (2019) conducted an adequate exploratory factor analysis to examine construct validity and found that all five questions loaded onto one factor that explained 71.5% of the variance. Confirmatory factor analysis has not been performed on the Five Questions; therefore, despite a very good COSMIN Risk of Bias rating for internal consistency reliability, the overall rating is indeterminate but high quality, based on the COSMIN and modified GRADE guidelines. One study rated as very good reported convergent validity with the Flemish CVI Questionnaire. No published studies have investigated the test–retest reliability of the Five Questions.
Preverbal Visual Assessment
A study of the Preverbal Visual Assessment (Pueyo et al., 2014) provided sufficient evidence of content validity, although the evidence was rated as low quality because of serious risk of bias. Pueyo et al. (2014) surveyed parents about the relevance and comprehensibility of the assessment but not about its comprehensiveness. They also did not survey professional content experts about the instrument’s relevance or comprehensiveness. The study also reported very good evidence of internal consistency reliability for all four subtests of the Preverbal Visual Assessment; however, because of the lack of confirmatory factor analysis, this moderate-quality evidence is considered indeterminate. Although a study rated as adequate reported that the Preverbal Visual Assessment had strong test–retest reliability (García-Ormaechea et al., 2014), it did not meet the required criteria specified in the COSMIN systematic review guidelines (Prinsen et al., 2018).
CVI Inventory
The CVI Inventory (Macintyre-Béon et al., 2013) is a modified version of the Structured History Taking Inventory, published as a letter to the editor the previous year (Macintyre-Béon et al., 2012). In a study rated as very good, a statistically significant difference was found on 18 of 48 items between children born at term and children born preterm; however, the analysis reported did not meet the COSMIN criteria for rating construct validity evidence. The CVI Inventory currently lacks construct validity evidence, internal consistency reliability evidence, and test–retest reliability evidence.
Structured History Taking Inventory
The Structured History Taking Inventory (Macintyre-Béon et al., 2012) was included in this review because it formed the basis for subsequent outcome measures. No content validity evidence was provided in the letter to the editor, nor was a confirmatory factor analysis reported, resulting in an overall rating of insufficient evidence, which was of very low quality for the internal consistency reliability of this instrument’s subtests. Sufficient test–retest reliability was reported; however, the study was rated as very low quality because of imprecision and serious risk of bias.
Functional Questionnaire
An exploratory factor analysis was performed to examine the construct validity of the Functional Questionnaire (Ferziger et al., 2011). The study identified two factors (Task-Oriented Visual Function and Basic Visual Skills), accounting for 81.12% of the variance; however, it earned a COSMIN Risk of Bias rating of inadequate because of its small sample. Because confirmatory factor analysis is required to meet the COSMIN systematic review guidelines criteria for structural validity and internal consistency, the overall rating of internal consistency was indeterminate (moderate-quality evidence due to imprecision). A convergent validity study with the Visual Classification Scale and the Coding Interactive Behavior rating system was rated as inadequate because of a lack of information about the comparator instruments. Ferziger et al.’s (2011) study, rated as very good, compared children with CVI with children without CVI and children with optic atrophy (known groups). It provided sufficient evidence of construct validity, but the evidence was rated as low quality because of imprecision and serious risk of bias. In addition, sufficient evidence of test–retest reliability was reported, although the evidence was low quality as a result of imprecision. Although the COSMIN Risk of Bias checklist does not provide criteria for appraising interrater reliability, Ferziger et al. (2011) also reported acceptable interrater reliability for the Functional Questionnaire.
Clinician-Reported and Performance-Based Outcome Measures
We classified two instruments as PerfOMs: the Austin Playing Card Assessment (McDowell, 2020) and the Children’s Visual Impairment Test (Vancleef et al., 2020a, 2020b). Two instruments were primarily ClinROMs, although each had elements of other types of outcome measures. The Structured Clinical Question Inventory (Philip et al., 2016) is similar to a PROM because it relies on answers given by the child’s caregiver; however, the clinician can revise the answers according to their observations. The CVI Range (Newcomb, 2010) is primarily a ClinROM, but it also includes performance-based items as would be found in a PerfOM, for example, having the child point out salient features of an image. It should be noted that the CVI Range does not have a standardized protocol or assessment kit. For the purposes of this review, we have classified both the Structured Clinical Question Inventory and the CVI Range as ClinROMs.
Austin Playing Card Assessment
A study of the Austin Playing Card Assessment’s construct validity demonstrated its ability to discriminate between children with CVI and controls (known groups) on visual matching tasks; however, this evidence was rated as low quality as a result of imprecision. To date, there are no published studies of the internal consistency or test–retest reliability of the Austin Playing Card Assessment. Although the COSMIN Risk of Bias checklist does not include criteria for appraising studies of interrater reliability, we note that one study found acceptable interrater reliability for this instrument (McDowell, 2020).
Children’s Visual Impairment Test
Of the four ClinROMs or PerfOMs, only the Children’s Visual Impairment Test was subjected to confirmatory factor analysis (Vancleef et al., 2020b). Although this study provided sufficient evidence for the instrument’s structural validity, the study’s COSMIN Risk of Bias rating was inadequate because it did not meet the sample size criterion. Two studies (Vancleef et al., 2020a, 2020b) rated as very good provided sufficient evidence that the Children’s Visual Impairment Test can discriminate between younger and older children who are typically developing, as well as between children with and without CVI, children with CVI and children with intellectual impairments, and children with CVI and children with simulated poor visual acuity (Vancleef et al., 2020b).
As a result of a lack of reporting of the measurement properties of the comparison measures, there was inadequate evidence for the Children’s Visual Impairment Test’s convergent validity when scores were correlated with scores on the L94 Visual Perceptual Battery. Similarly, studies of the Children’s Visual Impairment Test’s divergent validity (reported in Vancleef et al., 2020b) did not report the measurement properties of the comparator instruments, resulting in inadequate COSMIN Risk of Bias ratings. These divergent validity studies correlated scores of the Children’s Visual Impairment Test with scores on tests of visual acuity, visual–motor skills, and social skills (Vancleef et al., 2020b). Although the results of the convergent and divergent validity studies were consistent with the hypothesized correlations and therefore sufficient according to the COSMIN systematic review guidelines, the lack of reporting of measurement properties of the comparator instruments created a serious risk of bias and, along with imprecision, resulted in a low-quality rating for the construct validity evidence.
There is sufficient evidence of acceptable test–retest reliability of the Children’s Visual Impairment Test total test score, although evidence for three of its four subtests is insufficient (moderate-quality evidence; Vancleef et al., 2020b). We should also note that the study of the Children’s Visual Impairment Test was rated as inadequate because of a lack of internal consistency reliability data for its subscales (Vancleef et al., 2020b).
Structured Clinical Question Inventory
Only one of the four outcome measures classified as a ClinROM or a PerfOM, the Structured Clinical Question Inventory (Philip et al., 2016), reported evidence of content validity, which was rated as doubtful and inadequate because only ophthalmologists served as content experts, and clients were not surveyed about the relevance, comprehensiveness, or comprehensibility of the items (Mokkink et al., 2021; Terwee et al., 2018). An exploratory factor analysis was conducted for the inventory (Philip et al., 2016) and was rated as adequate; however, only confirmatory factor analysis met the COSMIN systematic review guidelines for structural validity evidence (Prinsen et al., 2018). A very good study of the Structured Clinical Question Inventory demonstrated internal consistency but was rated as indeterminate because of the lack of confirmatory factor analysis (Prinsen et al., 2018).
CVI Range
The CVI Range demonstrated very good, high-quality evidence of internal consistency, but the lack of confirmatory factor analysis studies of this instrument resulted in an indeterminate rating of its internal consistency (Newcomb, 2010). There is sufficient evidence of acceptable test–retest reliability of the CVI Range (low-quality evidence).
Discussion
General Recommendation for Use
Overall, the evidence for the measurement properties of currently available assessments of CVI that are appropriate for use by occupational therapists is limited. Content validity evidence was available for only two of the instruments, the Preverbal Visual Assessment (a PROM) and the Structured Clinical Question Inventory (a ClinROM). Prinsen et al. (2018) described content validity as the most important measurement property, because it provides evidence of the tool’s relevance, comprehensiveness, and comprehensibility. To have the potential to be recommended as most suitable for use, the COSMIN systematic review guidelines require sufficient content validity, regardless of the quality of the evidence, and at least low-quality evidence of sufficient internal consistency. Although the evidence for the Preverbal Visual Assessment’s content validity was sufficient, its internal consistency was indeterminate. Both the content validity evidence and the internal consistency evidence for the Structured Clinical Question Inventory were indeterminate. Despite this lack of evidence supporting use of these two instruments, note that the COSMIN guidelines only recommend against use of an outcome measure if the evidence of an insufficient measurement property is of high quality (Prinsen et al., 2018). Neither of these outcome measures, and none of the others included in this review, met these criteria, and thus all have the potential to be recommended for use if future research supports their measurement properties.
Internal Structure of the Instruments
After appraisal of content validity, the COSMIN approach to systematic reviews of outcome measures involves appraising the instruments’ internal structure. Included as part of the appraisal of the internal structure are structural (construct) validity, internal consistency, and cross-cultural validity. The guidelines require confirmatory factor analysis to verify structural validity (i.e., the relationships between the items and the quality of the individual items for measuring the intended construct; Prinsen et al., 2018). As our results show, there is little confirmatory factor analysis evidence for assessments of CVI. In fact, the only published evidence of structural validity is for one PerfOM, the Children’s Visual Impairment Test. Although the structural validity evidence for this test was sufficient, it was of low quality, providing limited confidence in the findings.
Internal consistency reliability evidence has been published for 8 of the 12 outcome measures; however, until confirmatory factor analysis studies are performed on the instruments, their internal consistency reliability is considered indeterminate (Prinsen et al., 2018). The one exception, the Children’s Visual Impairment Test, had sufficient internal consistency reliability, but the quality of the evidence was low as a result of indirectness and serious risk of bias. We note that there is high-quality evidence of insufficient internal consistency of the Parallel Processing in Multi-Tasking subtest of the Flemish CVI Questionnaire–CVI Questionnaire, suggesting that this subtest should only be used as part of the whole assessment, and caution should be used when considering scores on this individual subtest.
Prinsen et al. (2018) also considered cross-cultural validity as an aspect of structural validity. Despite the fact that several of the identified outcome measures were developed and studied in countries and cultures outside of the United States, currently no published cross-cultural validity studies of any of these instruments exist. Even though the research studies included in this systematic review were reported in the English language, we were unable to ascertain the language of the instrument used in the study. Practitioners who use CVI assessments outside the assessments’ countries of origin should do so with caution, because cultural differences could affect instructions, wording, and activities used as part of the measure.
Test–Retest Reliability
Despite the lack of evidence supporting the internal structure of the CVI assessments reviewed in this study, we also appraised the evidence related to other measurement properties, because they provide evidence of the overall quality of the instruments and their subscales, beyond the item level (Prinsen et al., 2018). Only four of the instruments reported evidence of test–retest reliability, and none of this evidence was high quality, suggesting that caution should be used when using these outcome measures to measure change over time. Of particular note is that three of the four subtests of the Children’s Visual Impairment Test demonstrated insufficient test–retest reliability (moderate-quality evidence), indicating that these subtests should not be used individually to measure progress. In addition to the lack of test–retest reliability evidence, none of the CVI assessments currently have any published evidence of responsiveness.
Hypothesis Testing for Construct Validity
Evidence for convergent validity, divergent validity, or both, as well as performance of the instruments with known groups, was provided for one-half of the CVI outcome measures. This construct validity evidence was sufficient for all but one instrument, the Insight Inventory; however, only the study of the convergent validity of the Five Questions and Flemish CVI Questionnaire–CVI Questionnaire provided high-quality evidence. As with the other measurement properties appraised, additional evidence of construct validity is needed for the currently available CVI outcome measures.
García-Ormaechea et al. (2014), Gorrie et al. (2019), and Hellgren et al. (2020) conducted studies of criterion validity to determine whether the CVI instruments studied were able to predict a CVI diagnosis. However, the lack of gold-standard instruments for assessing CVI precludes consideration of these studies as true criterion validity studies, according to the COSMIN systematic review guidelines (Terwee et al., 2007). Studies of criterion validity, as well as measurement error, are needed to increase therapists’ ability to make evidence-based decisions and recommendations.
Clinical Utility of PROMs
PROMs were available for trial in the clinical setting, with the exception of the Insight Inventory (Tsirka et al., 2020) and the Structured History Taking Inventory (Macintyre-Béon et al., 2012). Each of the PROMs demonstrated strengths and weaknesses that may affect their clinical utility. For example, clustering by domains or behavioral features makes them all, with the exception of the Five Questions, useful for directing occupational therapy interventions. In addition, the Flemish CVI Questionnaire–CVI Questionnaire and the Preverbal Visual Assessment are both easy to administer because they use yes-or-no questions. The Five Questions may meet the need for a short screening tool.
Other characteristics of the PROMs may limit their clinical utility. For example, six of the PROMs (Five Questions, Functional Questionnaire, Insight Inventory, EXPRESS CVI Inventory, CVI Inventory, Structured History Taking Inventory) use 5-point scales rather than dichotomous scoring, with few guidelines for interpretation. Two of the instruments are specific to particular populations. The Functional Questionnaire was developed for children with cerebral palsy, and the Preverbal Visual Assessment was normed only for children age 2 yr and younger. The Functional Questionnaire requires testing in two different conditions, bright and dimly lit environments. A limitation of the Flemish CVI Questionnaire–CVI Questionnaire is that it includes some questions that are culturally specific and may not be relevant for non-Flemish populations.
Clinical Utility of PerfOMs and ClinROMs
The PerfOMs and ClinROMs demonstrate several strengths that make them clinically useful as well as weaknesses that limit their clinical utility. For example, the Austin Playing Card Assessment uses inexpensive common objects (i.e., playing cards), and the Children’s Visual Impairment Test is purported to be available online at no charge (https://psytests.be/clinicians/test-centrum/cvi-t.php), although we were not able to access it (K. Vancleef, personal communication, December 15, 2022). The CVI Range is clustered by visual characteristics, which makes it easy to identify overall visual behaviors. This also assists in directing therapeutic efforts to problem areas and plotting phases and progress over time.
A common potential weakness of both the Children’s Visual Impairment Test and the Austin Playing Card Assessment is that both assessments require the understanding of symbolic representation; the Children’s Visual Impairment Test incorporates pictures, and the Austin Playing Card Assessment uses playing cards. The feasibility of the Austin Playing Card Assessment and the CVI Range may be limited by the fact that both assessments require testing in various settings and conditions. The Austin Playing Card Assessment requires testing and retesting in different environments, including a noisy classroom. The CVI Range requires interview, observation, and direct assessment; however, no standardized direct assessment tasks or protocols are included. The Structured Clinical Question Inventory is a questionnaire, although its method of scoring is unknown.
Availability of the ClinROMs and PerfOMs varied. Two of the four were accessible to us and available to be used in a clinical setting. Instructions for administering and scoring the Austin Playing Card Assessment were provided in the article, and coursework on administering and interpreting the CVI Range is publicly available. As noted earlier, although the Children’s Visual Impairment Test is reported to be freely available to providers, we could not access it. The Structured Clinical History Taking was not available in print or digital format.
General Recommendations
When determining an assessment plan, occupational therapists may want to first consider whether ClinROMs, PerfOMs, or PROMs (or a combination) will best provide the information needed to make clinical decisions. Depending on the caregivers’ understanding of CVI behaviors and their observation skills, therapists may find that use of a PerfOM or ClinROM is necessary to obtain comprehensive information for making recommendations and guiding intervention.
Overall, none of the CVI assessments met the COSMIN systematic review guidelines to be recommended as most suitable for use (i.e., demonstrating any level of sufficient content validity and at least low-quality evidence of sufficient internal consistency). Therefore, none of the instruments can be unequivocally recommended for use on the basis of the currently available evidence, although they have the potential to be recommended pending further investigation. However, there is currently no high-quality evidence that any of the outcome measures has insufficient measurement properties; thus, these assessments currently have the potential to be recommended pending future research supporting their measurement properties. Nevertheless, practitioners who use these instruments to make recommendations for CVI intervention should do so with caution.
Limitations
Overall, few studies reported measurement properties of CVI assessments that are within the scope of occupational therapy practice. In addition, not all measurement properties reported could be assessed using the COSMIN Risk of Bias tool (e.g., sensitivity, specificity, and interrater reliability). We note that the number of studies by researchers who were not creators of the instruments was also limited. Finally, we may have missed relevant articles as a result of the exclusion of unpublished literature and non–English language studies.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice: ▪ Occupational therapists who work with children with CVI may use these findings to assist them in choosing assessments that currently demonstrate the best measurement properties and meet their needs in the research or clinical setting. ▪ Caution should be used when using these assessments in clinical decision-making, given the limited information available about their measurement properties. ▪ Practitioners should be aware that these outcome measures may originally have been developed in languages other than English despite being published in an English-language article, without cross-cultural validity studies. Caution is therefore warranted when using these assessments.
Conclusions
The evidence of strong measurement properties of CVI assessments is limited. Because the COSMIN guidelines require high-quality evidence of sufficient content and structural validity to recommend use of an instrument, the CVI assessments appraised have the potential to be recommended but cannot be endorsed. Future research investigating the measurement properties of CVI assessments is needed to fill the identified gaps in the evidence. As additional high-quality evidence confirms or refutes the measurement properties of CVI assessments, more definitive recommendations can be made. Until then, occupational therapy practitioners must use their clinical reasoning and judgment to identify the assessment’s purpose and the most evidence-based approach to meet that goal, given the extant evidence. Practitioners’ approaches may include use of the currently available CVI assessments, but this should be done with caution, because of the current lack of evidence of strong measurement properties. Furthermore, not all the assessments are feasible to use in the clinical setting; some were found to be more suited to research. By monitoring the most current evidence of the measurement properties of CVI instruments, occupational therapy practitioners can perform the most evidence-based assessments to support the diagnosis of CVI and provide targeted interventions to meet their clients’ needs.
Supplemental Material
Supplementary material for Occupational Therapy Assessments of Cortical or Cerebral Visual Impairment: A Systematic Review
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2023.050313.pdf for Occupational Therapy Assessments of Cortical or Cerebral Visual Impairment: A Systematic Review by Orli Weisser-Pike, Anita Witt Mitchell and Leah Cordova in The American Journal of Occupational Therapy
Footnotes
*
Indicates studies included in the systematic review.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
