Website Accessibility: U.S. Veterans Affairs Medical Centers as a Case Study

Abstract

The Internet is a critical eHealth/eGovernment information source, and the U.S. Department of Veterans Affairs operates the United States’ largest integrated health care system. This case study used machine-based accessibility testing to assess accessibility for 116 VA Medical Center websites, based on U.S. Section 508 standards and international WCAG 2.0 guidelines. While we found accessibility issues on each website analyzed, problems were generally limited. Notable exceptions included PDF accessibility and fixed-text sizes. The study’s results offer implications for practitioners (accessibility problems likely overlooked and ways to check accessibility) and educators, particularly the need to better integrate accessibility into the curriculum.

Keywords

Department of Veterans Affairs accessibility SortSite Section 508 WCAG

Over the past three decades, the Internet has become a critical information source and communication tool for eGovernment and eHealth both in the United States (the focus of this case study) and internationally. More than 84% of Americans use the Internet (Perrin & Duggan, 2015), almost 60% look online for health information, including major and minor health concerns (Fox & Duggan, 2013), and around half use the Internet to get government information (Horrigan & Rainie, 2015). Yet researchers have found that despite this high Internet usage in the United States, many health and government sites have designs that limit accessibility to users with disabilities, such as problems with hearing, vision, motor skills, and cognition (Huerta, Hefner, Ford, McAlearney, & Menachemi, 2014;Olalere & Lazar, 2011). The Department of Veterans Affairs (VA) operates the United States’ largest integrated health care system. Around 28% of U.S. veterans use VA health services in at least 950 hospitals and outpatient clinics. These facilities are organized into 137 VA Medical Centers (VAMCs). VA health care services are the most frequently used veteran benefit, and almost 6 million of the country’s 22 million veterans are enrolled in the system (U.S. Department of Veterans Affairs, n.d.-a,n.d.-f,2016d). VAMC websites serve as health care hubs, and it is critical that these websites are usable by people with disabilities. While there is an ethical imperative for VAMC website accessibility, there is also a legal one—Standards forSection 508 of the Rehabilitation Act (2000)mandates federal website accessibility (specifically, Subpart B, Subsection 1194.22, the technical standards for web-based intranet and Internet information and applications).

Despite this U.S. mandate, federal websites are not always compliant.Olalere and Lazar (2011)found Section 508 violations on 90% of federal websites studied. While the VA health system has drawn attention in recent years, its online presence has been studied primarily through examining specific applications (Haun et al., 2015;Nazi, Turvey, Klein, Hogan, & Woods, 2015), and researchers have not addressed VA website accessibility fully, particularly for VAMCs. This case study asks how well VAMC websites meet Section 508 standards and how prepared VAMC websites were to meet the planned updates to Section 508. These updates were based on the World Wide Web Consortium’s (W3C) Web Content Accessibility Guidelines (WCAG 2.0), which theW3C (2008)described as “providing a shared standard for Web content accessibility that meets the needs of individuals, organizations, and governments internationally” (para. 3). To address these questions, we used the program SortSite to conduct automated accessibility analyses of all 116 VAMC websites based on Section 508 standards and international WCAG 2.0 guidelines, a process that can be adapted and applied to other organizations’ sites.

The following section provides a discussion of the literature and background for the study: an overview of accessibility laws and guidelines, followed by a discussion of how website accessibility has been studied and background information on the VA and VAMC. Next, the study posits two research questions regarding VAMC and accessibility rules, followed by a methods discussion and an overview of the results. Finally, we offer an analysis of the implications of our findings, including the implications for practitioners and pedagogy, and a discussion of the study’s limitations and possible directions for future research.

Literature Review and Background

Accessibility Laws and Guidelines

Accessibility regulations in America have traditionally been geared toward bridging architectural barriers. The government introduced the first architectural accessibility standards in 1961 and made accessibility mandatory for buildings housing federal agencies in 1968. TheRehabilitation Act of 1973 (1973)encouraged integrating people with disabilities into the workforce, particularly at the federal level, andSection 504 (1973)required that federally funded programs be accessible to those with disabilities. TheAmericans with Disabilities Act of 1990 (ADA), which is the widest sweeping federal accessibility law, prohibits disability-based discrimination in a range of areas including building access, transportation, employment, and participation in government programs. TheTelecommunications Act of 1996 (1996)moved accessibility into electronic media, including requiring televisions to have closed captioning receivers and ordering the Federal Communications Commission (FCC) to study video descriptions.

In 1998, with the Internet playing an increasing role in government information and services, Congress amendedSection 508 of the Rehabilitation Act of 1973 (1998)to require that federal-level information and communications technology, including online material such as websites, be made accessible. The original Standards forSection 508 (2000)provided 16 accessibility standards. These technical standards for the web are enumerated A through P, so anything that does not adhere to Standard A, for instance, does not have “a text equivalent for every non-text element” (para. 1); it might be an instance of missing meaningful alternative text for images. Accommodations for disabilities do include providing appropriate alternatives to nontextual information, but they also include actions such as writing code in a meaningful (i.e., semantic) manner (e.g., headers for rows and columns within data tables), captioning video, and taking into account the possible need for extra time for a response.

Accessibility was an early W3C concern, and its Web Accessibility Initiative (WAI) released its first formal accessibility guidelines, WCAG 1.0, in 1999. WAI released the current guidelines, WCAG 2.0, in2008in an effort to make it easier for designers to create and test for accessibility. The guidelines are built on four design principles—websites should be perceivable, operable, understandable, and robust—which serve as parents to 12 guidelines that determine website conformance levels. Conformance ranges from A (highest priority—what web developers need to do at a minimum for accessibility) to AAA (lowest priority), and the W3C does not encourage requiring AAA conformance, as some content cannot realistically satisfy all AAA criteria (W3C, 2008). To help designers apply WCAG guidelines, WAI codified specific techniques, including “Criteria of Failure” (e.g., “F25: Document title must not be blank,” with the “F” signifying failure and the numbers enumerating which way a site’s code has failed [W3C, 2016, Section 12]), as well as “General Techniques” (e.g., “G11: Creating content that blinks for less than 5 seconds,” with the “G” signifying general techniques that aid accessibility [W3C, 2016, Section 1]) and “HTML and XHTML Techniques” (e.g., “H25: Providing a title using the title element,” with the “H” signifying the “H” in the coding languages or HTML and XHTML [W3C, 2016, Section 2]). Several countries, including the United States, have already adopted WCAG 2.0 as the standard for their accessibility laws (Li, Yen, Lu, & Lin, 2012).

Over the past decade, the U.S. federal government has extended the ADA from the physical world to the virtual world. In 2010, the government used the ADA to mandate state and local government website accessibility. Around the same time, the government passed theTwenty-First Century Communications and Video Accessibility Act of 2010 (2010), extending aspects of the Telecommunications Act, such as requiring closed captioning of Internet-based video, and pushing for an increased use of audio descriptions to make video more accessible to users with vision problems. The courts have sided with the extension of the ADA into the virtual world, particularly in the area of captioning, with Judge Michael Ponsor in National Association of the Deaf et al. v. Netflix, Inc.(2012)arguing, “Congress intended the ADA to adapt to changes in technology” and communication delivery systems (p. 8). The application of the ADA to the online world has not been the only recent change in U.S. federal accessibility law. In January 2017, the U.S. government formalized new Section 508 accessibility rules, adopting the WCAG 2.0 A and AA level guidelines as the new national standard. This update brings U.S. laws more closely in line with international standards and current technology (U.S. Access Board, 2017). Despite federal accessibility laws, researchers (Olalere & Lazar, 2011) have found that many federal websites have accessibility issues.Jaeger (2004)argued that when users encounter a website’s accessibility problems, it diminishes the credibility of the site’s information. Jaeger’s argument seems particularly pertinent in situations in which the target audience is likely to have a high proportion of users with disabilities that might interfere with using the website, such as VAMCs.

Accessibility Research

Despite the importance of VAMCs, most research has focused on narrow issues, including studies of electronic medical records (Nazi et al., 2015) and a brief discussion of the development of a new VAMC site to help coordinate veteran medical information during the Katrina disaster (Geraci et al., 2008). Tangentially related to the web, others (McInnes et al., 2015) explored how mobile devices might be used to help homeless veterans access VA services, particularly in the area of health care. Although researchers have not examined the VA web presence in detail, researchers have examined hospital and online health care sites.Gonçalves et al. (2015)examined the online presence of Portuguese and Spanish health care facilities by using AcessWeb, a Portuguese-language automated accessibility testing tool.Huerta et al. (2014)looked at accessibility as part of their analysis of American hospital websites. Both studies found significant accessibility problems across the board. Hospital websites are far from unique in accessibility-related problems. Similar issues are prevalent across state, local, and federal government (Evans-Cowley, 2006;Olalere & Lazar, 2011;N. E. Youngblood, 2014) and among private sector websites including major corporations (Loiacono & Djamasbi, 2013). One approach to minimizing accessibility problems has been to promote the use of templates that can be reused for multiple sites. While studies on the effect of state-level website templates (Galvez & Youngblood, 2016;Lazar et al., 2010) have shown mixed results, overall there seems to be a positive relationship between template use and increased accessibility.

Studying Website Accessibility

Website accessibility has been studied using a wide variety of approaches, including expert evaluation, user-based testing, and web-manager surveys (Jaeger, 2006;Olalere & Lazar, 2011). There have also been studies into legal policies and standards (Jaeger, 2004;Lewthwaite, 2014). Furthermore, website accessibility is studied using automated analysis tools, which apply machine-testable heuristics to web page files, such as HTML, ASP, PHP, CSS, PDF, and image files. Web-AIM’s WAVE (Geiger et al., 2011) and AChecker (N. E. Youngblood, 2014) are among the commonly used free tools, but both are limited to testing a single web page at a time. Researchers have also used automated tools such as SortSite to analyze entire sites (Conway, 2011;Gonçalves, Martins, Pereira, Oliveira, & Ferreira, 2013), allowing them to delve deep into websites and draw more valid conclusions about accessibility issues than might be available by using a single home page as a surrogate for the entire site.

As automated testing relies on computer-based heuristics, it does raise the possibility of generating false positives and false negatives. For example, while software can be used to check for the presence or the absence of alternative text for images and can provide some limited checking of alternative text quality (e.g., the use of file names or repetitive text to describe images), software cannot check for the difference between the text “a very nice picture” and “doctors performing surgery,” nor can it check for the proper use of alt=“”, which is used to tell a screen reader to skip over an image.Jaeger (2006)recommended using multiple methods when possible, though Jaeger acknowledged that doing so may be impractical in studies with large data sets, such as the current study, as expert evaluation and hands-on, user-based testing are both time and labor intensive.

The Department of Veterans Affairs

Established in 1930, the VA expanded from providing disabled veterans pensions to encompassing the Veterans Benefits Administration, the National Cemetery Association, and the Veterans Health Administration (VHA). The VHA manages health benefits including health care, dental care, disability benefits, and prescription fulfillment. Health care benefits are the most frequently used VA benefits (U.S. Department of Veterans Affairs, 2016c). Pew Research Center data from 2010 suggest that veterans who use the VA health system are less likely to use the Internet than either veterans who receive health care from outside the VA system or the general nonveteran population (Houston et al., 2013). VA veteran demographics offer insight into why these veterans may be on the have-not side of the digital divide. Veterans are more likely than nonveterans to live in rural areas, be older, be uninsured, and have chronic health conditions (Hixson, Van Bebber, & Bertko, 2015). That said, in the VA’s 2010 National Survey of Veterans, almost 70% of veterans responded that they were interested in using the Internet to get information about VA programs, and 98.7% of veterans under 30 years of age reported using the Internet (U.S. Department of Veterans Affairs, 2010). Veteran adoption of smartphones appears high—Erbes et al. (2014)found that 76% of participants in a VA posttraumatic stress disorder program owned a tablet or a smartphone.

The VA’s digital health technologies are grouped under the Office of Connected Care. These technologies include online services for health records, appointments, and prescriptions, as well as provider-to-patient telemedicine services (Haun et al., 2015;U.S. Department of Veterans Affairs, n.d.-c). VAMC websites tend to be informational and provide a wide range of VAMC-related information, including contact information, information on available care services, VAMC events, and local VAMC news. VAMC websites also provide information about VA Community-Based Outpatient Clinics (CBOCs) and links to general VA-related information and resources. While VAMCs have individual domains, VAMC site designs match the main VA website and often share common components, particularly top navigation, and appear to share a common template. Local navigation usually appears on the left side of the page and is often, though not always, similar between the different sites. A review of VAMC web pages through the Internet Archive’s Wayback Machine (Internet Archive, n.d.) suggests that this has been common practice since at least 2007. Ideally, sharing of a common theme between the VAMC websites and the VA overall site should make it easier for users to move between sites. VAMC sites are currently mobile ready and include links to local and national VA social media. The U.S. federal government provides employees training on making websites accessible (U.S. General Services Administration, n.d.). In addition, the VA also provides employees with online information, including descriptions of regulations, links to resources, and eLearning courses (U.S. Department of Veterans Affairs, n.d.-e). Although the VA provides Section 508 employee-training modules, it is unclear how well VAMC websites adhere to the standards, as there has been no research on VAMC websites and only limited research on VA websites generally.

VAMC-served veterans may have illnesses and/or disabilities that limit physical movement, sight, hearing, or cognition, and the chance of VA clients having accessibility problems will likely increase: The average age of male veterans is age 64 versus age 41 among the nonveteran male population (U.S. Department of Veterans Affairs, 2016c), and aging populations are more likely to develop disabilities. While concrete numbers for specific disabilities can be hard to nail down, theU.S. Department of Veterans Affairs (2016a)estimated that out of a 21 million veteran population, there are 130,428 legally blind veterans, and that there are more than 1 million more who have a vision loss that interferes with daily activities. Hearing loss is more widespread, and theU.S. Department of Veterans Affairs (2016e)reported that more than 933,000 veterans receive disability benefits based on hearing loss. In all, more than 4 million veterans currently receive disability benefits (U.S. Department of Veterans Affairs, 2016b), and use of VAMC and neighboring CBOCs has increased over the past decade.

Research Questions

Clearly, VAMC websites need to be accessible. Although the VA provides training resources, it is unclear if VAMC websites meet accessibility rules and how prepared they are to meet the rising number of clients with disabilities. This uncertainty leads to the following research questions:

Research Question 1:How well do VAMC websites meet federal accessibility mandates?

Prior to data collection, the United States Access Board announced a plan to update Section 508 based on WCAG 2.0 guidelines, providing clearer accessibility guidance and reflecting changes in web-based technology—the new rules went into effect 6 months after data collection. The planned changes to Section 508 lead to the next research question:

Research Question 2:How prepared are VAMC websites for the adoption of WCAG 2.0 guidelines?

To address these questions, we investigated how pervasive Section 508 and WCAG 2.0 errors were within the VAMC websites by using SortSite, an automated analysis tool, to examine each website. The tool examines the HTML and associated CSS, JavaScript, PDF, and image files for each page for violations of Section 508 guidelines and WCAG 2.0 standards. Rather than reporting just the rules being violated, SortSite reports specific issues, such as specific types of images (linked versus nonlinked, etc.) that are missing alternative text, rather than just reporting Standard A violations. Similarly, when reporting WCAG-2.0-related problems, SortSite draws on WCAG 2.0 guidelines as well as Failure for Success Criteria. Such a complete list, then, which is the purpose of this study, can afford web designers more direction when redesigning websites and help researchers better understand site problems.

Method

The VA provides a list of links to all VAMC websites (U.S. Department of Veterans Affairs, n.d.-d). Although there are 137 VAMCs, in several cases, multiple VAMCs share a common website. Using this list, we obtained a census of all 116 VAMC website addresses to use for our analysis. Data collection occurred over 5 days in July 2016 and consisted of inputting the main web address for each site into SortSite and letting the software spider each site.

We used SortSite Professional version 5.6 to analyze each website twice—once using Section 508 standards and once using WCAG 2.0 guidelines (AA level). SortSite, used by one third of Fortune 100 companies, works by analyzing a web page using usability and accessibility heuristics as well as checking for potential search engine optimization problems, broken links, privacy compliance, and use of valid HTML and CSS. In the current study, we concentrated on just accessibility-related issues. The software detects pages by following links on a starter page, usually the home page, and following links on successive pages. Analyses include a browser-based report and allow users to generate reports based on specific issues (PowerMapper Software, 2015). For each VAMC analysis, we used the accessibility reports to generate a .csv inventory of all site files examined and an .xml list of issues tied to specific files. We then converted each of these files into Excel spreadsheets, yielding 232 files for Section 508 tests and 232 files for WCAG 2.0 tests, for a total of 464 files. Next, we combined the accessibility results from each Section-508-issue list into a single issues file and then did the same for the WCAG-2.0-issue files. Using the two resulting files, we built pivot tables in each. For each VAMC, the pivot tables listed the standards/guidelines violated and how many pages violated each standard/guideline, and then broke down the list by file type—PDF, CSS, or HTML/ASP. Using the inventory files for each site, we identified how many of these files each site had and used those numbers to calculate the percentage of files with an issue for each site. We also calculated an average of how many pages per site had a specific error. Finally, we averaged the percentages of errors for each site by problem to calculate an average percentage of pages with an error. Although Section 508 standards are broad, SortSite analyzes violations based on specific problems. For example, it reports an iframe missing alternative content as a Standard A violation and also identifies an image missing alternative text as a Standard A violation. If these problems occurred on one page, the violations would be counted as separate errors. Thus, in the results, we discuss the prevalence of individual problems (e.g., missing alternative text) rather than the number of pages that violated a specific standard or guideline.

Results and Discussion

Not one of the websites was error free, though that is not surprising as the smallest site had 356 HTML files (including ASP, PHP, etc.), 16 PDF files, and 19 CSS files, and the largest site had 2,027 HTML files, 563 PDF files, and 16 CSS files. On average, each site had 598 HTML files, 70 PDF files, and 19 CSS files. While the sites should be error free, the error rate was relatively low, and SortSite identified all sites as performing above average for accessibility.Figures 1,2, and3break down Section 508 violations by file type—HTML, PDF, and CSS—and list how many sites had a problem with each issue, the average number of files per site with the problem, and the average percentage of files that had the problem.Figures 4,5, and6do the same for WCAG data. Errors occurring on less than 2% of websites and on less than 1% of pages for sites with errors were not included in the figures. An analysis of the results presented in the figures follows.

Figure 1.

Section 508 standards violations: HTML/ASP files.

Figure 2.

Section 508 standards violations: PDF files.

Figure 3.

Section 508 standards violations: CSS files.

Figure 4.

WCAG guideline violations: HTML/ASP files.

Figure 5.

WCAG guideline violations: PDF files.

Figure 6.

WCAG guideline violations: CSS files.

As shown inFigures 1through3, the majority of Section 508 errors were related to Standard 1194.22 (A): “A text equivalent for every non-text element shall be provided (e.g., via “alt,” “longdesc,” or in element content)” (Standards forSection 508, 2000, para. 1). In essence, this standard mandates that pictures, graphs, and other embedded media have to be made accessible, usually by providing a text-based description of the item. These issues include all but one of the most common errors SortSite detected. Of the eight Standard A errors for HTML pages (Figure 1), the most prevalent was the use of iframes (embedded html web page content) missing alternative text-based information. The issue was common across all the sites but affected only a small percentage of pages per site—on average, less than 1%, with the highest error rate on a site being 1.96%. More troubling were widespread issues with PDF files (Figure 2) missing the appropriate alternative text, which occurred in 106 (91.38%) of the sites and showed up in an average of 21% of the PDFs on a site. In several cases, this problem appeared in more than half of a site’s PDFs. Similarly, 100 (86.21%) of the sites had problems with PDFs having no markup at all for screen readers—an average of around 11%, though the highest percentage for this issue on an individual site was around 35% on a site that had 563 PDFs. Links to PDFs also posed a problem for all of the sites: A quarter of pages, on average, had links to PDFs without the link to Acrobat Reader required by Standard M, which requires that when an application or plug-in (e.g., Adobe Acrobat) is needed to view content on a page, designers include a link for users to download the plug-in.

Forty-two sites (36.21%) had images with missing alternative text (Standard A), with one site having this issue on 6.4% of its pages. The problem was typically more limited, averaging around 0.5% of pages per site. Forty-eight sites (41.38%) also had issues with file names being used as alternative text for images. This problem showed up on an average of less than a half percent of a site’s pages, though one site had problems with the error on almost 3% of its pages. This problem also appeared in PDF files—an average of almost 7% of the PDFs on 72 (62.07%) sites.

All sites had issues with removing hyperlink underlining, raising potential issues with Standard C, which requires that all information conveyed with color be available without color (Figure 3), as users relying on screen readers and users who are color blind will not be able to use the color-based information. This problem appeared in the same two CSS files in each site, bootstrap.css and layout.css. A review of a few of these files suggests that the affected links are in the navigation, where the lack of underlining might not pose a problem. All of the sites also had issues with Standard G: “Row and column headers shall be identified for data tables” (Standards forSection 508, 2000, para. 7). The inclusion of row and column information is critically important for making tables accessible for users relying on text readers, as it provides them with the context for the table data. An average of 27.79 pages per site (3.89%) had tables missing table headers or, in the case of layout tables, missing statements indicating that tables were used for presentation, which can cause a screen reader to misinterpret the page element order. In some sites, the problem was widespread—in one case, appearing on more than 27% of a site’s pages. This issue showed up in an average of 8.33% of PDFs on 11 sites, including more than 40% of one site’s PDFs. More than half (52.9%) of the sites violated Standard I: “Frames shall be titled with text that facilitates frame identification” (Standards forSection 508, 2000, para. 9). Much like alternative text, frame titles help users relying on screen readers to navigate through a web page, as they cannot visually scan the information on the page but have to navigate through each frame. While common across sites, the error rate was below 1% of pages on all but one site.

All sites had issues with Standard N, form accessibility, and had form controls missing label elements (similar to alternative text) intended to assist screen reader users in navigating forms. More than half (56.03%) of the sites had labels referring to nonexistent controls, making it difficult for screen reader users to use forms and navigate pages. Finally, all sites had pages with broken skip links, preventing affected pages from meeting Standard O: “A method shall be provided that permits users to skip repetitive navigation links” (Standards forSection 508, 2000, para. 15). These links, often phrased as “skip navigation” or “skip to content,” are particularly useful for users of screen readers as they allow users to bypass having the primary page navigation read to them on each of a site’s pages. Again, however, the problem appeared on barely 0.5% of pages on average.

Not surprisingly, some Section 508 errors are also problems identified in the WCAG 2.0 analysis (seeFigures 4,5, and6). The examples discussed above include broken navigation skip links, problems with form labels, unidentified table rows and columns, nonaccessible PDFs, problematic alternative text, and problems with labels. The WCAG analysis also picked up a number of problems not identified by the Section 508 analysis, or it identified more specific problems than did the Section 508 analysis.

Some of the more widespread problems involved making code clearer. Many sites (67.24%) had pages that did not report what language they were written in by using the HTML lang attribute, which lets a screen reader know what language to speak in, though the problem only appeared on less than 2% of pages on average. PDFs proved more problematic, with 91.38% of sites missing language information in an average of 20.62% of PDF files. Four sites had PDFs missing language information in 60% or more of their PDFs. Similarly, 95.59% of sites had PDFs missing title information (which helps screen readers identify what the PDF is about)—an average of 29.92% of a site’s PDFs. SortSite identified all sites as having pages with coding errors, with an average error rate of 8.5% of pages, and a high of 30% of pages for one site. Many websites also had issues with empty heading elements (79.31%) and empty hyperlinks (58.62%), though, on average, these problems appeared on less than 1% of a site’s web pages. Both of the issues are particularly problematic for screen reader users. The WCAG analysis also identified an additional issue with alternative text and images: All but one (99.14%) of the sites had instances in which the alternative text matched linked text in the same image or linked text directly next to the image, which can make a screen reader appear to stutter. Again, however, the problem was not widespread on the sites themselves and, on average, occurred in 0.22% of pages per site.

All sites had issues with multiple IDs with the same name on a page, which can cause issues for users with or without a vision problem, particularly if the ID is used as a hyperlink target. On average, the problem appeared on around 1% of a site’s pages, and all but one site had the problem on less than 2% of its pages. Well over half of the sites (62.07%) had pages referencing missing IDs, raising the potential for hyperlinks to page sections not working, though the problem only appeared on an average of 0.89% of a site’s pages. In addition to problems with form labels identified in the Section 508 analysis, the WCAG analysis found that 56.9% of sites had issues with labels not properly associated with controls, though again, the problem appeared on average in less than 1% of a site’s pages, and only one site had problems on more than 3% of its pages.

The last group of issues involves the use of absolute font sizes, which prevents users from easily enlarging website text through their web browser. Almost three fourths of sites (73.28%) had web pages relying on fixed text size (e.g., pixel) rather than relative text size, such as percent or em (essentially a percentage of a site’s base font size), though the problem only appeared on an average of 1.02% of pages. The problem was more widespread when it came to CSS files, where it appeared in all sites and in an average of 57.73% of CSS files. Finally, all of the sites had issues with CSS styles that interfered with keyboard-based navigation of a page by obscuring the dotted line that surrounds the on-focus (active) area of the screen as a user arrows or tabs through the information on a page.

This study provides an analysis of VAMC website accessibility, shares benchmarks against which to study the VA’s long-term progress in website accessibility, and discusses website conformance to Section 508 standards and WCAG 2.0 guidelines. The accessibility problems identified were not as widespread as one might anticipate based on prior eGovernment research. Alternative text issues often present major accessibility stumbling blocks. VAMC web pages, though, typically had few alternative text issues. Alternative text is a greater problem when it comes to PDFs, and PDFs are a frequent source of accessibility errors across the board, with some sites missing alternative text for images in well over half of the PDFs. The VA has not ignored this problem, at least when it comes to training; the VA’s online accessibility training includes a module on creating accessible PDFs (U.S. Department of Veterans Affairs, n.d.-b). The low frequency of CSS issues compared with HTML and PDF issues belies their impact. For instance, a text size issue in one line of CSS could affect every page on a website. Coupled with that, we did not examine how many of these errors occurred per file, which means that the issue could be more widespread in the site than what a simple percentage of CSS files might indicate.

The Internet is a significant health information resource, both in the United States and elsewhere. Making health information accessible is critically important. The harder it is for users with disabilities to access health information online, the less likely they are to try to use that information source. The United States, in general, and the VA, specifically, face an aging population, a population particularly likely to have disabilities that may affect a user’s ability to use the Internet and the web. It is crucial that those entering the web design arena understand what accessibility is, how to design accessible products, and how to test online product accessibility.

Implications for Pedagogy

EGovernment websites are often the only easily identifiable source for government information, particularly in areas such as the VA, which provides important health care services. As such, they are exceptional examples to use for helping students understand the importance of accessibility. In the case of looking at e-commerce and accessibility, a student may argue that users with a disability could just go to another site. It is much harder to make this argument when it comes to eGovernment information, particularly at the local level.

While automated tools are not a substitute for expert evaluation or hands-on accessibility testing during website development, automated tools do offer an opportunity to provide quick, usable, and actionable feedback on website accessibility. Automated tools work well in identifying basic problems, particularly items that are missing entirely such as alternative text and form labels. As such, these tools are particularly useful in the classroom, where students are likely still learning how to address accessibility in their design work. SortSite is a commercial program and may not be available to students; however, other products such as WebAIM’s WAVE (WebAIM, n.d.) and ATutor’s AChecker (Inclusive Design Research Centre, 2011) are freely available and offer similar feedback, albeit one page at a time. Although students should be encouraged to design with accessibility in mind from the very beginning of the development process, these tools can help students check their work as they go and can provide a good summative measure that can be turned in along with a project. Each tool has its own relative advantages. WAVE provides a visual map of where the errors are on a page and provides explanations about what the implications are for each issue. This format can be particularly useful for helping students understand the problems on a page. It can be difficult, however, to print or record WAVE results, and the source file has to be located on a web server. While it does not provide a visual map, AChecker provides easily printable and savable results, as well as a list of results by line number, allowing students a way to identify flaws in their code and the user to upload a file or cut and paste a section of code. This format offers advantages for faculty who would like students to turn in proof that they have used the tool and for researchers interested in conducting an automated analysis.

There is a range of options available to help students evaluate and implement website accessibility. Both WAVE and AChecker can check for HTML contrast issues that might be problematic for colorblindness, butVischeck.comlets users upload an image and view it through simulations of three types of color blindness. There are also browser plug-ins that can help students better understand and evaluate accessibility issues. One of the most helpful plug-ins is the Fangs Screen Reader Emulator for Firefox, which converts a web page to a text-based page similar to what would be read by a screen reader.S. A. Youngblood (2013)offered a good discussion of how to implement this tool in a classroom environment and stressed that it is important to use a tool such as Fangs rather than try to have students who are sighted try suddenly relying on an actual screen reader or work blindfolded, as these types of simulations are “inadequate and inappropriate” and disregard the unique skills users who are blind have developed (p. 221). Students need to be aware of these skills and understand how users with disabilities use computers. While having students conduct interviews may be an option, it may be difficult to find a sufficient number of users with disabilities for this type of project, particularly on a reoccurring basis. One alternative is to have someone from the university accessibility office come talk to a class. Another alternative is to have students watch videos of students with disabilities talk about how they use computers and the challenges poorly designed web pages pose. WebAIM is a good source for this type of video.

The breadth of VAMC website PDF accessibility issues suggests an area that needs to be covered in communication courses—the creation of accessible documents in nonweb courses. Although accessibility is a standard theme in web design courses, it is important for students to learn that web designers are not the only ones with a responsibility for creating accessible material. Ingraining accessibility should be part and parcel for courses teaching students to create and design documents. Adobe offers an online guide for creating accessible PDFs, and Adobe Acrobat Pro has a built-in accessibility tool (Adobe, 2017). Accessibility needs to extend to audio and video material as well. Students need to understand the importance of creating quality captions and that captioning is not simply a transcript; faculty interested in including captioning in their courses should incorporateZdenek (2015)into their accessibility readings. Audio descriptions are another area that needs to be addressed in courses that include video-based material, and this is an area that needs more research. Above all, students need to understand that they should think about accessibility from the beginning of a project. AsZdenek (2009)pointed out, “on the fly” accessibility does not work well (“A Critique,” para. 1). The best chance for a product to be accessible is to include accessibility as part of the planning and design processes.

Limitations and Implications for Future Research

Automated accessibility testing brings with it built-in limitations. An automated tool can check for the absence of alternative text and for key problems such as the use of a file name or a specific phrase such as “placeholder,” but it cannot differentiate between the appropriateness of more generalized alternative text. SortSite is also unable to enter password-protected portions of a site without the proper credentials and does not check videos for the use of captions. In addition, automated tools cannot fully replace having an actual user performing tasks on the computer, particularly when it comes to accessibility testing. That said, automated tools offer organizations an opportunity to identify and fix major accessibility problems before bringing in actual users, which can be both a time-consuming and expensive process. We did notice some minor discrepancies between WCAG and Section 508 analyses for some sites, as the number of HTML files examined sometimes varied by a few files. This, in turn, led to some slight differences in the WCAG and Section 508 results, most notably in the percentage of sites identified as having a problem. This research has identified some key accessibility problems, particularly issues with PDFs, forms, and nonresizable text. The methods employed in this study can be used or adapted for other sets of sites, including in international contexts (focusing on the international WCAG 2.0 standards). Future research should consider bringing in users with specific disabilities, particularly veterans, to help identify specific issues in the document creation process and help triangulate website accessibility issues. Finally, researchers may also want to explore how content creators are being trained on accessibility, particularly in the area of PDF creation, which may occur outside of the web development area.

Footnotes

Acknowledgements

The authors would like to acknowledge the valuable reviewer feedback given to a conference version of this article when it was presented at the Broadcast Education Association Conference in 2017. The authors would also like to thank Sushil Oswal for his editorial work on this article and Susan Youngblood for her assistance in editing the article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Norman E. Youngbloodis an associate professor of media studies at Auburn University and director of Auburn’s Laboratory for Usability, Communication, Interaction, and Accessibility. He teaches undergraduate and graduate media studies courses including courses on web design, podcasting, and media history. His research focuses on eGovernment and eHealth website usability and accessibility.

Michael Brooksis a PhD student in the technical communication and rhetoric program at Texas Tech University.

References

Adobe. (2017).Create and verify PDF accessibility (Acrobat Pro). Retrieved fromhttps://helpx.adobe.com/acrobat/using/create-verify-pdf-accessibility.html

Americans with Disabilities Act of 1990.42 U.S.C. § 12101 et seq. (1990).

Conway

(2011).Website accessibility in Western Australian public libraries.Australian Library Journal,60,103-112.

Erbes

C. R.

Stinson

Kuhn

Polusny

Urban

Hoffman

. . . Thorp

S. R.

(2014).Access, utilization, and interest in mHealth applications among veterans receiving outpatient care for PTSD.Military Medicine,179,1218-1222.

Evans-Cowley

J. S.

(2006).The accessibility of municipal government websites.Journal of E-Government,2(2),75-90.

Fox

Duggan

(2013).Health online 2013. Retrieved fromhttp://www.pewinternet.org/2013/01/15/health-online-2013

Galvez

R. A.

Youngblood

N. E.

(2016).e-Government in Rhode Island: What effects do templates have on usability, accessibility, and mobile readiness? Universal Access in the Information Society,15,281-296.

Geiger

Evans

R. R.

Cellitti

M. A.

Smith

K. H.

O’Neal

M. R.

Firsing

S. L.

III Chandan

(2011).The healthy web: Access to online health information for individuals with disabilities.Global Journal of Health Education and Promotion,14,93-100.

Geraci

S. A.

Douglas

Algood

T. L.

Pinkston

Sanders

Kirchner

(2008).Hurricane Katrina: The Jackson Veterans Affairs Medical Center experience.American Journal of the Medical Sciences,336,116-123.

10.

Gonçalves

Martins

Branco

Pereira

Peixoto

Rocha

(2015).AccessWeb barometer.Paper presented at the Seventh International Conference on Evolving Internet,St. Julians, Malta. Retrieved fromhttps://www.researchgate.net/publication/281626045_AccessWeb_Barometer_-A_Web_Accessibility_Evaluation_and_Analysis_Platform

11.

Gonçalves

Martins

Pereira

Oliveira

M. A.-Y.

Ferreira

J. J. P.

(2013).Enterprise web accessibility levels amongst the Forbes 250: Where art thou o virtuous leader? Journal of Business Ethics,113,363-375.

12.

Haun

J. N.

Nazi

K. M.

Chavez

Lind

J. D.

Antinori

Gosline

R. M.

Martin

T. L.

(2015).A participatory approach to designing and enhancing integrated health information technology systems for veterans: Protocol.JMIR Research Protocols,4,e28.

13.

Hixson

J. D.

Van Bebber

S. L.

Bertko

K. M.

(2015).Interest in a digital health tool in veterans with epilepsy: Results of a phone survey.Military Medicine,180,387-390.

14.

Horrigan

J. B.

Rainie

(2015).Connecting with government or government data. Retrieved fromhttp://www.pewinternet.org/2015/04/21/connecting-with-government-or-government-data

15.

Houston

T. K.

Volkman

J. E.

Feng

Nazi

K. M.

Shimada

S. L.

Fox

(2013).Veteran Internet use and engagement with health information online.Military Medicine,178,394-400.

16.

Huerta

T. R.

Hefner

J. L.

Ford

E. W.

McAlearney

A. S.

Menachemi

(2014).Hospital website rankings in the United States: Expanding benchmarks and standards for effective consumer engagement.Journal of Medical Internet Research,16,e64.

17.

Inclusive Design Research Centre. (2011).IDI web accessibility checker. Retrieved fromhttps://achecker.ca/checker/index.php

18.

Internet Archive. (n.d.). Retrieved fromhttp://www.archive.org

19.

Jaeger

P. T.

(2004).Beyond Section 508: The spectrum of legal requirements for accessible e-government web sites in the United States.Journal of Government Information,30,518-533.

20.

Jaeger

P. T.

(2006).Assessing Section 508 compliance on federal e-government web sites: A multi-method, user-centered evaluation of accessibility for persons with disabilities.Government Information Quarterly,23,169-190.

21.

Lazar

Beavan

Brown

Coffey

Nolf

Poole

. . . Wenger

(2010).Investigating the accessibility of state government web sites in Maryland. In Langdon

Clarkson

Robinson

(Eds.),Designing inclusive interactions(pp.69-78).London, England:Springer.

22.

Lewthwaite

(2014).Web accessibility standards and disability: Developing critical perspectives on accessibility.Disability and Rehabilitation,36,1375-1383.

23.

S.-H.

Yen

D. C.

W. H.

Lin

T. L.

(2012).Migrating from WCAG 1.0 to WCAG 2.0: A comparative study based on Web Content Accessibility Guidelines in Taiwan.Computers in Human Behavior,28,87-96.

24.

Loiacono

E. T.

Djamasbi

(2013).Corporate website accessibility: Does legislation matter? Universal Access in the Information Society,12,115-124.

25.

McInnes

D. K.

Fix

G. M.

Solomon

J. L.

Petrakis

B. A.

Sawh

Smelson

D. A.

(2015).Preliminary needs assessment of mobile technology use for healthcare among homeless veterans.PeerJ,3,e1096.

26.

National Association of the Deaf et al. v. Netflix, Inc.United States District Court for the District of Massachusetts. (2012,June). Retrieved fromhttps://dredf.org/captioning/MJP-DENIED-JUDGE-PONSOR-ORDER-6-19-12.pdf

27.

Nazi

K. M.

Turvey

C. L.

Klein

D. M.

Hogan

T. P.

Woods

S. S.

(2015).VA OpenNotes: Exploring the experiences of early patient adopters with access to clinical notes.Journal of the American Medical Informatics Association,22,380-389.

28.

Olalere

Lazar

(2011).Accessibility of U.S. federal government home pages: Section 508 compliance and site accessibility statements.Government Information Quarterly,28,303-309.

29.

Perrin

Duggan

(2015).Americans’ Internet access: 2000-2015. Retrieved fromhttp://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015

30.

PowerMapper Software. (2015).SortSite technical overview. Retrieved fromhttp://www.powermapper.com/download/sortsite/sortsite-technical-overview.pdf

31.

Rehabilitation Act of 1973.29 U.S.C. § 701 et seq. (1973).

32.

Section 504 of the Rehabilitation Act of 1973.29 U.S.C. § 701, sec. 794(1973).

33.

Section 508 of the Rehabilitation Act of 1973,29 U.S.C. § 794d(1998).

34.

Standards for Section 508 of the Rehabilitation Act. 65 FR 80523 § D1194.22(2000).

35.

Telecommunications Act of 1996. Pub. L.104-104(1996).

36.

Twenty-First Century Communications and Video Accessibility Act of 2010. Pub. L.111-260(2010).

37.

U.S. Access Board. (2017).About the update of the Section 508 standards and Section 255 guidelines for information and communication technology. Retrieved fromhttps://www.access-board.gov/guidelines-and-standards/communications-and-it/about-the-ict-refresh/overview-of-the-final-rule

38.

U.S. Department of Veterans Affairs. (n.d.-a).About VA: History: VA history. Retrieved fromhttp://www.va.gov/about_va/vahistory.asp

39.

U.S. Department of Veterans Affairs. (n.d.-b).Creating accessible PDFs with Adobe Acrobat Professional. Retrieved fromhttps://www.section508.va.gov/support/tutorials/pdf/video/creating_artifacts.htm

40.

U.S. Department of Veterans Affairs. (n.d.-c).Health care professionals. Retrieved fromhttps://connectedcare.va.gov/terms/connected-health/tabs/Solutions-for-Health-Care-Professionals

41.

U.S. Department of Veterans Affairs. (n.d.-d).Location. Retrieved fromhttp://www.va.gov/directory/guide/rpt_fac_list.cfm

42.

U.S. Department of Veterans Affairs. (n.d.-e).Section 508 support. Retrieved fromhttp://www.section508.va.gov/support

43.

U.S. Department of Veterans Affairs. (n.d.-f).U.S. Department of Veterans Affairs. Retrieved fromhttp://www.va.gov

44.

U.S. Department of Veterans Affairs. (2010).National survey of veterans, active duty service members, demobilized national guard and reserve members, family members, and surviving spouses.Washington, DC:Department of Veterans Affairs.

45.

U.S. Department of Veterans Affairs. (2016a).About blind rehabilitation services. Retrieved fromhttp://www.rehab.va.gov/PROSTHETICS/blindrehab/BRS_Coordinated_Care.asp

46.

U.S. Department of Veterans Affairs. (2016b).Department of Veterans Affairs statistics at a glance[PowerPoint slides]. Retrieved fromhttp://www.va.gov/vetdata/docs/Quickfacts/Homepage_slideshow_06_04_16.pdf

47.

U.S. Department of Veterans Affairs. (2016c).Profile of veterans: 2014 data from the American community survey[PowerPoint slides]. Retrieved fromhttp://www.va.gov/vetdata/docs/SpecialReports/Profile_of_Veterans_2014.pdf

48.

U.S. Department of Veterans Affairs. (2016d).Unique veteran users report FY 2014[PowerPoint slides]. Retrieved fromhttp://www.va.gov/vetdata/docs/SpecialReports/Profile_of_Unique_Veteran_Users_2014.pdf

49.

U.S. Department of Veterans Affairs. (2016e).VA research on hearing loss. Retrieved fromhttp://www.research.VA.gov/topics/hearing.cfm

50.

U.S. General Services Administration. (n.d.).Section508.gov: GSA government-wide Section 508 accessibility program. Retrieved fromhttp://Section508.gov

51.

W3C. (2008).Web content accessibility guidelines (WCAG) 2.0. Retrieved fromhttps://www.w3.org/TR/WCAG20

52.

W3C. (2016).Techniques for WCAG 2.0: Techniques and failures for web content accessibility guidelines 2.0. Retrieved fromhttps://www.w3.org/TR/WCAG20-TECHS/complete.html

53.

WebAIM. (n.d.).WAVE web accessibility evaluation tool. Retrieved fromhttp://wave.webaim.org

54.

Youngblood

N. E.

(2014).Revisiting Alabama state website accessibility.Government Information Quarterly,31,476-487.

55.

Youngblood

S. A.

(2013).Communicating web accessibility to the novice developer: From user experience to application.Journal of Business and Technical Communication,27,209-232.

56.

Zdenek

(2009,Fall).Accessible podcasting: College students on the margins in the new media classroom.Computers and Composition Online. Retrieved fromhttp://seanzdenek.com/article-accessible-podcasting/

57.

Zdenek

(2015).Reading sounds: Closed-captioned media and popular culture.Chicago, IL:University of Chicago Press.