Abstract
Some U.S. Military Health System (MHS) beneficiaries face unique challenges accessing available behavioral healthcare because of the nature of their occupations, deployments to and permanent duty stations in isolated geographies, and discontinuity of services. The use of deployable telehealth centers such as modified shipping containers offers promise as an innovative solution to increase access to behavioral healthcare in remote and otherwise austere environments. The first telehealth modified 20-foot shipping container, known as a relocatable telehealth center (RTeC), was deployed to increase access to care for MHS beneficiaries on American Samoa. The goal of this study was to conduct an exploratory evaluation of patient satisfaction with and usability perceptions of this solution as a place to receive behavioral healthcare services. Twenty-eight beneficiaries participated in this evaluation. Results suggest that the RTeC is safe and private and ultimately an appropriate telebehavioral-originating site. These data provide insight into usability considerations and inform future research and deployable telehealth center development. Additionally, a brief discussion about potential cost offset is provided as cost efficiencies impact RTeC viability.
Introduction
The U.S. Military Health System (MHS) provides access to multiple levels of healthcare for its beneficiaries, consisting primarily of American military service members and their families. Although the MHS is vast, many specialty care services, including psychological health and traumatic brain injury care, known collectively as behavioral health, are typically resourced at large military treatment facilities often located on or near even larger military installations. This fosters a situation where some beneficiaries face challenges accessing available behavioral healthcare because of the nature of their occupations, deployments to and permanent duty stations in isolated geographies, and discontinuity of services. Furthermore, the prevalence of behavioral health problems has been shown to be higher among those deployed to combat theaters. 1 –6 Challenges to accessing timely care have been compounded by the more than 2 million service members who have deployed to, and returned from, combat zones in Iraq and Afghanistan. 7
These conflicts have also made great use of reserve component service members, comprising both the National Guard and Reserves. Accessing care can be particularly problematic for these reserve component beneficiaries and those from smaller or more remote installations who may travel long and sometimes arduous distances to military treatment facilities to receive care. Care that cannot be provided within a military treatment facility is augmented through TRICARE managed care support contractors; however, it is estimated that almost 89 million people in the United States live in a Mental Health Provider Shortage Area. 8 Service members, in particular those from the reserve component and their families, often populate these shortage areas, and it has been reported that one-third of veterans from these combat zones will return to rural or highly rural areas. 9 Many beneficiaries thus live in areas that are both geographically remote from major medical facilities and do not have sufficient organic non-military specialized services to meet the vast need for care. This has necessitated a need to develop access-to-care solutions for the broad MHS population.
Accessing Care Through Telebehavioral Health
The use of telecommunications technologies has been identified as a solution to bridging access to behavioral healthcare gaps and reaching out to underserved populations. Telebehavioral health is a broad term that encompasses the use of technologies such as computers and the Internet, telephones, mobile devices, videoteleconferencing, and broadband connectivity to increase access to and enhance the delivery of behavioral healthcare from a distance. Several studies have shown strong support for the use of telebehavioral health for the treatment of depression and posttraumatic stress disorder, 10 –13 including group settings. 14 For many underserved populations including service members in remote locations, telebehavioral health is not only a convenience, but also a lifeline that offers the ability to provide 24/7 care to patients in need, eliminate wait times and travel costs, and potentially reduce stigma.
Obtaining telebehavioral health services requires access to both the appropriate technologies and an approved originating site. Studies show that many service members not only have access to these technologies but are willing to use them for behavioral health. 15,16 In addition, when given access to these technology solutions service members express satisfaction and willingness to use them similar to in-person encounters, and some even prefer this mode of behavioral healthcare delivery. 17 Unfortunately, access to both appropriate technology and originating site space is not universal, with access to space being possibly the more limiting factor. For example, some policies do not approve the delivery of ongoing telemental health clinical care to a clinically unsupervised originating site. Therefore, even if beneficiaries have personal access to these technologies they may not be able to access them in personal space, such as a home, other community-based sites, or nonclinical military settings. Solutions are thus needed that can place the appropriate technologies in approved, patient-centric locations.
Mobile Health Clinics
Mobile health clinics have historically afforded a convenient and acceptable portal into healthcare systems for a range of at-risk populations seeking a diverse array of services, including those considered to be medically disenfranchised because of lack of adequate insurance or other access barriers, such as mobility, geography, stigma, or trust. 18 In this context mobile health clinics are viewed as serving healthcare systems as safety net clinics rather than as an important component of the broader healthcare spectrum, including specialty behavioral health services. Although synchronous telehealth may be considered a standard of care and the use of mobile clinics for a range of healthcare services is established, 19 –24 there are limited examples where telehealth and mobile clinics have been combined to deliver ongoing telebehavioral healthcare. An early project in Oklahoma staffed a mobile health clinic with a midlevel provider who was able to teleconsult with a physician about difficult diagnoses rather than making an immediate referral to the main clinic. 24 Initial reports were positive, noting that staffing a mobile clinic with a physician cost twice as much as using telehealth in the same vehicle. Others have described efforts in Maine to deliver telehealth to extremely rural island populations using a 71-foot vessel outfitted with telehealth equipment enabled to provide primary and behavioral healthcare. 19 Thus, while integrating telehealth with repurposed equipment is not an entirely new idea, it is an idea that has gone mostly unrealized or unreported in the literature.
The National Center for Telehealth and Technology (T2), a Department of Defense organization, developed a program to integrate telebehavioral health technologies and services into mobile health clinics, broadly known as deployable telehealth centers (DTeCs). The initial solution deployed by T2, a relocatable telehealth center (RTeC), consists of a modified shipping container, also known as CONEX, built within International Organization for Standards shipping standards, outfitted with telehealth equipment and designed for mobility to provide MHS beneficiaries with access to care regardless of location. This platform is designed to enable the provision of telebehavioral healthcare in remote and/or austere environments that lack appropriate space. The remainder of this article will focus on how the MHS, through a partnership among T2, Tripler Army Medical Center (TAMC), and the Veterans Affairs Pacific Islands Healthcare System, has taken a lead in identifying the role that flexible and deployable telebehavioral health platforms may play in overcoming access to care barriers for service members and their families, regardless of location. It will focus specifically on the exploratory evaluation of a project using an RTeC to deliver care to MHS beneficiaries on American Samoa. This was the first RTeC deployed in an operational clinic setting, and the exploratory evaluation project to assess its usability as a place to receive telebehavioral healthcare was approved by the TAMC institutional review board.
Subjects and Methods
Needs Assessment Methods and Results
T2 initially partnered with TAMC in February 2009 to conduct a healthcare needs assessment for MHS beneficiaries on American Samoa and to determine whether telebehavioral health capabilities could provide quality, economical services to help fill any defined gaps in care. The assessment focused on American Samoa for several reasons. It is the most remote territory of the United States, its population of approximately 68,000 serves the United States in great numbers, it had reported gaps in access to medical care, with high costs to deliver care, and the local Veterans Affairs Pacific Islands Healthcare System Community Based Outpatient Clinic had network communications in place that ran directly to the operations colocated on the TAMC campus in Hawaii. Additionally, the 100th Infantry Battalion of the 442nd Regimental Combat Team, the U.S. Army Reserve's only Infantry unit, has two companies based in American Samoa, which were mobilized for deployments in support of Iraq and Afghanistan.
Twenty-one individuals participated in either individual or group interviews and were categorized by status as MHS beneficiaries, healthcare providers, or healthcare administrators. Interviews were conducted using a structured, but flexible, format focused on five main topic areas: (1) healthcare needs and services, (2) technology-based care, (3) barriers to care, (4) current practices, and (5) technology and infrastructure. The results suggest that MHS beneficiaries on American Samoa face several healthcare barriers, including limited access to providers and other resources because of geography and lack of infrastructure, lack of education about mental healthcare, and a deeply held culture that may stigmatize seeking care. Most interviewees expressed an interest in using telehealth capabilities for several services focused on anger management, posttraumatic stress, marriage counseling, family services, and traumatic brain injury care such as concussion assessment and management. To maintain ethical standards, no identifying information was collected, and study participants were informed of the confidential nature of the interviews and their right to end the interview at any time.
RTeC Implementation
The tsunami that hit American Samoa in September 2009 coincided with the redeployment of the 100th Battalion and heightened awareness of the reported gaps in accessing care. It was determined that placement of a 20-foot RTeC partitioned into three offices on American Samoa offered a probable solution to increase access to behavioral healthcare while also reducing travel cost and time lost associated with sending patients off-island for care. Each office was about 5'3" wide by 7'1" deep, had its own lockable exterior door, a built-in desk, and a window, and was equipped with a videoteleconferencing station, telephone, and computer. A single air conditioner unit served the RTeC by use of recirculation fans in the walls adjoining each office (Fig. 1). Although colocated with the Veterans Affairs Pacific Islands Healthcare System Community Based Outpatient Clinic, the RTeC was designated solely for use by MHS providers and beneficiaries. Because of the complexity of working across organizations and connectivity networks, care was not initiated until October 2010.

Relocatable telehealth center placement in American Samoa.
Evaluation Methods
Adult beneficiaries completing a clinical encounter in the RTeC were eligible for this study. Using a standard recruitment script, a member of the study staff approached potential participants following a telebehavioral healthcare visit in the RTeC. Interested beneficiaries were then given an informational letter to read prior to completing the questionnaire. All data was collected using a secure online questionnaire tool that generated anonymous responses. All results were password-protected, and participants were reminded to not provide any identifying information. Network errors resulted in six questionnaires being completed initially in paper form and then later entered into the electronic format by an approved member of the study staff. Of those approached, 28 beneficiaries consented to participate by anonymously completing the Telehealth and Technology Subjective Usability Survey (T2SUS), a 68-item questionnaire focused on the usability of and their satisfaction with the RTeC as a place to receive telebehavioral health services.
The T2SUS questionnaire was designed specifically for this project to focus on broad topics of telehealth utilization rather than typical satisfaction measures 25 and included basic demographic information, items about past experiences with and access to mental healthcare and technology, and a free-form written response. The majority of items were designed to assess three main content areas related to the healthcare visit: (1) technology, (2) office setting, and (3) final thoughts. These items were scored with a Likert-type format that asked respondents to indicate how much they agree or disagree with each statement on a 5-point scale (1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, and 5=strongly agree), which were collapsed for ease of reporting. Because this study was exploratory, responses were examined on an item-by-item basis within these domains. The study protocol was approved by the Human Use Committee at TAMC. Investigators adhered to the policies for protection of human subjects as prescribed in 45 Code of Federal Regulation 46.
Results
Participants
Data from all 28 consenting participants are provided. Twenty-four respondents were sponsor beneficiaries, and four were family members. Almost all were part of the Reserve Component or eligible retirees, and all reported the Army as their current or previous branch of service. Participants were largely Polynesian (n=24), non-commissioned officers or dependents (n=26), and male (n=22). The mean (M) age of participants was 42 years (standard deviation [SD]=9.4), ranging from 28 to 58 years. Ten participants had a high school education, 13 had some college experience, and 3 had an associate, technical, or paraprofessional degree. The high ratio of Polynesian ethnicity, Army non-commissioned officer status, and the relatively advanced age of the study sample suggest it is not representative of the broader military or general population.
Nineteen participants stated that in the past year they typically received care in-person with a provider; however, over half of all respondents reported they had received telehealthcare at some point in the past. Fifteen participants reported traveling 30–60 min one way to arrive at their session, 10 reported traveling less than 30 min, and 3 reported traveling 1–2 h. Sixteen participants reported being able to receive care always or most of the time over the past year, with the remainder reporting they could receive care some of the time or rarely. However, 16 respondents said getting access to care over the past year was somewhat inconvenient or neither convenient nor inconvenient, and 12 participants reported accessing care was either somewhat or very convenient.
About the Technology Used Today
Participants had relatively favorable ratings of the technology used for their behavioral healthcare encounters within the RTeC (Table 1). The majority of participants did not need any help using the technology, and 20 participants agreed that “The technology did not interfere with my conversation with my provider” (M=3.75, SD=0.52). Ratings of network connectivity were expected to be on the lower end because of the remoteness of the end points, and ratings of the video quality were much lower than ratings of the audio quality. Three participants agreed with the item “The image on the screen was stable throughout the session” (M=2.46, SD=0.74), and 19 agreed with the item “The sound was stable throughout the session” (M=3.54, SD=0.84). Despite some concerns with the network most participants agreed with the item “I am satisfied with the quality of the service I received from this technology” (M=4.0, SD=0.61).
Participant Attitudes About the Technology Used (n=28)
ABOUT THE OFFICE SETTING
The usability of the RTeC office setting and ratings of beneficiary acceptability including perceptions of privacy, safety, and comfort are of critical importance to evaluating and even recommending future use. Ratings of the physical space were relatively low, with a slight majority of respondents agreeing that they did not feel confined in the setting (M=3.54, SD=0.58) and about half of participants reporting a neutral opinion about the quality of the RTeC construction (M=3.46, SD=0.58). However, nearly all of the respondents reported feeling relaxed in the setting (M=3.96, SD=0.34) and agreed with the items “I did not have concerns about my privacy during my healthcare visit” (M=4.0, SD=0.38) and “This was an acceptable place to receive healthcare” (M=3.81, SD=0.62). Questionnaire responses can be found in Table 2.
Participant Attitudes About the Office Setting (n=28)
Overall Perceptions
The RTeC was placed in American Samoa as a way to provide better access to behavioral healthcare for MHS beneficiaries on the island. All respondents agreed with the statement “I was better able to access my provider using this service” (M=4.31, SD=0.47), and almost all of the respondents said they would use the service again (M=4.26, SD=0.53). One participant disagreed with the statement “I found this service to be an acceptable way to receive healthcare” (M=4.07, SD=0.72), whereas nearly all of participants agreed that “Overall I am satisfied with the quality of care I received” (M=4.14, SD=0.45). Questionnaire responses can be found in Table 3.
Participant Attitudes About Overall Perceptions (n=28)
Discussion
The goal of this exploratory evaluation study was to collect data to assess the usability of and patient satisfaction with a modified shipping container as a setting in which to receive telebehavioral healthcare. The results suggest that the RTeC is safe and private and ultimately an appropriate telebehavioral healthcare setting. It is interesting that these positive impressions were reported despite some issues with the video connections and physical comfort of the office. To better understand these impressions it is important to note that there were fewer reported issues with audio connections, suggesting an overall sufficient connection, and that without the RTeC many of these beneficiaries would need to travel approximately 2,500 miles to Hawaii for care or make the difficult decision to go without MHS-delivered care. Although some beneficiaries may perceive financial or other incentives traveling to Hawaii for care, all participants reported that they were better able to access a provider using the RTeC, and the vast majority agreed that this service saved them time traveling to another clinic, further suggesting that telehealth technology and the RTeC are appropriate for behavioral healthcare in remote locations. It is difficult to provide an exact estimate of cost avoidance without specific information about clinical diagnoses and treatment protocols. However, substantial cost avoidance is likely given that a single trip to Hawaii to receive care was at least $2,600 at the time of the study. The American Samoa trial has therefore been very successful, with significant expansion of service, positive reports of usability and satisfaction, and cost avoidance given even conservative estimates.
There are limitations to this evaluation, including a relatively small sample size, lack of provider feedback, and use of a questionnaire that has not been validated. However, these original data provide useful information about how RTeCs and other DTeCs can provide real-life improvements in access to behavioral healthcare. More specifically, these data provide insight into usability considerations and inform future research and DTeC development. T2 has already incorporated these data into the development of second-generation RTeCs with a focus on upgrading construction and configuration of the office interior. These data are also informing the development of other DTeCs, such as a solution built on a truck chassis and one using individual sound booths to support private telehealth connections in remote areas.
Addressing policies that define where telehealthcare can be delivered will be important to utilize the full potential of these deployable platforms as they have the capacity to be placed in wide-ranging locations that lack clinical services. However, most standards require care be delivered to locations with on-site clinical oversight, thus requiring that these deployable units be colocated with existing clinical facilities or be staffed with clinical personnel. This limits how these units can be optimized to bridge access to healthcare gaps in locations where little to no care is available. A recent review article suggests that telebehavioral health safety concerns can be effectively managed in clinically unsupervised settings. 26 Although limited in scope, this review is an important step in the process to expand conceptualizations of not only where telehealthcare can be delivered, but how innovative solutions such as DTeCs can enhance access to care for those often most in need.
Footnotes
Acknowledgments
The authors wish to thank Patrick Doar and Apelu Manuleleua for their assistance with this research.
Disclosure Statement
No competing financial interests exist.
