Identifying Those Whom Health Promotion Hardly Reaches

Abstract

To understand what circumstances lend groups to be recognized as hardly reached by health services and research, we systematically reviewed studies that identified their priority populations as hard to reach. We classified attributes of hardly reached groups into cultural/environmental, individual, and demographic domains. Of the 334 identified studies, 78.74% used attributes that were classified into the cultural/environmental, 74.85% the individual, and 50% the demographic domain to identify those hardly reached. Of all possible combinations of domains, the most common was the use of all three domains (28.74%). Overall, papers were more likely to use attributes to identify their hardly reached population that fell into more than one domain (74.85%) compared to only one domain (25.15%; χ², p < .0001). Through this review, we identified the attributes of those who have been identified as hardly reached in published research. No single attribute is used to identify those who are hardly reached. This reflects a socioecological perspective, emphasizing that both intrapersonal and external elements may cause interventions to fail to reach those intended. Moreover, the focus not on populations hardly reached but on the attributes of those hardly reached suggests objectives for interventions to reach them better.

Keywords

hard to reach socioecological framework social determinants health equity hardly reached

Introduction

Many factors may account for health care and prevention interventions’ failure to reach those they are intended to help. Health practitioners and researchers have used the term “hard to reach” to identify such groups (Cassady et al., 2012; Cooper, Levay, Lorenc, & Craig, 2014; Hallum-Montes, Morgan, Rovito, Wrisby, & Anastario, 2013; Johnston et al., 2013; Kauffman et al., 2013; Minetti, Hurtado, Grais, & Ferrari, 2014). There are several problems with this. First, the term hard to reach appears to reify the problem as something within people that makes them hard to reach. Second, hard to reach definitions from the literature suggest there are a wide range of reasons individuals or groups may be hardly reached (Bonevski et al., 2014; Lambert & Wiebel, 1990; Sydor, 2013).

Failure of prevention and treatment programs to reach those they intend to help leads to avoidable health burdens and costs. It intensifies health disparities (Raphael & Beal, 2010; U.S. Department of Health and Human Services, 2014) that not only worsen population health but also burden health system finance and economic productivity (Bauer, Briss, Goodman, & Bowman, 2014; Centers for Medicaid and Medicare Services, 2012). Furthermore, failure to reach key groups may risk disease outbreak in the general population, such as when high-risk groups do not access immunizations or other infectious disease control practices (Arnot, 1998; Kim, Johnstone, & Loeb, 2011; Measles eradication, 1997). Additionally, failure to include groups in research denies them the physiological, psychological, and social benefits of trial participation (Dhalla & Poole, 2013; Rogers, 2004), limits the ability to determine safety of health innovations for hardly reached populations (Albain, Unger, Crowley, Coltman, & Hershman, 2009), impedes the understanding of health disparities (Singh, Azuine, & Siahpush, 2012), and compromises the generalizability of findings to the general population (Johnson, 1990).

Commonly, populations have been classified as hard to reach without researchers or practitioners specifying the attributes that lead to this classification. When we use the blanket term hard to reach in this way, the term’s generality and consequent ambiguity impedes detailed understanding of priority populations (Freimuth & Mettger, 1990) and generalizability of research findings (Shadish, Cook, & Campbell, 2002). The failure to understand priority populations and the lack of generalizable research findings hinders the overall reach and effectiveness of developed health promotion programs (Glasgow, Vogt, & Boles, 1999). In this review, we take a fundamentally different approach in how we identify the “hardly reached.” Through identifying groups as hardly reached by sets of attributes rather than the common approach of classifying populations as hard to reach (such as the homeless or illegal immigrants), we encourage researchers and practitioners to address the circumstances of those hardly reached in order to create effective health promotion programs for those too often not reached as well as the more general population.

As a step towards a more comprehensive understanding of the problems surrounding the failure of interventions to reach all they might, this systematic review aims to identify the attributes of those who have been identified as hardly reached in published research. It is also of interest to determine if attributes describing those who are hardly reached vary by health topic. For example, do individuals with acute infectious diseases versus chronic diseases differ in the attributes associated with their too often not being reached? Such information could help inform disease-specific interventions. Therefore, this review addresses:

What attributes are used to classify individuals or groups as being hardly reached?

How, if at all, do these attributes vary across different health topics?

Method

This systematic review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher, Liberati, Tetzlaff, Altman, & PRISMA Group, 2010). The protocol was not registered. Because this review assessed how articles have utilized the concept, hard to reach, and did not evaluate interventions, we modified PRISMA checklist items accordingly (Liberati et al., 2009). We conducted a systematic review that answered the following questions: How are hardly reached groups identified? How do these identifying attributes vary across different health problems, if at all?

We determined inclusion and exclusion criteria a priori. Articles had to define their study populations as hard to reach or in similar terms, detailed subsequently. Included articles were published in English between 2009 and 2014. We searched the most recent 5 years of literature in order to capture the current usage of hard to reach (Cronin, Ryan, & Coughlan, 2008). We excluded articles if they (1) referred to “hard to treat” clinical conditions, such as antibiotic resistant bacteria or (2) did not identify who comprised their hard to reach group.

We conducted an electronic search in PubMed on September 17, 2014 in consultation with a reference librarian. We identified search terms based on a previously published search query used to guide the identification and management of tuberculosis among hard to reach groups (Cooper et al., 2014). While this previous review was limited to a specific disease, the current review included all health outcomes. Our modified filter was “hard to reach” OR “hard to locate” OR “hard to find” OR “hard to treat” OR “difficult to locate” OR “difficult to engage” OR “difficult to reach” OR “difficult to find” OR “difficult to treat” (or hyphenated versions of these, e.g., “hard-to-reach”). One author screened citations from the initial search for eligibility according to inclusion and exclusion criterion. A second author independently screened 10% of these citations to assess screening reliability (Krippendorf’s α = .9443). All disagreements and cases in which the coder had questions were resolved through consensus.

Articles eligible after the first screen were assessed in a full-text review using the same inclusion and exclusion criteria, and data were then extracted from included articles. The first author assessed the retrieved full texts for inclusion and, if included, extracted the following elements in a standardized format: author’s name(s), year of publication, research or health services focused, study type, study size, stigmatization of health issue, primary health focus, location, and categories of hardly reached populations (evidence table available upon request to the corresponding author). We developed the data extraction table iteratively, and we added categories that emerged from attributes used to identify groups as hardly reached until we reached saturation and revised previously coded papers accordingly. A second author independently reviewed 10% of the full texts to confirm full-text inclusion/exclusion (100% agreement) and to determine abstraction reliability (Krippendorf’s α = .6541–1.000 across different categories, Table 1). As mentioned earlier, all disagreements and cases of uncertainty were resolved by consensus. The first author screened the reference lists of included articles for potential additions to the review. Since the current review focuses on term utilization rather than specific outcome data, it was not necessary to report risk of bias or summary measures of studies.

Table 1.

Percentage of Papers That Used Features From Each Domain and Category to Identify Hardly Reached Groups, With Interrater Reliability for Each Category.

Domain	Total Domain Percentage^a	Category	Category Percentage^b	Krippendorf’s α
Cultural and environmental	78.74	Ethnicity	31.74	.89
		Geography	27.25	.91
		Social network	16.47	.81
		Infrastructure	38.02	.78
		Discrimination	24.25	1.00
		Hidden	12.57	1.00
Individual	74.85	Psychological and/or cognitive factors	38.92	.93
		Occupation	13.77	.75
		Sexual orientation	11.38	1.00
		Transiency	25.15	.83
		Substance use	17.96	.75
		History of incarceration	6.59	1.00
		Disability	4.79	1.00
Demographic	50.00	Socioeconomic status	31.74	1.00
		Age	19.76	.65
		Sex	8.98	.78

Note. N = 334.

^aTotal domain percentages calculated by the number of articles that reported features classified within the specified domain divided by the total number of articles (N = 334). Domains are therefore not mutually exclusive. ^b Category percentages calculated by the number of articles that reported features within a specific category divided by the total number of articles (N = 334). Categories are therefore not mutually exclusive.

The first author analyzed and summarized key results according to how papers identified their hardly reached population, and a third author verified the analysis. Attributes used to identify hardly reached groups were divided into 16 categories, which were then allocated to one of three domains: demographic, cultural/environmental, and individual. Papers could be assigned multiple attributes, categories, and domains. Because some categories may fall into multiple domains, classification decisions were discussed and resolved in author research meetings. The percentage of papers that used attributes from each category as a basis for identifying hardly reached groups was calculated for the entire sample of papers as well as for each of several health topics. Chi-square analyses determined differences in the percentages of papers that were classified into one, two, or all three domains. All statistical analyses were conducted using SAS^® software, version 9.4.

Results

Literature Search

The PubMed database search provided 2,612 citations, of which 2,260 were excluded due to not meeting inclusion criteria (i.e., irrelevant to the topic or discussed clinically difficult to treat conditions, such as antibiotic resistant bacteria; Figure 1). The full texts of the remaining 352 articles were examined in greater detail, and 19 articles were excluded because they also focused on clinically difficult to treat conditions (n = 16) or did not explicitly define the hard to reach group (n = 3). Screening the reference lists of included papers provided an additional paper for inclusion, resulting in 334 included articles. No unpublished articles were obtained.

Figure 1.

Flow diagram for the systematic review of articles identifying hardly reached populations from a PubMed search from January 2009 to September 2014.

Characteristics of the Hardly Reached

The review identified attributes used to classify individuals or groups as hardly reached in the 334 included papers. These attributes were assigned to one or more of 16 categories, for example, ethnicity, psychological and/or cognitive factors, and socioeconomic status (SES). Applying a broad ecological approach (Sallis & Owen, 2015), we divided these 16 categories into three domains: individual, demographic, and cultural and environmental (Table 1).

The individual domain included seven categories: psychological/cognitive factors, occupation, sexual orientation, transiency, substance use, history of incarceration, and disability. The demographic domain included SES, age, and sex. Each of these categories included multiple attributes. To illustrate, the category “SES” included both low and high SES (Huang, Maman, & Pan, 2012; Kauffman et al., 2013; Print, 2013; Truong et al., 2013). Articles reported young, middle, and old “age” to identify hardly reached individuals. In the category “sex,” each of male and female were used to identify individuals as hardly reached (Straka et al., 2011; Vandelanotte et al., 2013). Within the six categories in the cultural and environmental domain, “social network” included both low and high social support (Nelson et al., 2012; Riggs et al., 2014). “Infrastructure” included attributes of the environment that contributed to accessing health care services, including resource availability and transportation infrastructure (Cooper et al., 2014; Drainoni et al., 2014; Metcalfe & Sexton, 2014; Priebe et al., 2012). “Hidden” referred to populations of low numbers or groups overlooked by common research methodologies (Fernandez-Balbuena et al., 2014; Jones et al., 2014; Tang et al., 2013). Papers reported attributes categorized in anywhere from 1 to 13 categories for identifying their hardly reached population, mean = 3.28 categories.

As shown in Table 1, 74.85% of papers utilized attributes that fell within the individual domain. Within this broad domain, “psychological and/or cognitive factors” was the most common category, and this category was also the most common of all categories, with 38.92% of papers utilizing psychological or cognitive attributes to identify hardly reached groups. Many papers also used attributes to identify hardly reached groups classified into the cultural and environmental domain (78.74%). Within this domain, common categories included “ethnicity” (31.74%) and infrastructure (38.02%). The most frequent demographic category was SES with 31.74% of papers using attributes from this category to identify groups as hardly reached (Table 1).

Many papers used attributes within cultural and environmental (78.74%) and individual (74.85%) domains to identify hardly reached groups (Table 2). Of all possible combinations of the domains from which attributes might be drawn to identify groups as hardly reached, the most common, 28.74% of all papers, was that of all three domains (Table 2). Overall, 74.85% of papers utilized attributes from at least two domains compared to 25.15% that used attributes from only one domain (χ², p < .0001).

Table 2.

Percentage of Papers That Used Features Classified Into Different Combinations of Domains to Identify Hardly Reached Groups.

	Total Percentage	Combination Percentage	Environmental/Cultural	Individual	Demographic
Three domains	28.74	28.74	X	X	X
Two domains	46.11	26.65	X	X
		11.38	X		X
		8.08		X	X
One domain	25.16	11.98	X
		11.38		X
		1.8			X

Note. N = 334.

Hardly Reached, Stigmatization, and Specific Health Topics

We divided health topics along two dimensions: whether or not they are stigmatized and health focus (Table 3). Stigmatized health topics included issues that are widely recognized as stigmatizing, including sexual health (Price, 1997), HIV/AIDS (Monteiro, Villela, & Soares, 2013), tuberculosis (Juniarti & Evans, 2011), Hepatitis C (Treloar, Rance, & Backmund, 2013), substance use (van Boekel, Brouwers, van Weeghel, & Garretsen, 2013), and mental health (Ando, Yamaguchi, Aoki, & Thornicroft, 2013; Parcesepe & Cabassa, 2013). Although the distinction is somewhat arbitrary and a wide range of health challenges are sometimes stigmatized, we identified as nonstigmatized maternal and child health, early childhood health, oral care, vaccination/immunization, occupational safety, cancer, some infectious diseases (i.e., pneumonia), asthma, diabetes, arthritis, cardiovascular disease, diet, and/or exercise.

Table 3.

Percentage of Papers That Used Features Within Each Domain and Category to Identify Hardly Reached Groups Among Two Different Health Topic Categorizations.

Domain	Category	Stigmatization		Primary Health Focus
Domain	Category	Stigmatized (n = 161)	Nonstigmatized (n = 113)	Prevention (n = 102)	Infectious Disease (n = 86)	Chronic Condition (n = 57)	Substance Use (n = 29)	Access (n = 29)	Research (n = 31)
Cultural and environmental	Total, cultural/environmental	72.67	83.19	77.45	82.56	71.93	68.97	86.21	87.10
	Ethnicity	26.09	37.17	33.33	19.77	47.37	20.69	20.69	51.61
	Geography	14.91	40.71	37.25	22.09	17.54	10.34	44.83	25.81
	Social network	17.39	14.16	15.69	12.79	21.05	17.24	27.59	9.68
	Infrastructure	33.54	45.13	42.16	43.02	31.58	24.14	62.07	12.90
	Discrimination	36.65	9.73	14.71	41.86	21.05	24.14	24.14	12.90
	Hidden	19.88	0.88	4.90	24.42	1.75	20.69	0	29.03
Individual	Total, individual	89.44	56.64	60.78	83.72	84.21	89.97	72.41	67.74
	Psychological/cognitive	40.37	38.94	35.29	19.77	70.18	55.17	44.83	25.81
	Occupation	21.74	3.54	7.84	27.91	8.77	6.90	6.90	16.13
	Sexual orientation	21.12	0	4.90	31.40	0	3.45	3.45	9.68
	Transiency	25.47	17.70	17.65	27.91	19.30	27.59	37.93	38.71
	Substance use	29.19	5.31	4.90	36.05	3.51	51.7	10.34	12.90
	History of incarceration	9.32	3.54	5.88	9.30	3.51	10.34	0	9.68
	Disability	3.73	3.54	5.88	0	7.02	0	6.90	12.90
Demographic	Total, demographics	47.20	54.87	53.92	39.53	61.40	48.28	55.17	41.94
	Socioeconomic status	24.22	37.17	34.31	25.58	29.82	24.14	48.28	35.48
	Age	22.36	19.47	23.53	11.63	31.58	20.69	10.34	16.13
	Sex	8.07	13.27	11.76	8.14	7.02	17.24	3.45	3.23

Note. N = 334.

Papers addressing stigmatized conditions were more likely than those addressing nonstigmatized conditions to use discrimination (36.65% vs. 9.73% of papers), hidden—attributes of groups overlooked by common research methodologies (19.88% and 0.88%), sexual orientation (21.12% and 0%), substance use (29.19% and 5.31%), and history of incarceration (9.32% and 3.54%) to identify those who are hardly reached. These are congruent with many sources of or factors associated with stigmatization. Papers addressing stigmatized conditions were also less likely to use geography (14.91% and 40.71%) as an identifying attribute. The latter may reflect connections between employment status and stigmatizing health conditions. Apart from these differences, however, the distribution and patterns of use of attributes and their categories and domains were similar for the two types of papers.

The six primary health foci included (1) infectious disease (HIV/AIDS, Hepatitis C, tuberculosis, Chagas, malaria, pneumonia, and trachoma), (2) chronic condition (mental health, cancer, asthma, diabetes, arthritis, and cardiovascular disease), (3) substance use, (4) prevention (maternal and child health, early childhood health, vaccination/immunization, diet and/or exercise, occupational safety, sexual health, and oral care), (5) health research (studies concerned with engaging hardly reached groups in research, that is, alternative sampling and recruitment method development and/or evaluation), and (6) health care access.

The percentage using attributes from each of the domains to define hardly reached varied across condition. The percentage using attributes from the cultural and environmental domain ranged from 68.97% for substance use to 86.21% for access. The percentage using attributes from the individual domain ranged from 60.78% for prevention to 89.97% for substance use, and the percentages using attributes from the demographic domain ranged from 39.53% for infectious diseases to 61.40% for chronic conditions. Inspection of Table 3 reveals appreciable differences at the level of categories, for example, ethnicity being used to identify hardly reached individuals in 47.37% of papers addressing chronic conditions but in only 19.77% of papers addressing infectious diseases. The overall pattern across these health topics was, however, consistent with that observed for the entire sample of papers: use of a wide range of attributes to identify those who are hardly reached from each of the domains of cultural and environmental, individual, and demographic attributes.

Discussion

Variety of Hardly Reached

As this review illustrates, there are many specific categories among the broad individual, demographic, and cultural and environmental domains of attributes that identify groups as hardly reached. No single category nor domain is dominant. For example, even the most common category of attributes utilized to identify hardly reached groups, psychological and/or cognitive factors, was not used in the majority of papers. Moreover, this general pattern of no single category nor domain predominating was observed across a variety of health challenges and among studies of both stigmatized and nonstigmatized conditions.

Rather than any single category or domain being dominant, the major pattern observed was the use of a variety of attributes spanning several categories and domains to identify those who are hardly reached. Almost three-quarters of papers utilized attributes from more than one domain. Indeed, the most frequent pattern for papers was to use attributes from all three to identify those hardly reached, and 28.74% of all papers did so. Further reflecting the pattern of invoking a wide range of attributes, the most frequently used categories of attributes to identify hardly reached groups—“psychological and/or cognitive factors” (38.92% of papers), ethnicity (31.74%), infrastructure (38.02%), and SES (31.74%)—represented all three domains. This general pattern was also true for both stigmatized and nonstigmatized conditions and across a range of health problems (infectious diseases, chronic diseases, etc.). This reflects the socioecological model (Stokols, 1992), emphasizing that both intrapersonal characteristics and external settings jointly influence being perceived as hardly reached.

For several of the attributes identified, more than one value or position may sometimes be associated with being hardly reached. For example, low social support was reported as a precursor for low service utilization (Riggs et al., 2014; Salem, Nyamathi, Idemundia, Slaughter, & Ames, 2013; Zanini et al., 2013), but parents with high social support may be more difficult to engage in child asthma care coaching (Nelson et al., 2012). In another example, although low SES groups with limited resources often do not access services (Bryant, Bonevski, Paul, & Lecathelinais, 2013; Kauffman et al., 2013; Kesse-Guyot et al., 2013; Sleed, James, Baradon, Newbery, & Fonagy, 2013; Van Dyck, Veitch, De Bourdeaudhuij, Thornton, & Ball, 2013), groups were identified as hardly reached based on high SES due to opportunities that compete with acquiring care as well as avoiding services for fear of stigmatization (Huang et al., 2012; Wright & Polack, 2006). So too with age, middle-aged men were difficult to engage in a physical activity intervention due to competing demands of work and family (Vandelanotte et al., 2013). Adolescents were difficult to engage in their own health care because of present good health and low perceived risks (Malbon & Romo, 2013). Older adults were hardly reached due to limited mobility and access to health services (Hinrichs et al., 2011). Sex was associated with reach based upon power relations—advantage to males and/or expected gender roles—females more likely to participate (Straka et al., 2011; Vandelanotte et al., 2013). What attributes cause interventions to fail to reach their intended audiences clearly depend on multiple factors surrounding the interventions and their audiences.

The Category Mistake

The variety of categories that identify the hardly reached indicates that hardly reached reflects a set of attributes not a single characteristic of any group or individual. This may seem like a nuance, but it has significant implications. At its worst, conceptualizing hard to reach as a single characteristic may evoke preconceptions about groups that lead to their stigmatization and depiction as powerless, apathetic, and isolated.

These issues may fit what the philosopher Gilbert Ryle describes as a category mistake: conceptualizing the category as something other than the elements that comprise it. In Ryle’s example, a visitor to Oxford who indicates appreciation of seeing the Bodleian Library, New College, Magdalen College, St. Catherine’s College, and the Ashmolean Museum but then asks “Where is Oxford?” is committing the category error. The university, Oxford, is the libraries, the colleges, museums, and so on that comprise it not something apart from or in addition to them (Ryle, 2009). We too often treat hard to reach as though it were some fixed characteristic of individuals or groups, rather than the effect of the wide variety of attributes, categories, and domains documented here. Hardly reached is not a thing or characteristic separate from the diverse attributes that are associated with being hardly reached. A group of low-income, isolated older adults whom interventions fail to reach does not share or possess some characteristic of being hard to reach apart from their low income, isolation, age, and the failure of the interventions in question to reach them.

Moving Forward

Through identifying individuals as hardly reached by sets of attributes rather than by population groupings, we encourage researchers and practitioners to consider and address the circumstances of being hardly reached in order to further understand priority populations and create more generalizable and effective health promotion programs. The 16 categories that emerged from this review can be utilized by researchers and practitioners to identify actionable objectives for interventions so that they do reach those they are intended to help. When uncovering the attributes that are used to identify groups as hardly reached, care must be taken not to infer causes from associations. Uncovering attributes of the hardly reached is used to contextualize where the problem occurs but does not fix the cause of the problem. For example, sex or age or income may identify those hardly reached in a particular instance, however they are not necessarily the causes of the problem. Thus, we suggest conceptualizing hardly reached as reflecting a wide range of attributes that vary among circumstances. They may point to the sources of problems and guide more vigorous efforts to reach individuals, but they are not necessarily the cause of interventions’ failure to reach those they intend. Labeling groups hard to reach may reify the problem and constitute a kind of victim blaming, whereas conceptualizing hardly reached as a set of attributes directs attention to the actionable characteristics associated with interventions or services hardly reaching them.

Limitations

A study limitation was the primary use of only one coder. However, agreement in the sample of 10% indicated it was neither efficient nor necessary to dually code all full texts. Another weakness was the limitation to PubMed; however, the breadth and number of articles retrieved suggest this did not bias the review.

Conclusion

There are many and varied attributes that may identify groups as hardly reached. This range makes clear that no single attribute of individuals or groups makes them hardly reached, and we need to reconceptualize hardly reached as a set of problems not as a single attribute of populations such as ethnic minorities, “the unemployed,” and so on. The 16 categories that emerged from this review can provide a framework to understand the circumstances of those who are too often hardly reached. Recognizing the many attributes, attributes, circumstances, or barriers of those who are hardly reached may help identify targets for which interventions that may lead to interventions better reaching those they intend.

Footnotes

Acknowledgment

We would like to thank Rachael Posey, Liaison Librarian at the University of North Carolina at Chapel Hill, who assisted in the literature search. We would also like to thank Barbara Rimer, Dr. P. H. for bringing our attention to the recognition that those whom we too often fail to engage are better thought of as “hardly reached” than “hard to reach”.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this work was provided by Peers for Progress of the American Academy of Family Physicians Foundation and by contributions from the Eli Lilly and Company Foundation and the Bristol-Myers Squibb Foundation. Edwin Fisher received compensation from the American Academy of Family Physicians Foundation as a consultant regarding matters unrelated to the focus of this article.

References

Albain

K. S.

Unger

J. M.

Crowley

J. J.

Coltman

C. A.

Jr. Hershman

D. L.

(2009). Racial disparities in cancer survival among randomized clinical trials patients of the southwest oncology group. Journal of the National Cancer Institute, 101, 984–992.

Ando

Yamaguchi

Aoki

Thornicroft

(2013). Review of mental-health-related stigma in Japan. Psychiatry and Clinical Neurosciences, 67, 471–482.

Arnot

(1998). The evolving efforts to control hepatitis B virus. The Pediatric Infectious Disease Journal, 17, S26–S29.

Bauer

U. E.

Briss

P. A.

Goodman

R. A.

Bowman

B. A.

(2014). Prevention of chronic disease in the 21st Century: Elimination of the leading preventable causes of premature death and disability in the USA. Lancet, 384, 45–52.

Bonevski

Randell

Paul

Chapman

Twyman

Bryant

… Hughes

(2014). Reaching the hard-to-reach: A systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Medical Research Methodology, 14, 42.

Bryant

Bonevski

Paul

C. L.

Lecathelinais

C. L.

(2013). A cross-sectional survey of health risk behaviour clusters among a sample of socially disadvantaged Australian welfare recipients. Australian and New Zealand Journal of Public Health, 37, 118–123.

Cassady

Castaneda

Ruelas

M. R.

Vostrejs

M. M.

Andrews

Osorio

(2012). Pandemics and vaccines: Perceptions, reactions, and lessons learned from hard-to-reach Latinos and the H1N1 campaign. Journal of Health Care for the Poor and Underserved, 23, 1106–1122.

Centers for Disease Control and Prevention, (1997). Measles eradication: Recommendations from a meeting cosponsored by the world health organization, the pan american health organization, and CDC. Recommendations and Reports: Morbidity and Mortality Weekly Report. Centers for Disease Control, 46, 1–20.

Centers for Medicaid and Medicare Services. (2012). Chronic conditions among medicare beneficiaries: Chartbook. 2012 ed. Baltimore, MD: U.S. Department of Health and Human Services.

10.

Cooper

Levay

Lorenc

Craig

G. M.

(2014). A population search filter for hard-to-reach populations increased search efficiency for a systematic review. Journal of Clinical Epidemiology, 67, 554–559.

11.

Cronin

Ryan

Coughlan

(2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17, 38–43.

12.

Dhalla

Poole

(2013). Motivators to participation in medical trials: The application of social and personal categorization. Psychology, Health & Medicine, 18, 664–675.

13.

Drainoni

M. L.

Farrell

Sorensen-Alawad

Palmisano

J. N.

Chaisson

Walley

A. Y.

(2014). Patient perspectives of an integrated program of medical care and substance use treatment. AIDS Patient Care and STDs, 28, 71–81.

14.

Fernandez-Balbuena

de la Fuente

Hoyos

Rosales-Statkus

M. E.

Barrio

Belza

M. J.

, & Madrid Rapid HIV Testing Group. (2014). Highly visible street-based HIV rapid testing: Is it an attractive option for a previously untested population? A cross-sectional study. Sexually Transmitted Infections, 90, 112–118.

15.

Freimuth

V. S.

Mettger

(1990). Is there a hard-to-reach audience? Public Health Repors, 105, 232–238.

16.

Glasgow

R. E.

Vogt

T. M.

Boles

S. M.

(1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322–1327.

17.

Hallum-Montes

Morgan

Rovito

H. M.

Wrisby

Anastario

M. P.

(2013). Linking peers, patients, and providers: A qualitative study of a peer integration program for hard-to-reach patients living with HIV/AIDS. AIDS Care, 25, 968–972.

18.

Hinrichs

Moschny

Brach

Wilm

Klaassen-Mielke

Trampisch

Platen

(2011). Effects of an exercise programme for chronically ill and mobility-restricted elderly with structured support by the general practitioner’s practice (HOMEfit)—Study protocol of a randomised controlled trial. Trials, 12, 263.

19.

Huang

Maman

Pan

(2012). Understanding the diversity of male clients of sex workers in China and the implications for HIV prevention programmes. Global Public Health, 7, 509–521.

20.

Johnson

M. S.

(1990). Perspectives in medicine. generalizability of homogenous samples in clinical trials. Journal of the Association for Academic Minority Physicians, 1, 31–33.

21.

Johnston

L. G.

Prybylski

Raymond

H. F.

Mirzazadeh

Manopaiboon

McFarland

(2013). Incorporating the service multiplier method in respondent-driven sampling surveys to estimate the size of hidden and hard-to-reach populations: Case studies from around the world. Sexually Transmitted Diseases, 40, 304–310.

22.

Jones

H. E.

Hickman

Welton

N. J.

De Angelis

Harris

R. J.

Ades

A. E.

(2014). Recapture or precapture? Fallibility of standard capture-recapture methods in the presence of referrals between sources. American Journal of Epidemiology, 179, 1383–1393.

23.

Juniarti

Evans

(2011). A qualitative review: The stigma of tuberculosis. Journal of Clinical Nursing, 20, 1961–1970.

24.

Kauffman

K. S.

Dosreis

Ross

Barnet

Onukwugha

Mullins

C. D.

(2013). Engaging hard-to-reach patients in patient-centered outcomes research. Journal of Comparative Effectiveness Research, 2, 313–324.

25.

Kesse-Guyot

Andreeva

Castetbon

Vernay

Touvier

Mejean

… Hercberg

(2013). Participant profiles according to recruitment source in a large web-based prospective study: Experience from the Nutrinet-Sante study. Journal of Medical Internet Research, 15, e205.

26.

Kim

T. H.

Johnstone

Loeb

(2011). Vaccine herd effect. Scandinavian Journal of Infectious Diseases, 43, 683–689.

27.

Lambert

E. Y.

Wiebel

(1990). The collection and interpretation of data from hidden populations. Washington, DC: US Department of Health and Human Services, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute on Drug Abuse.

28.

Liberati

Altman

D. G.

Tetzlaff

Mulrow

Gotzsche

P. C.

Ioannidis

J. P.

… Moher

(2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration. BMJ Clinical Research Ed, 339, b2700.

29.

Malbon

Romo

(2013). Is it ok 2 txt? Reaching out to adolescents about sexual and reproductive health. Postgraduate Medical Journal, 89, 534–539.

30.

Metcalfe

S. E.

Sexton

E. H.

(2014). An academic-community partnership to address the flu vaccination rates of the homeless. Public Health Nursing, 31, 175–182.

31.

Minetti

Hurtado

Grais

R. F.

Ferrari

(2014). Reaching hard-to-reach individuals: Nonselective versus targeted outbreak response vaccination for measles. American Journal of Epidemiology, 179, 245–251.

32.

Moher

Liberati

Tetzlaff

Altman

D. G.

, & PRISMA Group. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8, 336–341.

33.

Monteiro

S. S.

Villela

W. V.

Soares

P. S.

(2013). The interaction between axes of inequality in studies on discrimination, stigma and HIV/AIDS: Contributions to the recent international literature. Global Public Health, 8, 519–533.

34.

Nelson

K. A.

Highstein

Garbutt

Trinkaus

Smith

S. R.

Strunk

R. C.

(2012). Factors associated with attaining coaching goals during an intervention to improve child asthma care. Contemporary Clinical Trials, 33, 912–919.

35.

Parcesepe

A. M.

Cabassa

L. J.

(2013). Public stigma of mental illness in the United States: A systematic literature review. Administration and Policy in Mental Health, 40, 384–399.

36.

Price

(1997). Culture and condoms: A review of research literature. Modern Midwife, 7, 28–31.

37.

Priebe

Matanov

Schor

Strassmayr

Barros

Barry

M. M.

… Gaddini

(2012). Good practice in mental health care for socially marginalised groups in Europe: A qualitative study of expert views in 14 countries. BMC Public Health, 12, 248.

38.

(2013). Accessing hard to reach groups, travelers and looked after children. Human Vaccines & Immunotherapeutics, 9, 1372–1373.

39.

Raphael

J. L.

Beal

A. C.

(2010). A review of the evidence for disparities in child vs adult health care: A disparity in disparities. Journal of the National Medical Association, 102, 684–691.

40.

Riggs

Gussy

Gibbs

van Gemert

Waters

Kilpatrick

(2014). Hard to reach communities or hard to access services? Migrant mothers’ experiences of dental services. Australian Dental Journal, 59, 201–207.

41.

Rogers

W. A.

(2004). Evidence based medicine and justice: A framework for looking at the impact of EBM upon vulnerable or disadvantaged groups. Journal of Medical Ethics, 30, 141–145.

42.

Ryle

(2009). Descartes’ myth. In Tanney

Julia

(Ed.), The concept of mind (p. 6). New York, NY: Routledge.

43.

Salem

B. E.

Nyamathi

Idemundia

Slaughter

Ames

(2013). At a crossroads: Reentry challenges and healthcare needs among homeless female ex-offenders. Journal of Forensic Nursing, 9, 14–22.

44.

Sallis

J. F.

Owen

(2015). Ecological models of health behavior. In Glanz

Rimer

B. K.

Viswanath

(Eds.), Health behavior: Theory, research and practice (5th ed., pp. 43–64). San Francisco, CA: Jossey-Bass.

45.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. New York, NY: Houghton Mifflin Company.

46.

Singh

G. K.

Azuine

R. E.

Siahpush

(2012). Global inequalities in cervical cancer incidence and mortality are linked to deprivation, low socioeconomic status, and human development. International Journal of MCH and AIDS, 1, 17.

47.

Sleed

James

Baradon

Newbery

Fonagy

(2013). A psychotherapeutic baby clinic in a hostel for homeless families: Practice and evaluation. Psychology and Psychotherapy, 86, 1–18.

48.

Stokols

(1992). Establishing and maintaining healthy environments: Toward a social ecology of health promotion. The American Psychologist, 47, 6–22.

49.

Straka

Zubor

Novomesky

Stuller

Krajcovic

Kajo

Danko

(2011). Fatal alcohol intoxication in women: A forensic autopsy study from Slovakia. BMC Public Health, 11, 924.

50.

Sydor

(2013). Conducting research into hidden or hard-to-reach populations. Nurse Researcher, 20, 33–37.

51.

Tang

Yang

Mahapatra

Huan

Yan

… Detels

(2013). Feasibility of recruiting a diverse sample of men who have sex with men: Observation from Nanjing, China. PloS One, 8, e77645.

52.

Treloar

Rance

Backmund

(2013). Understanding barriers to hepatitis C virus care and stigmatization from a social perspective. Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America, 57, S51–S55.

53.

Truong

H. M.

Grasso

Chen

Y. H.

Kellogg

T. A.

Robertson

Curotto

… McFarland

(2013). Balancing theory and practice in respondent-driven sampling: A case study of innovations developed to overcome recruitment challenges. PloS One, 8, e70344.

54.

U.S. Department of Health and Human Services. (2014). Healthy people 2020. Retrieved from https://www.healthypeople.gov/2020/

55.

van Boekel

L. C.

Brouwers

E. P.

van Weeghel

Garretsen

H. F.

(2013). Stigma among health professionals towards patients with substance use disorders and its consequences for healthcare delivery: Systematic review. Drug and Alcohol Dependence, 131, 23–35.

56.

Van Dyck

Veitch

De Bourdeaudhuij

Thornton

Ball

(2013). Environmental perceptions as mediators of the relationship between the objective built environment and walking among socio-economically disadvantaged women. The International Journal of Behavioral Nutrition and Physical Activity, 10, 108.

57.

Vandelanotte

Caperchione

C. M.

Ellison

George

E. S.

Maeder

Kolt

G. S.

… Mummery

W. K.

(2013). What kinds of website and mobile phone-delivered physical activity and nutrition interventions do middle-aged men want? Journal of Health Communication, 18, 1070–1083.

58.

Wright

J. A.

Polack

(2006). Understanding variation in measles-mumps-rubella immunization coverage—A population-based study. European Journal of Public Health, 16, 137–142.

59.

Zanini

Benini

Pigozzi

M. G.

Furba

Giaco

Cinquegrana

… Lanzini

(2013). Addicts with chronic hepatitis C: Difficult to reach, manage or treat? World Journal of Gastroenterology, 19, 8011–8019.