Measuring Participation in Employer-Sponsored Health and Well-Being Programs: A Participation Index and Its Association With Health Risk Change

Abstract

Purpose:

To develop an index of participation in workplace health and well-being programs and assess its relationship with health risk status.

Design:

Study design comprised a retrospective longitudinal analysis of employee health risk assessment (HRA) and program participation data.

Setting:

Data from 6 companies that implemented health and well-being programs from 2014 to 2016.

Participants:

Employee participants (n = 95 318) from 6 companies who completed an HRA in 2014 to 2016. After matching those who completed the HRA in all 3 years, the longitudinal file included 38 789 respondents.

Measures:

Participation indicators were created for 9 different program components. The sum of these 9 components established the total participation index.

Analysis:

Descriptive and correlation analyses were conducted on all participation measures. Repeated-measures analysis of variance was used to assess the impact of participation level on health risk over time.

Results:

Higher levels of participation were associated with a greater reduction in risks, with each participation dose yielding a reduction of 0.038 risks (P < .001).

Conclusion:

Results suggest that employees who participate more in workplace health and well-being programs experience more health risk improvement. The study also supports a more granular definition of participation based on the number of interactions and type of program.

Keywords

employee wellness participation workplace specific settings research methods wellness population health interventions behavior change workplace setting

Purpose

High levels of active participation in workplace health and well-being programs are a key priority for employers^1,2 and a leading indicator of program success. Although participation goals may vary across program components, most employers aim to maximize participation in program activities, or combinations of activities, to maximize program impact.

Despite its importance to program evaluation efforts,³ defining and measuring meaningful participation has long been a challenge for employers and other stakeholders. Participation is most often defined as a binary measure; one either does or does not participate. In this approach, those with minimal participation are equated to those with extensive participation and health risk assessment (HRA) participation has often been used as a proxy measure. These limitations present an important opportunity to refine how participation is defined and measured, particularly the need to increase both the granularity and the scope of any such measurement.

Organizations such as the National Committee for Quality Assurance, Health Enhancement Research Organization (HERO), and Population Health Alliance (PHA)⁴ have provided guidance to organizations on current best-practice approaches to defining and measuring participation in health and well-being programs. The HERO/PHA guide for Measurement and Evaluation, developed by leading researchers and experts in workplace health and well-being evaluation, specifically recommends that “more research be done to focus on the amount of intervention necessary to produce a positive health outcome.”^4(p41) In addition to advocating for research on the effectiveness of different types of interventions, the HERO guide further recommends “determining if there is a dose-response relationship with regard to the number of contacts” (emphasis added).^4(p41)

Even with the binary measure of participation, substantial research over 3 decades documents the connection between participation and health risk changes, medical service utilization, health-care costs, and workplace productivity.^3

-9 The likely limitations of a binary measure of participation in capturing the effects of a comprehensive workplace health and well-being program, however, suggest it is time for a more nuanced measure of participation that may more precisely predict healthy behavior changes and risk reduction. Such an approach may be valuable to the many employers who provide robust, multifaceted health and well-being programs that include many components, addressing multiple wellness goals.¹⁰ For these organizations, participation is necessarily complex. Reducing that complexity to a binary measure of participation loses substantial information relevant to understanding and predicting health behavior change and risk reduction.

This study aims to address the call by HERO and PHA for more research on participation and to respond to employer needs for more compelling evaluation measures by looking at participation across different programs, across time, and within programs by the degree of participation. The goals of this research were to (1) develop a participation index that captured the quantity of participation in the diverse components of workplace health and well-being programs, that is, to measure the doses of program participation; and (2) assess the relationship between the index and health risk change, that is, the response. More specifically, we aimed to leverage the guidance provided in the HERO/PHA Program Measurement and Evaluation Guide⁴ to create a participation index and to explore the relationship between this measure and health risk status. Higher levels of participation, based on the index, were hypothesized to be positively associated with health risk reduction over time.

Methods

Design

This study was a retrospective analysis of data collected while implementing workplace health and well-being programs for 6 different employers. All information was collected via routine implementation of these programs from 2014 through 2016. Existing client data were assembled into a longitudinal file of employees and their spouses (if eligible for employer offerings). A prerequisite for inclusion in the study was completion of the HRA during each of the 3 years captured by this study. This was necessary to ensure that the primary dependent variable for the predictive validity analysis—the total number of health risks—was measured longitudinally.

Sample

Table 1 presents basic data describing the sample before and after the analysis file was built. Overall, 95 318 persons affiliated with these employers completed at least 1 HRA during the 2014 to 2016 time frame. Employers ranged in size from 1767 to 28 583 HRA participants.

Table 1.

Distribution of All Eligible Respondents by Company.

Company Code	Original Cross-Sectional Files		Matched 2014 to 2016 Longitudinal File
Company Code	Count	%	Count	%
Company 1	1767	1.9	623	1.6
Company 2	7818	8.2	3753	9.7
Company 3	18 630	19.5	8760	22.6
Company 4	28 583	30.0	9692	25.0
Company 5	22 620	23.7	12 699	32.7
Company 6	15 900	16.7	3262	8.4
Total	95 318	100.0	38 789	100.0

These employees and their spouses are not a random sample of the US population. All of the employers are relatively large, which itself makes the sample atypical of the US population. By requiring 3 continuous years of HRA completion, a further degree of self-selection is incorporated into the study design. Study participants were also limited to those aged 18 to 65 years at the time of the 2014 HRA. The combined effects of these inclusion criteria reduced the longitudinal file to 38 789 employees and spouses. Attrition rates varied across companies when these conditions were imposed (Table 1). Company 6 experienced nearly 80% attrition, while companies 1 and 4 experienced attrition rates of nearly two-thirds. In contrast, company 5 experienced attrition of only 43.9% over the 3-year period. Still, the net impact of these inclusion criteria on measured characteristics of the sample was minimal (see Table 2).

Table 2.

Sample Changes Introduced by Constructing Longitudinal File.

Indicators	Original Cross-Sectional Data		Matched Longitudinal File Age 18-65 in 2014
Indicators	Sample Size	Metric	Sample Size	Metric
Average age in 2014	66 254	47.79	38 789	47.60
Percent female in 2014	66 254	46.5	38 789	47.0
Percent employees in 2014	66 254	76.6	38 789	77.0
Average number of risks in 2014	64 740	2.52	38 789	2.46
Average number of risks in 2015	65 629	2.38	38 789	2.30
Average number of risks in 2016	67 679	2.32	38 789	2.26
Participation index 2014	66 254	1.85	38 789	1.96
Participation index 2015	67 778	2.14	38 789	2.24

Measures

During each program period, individuals were eligible to participate in 1 or more health and well-being offerings. Not every employer offered every program. Also, each employer used a variety of strategies, such as financial incentives, targeted outreach and communications, and onsite promotions to encourage program participation. Participation indicators were defined for each offering in each program period. Individuals who elected to participate in a program component received a score of 1 or more, with the maximum score dependent on the activity, while individuals who elected to not participate and individuals at employers who did not have the relevant offering were scored as 0.

Health risk assessment

Although included in creating the index, this measure had no practical impact on the analysis as it was required for inclusion in the study. This measure is functionally a constant. This tool includes a series of questions on chronic conditions, health status, lifestyle health behaviors, and demographics. The predictive validity of this tool has been established by linking its health risk measurements to health-care utilization, medical costs, and heart disease mortality risk derived from the Framingham study population.^11

-14

Lifestyle management

All employees and spouses had access to telephone-based lifestyle management (LM) coaching. Lifestyle management is a common tool used to support health behavior change and has demonstrated effectiveness.¹⁵ Two indicators of LM were developed for each program period. One ranged from 0 to 2 and counted the number of unique programs in which the individual participated that year. The second indicator counted the number of completed calls associated with their program experience. For example, if a person enrolled in a program and completed 3 calls, then enrolled in a different program and completed 3 more calls, that person would have participated in 2 programs and completed 6 calls. For analysis purposes, the range for completed calls was restricted to 0 to 7 for each year to minimize the skew in the data.

As expected, these measures were strongly correlated. Due to eligibility criteria based on risk and self-selection among those eligible, only a small proportion of the total population typically participates in LM. For individuals who did not participate in LM, there is also a structural 0 for coaching calls because persons who did not enroll in any programs necessarily have 0 completed calls.

Digital health education

This is a series of digital health education (DHE) programs, each designed to address specific health risks such as unhealthy weight, stress, and physical inactivity, among others. One indicator was generated for each year based on the number of programs completed that year. Annually, this measure ranged from 0 to 3. Three employers offered this in 2014 and 4 in 2015.

Challenges

These focus primarily on improving healthy eating behaviors and physical activity through walking or steps challenges and, depending on program design, may constitute a year-long population-level initiative that promotes consistent daily activity. An indicator of participation was developed to count the number of challenges completed in each program year. Three employers offered challenges in 2014, while only 2 offered them in 2015.

Biometric screenings

A key component of the health assessment process, biometric screenings are often offered as part of worksite health programs, not only to identify health risk for individual participants and across a population but also to serve as an awareness and engagement-building opportunity.¹⁶ Providing feedback after assessing health risks has been documented as a common and useful tool in workplace health.¹⁷ Annual onsite biometric screenings were made available to employees of 5 of the 6 companies included in the analysis. Participation was captured for each year.

Health webinars

Health webinars are virtual group lectures, provided online, in real time, where participants hear expert commentary and discuss issues involved in maintaining a healthy lifestyle. Recordings of these were not available. One employer offered health webinars in 2014, with a second adding them in 2015. Participation was measured as a count of the number of health webinars completed in each program year.

Digital workshops

Digital workshops present individuals with interactive educational content reviewing a range of health risks, and the strategies and behaviors people can use to reduce these risks. These were unavailable in 2014. One employer offered this program in 2015. Two indicators were developed for use in the participation index. The first counted the number of completed workshops. The second counted the number of unique months in which workshops were attended.

Health risk

Total number of health risks was used to measure program impact. Health risk scores were calculated for all HRA completers in the following 9 areas: alcohol use, back care, depression, driving, nutrition, physical activity, tobacco use, stress, and weight. Except for weight data, all measures were self-reported. Total number of health risks, defined as count of these areas for which a person was not low risk, was calculated at both baseline and follow-up.

Participation index

A participation index was defined as the sum of the 9 program-specific measures discussed above. Since each employer had different offerings as is typical of workplace programs, the total possible index score for participants varied across employers. The results were deliberately not standardized between employers to allow testing of a wider range of doses on health outcomes. Four alternative versions of the participation index were also tested, each applying differential weights to the various program components, with the weightings based on the authors’ collective experiences regarding the likely impact of that program component on health and well-being. The correlations among the different methods of scoring the index all exceeded 0.7 within any 1-year program. Because of these very high correlations, the simpler, more parsimonious index described above was used for our final analysis of health risk impact. It is important to acknowledge that the theoretical underpinnings of the measure are not consistent with the logic associated with classical psychometric theory. In the usual psychometric model,^18
-20 the indicators that make up a scale or an index are presumed to be influenced by an underlying trait that causes the respondent to answer the items or behave in a particular way. Therefore, to the extent that a trait exists and the items validly measure that trait, then consistently high correlations are expected across those items. A different situation exists here. Individuals may have a range of different health needs, and they work for employers with somewhat different health offerings. The individual can thus rationally pick from available offerings that suit them. Individuals would likely not choose to participate in all of these offerings. Rather, they would select those options they feel are the best for their particular health needs. While there may be a modest correlation among the items in the index related to general health activation, one should not necessarily expect to see a pattern of strong correlations among all of the items.

Analysis

Four sets of analyses were conducted. First, basic analyses documented the effects of the constraints imposed by the requirements of developing the longitudinal sample on the distributions for the key demographic, risk and participation measures of the study population (Table 2). Second, descriptive statistics were determined for the 9 participation index component measures (Table 3). Third, the correlations among the various demographic, risk, and participation measures were analyzed (Table 4). Finally, the impact of participation level on risk change over time was assessed (Table 5 and Figure 1). All analyses were conducted using SAS software version 9.4 (SAS Institute, Cary, North Carolina).

Table 3.

Component Measures in Participation Index.

Measure	Number of Companies Offering in 2014	Number of Companies Offering in 2015	Maximum Score Possible	Average Score 2014	Average Score 2015	Item-Total Correlation 2014	Item-Total Correlation 2015
Health risk assessments	6	6	1	1.000	1.000	NA^a	NA^a
Health webinars	1	2	2	0.001	0.001	0.034	0.044
Number of digital workshops	0	1	3	0.000	0.001	NA^a	0.021
Months of digital workshop participation	0	1	3	0.000	0.000	NA^a	0.020
Biometric screening	5	5	1	0.522	0.535	0.475	0.410
Number of digital health education programs	3	4	3	0.030	0.025	0.147	0.077
Number of coaching programs	6	6	2	0.060	0.083	0.884	0.888
Number of coaching contacts	6	6	7	0.279	0.406	0.923	0.922
Number of challenges completed	3	2	4	0.067	0.191	0.288	0.391

^aThe absence of variation precludes calculation of an item-total correlation.

Table 4.

Correlations Among Measures.

	Age	Female	Employee	Average Number of Risks in 2014	Average Number of Risks in 2015	Average Number of Risks in 2016	Participation Index 2014	Participation Index 2015
Age	1.000
Female	−0.029	1.000
Employee	−0.065	−0.276	1.000
Average number of risks in 2014	0.066	−0.041	0.079	1.000
Average number of risks in 2015	0.046	−0.042	0.086	0.682	1.000
Average number of risks in 2016	0.033	−0.032	0.088	0.637	0.685	1.000
Participation Index 2014	−0.039	0.010	0.193	0.132	0.120	0.102	1.000
Participation Index 2015	−0.050	0.024	0.116	0.138	0.146	0.111	0.484	1.000

Table 5.

Within-Participant Effects for Repeated-Measures Analysis.

Effect	F Test	Probability
Time	8.98	.0001
Time × gender	2.96	.0518
Time × employee	1.1	.3316
Time × age	31.31	<.0001
Time × company	134.47	<.0001
Time × participation index 2014	10.7	<.0001
Time × participation index 2015	18.02	<.0001

Figure 1.

Predicted risk scores by participation index.

Results

Characteristics of the Sample

To assess the impact of limiting the sample to those who completed the HRA in all 3 years, we compared selected characteristics of the final study sample to individuals who completed an HRA only 1 or 2 years (Table 2). The study sample was slightly younger, slightly more likely to be female, and more likely to be employees (as opposed to spouses) than those who completed the HRA in any 1 year. Persons included in the study were also slightly healthier, averaging slightly fewer health risks than excluded individuals. Taken as a whole, these analyses revealed only minor differences between the study group and all individuals who completed an HRA, suggesting that selection bias associated with developing the longitudinal file requiring 3 years of HRA participation was minimal.

Characteristics of the Participation Indicators

As indicated in Table 2, the overall average participation index score was 1.96 in 2014 and 2.24 in 2015. The maximum observed score was 12 in 2014 and 15 in 2015. Table 3 presents the 9 component measures included in the participation index for the final study sample. By virtue of this unit weighting, measures with a wider range can influence the total scores to a greater degree.

An HRA was offered by all companies, making it a constant in the analysis file for all employers and participants. Beyond the HRA, biometric screening was most common, with averages exceeding 0.5 for both 2014 and 2015 and reasonably strong item-total correlations. Lifestyle management coaching indicators, with a greater range and universal offering by the companies in this analysis, showed modestly high average scores and very strong item-total correlations. The indicators for challenges and DHE had lower prevalence in the study population and correspondingly lower item-total correlations. Finally, health webinars and digital workshop indicators were not very prevalent and, consequently, were only weakly correlated with the total summary index.

Given the scoring model used in creating the index, employees whose employer offered more components tended to have higher scores. The company that offered the most components (company 4 in Table 1) had significantly higher scores on the participation index (P < .001).

Table 4 presents the correlations among the different measures included in the study for the final study sample. As anticipated, the over time correlations for the number of health risks were substantial. For each adjacent year, the correlation exceeded 0.68, while the 2-year lagged correlation was 0.637. The correlation between the 2 summary indicators of participation was modestly high (r = 0.484). Persons who participated in 1 year tended to participate the following year.

Reflecting the targeting of some wellness initiatives, the participation measures were modestly correlated with the 3 risk measures. Employees were more likely to participate than spouses. Females were slightly more likely to participate than males, and older persons were slightly less likely to participate.

Age, gender, and employee status were modestly correlated. Older persons were somewhat less likely to be employees and less likely to be females. Employees were also less likely to be females.

To test the predictive validity of the index and assess whether the participation index was associated with risk change, a repeated-measures analysis of variance was conducted. The dependent measures for this analysis were the 3 indicators of risks measured in 2014, 2015, and 2016, respectively. The key independent variables were the participation indices for 2014 and 2015, with gender, employee/spouse status, employer, and age included as control variables. The results of this analysis are presented in Tables 5 and 6.

Table 6.

Between-Participant Effects for Repeated-Measures Analysis.

Effect	F Test	Probability
Female	20.12	<.0001
Employee	105.1	<.0001
Age	292.58	<.0001
Company	339.4	<.0001
Participation index 2014	93.13	<.0001
Participation index 2015	316.22	<.0001

The between-participant effects are all highly significant. Males tended to have more risks than females, employees had more risks than spouses, and older respondents had more risks than younger respondents. Persons at companies 2 and 4 had the fewest risks, while those at company 6 had the most. Persons with higher scores on the participation index had more risks, which was the intent of the targeting strategy designed to engage people with a greater risk burden in more intensive programming.

Over time, the average number of risks were significantly reduced. Older employees and their spouses benefited more from the programs. Participants at companies 1, 4, and 5 benefited more than the participants at companies 2, 3, and 6 after controlling for the effects of participation. Evidence of the predictive validity of the index was found in the within-participant effects presented in Table 5. The statistically significant time × participation index interaction effects indicate that persons with higher scores on the participation index better attenuated their risks over time. This pattern is graphically presented in Figure 1.

The higher baseline risk scores (2014) for those with higher engagement indicate that the program targeting mechanisms worked as intended. Persons with more risks had higher rates of participation. The efficacy of the programming and the predictive validity of the participation index are supported by the reduction in the predicted risk score for each successive year. Persons with higher levels of participation decreased their risk scores by a greater degree than those with lower levels of participation. Additionally, nonemployees had higher levels of risk at baseline but reduced their risks more than employees. Compared to younger individuals, older individuals achieved greater levels of risk reduction.

Discussion

This study clearly supports the notion of measuring participation in workplace program components based on the number of interactions/contacts—the dose model of participation suggested by HERO and PHA.⁴ Results generally suggest that more participation based on this recommended approach is associated with more health risk improvement. Additionally, the effects observed with participation over time suggest that people have predispositions for specific types of participation that persist and guide how they get involved with their health and well-being.

In this analysis, each type of contact was scored as fundamentally equal to every other type of contact; different contact types were not differentially weighted. Different scoring models were also developed and tested, and while the exact results differed slightly, the overall pattern of impact was essentially identical. Higher scores on the participation index were associated with both higher risk and with slowing the rate of increase in risks over time, that is, the dampening effect. Conceptually, we believe that weighting the components of the index makes sense, but our attempts at weighting added no measurement value. Further research is needed to develop empirical estimates of the proper weights to employ in the index.

Results were generally consistent with the hypothesis: Higher levels of participation were correlated with higher levels of health risk improvement. Previous studies have shown programs like these to be associated with behavior change and/or risk improvement,^1,3,5,7,8,15 so it follows logically that combining participation measures into a summary index would have a similar relationship. At least 2 explanations for this relationship are plausible. One is that the program components in which people participated were effective at helping participants change their behavior; the second is that people who participated were in the process of changing their behavior prior to participation, and they chose to participate to help them be more successful in doing so. It is likely that both these factors were at play.

It is important to keep in mind some of the ways participation was encouraged by employers in this study. Although organizational strategies vary, most of them include 1 or more of the following elements: financial incentives, reminder e-mails/mailings, targeted e-mails/mailings (eg, for individuals at high risk), telephone call reminders, verbal reminders from colleagues or supervisors, and more. Participation in any program is voluntary, but these and other strategies introduce a potential bias to this analysis, namely, that some participants may not be motivated to change their behaviors, which may have tempered the risk change found here.

Furthermore, every organization approaches participation differently, inherently limiting the external validity of these results. It is further limited because the offerings used in the participation index were all developed by one provider of workplace health and well-being programs. Changes to these elements may influence the relationships described in this study, which makes replication across multiple providers essential to assess generalizability.

The study has several other limitations. One is that the participation index was not identical to the conceptual model developed by HERO and PHA.⁴ In the HERO and PHA formulation, the suggested index is a sum of the contacts or touch points between the individual and the well-being program. The index we developed in this research largely parallels that model, but we nonetheless acknowledge minor differences in the formulation of the index.

Study findings clearly suggest that an increased degree of participation results in greater impact. We encourage further research using an even more granular approach to measuring and summarizing participation. Models that disaggregate participation into multiple components represent an extension of the thesis presented here that participation is more than a simple binary measure; treating it as multiple binary measures, or as a combination of some binary measures and some count-based indices, is consistent with that goal. Adding other measures relevant to participation may also add value to the research. Measures such as readiness to change, or the perceived value associated with wellness initiatives, could capture a social-psychological component of the change process related to both participation and outcomes. Such measures would also be more likely to be consistent with the psychometric model of measurement not captured by the participation index.

So What?

These findings have important implications for program practitioners as well as future measurement of participation and understanding the potential value of workplace health and well-being programs. Practitioners and researchers alike have long struggled to understand the relationship between participation in various workplace health and well-being programs and subsequent risk/behavior change. Results of this study can help move workplace health researchers, providers, and practitioners toward a better measurement approach and a clearer understanding of how participation fits in the value proposition for workplace health and well-being.

Footnotes

Authors’ Note

All authors meeting the core 4 ICMJE requirements for authorship.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Erin L. D. Seaverson, MPH, and Stefan B. Gingerich, MS, declare that there is a conflict of interest based on direct employment by StayWell. David J. Mangen, PhD, and David R. Anderson, PhD, declare that there is a conflict of interest based on contractual consultancy by StayWell.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Stefan B. Gingerich, MS

David J. Mangen, PhD

References

Goetzel

Shechter

Ozminkowski

Marmet

Tabrizi

Roemer

. Promising practices in employer health and productivity management efforts: findings from a benchmarking study. J Occup Environ Med. 2007;49(2):111–130.

Serxner

Anderson

Gold

. Building program participation: strategies for recruitment and retention in worksite health promotion programs. Am J Health Promot. 2004;18(4):1–6, iii.

Grossmeier

Terry

Cipriotti

Burtaine

. Best practices in evaluating worksite health promotion programs. Am J Health Promot. 2010;24(3):TAHP1–TAHP9, iii.

Program Measurement and Evaluation Guide. Core metrics for employee health management. 2015; 39–41. http://hero-health.org/wp-content/uploads/2015/02/HERO-PHA-Metrics-Guide-FINAL.pdf. Accessed June 22, 2018.

Serxner

Gold

Meraz

Gray

. Do Employee health management programs work? Am J Health Promot. 2009;23(4):1–12.

Mitchell

Ozminkowski

Hartley

. The association between health program participation and employee retention. J Occup Environ Med. 2016;58(9):896–901.

Burton

Chen

Schultz

Edington

. Reduction in health risks and disparities with participation in an employer-sponsored health promotion program. J Occup Environ Med. 2013;55(8):873–878.

Grossmeier

Seaverson

Mangen

et al. Impact of a comprehensive population health management program on health care costs. J Occup Environ Med. 2013;55(6):634–643.

Serxner

Gold

Grossmeier

Anderson

. The relationship between health promotion program participation and medical costs: a dose response. J Occup Environ Med. 2003;45(11):1196–1200.

10.

Embracing a Broader Definition of Well-Being . Washington, DC: National Business Group on Health and Fidelity Investments; 2017.

11.

Anderson

Brink

Courtney

. Health Risks and their Impact on Medical Costs, Research Report for the Chrysler Corporation and the International UAW Union. Milwaukee, WI: Milliman & Robertson, Inc.; 1995.

12.

Brink

. Health Risks and Behavior: The Impact on Medical Costs. Milwaukee, WI: Milliman & Robertson, Inc.; 1987.

13.

Goetzel

Anderson

Whitmer

Ozminkowski

Dunn

Wasserman

. The relationship between modifiable health risks and health care expenditures: an analysis of the multi-employer HERO health risk and cost database. J Occup Environ Med. 1998;40(10):843–854.

14.

Smith

McKinlay

Thorington

. The validity of health risk appraisal instruments for assessing coronary heart disease risk. Am J Public Health. 1987;77(4):419–424.

15.

Terry

Seaverson

Staufacker

Gingerich

. A comparison of the effectiveness of a telephone coaching program and a mail-based program. Health Educ Beh. 2010;37(6):895–912.

16.

Behling

Kelly

Ruppert

et al. Biometric Health Screening for Employers Consensus Statement of the Health Enhancement Research Organization, American College of Occupational and Environmental Medicine, and Care Continuum Alliance. J Occup Environ Med. 2013;55(10):1244–1251.

17.

Soler

Leeks

Razi

et al. A systematic review of selected interventions for worksite health promotion: the assessment of health risks with feedback. Am J Prev Med. 2010;38(2):S237–S262.

18.

Cronbach

Gleser

Nanda

Rajaratnam

. The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. New York, NY: John Wiley & Sons; 1972.

19.

Nunnally

. Psychometric Theory. New York, NY: McGraw Hill; 1967.

20.

Nunnally

Bernstein

. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill; 1994.