Developing a measure of engagement with telehealth systems: The mHealth Technology Engagement Index

Abstract

Introduction

Telehealth systems and mobile health (mHealth) devices allow for the exchange of both physical and mental healthcare data, as well as information from a patient to a practitioner, or care recipient to caregiver; but there has been little research on why users are motivated to engage with telehealth systems. Given this, we sought to create a measure that satisfactorily assesses human motivation to use telehealth devices.

Methods

532 survey responses were used in an exploratory factor analysis and confirmatory factor analysis, which tested and retested the feasibility of this new measure. Convergent and divergent validity analyses indicated that the mHealth Technology Engagement Index (mTEI) is a unique measure of motivation.

Results

The results indicated that autonomy, competence, relatedness, goal attainment, and goal setting underpin motivation to use telehealth systems.

Discussion

The mTEI shows promise in indexing human motivation to use telehealth technologies. We also discuss the importance of developing measurement tools based on theory and how practitioners can best utilize the mTEI.

Keywords

mHealth eHealth self-determination theory telehealth systems

Introduction

Telehealth systems, or mobile health devices (mHealth), have emerged to meet the demands of a growing population with increased healthcare needs.^1,2 Telehealth devices (i.e. computer, tablet, mobile computers, etc.) are meant to facilitate the exchange of important health-related information to a healthcare practitioner or provider electronically and in real-time.³ Telehealth systems have been used successfully to supplement face-to-face patient–practitioner interactions, which has substantially reduced the cost of medical- and psychological-related visits to hospitals. This has greatly improved affordability and removed many barriers associated with receiving healthcare.^4–8

Given the success of telehealth systems, researchers have begun to direct their attention towards the measurement of the need for telehealth devices in special populations.⁹ For example, in one survey patients with chronic conditions (e.g. depression and cardiovascular disease) were asked about the types of telehealth they would be interested in using in the future.¹⁰ Patients (N = 1478) in this study, who consisted of mostly older adults, reported that they were most interested in using phone-based telehealth (60.01%) or internet- or email-based telehealth systems (57.26%). In another survey of older adults (a Swedish sample), participants were asked about the anticipated psychosocial impact of current ehealth programs and of future mHealth applications.¹¹ The researchers found that adoption was again related to age and frequency of internet usage, but the researchers did not link these findings back to any latent individual difference variables.¹¹

In a study of nursing students (N = 308), researchers reported that intention to use various telehealth devices such as telenursing tablets, telerobots, and teleconferencing was generally high (66%).¹² While this research is helpful in understanding the attitudes of those who use telehealth systems, it does not explain why people are motivated to use telehealth technologies. One theory that seeks to understand human motivation to use telehealth technology is the self-determination theory (SDT).^13,14

Self-determination theory

SDT is one of the reigning theories of human motivation with over 30 years of publications supporting it, many of these being in the public health and healthcare domains. SDT assumes that all individuals have three basic psychological needs: the need for autonomy, the need for competence, and the need for relatedness.^13,14 The need for autonomy is based on the inherent desire of all individuals to have some degree of control over their own activities, whether it is a general perception of an internal locus of causality or mastery over a task.^13–15 The need for competence suggests that individuals need to feel effective and skilled in their activities.^13,14 The need for relatedness refers to the universal need for interpersonal relationships and belonging.^13,14,16 The universal needs of autonomy, competence, and relatedness are theorized to be the source of human well-being and engagement with daily activities, regardless of gender, social status, or cultural climate (i.e. collectivist or individualistic).¹³ SDT also suggests that people may vary in the amount of each need that they must have to be satisfied or engaged with a telehealth system.

This theory could be applied to the domain of telehealth, especially since it has had much success in improving health. For example, a telehealth system should allow users to have some degree of autonomy, or choice, regarding the information they provide or how they input the data.^17,18 Furthermore, a well-designed telehealth device will allow users to feel like they can competently complete tasks while interacting with the technology. If a system is designed with poor controls or does not provide users with accurate health data, users will discontinue the use of this technology. This will greatly affect attitudes towards the technology and intention to use it in the future. In terms of relatedness, a self-determination perspective argues that connecting users to nearby care providers, or even support groups, depending on the purpose of the technology, could fulfil the need for belonging.

According to SDT, a telehealth system that satisfies these three needs creates a motivational situation, or an affordance, that facilitates technological engagement and utilization.¹⁷ That is, telehealth devices should be designed in a way that satisfies autonomy, competence, and relatedness needs, which in turn improves attitudes towards the technology. Supporting this claim, SDT states that when the satisfaction of needs for autonomy, competence, and relatedness are supported by a system, patients are much more motivated to maintain their healthy behaviours over time.¹⁹ As noted by Szalma (2014), it is not enough to assume that technologies afford motivation and it is a misstep to ignore the measurement of motivation when engineering such technologies. This is a major issue that has only recently begun to be addressed by human factors practitioners.

Integrating the research on self-determination theory and telehealth

To date, and to the best of our knowledge, only two studies have used SDT to guide the design of telehealth systems. In one study, SDT was utilized when developing A-CHESS, a smartphone application “designed to improve competence, social relatedness, and motivation” in patients with alcohol addiction.²⁰ A-CHESS had an excellent rate of return users, with up to 70% of the participants continuing the usage of the system 16 weeks after its introduction.

In another study, SDT was applied to the development of an mHealth program used to manage medications and monitor blood pressure in patients.²¹ Sieverdes et al. concluded that this system was successful because it supported patient autonomy.²¹ Answering the call of Sieverdes et al., who insisted that it is important to develop a valid and reliable self-report measure of motivation to use telehealth systems, we have developed a measure of motivation to use and engage with these technologies.

That said, the primary goal for the present research was to develop a theoretically valid measure of human motivation to reliably predict engagement. A measure that capitalizes on the degree to which technology facilitates satisfaction of the three needs (i.e. autonomy, competence, and relatedness) will improve current and future human interactions with telehealth devices.^13,14 Thus, we have developed and tested the psychometric properties of a new measure of motivation to use telehealth systems – the mHealth Technology Engagement Index (mTEI).

Impact of the present work

Our measure, the mTEI, confirms that, based on the exploratory and confirmatory factor analyses, motivation to use telehealth technologies relies on a symbiotic relationship between the device and the human user. Using the mTEI can help researchers better quantify why individuals are motivated to use or avoid telehealth technology. Our findings also suggest that motivation to engage with telehealth systems is distinct from other constructs, such as usability or acceptance.

While SDT has had success in applied areas, such as video gaming and healthcare in general, the theory has been slow to being used broadly in researches on human–computer interaction and device usability.¹⁷ This is a limitation to the present study, but we hope that by incorporating SDT into a measure of motivation to engage with telehealth systems, we can make a small necessary step in bridging these two disparate literatures.

Methods

Participants

Four samples of participants were collected for each statistical analysis. Each sample was drawn from the same south-eastern university and included students currently enrolled in a psychology course in which they had to participate in research for course credit. Students were not allowed to participate in more than one of these studies (i.e. the samples presented are entirely unique from each other). The duration of each research session was a maximum of 30 minutes. Participants tended to use the full 30 minutes, but no participants took longer than 30 minutes to complete the surveys. Samples A, B, C, and D are described in Table 1.

Table 1.

Sample characteristics.

	Sample
Characteristic	A	B	C	D
Cronbach's alpha (α)	0.898	0.885	0.929	0.884
N	222	119	80	111
Age range (in years)	18–34	18–36	18–25	18–41
Age mean (in years)	19.99	20.01	18.56	19.84
Female %	64.0	67.2	68.4	73.9
Male %	36.0	32.8	31.6	26.1
African American %	10.4	10.9	11.4	12.6
Asian %	5.5	9.2	7.6	7.2
Caucasian %	65.3	57.1	56.2	49.5
Hispanic %	12.6	13.4	16.5	22.5
Native American/ Alaskan Native %	0.9	0	0	0.9
Biracial/Multiracial %	4.1	7.6	5.0	5.4
Other ethnicity %	1.4	1.7	1.3	1.8

Note: In Sample C, four participants did not indicate their gender and one participant did not indicate their preferred ethnicity.

Preliminary item selection for the mTEI

Initial items for the mTEI were based on SDT and its application in healthcare settings.^13,14,22 The initial item generation comprised 40 items and was divided into four subfactors: autonomy, competence, relatedness, and goals. Each subfactor consisted of a pool of 10 items corresponding to that factor. For example, all items about autonomy involved the degree of choice a user could have while interacting with a telehealth system to address a health concern. Items related to competence queried users about how well the system facilitated their self-reported feelings of efficacy while navigating and using the telehealth device. Items about relatedness addressed concerns about feelings of connectedness to a provider or support group while using the technology. Items related to goals broadly asked participants whether or not the telehealth system allowed them to achieve a desired outcome.

Preliminary item reduction for the mTEI

Subject matter experts (SMEs) were recruited via email solicitation from the SDT listserv. SMEs were required to be actively publishing on SDT and have a deep understanding of the theory. Eight SMEs completed a review of our measure in two weeks. SMEs were provided with a standard definition of a telehealth system and then rated the representativeness of each survey item in the measure.³

The eight SMEs rated how reflective an item was of each subfactor on a 5-point Likert scale (i.e. very well, well, fairly well, poor, and very poorly). SMEs also assessed how adequately an item served as a measurement tool on a 5-point Likert scale (i.e. very good, good, fair, poor, and very poor). The V statistic, a content validity coefficient, was calculated for the reflectiveness of each item and for the adequacy of each item across the eight SMEs.^23,24 Items rated above V = .65 were retained (indicating that all raters endorsed a high rating of reflectiveness and adequacy, with 1.0 representing a perfect item across all raters); this resulted in the removal of seven items. Two SMEs expressed concerns regarding our treatment and categorization of goals. One SME suggested that we divide our operationalization of goals into two parts: goal attainment and goal setting. To recapitulate, at this stage of item reduction, the measure consisted of 33 items and now five subfactors: autonomy, competence, relatedness, goal attainment, and goal setting.

Exploratory factor analysis

The 33-item scale was then evaluated via exploratory factor analysis (EFA). A total of 222 undergraduate students at a large south-eastern university completed the measure via the psychology research participation system (SONA) for credit (Sample A, Table 1). The remaining items on the mTEI were randomized into meaningful blocks to control for order effects and priming.²⁵ Because our item pool was relatively small and the items correlated highly with one another, maximum-likelihood estimation was selected as the method of parameter estimation and direct oblimin was selected for rotation. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.935, well above the recommended value of 0.60, and Bartlett's test of sphericity was significant (χ² (561) =3929.784, p < .05). Seven items were removed because their communalities were below the recommended value of 0.40. Ten additional items were removed for poor factor loadings (below the recommended value of 0.30) or because these items showed substantial cross-loading on another factor.

Table 2 presents the 16-item mTEI that was used in subsequent studies. Table 3 includes the intercorrelations among the subscales of the mTEI. The initial eigenvalues showed that the first factor explained 39.37% of the variance, the second factor 5.56% of the variance, the third factor 4.20% of the variance, the fourth factor 3.97% of the variance, and the fifth factor 3.42% of the variance. A five-factor structure was preferred because of our theoretical development of the item sets, although the scree plot indicated one general factor could be extracted from the data (which we presumed to reflect overall motivation to use the telehealth system).

Table 2.

Means, standard deviations, factor loadings, and item-total correlations for the mTEI.

Scale item	Subfactor	M	SD	FL	I-T
I should be able to pick and choose my treatment.	A	3.66	0.925	0.760	0.452
I should be able to make choices that improve my care.	A	3.96	0.776	0.687	0.684
I should be able to make choices that positively impact my care.	A	4.01	0.691	0.632	0.725
I should feel competent about my healthcare.	C	3.85	0.833	0.478	0.619
I should be able to better manage my health.	C	4.06	0.820	0.598	0.633
I should feel more knowledgeable about my health.	C	4.02	0.846	0.427	0.686
I should feel competent explaining my health to my healthcare provider.	C	3.79	0.892	0.608	0.634
I should feel like I can ask the right questions to my healthcare provider.	C	3.84	0.894	0.453	0.630
I should be able to connect with my healthcare provider.	R	3.99	0.910	0.562	0.650
I should have better interactions with my healthcare provider.	R	3.82	0.997	0.635	0.622
I should be able to connect to healthcare services I need.	R	4.01	0.785	−0.858	0.591
I should feel closer to peers experiencing my health concern.	R	3.51	0.958	0.718	0.409
I should be able to set long-term goals.	GS	3.94	0.857	0.415	0.499
I should be able to set short-term goals.	GS	3.88	0.816	0.425	0.512
I should be able to show my healthcare provider my progress.	GA	4.05	0.829	0.407	0.614
I should be able to visualize my progress.	GA	4.09	0.787	0.637	0.578

Note: All scores were based on Sample A data (N = 222). Items were introduced by the following: “Instructions: After reading the following definition of telehealth technology [definition was presented], please rate the degree to which you either agree or disagree with the following statements. After interacting with this telehealth system/software…” – using a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree).

Subfactors: A = autonomy; C = competence; R = relatedness; GA = goal attainment; GS = goal setting; FL = factor loading; I-T = item-total correlations. Possible scores on the mTEI range from 16 to 80, with lower scores being indicative of less motivation to use the system again/in the near future.

Table 3.

Intercorrelations between mTEI constructs.

Subscale	Autonomy	Competence	Relatedness	Goal attainment	Goal setting
Autonomy	-
Competence	0.680**	-
Relatedness	0.642**	0.713**	-
Goal attainment	0.465**	0.537**	0.380**	-
Goal setting	0.565**	0.645**	0.549**	0.564**	-

Note: Based on the data from Sample A (N = 222).

p < .01.

Confirmatory factor analysis

We then performed a confirmatory factor analysis (CFA), based on the results of our EFA, using a new sample of 119 students from the same university (Sample B, Table 1). The CFA was computed using maximum-likelihood estimation and LISREL^© Student 9.1 software. A satisfactory fit was obtained for the five-factor model, χ² (94, N = 119) = 307.19 (goodness-of-fit index (GFI) = 0.765; adjusted goodness-of-fit index (AGFI) = 0.661; comparative fit index (CFI) =0.908; incremental fit index (IFI) = 0.909; and root-mean-square error of approximation (RMSEA) = 0.130). The GFI and AGFI fit indices were above 0.50, which indicated moderate fit.²⁶ The CFI and IFI fit indices were above 0.90, which indicated good fit.²⁶ The RMSEA was closer to 0.10, which is approaching the cut-off for a poorer fitting model.²⁶ That said, this could be due to our smaller sample size; models with low sample sizes and smaller degrees of freedom can artificially inflate the RMSEA value.²⁷ Thus, our smaller sample should suffice because the relationships among our variables are strong.²⁸ All 16 items of the scale were significantly related to the latent factor (p < .01). Internal consistency of the measure was 0.885. The full model is depicted in Figure 1.

Figure 1.

Results for the structural equation model (based on the data from Sample B). GFI = 0.765; AGFI = 0.661; CFI = 0.908; IFI = 0.909; RMSEA = 0.130; chi-square = 307.19; degrees of freedom = 119.

Correlations with other measures

Next we examined the convergent and discriminant validity of the mTEI. As suggested by Mayer (2000), “a new measure must be similar, but different enough from extant measures.”²⁹ This presents some difficulty because no current measures of motivation to use telehealth technology exist, let alone other technologies. That said, there are several measures that approximate or relate to the measurement of engagement with telehealth devices, albeit from a different theoretical or atheoretical perspective. Some of these measures draw from the technology acceptance literature, the usability literature, and the research on assistive devices. Borrowing from these research domains, the overall mTEI score was correlated with several measures across the 532 participants.

Measures used for assessing convergent and divergent validity

Psychosocial impact of assistive devices (PIADS)

The PIADS is a 26-item inventory of the effect of assistive devices on the human user.^30,31 There are three subscales associated with the PIADS: competence (12 items), adaptability (six items), and self-esteem (eight items), with higher scores on each factor being indicative of positive effects from the assistive device.

Telehealth attitude questionnaire (TAQ)

The TAQ is a 22-item measure of patient post-satisfaction with a telehealth device.³² There are four subscales nested within the TAQ: comfort and confidence (six items), telepsychiatry usage (eight items), patient concerns (four items), and availability of telehealth resources (four items). High TAQ scores indicate a more accepting attitude towards a telehealth system.

Technology acceptance model questionnaire (TAM)

The TAM framework can be used to index user satisfaction and attitude towards telehealth technology – another advantage of this model. Generally speaking, the TAM posits that perceived usefulness (PU) and perceived ease of use (PEOU) are two major constructs that predict intention to use telehealth devices. The variance accounted for by the TAM has ranged from 40% to 70% in some studies.^33,34 The TAM questionnaire is a brief measure (nine items) of the acceptance of information systems.^33–35 There are three subscales related to the TAM questionnaire: intention to use (two items), PU (four items), and PEOU (three items).

System usability scale (SUS)

The SUS is a 10-item measure assessing system usability.³⁶ The SUS is intended for broad use in a variety of contexts and assesses the usability of any technology. The higher the SUS score, the greater perceived system usability.

Results

Correlations between the mTEI and related measures were assessed across the four samples (see Table 4). Correlational analyses were used to evaluate the convergent and discriminant validity of the mTEI. The mTEI should diverge from scales related to device usage, but should be similar to subscales that converge on a similar characteristic. For example, competence and confidence should be correlated, but the measure should not be highly correlated with usability.

Table 4.

Correlations of mTEI with relevant measures of technology acceptance and usage.

Measure	Sample(s)	Correlation(s)
PIADS
Competence	A, B, D	0.568, 0.391, 0.554**
Adaptability	A, B, D	0.452, 0.330, 0.398**
Self-esteem	A, B, D	0.429, 0.343, 0.414**
TAQ	C	0.367**
Comfort and confidence	C	0.229**
Telepsychiatry usage	C	–0.010
Patient concerns	C	0.447**
Availability of telehealth resources	C	0.368**
TAM Questionnaire	D	–0.744**
Intention to use	D	–0.566**
Perceived usefulness	D	–0.745**
Perceived ease of use	D	–0.399**
SUS	B, C, D	0.107, 0.365, 0.414

Note: N for Samples A, B, C, and D are 222, 119, 80, and 111, respectively.

PIADS: Psychosocial impact of assistive devices; TAQ: Telehealth attitude questionnaire; TAM: Technology assessment model items; SUS: System usability scale.

p < .01.

The mTEI had significant positive correlations across each subscale of the PIADS over three samples (i.e. Samples A, B, and D). This indicates that the mTEI is assessing a different latent construct or motivation, and not adaptability or self-esteem. Additionally, we found that competence on the mTEI was highly correlated with the PIADS subscale of competence, which was expected as these constructs should overlap. In Sample A, the competence subscale was positively correlated with the mTEI (r = 0.568). In Sample B, the correlation was positive and weak (r = 0.391), and in Sample D the relationship was positive and moderate (r = 0.554).

In terms of correlation to the TAM questionnaire, the mTEI was negatively correlated to the global acceptance index (r = –0.744), intention to use (r = −0.566), PU (r = –0.745), and PEOU (r = –0.399). These results show that, generally, overall acceptance and the factors that underpin acceptance are again distinct from the mTEI. Acceptance and behavioural intention to use a telehealth system are not the same constructs as motivation to use a device, though there could be some overlap between perceived usability and motivation to use telehealth technologies.

The mTEI was also positively correlated with the TAQ (r = 0.367), but only in one sample, which shows that, generally, attitude towards telehealth technology is distinct from motivation to use telehealth systems.

Finally, correlations between the SUS and mTEI indicated that our measure is not a measure of usability. The SUS was only significantly correlated with the mTEI in two samples: C (r = 0.364) and D (r = 0.414). The SUS was included to show that mTEI distinctly measures motivation to engage with telehealth systems and not perceived usability per se.

Discussion

While much is known about who uses telehealth devices and how telehealth devices have impacted well-being over the last 20 years,⁹ it is equally important to understand why some users are engaged with telehealth systems and why others abhor technology. A measure of motivation can better pinpoint the factors associated with engagement with telehealth devices. We found that autonomy, competence, relatedness, goal attainment, and goal setting we are all important in understanding why individuals are motivated to use a particular system. We believe that the mTEI can be used to assess technology from a humanistic perspective, meaning that telehealth technologies should afford the user with a sense of choice, belonging, and confidence.

Furthermore, we propose that the mTEI has several strengths over the other four measures used in our study. For example, the mTEI measures goals, autonomy, and relatedness, which are not currently assessed by any other measure of telehealth acceptance or intention to use. As the results have indicated, motivation is distinct from usability, attitude towards technology, and acceptance; thus, the mTEI extends theory by demonstrating how integral motivation is to acceptance and intention to use. We believe that the integration of motivation into the technology acceptance framework will work to make this model stronger.

Because the mTEI was developed using a robust theory of human motivation, we believe that it can best be used to assess patient or practitioner motivation as a baseline, which in turn predicts the likelihood that users will regularly interact and engage with a telehealth system. If motivation is low at baseline, the device will need to be changed to better support the satisfaction of motivational needs. Telehealth technologies that foster motivation will result in more usage over the long term, thereby impacting patient health and satisfaction.^19–22

To conclude, the mTEI serves to bridge two disparate literatures: the research on motivation and the research on telehealth systems. By unifying theory with practice, we can begin to assess and improve patient and practitioner engagement with telehealth technologies.

Footnotes

Acknowledgements

We would like to especially thank our SMEs for offering their help and outstanding suggestions for our measure. We would also like to thank three anonymous reviewers for their helpful feedback on this manuscript, as well as their diligent attention to detail; it served to make this paper much stronger.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Colby SL and Ortman JM. The baby boom cohort in the United States: 2012 to 2060. Report, US Census Bureau, Washington, DC, 2014.

National Center for Telehealth and Technology. 2013 annual report. Report, Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury, Joint Base Lewis–McChord, Tacoma, WA, 2013.

Malvey

Slovensky

. mHealth: Transforming Healthcare, New York: Springer, 2014.

Benavides-Vaello

Strode

. Using technology in the delivery of mental health and substance abuse treatment in rural communities: a review. J Behav Health Ser R 2013; 40: 111–120.

Healthcare Intelligence Network. 2013 Healthcare benchmarks: telehealth and telemedicine, 3rd ed. Sea Girt, NJ: Healthcare Intelligence Network, 2013.

Hilty

Luo

. Telepsychiatry: an overview for psychiatrists. CNS Drugs 2002; 16: 527–548.

Chong

Moreno

. Feasibility and acceptability of clinic-based telepsychiatry for low-income hispanic primary care patients. Telemed J E Health 2012; 18: 297–304.

Melenhorst

Rogers

. Older adults' motivated choice for technological innovation: evidence from benefit-driven selectivity. Psychol Aging 2006; 21: 190–195.

Wootton

. Twenty years of telemedicine in chronic disease management – an evidence synthesis. J Telemed Telecare 2012; 18: 211–220.

10.

Edwards

Thomas

. Are people with chronic diseases interested in using telehealth? A cross-sectional postal survey. J Med Internet Res 2014; 16: e123.

11.

Wiklund Axelsson

Nyberg

. The anticipated positive psychosocial impact of web-based e-health services and future mobile applications: an investigation among older Swedes. Int J Telemed Appl 2013; 2013: 1–9. .

12.

Glinkowski

Powlowska

. Telehealth and telenursing perception and knowledge among university students in nursing in Poland. Int J Telemed Appl 2013; 19: 523–529.

13.

Ryan

Deci

Self-determination theory and the role of basic psychology needs in personality and the organization of behavior. In: John

Robins

Pervin

(eds). Handbook of personality theory and research, 3rd ed. New York, NY: Guilford Press, 2008, pp. 654–678.

14.

Ryan

Deci

. On happiness and human potentials: a review of research on hedonic and eudaimonic well-being. Annu Rev Psychol 2001; 52: 141–166.

15.

Rotter

. Generalized expectancies for internal versus external control of reinforcement. Psychol Monogr 1966; 80: 1–28.

16.

Baumeister

Leary

. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol Bull 1995; 117: 497–529.

17.

Szalma

. On the application of motivation theory to human factors/ergonomics: motivational design principles for human-technology interaction. Hum Factors 2014; 56: 1453–1471.

18.

Deci

Eghrari

. Facilitating internalization: the self-determination theory perspective. J PERS 1994; 62: 123–142.

19.

Ryan

Patrick

. Facilitating healthy behaviour change and its maintenance: interventions based on Self-Determination Theory. EHPS 2008; 10: 2–5.

20.

McTavish

Chih

. How patients recovering from alcoholism use a smartphone intervention. J Dual Diagn 2012; 8: 294–304.

21.

Sieverdes

Gregoski

. mHealth medication and blood pressure self-management program in Hispanic hypertensives: a proof of concept trial. Smart Homecare Technol Telehealth 2013; 1: 1–10. DOI: 10.2147/SHTT.S49633.

22.

Williams

Improving patients' health through supporting the autonomy of patients and providers. In: Deci

Ryan

(eds). Handbook of self-determination research, Rochester, NY: The University of Rochester Press, 2002, pp. 233–254.

23.

Aiken

. Three coefficients for analyzing the reliability and validity of ratings. Educ Psychol Meas 1985; 45: 131–142.

24.

Aiken

. Content validity and reliability of single items or questionnaires. Educ Psychol Meas 1980; 40: 955–959.

25.

DeVellis

. Scale development: theory and applications, 3rd ed. Thousand Oaks, CA: SAGE Publications, 2012.

26.

Loehlin

. Latent variable models: an introduction to factor, path, and structural analysis, 3rd ed. Mahwah, NJ: Lawrence Erlbaum Associates, 1998.

27.

Marcoulides

Schumacker

. Advanced SEM: issues and techniques, Mahwah, NJ: Lawrence Erlbaum Associates, 1996.

28.

Millsap

. Structural equation modeling made difficult. Pers Indiv Differ 2007; 42: 875–881.

29.

Mayer

. Spiritual intelligence or spiritual consciousness. Int J Psychol Relig 2000; 10: 47–56.

30.

Day

Jutai

. Measuring the psychosocial impact of assistive devices: the PIADS. Can J Rehabil 1996; 9: 159–168.

31.

Jutai

Day

. Psychological Impact of Assistive Devices Scale (PIADS). Technol Disabil 2002; 14: 107–111.

32.

Grubaugh

Cain

. Attitudes toward medical and mental health care delivered via telehealth applications among rural and urban primary care patients. J Nerv Ment 2008; 196: 166–170.

33.

Venkatesh

Davis

. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci 2000; 46: 186–204.

34.

Venkatesh

. Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. ISR 2000; 11: 342–365.

35.

Schepers

Wetzels

. A meta-analysis of the technology acceptance model: Investigating subjective norm and moderation effects. Inform Manage 2007; 44: 90–103.

36.

Brooke J. SUS – a quick and dirty usability scale: smart phone applications technology across the NSW Brain Injury Rehabilitation Program (BIRP). Report, Digital Equipment Corporation, Reading, UK, 1996.