User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease

Abstract

Objective

Mobile health (mHealth) technologies are emerging to support the delivery of pulmonary rehabilitation (PR). This study aimed to explore the ease of use, satisfaction and acceptability of an Australian mobile pulmonary rehabilitation app (m-PR™) in people with chronic obstructive pulmonary disease (COPD).

Methods

In this mixed methods observational study, participants with COPD were recruited following PR assessment. Participants were educated on m-PR™ which contained symptom monitoring, individualised exercise training with exercise videos, education videos, goal setting, health notifications and medication action plan. Participants used m-PR™ for 4–8 weeks. At baseline, participants were surveyed to assess level of technology engagement. At follow-up, participants completed the system usability survey (SUS), a satisfaction survey and a semi-structured interview.

Results

Fifteen participants (mean age 70 [SD 10] years, 53% female) completed the study. Technology usage was high with 73% (n = 11) self-rating their technology competence as good or very good. The SUS score of 71 (SD 16) demonstrated above average perceived usability of m-PR™. The satisfaction survey indicated that 67% (n = 10) enjoyed m-PR™ and 33% (n = 5) were neutral. Most participants found the different m-PR™ components somewhat easy or very easy to use (range 69–100%) and somewhat helpful or very helpful (range 76–100%). Interview responses revealed that m-PR™ elicited divergent feelings among participants, who reported both positive and negative feelings towards the app's features, the effort required to use it and data security.

Conclusion

The majority of participants found m-PR™ enjoyable, easy to use and helpful in managing their COPD. Further research is warranted to understand the effectiveness of mHealth to deliver PR.

Keywords

Chronic < Disease chronic obstructive pulmonary disease mHealth < Psychology digital health < General exercise < Lifestyle pulmonary rehabilitation smartphone < Media app

Introduction

Pulmonary rehabilitation (PR) is an effective non-pharmacological intervention for people with chronic obstructive pulmonary disease (COPD).¹ Barriers to access, uptake, attendance and completion of traditional outpatient PR programs are well documented.² Home-based programs are a safe and feasible alternative to outpatient PR programs to improve availability and access³ and are comparable to outpatient PR in reducing dyspnoea and improving exercise capacity and health-related quality of life (HRQoL).^4,5

With increasing use of mobile health (mHealth) technology, providing a comprehensive digitalised PR program is now possible and has the potential to expand access to PR. People with COPD report a substantial engagement with technology and willingness to use technology-assisted PR⁶ and short-term PR programs using mHealth technology have now been developed and tested in multiple countries.^7–13 Of the studies to date, three have included some element of user experience and satisfaction testing of the mHealth PR programs with general acceptability being reported.^8,9,12 The mobile PR app (m-PR™) is the first Australian specific mHealth PR program which allows people with COPD to complete a home-based PR program supported by mHealth technology. Guidelines for reporting of mHealth interventions outline the importance of user experience testing and user feedback during the design process before investigating effectiveness,¹⁴ therefore this formative study was undertaken as part of the m-PR™ development process. The aim of this study was to determine the ease of use, satisfaction and acceptability of m-PR™ in people with COPD.

Methods

Study design and participants

This mixed methods observational study was conducted between November 2020 and November 2021. A convenience sample of participants with COPD was recruited following an initial PR assessment (either in-person or via teleconsultation) at three sites in Sydney, Australia. Any patient with COPD aged over 40 years was invited to take part in the study if they were willing and had access and ability to use a smartphone and had a data plan. People were excluded if they had: supplemental oxygen requirements during the exercise assessments or training procedures; limited skills in the use of mobile technology; or limited English language.

Intervention

The m-PR™ platform was co-created by a team of technology and health experts together with consumers. Software engineers from the Australian eHealth Research Centre (Commonwealth Scientific and Industrial Research Organisation [CSIRO]) built the platform based on the material design framework (https://m3.material.io/). The software is available on the Apple App Store (iOS) and the Google Play Store. The health experts consisted of academics and multi-disciplinary clinicians who are considered specialists in the field of PR. The m-PR™ platform contains both a patient app (being evaluated during this study) and a clinician web-based portal. The patient-facing m-PR™ app consists of daily monitoring of COPD symptoms, a tailored individualised exercise program with demonstration videos, education videos, medical management through a virtual “Lung Foundation Australia” action plan¹⁵ and goal setting for behaviours such as increasing daily physical activity levels and reducing smoking. Patient-reported outcome measures being the COPD Assessment Test (CAT) and the Modified Medical Research Council Dyspnoea Scale (mMRC) were also integrated within the m-PR™ app with users prompted to complete on a weekly or daily basis respectively. Health coaching was provided through regular in-app notifications and through weekly contact with the physiotherapist. The m-PR™ clinician portal allows the clinician to upload individualised exercise prescription and educational videos, track each participant's progress and also provide consistent data capture across the participating PR programs. Further details of the m-PR™ platform components are available.¹⁶ Participants were provided with a 30-min education session about how to use m-PR™, and were asked to engage daily with the m-PR™ app for 4 to 8 weeks whilst completing their standard 8 week PR program. The 4–8 weeks duration was given as an option as within 4 weeks of using the m-PR™ app all the possible functionalities within the app would have been used by the participant. The standard PR program was completed either in the centre with twice weekly exercise sessions with one face-to-face education session or independently at home with twice weekly exercise sessions, once-a-week telephone support and education provided through the m-PR™ app. Weekly contact with the physiotherapist was provided either in the centre or via telephone to address any questions the participant had about using the m-PR™ app or general questions about their PR program. Participants were also able to contact the physiotherapist by telephone at any point if they had any questions about the m-PR™ app or their general PR program.

Data collection

Demographic and clinical characteristics

Baseline demographic information collected included age, sex, smoking history, and comorbidities. At baseline, all participants either completed spirometry or had recent spirometry results (within the last 6 months if clinically stable) recorded. Baseline technology engagement was assessed with a custom-designed survey (in appendix), wherein participants indicated which devices they access, the frequency of use, use of their devices to access other types of digital information (e.g., shopping, banking, health information) and their self-rated mobile technology skill competency.

Outcome measures

Participant satisfaction survey

A custom-designed satisfaction survey (in appendix) was used to explore participants’ experience with the m-PR™ app. The survey contained four primary questions with responses on a 5-point Likert scale and one open-ended question. The questions were designed to determine the degree of enjoyment of using the m-PR™ app, the perceived ease and helpfulness of utilising different components of the m-PR™ app, and the helpfulness of the m-PR™ program at improving motivation to exercise and at monitoring and managing COPD symptoms. The closed-ended questions required participants to tick a box in response to each question. The open-ended question enabled participants to express in writing which parts of the m-PR™ app they found helpful or not helpful.

System usability scale

Participants rated their ease of use with the m-PR™ app via the System Usability Scale (SUS) which is a ten-item scale giving a global view of subjective assessment of usability of digital systems.¹⁷ Responses were on a five-point Likert scale and scores ranged from 0 to 100, with 100 representing the best score¹⁷ and a score of 68 or higher indicating an above average SUS score.¹⁸

Participant interviews

Participants were invited to complete an individual, semi-structured interview via tele-conference, telephone or in-person, with a study researcher (SW) with extensive clinical and research experience in PR, to explore their experiences of using the m-PR™ app. A semi-structured interview guide (in appendix) was developed to determine acceptability of the m-PR™ app. Open-ended and probing questions allowed in-depth discussion. Interviews were audio recorded and transcribed verbatim using an external transcription service. Transcripts were deidentified by each participant being assigned a code. The theoretical framework of acceptability (TFA) (version 2)¹⁹ was used as a multi-faceted construct to determine the acceptability of the m-PR™ app to people with COPD.

Data analysis

Participant characteristic data and outcome data were analysed using descriptive statistics of mean and standard deviation (SD) for continuous variables and numbers and percentages for categorical variables (IBM SPSS Statistics 27).

Satisfaction Survey: Each question in the participant satisfaction survey was evaluated separately and the absolute number and percentage of responses were determined for all participants across each scale. The anchor responses of the 5 point scale were grouped to provide a three level response. For example, when analysing the outcome of “enjoyment”: the “not at all enjoyable” was labelled as “not enjoyable”, the “neither” enjoyment level was labelled as “neutral”, and the “somewhat enjoyable”, “enjoyable” and “very enjoyable” were labelled as “enjoyable”. When analysing the outcome of ease of use and helpfulness, “very difficult / very unhelpful” and “somewhat difficult / somewhat unhelpful” were merged into “difficult / not helpful”, “neither” were labelled as “neutral”, “somewhat easy / somewhat helpful” and “very easy / very helpful” were labelled as “easy/helpful”. Scores were then determined for each question by allocating a “−1” to “not enjoyable/difficult/not helpful, “0” to “neutral” and 1 to “enjoyable/easy/helpful” with an average score and SD determined.

Participant Interviews: We conducted a systematic analysis of the interviews using a framework-based approach with five steps: 1. Familiarisation of the transcripts: A study researcher (MD) reviewed all transcripts identifying patterns and divergent views; 2. Identification of a thematic framework: From the initial coding, two study researchers (MD, ZM), both with extensive clinical and research experience working with patients with chronic respiratory diseases, identified key issues and concepts arising from the transcripts, applying these to the seven constructs of the TFA (v2); 3. Indexing: A study researcher (MD) applied single passages of text from all transcripts to the TFA (v2) which was reviewed and discussed by a different study researcher (ZM); 4. Charting: Two study researchers (MD, ZM) independently rearranged data according to the TFA, then finalised this step through an iterative process and presentation to a third researcher (SW); 5. Mapping and interpretation: Two study researchers (MD, ZM) explained the findings.²⁰

Results

Participant flow and characteristics

The flow of the study participants is presented in Figure 1. Fifteen out of 18 participants completed the study. Three participants did not complete the study with two participants withdrawing consent and one participant unable to be contacted after agreeing to participate. Participant characteristics are shown in Table 1. Participants had mild-to-severe COPD, a high proportion of co-morbidities and rated themselves as having good technology skills.

Figure 1.

Participant flow.

Table 1.

Participant characteristics.

Characteristic	All participants (n = 15)	Participants who completed interviews (n = 12)
Age (yr), mean (SD)	70 (10)	69 (11)
Sex, no. of female (%)	8 (53)	5 (42)
Lung function, mean (SD)
FEV₁ (l)	1.57 (0.62)	1.66 (0.65)
FEV₁ (% predicted)	62 (18)	64 (17)
FVC (l)	3.18 (1.03)	3.23 (1.14)
FVC (% predicted)	97 (23)	95 (24)
FEV₁/FVC (%)	53 (18)	56 (19)
Severity of COPD, n (%)^a
GOLD 1	4 (27)	3 (25)
GOLD 2	9 (60)	8 (67)
GOLD 3	2 (13)	1 (8)
GOLD 4	0 (0)	0 (0)
Comorbidities, n (%)^b
Musculoskeletal	8 (53)	6 (50)
Respiratory	9 (60)	8 (66)
Cardiovascular	8 (53)	7 (58)
Cancer	2 (13)	2 (17)
Mental	4 (27)	4 (33)
Metabolic	5 (33)	4 (33)
Other	11 (73)	9 (58)
Self-rated technology skill, n (%)
Poor	0 (0)	0 (0)
Adequate	4 (26)	3 (25)
Good	9 (60)	7 (58)
Very good	2 (13)	2 (17)

Legend: COPD: Chronic Obstructive Pulmonary Disease; FEV₁: forced expiratory volume in 1 s; FVC: forced vital capacity.

Classification of severity of COPD is determined according to the GOLD Report. (21) GOLD 1 - mild: FEV₁ ≥ 80% predicted. GOLD 2 - moderate: 50% ≤ FEV₁ < 80% predicted. GOLD 3 - severe: 30% ≤ FEV₁ < 50% predicted. GOLD 4 - very severe: FEV₁ < 30% predicted.

Multiple responses possible.

Participant satisfaction survey

The level of enjoyment experienced by participants using the m-PR™ app is presented in Table 2. Of the fifteen participants, 10 (67%) participants scored “enjoyable”, 5 (33%) participants were neutral and there were no participants scoring “not enjoyable”. The ease of use and the helpfulness of the m-PR™ app is reported in Table 3. All participants who responded to the set of questions on “ease of use” found the education video component and communicating with the therapist component “somewhat easy” or “very easy”, with the latter component also found to be helpful to all participants. The exercise program component had the lowest mean score for ease of use (mean [SD] 0.46 [0.9]). Table 4 presents the perceived helpfulness of using different components of the m-PR™ app to improve COPD self-management. Most of the participants found the app “somewhat helpful” or “very helpful” for all the self-management factors, particularly “understanding of education topics”, “motivation to do the exercise program” and “motivation to take more steps each day”. Eleven of the 15 participants provided text-based comments in the survey on what they found “helpful” or “not helpful” about the m-PR™ app which has been included in Table 5. The major components that participants stated they found helpful were the education resources, the weekly contact with the physiotherapist, and the added incentive the m-PR™ app provided to help them complete tasks. The survey also identified that some participants found the m-PR™ app complex with the technical aspects being challenging.

Table 2.

The enjoyment level of the m-PR™ app.

	Participants n (%)	Mean Score (SD)
		0.67 (0.49)
Not enjoyable	0 (0)
Neutral	5 (33)
Enjoyable	10 (67)

The mean score was calculated from an allocation of: −1 (not enjoyable), 0 (neutral), or 1 (enjoyable).

Table 3.

Number of participants who found the different components of the m-PR™ app easy to use and helpful to use.

	Ease of Use		Helpful to Use
	Participants n (%)	Mean Scores (SD)	Participants n (%)	Mean Scores (SD)
Daily Monitoring	14 (93)	0.87 (0.5)	12 (86)	0.71 (0.7)
Exercise Program	9 (69)	0.46 (0.9)	10 (76)	0.69 (0.6)
Exercise Videos	10 (83)	0.75 (0.6)	11 (92)	0.92 (0.3)
Education Videos	11 (100)	1.00 (0)	11 (92)	0.92 (0.3)
Communicating with the therapist	9 (100)	1.00 (0)	9 (100)	1.00 (0)

-The number of responders to each question was different and has been indicated below:

-Total numbers who responded to each “ease of use” component: Daily Monitoring (n = 15); Exercise Program (n = 13); Exercise Videos (n = 12); Education Videos (n = 11); Communicating with the therapist (n = 9).

-Total numbers who responded to each “helpful to use” component: Daily Monitoring (n = 14); Exercise Program (n = 13); Exercise Videos (n = 12); Education Videos (n = 12); Communicating with the therapist (n = 9).

-Scores for “ease of use” were calculated from an allocation of: −1 (difficult), 0 (neutral), or 1 (easy).

-Scores for “helpful to use” were calculated from an allocation of: −1 (not helpful), 0 (neutral), or 1 (helpful).

Table 4.

Number of participants who found the different components of the m-PR™ app helpful to use to improve self-management.

Factors	Participants n (%)	Mean Scores (SD)
Motivation to do the Exercise Program	10 (71)	0.57 (0.8)
Motivation to take more steps each day	9 (75)	0.58 (0.8)
Understanding of education topics	13 (93)	0.93 (0.3)
Ability to monitor your symptoms	9 (64)	0.43 (0.9)
Ability to manage your symptoms	8 (57)	0.50 (0.7)

-The number of responders to each question varied and has been indicated below:

-Total numbers who responded to each component: Motivation to do the Exercise Program (n = 14); Motivation to take more steps each day (n = 12); Understanding of education topics (n = 14); Ability to monitor your symptoms (n = 14); Ability to manage your symptoms (n = 14).

-Scores were calculated from an allocation of: −1 (not helpful), 0 (neutral), or 1 (helpful).

Table 5.

Respondent comments on the satisfaction survey of the helpful and unhelpful parts of the m-PR™ app.

	Respondent comment	Sex/Age	Enjoyment level
Positive	It gave you an incentive to complete tasks.	Female, 65	Enjoyable
	The app is like having a physio sitting on your shoulder which I felt prompted me to do the exercises daily.	Female, 60	Enjoyable
	I found the weekly phone calls useful.	Female, 75	Enjoyable
	I liked the educational pieces.	Female, 52	Enjoyable
	Very helpful to still have contact with the clinician and not be on own.	Male, 78	Enjoyable
	The exercise videos are very basic and once seen them don’t need to look at them again.	Male, 78	Enjoyable
	It took some time getting used to the process of recording elements of the exercise program, eg walking - however after downloading Google's pedometer app, this was simplified greatly.	Male, 82	Neutral
	Found the “how to breath” during activity video's really helpful and would be good to put in the “weekly rehab”.	Male, 78	Enjoyable
	I didn't believe it would/should be enjoyable perse. The functions were easy to use or maybe fun.	Male, 50	Neutral
	It was very handy to have access to all the videos information, guides and measures.	Female, 64	Neutral
Negative	The app was confusing at times, too many things to go through to get where you wanted to go. The mindfulness sessions were too short.	Female, 78	Enjoyable
	I found it difficult to put in the exercises as it didn't roll over each day, I found I had to manually select the day I was exercising.	Female, 52	Enjoyable
	Not enough definition/choices for “breathing” and “sputum colour” options in daily symptoms.	Male, 67	Enjoyable
	The play button is too close to the progress bar, some people need/like bigger screen.	Male, 50	Neutral
	Because I needed assistance to use the app I personally didn’t find it helpful.	Male, 85	Neutral
	Would be useful to have new exercise videos or ones that changed every month or gave new options.	Male, 78	Enjoyable

System usability scale

The SUS mean score of 71 (SD 16) demonstrates an above average score for the perceived usability of the m-PR™ app.

Participant interviews

Twelve participants consented to be interviewed following completion of the user testing. The interviews ranged in duration from 20 to 46 min (mean 33 min). Characteristics of participants who were interviewed are presented in Table 1 together with characteristics of the entire group. Participant quotes mapped to the TFA (v2) are presented in Table 6, with an indication after each quote on the participant's enjoyment level from the m-PR™ app survey data.

Table 6.

Consumer acceptability of the m-PR™ app.

Component Construct of TFA	Indicative quotes highlighting a range of responses to the m-PR™ app's acceptability positive negative
1. Affective Attitude	I found the app was like your physiotherapist on your shoulder because it gave you the motivation to do your exercises for the day because you knew that you were being watched. So, I thought it was great. It was a great motivational tool…(female, 60 years, enjoyable)	I have to say that I did feel, whether it was guilty or just anxious, like oh god, (the physio) needs this and I've got to get out and do it, and I thought it was to do all of this all on every single day, and I just couldn't manage it. (female, 52 years, enjoyable)
	My concern is motivation, and we all have problems with that, is to keep those exercises going when I've finished with the program. So the app could be useful for that. (female, 74 years, neutral)	I would like to be able to open up the app and look at the series of things that were required exercise-wise to maintain a certain state of health with a respiratory condition, just follow the steps, follow the instructions without any other distracting things like having to go back to the instructional videos to check on this or check on that…. I would like to see an app that just does that; just runs through the - a series of exercise for the day. (male, 85 years, neutral)
	I felt it was great to have all the different things that they had in there, so that the monitoring of symptoms and the exercise and the information I think it's quite a comprehensive that's very helpful to have that all in the one spot. (female, 64 years, enjoyable)
	It's like I’ve got someone on tap there. I’ve got some support during it, just from the app alone. I know it's not specific and it can’t tell me to increase on a specific day or whatever, but it's something that I can refer to to help me along, to get me going, just as a reminder just to slow down or get the breathlessness to a scale of four (male, 67 years, enjoyable)	I mean to me, the most – the thing about pulmonary rehab is sort of a chance for some interaction with other people, chatting, hello, how are you going and that sort of thing. (male, 76 years, enjoyable)
	It was useful in that it helped me track my symptoms and it triggered when I needed to see the doctor or activate my action plan. (female, 74 years, neutral)
	I did watch some videos……. if you were dealing with people who were just doing it all remotely via the app, then that would be very helpful. (female, 64 years, enjoyable)
2. Burden	I must admit I found it pretty straight forward once I had been introduced to the system. (male, 82 years, neutral)	I don’t know if there's anything simpler but I’d say the simple the better when you get to the – particularly the over 70 s. (female, 75 years, enjoyable)
	I didn't find it too complicated. I'm a person who's still working pretty much full time, and I'm on a computer all day, and I use lots of different apps, so it might be different for someone who's not so familiar with such things, but for me, I find it fine.(female, 64 years, enjoyable)	She probably prescribed it right, I may not have interpreted it right. I had looked at it as being it was what I had to do daily, and for me, it was too much. (female, 52 years enjoyable)
	I found the app easy to use. I like the app. I liked phrase of the day. I felt some of the questions probably didn’t relate to me but no, I found it great… like the phlegm and coughing up sputum and stuff, things like that. (female, 60 years, enjoyable)	I did all the things that (were) recommended. I download – sort of logged out of the app and downloaded it again. I tried everything possible, and it just, I just couldn’t get through to the rehab (section). (male, 67 years, enjoyable)
3. Ethicality	It wouldn’t worry me at all (having personal information stored on the App and therefore on the server) (female, 60 years, enjoyable)	You wouldn’t want to be going into the detail of the security, you’d just want people to know or as a person, you would just want to know that a) a hack is not going to be able to get the data as it's trans versing the internet and b) that the only people seeing the data are clinicians and that the servers are obviously secure, i.e., don’t want the data getting out to who it shouldn’t be getting out to (male, 50 years, neutral)
	It doesn’t concern me at all [name being connected to this type of e-health platform]. (male, 78 years, enjoyable)	There's nothing really personal in this thing anyway. If someone knows that I cough a bit then what the hell so in this case I wouldn’t worry. But I would generally be careful about making information available………..I don’t like Big Brother watching over me. (male, 76 years enjoyable)
		I’d like to know that this is secure because it's my medical information. (male, 50 years, neutral)
4. Intervention Coherence	I think it's wonderful. It's got the exercise – you tap on the exercise that's a really great way to look at that and see what works and what doesn’t. (male, 67 years, enjoyable)	I’m notoriously bad at doing exercise and all these things, so I confess that I didn’t use it every day, and sometimes, I skipped those first things and just went and used the recording walk I was doing or something like that. But when I first got the app, I was doing that. (female, 64 years, enjoyable)
4. Intervention Coherence	I thought that was great. Because if you’re doing a regular training, you know you’ve got to do three lots of 10 with a certain weight, but it's good to have this program in front of you so you that you can count – you can fill it out as you go and know where you are, and if you want to increase the weight you increase the weight or go down or whatever, so I think that's great. I love it. I love that exercise part of it. (male, 78 years, enjoyable)	So I didn’t worry too much about the breathlessness scale and notifications, educational settings were more straight forward and the other things, the stretches, well, that part of the exercise program. The effort scale, I didn’t really do that. (male, 82 years, neutral)
5. Opportunity Costs	The walking, the problem I have with that is that I’m walking all day. I stop and start, stop and start, stop and start. It's not - I’m not doing a structured walk – if I’m doing a structured walk, I’d probably use it and that would be fine, but if I’m not going on a structured walk, I’m simply walking around up the shops or around the shops or around the garden or whatever, I can’t keep – or golf, I can’t keep turning it on and off, that's the problem.(male, 78 years, enjoyable)	When I had no appointments and I wasn’t time poor, I just picked up the iPad and I did these daily symptoms and then I went on and I put my exercise. How long it took me to walk, what steps I did, how many lunges, any of the myriad of exercises. (male, 69 years, enjoyable)
6. Perceived effectiveness	It's pretty much the same [comparing to centre PR] except I don’t have a treadmill and I don’t have a cycle, so now, I mean my exercising at home is down to two-kilogram weights, which I’ve already had, and just sort of making sure I sit down and use the weights (male, 76 years, enjoyable)
6. Perceived effectiveness	Certainly if I get sick again, and I’m hoping I won’t, that – to be able to look at those measures, that would be probably good. (female, 74 years, neutral)
7. Self-efficacy	I just played with it, and I guess I’m experienced enough – I really am a bit of a phobe, so if a – it took me a while, but I’m getting more adventurous on the phone………….So I just explored the app and found things that way. (male, 67 years, enjoyable)
	I think as long as the patients are willing to actually play as such, as in press buttons and have a look and see what it's doing, they'll probably figure it out reasonably quickly. I think it's a great program, it's fantastic. (male, 50 years, neutral)
	I generally found it easy to use once I played with it a bit and found my way around it, and good. (female, 64 years, enjoyable)
	But I tended to just keep exploring, pressing buttons and exploring. But it didn't take me long to find my way around, that wasn't an issue. (male, 78 years, enjoyable)

TFA v2 = Theoretical Framework of Acceptability version 2 ¹⁹

The m-PR™ app elicited divergent feelings among participants, who reported both positive and negative acceptability. Some participants liked the m-PR™ app features which tracked their progress, motivated them to exercise, monitored their symptoms and had educational components (e.g., educational videos). Participants also felt the m-PR™ app may help them to maintain their exercise program on completion of PR. However, some participants said they found the m-PR™ app contained too much information, they would miss the in-person interaction if they only used the m-PR™ app to complete their PR program, and it may be too difficult to use for those people unfamiliar with technology.

Some participants reported that using the m-PR™ app required minimal effort, whereas other participants perceived the m-PR™ app as requiring too much effort, with increased burden commonly due to too much information presented or technology problems. Participants expressed mixed views on sharing personal information and the need for data security.

Most participants reported that they understood the intervention and what was involved with using the m-PR™ app, although many said they were not compliant with all aspects of the m-PR™ app during user testing. Most participants perceived the m-PR™ app would be effective in delivering PR and felt confident that they would be able to participate in the program via the m-PR™ app, particularly those who were familiar with technology and willing to explore the m-PR™ app.

Discussion

There is limited research examining the user experience of mHealth platforms to deliver PR in people with COPD and this is the first study to examine the user experience of an Australian mobile pulmonary rehabilitation app (m-PR™). The findings of this study indicate that the majority of participants found the m-PR™ app enjoyable, easy to use and that the components helped to support their COPD self-management. The interview data indicated overall satisfaction with the m-PR™ app, however, there were differing opinions on the app's features, the effort required to use it and data security.

Overall, 67% of the participants found the m-PR™ app enjoyable whilst 33% were neutral. This level of enjoyment is similar to a previous study that reported high levels of satisfaction with a mHealth PR program in a small sample of participants.¹² The perceived enjoyment in utilising technology includes physiological, cognitive, and psychological components that are closely interconnected to each other. In the process of performing activities, enjoyment is likely to be experienced when a balance between skill and challenge is achieved.²¹ Interestingly, when the participants who rated their enjoyment as “neutral” were questioned about why their enjoyment was “neutral”, they remained positive describing their experience as being “useful”, “OK” and “satisfactory”. This level of enjoyment with the m-PR™ app is not dissimilar to a study exploring another COPD self-management app in the Netherlands²² which had a mean satisfaction level of 7.7 (SD 1.7) on a scale of 1 to 10 (with 1 representing not being satisfied at all, to 10 representing being very satisfied).²²

The majority of the participants considered all m-PR™ components easy to use, particularly for the “education videos”, “communicating with the therapist”, “daily monitoring” and “exercise videos” components. A similar result was found for the helpfulness of these components. These findings related well to the SUS score of 71 (SD 16) which also indicates an above average result for overall perceived usability of the m-PR™ app. The component with the lowest score in this section was the ease of use of the exercise program itself. Similar feelings about the exercise component of mHealth PR apps have been previously reported.⁹ Given participants have to actively engage in this component by both completing the exercises and recording the dosage information, it is likely that this was the most complex task that was undertaken on the m-PR™ app and is the functionality that, in the future, people with COPD should be well supported about at the start of a program.

The self-management factors outlined in Table 4 had more variability in how people found them helpful, with the highest ranked factor being “understanding of education topics” and the lowest being “ability to manage your symptoms”. The latter result may be because the participants did not perceive that the management functions in the app (such as exercise training) changed symptoms such as breathlessness given the short-time frame in which they may have used the app (e.g., 4 weeks). Interestingly, in a randomised controlled trial of the MyCOPD online platform,²³ in which participants’ knowledge, skill and confidence for self-management was measured with the patient activation measurement (PAM), participants demonstrated a greater PAM level after 12 weeks of PR using MyCOPD when compared to usual care. This finding suggests that it may take time to master self-management skills and change health behaviours which is understandable given the complexity of tasks required to improve or maintain wellbeing in people living with COPD.

Similar to other mHealth app feedback,^8,9,24 participants provided a mixture of positive and negative comments on the aspects they found helpful about the m-PR™ app (Table 5), and on the acceptability of components of the m-PR™ app (Table 6). Overall our findings build on previous work that has reported generally positive user experience and satisfaction in using mHealth platforms to deliver PR^8,9,12 with our study indicating participants enjoyed the delivery of education through these platforms and were motivated by the technology to complete tasks and track progress. However, we have also determined that this innovative model of PR delivery may not be suitable or acceptable to all patients, particularly those who enjoy in-person interactions or who find technology too difficult to use. A future avenue for research is to determine who is best suited to participating in PR via an app together with measuring consumer adherence and engagement with the technology.

There are some limitations to this study. A convenience sample of COPD patients attending PR programs was used and recruitment was hindered due to COVID-19 restrictions resulting in a small sample size which may undermine the validity of the study findings. Participants in this study had mild-to-severe COPD, and therefore the findings of this study might not be transferable to people with severe-to-very severe COPD or those on long-term oxygen therapy, and future studies should explore how patients with a higher disease burden or those that use supplementary oxygen during exercise experience receiving PR via mHealth technology. All participants from this study self-rated their technology skills as at least being adequate, creating a potential sample bias of technology-engagers and therefore the study findings may not be applicable to COPD patients with limited technology skills. This bias might be due to the nature of the study: participants willing to engage with mHealth were highly likely to be familiar with mobile technology, i.e., technology-engagers themselves in life. Although this might be a limitation in terms of sample bias, in clinical practice it is likely that mHealth technology will be of more benefit to patients who are reasonably confident in their technology skills.

In conclusion, this user experience study has indicated that the majority of participants found the m-PR™ app enjoyable, easy to use and helpful in managing their COPD. Furthermore, the study provided useful information about the components of the m-PR™ app which people with COPD are willing to engage to improve self-management. These results suggest that mHealth programs are an acceptable way to deliver a digitally enabled PR program at home or in a centre, to people with COPD who have adequate technology skills.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241237381 - Supplemental material for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease

Supplemental material, sj-docx-1-dhj-10.1177_20552076241237381 for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease by Sally L Wootton, Marita T Dale, Ye Tian, Meredith King, Jennifer A Alison, Andrew S L Chan, Marlien Varnfield, Ian A Yang and Zoe J McKeough in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241237381 - Supplemental material for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease

Supplemental material, sj-docx-2-dhj-10.1177_20552076241237381 for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease by Sally L Wootton, Marita T Dale, Ye Tian, Meredith King, Jennifer A Alison, Andrew S L Chan, Marlien Varnfield, Ian A Yang and Zoe J McKeough in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076241237381 - Supplemental material for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease

Supplemental material, sj-docx-3-dhj-10.1177_20552076241237381 for User experience testing of the mobile pulmonary rehabilitation (m-PR™) app in people with chronic obstructive pulmonary disease by Sally L Wootton, Marita T Dale, Ye Tian, Meredith King, Jennifer A Alison, Andrew S L Chan, Marlien Varnfield, Ian A Yang and Zoe J McKeough in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors would like to acknowledge the contributions of the NSLHD Chronic Disease Community Rehabilitation Service at Royal North Shore Hospital, Hornsby Ku-ring-gai Hospital and Mona Vale Hospital. The authors would like to acknowledge CSIRO software engineers Derek Ireland and Simon Gibson for the creation of the m-PR™ app. The authors would like to acknowledge and thank the participants that took part in the study.

Authors’ contributions

The study was initially conceived by SW, ZM, JA, MD, MV, AC, and IY. All authors contributed to the study design. SW wrote the first draft of the manuscript. All authors contributed to writing of the manuscript and critically reviewed it. All authors approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics approval and consent to participate

This study has been approved by the NSLHD Human Research Ethics Committee (HREC reference: 2019/ETH14039). All participants gave their written informed consent to participate in the study. Data will be stored according to, and as required by, the ethics committee and will be available from the corresponding author on request.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The m-PR™ platform development was funded by a Northern Sydney Local Health District (NSLHD) innovation grant, a Metro North Hospital and Health Service seeding grant, and the Australian government Commonwealth Scientific and Industrial Research Organisation (CSIRO). A NSLHD Chronic and Complex Care, Primary and Community Health seeding grant supported the research.

Guarantor

Sally Wootton.

ORCID iDs

Sally L Wootton

Andrew S L Chan

Supplemental material

Supplemental material for this article is available online.

References

McCarthy

Casey

Devane

, et al. Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2015; 2: CD003793.

Houchen-Wolloff

Spitzer

Candy

. Access to services around the world. In: Holland

Dal Corso

Spruit

(eds) Pulmonary rehabilitation (ERS monograph). Sheffield: European Respiratory Society, 2021, pp.258–272.

Hanssen

Nolan

CM.

Emerging models. In: Holland

Dal Corso

Spruit

(eds) Sheffield: European Respiratory Society; 2021, pp.294–310.

Wuytack

Devane

Stovold

, et al. Comparison of outpatient and home-based exercise training programmes for COPD: a systematic review and meta-analysis. Respirology 2018; 23: 272–283.

Neves

Reis

Gonçalves

. Home or community-based pulmonary rehabilitation for individuals with chronic obstructive pulmonary disease: a systematic review and meta-analysis. Cad Saude Publica 2016; 32: 1–25.

Seidman

McNamara

Wootton

, et al. People attending pulmonary rehabilitation demonstrate a substantial engagement with technology and willingness to use telerehabilitation: a survey. J Physiother 2017; 63: 175–181.

Kwon

Lee

Jung

, et al. An mHealth management platform for patients with chronic obstructive pulmonary disease (efil breath): randomized controlled trial. JMIR Mhealth Uhealth 2018; 6: e10502.

Deng

Sheng

Jiang

, et al. A home-based pulmonary rehabilitation mHealth system to enhance the exercise capacity of patients with COPD: development and evaluation. BMC Med Inform Decis Mak 2021; 21: 325.

Whittaker

Dobson

Candy

, et al. Mobile pulmonary rehabilitation: feasibility of delivery by a mobile phone-based program. Front Comput Sci 2021; 3: 1–9.

10.

Rassouli

Boutellier

Duss

, et al. Digitalizing multidisciplinary pulmonary rehabilitation in COPD with a smartphone application: an international observational pilot study. Int J Chron Obstruct Pulmon Dis 2018; 13: 3831–3836.

11.

Jiang

Liu

Guo

, et al. Evaluating an intervention program using WeChat for patients with chronic obstructive pulmonary disease: randomized controlled trial. J Med Internet Res 2020; 22: e17089.

12.

Cerdán-de-Las-Heras

Balbino

Løkke

, et al. Effect of a new tele-rehabilitation program versus standard rehabilitation in patients with chronic obstructive pulmonary disease. J Clin Med 2021; 11: 1–11.

13.

Candy

Reeve

Dobson

, et al. The impact of patient preference on attendance and completion rates at centre-based and mHealth pulmonary rehabilitation: a non-inferiority pragmatic clinical trial. Int J Chron Obstruct Pulmon Dis 2023; 18: 1419–1429.

14.

Agarwal

LeFevre

Lee

, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. Br Med J 2016; 352: i1174.

15.

Lung Foundation A. COPD Action Plan 2023. https://lungfoundation.com.au/resources/copd-action-plan/. [Available from.

16.

Wootton

Dale

Alison

, et al. Mobile health pulmonary rehabilitation compared to a center-based program for cost-effectiveness and effects on exercise capacity, health status, and quality of life in people with chronic obstructive pulmonary disease: A protocol for a randomized controlled trial. Phys Ther 2023; 103: 1–9.

17.

Brooke

. SUS: a “quick and dirty” usability scale. London: Taylor and Francis, 1996.

18.

Sauro

Measuring Usability with the System Usability Scale 2011 [Available from: https://measuringu.com/sus/.

19.

Sekhon

Cartwright

Francis

. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017; 17: 88.

20.

Pope

Ziebland

Mays

Analysing qualitative data. Qualitative Research in Health Care 2006: 63–81.

21.

Crutzen

van ‘t Riet

Short

. Enjoyment: a conceptual exploration and overview of experimental evidence in the context of games for health. Games Health J 2016; 5: 15–20.

22.

Kooij

Vos

PJE

Dijkstra

, et al. Effectiveness of a mobile health and self-management app for high-risk patients with chronic obstructive pulmonary disease in daily clinical practice: mixed methods evaluation study. JMIR Mhealth Uhealth 2021; 9: e21977.

23.

Crooks

Elkes

Storrar

, et al. Evidence generation for the clinical impact of myCOPD in patients with mild, moderate and newly diagnosed COPD: a randomised controlled trial. ERJ Open Res 2020; 6: 1–10.

24.

Kadiri

Kerr

Oswald

, et al. Fit 4 surgery, a bespoke app with biofeedback delivers rehabilitation at home before and after elective lung resection. J Cardiothorac Surg 2019; 14: 132.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB