Abstract
Introduction
Home-based programmes for cardiac rehabilitation play a key role in the recovery of patients with coronary artery disease. However, their necessary educational and motivational components have been rarely implemented with the help of modern mobile technologies. We developed a mobile health system designed for motivating patients to adhere to their rehabilitation programme by providing exercise monitoring, guidance, motivational feedback, and educational content.
Methods
Our multi-disciplinary approach is based on mapping “desired behaviours” into specific system’s specifications, borrowing concepts from Fogg’s Persuasive Systems Design principles. A randomised controlled trial was conducted to compare mobile-based rehabilitation (55 patients) versus standard care (63 patients).
Results
Some technical issues related to connectivity, usability and exercise sessions interrupted by safety algorithms affected the trial. For those who completed the rehabilitation (19 of 55), results show high levels of both user acceptance and perceived usefulness. Adherence in terms of started exercise sessions was high, but not in terms of total time of performed exercise or drop-outs. Educational level about heart-related health improved more in the intervention group than the control. Exercise habits at 6 months follow-up also improved, although without statistical significance.
Discussion
Results indicate that the adopted design methodology is promising for creating applications that help improve education and foster better exercise habits, but further studies would be needed to confirm these indications.
Introduction
Cardiovascular diseases (CVDs) are a major global health problem. In 2008, an estimated 17.3 million people died from CVDs, representing 30% of all deaths worldwide.
1
Exercise may help reduce the burden of CVD by not only delaying the onset of CVD, but also by improving recovery in those unlucky enough to develop it. Exercise training is a standard component of cardiac rehabilitation programmes offered to patients with coronary artery disease (CAD),
2
Traditionally, rehabilitation programmes are delivered in hospitals or other community centres by multidisciplinary teams. Referral and uptake rates of these programmes are known to be poor mainly because patients find it difficult to attend them.
8
Information and telecommunications technologies (ICT) can facilitate the delivery of these home-based programmes. In particular, mobile health (m-Health) systems allow health services to be delivered remotely, reducing costs and facilitating patients’ engagement and empowerment. In this context, we present the educational and motivational design approach and the results achieved within the HeartCycle project where a mobile-based cardiac rehabilitation system called “Guided Exercise” (GEx) was created. GEx was designed to guide patients through the rehabilitation process by means of physical training and educational strategies to promote a healthy lifestyle while simultaneously ensuring the safety of the patient.11–13
This paper reports only those results influenced by or influencing the motivational and educational strategies and does not cover the clinical results.
Related work
Electronic systems for motivation and education have been adopted in different medical fields, and particularly for fostering physical exercise and a healthy lifestyle.14–19 From these studies, it is possible to deduce that each type of patient (under rehabilitation, with chronic conditions, etc.) and each objective (weight-loss, physical exercise, adherence to intervention, etc.) has its own characteristics and challenges. Generalising, key factors to be considered when designing such systems are: setting goals, timing and personalisation of feedback, usability and acceptance of the system and the use ofwell-grounded behavioural change theories. Relevant to this work are mobile applications that were specifically built for cardiac rehabilitation and that included educational and motivational strategies.20–25
For example, in a study conducted in the Mayo clinic, 20 25 patients used an m-Health application for monitoring blood pressure (BP), weight, glucose, exercise and dietary habits over a three-month period. The application also provided patients with educational content about healthy lifestyle to prevent further cardiac complications. The use of the application led to reductions in weight and BP, better diet and reduced re-hospitalisations and visits.
In a preliminary study conducted by Worringham et al., 21 seven patients in cardiac rehabilitation were provided with a mobile phone application, a GPS tracker and an electrocardiogram (ECG) sensor for real-time monitoring of physiology. By the end of the study, patients increased the walking distance in the 6-minute walk test (6MWT) and improved their physical health-related quality of life (QoL).
In a study conducted by Antypas and Wangberg, 22 the intervention group was provided with a website with tailored content based on health behaviour models and with automated text messages on the phones. At the end of the pilot, the authors observed a higher level of overall physical activity in the intervention group, smaller adherence to the consultation of the website and no statistically significant differences in stage of change, self-efficacy, social support, perceived tailoring, anxiety or depression.
Varnfield et al. 23 provided 60 patients with a mobile app to track exercise, symptoms and physiological variables, complemented with a set of periodic telephone calls of a mentor. The authors reported similar improvements in the control and intervention groups regarding physiological and psychological health outcomes (6MWT, emotional state, QoL), but improved uptake of post-MI cardiac rehabilitation, better adherence and higher completion rates in the intervention group.
Dale et al.
24
Most of the cited works integrate physical and educational aspects into mobile applications for improving safety and comfort in the rehabilitation process while positively influencing lifestyle. In three cases,20–23 they did not specifically base their design on behaviour change strategies, which could potentially improve long-term lifestyle changes. Although the need to create individualised intervention plans is recognised, only two studies implemented real personalisation,
23
Methods and materials
Since cardiac rehabilitation programmes have multifactorial components, the HeartCycle GEx system was developed following an holistic and multidisciplinary approach. A team formed by cardiologists, nurses, psychologists, engineers and software developers was set-up to design the system. Although not part of the core team, volunteer patients were also frequently consulted from the early stages of the development. The team reviewed relevant medical literature for cardiac rehabilitation including guidelines
26
Starting from the clinical requirements, the system design and development were guided by a combination of methodologies: goal-directed design (GDD), 29 persuasive systems design (PSD) 30 and agile software development. 31 These methods were selected because they had proven to be beneficial in the context of health and fitness systems.32–36 Details about the development process and preliminary evaluations are provided in previous publications, concretely in Ottaviano et al., 37 Vera-Muñoz et al. 38 and Salvi et al. 39 This paper focuses on the description of the final prototype and the design of its educational and motivational features.
The GEx system
The GEx system is composed of three main parts: the mobile station, for monitoring physical exercise and providing live guidance during exercise sessions; the patient station, which acts as a collector and gateway of patient’s data and is responsible for delivering educational contents to the user; and the professional station, a web-based application used by doctors to prescribe and tailor each exercise programme, to visualise progress and to be alerted in case of problems. The overall system is sketched in Figure 1. In the following, more details are provided for each sub-system.
The GEx system comprising the mobile station (sensor and smartphone), the patient station and the professional station with its web interface.
The mobile station
The mobile station includes a wearable sensor capable of measuring biomedical information including a one-lead ECG, heart rate (HR), respiration rate and activity level based on 3D acceleration. The sensor is attached to a special shirt (produced by Clothing+, Finland) that positions it correctly against the skin. The mobile station also includes a smartphone application for user interaction and temporary data storage during exercise sessions. The sensor and the smartphone are connected through a Bluetooth serial link using an ad-hoc application protocol.
The mobile station offers guidance in two ways: (1) it helps the user to follow the exercise protocol set by the doctor and (2) it detects and provides warnings about risky situations. Regarding the first aspect, the smartphone software helps patients to reach and maintain their HR in the specified range: it gathers real-time HR and activity level values from the sensor, smooths the signals and uses HR and activity level trends to provide instructions to the user (further details about the signal processing approach are provided by Runtti et al.
40
). Live feedback is provided to the user during the exercise in a multi-modal fashion. Three kinds of messages are delivered, depending on the measured HR and the thresholds set in the protocol. The user is guided to increase or decrease the exertion level or to stop completely, thanks to a combination of text messages on the user interface, blinking, bold colours in the background and vibration patterns (Figure 2a). In addition, at the end of the exercise, indications are given about how the actual HR was kept within the thresholds for each phase of the exercise (warm-up, core, cool down) and globally, as an exercise “index” (Figures 2b and c).
Examples of feedback provided by the mobile station. (a) Live feedback, (b) End-of-exercise indicators per exercise phase and (c) Overall exercise index.
Regarding the detection and warning of risky situations, a set of safety measures were designed by the medical team. Before starting a session, the patient’s medical condition are checked using BP values and the responses to a symptoms questionnaire, both manually inserted by the patient on the smartphone interface. If the situation is not completely safe, the application warns the user to avoid exercise for a certain period of time and a notification is sent to the professional station. During exercise, moreover, the smartphone checks the HR and respiration rate values. If HR values are outside the safety ranges or if HR increases abruptly but the respiration rate does not increase, the user is instructed to stop exercising and the smartphone prevents the user from using it further for the rest of the day.
The patient station
The patient station is a dedicated device (a tablet PC) that is meant to be used at home. Patients can use it for synchronising data with the professional system over a wireless connection, for checking and personalising their exercise plan, for analysing their performance and progress and for receiving feedback and educational information.
The patient station user interface comprises five main sections (Figure 3).
A ‘home’ section with a virtual assistant (or ‘avatar’) used for delivering tips and suggestions in a direct and friendly way. A ‘messages’ section where users can read messages generated by the system or sent by the caregiver. Messages can contain links to educational material and questionnaires. A ‘calendar’ section where to consult and configure the scheduling of the exercise sessions. An ‘exercise’ section where patients can review indicators of past sessions. Progress indicators are also shown for each individual session, on a weekly basis and as global progress over the rehabilitation process. A ‘learning’ section with multimedia material to help patients understand their health condition and the benefits of exercising. Example of persuasive features of the patient station: performance indicators.

The professional station
The professional station is the system used by medical professionals to prescribe personalised exercise routines as part of the cardiac rehabilitation programme. The system supports patients’ initial assessment, provides a set of visualisation tools to monitor their progress, and generates alerts in the case of medical complications.
The user interface displays a list of all the patients sorted by an algorithm that takes into account the number and the severity of alerts that have been triggered for each patient, placing those with a higher risk on top. These alerts are related to increased BP values before the exercise, increased HR values during exertion beyond safe thresholds, discrepancies between prescribed and reported Borg intensity higher than 2 units, and non-compliance to rehabilitation programme longer than 7 days without exertion. When a patient is selected, a summary area is shown together with detailed information on the data available. From this section, caregivers can prescribe exercise routines and modify existing ones by adjusting the intensity (HR limits), the frequency (number of sessions per week), the duration of each session and the type of the exercise (endurance training versus strength training). The patient summary section also provides access to the data recorded during exercise sessions, displaying charts of recorded HR versus prescribed HR, ECG signal, respiration rate, workload and the answers to questionnaires. Indicators about adherence are also shown, both in relation to the overall programme as the percentage of sessions done and, to each specific session in terms of percentage of time within prescribed thresholds.
These interfaces, and the information they provide, were co-designed by engineers and the medical team of the project. The continuous feedback of the doctors was taken into consideration for incremental improvements during the projects’ iterations.
Implementation of the educational and motivational strategies
When designing the GEx system, health belief models
41
were used to classify patients on the basis of the perceived benefits and barriers to self-efficacy in healthy behaviour. A team of engineers and clinical experts worked together to provide proper strategies for modifying patients’ beliefs associated to risky behaviours and transforming them into ‘desired behaviours’, i.e. health-related activities that patients could achieve with the help of the system. Seven desired behaviours were identified as follows:
DB1 the patient starts using the GEx system; DB2 the patient regularly performs exercises using the GEx system; DB3 the patient understands the benefits of exercising; DB4 the patient intensifies the exercise programme; DB5 the patient has general knowledge about CVDs, heart attack and risk factors; DB6 the patient has knowledge about how to reduce and control risk factors; DB7 the patient continues exercising and engages healthy behaviours even without the GEx system.
These behaviors were validated by 11 patients in a focus group, where they were all very positively accepted. Desired behaviours were then mapped to a set of motivational strategies, which were derived from Fogg’s functional triad: tools, social actors and media. 42
The tools for persuasion were defined as features in both the mobile and the patient stations. The PSD principles were taken into consideration and self-monitoring, rehearsal, rewards, reminders, suggestions and trustworthiness were considered in the implementation.
30
The messaging system of the patient station, in particular, was exploited, which was used to provide personalised feedback and information to help patients follow their training programme. A set of predefined messages was defined, according to the following categorisation.
Messages to inform about the exercise plan prescribed by the doctor (e.g. type of exercise, duration, frequency, level of exertion, etc.). Practical instructions to facilitate the use of the system (e.g. how to synchronise the patient station with the mobile station, how to wear the sensor, etc.). Feedback about the exercise sessions (e.g. indicators about performed session, level of exertion, etc.). Tips about health and lifestyle, scheduled according to the rehabilitation phase.
Also, the performance indicators were available for the patient for each exercise, or weekly or on a monthly basis. The indicators were computed by mixing a set of statistics including the percentage of time the user was within the prescribed range of HR and the number of exercises performed compared with those prescribed. Based on these indicators, a reward messaging strategy was built taking into account the adherence to the exercise programme (Figure 3).
From the technical perspective, GEx made use of a development platform for personalised health which allowed part of the application’s logic to be programmed through human readable rules. 39 Rules could include variables, such as the user’s name or the value of indicators of the last exercise, which allowed messages to be more personalised. In total, 114 rules and 112 messages were written, divided into eight groups: two groups dedicated to the generation of messages related to changes in the prescribed exercise plan, a group dedicated to the personalised feedback after the execution of each exercise, two related to periodic progress indicators, one related to the automatic suggestion of educational content based on the answers given to questionnaires and one to generate reminders about planned sessions.
These rules were implemented with the use of JBoss Drools (see https://www.drools.org/). Drools was chosen for the availability of visual editors and also for the possibility of creating domain-specific languages that make rules very easy to be read by people with limited technical skills. An example rule is provided in Figure 4.
An example of a rule defined with a domain-specific language. The message includes a variable, $EXINDEX, that is statically configured as the exercise index of the last exercise session.
As a social actor, the virtual coach or “avatar” was considered. Messages delivered by the avatar were part of the already-mentioned messaging strategy, but had a more concise and informal character.
The medium was identified as a personalised educational programme. Educational material was selected from relevant literature and tailored to patient’s different levels of knowledge, self-confidence and progress in the rehabilitation process. The majority of it was adapted from the Heart Manual and includes, among others, comprehensive information about the exercise, technical instructions for using the system, medical information about the disease and its causes, modifiable risk factors and lifestyle changes. In total, the content included 131 HTML pages with text, pictures and videos, translated into three languages: English, Spanish and German. The content was structured into 20 chapters with 1–15 pages per section. A misconceptions questionnaire was used in order to personalise the delivery of the content. According to the answers provided in the questionnaire, the system was able to detect patients’ incorrect assumptions and to provide them with suggestions about appropriate educational material from the repository.
Desired behaviour, motivational and educational strategy and used software tools.
Study design
The GEx system was tested in a randomised controlled trial conducted in three centres in Spain, Germany and the United Kingdom. The study included a control group, receiving standard rehabilitation according to the national procedures of the three involved countries, and an intervention group, receiving the rehabilitation through the GEx system. Standard care of MI patients differs between countries and even centres, but, approximately, it can be generalised with three phases. Phase 1 is shortly after the acute event and it implies hospital care. During this phase a thorough physical examination is performed together with initial educational and counselling. Phase 2 is when the actual rehabilitation programme starts. Patients may be initially kept in hospital, then sent home and referred to community-based programmes. This phase includes exercise training and usually lasts 12–21 weeks. Phase 3 involves long-term maintenance of physical activity and life-long rehabilitation. It is usually provided by ambulatory coronary sports groups and the primary care. All patient in HeartCycle were recruited during phase 2 and the study was performed at home after discharge from rehabilitation facilities for a recommended period of 21 weeks.
The main objective of the trial was to determine whether the GEx system can improve the physical exercise capacity at 6 months follow-up after the rehabilitation compared with national standards. Secondary objectives, the actual topic of this paper, were to assess the user experience of the GEx system in terms of acceptance and perceived usefulness, and whether its use leads to higher adherence to exercise prescriptions, improved exercise behaviours and better education about heart-related health.
Patients who had suffered a cardiac event were selected and analysed for suitability (medical inclusion and exclusion criteria) shortly after the event by physicians involved in the project. If eligible, they were approached and given a brochure explaining the objectives of the trial and how to use the system, and non-medical inclusion/exclusion criteria (e.g. able to use a computer/mobile device) were verified. The programme started 4–8 weeks after they left the hospital, in accordance with guidelines.
Inclusion criteria were: patients suffering from CAD and presenting after an acute MI or elective coronary intervention, ejection fraction (EF) >30%, willing to exercise with a preference for walking, running or cycling, eligible for standard local rehabilitation and ability to use computer and Internet. Exclusion criteria were: EF <30%, severe congestive heart failure (New York Heart Association class IV), inability to exercise, severe valve disease, recent cardiac surgery (<4 weeks), implantable devices (implantable cardioverter–defibrillator or cardiac resynchronisation therapy devices, pacemaker) or open thorax wound (see also Skobel et al. 43 ).
The intervention group was given the GEx system and was assigned a personalised training programme based on an initial assessment done at the hospital. All training programmes followed a common pattern, which was based on international guidelines about exercise training prescription for cardiac rehabilitation.
26
The training programme was developed in five ‘stages’ along the 21 weeks. In each stage, the frequency of sessions, their duration and the HR parameters were increased. Caregivers used this standard prescription as an initial recommendation, but were also allowed to modify the training programme during the execution of the trial in order to adjust the exertion level to the actual conditions of the patient. Sample size estimation identified 130 individuals (65 per group) for obtaining sufficient power and level of significance with what regards the clinical objectives of the study.
The Regional Ethical Review Board at the University of Aachen (00017326, EK218/11), Hospital Clínico San Carlos of Madrid (C.I. 11/232-E) and University of Hull (12/YH/0072) approved the study. Principles according to the Helsinki declaration 45 were followed. Written informed consent was obtained from all participants prior to inclusion in the study. The study was registered at ClinicalTrials.gov with the identifier NCT01761448.
In the following, details are provided about how specific aspects were measured. For each measurement, simple statistics were computed such as sample size, mean and confidence interval at 95% assuming a normal distribution. When differences were under analysis, Student’s t-tests were also performed.
User acceptance and perceived usefulness
User acceptance and perceived usefulness are known to be associated with a long-term adoption of health mobile apps.
46
These were measured at the intervention group with a questionnaire inspired by the Technology Acceptance Model (TAM),
47
which has been repeatedly used in tele-health.
48
The questionnaire was sent four times to patients: soon after the GEx systems was presented at baseline visit, after 1 week, after 5 weeks and at the end of the study. Questionnaires were filled out on paper during visits at the beginning and at the end of the study and at home during the study. Separate questions covering similar aspects were asked for the three used modules, namely the sensor, the smartphone and the patient station to assess differences among these modules.
Education about heart-related health
Education about heart-related health was measured at the beginning and at the end of the study with a ‘quiz’ about the disease and exercise. The quiz was designed together with the medical team to measure whether patients had achieved an increased level of knowledge after the end of the rehabilitation programme. It encompassed the following topics: coronary heart disease, angina, importance of exercise and practicalities, health habits and risk factors, and level of self-management. Six of the 17 questions were taken from the Cardiac Knowledge Scale developed by Maeland. 50 As further analysis, patient station’s logs were also examined to understand how often the patient accessed the educational content and which type of educational content was mostly accessed.
Exercise adherence and long-term habits
Adherence during the rehabilitation programme was calculated by comparing the total number of prescribed exercise sessions with the number of sessions that were actually started by patients. Further indications were derived from the percentage of exercise sessions explicitly cancelled by users and from the number of minutes of performed exercise compared to the total number of minutes prescribed. To understand how the coaching programme affected exercise habits in the long run, exercise habits were measured at baseline and at the 6 months follow-up after the end of the study. A custom questionnaire was used that included two main aspects: the time per week spent in sweating physical activity and the subjective level of intensity.
System reliability
The reliability was derived from the logs generated by GEx. Thanks to a remote logging facility, all the instances of the application sent their logs periodically to a remote server where logs were stored and made accessible on a web interface. Logs were used also to gather statistics about what parts of the applications were more frequently accessed by patients.
Results
The clinical results of the study are published in Skobel et al.; 43 here we report those related to acceptance, motivation and education.
The trial lasted over 2 years (April 2011–August 2013). A total of 132 patients (59 ± 14 years, 11% females) gave consent, 33 in Germany, 44 in the UK and 55 in Spain. Of consented patients, 118 were randomised and included in the study: 55 (47%) were assigned to the intervention group and 63 (53%) were assigned to the control group. Patients did not show significant differences in terms of MI severity, as all patients had a left ventricular ejection fraction (LVEF) above 30% to meet inclusion criteria, nor were there significant differences between control and intervention group at baseline (intervention group median LVEF, 59; interquartile range (IQR), 11.3; control group median LVEF, 59; IQR, 10). In the intervention group, 15 of 55 patients (27%) did not receive the rehabilitation plan, or received it too late because of the unavailability of the clinical team and 21 (38%) dropped out. The reason for dropping out can be summarised as follows.
Two patients had medical complications independent from the rehabilitation. Six patients withdrew because of lack of time. Seven patients found the system too cumbersome to use. Six patients were demotivated because the system prevented or stopped the exercise for safety reasons too often. CONSORT diagram of the HeartCycle GEx study.
In the control group 12 of 63 patients (19%) joined the programme too late and 9 (14%) patients were lost to follow-up. Only 8 of 55 (14%) patients of the intervention group and 7 of 63 (13%) of the control group participated to the 6-month, long-term habits follow-up. A CONSORT
51
diagram of the study is presented in Figure 5.

Given that education and motivation were secondary objectives of the trial, the collection of related questionnaires was often performed poorly, fact that caused some data points to be missing.
User acceptance and perceived usefulness
User acceptance and perceived usefulness of the GEx system, mean ± confidence interval.
Separate answers for the three sub-parts of the system, the sensor, the smartphone and the tablet (patient station), are omitted for brevity. They confirm the general trend, and show that the most appreciated sub-system was the patient station, while the sub-system that received slightly worse rankings was the sensor, probably because of some wearability problems.
The acceptance questionnaire also included open questions where patients had to describe the characteristics of the system they preferred and those they did not appreciate. The feedback can be summarised as follows.
The overall system was considered as appropriate for its purpose.
Example comments: ‘It keeps me motivated.’, ‘I got used to it and like to exercise.’, ‘It gave me the opportunity to get out of the home and try and get myself fit after the operation. I believe it has achieved that and more. I feel better in myself and I can achieve most jobs without taking about it.’
Three patients remarked on the added value of performing monitored exercise session at home especially in terms of freedom.
For example: ‘Easy because you can exercise at home.’, ‘Having retired, I found it quite easy to fit into my everyday life. I organise my exercises around my life and grand children and of my wife.’
One patient considered the exercise routines too dull and ‘not challenging enough’. Three patients considered the patient station easy to use and four considered its content helpful.
For example: ‘Very good. Easy to use.’, ‘Helpful information.’, ‘It is very easy once you get the way of it.’
One patient complained about the responsiveness of the touch screen of the patient station. Another patient thought that the smartphone was too big and did not fit into a handlebar mount for cycling. Some patients experienced technical problems while using the system. Specifically, problems were reported by 22 of 55 patients (40%) in the 1st week questionnaire, by 8 (14%) in the 5th week questionnaire and by 10 (18%) in the last week. Three patients found the wearable sensor not very comfortable and three other patients had problems with the set-up of the system.
Example quotes: ‘I have no experience as there has always been a problem in the overall equipment.’, ‘A bit too long and bulky.’, ‘Dislike the shirt, for a woman can’t wear a bra.’, ‘Software gremlins are frustrating. Exercise routines are a bit monotonous. Warm up & cool down phase are almost impossible to achieve in real life. Heart monitors not reliable enough.’
As future improvements, one patient suggested to include the facility to send message to the health professional from the patient station, two patients proposed simplification of the overall system, particularly in relation to the high number of required devices and accessories and one patient asked for GPS tracking to be included during the exercise.
Education about heart-related health
Percentage of correct answers given to the knowledge questionnaire by control and intervention groups at baseline visit and at the end of the study, mean ± confidence interval.
Number of generated messages per category.
The sections of the patient station that were more accessed by users were those containing feedback, concretely messages (31%) and home (29%), followed by exercise (18%), education (13%) and calendar (8%). Regarding the use of educational content, patients consulted mainly chapters related to ‘practical information’ (30%), ‘resistance exercise’ (30%) and ‘how to exercise with GEx’ (14%), probably because these were the topics about practical aspects they were less trained at. With the exception of few chapters, all chapters were visited at least once, on average, by patients.
Exercise adherence and long-term habits
Percentage of started sessions, cancelled sessions and minutes of exercise performed versus prescribed, mean ± confidence interval.
Exercise time (in minutes) and effort (in 1 to 10 scale), mean ± confidence interval.
System reliability
During the trial some technical problems were identified. The analysis of the logs shows that 43% of the exercise sessions were fully completed, 37% were not started or interrupted by the safety algorithms, 18% were abandoned because of technical errors (e.g. poor signal quality in the sensor or poor Bluetooth connectivity) and 2% were stopped by the user. Therefore, many exercise sessions were aborted automatically, a fact that caused six of the drop-outs. Of those sessions interrupted for safety reasons, 63% were because of high BP, 26% because the HR was out of range for too long and 2% because of symptoms detected before starting the session.
Other technical problems gathered from the logs are related to the patient station, particularly communication problems with the server (93% of logged failures), mistaken rules in the rule engine (1%) and database access issues (4%). The reason why the number of connection errors to the server is so high is that the patient station used an aggressive synchronisation strategy that sent and received data from the server every 5 minutes and, when a communication error happens, it attempts three retries.
Discussion
User acceptance and perceived usefulness
The results of the questionnaires show that user acceptance was rated positively in all its aspects at all stages of the study. This is in accordance with studies such as Scalvini et al., 52 which reports high user satisfaction (values are not comparable), or Worringham et al., 21 where ease of use is rated 4.8 on a 1 to 5 scale. In comparison, HeartCycle’s ease of use was rated 3.5 on average at the end of the study, but the questionnaires used in both studies differ. The most similar measurement of acceptance in Dale et al. 25 was the number of patients who read all the messages, which was 85%. In HeartCycle all messages were read, but this was expected as the interface disabled all functionalities until the user had read all the pending messages.
These positive results, nonetheless, are in contrast with some of the drop-outs. Seven patients of 55 (13% of the intervention group) abandoned the use of the system because they thought it was too bulky and complicated, which may also justify the mild result in the ease of use dimension of the acceptance questionnaire.
Usability was assessed multiple times in HeartCycle, but while preliminary user tests of each separate part of the system showed encouraging results, probably the whole set of tablet, smartphone, sensor and t-shirt put some patients off, especially the older ones. As a recommendation for future studies, it is advisable to assess the acceptance of the whole integrated system early in the project, also considering practical and logistic consequences that may arise from its use.
The drop-outs due to lack of involvement were partially expected and may not be linked to the acceptance of the system. As Antypas explains: ‘Another reason for the users to stop using the intervention is that they might have achieved a satisfactory (for them) level of activity and, therefore, did not need the help of the intervention’. 22 The figures are, in fact, comparable between the intervention group (11%) and the control group (14%).
In general, drop-outs and refusals to join tele-care interventions have multiple dimensions as shown by Sanders et al. 53 The HeartCycle study confirms that some participants can reject a technology-based intervention because of felt ‘requirements for technical competence and operation of equipment’. 53 Nonetheless, other aspects such as potential disruption of identity (e.g. patients feeling more dependent or ill) or to conventional services (e.g. patients preferring physical contacts with physicians) were not analysed in this study.
For what regards perceived usefulness, answers to the questionnaire show that the GEx is perceived as a motivational tool for performing exercise in a safe way and increasing adherence to the prescribed exercises. The feedback strategy seems to play a positive role, although not all patients strongly agree on this. A positive result regarding perceived usefulness is also shown in Dale et al., 25 were 77% of participants felt the programme helped them change their behaviour.
Education about heart-related health
Results of the education questionnaire show an increased percentage of correct answers at the intervention group and no increase at the control group, although the effect was not high. The reason for this moderate effect relies on the fact that both groups already had a very high average of correct answers at baseline (83% and 81%), which may indicate that patients were already educated enough about the addressed subjects, probably thanks to personal counselling given by care professionals after discharge.
From the analysis of the logs it arises that a high number of messages were delivered to patients, half of which contained links to some sections of educational content. Thanks to these messages, patients were invited to read the educational content in a friendly and personalised way. As a result, the education section of the application was consulted 13% of the time. Within this section, patients consulted mainly chapters with the topics about practical aspects they were less trained at.
The only available comparison with other studies is related to the number of messages received by each patient, which, in Dale et al., 25 was 129 over a period of 21 weeks, compared with an average of 163 messages over 21 weeks in HeartCycle. The study interestingly also reports that the number of messages was considered adequate by 79% of the users.
Exercise adherence and long-term habits
Results about adherence differ depending on the indicator. By looking at the number of total performed minutes versus prescribed, adherence was low, which can be explained by the fact that a high percentage of sessions were interrupted by either connectivity problems or the safety algorithms. By looking at the number of exercise sessions or explicitly cancelled sessions, adherence can be considered satisfactory, especially for those patients who completed the training programme.
The comparison of adherence with other studies depends on the metric employed. Varnfield et al. 23 report a measurement of adherence that is different from that used in this study. The most comparable outcome is what Varnfield et al. define as ‘completion’, as attendance at the 6-week assessment based on those randomised to each group. Varnfield et al. report a completion of 80%, while in HeartCycle, considering the drop-outs and patients who did not join the study, completion was 35%. For what regards cancelled sessions, Worringham et al. 21 reported that 13.4% of the sessions were explicitly cancelled, against 2–4% (depending on the group) of HeartCycle.
For what regards long-term habits, results show that the intervention group increased both time and intensity at the 6-month follow-up, although the number of samples was not enough to be statistically significant. The reason for such poor data collection is due to the little emphasis that was placed on the motivational and educational aspects in the design and the execution of the clinical study. Future studies should address this problem by focusing on the behavioural change aspects of the rehabilitation. Instead of questionnaires, wearables and pedometers can be used to assess physical activity more objectively in the long term (as, for example, done by Thorup et al. 54 ).
Even if with limitations, the positive outcome about long-term habits confirms the results of Antypas and Wangberg, 22 where, 3 months after discharge, the intervention group reported a higher level of physical activity compared to the control group. Also in Dale et al., 25 participants in the intervention group reported increased adherence after 3 months, but adherence plateaued at 6 months. This measurement is not directly comparable with HeartCycle, as different metrics were employed.
System reliability
Three main problems affected the GEx system during the trial: the safety algorithms, which were too strict given the actual level of fitness of the patients, poor quality of the ECG signal and reduced connectivity between the smartphone and the sensor or between the tablet and the Internet. These problems caused more than half of exercise sessions to be aborted automatically, a fact that annoyed patients and led to six (11%) drop-outs.
Part of these issues were caused by contextual factors such as bad connectivity or when the sensor was worn incorrectly, and cannot be attributed to defects of the system. Other issues were related to proper malfunctioning code, such as improperly written rules or excessively strict safety algorithms. Some of these problems may have been solved with better testing of the integrated prototype with actual patients, especially for what concerns the thresholds used in the safety algorithms.
Technical maintenance was also a challenge by itself, as the development team was based in a different country than two of the study sites. Therefore, the clinical teams located far from the development team were also responsible for some of technical maintenance, such as preparation of the equipment, issue tracking and software updates. This suggested the implementation of the remote logging and an automatic software update mechanism. These features allowed a more frequent monitoring policy and timely detection of bugs, but it was introduced when several patients had already dropped-out. In total, 16 updates of the GEx software were produced after the trial started.
Compared with other studies such as Worringham et al., 21 which reported few technical errors, or Varnfield et al., 23 where only 3 of 60 patients abandoned the study for technical difficulties, the GEx system performed worse. However, the technological complexity of the system used in HeartCycle was higher than described in those two studies.
Future developments should consider reducing the complexity of the system considerably. Nowadays, modern smartphones and wearables offer capabilities that were out of reach at the time HeartCycle was designed. A possible simplification would be merging the patient station and the mobile station into one ‘app’ instead of having two different systems. Also the sensor could be replaced by a commercial device. In fact, although few commercial devices are capable of measuring the ECG reliably, in HeartCycle this signal was used mostly to derive the HR, which is now easily detectable also by off-the-shelf wearables.
Limitations
The study suffers from a low number of participants and a relatively high number of drop-outs. This can potentially introduce bias in the perception of the system as only a limited number of motivated patients completed the study. In addition, the outcomes reported in this paper were considered secondary and explorative in the design of the trial, therefore were not sufficiently powered nor was enough emphasis placed on data collection. A further difficulty is due to the fact that the participants were recruited in three different countries, with different health care systems and cultures. Given the small numbers, it is impossible to characterise patients according to the site where the intervention was performed.
Conclusions
Given the issues encountered during the trial, no strong conclusions can be derived from this work. However, if we consider the results of those patients who did not encounter technical issues, indications are supportive of the employed motivational and educational methods. It can be stated that the design methods used for ‘engaging’ the patient towards completing the rehabilitation programme and for adopting healthy lifestyles can be considered adequate for this kind of system. Therefore, the study can be seen as an interesting proof-of-concept that will need more substantial validation.
Footnotes
Acknowledgements
We would like to thank the whole HeartCycle Consortium for their valuable contribution to this work. Manuel Ottaviano, Salla Muuraiskangas, Cecilia Vera-Muñoz, Wim Stut and Carolyn Deighan designed the GEx system and the coaching strategies. Dario Salvi, Alvaro Martínez-Romero and Manuel Ottaviano implemented the software. Hilkka Liedes, Anita Honka and Jean Luprano were responsible for developing the HR sensor and the safety algorithms. Erik Skobel, John Cleland and Christian Knackstedt were responsible for running the trial. Maria Fernanda Cabrera Umpiérrez, Maria Teresa Arredondo Waldmeyer and Andreas Triantafyllidis helped analyse the results.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially funded by the European Commission through the HeartCycle Project, ‘Compliance and effectiveness in HF and CHD closed-loop management’ (FP7 - 216695).
