Abstract
In planning a telehealth project, a readiness assessment can help to improve the chances of successful implementation by identifying the stakeholders and the factors that should be targeted. We conducted a literature search and identified six questionnaires on readiness that can be used when implementing telehealth projects. Only one of them was sufficiently generic to be used with all kinds of telehealth projects and with different groups of participants (patients and public, health-care practitioners and organization personnel like health-care managers and technical support managers), but it had rather limited psychometric evaluation. Two of them had had good psychometric evaluation but they were specific to particular telehealth projects and groups of stakeholders. All six published questionnaires were in English. We have developed and validated a French-Canadian version of the practitioner and organizational telehealth readiness assessment tool.
Introduction
In planning a telehealth project, a readiness assessment can help to improve the chances of successful implementation by identifying the stakeholders and the factors that should be targeted. The term readiness embraces preparedness, receptiveness and the willingness to achieve something. We have examined the published questionnaires on readiness. (We focused on telehealth specifically, and excluded questionnaires that addressed the readiness for the future adoption of e-health generally.)
Literature
A literature search was used to identify published questionnaires. The following databases were searched for relevant articles in English or French up to January 1995: Medline, CINAHL, and Health and Psychosocial Instruments. A Google search was also conducted. The keywords readiness, and the combination of scale or assessment or indicator with readiness, were combined with telehealth, cyber and e-health. Six questionnaires were found:
The Organizational Information Technology/Systems Innovation Readiness Scale (OITIRS)
1
was developed in 1996 to guide telehealth project managers in evaluation, diagnosis, treatment selection and resources for the different steps in patient care. In 2002, the validity and reliability of the OITIRS was tested in a pilot study
2
as part of a telehealth project. In this model, positive readiness towards information technology/systems would reduce the possibility of failure of an innovation and would be influenced by different environmental and organizational factors. The OITIRS consists of eight sub-dimensions, each containing ten items, namely: resources, end-users, technology, knowledge, processes, values and goals, management structures and administrative support. These sub-dimensions were validated by a group of experts. The OITIRS is thus made up of 80 items that can be scored using a 7-point Likert scale, which also offers the option of answering ‘unable to respond’. It can be completed in 20 minutes. The participants in the OITIRS validation study were nurses with management positions, as well as other types of clinical and support personnel.
2
The results suggest that the OITIRS may be reliable and valid. However, there was much missing data and there was a lack of construct validity for the administrative support sub-dimension. The Organizational Readiness for Change (ORC)
3
tool was developed in 2002 to evaluate the readiness toward change in substance abuse treatment agencies. This was the first questionnaire used in the context of online clinical service provision. The ORC is based on the model of Simpson,
4
which describes the introduction of new technologies or knowledge. It was based on scales used in other studies on organizational climate
5–6
by adding new items specific to change readiness. Unlike the OITIRS, there are two versions of the ORC, i.e. the staff version (ORC-S)
7
and the director's version (ORC-D).
8
The questionnaires take into account the differences between these two groups and are divided into four areas: motivation to change, institutional resources for the programme, attributes of personnel and organizational climate of the programme. The ORC-S includes 129 items, whereas the ORC-D has 115. These items cover 18 areas, and can be scored using a 5-point Likert scale. The results of the psychometric study (n = 500) showed that the ORC items are reasonably reliable and generally one-dimensional. However, this tool has certain limitations since it has no questions on technology transfer or organizational processes. In 2004, inspired by the previous two questionnaires, the Assessment of the Readiness of Hospice Organizations to Accept Technological Innovation
9
was developed to measure the readiness of hospice organizations for using videophones in their patients' homes. This instrument comprises 19 questions, the majority of which focus on the different uses of technology in the field. Only four questions ask about the employees' perception of using videophones. This questionnaire has not had its content validated. In 2003, using OITIRS as a base, the Assessing Care Agencies' Readiness for Telehealth Tool
10
was created to measure the readiness for telehealth in home care. The questionnaire contains 35 questions, including aspects that had to be targeted before and during telehealth implementation. However, there is no information in the literature about the psychometric qualities of this questionnaire. In 2007, the E-health Readiness Assessment Tools for Healthcare Institutions in Developing Countries
11
were developed to assist in planning telehealth programmes. The tools were developed with a participatory action research that captured partners' opinions, a revision of the existing tools and the creation of a conceptual framework based on the literature about e-health in developing countries. The questionnaires cover four areas. Three of them – core readiness, societal and policy readiness – are common to both questionnaires. Technological readiness is aimed strictly at managers while learning readiness is for stakeholders. The questionnaire for managers contains 54 items, and the one for stakeholders contains 50 items. The questionnaire items are scored using a 5-point Likert scale. The validity and reliability of both questionnaires were tested in Pakistan using qualitative and quantitative methods.
11–13
The results demonstrated good validity and reliability for the two tools. However, the results were limited by the lack of participants at certain service levels and the under-representation of women. In 2004, Jennett et al.
14
developed three generic telehealth readiness assessment tools, which could be used in several clinical contexts. They were designed for three groups of people: practitioners, organizational representatives and patients/members of the public. These assessment tools originated from a readiness model developed by the same team of researchers.
15–18
The researchers were experts in various areas of health-care promotion (i.e. telepsychiatry, interdisciplinary trauma videoconferencing, family medicine, cardiology, health of the northern communities and women's health) and were knowledgeable in the different uses of information technology. Six main themes were covered by the questionnaires: general readiness, infrastructure readiness, commitment, planning, workplace readiness and technical readiness. The Practitioner Telehealth Readiness Assessment Tool and Patient/Public Telehealth Readiness Assessment Tool have three sections and 17 items (maximum score: 85). The Organizational Telehealth Readiness Assessment Tool has four sections with a total of 28 items (maximum score: 140). The items are scored using a 6-point Likert scale. The first two questionnaires have three sub-scores, whereas the third one has four. There appears to be no information in the literature about the psychometric qualities of these questionnaires.
Conclusion
Although six questionnaires have been published, there is information about the reliability and internal validity for only two, i.e. the OITIRS and the E-health Readiness Assessment Tools for Healthcare Institutions in Developing Countries. Unfortunately, there is a lack of construct validity in the first case and, in the second case, the constitution of the sample was not representative. Only the tools from Jennett et al. involve the readiness of three participant groups (practitioners, patients and organization personnel, including, among others, health-care managers and technical support managers), and can be generalized for all telehealth projects, including health-care service provision. Only the Jennett et al. questionnaires include scores and sub-scores, along with their interpretation. However, their reliability and internal validity has not yet been demonstrated. In future, therefore, the construct validity and psychometric qualities of all readiness assessment tools should be improved and different groups of stakeholders should be included. The six tools that are described above are only available in English. For transcultural validation in French-Canadian, we have retained an existing readiness tool (the Jennett instrument) that can be generalised for all telehealth projects and for different groups of respondents. 19
Footnotes
Acknowledgements
We thank everyone who took part in the literature review. Financial support was provided by the Centre de réadaptation InterAction, the Institut de réadaptation en déficience physique de Québec, the Agence de santé et de services sociaux de la Capitale Nationale and the Agence de santé et de services sociaux du Bas St-Laurent. We also thank the Elan Foundation, which supported É Légaré as part of a research internship at the Centre for Interdisciplinary Research in Rehabilitation and Social Integration. M-P Gagnon is a research fellow for the Canada Institutes of Health Research and P Lehoux holds a Canada Research Chair on Health Innovation. We thank D Kairy and the Centre for Interdisciplinary Research in Rehabilitation for their participation in data collection.