Telehealth Centers: A Proposal of a Theoretical Model for Evaluation

Abstract

Background:

Telehealth is a form of assistance support and continuing education for professionals who work in Primary Healthcare.

Introduction:

The implementation of the Brazilian Telehealth Program has extended the need for evaluation of its results to improve its use and benefits. Thus, it is necessary to systematize the theory, principals, and guidelines that guide the program organization, its activities, and resources. The objective of this study is to propose a theoretical model of functioning and organization for telehealth centers of the Brazilian Telehealth Program.

Materials and Methods:

A study that combined bibliographic, documental research and empiric data to systematize information, which are the basis functioning of the program, and verify the availability of telehealth centers in Brazil.

Results:

The results were the construction of a theoretical model that demonstrated the evaluability of the object and the definition of methodological paths to continue the evaluation research that respond with fragilities and potentialities.

Discussion:

In Brazil, there is public funding for telehealth activities offered by telehealth centers linked to the Ministry of Health. Its operation is guided by support for training and assistance activities, enabling qualification and strengthening of professionals to improve patient care and increase access. The transformation of reality is possible and favored by the articulation of teaching and service. More research is needed to evaluate these services offered. Theoretical guidance through modeling is fundamental to qualify the evaluation.

Conclusion:

The theoretical model is an important tool to qualify and optimize the evaluative research process.

Introduction

Primary Healthcare (PHC) is the user's main doorway in the Brazilian Unified Health System (SUS). It is expected that it will take responsibility for the users it serves, will solve between 85% and 90% of the health demand, and will be a leader of healthcare networks, performing qualified referrals to other levels of care.^1,2

The work process of professionals in PHC is complex and requires a differentiated way of healthcare, especially in relation to the object of care and the way of organizing service and actions: the focus is the individual in his or her family context and in the physical and social environment, where healthcare is guided by the principle of health surveillance, combining actions of promotion, prevention, and cure, developed by a multidisciplinary team of professionals who have the sanitary responsibility on a defined territory.³

In this context, the Ministry of Health identified the need to qualify these professionals to raise the solvability of PHC with continuing education initiatives, such as multiprofessional residences in family health, and introductory courses, on attendance and distance specializations, among others. To support professionals, particularly those who work far from the main training centers, and to promote Continuing Education, the Ministry of Health created the Brazilian Telehealth Project in 2007, which in 2011 was established as the Brazilian Telehealth Networks Program.⁴

Telehealth is the use of information and communication technology on health for education actions and distance support, expanding access, qualifying care, and improving the solvability in the health system.⁵

Nine telehealth centers were set in Brazil as a pilot project of this program, which have been developing activities since 2007. Currently, these pilot centers are consolidated and offering services recommended by the Ministry of Health: Teleconsulting, Tele-education, Telediagnostics, and Formative Second Opinion.

Telehealth evaluations in Brazil have already demonstrated that its use extends the possibilities of patients' access to health services by means of health professionals' support.^6,7 However, the use of models to represent reality and guide evaluative processes points to still different scenarios of access to telehealth services in Brazil and their uses, as well as incipience of norms to regulate their use and fragile initiatives by the Federal Government to guide telehealth actions and the articulation of its application in the teaching and service areas.^8,9

Brazilian Telehealth Networks is a young program and its actions have supported the construction of improvements in the health system; therefore, further studies are needed to evaluate the fragilities and potentialities of the actions that have been developed.^8

–12 In this scenario, the supply and use of telehealth services have grown, also increasing the need to evaluate its results to improve the use and benefits of all the actors involved,^5,9,10 strengthening its structure and organization to allow the expansion of the program, whose range and reach benefit the whole country.

Despite the proposition of monitoring and evaluation indicators of telehealth centers,^8,9,13 theoretical and evaluative models specifically proposed for the orientation of evaluative researches of center of Telehealth Program Brazil Networks that can be replicated at the national level have not been found in the literature.

Therefore, the objective of this article is to propose a theoretical model for functioning and organization of telehealth centers of the Brazilian Telehealth Networks Program, from which it is possible to propose evaluative studies in the light of the program theory, principles, and guidelines on which they were created, which may result in useful information for the program, helping in the reorganization of planning activities and resources, adapting them so the proposal objects may be achieved.¹⁴

Materials and Methods

Study strategy

An availability study was carried out, whose objective is to realize a complete description of a program, which is graphically represented in a theoretical model, so, later, it will define an evaluation plan and establish an agreement among the interested ones.¹⁵ The study was developed between January and September 2016. To achieve the objectives of an availability study, the elements of Thurston and Ramaliu¹⁶ methodology were used to provide an understanding of the literature review systematization on the object for later design of the theoretical model of a telehealth center:

(1) Identification of goals, objectives, and activities of a telehealth center;

(2) Identification of available documents on the Brazilian Telehealth Networks Program and their telehealth centers;

(3) Identification of the available resources, the aimed activities of the program, expected impacts, and presumed casual connections;

(4) Understanding of the manner in which telehealth centers work;

(5) Development of an evaluation theoretical model;

(6) Identification of evaluation users and other involved principals;

(7) Acquisition of an agreement regarding procedure of an evaluation.

The expected output of an Availability Study is an exhaustive description of the program that should support the definition of an evaluation plan and the establishment of an agreement among the interested ones¹⁵ through a description of the mission and vision of the program, as well as its objectives, activities, methodology, and expected results involving and compromising those interested in the use of the study.^14,16

From the exhaustive description of the program, it follows the modeling, which is considered the most challenging stage of an evaluation study since it intends to graphically represent the theory of a program by identifying whether there is congruence between what the documents guide and what is observed in the effective development of the program.^15,17

In this process, the possibilities of evaluation of the program that are agreed among the participants are identified to qualify and make their practices more assertive.

Data collection

The techniques used to collect data according to the elements described above were a bibliographic review, interviews with key informants, visits to official Web sites, and documentary analysis, from 2007 to 2015.¹⁸ The interviews included professionals working in the Brazilian Telehealth Networks Program since the beginning of their activities, intentionally selected by the authors.

Data collection was carried out from the documental analysis of the publications (Fig. 1).

Fig. 1.

Publications included in the documental analysis. Source: Elaborated by the authors.

The analysis of the information found in the documents and articles selected by the bibliographic review was based on the principles of exhaustiveness, homogeneity, and adequacy.

The findings and products elaborated in this study were appreciated by professionals involved in the program for feedback and suggestions.

Ethics aspects

This study was approved by the Research Ethics Committee of the Federal University of Santa Catarina, according to Opinion no. 1.466.605. All the ethical precepts were respected for the construction of a study involving human beings and free consent and consent of all those interested and involved were obtained.

Results

The collected data on telehealth centers allowed the understanding of the theoretical bases of the Brazilian Telehealth Networks Program, as well as the context of their creation and insertion in SUS. Hence, it was possible to respond to each of the seven elements of Thurston and Ramaliu.¹⁶ The results for each element are summarized in Table 1.

Table 1.

Results from the Bibliographic Review on a Telehealth Center per Elements of Thurston and Ramaliu's Methodology^a

ELEMENTS	RESULTS
Description of goals, objectives, and activities of a telehealth center	Objectives: To develop actions of healthcare support and continuing education support of the teams, to broaden PHC solvability and promote its integration with healthcare networks
	Goals: The goals and indicators are set by each technical-scientific center in the projects submitted to the Ministry of Health for their financing and maintenance
	Activities: planning; provision of services: teleconsulting, formative second opinion, tele-education and telediagnosis; monitoring and evaluation of participation and services; dissemination and communication
Identification of documents of telehealth centers	• Principles and Guidelines of Brazilian Healthcare System (Organic Laws nos. 8.080/90 and 8.142/90, SUS defense pact—Order 399/2006, Decree 7.508/11); • Primary Healthcare (National Policy of Primary Health Care—Ordinance 2.488 of October 21, 2011; Primary Healthcare Principles and Guidelines, Integrated Health Actions, community guideness, and team/interdisciplinary work); • Continuing Education in Health (National Policy for Health Continuing Education—Ordinance GM/MS 1.996 from August 20, 2007); • Regulatory Instruments of Brazilian Telehealth Networks Program (Ordinance 2.546/2011; Ordinance 2.554/2011; Telehealth Guidelines for Primary Health; Technical Note 050/2015-DGES/SGTES/MS); • Articles: 1) The case of Telemedicine University Network: analysis of telehealth entry in the Brazilian political agenda—Silva and Moraes (2012) and Analysis of the implementation of the TeleHealth Program in Pernambuco State, Brazil: a case study—Oliviera et al. (2015)
Identification of available resources, intended activities programs, expected impacts, and presumed causal connections	Financial resources:
	• Projects sent to the Ministry of Health to maintain activities for a particular time (established in the project). • The resource is administered by foundations or organizations linked to educational institutions • Agreements or projects by State or Municipal Health Secretariats/Bureaus • Partnerships with departments of state or other public organization managements for production of materials and service offer
	Human and Physical Resources:
	• Teaching institutions and managers (state or municipality) are responsible for providing physical space, equipment, and hiring of professionals for the development of activities (by means of scholarship, service rendering, or contractual link).
	Intended activities:
	• The offered services/activities are as follows: Teleconsulting (synchronous and asynchronous), Formative Second Opinion (through Web site of the • Virtual Health Library), Tele-education (courses, Web lectures, videolessons), Telediagnosis (exams reports). • Guided by the reality of healthcare networks, taking into account the guidelines and principles of PHC
	Expected impact:
	• To contribute for the achieving of integrality care model by qualifying PHC team staff; • Referral reduction and qualification to other care levels; • Cost reduction; • Professional stablishing.
Understanding of a telehealth center functioning	• Coordination carried out by a teaching and/or health management institution or in partnership between both, supported by a Telehealth State Managing Committee. • Guidelines and assumptions of legislations and agreements of the Ministry of Health are followed (by means of Secretariat of Work and Education Management in Health—SGTES and the Secretariat of Healthcare—SAS, through the Department of Primary Healthcare—DAB), always considering the local context and the scope of the telehealth center, which can be municipal, intermunicipal, or state. • Services methodology is organized and structured considering two main components: (1) promotion of activities of Continuing Education in Health and (2) promotion of assistance support strategies. Both elements are objects of evaluation of this study. • Monitoring of participation in services is done monthly by each center through specific worksheets with a description of the achievement of the goals and indicators. They are integrated through the Monitoring and Evaluation of Results System (SMART) that allows analyzing the indicators proposed at a national level.
Development of an evaluation theoretical model	The theoretical model developed from the collected information presents graphically the context of insertion of a Telehealth Center, its main theoretical bases and operation modes (See Fig. 2).
Identification of the evaluation users and other main involved	Evaluation of telehealth center might serve the following: • Center coordination; • Professionals who work in the telehealth center; • Telehealth Service users (managers, professionals working at PHC, students); • Coordination of the Brazilian Telehealth Networks Program
Acquisition of an agreement regarding evaluation procedure	The theoretical model, as the final product of the evaluation study, will be presented to interested actors and experts in the field to validate and agree on the most appropriate ways to implement more efficient evaluation processes that can in fact support the improvement of the services offered and their results.

Source: Elaborated by the authors.

References.^15,16

PHC, Primary Healthcare; SAS, Health Assistance Secretariat; SUS, Unified Health System of Brazil.

Proposition of a theoretical model for evaluation of telehealth centers

From the development of the evaluation study, it was possible to graphically develop a theoretical model for telehealth centers (Fig. 2).

Fig. 2.

Theoretical model for telehealth centers of the Brazilian Telehealth Networks Program. Source: Elaborated by the authors. DAB, Primary Health Care Department; PHC, Primary Health Care; SGTES, Secretariat of Work and Education Management in Health; SUS, Unified Health System of Brazil.

This theoretical model attempts to express objectively the national context of telehealth centers, their main epistemological bases (Continuing Education in Health and Primary Health Care), the local context (management and structure), as well as the services offering process (for assistance support and continuing education support), and the expected results (as described in each element of analysis), according to Donabedian's proposal.²³

A telehealth center is inserted in the national context of SUS by means of Brazilian Telehealth Networks Program. Its conception is intersectoral, permeating the Ministries of Health, Education, Science and Technology, and Communication and Defense.

Its objective image is to promote the integrality and improvement of access and quality in PHC, through distance support that promotes qualification and security for the performance of human resources.

The main epistemological bases of the program are the Primary Healthcare and Continuing Education in Health, following the directives that guide them.^3,24

The telehealth centers are officially linked to the Ministry of Health through the Secretariat of Work and Education Management in Health (SGTES)²⁰ and the Health Assistance Secretariat (SAS), through the Department of Primary Health Care (DAB),²¹ which guides the manner in which the structure, the processes of offering telehealth services, and the expected results of a telehealth center in local contexts should be.

Regarding the structure of a telehealth center, it is foreseen that Teaching Institutions and Health Managing Organizations will be jointly or individually responsible for coordination, physical structure, and financial management. Several organizational designs are possible, with variations in the interaction between the Teaching Institutions with the health managers that can be of municipal, regional, or state scope, or even with foundations and public health sectors, where the approval of a work plan by the Bipartite Interagency Commission is required.

A telehealth center is formed by the participating telehealth points (primary health teams and professionals) that must have connectivity and equipment available for the access of the services offered, and by qualified teams of professionals hired to meet the demands of the participants by means of the offered telehealth services.

With this structure available, the telehealth centers seek to offer assistance and support for the continuing education of health work teams through the types of planned services based on scientific evidence. This offer must be made from the survey of the epidemiological demands and actual work processes of the participating health teams and professionals, with the purpose of transforming and qualifying such work practices, always based on the principles and guidelines of SUS, guided by Primary Health Care and Continuing Health Education.

The searched results aim at the qualification and strengthening of PHC to respond to the needs of the users it serves.

Discussion

Currently, there are over 40 centers implanted in 23 Brazilian states and others in the implantation phase. In Brazil, telehealth plays an important role in strengthening and expanding PHC as the network coordinator by supporting ministerial strategies to strengthen the proposed care model for SUS and qualifying the care provided to users.⁴

The two ordinances ruling the telehealth centers (one from the SGTES and another from the DAB) are coherent with each other regarding the objectives and functioning of the telehealth centers, differing only in the ways of financing and transferring resources from these two sectors. The telehealth centers financing may include a single source or both, depending on the type of local coordination and the project submitted to the Ministry of Health for approval of the services offered.

The expansion of this program coverage assumes that the telehealth centers are articulated with the service network of SUS, organizing and structuring the services in response to the needs demanded by the professionals and managers, which compound the healthcare networks in the area under influence of the telehealth center, with a strong alignment with PHC management, creating opportunities for reality orientation and its transformation by continuing education.

The support offered strengthens and qualifies the professionals and avoids the need to remove patients to other health services, which contributes to the qualified and resolute care and optimization of public resources use in health.^10
–12,25 The services allow professionals in different locations to access support for the various doubts that arise in the daily work.⁴

The monitoring of the participation in the services offered by the telehealth centers allows identifying the growth in their use over the years. Although the program is deployed and under growth, it is still little used in comparison with the offer capacity and support by telehealth services.^10,25 The existing official documents guide the practice of telehealth, but do not guarantee the incorporation of services by management and the access and use by professionals of different health teams.

Assessing fulfillment of the objectives proposed in daily life (expansion of PHC solvability and offer of support to healthcare, and continuing education of the teams) is still necessary.

Modeling is an essential element in an assessment study to give more precision and safety to those involved in decision making by certain methodological paths. By proposing a theoretical model for the telehealth center, it was possible to identify the object as subject to evaluation and provided the participants with a deeper understanding of the program and clarification about the theory that supports it.¹⁷

The construction of this theoretical model allowed the identification of several spotlights and possibilities of evaluative studies such as the evaluation of the structure of the centers, evaluation of the interaction between the managing organizations of the centers, evaluation of the connectivity of the telehealth points, and evaluation of the implementation of each service according to proposed objectives, among others. Each telehealth center may analyze and prioritize the aspects that are most necessary to evaluate.

However, in this process, two lines of evaluative studies were highlighted, which presented themselves as fundamental issues in the work process of a telehealth center, namely, care support and support for continuing health education. Each of these lines, although complementary to each other, presents a development complexity that requires specific attention of the researchers.

Although these proposed lines of research represent the two main objectives of a telehealth center, no study presenting models for evaluating the fulfillment of these objectives and considering specific indicators for this purpose were found.

To deepen this study, we propose the construction of two logical models and evaluation matrices, one for each identified line of research (care support and support for continuing health education), defining indicators, criteria, and evaluation parameters that are replicable to other telehealth centers.

Conclusion

Through this availability study, we see the validity of theoretical model construction, which contributes in identifying more appropriate and useful methodological paths to evaluate health programs, as the Brazilian Telehealth Networks Program.

Another contribution of this study was the formative nature of the process, which allowed clarifying the participants about possible contexts for the insertion of telehealth centers of the Brazilian Telehealth Networks Program and its modes of operation, allowing to resignify the work processes within the program and guide the planning and execution of actions.

Footnotes

Acknowledgment

The authors would like to acknowledge the National Council of Scientific and Technologic Development (CNPq) for funding this research (MCTI/CNPq/MS—SCTIE—Decit No. 08/2013—Research in continuing education for SUS and evaluation on health labor force).

Disclosure Statement

No competing financial interests exist.

References

Mendes

. The care of chronic conditions in primary healthcare: the imperative of the consolidation of family health strategy. Brasília: Pan American Health Organization, 2012. Brasília: Organização Panamericana da Saúde, 2012. Available at http://bvsms.saude.gov.br/bvs/publicacoes/cuidado_condicoes_atencao_primaria_saude.pdf (last accessed March 30, 2016 ).

Brasil.. Conselho Nacional de Secretários de Saúde. Primary Care and the Health Care Networks. Brasília: CONASS, 2015. Available at www.resbr.net.br/wp-content/uploads/2015/11/Conass-APS-e-RAS.pdf (last accessed March 30, 2016 ).

Brasil.. Ministry of Health. Secretariat of Health Care. Department of Primary Health Care. National Policy of Primary Health Care. Brasília: Ministério da Saúde/Departamento de Atenção Básica, 2012. Available at http://189.28.128.100/dab/docs/publicacoes/geral/pnab.pdf (last accessed February 20, 2017 ).

Brazilian Telehealth Networks website.. Telehealth History. Available at http://www.telessaudebrasil.org.br and http://dab.saude.gov.br/portaldab/ape_telessaude.php (last accessed February 2, 2016 ).

World Health Organization.. Telemedicine: Opportunities and developments in Member States: Reporto on the second global survey on eHealth. Global Observatory for eHealth Series. Geneva: World Health Organization, 2010.

Campos

, Ferrari

. Teleaudiology: evaluation of teleconsultation efficacy for hearing aid fitting. J Soc Bras Fonoaudiol, 2012; 24:301–308.

Novaes

, Machiavelli

, Villa Verde

, Campos Filho

, Rodrigues

TRC

. Tele-education for continuing education in mental health for family healthcare teams: an experience in Pernambuco, Brazil. Interface Comun Saúde Educ, 2012; 16:1095–1106.

Oliviera

, Frias

, Vanderlei

LCM

, Vidal

, Novaes

, Souza

. Analysis of the implementation of the teleHealth program in Pernambuco State, Brazil: a case study. Cad Saúde Pública, 2015; 31:2367–2378.

Silva

, Moraes

IHS

. The case of Telemedicine University Network: analysis of telehealth entry in the Brazilian political agenda. Physis, 2012; 22:1211–1235.

10.

Alkmin

MBM

, Marcolino

, Figueira

, Sousa

, Nunes

, Cardoso

, Ribeiro

. Factors associated with the use of a teleconsultation system in brazilian primary care. Telemed eHealth, 2015; 21:1–11.

11.

Piropo

TGN

, Amaral

HOS

. Telehealth, contexts and implications in Bahia scenario. Saúde Debate, 2015; 39:279–287.

12.

Pessoa

, Sousa

, Ribeiro

, Oliveira

, Silva

JLP

, Alkmim

, Marcolino

. Description of factors related to the use of the teleconsultation system of a large telehealth service in Brazil—The Telehealth Network of Minas Gerais. J Int Soc Telemed eHealth, 2016; 4:1–9.

13.

Brasil.. Ministry of Health. Manual of telehealth for primary health care. Brasília: Ministry of Health; Federal University of Rio Grande do Sul, 2012. Available at http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_telessaude.pdf (last accessed February 20, 2016 ).

14.

Leviton

, Khan

, Rog

, Dawkins

, Cotton

. Evaluability assessment to improve public health policies, programs and practices. Annu Rev Public Health, 2010; 31:213–233.

15.

Thurston

, Potvin

. Evaluability assessment: A tool for incorporating evaluation in social change programmes. Evaluation, 2003; 9:453–469.

16.

Thurston

, Ramaliu

. Evaluability assessment of a survivors of torture program: Lessons learned. Can J Program Eval, 2005; 20:1–25.

17.

Walser

, Trevisan

. Evaluability assessment thesis and dissertation studies in graduate professional degree programs: Review and recommendations. Am J Eval, 2016; 37:118–138.

18.

Trevisan

. Evaluability assessment from 1986 to 2006. Am J Eval, 2007; 28:290–303.

19.

Brasil.. Ministry of Health. National policy of continuing education in health. Ministry of Health, 2009. 64. Available at http://bvsms.saude.gov.br/bvs/publicacoes/politica_nacional_educacao_permanente_saude.pdf (last accessed February 15, 2016 ).

20.

Brasil.. Ministry of Health. Ordinance n. 2.546 of October 27, 2011. Redefines and extends the Brazilian Telehealth Program, which is now Brazilian Telehealth Networks Program. Ministry of Health, 2011. Available at www.telessaudebrasil.org.br (last accessed February 16, 2016 ).

21.

Brasil.. Ministry of Health. Ordinance n. 2.554 of October 28, 2011. Establishes in the Rehabilitation of Primary Health Units Program, the information component and the Brazilian Telehealth Networks in primary healthcare, integrated to the Brazilian telehealth networks program. Ministry of Health, 2011. Available at www.telessaudebrasil.org.br (last accessed February 16, 2016 ).

22.

Brasil.. Ministry of Health. Technical Note n. 50/2015-DGES/SGTES/MS: Guidelines for the offer of activities of the Brazilian Telehealth Networks Program. 2015. Available at http://189.28.128.100/dab/docs/portaldab/notas_tecnicas/Nota_Tecnica_Diretrizes_Telessaude.pdf (last accessed October 16, 2016 ).

23.

Medina

, Silva

GAP

, Aquino

, Hartz

ZMA

. Use of theoretical models in the health evaluation: conceptual and operational aspects. In: Hartz

ZMA

, Silva

LMV

[org.]. Evaluation of the theoretical models in the evaluation of the Programs and health systems. Salvador: EDUFBA/Rio de Janeiro: Fiocruz, 2005:41–63.

24.

Brasil.. Ministry of Health. Ordinance n. 35, of January 4, 2007: Establishes, in the Ministry of Health, the Brazilian Telehealth Program. Official Diary of the Union, 2007.

25.

Figueiredo

, Guedes

TAL

. Brazilian Telehealth Networks Program,. In: Valentim

RAM

, Araújo

, Guedes

TAL

, Figueiredo

, eds. Telehealth in Brazil and the technological innovation in Primary Health Care. Natal, Brazil: EDUFRN, 2015:27–46.