Developing a BI-based framework for assessing the smart levels of hospitals

Abstract

Objective: Business intelligence (BI) applications can result in higher smartness in hospitals. However, up to now, there has been a shortage of validated frameworks for assessing the smart levels of hospitals. Therefore, this study aims to develop a BI-based framework to assess the smart levels of hospitals. Methods: The research method is a combination of literature review, pilot test, and multi-case study. Results: The results of a multiple-case study approve the applicability of the framework since it can explain the status quo of the hospitals. Based on results of framework validity assessment with eight hospitals in Vietnam, the managerial implications for hospital directors to improve their smartness are suggested. Conclusions: The framework is anticipated to give hospitals the knowledge they need to address issues and difficulties in an organized manner, serving as both a benchmark for evaluating current capabilities and a guide for future development. However, the frameworks should be validated in other countries with the involvement of patients for more generalization.

Keywords

business intelligence smart hospital assessment framework smart levels

Introduction

There is currently a lot of demand for healthcare systems around the world to lower costs, increase and improve service efficiency, and increase access while maintaining or even raising the quality of healthcare services offered.¹ This demand could be satisfied with the analysis of the hospital data to understand its operational status quo and to make fact-based decisions. Hence, the collection, storage and processing of hospital’ s data are of importance. Hospitals’ data is huge and varied in format. Thus, the acceptance and successful application of health information systems and technology are essential to the success of the healthcare sector which is a significant social and economic component of contemporary societies.^2–4 The health information system and IT applications in hospitals are claimed to make hospitals smarter. Business intelligence (BI) as a kind of information system has grown in importance as a field of study for both researchers and practitioners due to the critical role that data plays in aiding organizational improvement.⁵ By using data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to inform decisions and actions, BI is defined by Davenport and Harris⁶ as a collection of tools to collect, store and process data to provide a precise understanding of corporate performance. Hence, BI could be used to create smartness for hospitals. The last 10 years have seen tremendous advancements in the field of business intelligence.⁷ Chen, Chiang and Storey⁵ have noted that BI holds promise for the healthcare industry. It is well known that BI may help healthcare firms with better patient care and results, efficient use of human resources, increased process efficiency, and cost savings.⁸ BI can assist healthcare organizations to achieve better performance and advance the continuum from intuitive to precision medicine.^9,10 By giving hospitals strategies for developing tailored data, enhancing medical safety and quality control, optimizing medical procedures, and enhancing smart hospital administration, BI applications can lead to a higher level of smartness in hospitals. These enhancements aid in achieving goals for the quality of health services.¹¹

A definition of smartness concerning IT-enabled technologies used in healthcare systems is necessary for a meaningful discussion about smart hospitals. From “not smart at all” to “unscripted/partially scripted invention”, there are five categories of intelligence. According to Mettler and Vimarlund,¹² the complexity of the clinical component and the multiplicity of stakeholders make it difficult to integrate BI systems to create smarter hospitals. In truth, there is no comprehensive framework for the development of BI that is available to medical institutions.¹³ The problems with the current BI frameworks are emphasized, especially when it comes to how well they work in a complicated field like healthcare. The current BI frameworks are inadequate to meet the needs of healthcare industry.¹⁴ Thus, domain-specific BI frameworks for assessing smartness are important, especially for evaluating the smartness of hospitals. To facilitate processes of expansion and continual improvement, it is crucial to monitor and evaluate the smart levels of BI-based solutions in hospitals.¹⁰

The world is currently concentrating on the 4.0 Industrial Revolution and digital transformation. Clinicians and the public in Vietnam are generally optimistic about the potential of digital technology.¹⁵ The Vietnam Ministry of Health has released new digital health policies and guidelines in conjunction with the development of national health databases like electronic health records (EHRs).¹⁶ However, most hospitals in Vietnam have only partially embraced telemedicine, eHealth, and electronic medical records.¹⁷ Moreover, there is a lack of a framework for evaluating the smartness of hospitals, which is necessary for ensuring the success of digital transformation of the healthcare industry in Vietnam. Therefore, more research is needed to guide the implementation of smart hospital models in Vietnam.

BI application in healthcare industry is limited, despite its promising application and benefits.¹⁸ Due to the complexity of the healthcare industry, domain-specific BI frameworks for hospital smartness evaluation are still in shortage. There hasn’t been much research done on how to effectively apply a BI solution in this field.⁸ More precisely, there are no complete models available to guide practitioners in setting priorities for creating effective BI solutions.^5,10 Hospitals find it challenging to assess their existing problems and guide future development toward a smarter organization since there is a lack of a BI-based assessment framework, particularly for emerging markets like Vietnam. Therefore, the goal of this study is to create a BI-based framework to evaluate hospitals’ levels of smartness and to make managerial recommendations for hospital managers to increase their smartness based on the framework’s validity regarding multi-case assessment.

The remaining sections of the paper are structured as follows: section 2 presents the theoretical foundation, section 3 outlines the research method, section 4 summarizes the findings from the BI-based framework’s multi-case, section 5 suggests the management implications, and section 6 concludes the study.

Literature review

Smart levels of hospitals

According to Gomez-Sacristan et al.,¹⁹ a smart hospital is a technologically advanced, highly linked hospital that operates on tablets. Systems that employ IoT and mobile internet to connect people, documents, and organizations involved in the healthcare industry as well as access information, manage, and intelligently respond to patient requirements are examples of smart hospitals.²⁰ The term “smart hospital” is used in this paper to refer to a model of comprehensive healthcare that can best meet the needs of healthcare organizations, businesses, and patients,²¹ using technology to access information, and connect people,²² materials, and related organizations.^20,23 This model of healthcare can also help to increase clinical effectiveness, lower costs, share responsibility, and improve safety.¹⁹ Hospital ‘smartness’ is different from ‘readiness’ and is more important in a fast-changing environment. While ‘smartness’ refers to the capability of hospitals to utilize advanced technologies for better decision-making and efficiency, ‘readiness’ typically concerns the preparedness of an organization to adopt and implement new technologies.

A system’s or an organization’s level of smartness can be divided into five categories, according to Alter²⁴: Not smart at all; Scripted execution; Formulaic adaptation; Creative adaptation; Unscripted or partially scripted invention. In this research, an assessment system for smart hospitals with 5 smart levels by using the methodology of Alter²⁴ is established.

Business intelligence (BI) as a cornerstone for smart hospitals

The factors of business intelligence

BI (business intelligence) comprises a vast number of advanced technologies including the Internet of Things (IoT), cloud, data analytics, big data, data mining, Artificial Intelligence (AI), etc. In this research context, BI is a part of the framework used to assess the smartness of organizations in adopting new technology in general. A set of BI performance criteria was developed by Jahantigh, Habibi and Sarafrazi²⁵ and includes factors like culture, technology, knowledge, responsibility, organizational structure, business environment, practices, and resistance to change, as well as factors like power and policy. Data analysis, data storage, business performance management, and user interface are the four basic purposes of BI.⁵ In it, data storage and analysis serve to illustrate the technical aspects of BI. BI factors should be categorized into the following four classes, which can be thought of as the foundation for creating a smart hospital, according to a recent study by Gastaldi et al.¹⁰: (1) Functional factors; (2) Technological elements; (3) Diffusive factors; and (4) Organizational factors. However, in hospitals, data and processes are very important to healthcare quality, and hospital management, especially during the early phases of digital transformation. According to WHO,²⁶ healthcare quality is the extent to which health services for both individuals and populations raise the probability of desired health outcomes. There are many different ways to define quality health care, but it is increasingly accepted that good health care should be: People-centered (that is, care that takes into account each patient’s preferences, needs, and values); Safe (that is, care that does not harm the people for whom it is intended); and Effective (that is, provide evidence-based healthcare services to those who need them). The data and processes play an important role in ensuring healthcare quality since the data analysis combined from many sources at a hospital along with data from other hospitals and public knowledge bases will provide important information about disease mechanisms to improve diagnosis, prognosis, and intervention choices.²⁷ Thus, data and processes should be reliable, precise, and updated with the development of the hospital and contribute to the smartness of the hospital. The data and processes are proposed as one of the key factors in measuring the success of the information system^28–30 but are not fully mentioned in the BI framework of Gastaldi et al.¹⁰. Hence, the assessment framework proposed in this study will be based on Gastaldi et al.¹⁰ with the addition of data process based on Ariyachandra and Watson.³¹ Thus, the fifth class consists of (5) Data and processes. Table 1 summarizes the BI factors conducted by recent studies.

Table 1.

Factors of business intelligence.

Classification	Factors	Ariyachandra and Watson³¹	Olszak and Ziemba³²	Yeoh and Koronios³³	Jahantigh et al.²⁵	Gastaldi et al.¹⁰
Function	Set goals for hospital periodical performance			x	x	x
	Measure hospital performance based on data, periodical reports	x		x	x	x
	Analyze the gap between actual and target performance based on overall hospital data					x
	Make decisions based on general data and hospital reports			x		x
Technology	Smart structure	x	x	x	x	x
	Reports				x	x
	User interface	x	x	x		x
	User profile		x	x		x
	Technology integration		x	x	x	x
	The standard for smart solution tools	x				x
	Updating data		x	x	x	x
Diffusion	The hospital’s IT system supports people to access and use the hospital’s IT system			x	x	x
Diffusion	The hospital’s IT system supports management functions and work processes	x		x	x	x
Organization	Smart hospital development strategy			x	x	x
	Funds for smart development			x		x
	Organizational support, internal resources for smart development	x		x	x	x
	Competency of IT staff				x	x
	User capacity				x	x
	IT capacity improvement	x				x
	Coordinate with suppliers of smart tools and solutions		x	x	x	x
Data and process	Grasp the operation process and medical examination and treatment process on the basis of new technology	x			x	x
	Analyze the effectiveness of the process	x		x		x
	Process improvement	x				x
	Data integration of smart hospital applications	x	x	x	x	x
	Data mining	x	x	x	x	x
	Data quality	x	x	x	x	x
	Integration of workflows and data	x	x		x	x

The relationship between factors of business intelligence and smart hospital

Today, a variety of medical gadgets,³⁴ biomedical tools, and healthcare resources, including mobile devices with biometric sensors, are stored in numerous information systems in hospitals.³⁵ Dash et al.³⁵ claim that analyzing those data can reveal information on procedures, engineering, medicine, and other healthcare innovation areas. To optimize service quality while lowering costs, smart hospitals require electronic healthcare applications.³⁶ To increase their productivity and smartness, healthcare firms are attempting to adopt techniques like BI.³⁷ In fact, for effective hospital management, BI system improvement in hospitals should be a part of the plan of all departments. To track medical performance and enable quick and precise decision-making, it is crucial to develop BI for all levels of hospital management.¹⁰ The development of smart hospitals needs data analysis, data visualization, and interface design based on user requirements. According to several studies,^38,39 efficient BI applications have resulted in cost savings and greater productivity. In the case of BI application in healthcare, if successfully implemented, BI gives hospitals a practical method for growing and continuously enhancing their smartness.¹⁰ This will also enable them to respond to changes rapidly, modify their plans, and, most importantly, build more advanced and smarter hospitals. As such, previous studies have shown a positive relationship between the factors of BI and the development of smart hospitals. In other words, building a smart hospital requires solid foundations of BI. Therefore, in this study, the assessment of the smart levels for hospitals will be made based on the factors of BI.

Previous assessment frameworks

To compare our framework to the others of previous studies, the recent frameworks for assessing IT healthcare systems in hospitals are summarized. Table 2 lists some previous frameworks for assessing the effectiveness of hospitals based on diversified criteria related to IT application.

Table 2.

Previous assessment frameworks for the healthcare industry.

Author	Framework	Assessment criteria	Contribution	Comments
Martínez-García et al.²⁷	FAIR workflow for health project	Findability, accessibility, interoperability, and reusability of digital data	Analyze the influence of FAIR principles on health research management outcomes	- Only focus on developing a FAIR workflow to improve the time and cost of health research execution- Not consider the assessment of smart levels of hospitals
Queralt-Rosinach et al.⁴⁰	The ontological data model for the patient dataset	Findability, accessibility, interoperability, and reusability of digital data	Apply the FAIR principles in the hospital to make patient data more findable, accessible, interoperable, and reusable	- Only focus on developing an ontological data model for FAIR data analytics in the hospital- Not consider the assessment of smart levels of hospitals
Yang, Hsu & Wu⁴¹	A resource-constraints model of a smart healthcare management system	Four factors: Healthcare technology, optimal processes, financial incentives, and healthcare policy	Provide a four-factor approach to evaluate the performance of smart hospitals	- Only focus on developing a hybrid multiple-criteria decision portfolio with the resource constraints model of a smart healthcare management system for public medical centers- Not consider the assessment of smart levels of hospitals
Carvalho et al.³	A maturity framework for hospital information systems	Seven influencing factors: People, electronic medical records, systems and IT infrastructure, strategy, information security, and data analysis	Provide a maturity framework for hospital information system	- Only focus on six stages of the hospital information system growth and maturity progression- Not consider the assessment of smart levels of hospitals
Gastaldi et al.¹⁰	A framework for measuring the maturity of BI in healthcare	Four dimensions: Function, technology, diffusion, and organization	Provide a comprehensive model for the healthcare industry to address the key issues related to BI solutions	- Only focus on measuring the maturity of BI in healthcare based on four maturity levels- Not consider the assessment of smart levels of hospitals
Balasubramanian et al.⁴²	A readiness assessment framework for blockchain adoption in healthcare	Four dimensions of individual stakeholder readiness: Motivational readiness, engagement readiness, technology readiness, and structural readiness	Provide a readiness assessment framework with different underlying factors for key stakeholders in healthcare	- Only focus on assessing the readiness of blockchain adoption in healthcare- Not consider the assessment of smart levels of hospitals
This study	A BI-based framework for assessing the smart levels of hospitals	Four BI dimensions (Function, technology, diffusion, and organization) are assessed on 5 smart levels: (Level 1: Not smart; Level 2: Pattern-based operation; Level 3: Adapt to the pattern; Level 4: Creative adaptation; Level 5: Un-patterned or partially patterned inventions)	Provide a BI-based framework for assessing the smart hospital and its application to improve the healthcare efficiency	- Focus on assessing BI dimensions of hospitals- Take into account the assessment of smart levels of hospitals

Based on Table 2, recent frameworks for assessing healthcare systems have not focused on measuring the smart levels of hospitals, instead, they mostly assess the readiness or maturity of IT systems adoption. To the best of our knowledge, there is a shortage of frameworks for assessing smart levels of hospitals. As mentioned in the above review, the BI dimensions are the cornerstone of developing a smart hospital. Thus, it is necessary to develop a framework based on BI dimensions to assess the smart levels of hospitals. The smart framework plays an important role in suggesting recommendations for enhancing the smart level and improving the efficiency of hospitals in the era of IT advances.

Besides, this study’s BI-based framework also follows the FAIR principles. Specifically, the five factors of BI (Technology, Data and Process, Function, Diffusion, and Organization) are applied to assess the hospital’s smartness under FAIR principles. The two factors “Technology” and “Data and Process” are the critical measurements to assess hospitals’ smart levels, ensuring that they facilitate digital data to be findable, accessible, interoperable, and reusable. In the meantime, the other BI factors (Function, Diffusion, and Organization) are used to assess the smartness of hospitals applying FAIR data to improve their strategic and operational performance.

Research method

This research is based on Design Science methodology,⁴³ which focuses on the development of artifacts with the explicit intention of improving the functional performance of the artifact. The main purpose of this research is to develop a BI-based framework for assessing smart levels of hospitals. The method for this research is a combination of literature review, pilot test, and multi-case study. The research process is presented in Figure 1.

Figure 1.

Research process description.

In accordance with the research objective, the research method is composed of three phases. For the first phase, the systematic literature review is conducted to find out previous frameworks with the measurement scale for evaluating the smart levels in general, and for the smart hospital. These are analyzed to identify the gaps in the assessment scale for evaluating smart levels of hospitals. Then, an assessment scale for evaluating the smart levels of hospitals is proposed. A systematic literature review is a strategy that gathers prior studies that address the same subjects as the keywords. The major features and critical points of these studies are then identified through analysis to gain a general understanding of the state-of-the-art of subjects and research gaps. Based on this analysis of their contents, the ideas for a more thorough framework and assessment scale are derived. The process of the systematic literature review is based on PICOC^44,45 and presented in Figure 2.

Figure 2.

Literature review process.

PICOC refers to Population, Intervention, Comparison, Outcome, and Context. PICOC description will help to define the scope of the search process. From the PICOC, the keywords for search are identified and the search query string is composed with Boolean connectors (AND, OR). Exclusion and inclusion criteria for article selection are also stated. Then, the search query process is started with the search query string and the criteria to select suitable articles. The selected articles are screened primarily for their contents associated with the objectives of this study; otherwise, they are deleted. The remaining articles are read and deeply analyzed to find out gaps to propose new study issues. In this study, the main topic is the smart hospital measurement scale.

The search query string is composed of the keywords such as (“Hospital management” OR “smart hospital management”) AND (“IT technologies” OR “information system” OR “business intelligence” OR “hospital information system”) AND (“Information system maturity” OR “information system success” OR “smartness measurement scale”), and related keywords. The search query process is limited to English articles having at least two keywords in the title, abstract, or keywords. The chosen period of articles is from 2005 to 2023 because the Forth Industrial Revolution is believed to start in 2005. The remaining articles are further analyzed for their objectives, theoretical frameworks, assessment scales, and applicability. The selected database is the SCOPUS/ISI, which is suitable for this topic because it is the integration of management and information field and requires a broad database with multiple disciplinary articles.

Then, the evaluation of the proposed assessment scale is conducted by a pilot test. To do so, a workshop is organized with related experts (hospital managers, IT department heads of the hospitals, and IT providers) who are decisive people to establish the IT system for the smart hospital implementation. In the workshop, these experts state different perspectives on the smart hospital (management, operation, and technology) and judge the suitability of the proposed scale. Based on the pilot test, a revision is made to get the final version of the assessment scale. Finally, the final assessment scale is tested for its applicability with a multi-case study, which is conducted in several hospitals. The details of these phases are presented in the next section.

Research results

BI-based framework for assessing the smart levels of hospitals

Based on Alter²⁴ and Gastaldi et al.¹⁰ (Table 1), a framework for assessing the smart levels of hospitals is proposed. The assessment framework, which consists of five groups of BI factors divided into 27 elements, is built in accordance with five levels of smartness. To evaluate the proposed assessment scale, a workshop was organized to deeply discuss the structure, contents, and wordings of the scale. The criteria to select the participants included: (i) hospital managers having more than 10-year experience at hospitals implementing IT systems to become smart, (ii) IT providers with software provision of any kind for at least three different hospitals, and (iii) IT department heads from the hospitals having the selected managers. There were seven hospital managers, 8 IT providers, and 5 IT division heads joining the workshop. Two weeks before the workshop date, the participants received a letter presenting the assessment scale and the objective of the workshop. Some wording was suggested for adjustment. Finally, all participants came to the final version of the BI-based framework with the assessment scale (Appendix 1) and agreed that the scale was clearly structured, understandable, and feasible to be used for assessing the smart levels of hospitals.

Prior research in the healthcare industry has established the significance of the Hospital Information System (HIS) in overseeing organizational procedures to accomplish healthcare goals, specifically in relation to person-enabled healthcare, predictive analytics, workforce and governance, and interoperability, all of which have a significant influence on the health and wellness of patients and populations. However, different HIS structures and technology result in diverse approaches to achieving these goals.⁴⁶ In order to evaluate the maturity levels and provide guidance for developing HIS to raise them, the Healthcare Information and Management Systems Society (HIMSS) framework has been designed.^47,48 Our proposed framework is intended to complement HIMSS by offering an additional method for evaluating the healthcare systems’ maturity using smartness levels. The framework centers on the hospitals’ capacity to utilize IT to automate their procedures. As a result, several components of our proposed framework are strongly related to HIMSS elements: Patient-enabled health is demonstrated by factors 2.1, 3.1, 3.2, 4.3, 4.5, and 4.6; predictive analytics is expressed by factors 1.1, 1.2, 1.3, 1.4, 2.1, 2.5, 2.7, 3.2, 4.6, 5.4, 5.5, 5.6, and 5.7; governance and workforce are covered by factors 3.1, 4.4, 4.5, and 4.6; and interoperability is related to factors 2.1, 2.5, 3.2, 5.4, and 5.7. In addition to the HIMSS components, our proposed framework also incorporates organizational aspects (such as budget and smart strategy) and technical variables (such as establishing technological standards, integration, report formats, and automation) to sustain the IT systems.

Applicability of the BI-based assessment framework for smart hospitals

To test the applicability of the framework, the multiple-case study with the presence of hospital directors and IT department heads was proceeded. The selected hospitals were the diversified cases having implemented some IT systems in clinical treatment and operation management such as Hospital Information System (HIS), Picture Archiving and Communication System (PACS), Radiology Information System (RIS), Laboratory Information System (LIS), Health Level Seven (HL7) in Vietnam. A questionnaire involving two parts was prepared. The first part was to introduce the objectives of the interview, and the second part included the assessment scale for evaluating smart levels of hospitals. The respondents were required to select the smart level for each statement of the assessment scale, which reflects the current situation of the hospitals. At the beginning of the interview, the objective of the study and the structure of the assessment scale were presented. The directors and the IT department heads could share current information on their smart software implementation so that the researchers could understand their status quo, then, the directors and heads of IT departments of the hospitals would fill out the questionnaires separately. To invite the hospitals to join the interviews, telephone calls were conducted to the directors and IT department heads of these hospitals to make appointments. Due to the high workload of the directors, there are eight hospitals in Ho Chi Minh City (Vietnam) joined the interviews. The questionnaires were sent to them for their preparation. The structure of the interview sample is presented in Table 3.

Table 3.

Structure of the interview sample.

Grade	Grade 1		Grade 2
Ownership	Public	Private/Foreign investment	Public	Private/Foreign investment
Specialized hospital	Hospital 1Hospital 2	0
General hospital	Hospital 3Hospital 4Hospital 5Hospital 6	Hospital 7	Hospital 8	0
Sum	6	1	1	0

As shown in Table 3, all the public hospitals of grade 1 are of large size, well known for high-quality treatment. The public hospital of grade 2 is of small size as compared with grade 1 hospitals, but it has high-quality treatment and patient services. The private/foreign investment hospital is one of the foreign investment hospitals with large size, which has achieved good treatment quality and patient service. Table 4 presents the interviewees in the sample.

Table 4.

List of the interviewees.

Hospital code	Interviewees
Hospital code	Director	Vice director	Head of IT department	Sum
Hospital 1		1	1	2
Hospital 2		1	1	2
Hospital 3	1		1	2
Hospital 4	1		1	2
Hospital 5		1	1	2
Hospital 6		1	1	2
Hospital 7	1		1	2
Hospital 8	1		1	2
Sum	4	4	8

The data received from the directors and IT department heads at the ends of the interviews were collected to evaluate the applicability of the framework to each hospital. The average value of each component of the assessment scale and the final smartness level of each hospital are computed and presented in Table 5.

Table 5.

Results of smart levels of hospitals in the sample.

Hospitals	Dimensions					The average value for each hospital	Smart level
Hospitals	Function	Technology	Diffusion	Organization	Data and process	The average value for each hospital	Smart level
Hospital 1 (public grade 1)	2.5	2.6	2.5	3.7	3.9	3.0	2
Hospital 2 (public grade 1)	3.6	3.6	4.0	4.1	4.3	3.9	3
The average value for public specialized hospitals	3.1	3.1	3.3	3.9	4.1	3.5	3
Hospital 3 (public grade 1)	3.5	3.6	2.0	3.0	3.3	3.1	2
Hospital 4 (public grade 1)	4.5	4.9	4.0	4.7	4.7	4.6	4
Hospital 5 (public grade 1)	2.0	5.0	2.5	2.4	5.0	3.4	2
Hospital 6 (public grade 1)	4.0	3.0	3.5	2.9	3.0	3.3	2
The average value for public general hospitals	3.5	4.1	3.0	3.3	4.0	3.6	3
Hospital 7 (Foreign investment grade 1)	2.5	4.1	5.0	4.3	4.6	4.1	2
Hospital 8 (public grade 2)	5.0	4.6	5.0	4.7	4.7	4.8	4
The average value for each dimension	3.5	3.9	3.6	3.7	4.2	3.8	3

As shown in Table 5, overall, the data-process dimension (4.2) is at the greatest level for the sample’s hospitals, while the technology dimension (3.9) is in second place. The development of Health Level 7 (HL7) in recent years for all hospitals and the current implementation of the Electronic Medical Record (EMR) at Grade 1 Hospitals may all be attributed to the Ministry of Health’s demands for the development of HIS. The general hospitals develop IT system technologies more than the specialized hospitals because they have more activities, processes, and complicated management with high workloads (4.1 vs. 3.1). As a result, the general hospitals have a greater level of technological dimension than the specialized ones. However, the technological dimension’s overall score (only 3.9) falls short of reaching a level higher than 4. This is because IT solution providers’ IT technology varies, and it takes longer for hospitals to deploy these IT systems. HIS, PACS, RIS, LIS, and EMR are among the IT applications used in hospitals where most IT solution providers in the city are not very proficient. They employ many technologies, and some of them implement the findings of their research and clinical trials in hospitals.

At the levels of function dimension (3.5) and diffusion dimension (3.6), these average values are the lowest. This demonstrates that management does not fully support hospital IT systems and that neither the management nor the medical staff makes full use of them. The function dimension is similarly at the lowest level for the foreign-invested hospital (hospital 7, 2.5), despite its IT system being practically completely completed. This finding demonstrates that IT technologies are not necessary for administrative tasks to be supported by hospital managers. At the function dimension, general hospitals score higher than specialized hospitals (3.5 vs. 3.1). This can be explained by the fact that general hospitals require more sophisticated, and high workload management activities than specialized hospitals. Because of this, the managers of these hospitals require more IT system support than the managers of specialized hospitals, and they also need more demanding IT systems to develop the function dimension. The specialized hospitals, therefore, have a higher degree of the data-process dimension (4.1 vs. 4.0). In a similar vein, general hospitals with more complicated operational procedures find it difficult to support all their employees and patients in the utilization of IT systems, and their operational procedures are not fully developed to support all hospital operations. Because of this, general hospitals’ diffusion level is lower than that of specialized hospitals (3.0 vs. 3.3). The specialized ones benefit from advantageous conditions for investing in IT systems and for training IT employees, users, and system administrators. They, therefore, have a better level of organization (3.9) than the average one (3.7). Besides, based on the analysis of the 5 factors of the assessment scale, the organization and diffusion factors are considered very important (high distance) to distinguish between the smart and non-smart groups. Therefore, improving these two factors can be helpful in improving the smartness of hospitals.

As shown in Table 5, Hospital 4 (public grade 1) and Hospital 8 (public grade 2) are the two hospitals with the greatest levels across all dimensions. It becomes clear from the interviews with the managers of these hospitals that they think more strategically than the other managers and focus on how to use IT systems to give them data and information for management. They are eager to pick up new skills and advance the capacity of IT workers and systems. This is proved by high levels at the function dimension (4.5 and 5.0) and results in high levels at the remaining dimensions. These findings support the notion that hospital administrators’ managerial outlooks play a key role in the growth of smartness in healthcare systems.

As shown in Figure 3, hospitals three and five need to improve only the diffusion and the organization dimension respectively to level 3, then, these hospitals will achieve the smart level at 3. Moreover, the improvement of these factors should be balanced between two groups: (1) technological group (technology, function, and data and process), and (2) social group (organization, and diffusion). Most of the hospitals in the sample focus too much on the technological group. So, they should rebalance the investment to focus more on the social group to improve the smart level in practice.⁴⁹ By using the framework with the assessment scale, hospitals can recognize their status and easily build a roadmap to improve their smart levels. These results prove the applicability of the framework in measuring hospitals’ smart levels. After discussing with hospitals’ managers in the sample, the framework was realized as a helpful tool to improve their digitalization process.

Figure 3.

Radar chart of smart levels of hospitals.

Discussions

The BI-based framework with an assessment scale to evaluate the smart levels of hospitals includes various cutting-edge features. According to the social-technical system theory, the development of smart hospitals should prioritize both the social (people utilizing and impacted by the system) and technical aspects. Our framework assesses how smart hospitals are based on those aspects and a five-level scale. Most previous studies have concentrated on IT applications (IoT, AI, telemedicine, etc.), whereas the assessment scale developed for this study also considers function, diffusion, and organization in addition to technology, data, and process. This makes it easier for the application of the information system (IS) to be accepted by the stakeholders and to provide organizational benefits.³⁰ The quality of the information and procedures in hospitals is not a major problem for the previous research, even though they are focused on ITs.^29,30 Our framework, which includes a broad range of areas of IS success,²⁹ including the technology, the smart level, staff support, patient contacts, and efficacy, is expanded to incorporate the quality of information and processes to fill this gap. The assessment scale for smart levels is dynamic rather than static. As a result, the scale might be able to adapt to technological advancements. Additionally, the scale does not reflect the level of technology utilized to create the information system; rather, it assesses how smart the hospital is, which is a result of its internal information systems. Consequently, the smart scale works well and effectively over time.

The directors must decide which technologies to use, establish the ultimate goals of digitalization, and create a roadmap for its completion to develop and implement the digitalization plan.^50,51 However, most hospital directors are medical practitioners who lack the IT expertise and experience needed to set the goals for digitalization and create a plan for it. The findings indicate that function is the most crucial factor for determining the smart level of hospitals. For this scarcity, hospitals 1, 5, and seven are usual. A typical example is Hospital 7, which has high levels of technology, data, and processes but low levels of function. In terms of change management, hospital directors could affect how other staff members act and feel about utilizing the system.^52,53 However, only two hospital directors in the sample (Hospitals four and 8) have a strong desire to take advantage of the system and apply it to all aspects of the hospitals’ operations; as a result, their hospitals are smarter than the others. Hospitals 1 and 5 perform poorly in terms of organization, function, and diffusion but excel in terms of technology, data, and processes. Prior to implementing IT systems, these hospitals should strengthen their capacity for organization and function. From the above analysis, hospitals should identify the components with low levels that are making bottlenecks and enhance them for a higher status to increase the smart level. Recommendations for key stakeholders to build smart hospitals are presented in Table 6.

Table 6.

Recommendations for key stakeholders to build smart hospital.

Position	Actions
Directors	− Advancing in the management thinking and capabilities, particularly, the strategic management, project management, change management and culture of change− Taking active roles in digitalization process from the strategy to road map establishment and implementation project− improving the staff’s awareness of the information technology applications, smart hospital in order to develop the organizational investment and staff capability improvement to accept and exploit the information systems in their activities− Organizing training courses to instruct the patients to improve the patients’ interactions, centricity, and satisfaction via the information systems of the hospital− measuring the smartness level of the hospital and setting the road map to develop this level to a higher level− Building a clear strategy and reserve the budget to complete the smart hospital development
Medical and and aministrative staff	− Learning the information systems of the hospital and smart hospitals in general to improve the awareness of the systems and acceptance to use them− improving the IT skills to use and exploit the information systems of the hospital to enhance the working capability− Updating the IT skills to adapt to the development of the smart level of the hospital− Setting requirements for the information system developers and providers to design the system development
IT staff	− Learning to improve information technology knowledge and skills− Learning new information technologies to upgrade hospital information systems− Working closely with the hospital directors and IT providers during the whole digitalization process to make the digitalization process efficient and effective− Supporting the hospital staff and patients in using and exploiting the hospital information systems− Designing and implementing training courses for the staff and patients in the awareness of the systems and using them
Patients	− Joining actively the training and/or instructing courses provided by the hospital to learn how to interact with the hospitals using the information systems of the hospital− Using and exploiting actively the information systems of the hospital to serve her/himself in the best way− giving requirements for the hospital to develop the information systems to satisfy more her/himself

However, there are some implementation barriers for the above solutions, such as budget constraints, staff resistance, and technology integration issues. Therefore, the hospital managers should have an incremental strategy for improving the smartness of their hospitals. For example, based on BI-framework evaluation, the hospital should develop a roadmap for implementing its smartness. The roadmap should include short-term and long-term plan for improving their IT infrastructure, training users (internal and external), and integrating various functions and features of BI systems. Management supports and collaboration between IT staff and business functions should also be important elements for ensuring the success of the implementation plan.

Conclusions

The development of smart hospitals is urgently required to boost patient-centricity, medical quality, and hospital administration, as well as solve current problems of cost increase, waiting time for patients, and heavy workload for hospitals. Hospitals can implement different models to become smart hospitals. However, many hospitals place a greater emphasis on information technology applications than they do on hospital activity management. As a result, the development of the hospital information system has not adequately supported the management activities, and the hospital staff has not fully accepted and exploited it. Additionally, because there is a shortage of an assessment framework for evaluating smart levels, hospitals are unable to create a roadmap for developing their smartness effectively and systematically. The goal of this study is, therefore, to develop a BI-based framework for assessing the smart levels of hospitals to support them for improvement.

By systematic literature review, this study has found the gap in constructing the assessment scale for the evaluation of smart hospitals. Then, a comprehensive BI-based framework that includes five aspects of function, technology, diffusion, organization, and data and processes with 5 smart levels was proposed. The scale is assured of its validity and applicability by the pilot test and multiple-case studies. The results of a multiple-case study approve the applicability of the framework since it can explain the status quo of the hospitals. From then, some recommendations for hospitals are suggested to improve their smart levels.

However, there are some limitations of this research, such as the small sample size, a shortage of interviews with hospital suppliers and IT providers, etc. The sample collected in Vietnam has limited the possibility to generalize the results. The technological aspect of the framework may need to be revised and updated if there is a new technological revolution in the future. Further research can focus on the following topics: validating the assessment scale with a larger sample size; or studying the roles of hospital suppliers and IT providers to improve their smart levels. The proposed framework may be suitable for application in Vietnam and some countries with similar conditions. The further validation of the scale in other countries could also be a future research direction.

Footnotes

Acknowledgements

This research is funded by Vietnam National University - Ho Chi Minh City (VNU-HCM) under grant number DS 2022-20-08. We would like to thank Ho Chi Minh City University of Technology (HCMUT), VNU-HCM, and hospitals in the sample for the support of their time and facilities for this study.

ORCID iD

Quoc Trung Pham

Ethical approval

This study received ethical approval from the School of Medicine – Vietnam National University Ho Chi Minh City IRB (approval 8/QD-IRB-VN01.017) on Month 09, 2022.

Authors’ contributions

Conceptualization, M.C.T., P.L.L., Q.T.P. and V.T.N.; methodology, Q.T.P. and V.T.N.; validation, P.L.L., Q.T.P. and V.T.N.; formal analysis, M.C.T., P.L.L., Q.T.P. and V.T.N.; investigation, M.C.T., P.L.L., and V.T.N.; writing—original draft preparation, P.L.L. and V.T.N.; writing—review and editing, P.L.L., Q.T.P. and V.T.N.; supervision, M.C.T. All authors have read and agreed to the published version of the manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is funded by VNU-HCM under grant number DS 2022-20-08

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author, upon reasonable request. *

Appendix

Appendix 1.

BI-based assessment framework for smart hospitals.

No.	Smart levels	Level 1: Not smart at all	Level 2: Scripted execution	Level 3: Formulaic adaptation	Level 4: Creative adaptation	Level 5: Unscripted or partially scripted invention
1.0 functions:
1.1	Goal definition	Hospital defines the goal for annual performance of more than 50% of activities based on the hospital-wide annual data	Hospital defines the goal for quarterly performance of more than 50% of activities based on the quarterly reports of each faculty/ department	Hospital defines the goal for monthly performance of more than 50% of activities based on the monthly reports of each faculty/ department	Hospital defines the goal for weekly performance of more than 50% of activities based on the weekly/daily reports of each faculty/ department	Hospital defines the goal and all activities the trend for daily performance of faculty/department based on the daily data of each process of each faculty/ department
1.2	Measurements	Hospital measures the annual performance by hand (not automatic) of some main activities based on hospital-wide data and annual report, such as: reception rate, medication quality, financial report, equipment usage, etc.	Hospital measures the quarterly performance by hand (not automatic) of some main activities based on quarterly data and report of faculty/ department, such as: reception rate, medication quality, financial report, equipment usage, etc.	Hospital measures the monthly performance automatically of some main activities based on monthly data and report of faculty/ department, such as: reception rate, service quality, customer satisfaction, equipment usage, human resources, balance sheet, investment and development, etc.	Hospital measures the weekly performance automatically of some main activities based on weekly/ daily data and report of each process of faculty/ department, such as: reception rate, service quality, customer satisfaction, equipment usage, human resources, balance sheet, investment and development, etc.	Hospital measures the daily performance and realizes the trends automatically/online of all activities based on daily data and report of each process of faculty/ department, such as: reception rate, service quality, customer, equipment usage, human resources, balance sheet, investment and development, etc.
1.3	Gap analysis	Every year, hospital by hand (not automatic) the distance between the real result and goal based on common data of the hospital and annual report on gap analysis	Every quarter, hospital analysis by hand (not automatic) the distance between the real result and goal based on data of each/ department and previous reports on gap analysis	Every month, hospital automatically the distance between the real result and goal based on data of each faculty/ department and previous month reports on gap analysis, and visualize the gap analysis results	Every week, hospital analysis automatically/ online the distance between the real result and goal based on data of each process of faculty/ department and previous weekly reports on gap analysis, and visualize the gap results	Hospital automatically/ online the distance between the real result and goal based on real-time data of each process of faculty/ department, and visualize the gap analysis results in an easy-to-understand form, and predict the trend on gap
1.4	Decision making	Hospital makes decision (without the support of IT) based on common data of the hospital and yearly updated reports	Hospital makes decision (without the support of IT) based on data of faculty/ department and quarterly-updated reports	Hospital makes decision based on OLAP report, data of each process/ event of each faculty/ department and monthly-updated reports	Hospital makes decision weekly based on OLAP report, data of each process/ event of each faculty/ department and daily/weekly-updated reports	Hospital makes decision based on real-time data of each process/ event of each faculty/ department and daily-updated reports/ trends
2.0 Technology:
2.1	Smart architecture	The hospital has not yet fully developed working procedures to be able to computerize. Data collection and analysis are done manually	The hospital has complete internal management processes that can be computerized. The data collection and analysis are done manually	The hospital has full internal management processes, computerized, some processes of interaction with the external partners are computerized, some key processes are integrated online	The hospital has fully established internal management processes, external interactions, and online integration. Interaction with patients is done online	The hospital has fully established internal management processes and integrated with the external partners online and in real time. Interaction with patients is done online in real-time
2.2	Reports	Static reports (tables, tables, etc.) are made by hand and printed on paper	Static reports (including basic calculations, tables, tables, graphs, etc.) are done manually and are transferred in digital form	Static reports (including basic calculations, statistics, documents, tables, graphs, etc.), are performed automatically, always ready to be transferred and retrieved automatically	Dynamic reports (for different data sets, the report will change accordingly) with basic calculations, statistical analysis is done automatically and always ready to be transferred and retrieved automatically	Dynamic reports, structure, report content can be changed according to user needs automatically and be always ready to be transferred and retrieved automatically
2.3	User graphic interface	No interface	Have an interface at the local level	Hospital sets up website which can be accessed from personal computers	Hospital sets up website which can be accessed from personal computers or smartphones	Hospital sets up website which can be accessed from personal computers or smartphones
2.4	User profile	Hospital does not record user profiles in digital form	Hospital records user profiles as files in the local computers	Hospital records user profiles in digital form in centralized database	Hospital records user records in digital form and has user classifications (according to users, purposes of use, commonly used functions, etc.)	Hospital builds IT tools to identify and support external users depending on the users, purposes of use, commonly used functions, etc. to increase convenience and reduce system penetration time
2.5	Technology integration	Hospital has not yet applied smart solution tools (technology 4.0) to perform work and interact with the outsiders. All are done by hand	Hospital has not yet applied smart solution tools (technology 4.0) to perform work and interact with the outsiders. All are done manually. Data from internal medical devices are stored internally and cannot be transferred. No data received from external medical devices	The hospital adopts smart solutions tools (technology 4.0 such as IoT, cloud, etc.) to perform part of the internal work and some interaction with the outsiders Medical data from internal medical devices is integrated, circulated internally and transmitted to external partners Data from external medical devices are received, integrated and circulated inside the hospital	Hospital adopts smart solutions tools (technology 4.0 such as IoT, cloud computing, big data, etc.) to carry out all internal work and all external interactions. Medical data from internal medical devices is integrated, circulated internally and transmitted to external partners Data from external medical devices are received, integrated and circulated inside the hospital	Hospital adopts new smart solutions tools (integrating 4.0 IoT technologies, big data, AI, etc.) to carry out all the work and interactions with the outside. The new smart solution tools are fully integrated with existing smart solutions tools. Medical data from internal medical devices is integrated, circulated internally and transmitted to external partners Data from external medical devices is received, integrated and circulated inside the hospital
2.6	Standards	Hospital has not applied IT, so there is no standard for smart solution tools to perform work and interact with the outsiders	Hospital applies IT at a simple level, and does not yet apply standards for smart solution tools to perform work and interact with the outsiders	Hospital adopts the standard for a few smart solution tools to perform work and interact with the outsiders	Hospital applies the standard for all smart solution tools to perform work and interact with the outsiders	Hospital adopts a new standard for a number of new smart solution tools to perform work and interact with the outsiders
2.7	Data update	Data are updated at the hospital level and are updated annually	Data are updated at departmental level and updated quarterly	Data are updated at departmental level and updated monthly	Data are updated at process level and updated weekly	Data are updated at the process level and updated online in real time
3.0 Diffusion :
3.1	Accessing users	Hospital IT system does not support any tool for doctors, nurses, and administrative staff to work. No mean for patients, customers, vendors to contact with the hospital	Hospital IT system supports tools for less than 50% of doctors, nurses, and administrative staff to work. Tools for less than 25% of patients, customers, vendors to contact with the hospital	Hospital IT system supports tools for 50%-70% of doctors, nurses, and administrative staff to work. Tools for 25%-50% of patients, customers, vendors to contact with the hospital	Hospital IT system supports tools for 70%-80% of doctors, nurses, and administrative staff to work. Tools for 50%-75% of patients, customers, vendors to contact with the hospital	Hospital IT system supports tools for more than 80% of doctors, nurses, and administrative staff to work. Tools for more than 75% of patients, customers, vendors to contact with the hospital
3.2	Process coverage	Hospital IT system does not support management functions (goal definition, measurement, gap analysis, decision making) and business processes/ medication processes of faculty/ department	Hospital IT system supports some of management functions (goal definition, measurement, gap analysis, decision making) and business processes/ medication processes of several faculties/ departments. IT system supports interaction functions with customers, patients, such as: medication registration, meeting schedule, personal profile. Users’ data are stored in file format	Hospital IT system supports some of management functions (goal definition, measurement, gap analysis, decision making) and business processes/ medication processes of several faculties/ departments. IT system supports all online interaction functions with patients, customers, and vendors. Users’ data and trace files are stored in database	Hospital IT system supports all of management functions (goal definition, measurement, gap analysis, decision making) and all business processes/ medication processes of all faculties/ departments. IT system supports all online interaction functions automatically with patients, customers, and vendors. Users’ data and trace files are stored in an integrated database	Hospital IT system supports all of management functions (goal definition, measurement, gap analysis, decision making) and all business processes/ medication processes of all faculties/ departments. IT system supports all online interaction functions automatically in the real-time manner with patients, customers, and vendors. Users’ data and trace files are stored in an integrated database
4.0 Organization :
4.1	Smart strategy	Hospital does not focus on smart hospital strategy development	Hospital develops strategies for each department in the smart direction	Smart strategies of all faculties and departments are integrated into the general strategy development of the hospital	Smart strategies guide change management across departments and hospital-wide	Hospital has the ability to creatively analyse conditions or information to refine smart strategies
4.2	Smart budget	Hospital has no funds for smart development	Hospital uses available guidelines to build a budget for smart development	Hospital uses existing facilities to increase funding for smart development	Hospital has the ability to creatively analyse conditions or information to increase funding for smart development	Hospital has the ability to come up with new ways to increase funding for smart development
4.3	Organizational coverage	Hospital has no intrinsic resources for smart development	Hospital has some intrinsic resources for its development but these resources are not specifically dedicated to smart development	Hospital has a number of specialized intrinsic resources for smart development	Hospital forms a specialized unit in charge of smart development and leads the changes towards smartness	The specialized unit in charge of smart development for hospital is capable of self-analysing and assessing the development of the technology, patient and customer needs to propose changes in management, operation and provision of smart health care services
4.4	IT staff capabilities	No IT staff	Hospital IT staff only has basic IT technical capacity, not yet equipped to deploy 4.0 technologies for smart hospital	Hospital IT staff is capable of operating and maintaining 4.0 technologies and has applied some 4.0 technologies to meet the needs of smart hospital	Hospital IT staff is qualified to self-deploy, operate, and maintain 4.0 technologies for smart hospital to meet the hospital’s requirements	Hospital IT staff has the ability to identify new needs as well as build and deploy 4.0 technology solutions to meet needs for smart hospital
4.5	User capabilities	Heads of Departments are not capable of using smart tools and solutions to do work and create reports. Managers and doctors are not capable of using smart tools and solutions to do the job and generate reports. Patients, customers, and partners of the hospital are not capable of using smart tools and solutions to interact with the hospital	Heads of Departments are only capable of using basic smart tools and solutions to perform tasks and create static reports that only administrators and doctors can use. Simple smart tools and solutions are used to do the job and generate static reports patients, customers, and partners of the hospital are only capable of using simple smart tools and solutions to interact with the hospital	Heads of Departments have the ability to use more complex smart tools and solutions to do most of the work, create and manage dynamic reports. Competent management staff and doctors use more complex smart tools and solutions to do most of the work, create and manage dynamic reports. Patient, customer, and partner are capable of using complex smart tools and solutions to interact with the hospital	Heads of Departments have the ability to use more complex smart tools and solutions to perform the entire job, create and manage dynamic reports, and competent management staff and doctors can use more complex smart tools and solutions to do all the work, create and manage dynamic reports. Patients, customers, partners have the ability to use complex smart tools and solutions to fully interact with the hospital	Heads of Departments and Faculty have the competence to use the tools and solutions of Technology 4.0 to analyse work outcomes and identify necessary improvements to perform work better. Management staff and doctors are competent to use the tools and solutions of technology 4.0 to analyse the results of the work, identify the necessary improvements to perform the work better. Patients, clients, and partners are able to identify the new requirements that hospitals need to meet with the support of tools and smart solutions
4.6	Competence improvement	Hospital does not have a program to improve the IT capacity of its staff	Hospital has a training program to improve IT capacity for staff to use some basic software and hospital’s IT system	Hospital has a training program to improve IT capacity for staff to use some professional software for data processing and use the hospital IT system. The hospital has drafted a manual instruction for patients and external partners to interact with the hospital through the IT system	Hospital has a training program to improve IT capacity for staff to use some professional software. Basic analysis is supported by professional data processing software, and 4.0 technology tools and system. The hospital has identified the requirements of patients and partners to adjust the IT system and makes them easier to interact with the hospital	Hospital has a training program to improve IT capacity for staff to use professional software. In-depth analysis is supported by professional data processing software as well as 4.0 tools and systems. The hospital has identified the requirements of patients and partners to adjust the IT system and makes them easier to interact with the hospital
4.7	Partner coordination	Hospital does not have mechanisms to coordinate and cooperate with tool and solution providers for smart development	Hospital coordinates with suppliers to build smart-oriented workflows, process, database, etc.	Hospital coordinates with suppliers to deploy smart tools and solutions	Hospital works with suppliers to improve smart tools and solutions and solve problems arising during the use of these tools and solutions	Hospital coordinates with suppliers to analyse and improve the performance of the IT system and upgrades the IT system capacity to meet the new requirements
5.0 Data and process:
5.1	Process capturing	Hospital does not conduct any activity for capturing and standardizing processes	Hospital uses available instructions for presenting the common processes	Hospital uses available forms and private rules for capturing and standardizing important processes	Hospital creatively uses available conditions and information for capturing and standardizing medication processes and operation processes	Hospital can create new processes for supporting creative solutions. Hospital captures and standardizes medication processes and operation processes based on new technologies
5.2	Process performance analysis	Hospital does not conduct any activity for analysing the performance of current processes	Hospital uses available instructions and basic tools for analysing the performance of repetitive processes	Hospital uses available supports and specific analysis tools for analysing the performance of important processes	Hospital can take advantages of new technologies and use information creatively for analysing the performance of medication processes	Hospital can create new and smart solutions based on new technologies to support for analysing the performance of medication processes
5.3	Process improvement	Hospital does not conduct any activity for improving the performance of current processes	Hospital uses process analysis results for improving the performance of repetitive processes	Hospital uses process analysis results and internal networks for continuously improving the performance of important processes and implementing the process change	Hospital can take advantages of new technologies and use information creatively for improving the performance of medication and operation processes and implementing the change for a better performance	Hospital can create new and smart solutions based on new technologies to support for analysing and improving the performance of medication and operation processes and implementing the change for a better performance
5.4	Data accumulation	Hospital does not integrate database of various hospital information systems automatically	Hospital integrates some databases of hospital information systems based on available guidelines	Hospital integrates various databases of hospital information systems on the cloud platform based on available conditions/ systems	Hospital integrates various databases of information systems on the cloud platform creatively (based on specific conditions/ systems)	Hospital integrates various databases of information systems in real-time manner on the cloud platform using modern technologies (IoT, cloud computing, wearable devices, etc.)
5.5	Data exploration	Hospital does not have any reporting system based on information system automatically	Hospital exploits data and reports generated from information systems based on available guidelines	Hospital exploits online data and reports generated from information systems based on available conditions/ systems	Hospital exploits online data and reports generated from hospital information systems creatively (based on specific conditions/ systems)	Hospital exploits real-time data and reports generated from smart hospital information systems on the cloud platform using modern technologies (BI, Data mining, AI, KMS, etc.)
5.6	Data quality	Hospital does not have any control system for providing the quality of input and output data of the information system	Hospital monitors and controls traditionally the quality of input and output data of faculties/ departments based on quarterly updated data	Hospital monitors and controls online the quality of input and output data of all faculties/ departments automatically (based on statistics)	Hospital monitors and controls online the quality of input and output data of all faculties/ departments automatically (based on statistics). Using data analysis tools for controlling data quality	Hospital monitors and controls in real-time manner the quality of input and output data of all faculties/ departments automatically (based on statistics). Using data analysis tools, such as: visualization, simulation, forecasting, etc., for realizing and analysing the causes of data trend and distribution
5.7	Functional integration	Hospital does not integrate operation processes and data for supporting the collaborative work between faculties/ departments	Hospital integrates business processes and data for supporting the collaborative work between internal medication group, quality group, and administrative group	Hospital integrates business processes and data for supporting the online collaborative work between internal medication group, quality group, and administrative group, and between them (basic ERP system)	Hospital fully integrates business processes and data for supporting the online collaborative work between internal medication group, quality group, and administrative group, and between them (full ERP system)	Hospital fully integrates business processes and data in real-time manner for supporting the online collaborative work between internal medication group, quality group, and administrative group, and between them (full ERP system integrated with outside partners in real-time manner)

References

Ahtonen

. Healthy and active ageing: turning the ‘silver’ economy into gold. In: European policy centre, Europe's political economy - coalition for health. Ethics and Society (CHES), 2012.

Wimble

Leroy

. Health information technology: promise and progress. Health Systems 2018; 7(3): 161–165. DOI: 10.1080/20476965.2018.1473944.

Carvalho

Rocha

van de Wetering

, et al. A Maturity model for hospital information systems. J Bus Res 2019; 94: 388–399. DOI: 10.1016/j.jbusres.2017.12.012.

Øvrelid

. Exploring adaptive mirroring in healthcare IT architectures. Health Systems 2023: 1–12. DOI: 10.1080/20476965.2023.2182238.

Chen

Chiang

Storey

. Business intelligence and analytics: from big data to big impact. MIS Q 2012; 36(4): 1165–1188. DOI: 10.2307/41703503.

Davenport

Harris

. Competing on analytics: updated, with a new introduction: the new science of winning. Boston, MA, 2017.

Gandomi

Haider

. Beyond the hype: big data concepts, methods, and analytics. Int J Inf Manag 2015; 35(2): 137–144. DOI: 10.1016/j.ijinfomgt.2014.10.007.

Foshay

Kuziemsky

. Towards an implementation framework for business intelligence in healthcare. Int J Inf Manag 2014; 34(1): 20–27. DOI: 10.1016/j.ijinfomgt.2013.09.003.

Tremblay

Hevner

Berndt

. Design of an information volatility measure for health care decision making. Decis Support Syst 2012; 52(2): 331–341. DOI: 10.1016/j.dss.2011.08.009.

10.

Gastaldi

Pietrosi

Lessanibahri

, et al. Measuring the maturity of business intelligence in healthcare: supporting the development of a roadmap toward precision medicine within ISMETT hospital. Technol Forecast Soc Change 2018; 128: 84–103. DOI: 10.1016/j.techfore.2017.10.023.

11.

Iqbal

Altaf

Aslam

, et al. Application of intelligent agents in health-care. Artif Intell Rev 2016; 46: 83–112. DOI: 10.1007/s10462-016-9457-y.

12.

Mettler

Vimarlund

. Understanding business intelligence in the context of healthcare. Health Inf J 2009; 15(3): 254–264. DOI: 10.1177/1460458209337446.

13.

Kraus

Schiavone

Pluzhnikova

, et al. Digital transformation in healthcare: analyzing the current state-of-research. J Bus Res 2021; 123: 557–567. DOI: 10.1016/j.jbusres.2020.10.030.

14.

Ramalingam

Subramanian

Reddy

, et al. Exploring business intelligence applications in the healthcare industry: a comprehensive analysis. Egyptian Informatics Journal 2024; 25: 100438. DOI: 10.1016/j.eij.2024.100438.

15.

PwC . Vietnam digital readiness report. https://www.pwc.com/vn/en/publications/vietnam-publications/digital-readiness.html (2021, accessed 20 May 2024).

16.

VIR . Digital healthcare to create motivation for investment. https://vir.com.vn/digital-healthcare-to-create-motivation-for-investment-81881.html (2020, accessed 15 May 2024).

17.

Tran

Thwaites

Van Nuil

, et al. Digital health policy and programs for hospital care in Vietnam: scoping review. J Med Internet Res 2022; 24(2): e32392. DOI: 10.2196/32392.

18.

Hanson

. Good health information–an asset not a burden. Aust Health Rev 2011; 35(1): 9–13. DOI: 10.1071/AH09865.

19.

Gomez-Sacristan

Rodriguez-Hernandez

Sempere

. Evaluation of quality of service in smart-hospital communications. J Med Imaging Health Inform 2015; 5(8): 1864–1869. DOI: 10.1166/jmihi.2015.1660.

20.

Comito

Falcone

Forestiero

. Current trends and practices in smart health monitoring and clinical decision support. In: Park

Taesung

Cho

Young-Rae

Xiaohua

, et al. (ed). 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2020, pp. 2577–2584.

21.

Coronato

Esposito

. Towards an implementation of smart hospital: a localization system for mobile users and devices. In: 2008 Sixth Annual IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 2008, pp. 715–719.

22.

Alter

. Theory of workarounds. Commun Assoc Inf Syst 2014; 34(55): 1041–1066.

23.

Hussein

Omar

, et al. Smart collaboration framework for managing chronic disease using recommender system. Health Systems 2014; 3(1): 12–17. DOI: 10.1057/hs.2013.8.

24.

Alter

. Making sense of smartness in the context of smart devices and smart systems. Inf Syst Front 2020; 22(2): 381–393. DOI: 10.1007/s10796-019-09919-9.

25.

Jahantigh

Habibi

Sarafrazi

. A conceptual framework for business intelligence critical success factors. Int J Bus Inf Syst 2019; 30(1): 109–123. DOI: 10.1504/IJBIS.2019.097058.

26.

WHO . Quality of care. Accessed in 2024.https://www.who.int/health-topics/quality-of-care#tab=tab_1

27.

Martínez-García

Alvarez-Romero

Román-Villarán

, et al. FAIR principles to improve the impact on health research management outcomes. Heliyon 2023; 9(5): e15733. DOI: 10.1016/j.heliyon.2023.e15733.

28.

Berg

Doktorchik

Quan

, et al. Automating data collection methods in electronic health record systems: a Social Determinant of Health (SDOH) viewpoint. Health Systems 2022: 1–9. DOI: 10.1080/20476965.2022.2075796.

29.

DeLone

McLean

. Information systems success: the quest for the dependent variable. Inf Syst Res 1992; 3(1): 60–95. DOI: 10.1287/isre.3.1.60.

30.

Sulistiyani

Tyas

SHY

. What is the measurement of the IT project success? Procedia Comput Sci 2022; 197: 282–289. DOI: 10.1016/j.procs.2021.12.142.

31.

Ariyachandra

Watson

. Which data warehouse architecture is most successful? Bus Intell J 2006; 11(1): 4–6.

32.

Olszak

Ziemba

. Approach to building and implementing business intelligence systems. Interdiscipl J Inf Knowl Manag 2007; 2(1): 135–148.

33.

Yeoh

Koronios

. Critical success factors for business intelligence systems. J Comput Inf Syst 2010; 50(3): 23–32. DOI: 10.1080/08874417.2010.11645404.

34.

El-Adaileh

Foster

. Successful business intelligence implementation: a systematic literature review. Journal of Work-Applied Management 2019; 11(2): 121–132. DOI: 10.1108/JWAM-09-2019-0027.

35.

Dash

Shakyawar

Sharma

, et al. Big data in healthcare: management, analysis and future prospects. J Big Data 2019; 6(1): 1–25. DOI: 10.1186/s40537-019-0217-0.

36.

Miah

Gammack

Hasan

. Methodologies for designing healthcare analytics solutions: a literature analysis. Health Inf J 2020; 26(4): 2300–2314. DOI: 10.1177/1460458219895386.

37.

Wang

Kung

Byrd

. Big data analytics: understanding its capabilities and potential benefits for healthcare organizations. Technol Forecast Soc Change 2018; 126: 3–13. DOI: 10.1016/j.techfore.2015.12.019.

38.

Borzekowski

. Measuring the cost impact of hospital information systems: 1987–1994. J Health Econ 2009; 28(5): 938–949. DOI: 10.1016/j.jhealeco.2009.06.004.

39.

Lucas

Sirovich

Gallagher

, et al.

Variation in cardiologists’ propensity to test and treat: is it associated with regional variation in utilization?

Circulation: Cardiovascular Quality and Outcomes 2010; 3(3): 253–260. DOI: 10.1161/CIRCOUTCOMES.108.840009.

40.

Queralt-Rosinach

Kaliyaperumal

Bernabé

, et al. Applying the FAIR principles to data in a hospital: challenges and opportunities in a pandemic. J Biomed Semant 2022; 13(1): 12. DOI: 10.1186/s13326-022-00263-7.

41.

Yang

C-H

Hsu

Y-L

. A hybrid multiple-criteria decision portfolio with the resource constraints model of a smart healthcare management system for public medical centers. Soc Econ Plann Sci 2022; 80: 101073. DOI: 10.1016/j.seps.2021.101073.

42.

Balasubramanian

Shukla

Sethi

, et al. A readiness assessment framework for Blockchain adoption: a healthcare case study. Technol Forecast Soc Change 2021; 165: 120536. DOI: 10.1016/j.techfore.2020.120536.

43.

Hevner

March

Park

, et al. Design science in information systems research. Management Information Systems Quarterly 2008; 28(1): 6.

44.

Carrera-Rivera

Larrinaga

Lasa

. Context-awareness for the design of Smart-product service systems: literature review. Comput Ind 2022; 142: 103730. DOI: 10.1016/j.compind.2022.103730.

45.

Carrera-Rivera

Ochoa-Agurto

Larrinaga

, et al. How-to conduct a systematic literature review: a quick guide for computer science research. MethodsX 2022: 101895. DOI: 10.1016/j.mex.2022.101895.

46.

CMMI Product Team . CMMI for development, ver. 1.3, software engineering process management program. https://www.sei.cmu.edu

47.

Carvalho

Rocha

Abreu

. Maturity assessment methodology for HISMM - hospital information system maturity model. J Med Syst 2019; 43: 45.

48.

HIMSS . Digital Health: a framework for healthcare transformation. https://www.himss.org/sites/hde/files/media/file/2022/12/21/dhi-white-paper.pdf. 2020.

49.

Pham

. A knowledge management approach for ensuring the success of IT industries in Vietnam. New York: Nova Science Publishers, 2017.

50.

Arbab

Spaulding

Gamm L

, et al. Healthcare strategic management and the resource based view. Journal of Strategy and Management 2014; 7(3): 251–264. DOI: 10.1108/JSMA-06-2013-0040.

51.

Sinnaiah

Adam

Mahadi

. A strategic management process: the role of decision-making style and organisational performance. Journal of Work-Applied Management 2023; 15(1): 37–50. DOI: 10.1108/JWAM-10-2022-0074.

52.

Hambrick

Mason

. Upper echelons: the organization as a reflection of its top managers. Acad Manag Rev 1984; 9(2): 193–206. DOI: 10.1071/AH09865.

53.

Qin

Wang

Huang

Y-N

, et al. Organisational culture research in healthcare: a big data bibliometric study. In: Healthcare 2023. MDPI, p. 169.