Abstract
BACKGROUND:
To effectively monitor medical insurance funds in the era of big data, the study tries to construct an inpatient cost rationality judgement model by designing a virtuous cycle of inpatient cost supervision information system and exploring a complete set of inpatient cost supervision methods.
OBJECTIVE:
To lay the foundation for applying artificial intelligence (AI) technology in medical insurance cost control supervision and provide feasible paths and available tools for medical insurance cost control managers.
METHODS:
By way of collecting and cleaning electronic medical record (EMR) data from 2016 to 2018 of a city in East China, focusing on basic patient information and cost information, and using a combination of machine learning modeling and information system construction, the study tries to form a feasible inpatient cost supervision method and operation path.
RESULTS:
The set of the regulatory method, applied in nursing homes of a city in East China, is compelling. The accuracy rates of rationality judgement in different main diseases are stable up to 80%, the false positive rate is steady within 10%, and rehabilitation fee days of hospitalization, and the number of complications are important factors affecting the rationality of the inpatient cost.
CONCLUSION:
The model construction and optimization method combining machine learning and information system can make practical cost rationality judgement on medical institution’s inpatient cost data, which can directly reflect the key influencing factors of relevant inpatient costs, and achieve the effect of guiding medical behavior and improving the efficiency of medical insurance fund use.
Background
As health insurance coverage expands and the level of protection increases, problems such as irregularities in insurance fraud, excessive treatment, and waste of resources have intensified, causing the fund to grow too fast. At the same time, according to the requirements to further promote the sustainable and healthy development of socially run medical institutions, medical insurance will face more risk and regulatory pressure on medical insurance funds.
In the past 20 years, many scholars have applied data mining and machine learning to health care cost regulation, hospitalization cost analysis, and fraud screening. Scholars such as Biafore used data mining techniques to discover specific data patterns and trends from a large amount of complex and heterogeneous data and used them to provide decision supports [1]. Milley applied data mining techniques to health care cost detection and gave successful implementation cases [2]. Rudman used data mining techniques to construct models for preventing fraud in US health insurance [3]. Gao proposed the use of data mining and machine learning to build a library of models and methods [4]. Lan applied machine learning and neural networks to the analysis of hospitalization costs for a variety of diseases [5].
Nowadays, there are three main medical insurance fee control method: rule-set-based, Pharmacy Benefit Management (PBM) and Diagnosis Related Groups (DRGs). Each of the three approaches has its advantages, but there are also incomplete rules, non-transparent third-party profit points, and sketchy details of paid service items. Focusing on primary care institutions, the nursing home (health institution category code A710) was used as the entry point for the study. Taking a city in East China as an example, there are now more than 40 nursing homes, with the lack of effective means to regulate hospitalization costs. According to the progress of literature research and actual management, the following problems mainly exist.
(1) Traditional modeling problems
Based on the preliminary study results and the feedback from clinicians, experts indicated that it was difficult to provide clear regulatory standards. Thus, the traditional modeling approach of forming rules through expert consultation was not feasible. In addition, conventional manual mathematical modeling is weak in dealing with nonlinear and complex data. Like the ruleset modeling approach, it is not dynamic or sustainable enough to respond to data expansion and technological development.
(2) Machine learning modeling problem
Referring to the relevant literature, artificial intelligence techniques were introduced to model physicians’ experiences inductively. However, limited by labor costs, most of the previous hospitalization cost studies have used unsupervised learning, neural networks, and other methods that do not require manual labeling. Such modeling approaches are more effective but less interpretable, making it difficult to generate effective feedback on actual management operations. Also, for predicting reasonable hospitalization costs, most studies adopted methods that do not require manual labels, such as unsupervised learning and neural networks. However, these methods do not distinguish between reasonable and unreasonable hospitalization cost data, but are based on the premise that ‘most of the data are reasonable’. Such a premise is not objective enough and negatively affects the learning effect of machine learning.
(3) Audit and supervision issues
In a city in East China, limited by the clinical expertise and human cost of administrators, there has never been a well-formed standard for the regulation of relevant hospitalization fees, with just a one-size-fits-all regulatory approach, i.e., the average bed-day fee must not exceed 400 Yuan. This traditional regulatory method is not only not conducive to prior and intermediate guidance, but also allows some physicians with a tendency to charge irregularities to take the initiative and adopt a direction of treatment that circumvents penalties.
To solve the above problems, the study adopts the following methods:
To address the problem that experts have difficulty in giving standards in a traditional modeling way, the study adopts machine learning to mine and summarize experts’ experience, and combine it with the construction of information systems to form a dynamic improvement of regulatory standards; to address the problem of sustainability of traditional modeling, the study combines machine learning models with a recyclable information management system to continuously update data and labels to achieve the purpose of sustainable improvement of models and systems. To address the problem of the high labor cost of machine learning modeling, the study adopts the gradient descent algorithm logic-based sampling, and also adopts active learning algorithm to screen the high-value labeled data that are difficult to classify and make key markings, to improve the effectiveness of the algorithm with less cost; to address the problem of poor interpretability of machine modeling, the study adopts the decision tree model with strong interpretability in machine learning, to effectively feedback; to address the problem of insufficient objective data, the study combines expert expertise with machine processing capability to construct a reasonable hospitalization cost prediction model by using neural networks with the reasonable data judged by qualified physicians. To address the problem of one-size-fits-all in the existing supervision standards, the study includes all basic patient information and cost information, combines expert experience and machine efficiency in a data-driven manner to more accurately judge the rationality of relevant hospitalization cost data.
In summary, to capture the appropriate point of combining public health, management, and computer science, to understand physicians’ thinking, and to assist health insurance in a more effective regulatory control of hospitalization costs, research objectives are as follows. To explore a dynamic method of discriminating the rationality of hospitalization costs by using single-disease hospitalization costs in nursing homes as an empirical entry point. To construct a model to judge the rationality of inpatient costs in nursing homes with a model to predict the reasonable costs, and support the research of big data monitoring methods for medical institutions. Design an inpatient cost cycle-monitoring information system to lay the foundation for the practical application of artificial intelligence technology in related regulation. Ultimately, provide feasible methods and available tools for medical insurance fee control managers.
Firstly, the research data source, mainly the inpatient medical record front sheet data from 2016 to 2018 of a city in east China provided by the Medical Insurance Center, was cleaned for the study after the preliminary preparation. In the second step, clinicians who have been practicing medicine for more than five years were invited to participate in an extensive online expert consultation. The sampled data are judged for the first round of rationality, and features are used to construct a preliminary model. All relevant qualified physicians judged the rationality of the cost volume and cost composition based on their own experience by reading the complete inpatient medical record front sheet. Three experts judged each piece of data, and the study used the data with consistent judgement for modeling to ensure the accuracy of the label and model. In the third step, we put the raw data into the model constructed in the first round to derive the machine’s judgement, and give the machine’s judgement to the doctors for the second round of validation to verify the model’s reliability. During the second round of warranty, each piece of data was judged by six experts. Due to the high proportion of positive labels in the data, the majority rule was sufficient to screen unreasonable data and determine the validation results. The data in which multiple classifiers judged inconsistently was also given feedback for Active Learning with the same principles as in the first round. The physicians will provide a re-label to put into the model. In the fourth step, we systematically design the model, which has constituted a complete system of model discrimination
Materials and methods
Source of data
The source of data was the Medical Insurance Center’s database of the inpatient medical record front sheet from 2016 to 2018 of a city in East China, which collected patient information data, including age, days of hospitalization, number of hospitalizations, presence of surgery, details of complications; and inpatient cost data, including general medical service fees, general treatment operation fees, western medicine fees, and nursing fees.
Feature dimensionality reduction
Expert consultation
Clinicians with at least five years of experience were invited to consult on the feasibility of the study. At the same time, experts focused on critical features by ranking the importance of features contained in the medical record front sheet data. The ranking order was determined by the composite score, which
PCA downscaling
With the help of the Principal Component Analysis (PCA) algorithm, the amount of information carried on the features can be measured, and the number of features is reduced while retaining most of the valid information, gradually creating a new feature matrix with fewer features that can represent most of the information in the original feature matrix. In dimensionality reduction, the measure of information used by PCA is the sample variance, also known as an interpretable variance. The larger the variance, the more information features carry.
In this method, the number of features to be retained after dimensionality reduction is used as the horizontal axis variable, and the contribution of interpretable variance captured by the new feature matrix is used as the vertical axis to visualize the cumulative interpretable variance contribution curve for reference and to initially determine the specific features and the number of features to be included in the study from the medical record front sheet.
Data pre-processing
Excluding data with missing key patient information fields such as age and primary disease. Excluding data with more than 1/3 of the cost feature missing and filling in data with no more than 1/3 of the cost feature missing with the median. The patient hospitalization costs showed skewed distribution. Thus the data were box-cox transformed to approximate normal distribution. Following this, the data were normalized to converge between [0, 1]. One-hot encoding and dummy variables were used to deal with the categorized feature data, such as gender and medical payment methods. Among them, because the three payment methods of ‘new agricultural cooperative’, ‘urban workers’, and ‘other’ were all recorded as ‘medical insurance’ in the medical record front sheet, no further classification was done. Binarization and segmentation were used to handle the continuous type of features, such as age.
Sampling and labeling
Invite nursing home clinicians with at least five years of experience to participate in an extensive online expert consultation. The first round of rationality judgement is made on the sampled data, and a preliminary model is constructed using the front sheet features. All relevant qualified physicians will judge the rationality of the cost volume and cost composition based on their own experience while reading the complete medical record front sheet information. A piece of data is considered reasonable if it is judged by the doctor as ‘reasonable cost volume’ and ‘reasonable cost composition’; if it is judged by the doctor as ‘unreasonable cost volume’ or ‘unreasonable cost composition’, then it will be considered unreasonable. Three experts will judge each data, and the discriminated consistent data are used for modeling to ensure the accuracy of the label and the model.
Due to the limitation of the number of relevant qualified physicians, the paper is based on systematic sampling and draws on the idea of the gradient descent method for sampling and labeling. Sample records are drawn separately by main diseases, with the mean value as the centroid. The centroid is approached with a learning rate α starting from the farthest Euclidean distance and given to the physician to judge whether it is reasonable. Each data will be judged by multiple experts. The learning rate
Decision tree modeling
We are modeling with the C5.0 decision tree algorithm [6, 7], which uses nod dichotomy to pursue maximization of information gain. When a feature is selected as a node, we want the information entropy of the feature to be close to 0 (i.e., the probability is close to 1), and the uncertainty is minimized. The label of the model is determined by both ‘cost volume rationality’ and ‘cost composition rationality’. The study set 70% sample size for the training set and 30% sample size for the test set, feature selection criterion chooses information entropy, feature division point selection criteria chooses random, maximum depth chooses three layers, random seed number is chosen 420, the category weight is chosen balanced, and the minimum impurity of node division is chosen 0.3.
System design.
Technology roadmap.
Active Learning [8, 9, 10] is to find the most valuable labeled data in the sample data without category labeling by certain algorithms, and then hand over the labeled data and its category labels to experts for manual labeling, and then incorporate the labeled data and its category labels into the training set to iteratively optimize the classification model and improve the processing effect of the model. The Active Learning model is A
In the study, the machine learning model C is the decision tree model constructed by the first round of labeling; the query rule Q is that the data labeled by the first round are trained with five classifiers with different algorithms, and if five classifiers give different predictions in the ratio of 2:3 to a sample, the sample is considered to have re-labeling value; the expert group S is the group of doctors participating in labeling; the labeled sample set L is the data set labeled by the first round; the unlabeled sample set U is the data set not labeled by the first round. The unlabeled sample set U is the dataset that has not been labeled in the first round.
Specifically, we put the original data into the model constructed in the first round to derive the machine learning judgements, and give the results of the machine learning judgements to the doctors for a second round of validation for the reliability of the model. In the second round of validation, the data that are judged differently by multiple classifiers are selected for Active Learning feedback with the same principles as in the first round, and re-labeled for the doctors to put into the model. Each piece of data will be judged by six experts. Due to the high proportion of positive labels in the data, the majority rule is sufficient to screen unreasonable data and to determine the validation results. The re-labeling is re-input into the model to form feedback and refine the model until the model is stable at about 80% accuracy in the test set.
Neural network prediction
Based on the reasonable data after Active Learning feedback, a neural network [11] is applied to construct a reasonable cost prediction model using basic patient information as the feature and cost as the label. The validity of the model is evaluated by mean square error MSE and coefficient of determination
System design
Based on the methodology of the above study, we can carry out the system design for use by decision-makers. The model gives a judgement on the rationality of the cost data, the medical insurance agency makes a second-time judgement on the unreasonable data after clustering, the hospital explains the data that are still unreasonable in the second-time judgement, and the experts verify the explanation. If the explanation is acceptable, the data labels are updated and put back into the model. The purpose of continuously improving the data and the model is achieved.
The front-end of the system adopts React-based Antdesignpro, the back-end adopts Django server, the main implementation language is JavaScript and Python, the machine learning model is based on Sklearn and Pytorch, and is generated by training the labeled data from the expert feedback result. The system design diagram is shown in Fig. 1.
Technology roadmapping
The technology roadmapping of the study is as follows, covering the complete operation path of ‘data processing
The technology roadmap is shown in Fig. 2. The core results during the process are located on the utmost right side of Fig. 2, namely ‘Rationality judgement model’, ‘Hospitalization cost supervision information system’ and ‘Reasonable hospitalization cost prediction model’. The system was built based on the 2 models, and both models were improved based on the system. The three parts permeated and promoted mutually.
Empirical results
Model effects
An empirical study of the entire methodology was conducted in nursing homes of a city in East China. A total of 18,697 inpatient medical record front sheets were sampled for the rationality judgement of clinicians, and 15,488 of them were effectively recovered, with a recovery rate of 82.8%. Among the results of clinicians’ judgement, 4330 items of cost volume were unreasonable, and 3727 items of cost composition were unreasonable. The volume and composition of unreasonable were taken as the intersection of 3542 items, and the volume and composition of unreasonable were taken as the concurrent set of 4515 items. After considering the combination of diseases, the amount of data for each classification is too small. Thus for nursing homes, the modeling by main disease classification is adopted for the study. The model accuracy, false-positive rate, and feature importance ranking of the five main diseases with the highest volume of data on the front sheet of nursing homes are shown in Table 1. Among them, 2352 cases of cerebral infarction sequelae, 609 unreasonable and 1743 reasonable; 831 cases of coronary atherosclerotic heart disease, 203 unreasonable and 628 reasonable; 532 cases of hypertension grade III, 81 unreasonable and 451 reasonable; 449 cases of pulmonary infection, 151 unreasonable and 298 reasonable; 434 cases of cerebral infarction, 119 unreasonable and 315 reasonable
Based on the needs of the medical insurance agency and labor costs in management, we adopt the principle of ‘letting go’ but not ‘killing wrong’ for unreasonable cases. Therefore, the study identifies the key performance indicators of the model, focusing on two indicators: accuracy rate and false-positive rate. The accuracy rate is the rate of agreement between expert and machine discriminations in a test set divided by machine learning; the false positive rate refers to a case that is judged by the machine discrimination as unreasonable but by the doctor as reasonable. The rationality judgement model effects of different main diseases are shown in Table 1.
Rationality judgement model effects
Rationality judgement model effects
The effectiveness of the neural network model is evaluated by modeling the ‘average daily cost’ and other individual costs as predictor labels. The mean square error was used to evaluate the prediction performance of the model, and the decision coefficient was used to evaluate the fitting performance of the model. In the study, only the costs related to the sequelae of cerebral infarction, which had the largest amount of data, were predicted well, and the results of the reasonable cost prediction model are as shown in Table 2.
Reasonable cost prediction model effect of cerebral infarction sequelae
The model is meaningful when the correlation coefficient
We compared the existing rationality judgement method with the paper, and selected two data where the main disease was the sequelae of cerebral infarction for comparison. Using the traditional regulatory standard, i.e., the average daily cost of a nursing home should not exceed 400 Yuan per day, Data 1 is reasonable and Data 2 is unreasonable at this point. However, based on our rationality judgment model, the reasonableness results are completely opposite after combining the basic information such as the number of complications and the number of days in the hospital. Further observation reveals that the model obtained from the study is clearly more descriptive than the traditional method. For Data 1, although the average daily cost did not exceed 400 Yuan per day, many individual costs were zero, which did not fit the rational situation, while for Data 2, although the average daily cost exceeded 400 Yuan per day, the number of complications of the patient was found to be 11 and serious, which did have a rational basis. The traditional one-size-fits-all approach is clearly unable to discern the reasonableness of this type of data. The individual cost profiles of Data 1 and Data 2 and the results of the two approaches to discriminate are shown in Table 3.
Comparison of typical data
Comparison of feature importance
In order to meet the effective practical application of the methodology of this paper and to form a complete feedback loop of ‘model discrimination
Medical insurance console system
Data access module. Dashboard (visualization) display module. Rationality feedback module. Unreasonable cost processing module. Process tracking module.
Database. Judgement model. Clustering model. Automatic classification of unreasonable cost records. Model trainer.
Rationality feedback module. Process tracking module.
Distribution module. Rationality feedback module.
Model refinement
Theoretically, for classification models, Vapnik has classical conclusions. It is proved that if we want to build a robust machine learning model, we should need both a large amount of data and features [12].
Based on the medical insurance needs, this study identifies the direction of model improvement, focusing on the increase of model accuracy and the decrease of false-positive rate. The ‘data volume-model accuracy and false positive rate trend’ and ‘feature volume-model accuracy and false positive rate trend’ were plotted, respectively. Each data point was modeled ten times, and the extreme values at both ends were eliminated and averaged to eliminate the error caused by randomness; meanwhile, in the trend graph of feature volume, the features were input one by one, and the order was continuously adjusted by combining the medical law and model sensitivity. Finally, through the trend diagram, the two main refinement directions of data volume and feature volume are determined.
Based on the current positive and negative category ratio of 6:1, although this study can more accurately screen the rationality of the cost data, it also requires a more extensive audit workforce for screening false-positive cases.
Thus, combined with the above study, the direction of model refinement can be summarized as follows:
Specifying the features of the information on the medical record front sheet and increasing the number of features in an appropriate amount. Continuously collecting front sheet to increase the amount of model data. To form benign feedback through a complete system of ‘model discrimination
In addition, complications, a series of important features, were simplified in the study. We replaced all complications with the number of complications to include in the model to ensure the amount of data needed for modeling. However, as the number of data increases, we should try to explore it in more depth.
For the important basic data in the study, the quality of the medical record front sheet plays a decisive role. By comparing the ranking of the importance of the features during the preliminary expert consultation with the ranking of the importance of the features fed by the machine learning model, useful directions for the improvement of the medical record front sheet were obtained.
As an example, a decision tree was applied to analyze the importance of features influencing inpatient cost in a case where the main disease was cerebral infarction sequelae. The results showed that: the rehabilitation cost, the number of complications, and the number of days of hospitalization were the important factors affecting the rationality of the cost data. The higher value of feature_importances_indicates that the feature is more important relative to the model [13]. A total of nine features with feature importance higher than 0.03 were included in the model fit for the sequelae of cerebral infarction, and the results were compared with the ranking results of feature importance obtained from expert consultation in Table 4.
Cross-referencing the differences between the two, and observing the existing database of the medical record front sheet, we can see that the data of ‘admission condition’ can be perfected and are mostly consistent in the existing database; the data of ‘number of days of hospitalization’ and ‘discharge condition’ have differences in the importance of features in the ranking of the two methods, and the study has not yet found the reason.
For China, which is rolling out the DRG model, and other countries that have accumulated a lot of DRG experience, this study also has some reference value. For the DRG model, generally, the government will give a general grouping framework, but we need more detailed rules specific to the local level. Then how can we form rules that are adapted to local conditions and dynamically adjusted under the general framework? The methodology of this study is a sound basis, moreover, the right modeling direction for combining manual and machine modeling. Specifically, for cases that cannot be enrolled, or are enrolled but abnormal, the operation path of the study can be used to create dynamic grouping criteria and figure out new subgroups by combining artificial intelligence with the expert review.
Implications for outpatient cost supervision
For outpatient cost supervision, more complex data involved. We will consider establishing a supervision model with the doctor’s treatment behavior as the core. The specific ideas are as follows. First, on the basis of a comprehensive analysis of doctors’ treatment behaviors [14], focus on the problems of rehabilitation, physiotherapy, and TCM treatment, and extract relevant early warning indicators. Second, establish early warning indicators in terms of the number and frequency of treatment items prescribed by doctors and the association between items. Next, the unsupervised learning [15] method is used to determine the early warning indicators, establish a big data early warning model, divide doctors into different clusters according to varying characteristics of treatment behaviors. Finally, based on the model results, the clusters of suspect doctors and the range of suspect doctors were determined, and the Analytic Hierarchy Process (AHP) hierarchical analysis [16] is used to score the suspect doctors and establish a red-orange-yellow three-level warning mechanism.
Future of AI supervision in EMR and others
Based on the content of EMR, it is the main daily work of the medical statistical department to judge the rationality of the inpatient cost in medical records, review and correct the filling quality, and form a systematic medical knowledge graph.
For the future management and review of medical records, except for the supervision of hospitalization expenses, a large number of scholars started to focus on Named Entity Recognition (NER) and Knowledge Graph (KG), to identify violations and unreasonable expenses by efficiently extracting key information in EMR, and to construct the knowledge graph of related professions. Liu et al. designed different feature templates and context windows for conditional random field learning and analyzed the model entity recognition efficiency to find the best electronic medical record feature templates and context windows [17]. En Ouyang constructed a set of BiLSTM-CRF algorithm models and combined them with the entity recognition system of multi-phrase, word segmentation, part-of-speech tagging, medical vocabulary, and other corpus features [18]. In 2019, Liu and other scholars established RoBERTa based on the language masking strategy of the BERT model, modified the key hyperparameters in the BERT, adopted a larger amount of model parameters, and more data pre-training, and changed the static mask of the traditional BERT model. It becomes dynamic, removes the next sentence prediction (NSP), and improves the batch and text encoding, which makes RoBERTa better than BERT in the downstream tasks [19]. The construction of knowledge graphs also has gradually penetrated various fields. The ‘Galaxy’ knowledge graph [20], as a military knowledge graph, has various data sources such as dark web data, Internet data, traditional databases, and military books, according to the type and entity of military events. The types are divided, including weapons and equipment of 88 countries and 6 major combat spaces, a total of more than 100,000 equipment entity data, and 330 military ontology categories. Other typical domain knowledge maps also include IBM Watson Health medical knowledge map [21], Haizhixing financial knowledge map [22], and Hisense ‘Traffic Management Cloud Brain’ traffic knowledge graph [23].
To meet the requirements of EMR review under the current background, the research team is willing to integrate the existing research progress and introduce new AI algorithms like RoBERTa into the field of EMR entity recognition to improve the efficiency of medical record review, to establish a systematic database for related entities for building a more targeted knowledge graph in the medical field, and finally, to accomplish a more complete supervision process mining from EMR based on AI technology.
At the same time, it is obviously not enough to supervise the medical process only through EMR. AI technology should be popularized at all stages, including early diagnosis guidance, intermediate auxiliary treatment, late EMR mining and cost supervision, the quality control of medical device in the whole process.
Take diagnosis and treatment of pneumonia as an example, machine learning has shown promise in overcoming the diagnostic difficulties of pneumonia (i.e., low specificity of symptoms, lack of accessible diagnostic tests and varied clinical presentation) [24].
In addition to knowledge and experience of medical doctors, correct diagnosis and appropriate patient treatment largely depend on accuracy and functionality of medical devices [25]. Medical device (MD) represents the backbone of the modern healthcare system. Considering their importance in daily medical practice, the process of manufacturing, marketing and usage has to be regulated at all levels [26].
AI is revolutionizing healthcare, from medical applications to clinical engineering. As healthcare is generating a lot of data, as in fact every medical device is generating a lot of data, those big data structures can be used to predict safety and performance of medical devices. For instance, the use of smart infusion pump systems has become the preferred method to ensure the safety of intravenous drugs. Most of these systems are based on AI expert systems rather than ML, but the durability and reliability of such devices have led to more comprehensive ML-based applications such as implantable insulin pumps and emerging closed-loop artificial pancreas devices [27]. Obtained results suggest that by introducing ML algorithms in MD management strategies benefit healthcare institution firstly in terms of increase of safety and quality of patient diagnosis and treatments, but also in cost optimization and resource management [28].
Conclusions
After an empirical study in nursing homes of a city in East China, a more effective inpatient cost rationality judgement method with a regulatory operation path was finally formed by constructing a rationality judgement model and a reasonable hospitalization cost prediction model with the design of a circulatory monitoring system, which can help the quality of medical insurance services and management efficiency, and can be applied in different policy contexts. The methodology applies the modeling method of data mining to medical insurance cost supervision, which better solves the problem of insufficient dynamics of traditional audit mode; combines formal machine learning and deep learning, which better solves the manual labeling cost of the conventional way; combines expert expertise and machine processing ability, which better solves the low human efficiency and poor interpretability of AI.
Ethics statement
The study mainly focuses on the cost dimension, so the data are without patient identifiable information. The data source is direct, and the study has no ethical issues involved. Permission from the Healthcare Security Administration in China is necessary to use the data.
Consent for publication
Informed consent was obtained from all individual participants included in the study.
Availability of data and materials
The datasets analyzed during the current study are not publicly available because the institution of data source prohibits researchers from providing their research data to other third-party individuals but are available from the corresponding author on reasonable request with permission from the Healthcare Security Administration in China.
Funding
This research project was funded by Fudan University.
Author contributions
All authors contributed to the design of the study. Xu, Zhang, Sheng, and Liu were responsible for the investigation and consultation. Xu and Sheng were responsible for data collection and analysis. Sheng, Liu and Xu were responsible for software building. Xu was responsible for writing the manuscript. Xu and Sun were responsible for the revision. The corresponding author Luo attests that all listed authors meet authorship criteria. No other individuals meeting the criteria have been omitted. Luo is the guarantor. All authors have read and approved the final manuscript.
Footnotes
Acknowledgments
The authors thank Shanghai Medical Insurance Bureau and Shanghai Medical Insurance Center for providing data needed for the research and organizing the participation of relevant clinicians.
Conflict of interest
The authors declare that they have no competing interests.