Abstract
Background
Digital healthcare's advance has underscored an urgent requirement for solid medical record quality control, critical for data integrity, surpassing manual methods’ inadequacies.
Objective
The goal was to develop an AI system to manage medical record quality control comprehensively, using advanced AI like reinforcement learning and NLP to boost management's precision and efficiency.
Methods
This AI system uses a closed-loop framework for real-time record review using natural language processing techniques and reinforcement learning, synchronized with the hospital information system. It features a data layer for monitoring, a service layer for AI analysis, and a presentation layer for user engagement. Its impact was evaluated by comparing quality metrics pre- and post-deployment.
Results
With the AI system, quality control became fully operational, with review times per record plummeting from 4200 s to 2 s. The share of Grade A records rose from 89.43% to 99.21%, and the system markedly minimized formal and substantive record errors, enhancing completeness and accuracy. The implementation of the artificial intelligence-based medical record quality control system optimizes the quality control process, dynamically regulates the diagnostic behavior of medical staff, and promotes the standardization and normalization of clinical medical record writing.
Conclusions
The AI-driven system significantly upgraded the management of medical records in terms of efficiency and accuracy. It provides a scalable approach for hospitals to refine quality control, propelling healthcare towards heightened intelligence and automation, and foreshadowing AI's pivotal role in future healthcare quality management.
Keywords
Introduction
Medical records serve as the central repository for the assessment of medical quality, clinical practice, and clinical thinking, holding key data for evaluating medical processes, implementing medical safety systems, and accurately and reasonably assessing the quality of medical care. 1 With the rise of the medical informatization wave, the digital transformation of medical records has significantly enhanced the efficiency with which hospitals utilize medical record data. However, under the impetus of policies such as high-quality hospital development, DRG/DIP medical insurance payment reform, electronic medical record system grading, third-level public hospital evaluation, and performance assessment, the importance of medical record quality control has become increasingly prominent, exerting a profound and lasting impact on hospital operations and management. 2 Concurrently, the continuous growth in medical demand and the deepening of information construction have rendered traditional manual quality inspection and random medical record checks inadequate in the work of medical record quality control. These methods struggle to meet the modern hospital's high standards for comprehensiveness, efficiency, and accuracy in medical record data quality control. 3 Medical record quality control, as the cornerstone of hospital management, is facing unprecedented challenges and stricter requirements. Therefore, there is an urgent need to introduce new generation information technology to enhance the work efficiency and quality of medical record management, ensuring the accuracy and reliability of medical data, thereby providing solid data support for the sustainable development of hospitals.4,5
Machine learning is now being used more and more in life sciences.Mehdi Gheisari et al. have developed the COVID-19 Detection and Diagnosis Mobile AppThe application of machine learning to the detection and diagnosis of diseases, by incorporating mobile technology, can be an integral part of controlling diseases and improving patient survival. 6 Developing AI models that generalize and avoid bias can aid in the design of next-generation drugs, vaccines, and diagnostics that address infectious diseases and better serve the clinic. 7 Hwi Young Kim et al. 8 developed several models to predict the risk of hepatocellular carcinoma (HCC) in patients with chronic hepatitis B (CHB), which showed excellent performance in risk stratification compared to previous risk scores. A growing body of research suggests that machine learning has an increasingly important role in the life sciences.
As medical informatization progresses, the routine reporting of hospital medical record data has become a pivotal aspect in assessing the quality of diagnosis and treatment. However, the traditional process of quality control for medical records, heavily reliant on manual operations, is not only limited in efficiency but also struggles to ensure the comprehensiveness and accuracy of the results, posing a significant challenge for hospitals in data quality control. Against this backdrop, the introduction of artificial intelligence (AI) is particularly urgent, with its automated monitoring and feedback mechanisms expected to revolutionize the existing model of medical record quality control. 9 Traditional quality control methods, focusing on the final inspection of medical records, are often inefficient due to the enormous workload and fail to meet the high standards required in the modern hospital informatization process. These methods not only increase the urgency and work pressure of hospital medical record management but also limit the depth and breadth of quality control work. 10 Therefore, exploring and applying AI technology to enhance the intrinsic quality of medical record data, optimize workflow, and improve work efficiency has become the essential path for innovation in hospital medical record quality control. 11 Moreover, AI-based medical record quality control systems can not only meet the current specific requirements of hospitals for quality control but also provide effective tools and data support for refined management, promoting the development of hospital management towards a higher level of intelligence and automation in the long run. 12 Such systems, through advanced technologies like deep learning and Natural Language Processing (NLP), can conduct comprehensive analysis, real-time monitoring, and timely feedback of medical record data, ensuring the authenticity and accuracy of medical data, thereby providing strong data support for hospital decision-making.
In this research, the Reinforcement Learning (RL) applied is a machine learning technique that learns through the interaction of an intelligent agent with its environment, aiming to maximize the cumulative reward. Within the realm of medical record quality control, RL can act as an automated monitoring and feedback system, continuously refining its strategies to enhance the precision and completeness of medical documentation.13,14 The heart of RL lies in its ability to manage uncertainty and delayed rewards, making it exceptionally suitable for the intricate scenarios present in medical record quality control. For instance, during the medical record creation process, RL models can leverage real-time data to intelligently detect potential errors or omissions, providing timely feedback to guide medical staff in corrections. This capability not only elevates the quality of medical record composition but also boosts the efficiency of the overall healthcare process.
Moreover, the application of RL in the medical field extends beyond quality control of medical records; it has also shown broad potential in various other domains, including gaming AI, robotic control, NLP, and quantitative financial trading. Throughout the entire lifecycle of medical data quality control, RL can be integrated with NLP technologies, utilizing deep learning models to parse and comprehend medical record texts, achieving a more profound level of quality control. 15 It should be noted that the application of RL in the quality control of medical data is still in its exploratory phase, necessitating further research and practical application to confirm its efficacy and safety. Nevertheless, with ongoing technological advancements and the growing abundance of medical data, RL is poised to become a vital instrument for improving the efficiency and quality of medical data quality control.
Based on the national medical record quality standards and the requirements set by the National Health Commission and the National Healthcare Security Administration, this project aims to integrate the vision of healthcare reform with a commitment to high-quality development. The goal is to establish an AI-driven system for medical data quality control and oversight. This study focuses on achieving precise operation of an intelligent medical record quality control system to ensure rigorous monitoring and real-time feedback throughout the entire lifecycle of medical data, thereby continuously enhancing data quality. Through a detailed investigation and consideration of the specific circumstances of our institution, we have developed a customized quality control system. This system is designed to improve the accuracy and reliability of medical data across multiple hospital sites through intelligent monitoring and feedback mechanisms. This initiative not only meets the national standards for medical data quality but also provides a practical approach for the intelligent transformation of the healthcare industry. It supports the advancement of medical services toward greater efficiency and precision, and strengthens the enhancement of medical quality and the optimal allocation of healthcare resources.
Methods
Establishing a comprehensive closed-loop management system for medical record quality control
A thorough medical record quality control process is based on the PDCA (Plan-Do-Check-Act) cycle, which forms a management closed loop centered on quality control. 16 Therefore, the core functionalities of the medical record quality control system should further develop in the following areas:
Review Function of Quality Control Physicians: Both responsible physicians and departmental quality control physicians need to review and monitor the quality of medical records for their respective departments. They should also mark the status of these reviews.
Scheduled Notifications via OA System: Key and critical quality control results should be effectively communicated to quality control physicians and medical team leaders through an Office Automation (OA) system. This ensures that issues with medical records are promptly reported and corrective actions are enforced.
Statistical Analysis of Medical Record Quality: The system should provide multidimensional displays and analyses of medical record quality. This helps management personnel understand the current state of medical record quality, quickly identify potential risks, and enhance decision-making efficiency.
Leveraging natural language processing to develop a robust and scalable medical record quality control system
NLP is an AI technology focused on the analysis, interpretation, and generation of human language through computational methods. 17 Clinical documentation is often recorded in free text, which reflects both the personalized and complex nature of the information, leading to electronic health records (EHRs) containing semi-structured or unstructured data. This characteristic mirrors the nuanced nature of clinical reasoning. 18 Although current technological solutions offer more structured formats, such as form-based or tabular templates, these structured approaches can somewhat constrain physicians’ ability to express clinical narratives freely, hindering personalized case understanding and the enhancement of clinical reasoning. Consequently, despite the apparent increase in structure, a significant amount of electronic health record data still requires advanced NLP techniques for in-depth content recognition and comprehension.19,20
This study will initially integrate various third-party information systems, including Hospital Information Systems (HIS), Electronic Medical Record Systems (EMR), Laboratory Information Systems (LIS), and Picture Archiving and Communication Systems (PACS). This integration aims to achieve cross-campus and interdisciplinary clinical data consolidation and standardization. By establishing a data center that accurately reflects clinical processes and healthcare quality, the study will employ NLP techniques to transform semi-structured data from electronic medical records into actionable and analyzable formats. 21 This process is crucial for deepening the analysis of medical record content and enhancing the efficiency and accuracy of medical record quality control. It represents a key step in developing a new, highly scalable system for medical record quality control.
Optimization of medical record quality control algorithms using reinforcement learning
RL, a prominent branch of machine learning, is distinguished by its ability to enable agents to interact dynamically with their environment, self-educate based on feedback, and continuously refine strategies. 22 Unlike traditional supervised or unsupervised learning, RL does not rely on pre-labeled datasets but instead learns and adapts within a dynamic setting. In the realm of medical record quality control, RL demonstrates substantial advantages: it avoids rigid, manually-designed outputs and addresses the limitations of knowledge graph-based quality control methods. Furthermore, RL adapts effectively to various hospital settings and individual clinical documentation practices, showcasing exceptional adaptability and robustness. 23 Additionally, RL facilitates the creation of standardized, reusable algorithmic models that meet the demands for rapid deployment and scalability.
This project innovatively combines RL techniques with clinical documentation practices. By designing reward strategies based on various feedback mechanisms, it achieves a balance between the accuracy and broad applicability of quality control results. This approach enables the model to automatically explore optimal integration patterns for information within medical records, aiming to maximize overall feedback rewards. Through a dynamic feedback mechanism grounded in clinician behavior, RL effectively adapts to complex clinical environments, significantly improving the accuracy and efficiency of medical record quality control. This represents an innovative and effective solution for quality control throughout the entire lifecycle of healthcare data.
Design principles for the new medical record quality control system
Integrating AI technology with medical record quality management represents not only an innovation in management practices but also a transformation in management philosophy. Therefore, the design of this system adheres to fundamental requirements such as safety, practicality, openness, and reliability, while also following these key principles: 24
2.4.1 Comprehensive monitoring principle
The AI-based quality control system manages medical records throughout the entire patient care process, from admission to discharge. It ensures continuous monitoring of all diagnostic and treatment information throughout the patient's journey.
2.4.2 Real-Time quality control principle
The system allows hospital quality management personnel to monitor patient records within their authorized scope at any time, utilizing real-time data provided by the AI system.
2.4.3 Flexibility principle
The system supports flexible configuration of permissions and quality control rules based on the specific needs of different departments and personnel, accommodating varying clinical and administrative requirements.
2.4.4 Scalability principle
To meet the demands of business growth, the system supports increased application server capacity to enhance response speed and additional storage servers to expand data storage capabilities.
Architecture design of the new medical record quality control system
The overall architecture of the quality control system is illustrated in Figure 1 and encompasses four primary layers:

Quality control system architecture diagram.
2.5.1 Data layer
This layer is responsible for the dynamic monitoring of medical record data. It involves data collection, control, and structuring, as well as providing the transformed data through a unified communication protocol and information model to the other layers.
2.5.2 Service layer
This layer focuses on core business functions and provides operational management capabilities for upper-level applications. It includes analyzing the information from the data layer using AI technologies for quality control and deploying messaging interfaces within the system environment to communicate quality control results to medical staff via enterprise WeChat.
2.5.3 Presentation layer
This layer delivers a B/S (Browser/Server) architecture interface directly to system users. It displays information such as medical record defect comparisons, quality control result statistics, and approval workflows, offering a flexible and comprehensive user interface.
2.5.4 System users
The system supports interactions with clinical doctors, quality control personnel, and administrators, allowing them to utilize various functionalities based on their permissions. This includes medical record quality control, record modification tracking, appeals and reviews, and statistical analysis.
2.5.5 Technical framework
The quality control system is built on the J2EE standard architecture, with the core modules constructed on an Application Server. It employs a technology stack comprising Spring, Spring Cloud, and MyBatis. The backend database is managed using MySQL. The deployment architecture is divided into four main server components: Web Application Server, Front-End Server, Database Server, and Backup Server. 25
The AI-based medical data quality control system, provided by Beijing Yisheng Intelligent Technology Co., Ltd, was officially launched at the end of 2021. To assess its effectiveness, a comparative analysis was conducted using two groups of final medical cases. The control group, consisting of 84,680 cases from 2021, utilized the traditional manual quality inspection method. The study group, comprising 94,900 cases from 2022, employed the intelligent system for quality control. The evaluation focused on a comprehensive analysis of various quality control metrics for both groups, including quality control nodes, coverage rates, time spent per medical record, and the proportion of Grade A records. Additionally, the distribution of major forms and content defects was analyzed and compared between the two groups.
Statistical analysis
Quantitative data are presented as mean ± standard deviation, with comparisons between groups conducted using the t-test. Categorical data are expressed as rates, and group comparisons are performed using the chi-square (χ²) test. All statistical analyses are carried out using SPSS version 21.0, and a p-value of <0.05 is considered statistically significant.
Results
Key functions of the new medical record quality control system
The primary functions of the system include the following: quality control at various stages of medical record processing and at the final stage, tracking of record modifications, handling appeals and reviews related to quality control inquiries, scoring of medical records, and statistical analysis of defects. The system's functionalities are designed to address three main areas: departmental quality control, oversight of operational medical records, and supervision by the quality control department. The specific application features are outlined as follows:
Implementation of departmental self-monitoring
The new medical record quality control system leverages AI to achieve efficient automation of departmental self-monitoring. The system integrates real-time quality control functions, using admission and surgical completion times as monitoring checkpoints. It automatically enforces time-based quality control on critical documents such as admission records, progress notes, and physician rounds to ensure the timeliness and accuracy of medical records.
Moreover, the system conducts in-depth analysis of record content and provides real-time feedback to clinical staff, generating hospital-level feedback following physician corrections to establish a quality control loop. Record defects are visually distinguished by color based on their severity, facilitating rapid identification and correction by physicians, thus enhancing the quality of medical records. The system also offers features for querying modification history and handling inquiries and appeals, ensuring the authenticity and traceability of records. This support enables healthcare professionals to protect their legal rights in medical litigation and advances the intelligent and automated quality control of medical data throughout its lifecycle.
Intelligent oversight of operational medical records
The new medical record quality control system employs advanced intelligence to enable real-time online monitoring of both the writing and content quality of medical records. This system integrates the enforcement of essential medical procedures, such as consultations, physician rounds, emergency responses, and preoperative discussions, ensuring adherence to established protocols. When defects in the medical records are detected, quality control personnel can promptly reject and request corrections online. These corrections are immediately reported to the medical administration department, creating an efficient and closed-loop quality control process. This intelligent oversight mechanism not only enhances the efficiency of medical record quality control but also ensures high standards of record content, thereby providing robust support for intelligent and automated quality control throughout the entire lifecycle of medical data.
Comprehensive oversight by the quality control department
The new medical record quality control system provides a comprehensive oversight platform for the quality control staff in the medical administration department. Through a visual statistical module, the system enables precise filtering and localization of medical records based on various dimensions such as discharge department, discharge date, defect type, and penalty scores. This allows for a thorough statistical analysis and presentation of the quality status of different medical records. Quality control personnel, within their designated authority, can continuously monitor the quality of departmental medical records, ensuring the accuracy and completeness of medical data. The system's built-in AI not only facilitates automated monitoring but also establishes an effective early warning mechanism. With real-time interactive feedback and iterative optimization, this technology provides robust data support for decision-making by the hospital's quality control department, thereby enhancing the efficiency and quality of quality control throughout the entire lifecycle of medical data.
Enhancing overall quality of medical record control with advanced systems
The implementation of the AI-based medical record quality control system has led to a significant optimization of the quality control process. This shift has transformed the traditional end-of-process quality control into a more efficient real-time quality monitoring system, dynamically standardizing medical personnel's diagnostic practices and promoting the standardization and regulation of clinical record-keeping. Prior to the system's implementation, in 2021, the hospital managed an average of 232 discharges per day with just four quality control specialists, who could review only 25 records daily, resulting in a coverage rate of merely 10.8%. The introduction of the AI-driven system has increased the quality control coverage to 100% and dramatically reduced the time required to review each record from 4200 s to just 2 s, greatly enhancing work efficiency. Moreover, the system provides comprehensive oversight of the entire hospital's record quality, reducing potential risks associated with archived records and optimizing the cost structure of medical record quality control. In terms of accuracy, the proportion of grade A records in the control group was 89.43%, while the research group saw an increase to 99.21%, marking a 9.78% improvement. The unified standards of the AI quality control system eliminate human error, facilitating meticulous management of medical records and significantly reducing misdiagnosis rates. This enhancement is evident in the comparative analysis shown in Table 1, highlighting the exceptional effectiveness of the new medical record quality control system in improving overall record quality.
Comparison of medical record quality control metrics.
Comparison of medical record quality control metrics.
As illustrated in Figure 2, formal deficiencies in final medical records are predominantly found in progress notes (including senior physician round records and daily progress notes), admission records, discharge summaries, and perioperative documentation. Research data indicate that the application of the AI quality control system has led to a significant reduction in formal deficiencies across key categories. Specifically, the rate of deficiencies decreased by 8.63% in daily progress notes, 8.09% in senior physician round records, 7.41% in admission records, 7.94% in discharge summaries, 7.44% in initial progress notes, 7.90% in preoperative discussions and summaries, and 6.86% in surgical records. These results clearly demonstrate the exceptional effectiveness of the AI quality control system in managing and reducing formal deficiencies in medical records, thereby significantly enhancing the completeness and accuracy of documentation.

Distribution of Major form deficiencies.
In the realm of substantive quality, key areas of concern include consultation records, admission records, initial progress notes, and discharge summaries, as depicted in Figure 3. The implementation of the AI quality control system has resulted in a notable reduction in substantive deficiencies across these critical areas. Specifically, the rate of substantive deficiencies decreased by 10.22% in consultation records, 9.12% in admission records, 9.09% in initial progress notes, 9.60% in discharge summaries and death records, 6.02% in preoperative discussions and summaries, 7.11% in critical value records, and 6.57% in invasive procedure records. These figures compellingly demonstrate that the AI-driven quality control model has significantly improved the substantive quality of medical records, substantially enhancing the overall quality of medical documentation.

Distribution of Major content deficiencies.
AI is increasingly being used in the modeling, processing, and analysis of biomedical data, 26 and not only is AI useful in the classification of clinical outcomes and treatment of patients with viral infections, 27 but integrating AI into maternal-fetal medicine and obstetrics has the potential to significantly improve patient prognosis, healthcare efficiency, and personalized care planning. 28 All of this suggests that AI has an increasing role in biomedicine and plays an important role in clinical treatment, disease diagnosis, and information management.
The medical record quality control system developed in this research project aims to transcend the limitations of individual departments, specialties, or hospital campuses by establishing a robust quality control framework that supports inter-campus medical data oversight and ensures consistency in record management across different facilities. The long-term goal of this initiative is to expand to a regional healthcare network across multiple hospitals, thereby creating a regional model for medical record quality management. To achieve this, the design of the AI model emphasizes universality and standardization, facilitating rapid deployment and application in diverse settings. The newly developed medical record quality control system leverages advanced technologies such as NLP and RL, closely aligning with the practical needs of hospital record management. 29 The system focuses on data analysis and implements both real-time and post-process quality control measures. Through intelligent monitoring and feedback mechanisms, it enhances the efficiency and quality of medical record management. This approach not only provides a powerful tool for internal hospital quality control but also offers a viable pathway for developing a regional quality management model, underscoring the significant value and broad prospects of this project in the lifecycle quality control of medical data.
This study highlights the significant advantages of the AI-based medical record quality control system. Unlike traditional manual quality control methods, this system operates free from personal subjective and objective biases, adhering strictly to standardized quality control protocols. This approach enables meticulous management of medical records, enhancing accuracy and reducing diagnostic errors, thereby improving overall record quality. The system's implementation has notably increased operational efficiency, allowing for comprehensive oversight of hospital-wide record quality, mitigating potential risks associated with archived records, and optimizing the cost structure of medical record quality control. Additionally, the findings indicate that the new AI-driven quality control system excels in minimizing formal deficiencies in final medical records, effectively enhancing record completeness and accuracy. Under the AI quality control model, the substantive quality of medical records has also seen significant improvement, further elevating the overall quality of medical documentation. These results underscore the pivotal role and application value of the new quality control system in the lifecycle management of medical data.
The implementation of this project has profound social and economic benefits for the comprehensive enhancement of hospital medical record quality control. This system not only fosters management innovation by integrating cutting-edge technology with advanced management concepts but also achieves dynamic, closed-loop management of electronic medical records. This has significantly improved management efficiency and standards, creating an innovative management model. Furthermore, the system advances smart healthcare initiatives, reflecting the principles of intelligent healthcare. By employing real-time AI interventions during medical activities and record documentation, the system improves the clarity of diagnoses and the standardization of treatments from the outset. This effectively enhances the efficiency of clinical operations, optimizes patient hospitalization experiences, and reduces the incidence of medical disputes. 4 In terms of technological innovation, the system overcomes the limitations of traditional quality control systems that rely on annotated data for training algorithms. It introduces RL techniques into medical record quality control, offering a novel approach. The application of RL extends beyond record quality control to include process optimization and disease prediction, demonstrating substantial potential in the medical field. 23 This innovative practice provides new insights and methodologies for the healthcare information technology sector, indicating promising future applications.
To ensure the quality of operational medical records, the next phase of this project will enhance the quality control process for records submitted by physicians, in accordance with relevant policies and regulations. Specifically, when a physician submits a medical record, the system will automatically alert them and prevent submission if serious quality issues are detected, requiring the physician to address these issues before proceeding.
Through close collaboration with the medical affairs and medical records management departments, the project has identified the submission phase on the medical record's front page as a critical point for real-time detection of severe deficiencies via information sharing and interactive mechanisms. Should a defect be identified, the system will promptly notify the physician via a pop-up alert, necessitating corrections to ensure the record's completeness and accuracy before submission. This feature is expected to significantly reduce the incidence of Class C records, thereby improving the overall quality of archived records. 30 By employing intelligent quality control methods, this project not only boosts operational efficiency but also addresses the root causes of issues in record content, enhancing diagnostic clarity and treatment standardization. This will markedly improve clinical operational efficiency, enrich patient hospitalization experiences, and lower the incidence of medical disputes. Ultimately, the system will support the automation and intelligence of hospital medical record quality control processes, providing a solid technical foundation for the ongoing improvement of medical quality.
Of course, this researcher has some shortcomings. AI requires a large amount of high-quality medical data for training to ensure performance, but the medical industry is difficult to obtain data, medical cases are complex and varied, AI models are difficult to fully and accurately understand and process all case information, and there may be misdiagnosis or omission of diagnosis in the identification of rare diseases, complex conditions and so on. The medical industry is developing rapidly, case management needs are constantly changing, and AI technology needs to be frequently updated and optimized, all of which require a large amount of data support.
Conclusion
The AI-powered system greatly improves the efficiency and accuracy of medical record management. The system operates with strict adherence to standardized quality control protocols, which reduces diagnostic errors, enables comprehensive oversight of the quality of medical records across the hospital, reduces potential risks associated with archived medical records, and optimizes the cost structure of medical record quality control. It provides a scalable approach to fine-tuning quality control in hospitals, pushes healthcare toward a high degree of intelligence and automation, and foreshadows the critical role of AI in the future of healthcare quality management.
Footnotes
Ethical considerations
Not applicable.
Informed consent
Not applicable.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
