Enabling decision-making and innovation in learning health systems through simulation modelling

Abstract

Canadian healthcare systems require profound transformations to enhance patient experience, improve population health, reduce costs, and improve the work life of healthcare providers. Learning Health Systems (LHSs) are an approach for undertaking this transformation in an effective, efficient, and sustainable manner with digital technologies as a key enabler for change. However, the successful implementation of a LHS brings with it challenging and potentially risky changes to clinical practices and operations. Simulation modelling is an advanced analytics technique particularly well-suited for informing decision-making and planning prior to and during the transformation of complex systems such as LHSs. Yet, despite the use and demonstrated benefits of simulation modelling in many different industries including healthcare, its application in the context of LHSs has received limited attention. In this article, we discuss how simulation modelling can be leveraged to support better-informed, lower-risk decisions and innovation in LHSs.

Introduction

A Learning Health System (LHS) is a vision for transforming current health systems and healthcare provider organizations into high-performance entities able to provide effective and safe care at lower costs while providing excellent patient and care provider experience.^1,2 This vision is driven by the availability of large health-related datasets, such as those contained in electronic health records, combined with the possibilities offered by major technological advances in digital infrastructures and data processing capabilities to support timely and evidence-based decision-making.³ Digital technologies and data can thus be leveraged to support rapid learning cycles moving from raw data to knowledge creation and changes in practice, the results of which can then be used to initiate new learning cycles in an ongoing improvement approach.⁴

In Canada, LHSs have been identified as a key strategy for modernizing our health systems and addressing long-standing issues including unnecessary delays in access to care, siloed care delivery, human and financial resource constraints, and slow uptake and scaling-up of innovations.^1,5 Examples of LHS initiatives in Canada include the Alliance for Healthier Communities, a community-governed primary healthcare organization in Ontario that has reorganized itself as a LHS⁶ and ACCESS Open Minds, a LHS aiming to improve youth mental health services across Canada through a pan-Canadian online data repository that provides help to youth and their families.⁷ LHSs can also be found at the regional and organizational level, such as a LHS initiative driven by the Ottawa Hospital that aims to redesign lung cancer care processes in Eastern Ontario.⁸ The LHS model is also being proposed as a means to address pressing issues related to core Canadian health systems values such as improving Indigenous health,⁹ eliminating health inequities,¹⁰ and advancing environmental sustainability.¹¹

Nevertheless, transforming a health system into a LHS is challenging due to the inherent complexity of healthcare at scale. Successfully transforming a system into a LHS requires redesigning the whole system or organization around learning processes rather than instigating unconnected initiatives or projects.¹² This entails managing several learning cycles, each focusing on a specific clinical or operational problem, each moving at its own speed.¹³ Understanding the causes and consequences related to a diverse set of performance metrics and clinical outcomes within these learning cycles is key to their success. But since complexity tends to create opacity, changes to clinical and operational practices that are inherent to a LHS could increase the risk of unintended outcomes.

Implementing a shared infrastructure of digital, physical and human resources, methods, incentives, ethics, culture, governance, and more is a key enabler for successfully managing such interconnected yet distinct learning cycles.¹³ Making data available on all aspects of care and operations is also required to support clinical and operational decision-making in such an infrastructure.¹³ However, data availability is only the first step in creating a holistic view of a LHS and its learning cycles. Methods providing the ability to understand how a complex system works and what causes inefficiencies, errors, and lack of performance are needed to prevent and mitigate the risks that could arise from ongoing changes inherent to a LHS.

A useful approach for supporting transformation efforts and ongoing LHS processes is simulation modelling, which is a family of methods that rely on software tools to represent a system at different scales. These representations enable the identification of causes of observed events or problems and provide insights into the potential outcomes of anticipated changes. Simulation modelling is particularly well-suited for representing and generating insights for complex systems by facilitating the analysis of alternative downstream decisions on variables such as costs or outcomes.¹⁴ The analyses conducted using simulation models can thus inform decision-making and planning prior to making real-life changes to existing digital infrastructures, governance structures, the allocation of human and financial resource, clinical practices, and more.

Simulation modelling in healthcare

Simulation modelling involves creating a computer representation (i.e., a model) of a real process within a system in such a way that it captures its essential properties and relationships. Such a model provides an analysis tool to explore how a system is functioning, evaluate the impact of different potential interventions, and determine how the impact may be affected by other factors or features within the system itself.¹⁵ While descriptive and predictive analytics methods, respectively, focus on what happened and what will happen, simulation modelling is part of prescriptive analytics and helps to answer questions related to what may happen and why it happened. Moreover, simulation modelling offers more comprehensive support to decision-making than predictive techniques such as Machine Learning by providing the ability to evaluate alternative system configurations that differ in internal and external aspects.

Simulation modelling (hereafter “Simulation”) was first used in the defence industry in the 1950s, where early models were built using programming languages as FORTRAN and run in mainframes.¹⁶ Since then, simulation has evolved and been applied to modelling and improving processes in many different industries. In healthcare, applications of simulation have involved a wide range of operational and clinical decision-making problems^17,18 (see Table 1).

Table 1.

Overview of simulation methods.

Method	Description	Use	Examples of support to decision-making
MCS	Models the possible outcomes of an uncertain event through repeated random sampling	Evaluate functions (e.g., performance metrics) that are impossible to evaluate through direct analytical methods²⁰	Patient-treatment matching²¹; cancer screening and treatment²²; mental health status monitoring²³; setting maximum recommended wait time targets²⁴; diagnostic imaging operations²⁵; operating room scheduling²⁶; capacity planning²⁷
ABS	Models the behaviour of individuals (agents) and their interactions within a system or population	Understand how small changes in behaviour and interaction influence the overall system or population’s dynamics and emergent behaviours²⁸	Mental health treatment coverage and provision²⁹; targeting public health interventions³⁰; optimal treatment selection²¹
DES	Models the operation of a system, where entities flow through a number of activities, over a chronological sequence of events	Determine the impact of resource availability on waiting times and waitlists²⁸	Blood supply chain management³¹; design and operational management of mass vaccination centres³²; chronic obstructive pulmonary disease management³³; stroke treatment delivery³⁴; diagnostic imaging operations³⁵; capacity planning³⁶; capacity planning and appointment scheduling³⁷; cancer screening planning³⁸; preventing HIV transmission³⁹; treatment of renal failure⁴⁰; screening for diabetic retinopathy⁴¹
SD	Models the behaviour of a complex, dynamic system over time in terms of stocks (e.g., resources and people), flows between stocks and information that determines the values of the flows	Obtain long-term, aggregate-level insights about the response of the system to strategic decisions or decisions affecting a whole population²⁸	Hospital capacity planning⁴²; mental health intake and treatment management²⁹; management of AIDS epidemic⁴³; COVID-19 response⁴⁴; chlamydia screening⁴⁵

Four common simulation techniques used to improve healthcare systems are Monte Carlo Simulation (MCS), Agent-Based Simulation (ABS), Discrete-Event Simulation (DES), and System Dynamics (SD).¹⁹ These techniques are described in Table 1, which also provides examples of their application.

Simulation offers significant advantages for analyzing and understanding the behaviour of complex systems. Indeed, it is much easier, faster, more cost-effective, and safer to assess the potential outcomes that changes in a process may introduce than to directly intervene in a real system. Doing so encourages a “let’s try” attitude that fosters insights and innovation and supports better-informed, lower-risk decisions. Simulation models are also very helpful in understanding the broader impact of changes by highlighting system interdependencies and emerging behaviours, enabling holistic thinking. Related applications include examining the viability of new processes, conducting cost-benefit analysis, exploring system redesign options, examining key performance metrics in both the current and future states of the system, assessing implementation plans and risk, and disseminating the working of newly redesigned processes.¹⁶ Moreover, the mere process of building a simulation model is often beneficial, providing stakeholders with insights about causal determinants of a given problem, required trade-offs of a potential solution, and relevance of chosen metrics to measure outcomes.⁴⁶

Positive results associated with the use of simulation include reduced waiting times, earlier appointment confirmation times, higher appointment reliability, increased patient and care provider satisfaction, improved resource utilization and coordination, reduced overtime, balanced staff workload, decreased clerical rework, reduced costs, increased service levels, and increased adherence to clinical guidelines for diagnosis and treatment. However, despite the successful use of simulation techniques in healthcare and their proven ability to support understanding and decision-making at a system level, their uptake in the context of LHSs is still in early stages. The rapid emergence of LHSs across Canada is providing a timely opportunity to introduce simulation alongside other advanced analytics methods and digital technologies to support the development and implementation of LHSs.⁴

Supporting clinical and operational decision-making in LHSs

A core characteristic of a LHS is its organization around learning cycles aiming to address issues that impede the achievement of the Quadruple Aim by speeding up the implementation of innovations and ensuring their continued improvement.² Learning cycles within a LHS are composed of three core processes: Data to Knowledge (D2K), where new knowledge is created through accessing and processing clinical and operational data; Knowledge to Performance (K2P), where new knowledge and insights are used to change processes and practices to achieve desired outcomes; and Performance to Data (P2D), where changes to processes and practices are evaluated.¹³ Learning cycles are thus inherently iterative as the P2D process ensures that changes in performance are documented and analyzed to identify next steps for improving a clinical or operational problem of interest.

This view of LHSs helps to understand how simulation can support LHS leaders in making effective decisions by pinpointing its role in specific learning processes. In the D2K learning process, simulation modelling can support decision-making by helping to understand a problem and its causes.⁴⁷ Examples of this application of simulation techniques in the context of LHSs include the use of ABS to model a collaborative LHS as a population of patient- and doctor-agents with the goal of improving the patient-treatment matching process and its implementation while considering engagement, information and knowledge sharing among agents.²¹ More recently, MCS was used to generate synthetic patients for building an artificial data-centric, ML-enabled risk prediction LHS using lung cancer and stroke as examples.⁴⁸ These examples show the ability of simulation to create new understandings from data, whether at the individual, population, or system level.

Simulation can also be at the core of implementation strategies part of the K2P learning process by providing decision-makers with the ability to examine underlying assumptions, consider alternative strategies, and anticipate their consequences,⁴⁶ hence helping to select the best option for addressing a problem.⁴⁷ For example, simulation can be used to identify, predict, and prevent errors prior to implementing medical guidelines.⁴⁹ MCS (specifically, a Markov model) was used to conceptualize a patient journey for seniors receiving care in their homes in order to evaluate potential interventions and determine which interventions should be subsequently piloted as part of the following stages of a LHS cycle.⁵⁰ Simulation can also be used to identify conditions for successful implementation, for example, when designing coordinated service infrastructures (e.g., integrated practice units) in the context of HIV-AIDS care.⁵¹ Essentially, in cases where the implementation of innovations and improvements may have unforeseen effects, simulation offers a valuable analysis tool for anticipating and preventing negative impacts on efficiency, quality, and outcomes.⁴⁹

Simulation techniques may also help to plan P2D learning processes. For example, MCS (specifically, semi-Markov models) have been used to model populations at both the aggregate and individual levels to perform survival and cost-benefit analyses of varied decision policies.⁵² In another study, researchers performed a DES analysis of a stroke guideline that had already been implemented, showing that its cost-benefit was marginal while a much greater cost-benefit could have been achieved by reducing patients’ time from stroke onset to hospital arrival by 30 minutes.⁵³ Using simulation prior to implementing the guideline could have saved costs while increasing the number of treatable patients, thus directing resources to a more promising performance target. These examples show that using simulation within the P2D learning process can answer the call for methods that enable the assessment of a LHS’s performance and the conditions in which it may best operate.⁵⁴

Beyond its use in facilitating specific learning processes, simulation can support decision-making that relates to a LHS’s infrastructure or issues that span multiple learning processes. This includes clinical decision-support models designed to assist individual clinicians and their patients with decisions regarding their care. For example, clinical decision-support models developed using simulation methods can be used to plan for or replicate clinical trials, helping to estimate clinical performance metrics such as morbidity and mortality rates and cost.⁵² Simulation models are known for their usefulness in assessing the impact of introducing a medical, operational, or technological innovation on system-wide behaviour and performance.¹⁴ Policy decision-support models can thus be developed to more broadly evaluate whether particular healthcare technologies or other innovations should be implemented within the context of an organized healthcare system such as a LHS.¹⁴ More generally, simulation models can support operational decision-making by evaluating the behaviour of a health system using measures such as flow time, wait time, throughput, and resource utilization.⁵²

Innovating through simulation

Innovation should be a core value to foster rapid learning, but in practice, innovation is difficult to incorporate and evaluate in a LHS.⁵⁴ Embedding simulation in novel digital technologies could increase their value and capabilities, facilitating their uptake.

Two transformative digital technologies that are being embraced in healthcare are process mining and digital twins.^55,56 The first is a data-driven methodology used to analyze event logs from information systems to gain insights into how real processes operate, identify inefficiencies, and optimize workflows.⁵⁷ By offering data-driven insights into how healthcare processes are actually executed, process mining can be used to create rich data and representative process models for simulation. Simulation models can then be used to inform process optimization efforts in a manner that increases system-wide performance.

Digital twins are virtual replicas of physical systems that dynamically reflect their real-world counterparts.⁵⁸ While both simulation and digital twins involve replicating real-world systems in a virtual environment, a digital twin goes beyond simulation by connecting a virtual model to its physical counterpart in real-time, enabling continuous data exchange and two-way communication. Embedding simulation models in this environment not only allows to rigorously test and evaluate the potential impact of changes to the physical system before their real-world deployment but also supports real-time decision-making.

Leveraging the synergy of these technologies can empower organizations to make more informed decisions, implement targeted innovations, and ultimately deliver better care, more efficiently, and at a lower cost. In this context, simulation can act as a bridge between the descriptive and predictive capabilities of these technologies, contributing to the development and implementation of proactive analytics-driven and self-sustainable LHSs.^58,59

Take-aways and challenges to leverage simulation in LHSs

Despite the increasing complexity of healthcare organizations and the world in which they operate, the relentless time pressure for improving efficiency and patient outcomes, the massive amounts of data information systems generate, and the added and cheaper computer power, the use of system-wide simulation models in the context of LHSs is just beginning. Among the reasons for the lack of uptake are challenges related to the following.

Data management

Effective data management within LHSs faces obstacles related to data availability, access, linking diverse sources, reliability, cleaning, and maintenance.^60,61 While integrated data platforms are a starting point for addressing these issues, significant effort is needed to analyze and standardize data, especially across multiple organizations, to enable meaningful improvements and system-wide learning.

Digital infrastructure

The digital infrastructure of a LHS should provide essential services that can be applied across various learning cycles.¹³ In regard to simulation, these services should include sophisticated simulation software. Indeed, such applications are needed to support complex analysis and provide the full range of answers that may be needed by decision-makers.³⁶ Integrating dedicated software solutions for simulation modelling within a LHS infrastructure would thus be needed to fully leverage its optimization and innovation capabilities.

Expertise and culture

Successfully leveraging simulation in LHSs requires the right expertise and culture, including knowledge of best practices for selecting appropriate modelling approaches and suitable software. Furthermore, transforming raw healthcare data into useful datasets demands a blend of data science and clinical expertise to ensure the data preparation is both technically sound and clinically meaningful.⁶¹ Additionally, a broader set of complementary knowledge and roles may be needed throughout the simulation process to ensure its success, from model development to ongoing model governance, including model validation and deployment.⁶² In particular, in-depth practical expertise is needed to integrate simulation outcomes into routine operations. Successful examples of the latter highlight the importance of not only involving senior leadership overall but also of having local leaders and stakeholders that are highly knowledgeable about the system being intervened, motivated to address the issue being studied, and empowered to facilitate the process and implement desired changes.^63,64

Costs and benefits

Simulation brings with it costs related to: the use of specialized simulation software; staff time required to develop models, including time spent in data collection and data analysis activities; stakeholder engagement, for example, to support data collection and model validation; and the implementation of the changes suggested by the model.⁶⁵ The benefits of simulation vary based on the specific context and how it is applied. Operational benefits are typically measured in terms of savings (e.g., reduced overtime through better planning) or increased throughput (e.g., ability to serve more patients with the same number of staff and other clinical resources). Health benefits may be measured based on clinical outcomes (e.g., higher number of patients identified in an earlier stage of a disease) or in terms of downstream improvements (e.g., using quality-adjusted life years). While only a few studies report on the return on investment of real-life implementations in healthcare, those that do show high benefit-to-cost ratios.^63,65

Simulation has the capability to support better-informed, lower-risk decision-making and innovation in LHSs. While fully leveraging simulation requires resources and efforts, the benefits of using simulation techniques promise to outweigh required investments. Moreover, the rapid development of LHSs in Canada creates a timely momentum for integrating simulation as a core method for enabling the transformation of our increasingly digital Canadian health systems and healthcare organizations.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Institutional review board approval was not required.

ORCID iD

Lysanne Lessard

References

Noseworthy

. Innovation in the Canadian health system. Healthc Manag Forum. 2021;34(1):5-8. doi:10.1177/0840470420936709

Menear

Blanchette

Demers-Payette

Roy

. A framework for value-creating learning health systems. Health Res Pol Syst. 2019;17(1):79. doi:10.1186/s12961-019-0477-3

Friedman

Rubin

Brown

, et al. Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inf Assoc. 2015;22(1):43-50. doi:10.1136/amiajnl-2014-002977

Coates

Chung

AQH

Lessard

, et al. The use and role of digital technology in learning health systems: a scoping review. Int J Med Inf. 2023;178:105196. doi:10.1016/j.ijmedinf.2023.105196

Reid

Wodchis

Kuluski

, et al. Actioning the Learning Health System: an applied framework for integrating research into health systems. SSM Health Syst. 2024;2:100010. doi:10.1016/j.ssmhs.2024.100010

Nash

Rayner

Bhatti

Zagar

Zwarenstein

. The Alliance for Healthier Communities’ journey to a learning health system in primary care. Learn Health Syst. 2023;7(1):e10321. doi:10.1002/lrh2.10321

Malla

Iyer

Shah

, et al. Canadian response to need for transformation of youth mental health services: ACCESS Open Minds (Esprits ouverts). Early Interv Psychiatry. 2019;13(3):697-706. doi:10.1111/eip.12772

Fung-Kee-Fung

Maziak

Pantarotto

, et al. Regional process redesign of lung cancer care: a learning health system pilot project. Curr Oncol. 2018;25(1):59-66. doi:10.3747/co.25.3719

Rice

Mashford-Pringle

Qiang

, et al. Frameworks, guidelines, and tools to develop a learning health system for Indigenous health: an environmental scan for Canada. Learn Health Syst. 2024;8(1):e10376. doi:10.1002/lrh2.10376

10.

Bierman

Mistry

. Commentary: achieving health equity – the role of learning health systems. Healthc Policy. 2023;19(2):21-27. doi:10.12927/hcpol.2023.27236

11.

Barber

Sinclair

Cassidy

. Advancing environmentally sustainable learning health systems: perspectives from a Canadian health center. Learn Health Syst. 2025:e10470. doi:10.1002/lrh2.10470. Published online January 28, 2025.

12.

Melder

Robinson

McLoughlin

Iedema

Teede

. An overview of healthcare improvement: unpacking the complexity for clinicians and managers in a learning health system. Intern Med J. 2020;50(10):1174-1184. doi:10.1111/imj.14876

13.

Friedman

Rubin

Sullivan

. Toward an information infrastructure for global health improvement. Yearb Med Inform. 2017;26(1):16-23. doi:10.15265/IY-2017-004

14.

Caro

Briggs

Siebert

Kuntz

ISPOR-SMDM Modeling Good Research Practices Task Force . Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices task force-1. Value Health. 2012;15(6):796-803. doi:10.1016/j.jval.2012.06.012

15.

de Quiros

FGB

Dawidowski

Figar

. Representation of people’s decisions in health information systems. Methods Inf Med. 2018;56:e13-e19. doi:10.3414/ME16-05-0001

16.

Harrington

Tumay

. Simulation Modeling Methods. McGraw Hill Professional; 2000.

17.

Roy

Shah

Gajjar

. Application of simulation in healthcare service operations: a review and research agenda. ACM Trans Model Comput Simulat. 2021;31(1):1-23. doi:10.1145/3427753

18.

Beliën

Brailsford

Demeulemeester

Demirtas

Hans

Harper

. Fifty years of operational research applied to healthcare. Eur J Oper Res. 2024;326(2):189-206. doi:10.1016/j.ejor.2024.12.040. Published online December 22, 2024.

19.

Salleh

Thokala

Brennan

Hughes

Booth

. Simulation modelling in healthcare: an umbrella review of systematic literature reviews. Pharmacoeconomics. 2017;35(9):937-949. doi:10.1007/s40273-017-0523-3

20.

Choi

Kang

. Modeling and Simulation of Discrete Event Systems. 1st ed. Wiley Publishing; 2013.

21.

Seid

Bridgeland

Hartley

. A collaborative learning health system agent-based model: computational and face validity. Learn Health Syst. 2021;5(3):e10261. doi:10.1002/lrh2.10261

22.

Ekaette

Lee

Kelly

K-L

Dunscombe

. A Monte Carlo simulation approach to the characterization of uncertainties in cancer staging and radiation treatment decisions. J Oper Res Soc. 2007;58(2):177-185. doi:10.1057/palgrave.jors.2602269

23.

Mohiuddin

Brailsford

James

, et al. A multi-state model to improve the design of an automated system to monitor the activity patterns of patients with bipolar disorder. J Oper Res Soc. 2013;64(3):372-383. doi:10.1057/jors.2012.57

24.

Babashov

Sauré

Ozturk

Patrick

. Setting wait time targets in a multi‐priority patient setting. Prod Oper Manag. 2023;32(6):1958-1974. doi:10.1111/poms.13951

25.

Pisharody

Yarmohammadi

Ziv

, et al. Reducing wait times for radiology exams around holiday periods: a Monte Carlo simulation. J Digit Imag. 2023;36(1):29-37. doi:10.1007/s10278-022-00728-2

26.

Dexter

Macario

Traub

Hopwood

Lubarsky

. An operating room scheduling strategy to maximize the use of operating room block time: computer simulation of patient scheduling and survey of patients’ preferences for surgical waiting time. Anesth Analg. 1999;89(1):7-20. doi:10.1213/00000539-199907000-00003

27.

Patrick

Nelson

Lane

. A simulation model for capacity planning in community care. J Simulat. 2015;9(2):111-120. doi:10.1057/jos.2014.23

28.

Currie

CSM

Fowler

Kotiadis

, et al. How simulation modelling can help reduce the impact of COVID-19. J Simulat. 2020;14(2):83-97. doi:10.1080/17477778.2020.1751570

29.

Brice

Harper

Gartner

Behrens

. Modeling disease progression and treatment pathways for depression jointly using agent based modeling and system dynamics. Front Public Health. 2023;10:1011104. doi:10.3389/fpubh.2022.1011104

30.

Fetta

Harper

Knight

Williams

. Predicting adolescent social networks to stop smoking in secondary schools. Eur J Oper Res. 2018;265(1):263-276. doi:10.1016/j.ejor.2017.07.039

31.

Glasgow

Perkins

Tai

NRM

Brohi

Vasilakis

. Development of a discrete event simulation model for evaluating strategies of red blood cell provision following mass casualty events. Eur J Oper Res. 2018;270(1):362-374. doi:10.1016/j.ejor.2018.03.008

32.

Wood

Murch

Moss

Tyler

JMB

Thompson

Vasilakis

. Operational research for the safe and effective design of COVID-19 mass vaccination centres. Vaccine. 2021;39(27):3537-3540. doi:10.1016/j.vaccine.2021.05.024

33.

Yakutcan

Demir

Hurst

Taylor

. Patient pathway modelling using discrete event simulation to improve the management of COPD. J Oper Res Soc. 2022;73(4):754-778. doi:10.1080/01605682.2020.1854626

34.

Monks

Pitt

Stein

James

. Maximizing the population benefit from thrombolysis in acute ischemic stroke: a modeling study of in-hospital delays. Stroke. 2012;43(10):2706-2711. doi:10.1161/STROKEAHA.112.663187

35.

Singla

. Demand and capacity modelling in healthcare using discrete event simulation. OJMSi. 2020;08(4):88-107. doi:10.4236/ojmsi.2020.84007

36.

Edmonds

O’Connor

. The use of computer simulation as a strategic decision-making tool: a case study of an emergency department application. Healthc Manag Forum. 1999;12(3):32-38. doi:10.1016/S0840-4704(10)60715-6

37.

Liu

Sauré

Weber

Puterman

Tyldesley

. Improving access to chemotherapy through enhanced capacity planning and patient scheduling. IISE Trans Healthc Syst Eng. 2019;9(1):1-13. doi:10.1080/24725579.2018.1442376

38.

Rositch

Singh

Lahrichi

, et al. Planning for resilience in screening operations using discrete event simulation modeling: example of HPV testing in Peru. Implement Sci Commun. 2022;3(1):65. doi:10.1186/s43058-022-00302-5

39.

Rauner

Brailsford

Flessa

. Use of discrete-event simulation to evaluate strategies for the prevention of mother-to-child transmission of HIV in developing countries. J Oper Res Soc. 2005;56(2):222-233. doi:10.1057/palgrave.jors.2601884

40.

Davies

. A simulation model for planning services for renal patients in Europe. J Oper Res Soc. 1987;38(8):693-700. doi:10.1057/jors.1987.118

41.

Davies

Brailsford

Roderick

Canning

Crabbe

. Using simulation modelling for evaluating screening services for diabetic retinopathy. J Oper Res Soc. 2000;51(4):476-484. doi:10.1057/palgrave.jors.2600890

42.

Wolstenholme

McKelvie

. The Dynamics of Care: Understanding People Flows in Health and Social Care. Springer Nature; 2019.

43.

Roberts

Dangerfield

. Modelling the epidemiological consequences of HIV infection and AIDS: a contribution from operational research. J Oper Res Soc. 1990;41(4):273-289. doi:10.1057/jors.1990.49

44.

Duggan

Andrade

Murphy

Gleeson

Walsh

Nolan

. An age-cohort simulation model for generating COVID-19 scenarios: a study from Ireland’s pandemic response. Eur J Oper Res. 2024;313(1):343-358. doi:10.1016/j.ejor.2023.08.011

45.

Townshend

JRP

Turner

. Analysing the effectiveness of Chlamydia screening. J Oper Res Soc. 2000;51(7):812-824. doi:10.1057/palgrave.jors.2600978

46.

Sheldrick

Cruden

Schaefer

Mackie

. Rapid-cycle systems modeling to support evidence-informed decision-making during system-wide implementation. Implement Sci Commun. 2021;2(1):116. doi:10.1186/s43058-021-00218-6

47.

Global Commission on Evidence to Address Societal Challenges . Global Evidence Commission Update 2024: Building Momentum in Strengthening Domestic Evidence-Support Systems. Enhancing the Global Evidence Architecture, and Putting Evidence at the Centre of Everyday Life. McMaster Health Forum; 2024.

48.

Chen

. Simulation of a machine learning enabled learning health system for risk prediction using synthetic patient data. Sci Rep. 2022;12(1):17917. doi:10.1038/s41598-022-23011-4

49.

Garbayo

Stahl

. Simulation as an ethical imperative and epistemic responsibility for the implementation of medical guidelines in health care. Med Health Care Philos. 2017;20(1):37-42. doi:10.1007/s11019-016-9719-0

50.

Hall

. Development and Application of Economic Learning Health Systems – a Forecasting and Simulation Based Analysis of Home Health for Seniors. University of British Columbia; 2022. doi:10.14288/1.0421388

51.

Zeigler

. Discrete event system specification framework for self-improving healthcare service systems. IEEE Syst J. 2018;12(1):196-207. doi:10.1109/JSYST.2016.2514414

52.

Stahl

. Modelling methods for pharmacoeconomics and health technology assessment: an overview and guide. Pharmacoeconomics. 2008;26(2):131-148. doi:10.2165/00019053-200826020-00004

53.

Stahl

Furie

Gleason

Gazelle

. Stroke: effect of implementing an evaluation and treatment protocol compliant with NINDS recommendations. Radiology. 2003;228(3):659-668. doi:10.1148/radiol.2283021557

54.

Foley

Horwitz

. Learning health systems. Cambridge University Press; 2025. doi:10.1017/9781009325912. Published online March 3, 2025.

55.

Katsoulakis

Wang

, et al. Digital twins for health: a scoping review. npj Digit Med. 2024;7(1):77. doi:10.1038/s41746-024-01073-0

56.

Munoz-Gama

Martin

Fernandez-Llatas

, et al. Process mining for healthcare: characteristics and challenges. J Biomed Inf. 2022;127:103994. doi:10.1016/j.jbi.2022.103994

57.

Van der Aalst

WMP

. Process Mining: Data Science in Action. Springer; 2016. doi:10.1007/978-3-662-49851-4

58.

Croatti

Gabellini

Montagna

Ricci

. On the integration of agents and digital twins in healthcare. J Med Syst. 2020;44(9):161. doi:10.1007/s10916-020-01623-5

59.

Alazab

Khan

Koppu

, et al. Digital twins for healthcare 4.0—recent advances, architecture, and open challenges. IEEE Consum Electron Mag. 2023;12(6):29-37. doi:10.1109/MCE.2022.3208986

60.

Marshall

Burgos-Liz

Pasupathy

, et al. Transforming healthcare delivery: integrating dynamic simulation modelling and big data in health economics and outcomes research. Pharmacoeconomics. 2016;34(2):115-126. doi:10.1007/s40273-015-0330-7

61.

Bastarache

Brown

Cimino

, et al. Developing real-world evidence from real-world data: transforming raw data into analytical datasets. Learn Health Syst. 2022;6(1):e10293. doi:10.1002/lrh2.10293

62.

Luo

Mao

Sanchez-Pinto

, et al. Northwestern University resource and education development initiatives to advance collaborative artificial intelligence across the learning health system. Learn Health Syst. 2024;8(3):e10417. doi:10.1002/lrh2.10417

63.

Moretto

Comans

Chang

, et al. Implementation of simulation modelling to improve service planning in specialist orthopaedic and neurosurgical outpatient services. Implement Sci. 2019;14:78. doi:10.1186/s13012-019-0923-1

64.

Frisbee

Sousa

. Successful operational integration of healthcare analytics at Seattle Children’s. Learn Health Syst. 2023;7(2):e10331. doi:10.1002/lrh2.10331

65.

Soorapanth

Eldabi

Young

. Towards a framework for evaluating the costs and benefits of simulation modelling in healthcare. J Oper Res Soc. 2023;74(3):637-646. doi:10.1080/01605682.2022.2064780