Purpose: The purpose of this study was to compare the performance of artificial intelligence (AI)-generated and physician-written handoff summaries in a pediatric surgery population from nurses’ perspective, utilizing a validated quality assessment tool. Methods: Three clinical nurses independently reviewed and compared AI-generated summaries to physician-written summaries for patients admitted to the surgery service line utilizing the validated Physician Documentation Quality Instrument (PDQI-9) tool. The nurse evaluation team met prior to scoring to ensure consistency of interpretation and scoring of PDQI-9 domains. Results: Artificial intelligence-generated and physician written summaries for 117 patients were included, revealing a statistically significant higher ranking for AI-generated summaries (Z = 3.89, p < .001). Artificial intelligence-generated summaries outperformed physician written summaries in seven out of nine PDQI-9 domains (thorough, useful, organized, comprehensible, succinct, synthesized, and internally consistent). Fleiss’ Kappa analysis was conducted to assess the degree of agreement among the raters. The results showed fair agreement for five domains and slight agreement for four domains, highlighting the need and complexity of multirater review of clinical summaries. Practice Implications/Conclusion: This is the first study to examine how nurses evaluate the quality and usefulness of AI-generated summaries in their practice. Artificial intelligence-generated physician summaries improve the quality of handoff for nurses by providing clear and concise information, reducing variability, and providing structure for information sharing. Integration into practice may improve communication, enhance nursing workflow, and improve patient safety. Further research is needed to evaluate AI-generated physician summaries’ long-term effectiveness in patient safety, workload efficiency, and nursing satisfaction.
AmarasinghamR.SwansonT. S.TreichlerD. B.AmarasinghamS. N.ReedW. G. (2010). A rapid admission protocol to reduce emergency department boarding times. BMJ Quality & Safety, 19(3), 200–204. https://doi.org/10.1136/qshc.2008.028340
2.
ArsoniadisE. G.OlsenR. K.SkubeS.MarquardJ.MeltonG. B. (2019). Resident-physician preferences for electronic handoff note content: Implications for implementation of a system-wide electronic health record-integrated handoff tool. Studies in Health Technology and Informatics, 264, 1121–1125. https://doi.org/10.3233/SHTI190400
3.
Barak-CorrenY.WolfR.RozenblumR.CreedonJ. K.LipsettS. C.LyonsT. W.MichelsonK. A.MillerK. A.ShapiroD. J.ReisB. Y.FineA. M. (2024). Harnessing the power of generative AI for clinical summaries: Perspectives from emergency physicians. Annals of Emergency Medicine, 84(2), 128–138. https://doi.org/10.1016/j.annemergmed.2024.01.039
4.
CarrollJ. S.WilliamsM.GallivanT. M. (2012). The ins and outs of change of shift handoffs between nurses: A communication challenge. BMJ Quality & Safety, 21(7), 586–593. https://doi.org/10.1136/bmjqs-2011-000614
5.
CohenM. D.HilligossP. B. (2010). The published literature on handoffs in hospitals: Deficiencies identified in an extensive review. BMJ Quality & Safety, 19(6), 493–497. https://doi.org/10.1136/qshc.2009.033480
6.
DavisF. D. (1989). Technology acceptance model: TAM. In Al-SuqriM. N.Al-AufiA. S. (Eds.), Information seeking behavior and technology adoption (pp. 205–219). IGI Global.
7.
DeckerH.TrangK.RamirezJ.ColleyA.PierceL.ColemanM.BongiovanniT.MeltonG. B.WickE. (2023). Large language model-based chatbot vs surgeon-generated informed consent documentation for common procedures. JAMA Network Open, 6(10), e2336997. https://doi.org/10.1001/jamanetworkopen.2023.36997
8.
DesmedtM.UlenaersD.GrosemansJ.HellingsJ.BergsJ. (2021). Clinical handover and handoff in healthcare: A systematic review of systematic reviews. International Journal for Quality in Health Care, 33(1), mzaa170. https://doi.org/10.1093/intqhc/mzaa170
9.
GalatzanB. J.JohnsonE.JudsonT.ShanL. (2024). Linguistic dissection of nursing handoffs: Implications for patient safety in varied-acuity hospital settings. Journal of Clinical Nursing (John Wiley & Sons, Inc.), 33(8), 3077–3088. https://doi.org/10.1111/jocn.17190
10.
GhonemN. M. E.El-HusanyW. A. (2023). Handoff communication practices among nurses: A cross-sectional study. SAGE Open Nursing, 9, 23779608231159340. https://doi.org/10.1177/23779608231159340
11.
HarrisP. A.TaylorR.MinorB. L.ElliottV.FernandezM.O’NealL.McLeodL.DelacquaG.KirbyJ.DudaS. N. & REDCap Consortium. (2019). The REDCap consortium: Building an international community of software partners. Journal of Biomedical Informatics, 95, 103208. https://doi.org/10.1016/j.jbi.2019.103208
12.
HarrisP. A.TaylorR.ThielkeR.PayneJ.GonzalezN.CondeJ. G. (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010
13.
KeenanG.YakelE.Dunn LopezK.TschannenD.FordY. B. (2013). Challenges to nurses’ efforts of retrieving, documenting, and communicating patient care information. Journal of the American Medical Informatics Association, 20(2), 245–251. https://doi.org/10.1136/amiajnl-2012-001054
14.
MamalelalaT. T.HolzemerW. L.SeloilweE. S.IwuE. (2025). Experiences and challenges of rural nurses in Botswana handling births outside the hospital environment. Journal of Nursing Scholarship, 57(4), 695–702. https://doi.org/10.1111/jnu.70015
O'RourkeJ.AbrahamJ.RiesenbergL. A.MatsonJ.LopezK. D. (2018). A Delphi study to identify the core components of nurse to nurse handoff. Journal of Advanced Nursing, 74(7), 1659–1671. https://doi.org/10.1111/jan.13565
17.
PattonL. J.TidwellJ. D.Falder-SaeedK. L.YoungV. B.LewisB. D.BinderJ. F. (2017). Ensuring safe transfer of pediatric patients: A quality improvement project to standardize handoff communication. Journal of Pediatric Nursing, 34, 44–52. https://doi.org/10.1016/j.pedn.2017.01.004
18.
RhudyL. M.JohnsonM. R.KreckeC. A.KeigleyD. S.KraftS. J.MaxsonP. M.McGillS. M.WarfieldK. T. (2022). Standardized change-of-shift handoff: Nurses’ perspectives and implications for evidence-based practice. American Journal of Critical Care, 31(3), 181–188. https://doi.org/10.4037/ajcc2022629
19.
SchusterK.JenqG.ThungS.HershD.NunesJ.SilvermanD.HorwitzL. (2014). Electronic handoff instruments. Journal of the American Medical Informatics Association, 21(e2), e352–e357. https://doi.org/10.1136/amiajnl-2013-002361
20.
ShinnersL.AggarC.GraceS.SmithS. (2020). Exploring healthcare professionals’ understanding and experiences of artificial intelligence technology use in the delivery of healthcare: An integrative review. Health Informatics Journal, 26(2), 1225–1236. https://doi.org/10.1177/1460458219874641
21.
StetsonP. D.BakkenS.WrennJ. O.SieglerE. L. (2012). Assessing electronic note quality using the physician documentation quality instrument (PDQI-9). Applied Clinical Informatics, 3(2), 164–174. https://doi.org/10.4338/ACI-2011-11-RA-0070
22.
WaliaJ.QayumiZ.KhawarN.DygulskaB.BialikI.SalafiaC.NarulaP. (2016). Physician transition of care: Benefits of I-PASS and an electronic handoff system in a community pediatric residency program. Academic Pediatrics, 16(6), 519–523. https://doi.org/10.1016/j.acap.2016.03.005
