Patient‐reported outcomes (PROs) play an increasingly important role in medical decision making. Yet, patients whose objectives differ from their physician's may strategically report symptoms to alter treatment decisions. For example, athletes may underreport symptoms to expedite return‐to‐play (RTP) from sports‐related concussion (SRC). Thus, clinicians must implement treatment policies that mitigate the Price of Naïveté, that is, the reduction in health outcomes due to naïvely believing strategically reported symptoms. In this study, we analyze dynamic treatment cessation decisions with strategic patients. Specifically, we formulate the Behavior‐Aware Partially Observable Markov Decision Process (BA‐POMDP), which optimizes the timing of treatment cessation decisions while accounting for known symptom‐reporting behaviors. We then analytically characterize the BA‐POMDP's optimal policy, leading to several practical insights. Next, we formulate the Behavior‐Learning Partially Observable Markov Decision Process (BL‐POMDP), which extends the BA‐POMDP by learning a patient's symptom‐reporting behavior over time. We show that the BL‐POMDP is decomposable into several BA‐POMDPs, allowing us to leverage the BA‐POMDP's structural properties for solving the BL‐POMDP. Then, we apply the BL‐POMDP to RTP from SRC using data from 29 institutions across the United States. We estimate the Price of Naïveté by comparing the BL‐POMDP to naïve benchmark policies. Accordingly, the BL‐POMDP reduces premature RTP by over 44% and provides up to 3.63 additional health‐adjusted athletic exposures per athlete compared to current practice. Overall, changing the interpretation of reported symptoms can better reduce the Price of Naïveté over adjusting treatment cessation thresholds. Therefore, to improve patients' health outcomes, clinicians must understand how strategic behavior manifests in PROs.
ArendsP.WagnerR. J. S.MrazikM. (2019). SCAT symptom recall bias in concussed athletes. Archives of Clinical Neuropsychology, 34(5), 733–733. doi: https://doi.org/10.1093/arclin/acz026.03
2.
ÅströmK. J. (1965). Optimal control of Markov processes with incomplete state information. Journal of Mathematical Analysis and Applications, 10(1), 174–205. doi: https://doi.org/10.1016/0022‐247X(65)90154‐X
3.
AswaniA.KaminskyP.MintzY.FlowersE.FukuokaY. (2019). Behavioral modeling in weight loss interventions. European Journal of Operational Research, 272(3), 1058–1072. doi: https://doi.org/10.1016/j.ejor.2018.07.011
4.
AyerT.AlagozO.StoutN. K.BurnsideE. S. (2016). Heterogeneity in women's adherence and its role in optimal breast cancer screening policies. Management Science, 62(5), 1339–1362. doi: https://doi.org/10.1287/mnsc.2015.2180
5.
BarnettC. L.TomlinsS. A.UnderwoodD. J.WeiJ. T.MorganT. M.MontieJ. E.DentonB. T. (2017). Two‐stage biomarker protocols for improving the precision of early detection of prostate cancer. Medical Decision Making, 37(7), 815–826. doi: https://doi.org/10.1177/0272989X17696996
6.
BassC.HalliganP. (2014). Factitious disorders and malingering: Challenges for clinical assessment and management. The Lancet, 383(9926), 1422–1432. doi: https://doi.org/10.1016/S0140‐6736(13)62186‐8
7.
BaughC. M.KroshusE. (2016). Concussion management in US college football: Progress and pitfalls. Concussion, 1(1), cnc.15.6. doi: https://doi.org/10.2217/cnc.15.6
8.
BlackwellD. (1953). Equivalent comparisons of experiments. The Annals of Mathematical Statistics, 24(2), 265–272.
9.
BroglioS. P.MacciocchiS. N.FerraraM. S. (2009). Neurocognitive performance of concussed athletes when symptom free. Journal of Athletic Training, 42(4), 504–508. doi: https://doi.org/10.1016/S0162‐0908(09)79463‐2
10.
BroglioS. P.McAllisterT.KatzB. P.LaPraddM.ZhouW.McCreaM. A.HoyA.HazzardJ. B.KellyL. A.DiFioriJ.OrtegaJ. D.PortN.PutukianM.LangfordD.McDevittJ.CampbellD.JacksonJ. C.McGintyG.EstevezC., … MilesC. (2022). The natural history of sport‐related concussion in collegiate athletes: Findings from the NCAA‐DoD CARE Consortium. Sports Medicine, 52(2), 403–415. doi: https://doi.org/10.1007/s40279‐021‐01541‐7
11.
BroglioS. P.McCreaM.McAllisterT.HarezlakJ.KatzB.HackD.HainlineB. (2017). A national study on the effects of concussion in collegiate athletes and US Military Service Academy Members: The NCAA–DoD Concussion Assessment, Research and Education (CARE) consortium structure and methods. Sports Medicine, 47(7), 1437–1451. doi: https://doi.org/10.1007/s40279‐017‐0707‐1
12.
CantuR. C.GeanA. D. (2010). Second‐impact syndrome and a small subdural hematoma: An uncommon catastrophic result of repetitive head injury with a characteristic imaging appearance. Journal of Neurotrauma, 27(9), 1557–1564. doi: https://doi.org/10.1089/neu.2010.1334
13.
CapanM.KhojandiA.DentonB. T.WilliamsK. D.AyerT.ChhatwalJ.KurtM.LoboJ. M.RobertsM. S.ZaricG.ZhangS.SchwartzJ. S. (2017). From data to improved decisions: Operations research in healthcare delivery. Medical Decision Making, 37(8), 849–859. doi: https://doi.org/10.1177/0272989X17705636
14.
ConwayF. N.DominguesM.MonacoR.LesnewichL. M.RayA. E.AldermanB. L.TodaroS. M.BuckmanJ. F. (2020). Concussion symptom underreporting among incoming National Collegiate Athletic Association Division I College Athletes. Clinical Journal of Sport Medicine, 30(3), 203–209. doi: https://doi.org/10.1097/JSM.0000000000000557
15.
CoweeK.SimonJ. E. (2019). A history of previous severe injury and health‐related quality of life among former collegiate athletes. Journal of Athletic Training, 54(1), 64–69. doi: https://doi.org/10.4085/1062‐6050‐377‐17
16.
CrossM.KempS.SmithA.TrewarthaG.StokesK. (2016). Professional Rugby Union players have a 60% greater risk of time loss injury after concussion: A 2‐season prospective study of clinical outcomes. British Journal of Sports Medicine, 50(15), 926–931. doi: https://doi.org/10.1136/bjsports‐2015‐094982
17.
DehonE.WeissN.JonesJ.FaulconerW.HintonE.SterlingS. (2017). A systematic review of the impact of physician implicit racial bias on clinical decision making. Academic Emergency Medicine, 24(8), 895–904. doi: https://doi.org/10.1111/acem.13214
18.
DoshiP.GmytrasiewiczP. J. (2006, July). On the difficulty of achieving equilibrium in interactive POMDPs. In Proceedings of the National Conference on Artificial Intelligence, 21(2) (p. 1131). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.
19.
EpsteinR. M.StreetR. L. (2011). The values and value of patient‐centered care. The Annals of Family Medicine, 9(2), 100–103. doi: https://doi.org/10.1370/afm.1239
20.
GarciaG.‐G. P.BroglioS. P.LavieriM. S.McCreaM.McAllisterT. (2018). Quantifying the value of multidimensional assessment models for acute concussion: An analysis of data from the NCAA‐DoD care consortium. Sports Medicine, 48(7), 1739–1749. doi: https://doi.org/10.1007/s40279‐018‐0880‐x
21.
GarciaG.‐G. P.CzerniakL. L.LavieriM. S.LiebelS. W.McCreaM. A.McAllisterT. W.PasquinaP. F.BroglioS. P. (2022). Simulation‐optimization to distinguish optimal symptom free waiting period for return‐to‐play decisions in sport‐related concussion. In (Eds: FengB.PedrielliG.PengY.ShashaaniS.SongE.CorluC. G.LooC.LeeH., and LendermannP.), 2022 winter simulation conference (WSC) (pp. 1021–1032). IEEE. doi: https://doi.org/10.1109/WSC57314.2022.10015285
22.
GarciaG.‐G. P.LavieriM. S.JiangR.McAllisterT. W.McCreaM. A.BroglioS. P. (2019). A data‐driven approach to unlikely, possible, probable, and definite acute concussion assessment. Journal of Neurotrauma, 36(10), 1571–1583. doi: https://doi.org/10.1089/neu.2018.6098
23.
GarciaG.‐G. P.LavieriM. S.JiangR.McCreaM. A.McAllisterT. W.BroglioS. P. (2020). Data‐driven stochastic optimization approaches to determine decision thresholds for risk estimation models. IISE Transactions, 52(10), 1098–1121. doi: https://doi.org/10.1080/24725854.2020.1725254
24.
GarciaG.‐G. P.YangJ.LavieriM. S.McAllisterT. W.McCreaM. A.BroglioS. P. (2020). Optimizing components of the sport concussion assessment tool for acute concussion assessment. Neurosurgery, 87(5), 971–981. doi: https://doi.org/10.1093/neuros/nyaa150
25.
HermanD. C.JonesD.HarrisonA.MoserM.TillmanS.FarmerK.PassA.ClugstonJ. R.HernandezJ.ChmielewskiT. L. (2017). Concussion may increase the risk of subsequent lower extremity musculoskeletal injury in collegiate athletes. Sports Medicine, 47(5), 1003–1010. doi: https://doi.org/10.1007/s40279‐016‐0607‐9
26.
KerrZ. Y.EvensonK. R.RosamondW. D.MihalikJ. P.GuskiewiczK. M.MarshallS. W. (2014). Association between concussion and mental health in former collegiate athletes. Injury Epidemiology, 1(1), 28. doi: https://doi.org/10.1186/s40621‐014‐0028‐x
27.
KerrZ. Y.MihalikJ. P.GuskiewiczK. M.RosamondW. D.EvensonK. R.MarshallS. W. (2015). Agreement between athlete‐recalled and clinically documented concussion histories in former collegiate athletes. The American Journal of Sports Medicine, 43(3), 606–613. doi: https://doi.org/10.1177/0363546514562180
28.
KerrZ. Y.Register‐MihalikJ. K.MarshallS. W.EvensonK. R.MihalikJ. P.GuskiewiczK. M. (2014). Disclosure and non‐disclosure of concussion and concussion symptoms in athletes: Review and application of the socio‐ecological framework. Brain Injury, 28(8), 1009–1021. doi: https://doi.org/10.3109/02699052.2014.904049
29.
KerrZ. Y.ThomasL. C.SimonJ. E.McCreaM.GuskiewiczK. M. (2018). Association between history of multiple concussions and health outcomes among former college football players: 15‐Year Follow‐up From the NCAA Concussion Study (1999‐2001). The American Journal of Sports Medicine, 46(7), 1733–1741. doi: https://doi.org/10.1177/0363546518765121
30.
KeskinocakP.SavvaN. (2020). A review of the healthcare‐management (modeling) literature published in manufacturing & service operations management. Manufacturing & Service Operations Management, 22(1), 59–72. doi: https://doi.org/10.1287/msom.2019.0817
31.
KrishnamurthyV. (2012). Quickest detection POMDPS with social learning: Interaction of local and global decision makers. IEEE Transactions on Information Theory, 58(8), 5563–5587. doi: https://doi.org/10.1109/TIT.2012.2201372
32.
LawrenceW. F.FleishmanJ. A. (2004). Predicting EuroQoL EQ‐5D preference scores from the SF‐12 health survey in a nationally representative sample. Medical Decision Making, 24(2), 160–169. doi: https://doi.org/10.1177/0272989X04264015
33.
LevyA. G.SchererA. M.Zikmund‐FisherB. J.LarkinK.BarnesG. D.FagerlinA. (2018). Prevalence of and factors associated with patient nondisclosure of medically relevant information to clinicians. JAMA Network Open, 1(7), e185293. doi: https://doi.org/10.1001/jamanetworkopen.2018.5293
34.
LoboJ. M.DentonB. T.WilsonJ. R.ShahN. D.SmithS. A. (2017). Using claims data linked with electronic health records to monitor and improve adherence to medication. IISE Transactions on Healthcare Systems Engineering, 7(4), 194–214. doi: https://doi.org/10.1080/24725579.2017.1346728
35.
LohrK. N.ZebrackB. J. (2009). Using patient‐reported outcomes in clinical practice: Challenges and opportunities. Quality of Life Research, 18(1), 99–107. doi: https://doi.org/10.1007/s11136‐008‐9413‐7
36.
MasonJ. E.EnglandD. A.DentonB. T.SmithS. A.KurtM.ShahN. D. (2012). Optimizing statin treatment decisions for diabetes patients in the presence of uncertain future adherence. Medical Decision Making, 32(1), 154–166. doi: https://doi.org/10.1177/0272989X11404076
37.
McAllisterD. R.MotamediA. R.HameS. L.ShapiroM. S.DoreyF. J. (2001). Quality of life assessment in elite collegiate athletes. The American Journal of Sports Medicine, 29(6), 806–810. doi: https://doi.org/10.1177/03635465010290062201
38.
McCreaM.BroglioS.McAllisterT.ZhouW.ZhaoS.KatzB.KudelaM.HarezlakJ.NelsonL.MeierT.MarshallS. W.GuskiewiczK. M. (2020). Return to play and risk of repeat concussion in collegiate football players: Comparative analysis from the NCAA Concussion Study (1999–2001) and CARE Consortium (2014–2017). British Journal of Sports Medicine, 54(2), 102–109. doi: https://doi.org/10.1136/bjsports‐2019‐100579
39.
McCroryP.MeeuwisseW.DvorakJ.AubryM.BailesJ.BroglioS.CantuR. C.CassidyD.EchemendiaR. J.CastellaniR. J.DavisG. A.EllenbogenR.EmeryC.EngebretsenL.Feddermann‐DemontN.GizaC. C.GuskiewiczK. M.HerringS.IversonG. L., … VosP. E. (2017). Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51, 838–847. doi: https://doi.org/10.1136/bjsports‐2017‐097699
40.
MeierT. B.BrummelB. J.SinghR.NerioC. J.PolanskiD. W.BellgowanP. S. F. (2015). The underreporting of self‐reported symptoms following sports‐related concussion. Journal of Science and Medicine in Sport, 18(5), 507–511. doi: https://doi.org/10.1016/j.jsams.2014.07.008
41.
MintzY.AswaniA.KaminskyP.FlowersE.FukuokaY. (2020). Nonstationary bandits with habituation and recovery dynamics. Operations Research, 68(5), 1493–1516. doi: https://doi.org/10.1287/opre.2019.1918
42.
MoreauM. S.LangdonJ.BuckleyT. A. (2014). The lived experience of an in‐season concussion amongst NCAA Division I student‐athletes. International Journal of Exercise Science, 7(1), 62–74.
43.
PerloffR. M.BonderB.RayG. B.RayE. B.SiminoffL. A. (2006). Doctor‐patient communication, cultural competence, and minority health. American Behavioral Scientist, 49(6), 835–852. doi: https://doi.org/10.1177/0002764205283804
44.
RabinerL. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. doi: https://doi.org/10.1109/5.18626
45.
SchellG. J.GarciaG.‐G. P.LavieriM. S.SussmanJ. B.HaywardR. A. (2019). Optimal coinsurance rates for a heterogeneous population under inequality and resource constraints. IISE Transactions, 51(1), 74–91. doi: https://doi.org/10.1080/24725854.2018.1499053
46.
SchottmüllerC. (2013). Cost incentives for doctors: A double‐edged sword. European Economic Review, 61, 43–58. doi: https://doi.org/10.1016/j.euroecorev.2013.03.001
47.
ShechterS. M.BaileyM. D.SchaeferA. J.RobertsM. S. (2008). The optimal time to initiate HIV therapy under ordered health states. Operations Research, 56(1), 20–33. doi: https://doi.org/10.1287/opre.1070.0480
48.
SuenS.‐C.NegoescuD.GohJ. (2022). Design of incentive programs for optimal medication adherence in the presence of observable consumption. Operations Research, 70(3), 1691–1716. doi: https://doi.org/10.1287/opre.2021.2227
49.
WeberM. L.LynallR. C.HoffmanN. L.MillerE. H.KaminskiT. W.BuckleyT. A.BenjaminH. J.MilesC. M.WhitlowC. T.LintnerL.BroglioS. P.McCreaM.McAllisterT.SchmidtJ. D. (2019). Health‐related quality of life following concussion in collegiate student‐athletes with and without concussion history. Annals of Biomedical Engineering, 47(10), 2136–2146. doi: https://doi.org/10.1007/s10439‐018‐02151‐7
50.
YasudaM. (1988). The optimal value of Markov stopping problems with one‐step look ahead policy. Journal of Applied Probability, 25(3), 544–552. doi: https://doi.org/10.2307/3213983
51.
ZhangH.WernzC.HughesD. R. (2018). A stochastic game analysis of incentives and behavioral barriers in chronic disease management. Service Science, 10(3), 302–319. doi: https://doi.org/10.1287/serv.2018.0211
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
