Abstract
Purpose:
Osteoporosis is an under-screened musculoskeletal disorder that results in diminished quality of life and significant burden to the healthcare system. We aimed to evaluate the ability of Rho, an artificial intelligence (AI) tool, to prospectively identify patients at-risk for low bone mineral density (BMD) from standard x-rays, its adoption rate by radiologists, and acceptance by primary care providers (PCPs).
Methods:
Patients ≥50 years were recruited when undergoing an x-ray of a Rho-eligible body part for any clinical indication. Questionnaires were completed at baseline and 6-month follow-up, and PCPs of “Rho-Positive” patients (those likely to have low BMD) were asked for feedback. Positive predictive value (PPV) was calculated in patients who returned within 6 months for a DXA.
Results:
Of 1145 patients consented, 987 had x-rays screened by Rho, and 655 were flagged as Rho-Positive. Radiologists included this finding in 524 (80%) of reports. Of all Rho-Positive patients, 125 had a DXA within 6 months; Rho had a 74% PPV for DXA T-Score <−1. From 51 PCP responses, 78% found Rho beneficial. Of 389 patients with follow-up questionnaire data, a greater proportion of patients with Rho results reported vs not reported by a radiologist had discussed bone health with their PCP since study start (37% vs 21%, P < .001), or were newly diagnosed with osteoporosis (12% vs 5%; P = .02).
Conclusion:
By identifying patients at-risk of low BMD, with acceptability of reporting by radiologists and generally positive feedback from PCPs, Rho has the potential to improve low screening rates for osteoporosis by leveraging existing x-ray data.
Introduction
Osteoporosis affects over 2.5 million Canadians. 1 Its associated fractures lead to diminished quality of life and increased mortality, and considerable burden on healthcare systems. 2 Bone mineral density (BMD) decreases with age and is one contributing factor to increasing a patient’s fracture risk. Dual-energy x-ray absorptiometry (DXA) is considered the gold standard for BMD measurement. 3 Current Canadian guidelines suggest primary care providers should assess fracture risk factors in adults 50 years of age and older, and, based on the outcome, some patients should be referred for DXA to determine a more accurate fracture risk score before appropriate treatment initiation. 4 Despite the clinical guidelines, osteoporosis is under-screened and under-diagnosed. This may result, at least in part, from the silent nature of the disease and time constraints during primary care visits. 5 Given that the incidence of fragility fractures are expected to continue to rise due to the increasing aging population, improving the management of osteoporosis and its complications is a significant public health priority. 6
Rho (16 Bit, Toronto ON) is a software device that utilizes artificial intelligence (AI) to identify patients at-risk for low BMD from x-rays of the chest, thoracic and lumbar spine, pelvis, knee, and hand/wrist, approved for clinical use by Health Canada. 7 Rho is considered an opportunistic screening tool as it analyzes x-ray images that have been acquired for other (unrelated) clinical indications. Rho was trained using retrospective data composed of over 62 000 x-ray/DXA pairs in patients over age 50 years obtained from multiple sites across Ontario. Using retrospective data, Rho achieved an area under the receiver operating curve (AUROC) for classifying patients as normal versus low BMD ranging between 0.80 and 0.89 in multiple independent data sets. 7
The purpose of this study is to investigate if Rho is effective at identifying patients at-risk of low BMD in a prospective clinical setting, has the potential to be adopted by radiologists and clinicians, assess the impact on DXA screening, and assess the results of those additional DXAs being performed.
Methods
This prospective study was approved by the institution’s Research Ethics Board and all procedures were performed according to the recommended policies and regulations. Patient recruitment occurred at 2 x-ray department locations within the institution’s network, between 2022-11-21 and 2023-03-31. X-ray systems were different at the 2 departments; one department had x-ray machines from 3 different manufacturers, the other department had x-ray machines from a single manufacturer. According to the manufacturer, the performance of Rho has been demonstrated to be independent of X-ray manufacturer and technical parameters such as kVp and exposure. 8 In a previous publication, they demonstrated similar performance at multiple imaging sites within a given independent population, and between multiple independent studies.7,8 Patients who met the inclusion criteria were invited to participate in the study, and were informed about the purpose of the study, risk/benefits, and data security. Those who were interested signed informed consent. The inclusion criteria required patients to be aged 50 and above, undergoing outpatient x-ray examinations of the chest, thoracic or lumbar spine, pelvis, knee, and hand or wrist. These patients were referred for x-ray imaging from multiple departments including the fracture clinic, orthopedics, family practices, respiratory medicine, neurology, and neurosurgery, based on various clinical needs. There were no exclusion criteria at time of recruitment. At time of analysis, Rho excludes patients who have a DXA in the PACS that occurred within the last 2 years. Further, the x-ray must pass the built-in quality control that confirms the x-ray is a frontal view of a Rho-eligible body part.
Radiologist Participation
Prior to study initiation, a 30-minute voluntary information session was held for the radiology group where the Rho software was presented in a series of slides. A recording of the session was shared with Radiologists who could not attend the session. In these slides, the radiologists were informed of the study but ultimately could choose whether or not to include the Rho findings in their x-ray report, either using their own free text or using a pre-defined text snippet programmed into their voice dictation software: “Based on Rho, an opportunistic AI-based screening device, this patient has been identified to be at risk for having low BMD. Suggest fracture risk assessment, and if indicated, BMD assessment with DXA. For more information on Rho: https://16Bit.ai/rho/faq.”
X-Ray Image Acquisition and Analysis by Rho
Digital radiographs were acquired as part of routine care for indications unrelated to low bone density. Rho analyzed the x-ray at time of acquisition unless it detected a recent DXA (within the last 2 years) in the picture archiving and communication system (PACS), in which case bone health status was assumed to be known and, therefore, the x-ray was not analyzed. Rho has a built-in image classifier as a form of quality control, and will only analyze images that pass the classifier, that is, are confirmed good quality frontal projections of a Rho-eligible body part. The Rho result was sent to PACS only if the patient was “Rho-Positive,” that is, if the patient was likely to have low BMD (Rho Score ≥6). The radiologist would see the Rho result when they reviewed the x-ray.
Primary Care Physician Follow-Up
At the onset of the study, we expected PCPs to receive x-ray reports that included Rho-Positive findings. Over the course of the study, it became apparent that not all PCPs were receiving x-ray reports, as they were often received by specialists instead. Given that a key element of the study was to assess if referring PCPs would act on Rho-Positive findings, the study team faxed Rho results to PCPs of Rho-Positive patients. PCPs were later sent a questionnaire via Google surveys to gather insights into the potential applications of AI systems as opportunistic screening tools in primary care, along with their perceived benefits and drawbacks.
Follow-Up DXA
DXAs acquired within 6 months following the Rho analysis were extracted from the patient health record. Ground truth diagnosis of low BMD (DXA T-Score <−1, ie, low bone mass or osteoporosis) was determined by the DXA findings. Moderate/high fracture risk was assessed by a radiologist at time of DXA using a clinically validated fracture risk model (CAROC 2010). 9
Pre- and Post-Rho Questionnaire
Pre-Rho questionnaires administered at the time of screening were used to collect patient demographics (age, gender, and ethnicity) and included subjective inquiries about patients’ awareness of osteoporosis, their perceived risk of low bone density, medications they were taking for bone health, and their knowledge of practices to maintain good bone health. Post-Rho questionnaires were administered 6 months after screening and assessed whether patients had discussed bone health with their PCP since the Rho screening, including whether they received guidance from their PCP or had implemented any changes to improve their bone health (eg, initiated new osteoporosis medication, calcium, or vitamin D use). All questionnaires can be found in the Supplemental Material.
Statistical Analysis
Kruskal-Wallis rank-based test was used to assess sex differences in continuous variables. Chi square tests were used to assess sex differences in proportions at baseline, radiologist reporting, and subsequent DXA rates.
Results
1145 participants aged 50 to 97 years completed the pre-Rho questionnaire. Of these, 987 (86%) had Rho-eligible x-rays that passed Rho’s built-in quality control module and were analyzed by Rho. Baseline characteristics are presented for patients with Rho-analyzed x-rays (Table 1).
Baseline Characteristics of Participants.
A greater proportion of female versus male patients had previously discussed osteoporosis or fracture risk with their PCP, were taking calcium, vitamin D, or osteoporosis medications, and had had a previous fragility fracture.
Of these participants, 655 (66%) were flagged as Rho-Positive, that is, likely to have low BMD. The rate of Rho-Positivity did not differ between the 2 imaging departments (67% and 66%, P = .68) despite different x-ray systems. Radiologists included this finding in the x-ray reports of 524 patients (80%), but all Rho-Positive findings were sent to PCPs by the study coordinator. Of 655 Rho-Positive patients, 125 underwent a subsequent DXA (Table 2). In contrast, only 7 of 332 Rho-Negative patients underwent a subsequent DXA.
Results of Rho Screening.
Note. The % of Rho positive patients with DXA T-Score <−1. These patients could have either DXA-defined low bone mass or osteoporosis. PPV = positive predictive value.
The percentage of patients that underwent a DXA within 6 months following a Rho-Positive finding did not differ between females and males (19% for both; P = .92), and neither did PPV for low BMD (78% and 69%; P = .37; Table 2). Females versus males had a greater proportion of patients with moderate- or high-fracture risk (59% vs 33%; P = .007). Of the 7 Rho-Negative patients who underwent a subsequent DXA, none had true low BMD.
All PCPs (n = 172) of Rho-Positive patients were faxed a form containing their patient’s Rho finding and were later asked to complete a survey regarding the utility of the AI-based screening tool. Fifty-one physicians completed the survey (Table 3).
Family Physician Feedback.
Six months following the x-ray and Rho screening, a second questionnaire was administered to patients. Of 987 participants who had Rho-eligible x-rays analyzed by Rho, follow-up questionnaire data was available in 389 patients (Table 4). Loss to follow-up was caused by death (n = 4), inability to contact (n = 46) or non-respondents (n = 548). Following the Rho screen, a greater proportion of patients with Rho-Positive results reported by a radiologist versus those without (ie, Rho Positive but not reported and Rho-Negative patients) self-reported that they were diagnosed with osteoporosis (26/215 [12%] vs 9/174 [5%]; P = .018), had discussed bone health with their doctor (80/215 [37%] vs 37/174 [21%]; P < .001), and, although not statistically significant, self-reported taking new osteoporosis medication (13/215 [6%] vs 5/174 [3%], P = .14). The groups did not differ statistically in self-reported rate of new vitamin D or calcium use. Patients were asked to rate their satisfaction that their x-ray was screened for signs of suboptimal bone health with AI. 60% were satisfied, 20% were neutral, and 20% were not satisfied or unsure.
Follow-Up Data in Participants.
Discussion
This study demonstrates the impact of Rho when used in a clinical setting. Most PCPs found value in receiving the Rho-Positive findings, and some referred patients for follow-up DXAs. A much greater proportion of Rho-Positive versus Rho-Negative patients returned for a DXA (19% vs 2%; P < .001), and Rho demonstrated a similar positive predictive value (74%) to other published reports. 7 Finally, a greater proportion of patients with a Rho-Positive versus Rho-Negative finding self-reported that they (i) were newly diagnosed with osteoporosis, and (ii) had discussions with their health care provider about bone health following Rho screening.
The value of Rho lies in its potential to complement the current recommended practice for osteoporosis screening, which is for PCPs to identify fracture risk factors in adults over age 50 years, and refer for DXA according to guidelines. 4 When a Rho-Positive finding is sent to a physician, Rho may prompt the physician to assess the patient for risk factors of fracture and consider DXA referral. Consideration of fracture risk factors is often overlooked, as PCPs are challenged to consider acute concerns of the patient and screening of other conditions, such as hypertension, diabetes, and cancer, all within limited appointment times. 10 This study suggests that the receipt of the Rho-Positive finding by the PCP did lead to discussions about bone health, some DXA referrals and new diagnoses of osteoporosis. Other studies have reported the potential for AI to flag patients with low BMD from x-ray,11-13 but to our knowledge, none have reported the downstream impacts when used in routine clinical practice.
The clinical utility of Rho is limited by a radiologist’s decision to include Rho findings, and a PCP receiving and acting on those findings. In this study, more than 80% of Rho-Positive patients had Rho-documented findings in their radiology reports, suggesting that radiologists embraced this AI-tool in their routine clinical care. To achieve greater radiologist adoption, hospitals or imaging centers could consider instructing radiologists to always include Rho Positive findings in their x-ray report; conceivably, this could be automated (eg, automatic integration of the Rho positive report in the radiology report at the discretion of the radiologist to keep or delete). To achieve improved communication of the result to PCPs, reports could be integrated with the electronic medical records, or sites could copy PCPs on x-ray reports being sent to specialists; we note, however, that PCP contact information is not always available. Most PCPs (78%) agreed that the information from Rho was useful for patient care and 74% stated they would use a Rho finding in the care of their patient. Some comments from PCPs reported perceived benefits, including: “It helped me to treat one patient who was missed in COVID 19 pandemic,” “This was helpful for further risk stratification,” and “assisting with PCP burden.” In contrast, some comments pointed out perceived problems, including: “More work for family physicians who are already drowning,” “. . . osteoporosis treatment is not great so identifying early is not necessarily desirable,” and “Main issue is patient compliance.” These comments highlight the complexities of addressing the care gap in osteoporosis. In addition to better screening, awareness of effective treatment options and adherence to treatment are challenges that also need to be addressed.
Rho’s ability to analyze standard x-rays from multiple body parts is an advantage over other AI-based opportunistic screening tools. While opportunistic screening by CT is growing, standard x-ray is far more commonly conducted. 14 Multiple studies have reported algorithm performance in x-rays of single focus areas, particularly chest.12,15,16 One similar device analyzes hip and pelvic x-rays. 17 At time of publication, no similar devices were licensed by Health Canada. Rho analyses x-rays of the chest, lumbar and thoracic spine, pelvis, knee, and hand/wrist, thus allowing for more widespread use to capture more patients.
As described in the methods, we observed that x-ray reports with Rho-Positive findings were not reaching PCPs. It is possible that as the study was conducted at an academic institution, more x-ray referrals came from specialists rather than PCPs. To ensure that the Rho finding was communicated to PCPs, the research team faxed results to PCPs of all Rho-Positive patients. It is possible that faxing PCPs introduced bias by raising PCP awareness of the Rho finding more than it would have been if they read the Rho finding in the x-ray report. Further, while radiologists included Rho findings in their report for 80% of Rho Positive patients, the PCPs were faxed reports for 100% of Rho Positive patients, which could have artificially increased the number of PCP-patient discussions about both health and any downstream consequences. Whether a greater proportion of PCPs in tertiary settings would have received x-ray reports with the Rho-Positive findings should be investigated. In routine clinical practice, faxing Rho-Positive results to PCPs would be burdensome, and another means of patient follow-up might be considered. Potentially, a model where Rho notifies a dedicated nurse practitioner, who then follows up directly with flagged patients and conducts a clinical fracture risk assessment and triage, could be a cost-effective future direction for a provincial screening program.
Additional limitations include that (i) baseline and follow-up questionnaire data was self-reported rather than collected from patient charts, (ii) the relatively small number of participants (389 of 987) who completed the post-Rho questionnaire, (iii) the relatively small number of Rho-Positive patients who returned for DXA (125 of 655), and (iv) the study design did not allow assessment of the false negative rate. (i) Future studies that wish to investigate PCP discussion with patients and follow-up DXA results should consider including PCPs and/or electronic medical records in the study design in order to have access to reliable (chart-based) information on bone health discussions, treatment initiation, DXA referral, and DXA results when they are not accessible through the PACS on which Rho is installed. (ii) Follow-up emails and phone calls were sent to all participants. (iii) Regarding the small number of DXA follow-ups, there are many possible contributing factors. First, Rho will not analyze x-rays of patients with recent DXAs in the PACS, as their bone health status is presumed known. Given that the study was conducted at an academic institution, some x-rays may have been analyzed for patients with a recent DXA at a different imaging center, and as such, the patient’s PCP may already have been aware of their bone health status and, therefore, elected to not send the patient for another DXA scan. Second, some DXA follow-ups may not have been completed within 6 months due to patient or DXA machine availability. Third, in some Rho-Positive patients, follow-up with their PCP may have resulted in a discussion of bone health that determined a DXA was not required at this time. (iv) Finally, given that the main objective was to investigate the impact of Rho in a prospective clinical setting, we did not obtain DXAs on patients that were Rho-Negative, and thus could not determine the false negative rate. Care providers of patients flagged as Rho Negative are not notified of a negative Rho finding. These patients are still subject to the standard of care. In a previous study of 3 independent datasets that reported the performance of Rho, the false negative rate, which depends on the prevalence of low BMD in the dataset, ranged from 12% to 38%. 7
Conclusion
In conclusion, this study effectively demonstrates Rho’s positive impact in a clinical setting. It successfully identifies patients at risk of low bone density, with high adoption rates among radiologists. The receipt of Rho-Positive findings by PCPs led to discussions about bone health and, in some cases, referrals for follow-up DXA scans. A significantly higher proportion of Rho-Positive patients underwent subsequent DXA scans compared to Rho-Negative patients, and Rho exhibited a comparable positive predictive value to previously published reports. Ultimately, a greater percentage of patients with a Rho-Positive finding were newly diagnosed with osteoporosis and engaged in discussions about bone health with their healthcare providers following Rho screening. By leveraging existing x-ray data to identify at-risk patients, Rho holds significant potential to enhance the early detection of osteoporosis.
Supplemental Material
sj-docx-1-caj-10.1177_08465371251380240 – Supplemental material for AI Screening Tool Based on X-Rays Improves Early Detection of Decreased Bone Density in a Clinical Setting
Supplemental material, sj-docx-1-caj-10.1177_08465371251380240 for AI Screening Tool Based on X-Rays Improves Early Detection of Decreased Bone Density in a Clinical Setting by Andrew N. Jayarajah, Angela Atinga, Linda Probyn, Thiru Sivakumaran, Monique Christakis and Anastasia Oikonomou in Canadian Association of Radiologists Journal
Footnotes
Acknowledgements
The authors acknowledge the participation of 16 Bit Inc. as part of a commercialization pilot project funded by CanHealth.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was resulted from a commercialization pilot project funded by CanHealth, research funding provided by Amgen Canada, and generous support from TD Bank.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
