Artificial intelligence research in Canadian hospitals: The development of metropolitan competencies

Introduction

The healthcare sector represents a critical area where the deployment of Artificial Intelligence (AI) and robotics can significantly improve patient care, increase diagnostic and treatment capabilities, or help in decision-making. The advent of machine learning, and more specifically deep learning, has enabled the emergence of data-driven solutions for health informatics and biomedical research. ¹ AI is used in a wide range of applications in dermatology, radiology, anaesthesiology, psychiatry, surgery, genomics, and medical records.^2,3 It can improve patient care not only by reducing costs and improving safety but also by intervening upstream through early detection of chronic disease or preventive medicine. ⁴

Challenges such as the lack of interpretability in AI outputs, the high costs of surgical systems, the requirement of tacit knowledge from surgeons to perform procedures adequately, and the scarcity of usable clinical data necessary for building accurate AI models hinder the widespread adoption of these technologies.^5–7 To overcome these issues, Canada is supporting initiatives to develop a Canadian health data platform for advancing precision medicine through the Digital Health and Discovery Platform (DHDP) coalition. This initiative connects partners from healthcare institutions, companies, and universities across Canada. ⁸ The case of Canada is particularly interesting, given the country’s influential contributions to the field of AI. Canada has a large community of AI scientists and practitioners with three AI hubs growing in Toronto, Montreal, and the Edmonton region.^9,10 In particular, in 2023, Toronto’s performance in AI start-ups has been noticeable, being considered one of the top spots at the world level. ¹¹

While Canada is well known for its AI research, AI devices are not integrated uniformly into Canadian healthcare institutes. ¹² Practitioners and medical students are not sufficiently trained in Canada, and there is some reluctance to adopt AI technologies within the medical community due to uncertainties around liability issues.^12,13 Most of the studies on AI in healthcare in Canada are qualitative and provide a limited understanding of overall diffusion trends. ¹⁴ There is a need for a more detailed grasp of the interdependence between the public sector, companies, and hospitals in the process of diffusion of AI in medicine.

In this article, we provide the first systematic quantitative analysis of the deployment of AI in Canadian hospitals at the metropolitan area level for the period 2000-2021. The diffusion of AI technologies in hospitals is linked to (a) the existence of a thriving local ecosystem supporting the development of the technologies inclusive of public and private organizations, (b) the ease of access to national and international hotspots for frontier research in AI, and (c) the ease of access to advanced users in national and international hospitals that have already adopted the technologies. Absorptive capacity of the hospital and of its employees also plays an important role in successful adoption. Hospitals with previous experience in these technologies will be more prone to adopt advanced AI systems.

We find evidence of the importance of both metropolitan and external knowledge sources for the uptake of AI in hospitals. AI research in local public organizations and companies in the past is correlated with a higher level of deployment of AI technologies in hospitals. Public research is relatively more important than company research, but this result may depend on the focus on hospital pre-clinical activities. Most interestingly, from a managerial perspective, is the fact that external knowledge flows coming from hospitals active in AI are highly correlated with the uptake of AI in local hospitals. Hospitals that succeeded in developing relationships with national and international hospitals active in AI are much more active in AI research. However, only a small group of metropolitan areas has been able to activate this channel of knowledge exchange. Toronto is leading, followed by Montreal, Vancouver, and Ottawa; the other metropolitan areas have not been able to exploit this source of knowledge so far. Hospital and health managers, especially in less important metropolitan areas, should be more proactive in supporting collaborations with both local public research and other hospitals at the national and international levels.

Data and methodology

We analyze the relation between the characteristics of a metropolitan ecosystem’s competencies in AI in the period 2000-2016 and the last 5 years of activity in AI in hospitals (2017-2021). Data availability and citation windows constrain the analysis to 2021. Table 1 shows the performance of the top 20 metropolitan areas in the world in the last period. The two largest Canadian cities are, respectively, in sixth and sixteenth positions, showing a major role for Canada.

OPEN IN VIEWER

Table 1.

Top contributors in AI research in hospitals (worldwide).

Metropolitan area	Country	# Articles (Hosp., after 2017)
Boston	United States	7,834
New York	United States	4,122
Rochester	United States	3,149
Seoul	South Korea	3,060
Beijing	China	2,655
Toronto	Canada	2,375
Houston	United States	2,024
London	United Kingdom	1,942
Dallas	United States	1,799
Cleveland	United States	1,735
Baltimore	United States	1,712
Berlin	Germany	1,457
Philadelphia	United States	1,442
Rotterdam	Netherlands	1,403
Los Angeles	United States	1,321
Montréal	Canada	1,313
New Taipei	Taiwan	1,291
South Bend	United States	1,234
Chicago	United States	1,133
Nashville	United States	1,094

AI technologies can be used in hospitals to support the management of the institution, in the pre-clinical phase of research and care experimentation, and to deliver care to patients in the clinical phase (diagnosis, pre-operative, operative, and postoperative). To study the deployment of AI in hospitals, we use bibliometric data of scientific publications indexed in OpenAlex. Capturing the deployment of AI in hospitals with publications can be considered an acceptable proxy for activities going on at the border between pre-clinical and clinical phases. As such, it provides a lower bound estimation of the real use of AI. We classified articles as pertinent to AI using OpenAlex’s “concepts.” We used a broad definition of AI including first-level concepts of “artificial intelligence” and “machine learning” and their respective ancestors. This allows us to extract 4,496,336 documents for the period 2000-2021. Authors with at least one Canadian affiliation accounted for 188,316 publications.

Using the Research Organization Registry (ROR) classification, we were able to identify 15,354,871 affiliations out of the 16,064,390 in our AI sample. We classified affiliations into (1) involving only public research (Only PR: all affiliations are either “Education” or “Government”); (2) private companies (Comp: at least one “Company” affiliation); and (3) hospitals (Hosp.: at least one “Healthcare” affiliation). We use this information to carry out the organizational analysis described below. We then aggregate our analysis at the Census Metropolitan Area (CMA) level. Our final sample is composed of 2,910 metropolitan areas participating in AI worldwide, with 38 of them in Canada.

Our focus lies in understanding the relationship between past performance in AI technologies by local public research organizations and businesses and their subsequent deployment by hospitals. We measure performance using indicators of scientific output—specifically, the number of local publications—and specialization, as defined by the Revealed Comparative Advantage (RCA). Additionally, we investigate the diffusion of international knowledge into the local ecosystem by analyzing the positioning of Canadian cities in the collaborative network of AI research. This entails assessing cities’ eigenvector centralities in national and international co-publication networks to understand their positioning. In particular, we analyze hospital centrality in the international network of hospitals engaged in AI technologies to delineate information flows among hospitals utilizing AI.

Finally, we conduct a regression analysis to account for other potential factors that may influence the correlations between performance and deployment in hospitals. Specifically, we perform a series of OLS regressions focused exclusively on data from Canada, as well as on a global dataset encompassing all countries included in our study (see On-line Appendix A.5.1 and A.5.2). Our control variable included the population size of CMAs, used as a rough proxy for economic development, concentration of economic activities, and research funding opportunities. We include as independent variables output measures and network measures. We do not include specialization measures as the descriptive analysis shows their lack of relevance. For comprehensive details regarding our methodology, please consult the On-line Appendix (Download here).

Results

In line with findings from previous research,^15,16 our analysis reveals a marked exponential growth in the number of AI publications globally, as illustrated in Figure A1 (See On-line Appendix). The United States (US), China (CN), and the European Union (EU) are at the forefront of this expansion. Canada (CA) exhibits a growth pattern that parallels the US and EU trends in terms of AI publications. Especially, there is a noticeable shift in the global distribution of AI research, as the proportional contributions of the US and EU (and Canada) are gradually declining in contrast to the rising shares from China and other emerging participants in the field.

Figure 1 (first row, left panel) shows a strong correlation (0.80) between hospital AI publication output in the period 2017-2021 and papers in AI published by authors affiliated only to PR in the period 2000-2016. The graph includes only 27 CMAs that had at least one hospital publication in the final period and 100 publications in PR before 2017. Eleven of the total 38 CMAs involved show no evidence (as measured by publications) of competences/involvement in AI at the hospital level. Toronto and Montreal, the two largest Canadian CMAs (respectively, 6.2M and 4.3M inhabitants in 2021), outperform all other CMAs considering publication output in AI in hospitals in the period 2017-2021. Toronto is the CMA with the highest output with about 2,605 articles. The leading role of Toronto is confirmed by the results of the Newsweek World’s Best Smart Hospitals 2021 ranking ¹ ; of the 12 Canadian hospitals included, 9 are in the Toronto CMA and the other 3 are in Montreal, Ottawa, and Vancouver. Two distinct clusters including, respectively, 6 and 8 CMAs follow the first two CMAs. In cluster 3, Waterloo stands out as the good performance of PR researchers in AI (at the level of Ottawa) is less correlated to AI research in hospitals. Finally, a fourth cluster is composed by CMAs that published less than one paper a year in AI with hospital affiliation in the last period, indicating very weak activity in that area. Clusters 2, 3, and 4 are composed by CMAs of different average size, with an average population of about 1.4M, 0.5M, and 0.2M, respectively.OPEN IN VIEWER

All CMAs in the top three groups have a medical school, while the two smallest medical schools of St. John’s and Sudbury/Thunder Bay are included in the lower performance group 4. Medical school sizes in the second and third clusters are not significantly different. These results indicate that the existence of a minimum size (more than 100 students) medical school in the CMA is a precondition for having some AI research activity in health; however, the size of the medical school does not discriminate between the two mid-performance clusters 2 and 3. We have repeated the same analysis (see Figure 1, first row, right panel) looking at company publications. Companies publish much fewer papers in AI; for example, the highest number of publications by companies is in Toronto with about 800, while the highest number of publications by public research organizations is in Montreal with about 17,000. However, the correlation coefficient of 0.78 is similar to the one of public research organizations. Moreover, there is some change in the relative position of the CMAs in the 3 clusters. These results indicate that companies have varying roles in the ecosystem supporting the deployment of AI in hospitals according to the different CMAs.

AI specialization at the metropolitan area level is not correlated to the deployment of AI in hospitals (see Figure A2 in the On-line Appendix). The only information worth noticing is the high relative specialization of Waterloo in AI PR research (compared to other CMAs in cluster 3) that is not mirrored in a higher diffusion of AI in hospitals. Overall, relative specialization does not provide any added explanatory power to understand what supports AI deployment in hospitals.

The analysis of collaborative networks in AI research, illustrated in Figure 2, shows that hospitals in Toronto, Montreal, and Vancouver are among the most connected and occupy central and influential positions in the international collaboration network. Our findings (see Figure 1, second row, right panel) reveal a robust positive correlation between the centrality of CMAs in terms of collaboration between public research and businesses in AI during the initial period and the subsequent increase in AI paper production within hospitals. This suggests that cities linked as major hubs for AI production experience heightened engagement in AI research within hospital settings, implying a diffusion of AI knowledge into healthcare contexts. When examining collaboration networks focusing solely on papers in AI authored by hospital-affiliated researchers, a stronger positive correlation emerges between the centrality of CMAs and subsequent AI production in hospitals. This higher correlation indicates the more important role played by the hospital network in diffusing AI knowledge into healthcare systems. It also indicates the importance of internal drivers and capabilities of hospitals required for the absorption of external knowledge. However, much of this result is driven by the particular role played by Toronto, Montreal, and Vancouver, the only Canadian CMAs highly connected into the international hospital network.OPEN IN VIEWER

Finally, the results of the regression analysis indicate the significant role of the local ecosystem, encompassing both public research organizations and companies, with the former being much more relevant than the latter. External sources of knowledge also play a vital role, with centrality in the hospital network in AI research in the past being strongly correlated with current AI activity in hospitals in the full model specification.

Conclusions

Using global AI publications as a proxy for organizational capabilities, this article presents the first systematic quantitative analysis of AI deployment within Canadian hospitals. Our findings reveal a correlation between the utilization of AI technologies in healthcare at the metropolitan level in Canada and the presence of a local ecosystem of both public and private research. Beyond local competencies, hospitals’ capacity to leverage the national and international network of AI technology users in hospitals also emerges as a significant factor.

We validated the Canadian findings by examining approximately 3,000 metropolitan areas worldwide. Drawing from this extensive sample, we confirmed that the presence of AI companies in a metropolitan area correlates with increased AI activity in hospitals, albeit with an impact magnitude approximately 4 to 6 times smaller than that of public research. Notably, the magnitude of this effect depends on the proxy used to gauge AI deployment in hospitals, which captures the utilization of AI technologies at the border between pre-clinical and clinical phases.

Furthermore, our analysis underscores the significance of external knowledge flows. However, the evidence suggests that it is more crucial for hospitals to establish connections with external hospitals utilizing AI technologies rather than with external public or private research organizations. This finding aligns with the perspective that sourcing knowledge from distant organizations is more complex than sourcing it locally. Consequently, the connection between hospitals (which is easier to manage due to affinity) yields more substantial outcomes compared to the link between hospitals and public or private organizations.

The positive hospital network effect is driven by the two highly internationally connected large cities of Toronto and Montreal. The third largest Canadian city, Vancouver, is well connected in the public research and companies’ network but not in the hospital network. The role played by hospital AI user networks underscores the importance of making simpler international collaborations among hospitals to facilitate the uptake of AI technologies. In the Canadian context, medical schools and hospitals of the second cluster should increase their international collaborations to successfully tap into the international hospital network that is the driving force in the diffusion of AI-based methods.

Supplemental Material