Comment on “Can ChatGPT detect breast cancer on mammography?”

Dear Editor,

We would like to comment on the recently published “Can ChatGPT detect breast cancer on mammography?” ¹ The purpose of this study was to investigate the capabilities of the ChatGPT artificial intelligence (AI) program, notably in the domain of mammography (MMG) image processing, for the diagnosis of breast cancer before treatment in patients with pathologically confirmed invasive breast cancer with obvious lump formation. However, the research methodology used has several statistical limitations, including the fact that the sample size was not specified, the number of cases analyzed influenced the reliability of the kappa measurement, and the analysis using only two ChatGPT subprograms without comparing them to specialized AI models in radiology may not be sufficient to draw clear conclusions about the capabilities of these large language models (LLMs).

Furthermore, the use of consensus assessment by two radiologists may bias the data or exclude assessments from more diverse views. There are no published confidence ranges for the results, and the quality or resolution of the MMG images used is not specified, limiting the model's ability to evaluate. Furthermore, because ChatGPT is trained from text rather than images, image analysis via data transformation may result in fundamental restrictions for the model.

In terms of extension and innovation, future research might combine the LLM with a vision transformer or convolutional neural network models trained specifically for medical imagery. Another technique for developing a hybrid model that can effectively assess both images and text is to use image data from many sources, including real-world settings, which will help make the results more general and widely applicable. Issues that should be widely debated include the appropriateness of using LLMs such as ChatGPT for interpreting medical images that require high accuracy, the feasibility of using LLMs as report summaries rather than direct diagnosis, and the ethics of using AI in screening for cancer, which, if misdiagnosed, can have serious consequences for patients. To maximize the benefits of AI in medicine, norms and criteria for its application must be carefully developed in partnership with clinicians.