Study Analyzing Transparency of Clinical Evidence for AI Medical Devices in Radiology Published in the Japanese Journal of Radiology

Research involving Medical AI Promotion Institute, Inc. (Head Office: Chuo-ku, Tokyo; Director General: Yuki Shimahara; Representative Director: Sho Kawabe; hereinafter “MAPI”) has been published in the Japanese Journal of Radiology, the official English-language journal of the Japan Radiological Society.

This study systematically organized and analyzed clinical evidence described in package inserts for AI-based medical devices (Software as a Medical Device: SaMD) in the radiology field that have been approved in Japan.

Background

In recent years, medical devices utilizing artificial intelligence (AI) have been rapidly introduced, particularly in radiology, and the number of approvals by the Pharmaceuticals and Medical Devices Agency (PMDA) (Note 1) has been increasing in Japan.

However, the performance of AI medical devices may vary depending on patient characteristics, imaging conditions, and clinical environments. Therefore, sufficient disclosure of clinical evidence is essential to ensure appropriate use in clinical practice.

Study Overview

This study conducted a scoping review (Note 2) of PMDA-approved SaMD products using AI technology in radiology as of December 31, 2024.

Unlike previous studies relying on academic publications or manufacturer materials, this analysis used only information described in package inserts, which are the primary reference for healthcare professionals in clinical settings.

The study systematically reviewed product characteristics, clinical indications, study designs used for approval, sample sizes, patient characteristics, and disclosed performance metrics, with the aim of clarifying the transparency and scope of clinically available information.

Main Findings

Among 151 PMDA-approved SaMD products, 20 were AI-based radiology products.

The analysis found that no product fully disclosed comprehensive patient background information such as age, sex, and nationality.

There was also significant variability in performance metrics disclosure, making direct comparison between similar products difficult using package insert information alone.

Although physician-assist studies demonstrated improved diagnostic performance when AI was used, stratified analyses by patient characteristics were not disclosed in any product.

Future Outlook

Leveraging its expertise in the Next-Generation Medical Infrastructure Act and personal data protection regulations, MAPI provides regulatory and R&D support while considering the transparency challenges of clinical evidence for AI medical devices.

MAPI will continue to support evidence development and validation necessary for trusted and sustainable clinical implementation of medical AI, from research through deployment.

Publication Information

• Journal: Japanese Journal of Radiology
• Title:
  Scoping review of regulatory transparency in AI-based radiology software: analysis of PMDA-approved SaMD products
• Authors:
  Tomohiro Kikuchi, Shannon L. Walston, Hirotaka Takita, Yasuhito Mitsuyama, Rintaro Ito, Masahiro Hashimoto, Takeshi Nakaura, Hiroaki Hyakutake, Sho Kawabe, Harushi Mori & Daiju Ueda
• DOI:
  10.1007/s11604-025-01942-y
• Link:
  https://link.springer.com/article/10.1007/s11604-025-01942-y

Note 1: Pharmaceuticals and Medical Devices Agency (PMDA)
Note 2: A research method that systematically maps and summarizes existing evidence