1st workshop on Medical Video AI Assessment and Uncertainty Quantification

Call for Papers

Introduction

Artificial Intelligence and Machine Learning (AI/ML)-enabled medical devices are advancing rapidly to address the needs of patients, clinicians, and manufacturers in the MedTech industry. However, the pace of technological innovation has outstripped the development of evaluation methods in some instances, creating challenges for developers and for regulatory bodies charged with ensuring safety and efficacy.

The workshop aims at introducing regulatory science for AI-enabled devices to the ISBI community, with a focus on the assessment of medical video AI and methods for uncertainty quantification. These areas present unique technical challenges, including frame-to-frame variability, motion artifacts, and temporal consistency. We aim to foster dialogue among researchers, clinicians, and regulators to discuss technical and regulatory science challenges and help reduce the gap between development of novel AI technologies and their clinical adoption. We will discuss the development of regulatory science tools, testing methods, and metrics for assessing AI-assisted devices. While this workshop will focus on medical video AI systems, assessment many of the concepts generalize to other device types.

Rationale

The rapid adoption of AI across a broad range of medical devices highlights the need for robust evaluation frameworks and assessment approaches tailored to AI-based technologies. Critical regulatory science challenges include assessing generalizability and uncertainty. Addressing these challenges is essential to support evidence-based regulatory decision-making and to ensure that safe and effective AI-enabled medical devices are developed by innovators.

Extending these considerations to the domain of medical video AI introduces additional complexities. Medical video AI systems are relatively new, designed to assist care providers by improving the identification of abnormalities within temporal streams of imaging data. Taking the colonoscopy computer-aided detection (colon-CADe) devices as an example, six colon-CADe devices are authorized for marketing in the US. Disease detection in video imaging procedures differs from many radiology AI applications which typically operate on static images and potentially non-real time image interpretation. In contrast, video AI systems must handle challenges such as frame-to-frame variability, motion artifacts, and lesion persistence across time, and often the need for real-time prompting as diagnosis are commonly made during the clinical procedure/video collection such that appropriate timely intervention can be performed. It remains unclear which specific performance metrics and standalone study protocols are adequate for comparing colon-CADe algorithms as AI testing protocols and metrics characterizing video-based performance are still evolving. While many ISBI workshops have explored algorithm development, novel applications and platforms, a significant gap remains tackling practical assessment challenges, real‐world deployment hurdles, and core regulatory science questions necessary for the development of innovative technology.

If you are working on research related to medical AI, clinical deployment, or evaluation, then: Medical Video AI Assessment and Uncertainty Quantification is the right place for your work!