Post by INSIGHT Eye Hub

Despite the increasing number of medical AI foundation models being developed, there is limited understanding of how the composition of pre-training data, the “guts” of these models, affects capabilities such as generalisability (predicting accurately on both training and unseen data) and fairness (performing equally on data from different demographic groups).   To investigate this underexplored issue, our research group here at UCL Institute of Ophthalmology and Moorfields Eye Hospital, London worked with UK and international collaborators to train parallel retinal foundation models using two separate datasets.   They pre-trained the two models on 904,170 fundus photographs: one with data from INSIGHT Eye Hub at Moorfields and the other with data from the Shanghai Diabetes Prevention Program. Both models were adapted to downstream tasks for disease detection and prediction, and analysed for the fairness of their performance according to age, sex, and ethnicity. The results show that retinal foundation models exhibit strong generalisability. Although developed with distinct training data, they have comparable performance when adapted for different detection and prediction tasks. In the experiments, pre-training data attributes impacted model fairness differently: age distribution was associated with significant difference in performance across age subgroups, while not observed with ethnicity or sex subgroups.    Now in pre-press and available to download at Nature Communications, Nature Portfolio. https://lnkd.in/eMwBgREU Work by Yukun Zhou, Zheyuan Wang, Yilan W., Ariel Yuhan Ong, Siegfried Wagner, Eden Ruffell, Mark Chia, Zhouyu (Patrick) Guan, MD, Lie Ju, Justin Engelmann, David Merle, Tingyao Li, Jia Shu, Paul Nderitu, Ke Zou, Jocelyn Hui Lin Goh, Qingshan Hou, Xiao Liu, Yaxing Wang, Yih Chung Tham 覃宇宗, Andre Altmann, Carol Cheung, Daniel Alexander, Eric Topol, MD, Alastair Denniston, Tien Yin Wong FRS 黄天荫, Bin SHENG, Pearse Keane   Figure reproduced from the paper under Creative Commons Attribution 4.0 International License https://lnkd.in/gdWq3CF