Study population<\/p>\n
For the model training, we used CFPs from the UKB and three hospital-based centers in China to maximize ethnic and demographic heterogeneity in the training dataset (Extended Data Fig. 6<\/a>). The UKB is an ongoing prospective cohort study with extensive phenotypic data26<\/a>. In brief, CFPs of the macula were obtained at enrollment using the Topcon 3D OCT-1000 Mark II system without pupil dilation at the baseline visit between 2009 and 2010. Participants were randomly split into two groups in a 2:1 ratio from six assessment centers for retrospective diagnostic modeling and predictive modeling while minimizing potential center-specific biases. Participants with CFPs and complete data on disease outcomes, and relevant covariates, were included in the analysis.<\/p>\n The hospital-based cohort was retrospectively established using data from three clinical centers: Guangdong Provincial People\u2019s Hospital, the Third Affiliated Hospital of Sun Yat-sen University and Linyi People\u2019s Hospital. CFPs of the macula were captured using a variety of standard fundus cameras, including the Topcon TRC-NW6, Canon CR6-45NM and Kowa Nonmyd \u03b1-DIII with pupil dilation.<\/p>\n For the external test, we used datasets from distinct healthcare settings in China and the multi-ethnic cohort SEED. The resource-limited dataset was retrospectively assembled from three clinical sites in China: The First Affiliated Hospital of Xinjiang Medical University, Linzhi People\u2019s Hospital of Tibet and the People\u2019s Hospital of Guangxi Zhuang Autonomous Region. This dataset included 1,373 participants (2,746 images) with dilated pupils. The high-resource dataset comprised retrospectively collected data from physical examination centers (2,003 participants and 4,006 images) and large tertiary hospitals (587 participants and 1,174 images), captured predominantly under nondilated conditions. Additionally, we evaluated the model on the SEED study, a population-based cohort encompassing Chinese, Malay and Indian adults in Singapore, as described previously27<\/a>. A total of 15,306 CFPs from 7,653 Singaporean adults (2,800 Chinese, 2,385 Malay and 2,468 Indian) were included in the external test set.<\/p>\n This study was conducted with approval from the Institutional Review Board of Guangdong Provincial People\u2019s Hospital, Guangdong Academy of Medical Sciences (no. KY-N-2022-134). For the UKB data component, ethical approval was obtained through the NHS National Research Ethics Service (ref. no. 11:\/NW\/0382; project ID: 86091). All participants provided written informed consent before study participation. To ensure confidentiality, all data were systematically de-identified by removing all personal identifiers before analysis.<\/p>\n Definition of endocrine and metabolic outcomes<\/p>\n To define osteoporosis, gout and thyroid disease at baseline, we used self-reported information and inpatient records using International Classification of Diseases, 10th Revision (ICD-10) codes, down to the three-character category in the ICD hierarchy (Supplementary Table 8<\/a>).<\/p>\n For the diagnosis of T2DM, hypertension and hyperlipidemia at baseline, we considered blood markers and medication use in addition to self-reported information and ICD-10 codes. T2DM was diagnosed in patients who had a physician-diagnosed case of diabetes mellitus, were using antihyperglycemic medications or insulin, or had a glycated hemoglobin level of \u226548\u2009mmol\u2009mol\u22121 (ref. 28<\/a>). Hypertension was defined by the use of antihypertensive medications and an average systolic blood pressure of at least 130\u2009mmHg or an average diastolic blood pressure of at least 80\u2009mmHg (ref. 29<\/a>). Hyperlipidemia was defined by the use of medications for hyperlipidemia or statins, or a total blood cholesterol level of \u22656.22\u2009mmol\u2009l\u22121 (ref. 30<\/a>).<\/p>\n For the prediction tasks, incident cases of individual diseases were identified using inpatient hospital records and mortality registers. Follow-up visits began on the date of attendance at the assessment center and continued until the earliest recorded date of diagnosis, the date of mortality, or the last available date provided by the hospital or general practitioner, whichever occurred first. Individuals with prevalent disease at baseline were excluded. A sensitivity analysis was performed excluding individuals who developed incident disease within the first year of follow-up.<\/p>\n Reti-Pioneer development and evaluation<\/p>\n Overall architecture of the proposed Reti-Pioneer framework is presented in Extended Data Fig. 1<\/a>. For feature extraction, Reti-Pioneer uses three large-scale pre-trained models, that is, Swin Transformer, Vision Mamba and RETFound, which remain frozen during training11<\/a>,12<\/a>,31<\/a>. To enable multimodal integration, two bilinear modules were used for features fusion, integrating image quality and clinical characteristics, including age, sex, ethnicity and weight32<\/a>,33<\/a>.<\/p>\n Specifically, the Reti-Pioneer framework introduces an integration of a quality-aware module into retinal image analysis, enabling the retention and informed use of low-quality CFPs, such as those affected by cataract-induced obscuration that may nonetheless reflect systemic endocrine status34<\/a>. Rather than excluding these images, the framework leverages image quality metrics to guide model decisions during the early processing stages35<\/a>. The quality-aware and clinical characteristic fusion modules use a bilinear operation followed by scaled exponential linear units (SELUs) and a linear layer. The bilinear module combines quality metrics (probability scores for good, usable, bad classifications) with F-dimensional deep features, generating fused features of identical dimension (F). This fusion was followed by a linear layer for disease probabilities. The SELU activation function was applied between intermediate layers to enhance training stability and maintain self-normalizing properties36<\/a>. Both SELU activation and linear transformation maintain this dimensionality, ensuring feature representation consistency.<\/p>\n Fine-tuning was applied to a diverse set of downstream tasks, including screening and prediction tasks. The generalizability of the Swin Transformer, Vision Mamba and RETFound models was systematically evaluated through independent external testing using bilateral CFPs combined with clinical characteristics. The unit of analysis for all model development and evaluation was the individual participant. For each participant, fundus images from both eyes were used as input, three trained heads were integrated via a weighted soft voting ensemble and fused by the model architecture to generate a single, unified prediction.<\/p>\n To quantify the contribution of its core components, we conducted a series of ablation studies on the internal validation set. The full Reti-Pioneer framework was compared against (1) an ablated version without the quality-aware fusion module, trained on all images, (2) the same ablated version trained exclusively on high-quality images and (3) unimodal baselines using either fundus images or clinical metadata alone. These comparisons were designed to isolate the performance gain attributable to the model\u2019s full multimodal architecture.<\/p>\n Explainability and biological plausibility<\/p>\n To understand the retinal features associated with endocrine and metabolic disorders, we used two complementary approaches. Saliency mapping was performed to identify and visualize spatially resolved regions of interest within CFPs that contributed most to the model\u2019s diagnostic and predictive decisions, after Gaussian filtering. We used the integrated gradient algorithm to produce visual explanations of model behavior, attributing prediction relevance to individual pixel37<\/a>,38<\/a>. The saliency maps highlight the relative contribution of each pixel to the final predictions, thereby enhancing model interpretability.<\/p>\n To investigate the biological basis of disease predictions, we leveraged plasma proteomic profiling data from UKB participants. Proteins were quantified using the Olink Explore 3072 platform, which integrates cardiometabolic, inflammation, neurology and oncology panels, capturing 2,923 unique proteins. Expression levels are reported as Normalized Protein eXpression on a log2 scale. After quality control, excluding proteins with >30% missing data (GLIPR1, NPM1, PCOLCE) and participants with >50% missing data, 2,920 proteins were retained for analysis. Further details on sample processing and quality control are described elsewhere39<\/a>. To identify disease-specific protein signatures associated with metabolic and endocrine diseases, we used a two-step analytical approach. First, elastic net regularization analysis was used for retinal latent feature selection, enabling dimensionality reduction and identification of key predictors from the Reti-Pioneer framework. Second, we prioritized candidate proteins based on their coefficient to refine the selection of biologically relevant biomarkers. The OPLS-DA model was performed using the variational autoencoder-derived retinal latent features to assess the separation between the control and case groups. The predictive component, capturing the systematic variance in the data that is directly correlated with the inter-group differences. We used logistic regression to test the impact of predictive components on the prevalence of the corresponding diseases. Pearson correlation analysis was used to quantify the linear relationships between individual protein levels (continuous) and the corresponding predictive components. The association between retinal latent features derived from OPLS-DA and the top-ranked plasma proteins across the six endocrine and metabolic diseases was further evaluated using logistic regression. Both unadjusted models and models adjusted for age, sex, ethnicity, body mass index, image quality and assessment center are presented.<\/p>\n Similarly, we assessed the association between OPLS-DA-derived retinal latent features and disease-relevant PRS using logistic regression. Standard PRS were obtained from the UKB, including scores for T2DM, HbA1c, hypertension, high-density and low-density lipoprotein cholesterol, total cholesterol, triglycerides, osteoporosis and estimated bone mineral density T-score. Similarly, both unadjusted models and models adjusted for age, sex, ethnicity, body mass index, image quality, assessment center and top four ancestry principal components are presented. For multiple comparisons between retinal latent features and top-ranked plasma proteins or PRS, the false discovery rate correction was applied.<\/p>\n Clinical pipeline of silent trial<\/p>\n The prospective silent trial was conducted between September 2025 and October 2025 to evaluate the technical robustness and seamless integration of the Reti-Pioneer framework within real-world clinical workflows under blinded conditions. Figure 3a<\/a> illustrates the deployed clinical pipeline for real-time multimorbidity detection. During the trial, the system routinely processed 30\u201350 cases per day from patients referred for endocrine, metabolic and hematological testing in a primary care setting. The screening performance of Reti-Pioneer for six endocrine and metabolic diseases was evaluated against a reference standard incorporating laboratory measurements, physical examination findings and self-reported history, and was compared with the FINDRISC questionnaire for T2DM. In the clinical workflow, bilateral fundus images from all eligible participants were systematically captured at the point of care, irrespective of operator-related or environmental variations in image quality, and transmitted in real time to a centralized GPU computing infrastructure for immediate analysis without disrupting routine clinical procedures. AI-generated predictions were not disclosed to clinicians or patients and did not influence clinical decision-making, maintaining the trial\u2019s blinded nature.<\/p>\n Key operational metrics were automatically recorded throughout the trial, including image acquisition success rate, AI model inference success rate and system throughput efficiency (defined as total time from image capture to report generation). These data enabled direct comparison between the Reti-Pioneer-assisted screening pipeline and the current standard workflow based on laboratory testing.<\/p>\n Prospective pilot study<\/p>\n A prospective pilot study was conducted between September 2025 and November 2025 at a community health service center and a physical examination center to evaluate the real-world impact of the Reti-Pioneer AI system on clinical practice, patient behavior and clinician-patient perceptions in an open-label setting. Participants undergoing routine physical examinations were recruited and offered the Reti-Pioneer AI test by their PCPs. CFPs and clinical metadata were collected from all participants, alongside comprehensive laboratory-based physical examinations. AI-generated predictions were provided to both clinicians and patients as part of the clinical information. Diagnostic consistency was assessed by calculating the sensitivity, specificity and accuracy of the model\u2019s outputs against a composite diagnostic standard incorporating laboratory results and self-reported medical history.<\/p>\n Satisfaction and acceptance among participants and clinicians were evaluated using structured questionnaires. Participants rated their experience on a 5-point Likert scale across multiple domains, including ease of use, report comprehensibility, clarity of management recommendations, overall satisfaction, willingness to reuse and recommend the service, perceived efficiency of multidisease screening, information load, perceived convenience compared to traditional testing and willingness to pay (Supplementary Table 9<\/a>).<\/p>\n Clinicians completed a separate acceptance questionnaire, also based on a 5-point Likert scale, assessing overall satisfaction, willingness to integrate the system into practice, motivation to conduct multidisease screening, workflow compatibility, perceived decision-support value, trust in AI predictions and suitability for resource-limited settings. The complete questionnaire items are detailed in Supplementary Table 10<\/a>.<\/p>\n Statistical analysis<\/p>\n Descriptive statistics are reported as the mean\u2009\u00b1\u2009s.d. for continuous variables and as frequency (%) for categorical variables. Discriminative performance was evaluated using AUROCs. Comparisons between AUROCs were conducted using the DeLong\u2019s test40<\/a>. Calibration was assessed using calibration plots (observed versus predicted risk using deciles). Overall performance was evaluated using the Brier score. Sensitivity, specificity, PPV and NPV for classification are reported with 95% CIs. CIs were obtained using the percentile bootstrap method on 1,000 bootstrap samples. To evaluate clinical utility across a range of risk thresholds, decision curve analysis was considered41<\/a>,42<\/a>.<\/p>\n Data analyses and visualizations were conducted under Python (v.3.12) and R (v.4.2.0). Reti-Pioneer was implemented on PyTorch (v.2.8.0) and optimized using the AdamW optimizer with a warm-up strategy and a cosine annealing learning rate scheduler. The construction, fine-tuning and testing processes were performed on an RTX 3080 GPU (10\u2009GB dedicated memory, CUDA v.12.8).<\/p>\n Reporting summary<\/p>\n