Yavuz ÖZDEN, Nuh Mehmet BÜYÜKBERBER
Annals of Clinical and Analytical Medicine - 2026;17(10):1066-1072
Aim: The Mayo Endoscopic Subscore (MES) in ulcerative colitis (UC) is limited by interobserver variability, whereas central reading is resource-intensive. We developed and temporally validated a video-based deep learning system for automated MES assessment. Methods: This retrospective diagnostic accuracy study included 1,000 adults with UC who underwent white-light colonoscopy between November 2020 and November 2025. The model was developed in 750 patients using five-fold patient-level cross-validation and evaluated after model freezing in a temporally separated cohort of 250 patients. Two expert gastroenterologists assigned MES 0 to 3 from full-length videos, with third-reader adjudication of discordant cases. An EfficientNet-B0 model with attention-based multiple instance learning generated predictions. The primary outcome was four-class MES classification; dichotomous endpoints were MES 0 versus MES >=1 and MES <=1 versus MES >=2. Results: Pre-adjudication interobserver agreement was high (quadratic weighted kappa, 0.84; 95% CI, 0.81-0.87). In temporal validation, macro-averaged AUROC was 0.95 (95% CI, 0.93-0.97), overall accuracy was 83.6%, macro-F1 score was 0.84, and quadratic weighted kappa was 0.79 (95% CI, 0.68-0.87). AUROCs were 0.97 (95% CI, 0.95-0.98) for MES 0 versus MES >=1 and 0.96 (95% CI, 0.94-0.97) for MES <=1 versus MES >=2. MES 1-2 errors accounted for 58.5% of classification errors. Brier scores were 0.07 and 0.06, respectively, and decision-curve analysis demonstrated a positive net benefit. Conclusions: The model demonstrated high discrimination, good agreement with the adjudicated reference standard, good calibration, and interpretable errors. Multicenter external validation and prospective studies are required before implementation.
