Table 4

Performance of models applied without deviations

StudyValidation performance—reported from studyValidation performance—NYU dataPerformance difference between study validation performance and NYU original validation performanceValidation performance—NYU retrained
(95% CI)
Performance difference between study validation performance and NYU retrained validation performance
Xie et al6AUROC=0.98AUROC=0.67AUROC difference=0.31AUROC=0.76 (0.72 to 0.80)AUROC difference=−0.22
Yan et al7Avg F1=0.97Most recent values: Avg F1=0.63F1 difference=0.34Most recent values: AUROC=0.95 (0.93 to 0.96)*
Yan et al7*Earliest values: Avg F1=0.51*Earliest values: AUROC=0.70 (0.65 to 0.75)*
Mean AUROC=0.98Mean AUROC=0.67Mean AUROC difference=0.31Mean AUROC=0.82Mean AUROC difference=−0.22
  • *Value unavailable because authors did not provide an AUROC value when reporting validation performance.

  • AUROC, area under the receiver–operator curve; NYU, New York University.