Table 5

Performance of models applied with deviations

StudyValidation performance—reported from studyValidation performance—NYU dataPerformance difference between study validation performance and NYU original validation performanceValidation performance—NYU retrained (95% CI)Performance difference between study validation performance and NYU retrained validation performanceNYU deviations from original
Gong et al5Cohort 1: AUROC=0.85AUROC=0.59AUROC difference=0.26AUROC=0.68
(0.64 to 0.74)		AUROC difference=0.17PaO2 data not available. Target excluded
Gong et al5Cohort 2: AUROC=0.75AUROC=0.60AUROC difference=0.16AUROC=0.68
(0.64 to 0.74)		AUROC difference=0.07PaO2 data not available. Target excluded
Zhou et al8AUROC=0.88AUROC=0.74AUROC difference=0.14AUROC=0.75
(0.71 to 0.78)		AUROC difference=0.14‘Severe respiratory distress’ target not characterisable. Target excluded
Zou et al9*AUROC=0.70*AUROC=0.72 (0.67 to 0.77)*Altered mental status measured as Glasgow Coma Score <15
 Mean AUROC=0.83Mean AUROC=0.66Mean AUROC difference=0.19Mean AUROC=0.71Mean AUROC difference=0.13 

  • *Value unavailable because authors did not provide an AUROC value when reporting validation performance.

  • AUROC, area under the receiver–operator curve; NYU, New York University; PaO2, partial pressure of oxygen.