Performance of models rebuilt with deviations
| Study | Validation performance—reported from study | Validation performance—NYU retrained (95%CI) | Performance difference between study validation performance and NYU retrained validation performance | NYU deviations from original |
| Zhang et al12 | Task 1: AUROC=0.74 | Task 1: AUROC=0.78 (0.69 to 0.86) | AUROC Difference=+0.04 | ECMO, ARDS and intubation targets excluded. These targets were excluded during original validation |
| Zhang et al12 | Task 2: AUROC=0.72 | Task 2: AUROC=0.69 (0.64 to 0.73) | AUROC difference=−0.03 | ECMO, ARDS and intubation targets excluded. These targets were excluded during original validation |
| Guo et al13 | AUROC=0.78 | AUROC=0.79 (0.74 to 0.84) | AUROC difference=+0.01 | PaO2 and radiographic progression data not available. Circulatory shock and multiorgan dysfunction not characterisable. ICU admission used as surrogate target |
| Hu et al14 | AUROC=0.88 | AUROC=0.74 (0.69 to 0.78) | AUROC difference=−0.14 | PaO2 data not available. Target excluded |
| Mean AUROC=0.78 | Mean AUROC=0.75 | Mean AUROC difference=0.03 |
ARDS, acute respiratory distress syndrome; AUROC, area under the receiver–operator curve; ECMO, extracorporeal membrane oxygenation; ICU, intensive care unit; NYU, New York University; PaO2, partial pressure of oxygen.