Model performance summary
| Study type | Mean validation performance—reported from study | Mean validation performance—NYU data | Mean performance difference between study validation performance and NYU original validation performance | Mean validation performance—NYU retrained | Mean performance difference between study validation performance and NYU retrained validation performance |
| Applied without Deviation (n=3) | Mean AUROC=0.98 (n=1) | Mean AUROC=0.67 (n=1) | Mean AUROC difference=0.31 (n=1) | Mean AUROC=0.82 (n=3; 0.75–0.93) | Mean AUROC difference=0.21 (n=1) |
| Applied with deviation (n=4) | Mean AUROC=0.83 (n=3; 0.75–0.88) | Mean AUROC=0.66 (n=4; 0.59–0.74) | Mean AUROC difference=0.19 (n=3; 0.14–0.26) | Mean AUROC=0.71 (n=4; 0.68–0.74) | Mean AUROC difference=0.13 (n=3; 0.07–0.17) |
| Rebuilt without deviation (n=2) | Mean AUROC=0.73 (n=1) | – | – | Mean AUROC=0.77 (n=2; 0.71–0.80) | Mean AUROC difference=0.01(n=1) |
| Rebuilt with deviation (n=4) | Mean AUROC=0.78 (n=4; 0.72–0.88) | – | – | Mean AUROC=0.75 (n=4; 0.72–0.79) | Mean AUROC difference=0.03 (n=4; −0.01–0.09) |
–=Value unavailable because authors did not provide feature weights when reporting model development.
AUROC, area under the receiver–operator curve; NYU, New York University.