Characteristics of peer-reviewed protocols and completed RCTs evaluating artificial intelligence tools
| Characteristics | AI-RCTs (n=23) | Protocols of AI-RCTs (n=6) | Completed AI-RCTs (n=17) |
| No of centres, n (%) | |||
| Single | 11 (48) | 1 (17) | 10 (59) |
| Multicentre | 12 (52) | 5 (83) | 7 (41) |
| Geographic area, n (%) | |||
| Asia | 8 (35) | 2 (33) | 6 (35) |
| Europe | 5 (22) | 1 (17) | 4 (24) |
| North America | 9 (39) | 3 (50) | 6 (35) |
| Other | 1 (4) | 0 (0) | 1 (6) |
| Arms of randomisation, n (%) | |||
| Two | 20 (87) | 5 (83) | 15 (88) |
| Three | 3 (13) | 1 (17) | 2 (12) |
| Level of randomisation, n (%) | |||
| Patients | 22 (96) | 6 (100) | 16 (94) |
| Clinicians | 1 (4) | 0 (0) | 1 (6) |
| Sample size | |||
| Median (IQR) | 214 (108–571) | 298 (219–830) | 214 (100–437) |
| Min | 20 | 100 | 20 |
| Max | 22 641 | 18 000 | 22 641 |
| Power calculations, n (%) | |||
| Yes | 18 (78) | 6 (100) | 12 (71) |
| No | 5 (22) | 0 (0) | 5 (29) |
| Type of control intervention, n (%) | |||
| Standard of care | 22 (96) | 6 (100) | 16 (94) |
| Sham procedure | 1 (4) | 0 (0) | 1 (6) |
| Funding source, n (%) | |||
| Industry related | 4 (17) | 1 (17) | 3 (18) |
| Non-industry related | 10 (43) | 4 (66) | 6 (35) |
| None reported | 7 (30) | 1 (17) | 6 (35) |
| None | 2 (9) | 0 (0) | 2 (12) |
| Data sources, n (%) | |||
| Dedicated personnel | 5 (22) | 2 (33) | 3 (18) |
| Dedicated personnel and EHR | 4 (17) | 2 (33) | 2 (12) |
| EHR | 4 (17) | 2 (33) | 2 (12) |
| Not applicable | 4 (17) | 0 (0) | 4 (23) |
| Not specified | 6 (27) | 0 (0) | 6 (35) |
| Strategies for missing data, n (%) | |||
| Specified | 4 (17) | 4 (67) | 0 (0) |
| Not specified | 19 (83) | 2 (33) | 17 (100) |
| Primary outcome(s), n (%) | |||
| Binary | 7 (30) | 0 (0) | 7 (41) |
| Binary and continuous | 1 (4) | 0 (0) | 1 (6) |
| Categorical | 1 (4) | 1 (17) | 0 (0) |
| Continuous | 14 (61) | 5 (83) | 9 (53) |
| Primary outcome favours AI tool, n (%) | |||
| Yes | 13 (57) | 0 (0) | 13 (76) |
| No | 2 (9) | 0 (0) | 2 (12) |
| Not applicable | 8 (34) | 6 (100) | 2 (12) |
| Different geographic area of study population in development study and AI-RCT, n (%) | |||
| Yes | 3 (14) | 1 (17) | 2 (12) |
| No | 12 (52) | 1 (17) | 11 (65) |
| Not applicable* | 8 (34) | 4 (66) | 4 (23) |
| External validation of AI tool, n (%) | |||
| Yes | 11 (48) | 2 (33) | 9 (53) |
| No | 12 (52) | 4 (67) | 8 (47) |
| Different geographic area† | 0 (0) | 0 (0) | 0 (0) |
| Different time period† | 11 (48) | 2 (33) | 9 (53) |
*The respective development study was not identified.
†Compared with the development study.
AI-RCTs, artificial intelligence randomised controlled trials; EHR, electronic health records.