Artificial intelligence projects in healthcare: 10 practical tips for success in a clinical environment

1. Build a collaborative science team

No individual has all the skills and resources needed to make an AI healthcare project succeed so a collaborative science team (CST) is essential. The team’s composition may vary but includes HCPs, data scientists and possibly statisticians, project managers and software engineers. This is the most important tip; the shared knowledge of the CST overcomes many barriers presented to an AI healthcare project. Some barriers are obvious to a data scientist, some to a HCP and vice versa. We propose that such teams are the building blocks of what Cosgriff et al have described as the clinical AI departments of the future.6

It is also important to identify the project ‘gatekeepers’ from the outset; senior individuals who may be aware of organisational barriers to success. They could be heads of strategy, department leads or chief informatics officers. Early meetings with gatekeepers are essential; they may recommend additional team members and can access useful resources if a project proposes to change clinical pathways.

2. Engage frequently with the end user

HCPs often have little or no experience with AI. Many will be unfamiliar with the capabilities and limitations of AI7 which can lead to both unachievable expectations or unfair dismissal from the outset. In addition, HCPs may struggle to specify in advance how they want AI solutions to improve their workflow. Some may have never even mapped out or quantified their daily work. What they say they want may change as the project develops. There is a complex interaction of human factors at play which CSTs must not ignore.

Healthcare AI projects must therefore strive to offer digital transformation through ‘organisational, service and social innovation’ as suggested by Creswell et al.8 The clinical AI departments of the future must be prepared to offer in-reach into the other clinical departments. Defining the problem to be solved may take considerable time and may require CSTs to study clinical workflows carefully, perhaps even shadowing HCPs. In addition to building the solution, consideration should be given to how it will be employed. For example: how it is presented in a human understandable way,9 its impact on other workflows, on patient–caregiver interaction, the risks of automation complacency10 and of biasing clinical decision making.4

End user engagement is not a single event. We encourage CSTs to adopt an agile approach, where continuous user engagement with rapid modifications creates healthcare tools of real value. There is a tension here between agile development and ensuring patient safety as an AI solution is implemented and refined.

3. Build collaboration agreements early

Although healthcare organisations and universities frequently collaborate, overarching agreements for multisite working and secure data sharing are rare. Therefore, collaboration agreements need to be developed early to avoid major delays later. Conversely, generating new agreements for every project is inefficient. Creating communities of practice can facilitate dialogue between CSTs, supporting shared solutions or lobbying both organisations for high-level agreements.

Data-flow diagrams are an important part of any collaboration agreement (figure 2). They help design a transparent and detailed application for ethical review, and aid in the formation of a data management plan, which is a prerequisite for collaboration.

Request permission

Many healthcare organisations and universities have specific teams to help create collaboration agreements. Such teams can also advise on project costs, intellectual property, publication rights, data-sharing agreements and information governance. While drafting agreements, it is important to identify the data controller, who designs and supervises the project, and data processors, who carry out processing tasks under instruction from the controller.

4. Ethics: present a balanced view

Ethical oversight of AI-driven healthcare research can be problematic. There are numerous tension points that need to be resolved including the role of patient consent in the use of routinely collected healthcare data,11 the potential for under-representation of certain patient groups in AI projects12 and concerns about adaptive algorithms whose performance may change with new data. Most approval committees still have limited experience of AI and the review process may be ill-suited to AI-driven research.

The ethical review process can be improved if CSTs follow best practice such as the Standard Protocol Items: Recommendations for Interventional Trials-AI (SPIRIT-AI) protocol guidelines.13 A moral justification for transitioning to learning health systems supported by data science has been proposed,14 which may also help to inform the review process. A balanced view of AI healthcare projects should be offered; while AI presents ethical challenges it also offers opportunities to enhance the quality of existing medical evidence in areas where evidence is lacking or subject to bias.12

CSTs should be aware that ethical review may be protracted and should consider seeking review by committees with experience of similar projects

5. Invest in data science training for healthcare professionals in your team

There is currently a lack of data science training in healthcare education programmes. Where training programmes do exist, they are often optional. The knowledge demanded of HCPs focuses on interpretation of research evidence, screening tests and the output of randomised controlled trials. Although this is an excellent foundation for traditional medical research, it does not provide a common language for data-driven clinicians of the future. Online supplemental appendix 2 offers a glossary to help build this common language.

The Topol Review15 emphasised the need for data science training if NHS staff are to reap the benefits of the digital revolution. There are a range of resources that HCPs can use to learn about the principles of tidy data,16 how to categorise and address missingness in their data17 and to gain an overview of AI techniques and their limitations.7 There are communities of practice18 who can offer support and growing opportunities for additional self-directed learning.19

We recommend that HCPs pay attention to these learning needs and use the expertise of data scientists in their team to guide them. Such preparation can prevent errors of understanding and improve interactions throughout an AI healthcare project.

6. Don’t underestimate the challenge of data extraction

Figure 2 illustrates a process of data extraction and linkage in critical care where significant effort was needed to link three disparate data sets before any AI solutions could be considered. Sharma et al20 have highlighted how this lack of data integration is a major barrier to the success of clinical risk prediction tools. HCPs use a huge range of information technology (IT) systems which are not always integrated and are often supported by paper records. These systems were designed for healthcare delivery and manufacturers may offer limited support for research. Furthermore, there is no universal standard for coding and capture of healthcare data despite laudable efforts to promote this.21 22 Coupled with local variation,23 this makes it hard to aggregate or extend data sets between institutions. CSTs should anticipate the need for extensive data wrangling to align data sets from different systems and institutions for use in their research.

Faced with this complexity, help from healthcare IT teams is essential both to access the data, which is often stored on secure servers, and to navigate the data structure, which is complicated in most electronic health records. Unfortunately, many healthcare IT teams are overwhelmed by day-to-day operational work leaving little time to support healthcare AI projects.

CSTs should make allowances for these data encoding and extraction challenges. Accurately defining data required for the project may improve the interaction with hospital IT teams and avoid the need for time-consuming, repeated extraction. Consider promoting and contributing to a meta-data catalogue at your institution,24 so that other CSTs can benefit from the extraction work you have done.

Identifying a ‘data champion’; a gatekeeper (tip 1) who supports the proposed project and can leverage the IT team can be helpful. Even better is to make the healthcare IT team part of the CST. CSTs are often seen as ‘service users’ by healthcare IT teams. Try to move beyond this relationship by recognising and rewarding the contribution of the healthcare IT team in the research.

7. Protect patients’ data

At present, most patients support the use of their data to improve healthcare,25 but large-scale leaks of confidential information could easily damage trust. Patient data must be anonymised or pseudonymised prior to analysis or storage somewhere other than where it was collected. However, the proliferation of digital data increases the risk of inadvertent release of identifiable information, or identification through deductive disclosure.26 Moreover, healthcare records often contain unexpected identifiers such as pixelated names in image files or names in narrative entries.

One solution is to bring data scientists and data processing capabilities into the healthcare organisation. An alternative is a trusted research environment in which identifiable patient data are stored with the required analysis tools.27 Such environments incorporate strict access controls that mitigate many anonymisation concerns. Figure 2 illustrates a hybrid approach in which role-based access control to a research database applies deidentification at source depending on a user’s credentials.

Regardless of approach, the aim is to ensure that data are extracted in a manner that allows fluid access for analysis, while adhering to information governance regulations. Protecting patient data is a process, not a one-time event. Teams should develop a strategy to prevent the inadvertent leakage of patient information and consider how they will periodically test and monitor its effectiveness.

8. Remember that healthcare data varies in quality and reliability

Automatically captured observations, such as clinical images or continuous vital signs exemplify the ‘cleanest’ sources of healthcare data. At the other end of the spectrum are those that rely on manual input (eg, subjective examinations and medical history). As Sujan et al10 note, not all events are captured in the medical record which may make it hard for AI algorithms to function optimally. Furthermore, an individual’s healthcare record rarely describes the wider context of their care. Periods of bed pressure and emergency situations are examples of circumstances that may affect an individual’s record, for example, by influencing the timing of hospital discharge. This wider context reduces the reliability of some seemingly definitive, time-specific events. It can help to try to identify ‘anchors’:28 events in the healthcare record that are independent of these human factors.

Paying attention to data quality and reliability can pay huge dividends. It has been demonstrated that data cleaning and artefact removal can yield greater improvements in model prediction than applying more sophisticated AI algorithms to the original data.29 CSTs should go to considerable lengths to address missing values and artefacts and to understand the clinical nuances of their variables.

9. Design AI that can be trusted with patient care

AI healthcare projects must be trusted to enable safe patient care. Engaging frequently with the end user (tip 2) during the project and educating HCP in data science techniques (tip 5) can go a long way towards this aim. However, the CST should also ensure that they tailor their algorithm to the data available, promote transparency where possible and make it clear how the AI solution will support rather than replace HCPs. In meeting these aims, the UK government’s code of conduct for data-driven health technology,30 research standards such as CONSORT-AI31 and clinician focussed checklists9 are valuable resources.

CST’s should tailor the algorithms they use towards characteristics of the data set available. For example, deep neural networks offer excellent performance with large data sets but are prone to overfitting when used for smaller, high dimensional data sets, which may be the only option in some clinical settings. A plethora of other techniques including random forests, linear regression, support vector machines and K nearest neighbours7 can be utilised and ensembled to produce tangible results in these circumstances.

Choosing the correct algorithm also involves addressing the concerns of HCPs who may worry about the use of AI algorithms in clinical practice if they cannot explain them.7 In this context, so-called black-box solutions32 may need to be avoided. Alternatively, CSTs can use a variety of techniques such as graphical explanations, model-agnostic explainers, heuristic examples or counterfactual explanations that may offer insight into how a black-box model reached its conclusions.33

Degradation of algorithm performance must be mitigated to minimise error and patient safety issues. The CST must ensure that AI algorithms are as generalisable as possible, functioning equitably for all patient groups and across geographic regions. Algorithms should also demonstrate robustness to the introduction of new data (eg, not prone to ‘catastrophic forgetting’34). In order to achieve this, it is paramount that there is expertise within the CST to internally validate AI algorithms. There should be a clear plan to externally validate the algorithm with unseen data, using an appropriately sized demographic and geographic cohort across an appropriate time frame.

10. Be mindful of medical device regulation

Many CSTs may be uncertain about the regulatory status of the AI solutions they develop. Existing US and EU guidelines are reasonably clear:35 AI software is classed as a medical device if its purpose is specifically medical and its function goes beyond data curation and transmission. There are subtleties related to ‘intended purpose’ and what constitutes a ‘medical purpose’ that are beyond the scope of this paper; for an excellent summary see Ordish et al.35

Medical device regulation is unlikely to impede the initial phases of AI development, but has significant implications for marketing or clinical application. Regulatory challenges include the adaptive nature of some algorithms, where outputs change substantially as new data are processed and the aforementioned black-box dilemma. Initiatives such as the Food and Drug Administration (FDA) AI framework36 and the National Institute for Health and Care Excellence (NICE) evidence standards framework for digital health technologies37 have shifted the emphasis to the quality of the algorithm development process, along with postimplementation quality assurance measures, rather than the need to meet medical device standards at a specific time.

CSTs should be mindful of such regulations during project planning. The NHS X AI governance framework38 is an example of best practice principles that may aid this process.