OpenClinical.net: Artificial intelligence and knowledge engineering at the point of care ======================================================================================== * John Fox * Matthew South * Omar Khan * Catriona Kennedy * Peter Ashby * John Bechtel ## Abstract **Objective** OpenClinical.net is a way of disseminating clinical guidelines to improve quality of care whose distinctive feature is to combine the benefits of clinical guidelines and other human-readable material with the power of artificial intelligence to give patient-specific recommendations. A key objective is to empower healthcare professionals to author, share, critique, trial and revise these ‘executable’ models of best practice. **Design** OpenClinical.net *Alpha* ([www.openclinical.net](http://www.openclinical.net)) is an operational publishing platform that uses a class of artificial intelligence techniques called *knowledge engineering* to capture human expertise in decision-making, care planning and other cognitive skills in an intuitive but formal language called PRO*forma*.3 PRO*forma* models can be executed by a computer to yield patient-specific recommendations, explain the reasons and provide supporting evidence on demand. **Results** PRO*forma* has been validated in a wide range of applications in diverse clinical settings and specialties, with trials published in high impact peer-reviewed journals. Trials have included patient workup and risk assessment; decision support (eg, diagnosis, test and treatment selection, prescribing); adaptive care pathways and care planning. The OpenClinical software platform presently supports authoring, testing, sharing and maintenance. OpenClinical’s open-access, open-source repository *Repertoire* currently carries approximately 50+ diverse examples ([https://openclinical.net/index.php?id=69](https://openclinical.net/index.php?id=69)). **Conclusion** OpenClinical.net is a showcase for a PRO*forma*-based approach to improving care quality, safety, efficiency and better patient experience in many kinds of routine clinical practice. This human-centred approach to artificial intelligence will help to ensure that it is developed and used responsibly and in ways that are consistent with professional priorities and public expectations. * BMJ Health Informatics * computer methodologies ## Introduction ### The knowledge crisis Every week, hundreds of papers by expert medical researchers are published in high-quality journals and this research fuels continuous improvements in treatments and medical practice. New knowledge produced through research is often summarised and disseminated in the form of clinical practice guidelines (CPGs) which give short, evidence-based summaries of best practice for specific medical conditions. CPGs are generally considered to be a vital way of disseminating up-to-date recommendations for high quality and safe clinical practice. However, there are important difficulties for achieving all the potential benefits of CPGs: they take time to read and absorb; they are difficult to keep up to date, and they only provide general guidance not patient-specific recommendations. Chidgey *et al*1 reviewed the successes and issues facing the UK National Institute for Health and Care Excellence (NICE) in improving the delivery of medical services in the National Health Service. The focus was the NICE programme of guideline development, ‘arguably the largest in the world’, whose goal is to carry out rigorous and up-to-date reviews of evidence for alternative treatments and develop recommendations for practice. Chidgey *et al* discussed successes in changing practice but concluded that the record of translating guidance into successful implementation was ‘mixed’. The NICE guidance programme has grown to cover approaching 500 areas of medical practice and diversified into a wide range of products: CPGs, clinical pathways, quality standards, technology assessments, evidence summaries and more. NICE is presently in the process of reviewing its guidance development processes in a programme called *NICE Connect* ([https://www.nice.org.uk/about/who-we-are/nice-connect](https://www.nice.org.uk/about/who-we-are/nice-connect)) whose goal is to facilitate more effective use of NICE content in clinical practice. In response to the emergence of ‘Rapid Learning Systems’ in healthcare and ‘Computable Biomedical Knowledge’, NICE is also investigating options for disseminating knowledge of best practice in a computable form that can support individualised care. There are many options to be considered, from restructuring and enriching guideline documents so they can be read by computers and applied in more context specific ways (eg, GEM Guideline Elements Model ([https://www.astm.org/DATABASE.CART/HISTORICAL/E2210-02.htm](https://www.astm.org/DATABASE.CART/HISTORICAL/E2210-02.htm)), MAGIC enriched guideline content ([http://magicproject.org/](http://magicproject.org/))) to more technical approaches like statistical and knowledge-based decision-support systems ([https://en.wikipedia.org/wiki/Clinical\_decision\_support\_system](https://en.wikipedia.org/wiki/Clinical\_decision\_support_system)); rule-based systems for making patient-specific recommendations (eg, Arden syntax ([https://en.wikipedia.org/wiki/Arden_syntax](https://en.wikipedia.org/wiki/Arden_syntax)), Clinical Quality Language ([https://ecqi.healthit.gov/cql](https://ecqi.healthit.gov/cql))) to *Computer Interpretable Clinical Guidelines* or CIGs (Peleg *et al*2) which can also be integrated with the other methods if required.3 ### Computer interpretable clinical guidelines: publets OpenClinical was conceived as a way of developing CIGs using a class of artificial intelligence techniques called knowledge engineering.3 It is a web-based knowledge-sharing platform which offers a radically different approach to summarising knowledge of best practice in the form of executable models of practice that we call publets. Publets include structured data models and executable logic (rules, decisions, pathways) as well as traditional CPG content. When a publet is consulted in a patient’s care, it can request and interpret information about the patient and offer *personalised recommendations* and *explanations of the reasons* for its recommendations and the supporting evidence if required. OpenClinical ([https://www.openclinical.net/](https://www.openclinical.net/)) was originally developed at Cancer Research UK and later at Oxford University and UCL/Royal Free Hospital in London. It was launched in its first ‘alpha’ version at the Royal Free in 2013. We plan to deploy a significantly improved technology and publishing platform in 2020. Publets can support a clinician through just the parts of the CPG that are relevant for each patient, summarise the patient-specific *pros* and *cons* of each decision option and provide the supporting evidence. Users can either act on or critique suggestions as they see fit in light of the patient-specific rationale provided by the publet and their own professional judgement. The OpenClinical concept is that clinicians, researchers and other authors develop models of best practice and submit them for review and publication. Authors will submit publets to OpenClinical in much the same way that researchers submit research papers to conventional research journals; each publet can be peer reviewed by independent experts and validated against cases before publication. Several publet demonstrations can be accessed at [https://openclinical.net/index.php?id=68](https://openclinical.net/index.php?id=68) and a repository of diverse examples is at [https://openclinical.net/index.php?id=69](https://openclinical.net/index.php?id=69). Publets empower healthcare professionals and clinical researchers to develop and share models of practice in any area of medicine. Colleagues in other institutions, countries and specialties can download, assess and adapt the publet to meet local requirements and constraints. ### Methods for creating publets: knowledge engineering The tools for developing publets provided by OpenClinical exploit a range of artificial intelligence techniques known collectively as *Knowledge Engineering*. This discipline emerged from artificial intelligence and computer science but has also drawn on insights from cognitive science in that it exploits our understanding of human decision-making and the cognitive skills needed to carry out complex tasks. Medicine and clinical practice have been among the most important targets and challenges for this branch of artificial intelligence research. An important use of knowledge engineering is to formalise human-readable CPGs (traditionally text, tables, flow charts, etc) in a logical or other symbolic form rather than as a mathematical function or algorithm. Like algorithms knowledge models can be executed on a computer and assist clinicians in routine tasks, but symbolic task models are easier for healthcare professionals to understand and critique than conventional algorithms. In a classic paper, Peleg *et al*4 reviewed a number of approaches to modelling clinical guidelines that draw on concepts from knowledge engineering including EON (USA), ASBRU (Israel), GUIDE (Italy), GLIF (USA) and PRO*forma* (UK5). Peleg *et al* systematically compared the abilities of these different methods in capturing practice guidelines. OpenClinical uses the PRO*forma* language model and CIG authoring tools for modelling decision-making and pathways, but we wish to accommodate other approaches as they emerge. There are many differences between publets and other CIG models that have been proposed. PRO*forma* is based on a theory of human expertise and it is easily understood and written by clinicians so they can create, validate and share publets with their peers. Despite its naturalistic form, PRO*forma* has sound foundations in logic and decision theory. In practical terms, it has proved to be a versatile, scalable and effective tool for improving decision-making in many clinical settings and specialties (see table). Tools for authoring and testing publets are available from OpenClinical. View this table: [Table1](http://informatics.bmj.com/content/27/2/e100141/T1) The PRO*forma* approach to designing and deploying artificial intelligence at the point of care is different from the currently popular machine learning and data science approaches, in that it is grounded in an understanding of human expertise and getting the best from a combination of natural and artificial intelligence. It has shown that knowledge engineering methods are effective and versatile and that publets can be deployed at scale in a way that healthcare professionals can understand, critically engage with and, if warranted, challenge. ## Discussion Despite the current excitement around artificial intelligence, many clinicians struggle with the claims of ‘medical revolution’ scenarios which are promulgated by journalists, politicians and some healthcare professionals. However, until recently, the bulk of the medical community has stayed largely silent; claims of ‘breakthroughs’ come and go in medicine, with many technologies eventually falling by the wayside despite early promise. Although the current focus of public and business attention is on the use of data science and machine learning in medicine, the executable knowledge approach is in our view key to the acceptance of artificial intelligence as a useful tool in supporting clinicians and other healthcare professionals who are trying to cope with the knowledge crisis. It is not only that published evidence shows that knowledge engineering methods are effective and clinically acceptable at the point of care, they can also empower healthcare professionals who are not programmers to understand the content which is being used in an artificial intelligence system and when appropriate discuss and overrule an artificial intelligence’s recommendations. This is not to say that data science and machine learning algorithms are not important; they will surely find an important role in delivering precise and personalised care, but this is far better done within a framework in which abstract data can be understood in terms of human concepts and practice. What is arguably even more revolutionary about knowledge engineering is not only that it can improve consistency, quality and safety of care but healthcare professionals can themselves be the authors of artificial intelligence models which can be debated critically with other clinicians and shared fully with their patients. In our view, people who are primarily technologists should not ‘own’ artificial intelligence in medicine. Our role is to provide healthcare practitioners with the tools to deliver better care and to be able to assess and audit the services that AI technologies provide. The mission of OpenClinical is to provide a means to support the creation and dissemination of effective, appropriate services for improving quality and safety of patient care in an ethical, transparent and trustworthy way. The OpenClinical technology is now mature enough to deliver a practical service, and we hope to transform it from a practical demonstration of capability to a scalable publishing platform that delivers the benefits of artificial intelligence and knowledge sharing to healthcare professionals. ## Footnotes * Contributors JF wrote and revised the paper. Other authors contributed to OpenClinical technical development and project management. * Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. * Competing interests None declared. * Patient consent for publication Not required. * Provenance and peer review Commissioned; externally peer reviewed. * Data availability statement There are no data associated with this paper. * Author note JF is founder and director of Deontics Ltd., a for-profit medical AI Company in London. * Received March 6, 2020. * Revision received May 11, 2020. * Accepted May 22, 2020. * © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/). ## References 1. Chidgey J, Leng G, Lacey T. Implementing NICE guidance. J R Soc Med 2007;100:448–52.[doi:10.1177/014107680710001012](http://dx.doi.org/10.1177/014107680710001012)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17911126 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1258/jrsm.100.10.448&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=17911126&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 2. Peleg M, Tu S, Bury J, et al. Comparing computer-interpretable guideline models: a case-study approach. J Am Med Inform Assoc 2003;10:52–68.[doi:10.1197/jamia.M1135](http://dx.doi.org/10.1197/jamia.M1135)pmid:http://www.ncbi.nlm.nih.gov/pubmed/12509357 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M1135&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=12509357&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 3. Fox J, Gutenstein M, Khan O, et al. OpenClinical.net: A platform for creating and sharing knowledge and promoting best practice in healthcare. Computers in Industry 2015;66:63–72.[doi:10.1016/j.compind.2014.10.001](http://dx.doi.org/10.1016/j.compind.2014.10.001) 4. Peleg M. Computer-interpretable clinical guidelines: a methodological review. J Biomed Inform 2013;46:744–63.[doi:10.1016/j.jbi.2013.06.009](http://dx.doi.org/10.1016/j.jbi.2013.06.009) [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2013.06.009&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=23806274&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 5. Sutton DR, Fox J. The Syntax and Semantics of the PRO forma Guideline Modeling Language. J Am Med Inform Assoc 2003;10:433–43.[doi:10.1197/jamia.M1264](http://dx.doi.org/10.1197/jamia.M1264) [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M1264&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=12807812&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 6. Walton RT, Gierl C, Yudkin P, et al. Evaluation of computer support for prescribing (capsule) using simulated cases. BMJ 1997;315:791–5. [Abstract/FREE Full Text](http://informatics.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTUvNzExMS83OTEiO3M6NDoiYXRvbSI7czoyNToiL2JtamhjaS8yNy8yL2UxMDAxNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 7. Taylor P, Fox J, Pokropek AT. The development and evaluation of cadmium: a prototype system to assist in the interpretation of mammograms. Med Image Anal 1999;3:321–37. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/S1361-8415(99)80027-9&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=10709699&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 8. Emery J, Walton R, Murphy M, et al. Computer support for interpreting family histories of breast and ovarian cancer in primary care: comparative study with simulated cases. BMJ 2000;321:28–32. [Abstract/FREE Full Text](http://informatics.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjExOiIzMjEvNzI1Mi8yOCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qaGNpLzI3LzIvZTEwMDE0MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 9. Tural C, Ruiz L, Holtzer C, et al. Clinical utility of HIV-1 genotyping and expert advice: the havana trial. AIDS 2002;16:209–18. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1097/00002030-200201250-00010&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=11807305&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000173711400009&link_type=ISI) 10. Bury J, Hurt C, Roy A, et al. LISA: a web-based decision-support system for trial management of childhood acute lymphoblastic leukaemia. Br J Haematol 2005;129:746–54. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2141.2005.05541.x&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=15953000&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000229635300005&link_type=ISI) 11. Early referrals in primary care for patients with suspected cancer. London, UK Tenth World Congress on Health and Medical Informatics; 2001. 12. Patkar V, Hurt C, Steele R, et al. Evidence-Based guidelines and decision support services: a discussion and evaluation in triple assessment of suspected breast cancer. Br J Cancer 2006;95:1490–6. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1038/sj.bjc.6603470&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=17117181&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 13. Glasspool DW, Oettinger A, Braithwaite D, et al. Interactive decision support for risk management: a qualitative evaluation in cancer genetic counselling sessions. J Cancer Educ 2010;25:312–6. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s13187-009-0035-8&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=20162470&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000283105800010&link_type=ISI) 14. Patkar V, Acosta D, Davidson T, et al. Using computerised decision support to improve compliance of cancer multidisciplinary meetings with evidence-based guidance. BMJ Open 2012;2. doi:[doi:10.1136/bmjopen-2011-000439](http://dx.doi.org/10.1136/bmjopen-2011-000439). [Epub ahead of print: 25 06 2012].pmid:http://www.ncbi.nlm.nih.gov/pubmed/22734113 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 15. Peleg M, Fox J, Patkar V, et al. A Computer-Interpretable version of the AACE, AME, ETA medical guidelines for clinical practice for the diagnosis and management of thyroid nodules. Endocr Pract 2014;20:352–9. 16. Ranta A, Dovey S, Weatherall M, et al. Cluster randomized controlled trial of TIA electronic decision support in primary care. Neurology 2015;84:1545–51. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1212/WNL.0000000000001472&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25795645&link_type=MED&atom=%2Fbmjhci%2F27%2F2%2Fe100141.atom) 17. Miles A, Chronakis I, Fox J, et al. Use of a computerised decision aid (dA) to inform the decision process on adjuvant chemotherapy in patients with stage II colorectal cancer: development and preliminary evaluation. BMJ Open 2017;7:e012935. 18. González-Ferrer A, Valcárcel M Ángel, Cuesta M, et al. Development of a computer-interpretable clinical guideline model for decision support in the differential diagnosis of hyponatremia. Int J Med Inform 2017;103:55–64.