Automated conflict resolution between multiple clinical pathways: a technology report
=====================================================================================

* Ian Litchfield
* Alice Turner
* Ruth Backman
* João Bosco Ferreira Filho
* Phil Weber
* Mark Lee

## Abstract

**Background** The number of people in the UK with three or more long-term conditions continues to grow and the management of patients with co-morbidities is complex. In treating patients with multimorbidities, a fundamental problem is understanding and detecting points of conflict between different guidelines which to date has relied on individual clinicians collating disparate information.

**Objective** We will develop a framework for modelling a diverse set of care pathways, and investigate how conflicts can be detected and resolved automatically. We will use this knowledge to develop a software tool for use by clinicians that can map guidelines, highlight root causes of conflict between these guidelines and suggest ways they might be resolved.

**Method** Our work consists of three phases. First, we will accurately model clinical pathways for six of the most common chronic diseases; second, we will automatically identify and detect sources of conflict across the pathways and how they might be resolved. Third, we will present a case study to prove the validity of our approach using a team of clinicians to detect and resolve the conflicts in the treatment of a fictional patient with multiple common morbidities and compare their findings and recommendations with those derived automatically using our novel software.

**Discussion** This paper describes the development of an important software-based method for identifying a conflict between clinical guidelines. Our findings will support clinicians treating patients with multimorbidity in both primary and secondary care settings.

*   multimorbidity
*   clinical guidance
*   conflict identification
*   decision support
*   patient pathways

## INTRODUCTION

By 2018, it is estimated that the number of people in the UK with three or more long-term conditions will have grown from 1.9 to 2.9 million.1 The management of patients with co-morbidities is complex, relying on a range of interacting social agents including physicians, administrators and the drive for patient-centred care. Yet support for clinicians intended to improve the quality of healthcare is based on some 250 clinical guidelines that almost exclusively focus on single conditions.2 The result is that applying multiple guidelines to a patient can result in conflicting recommendations for care leading to calls for an improved integration of existing guidelines to better support patients with multimorbidities.3

These guidelines frequently use graphical descriptions of evidence and are often represented in a single or series of flowcharts.4,5 As such they share with software system specifications, the central tenets of a series of executions of ordered sequences of activities or tasks. The interactions these sequences describe can be modelled using a number of approaches such as sequence or activity diagrams,6 workflow languages such as business process model and notation (BPMN),7 or variants of Petri nets.8 Previous studies has explored the creation of algorithms for merging or ‘composing’ models in these languages9–13 in order to create a unified model from smaller constituent models, or views. However, in healthcare as in software engineering, a conflict may arise when models are merged but individual executions or actions are incompatible with others.

In this study, we are investigating the use of automated methods of detecting conflicts between multiple clinical pathways and proposing solutions that resolve the conflict. We will consider the specific nature and parameters of each guideline, specific conditions of individual patients, cross-referencing pathways to determine which aspects of the relevant pathway are followed and when. These methods will be developed into a prototype software tool presenting an automated method for navigating multiple clinical pathways for patients with multimorbidity that detects any conflict between pathways and makes suggestions as to how these conflicts might be resolved, sympathetic to the priorities of the care provider and patient. Ultimately, our tool will lead to improved patient outcomes and increase the cost-effectiveness of treating patients with multimorbidity.

## DEVELOPMENT

To identify possible conflicts, we are transforming models into logical statements and using them to detect conflicts between various steps on each ‘pathway’ using constraint solvers.14–17 These have been used successfully to detect conflict in the composition of static models of processes.18–20 More recently, the use of constraint solvers has been explored for dynamic models similar to those presented in healthcare.19,21 In this instance, the solvers recognise that the resolution of a conflict can entail different compromises and affect various measures such as time, resource and cost.

Our work consists of three phases.

### Phase 1: modelling clinical pathways

The first phase builds on the graphical process modelling language BPMN7 to accurately model clinical pathways and applies it to six pathways representing some of the most common chronic diseases: hypertension, ischaemic heart disease; chronic obstructive pulmonary disease (COPD); Type 2 diabetes mellitus; depression and osteoarthritis (OA). Describing this range of pathways in BPMN allows us to clarify any issues related to the expressiveness of the language or any ambiguities that might exist when attempting to capture clinical pathways.

Having developed the language and mapped the specified pathways, we can produce a formal model that captures care pathways unambiguously. This model will capture pathway characteristics including the periodic nature of appointments and the temporal validity of certain parameters,22,23 for example, the value of blood test results at a specific point in time, drug interactions or intolerance and so on. It will be capable of specifying constraints over the compatibility of treatment within three domains: the scheduling of appointments, medication and behavioural or lifestyle advice. This formal representation will be based on a restriction of the BPMN notation endowed with semantic concepts inspired by coloured petri nets (CPN)24 to express the dynamic behaviour of the models. CPN and equivalent models have previously successfully been used to model and reason about distributed systems for the dynamic interpretation of behavioural composition mechanisms.13,19,21,25 Elsewhere, BPMN has been successfully used to model clinical workflows in a static manner.26,27 We will confirm the most suitable way to define additional relations and functions over events to encode the necessary dynamic information relating to patients’ interaction with their care providers and subsequent actions. An example of one of our modelled pathways is shown in Figure 1.

![Figure 1](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/3/142/F1.medium.gif)

[Figure 1](http://informatics.bmj.com/content/25/3/142/F1)

Figure 1 An excerpt of the modelled pathway for osteoporosis illustrating the use of a restricted subset of BPMN notation enhanced to describe dependencies on, and modification of, data

### Phase 2: recognition and resolution of conflict

#### Pathway composition and care conflicts

Each care pathway represents sequences of medical procedures (events) that may be applied to a patient, together with decision criteria guiding which sequence of events will apply in a particular case, including interactions between the patient and provider. These care pathways are similar to workflow models,28,29 which represent the partially ordered occurrence of events and interactions between components in a software system. As a result, automated constraint solvers such as Z3,17,30 can be used to automatically detect whether the ordering of events specified in one model is in conflict with the ordering of events in others.13,31 In the context of healthcare, one example is when two pathways prescribe drugs which should not be used at the same time. Technically, this can be defined as a constraint, that is, the occurrence of one event disallows the occurrence of another.

We give a limited example in Figure 2. Here, we show a fragment of a pathway model for treating osteoporosis (OA, left) and COPD (right). The majority of the models have been omitted for clarity. The OA model specifies for a number of activities to be carried out in parallel, including the prescription of non-steroidal anti-inflammatory drugs (NSAIDs) to the patient, but only if neither NSAIDs nor corticosteroids are already prescribed. The medications prescribed to a patient are described by the values of data variables in the model. (Here, we use integers, to allow for future dependency on how much medication is being taken.) The ‘guard’ annotations indicate the restriction on prescribing NSAIDs, while the ‘data’ annotations indicate the need to record that the patient is now taking NSAIDs, once a prescription has taken place. The COPD fragment similarly depends on, and modifies the status of, corticosteroids. If a patient is following both the pathways, then there is potential for a conflict between these activities: if NSAIDs is prescribed by the OA pathway, then corticosteroids must not be prescribed by the COPD pathway, and vice versa. Depending on the stage at which the patient is on either pathway, these activities may occur at different times, the set of drugs prescribed to the patient may be different, and so on. Capturing pathways based on BPMN and specifying their composition and constraints based on data values will allow constraint solvers to solve the composed model for the sequences of activities which are valid for a particular patient and situation, and which must be avoided (and why). This is a very simple illustration of a conflict between models and our continuing work will extend this principle to account for other types of conflict. Principally, those that can occur between lifestyle recommendations such as a particular diet or exercise regime though we expect that further sources of conflict will be considered relating to the preferences and priorities of clinicians and patients and the scheduling of appointments or procedures, during subsequent stages of our work.

![Figure 2](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/3/142/F2.medium.gif)

[Figure 2](http://informatics.bmj.com/content/25/3/142/F2)

Figure 2 Fragments of modelled pathways for osteoporosis and COPD illustrating the potential for conflict between medication (corticosteroids and NSAIDs)

For each of the care pathways, we are using the Z3 solver to search for a solution that satisfies all constraints. If the care pathways have no conflicts between them, we obtain a new model of the integrated pathways. Otherwise, the analysis identifies the conflicting logical constraints, which allows us to identify which parts of the pathway models are in conflict. This underpins the creation of algorithms to map the solution (or conflicts) back onto the original pathway models for interpretation by clinicians and provides a basis for automated resolutions to be recommended.

Since our models of pathways are always finite, a conflict exists only if the solver can detect it.21 Where there is a loop – a common trait within clinical guidance, for example, testing for HbA1 in diabetics – we will generate a finite number of iterations of the loop over which to evaluate the model. The use of automated solvers in detecting these conflicts means that we can seek conflicts at scale and in a timely fashion.

#### Automated conflict resolution

As the number of pathways increases, it is critical to adopt the most suitable method of conflict detection and resolution. We will investigate automated methods for changing those parts of pathway models which are causing conflict. To enable this, we are developing a method of identifying the solutions that require the least changes to the models. To do so, we will incorporate a concept of distance metrics into our constraint solving practices. Since sequences of tasks representing a pathway can been seen as graphs, graph edit distances, which measure the number of changes between two graphs required to make them equal, are a suitable basis for such a metric.32 This will allow us to present the clinician with a series of choices ranked in a meaningful manner, as the tool will propose all possible solutions referencing the number of changes to the various pathways needed to accommodate the solution, and so allow a better-informed decision to be made.

### Phase 3: development of a prototype software tool and case study

The final phase involves the creation of a prototype tool that will allow the user to create models of pathways, perhaps reflecting local priorities or initiatives, and to identify and resolve conflicts between them. The validity of our tool will be evaluated using a case study based on an example proposed by Boyd *et al*.33 of a 79-year-old lady with multiple morbidities including diabetes, hypertension, OA, and COPD. To evaluate whether our method can automatically detect the published conflicts, we will use the models of the six pathways we have defined in Phase 1 and which we know have conflict. Using Z3 as outlined in Phase 2, we will validate whether we can detect the published conflicts and also any other conflicts that may exist between the models. We will then apply the automated method of conflict resolution to produce potential solutions to these conflicts. Finally, a team of clinicians including the author Alice Turner will confirm whether the conflicts discovered and the proposed solutions are medically valid and applicable to the clinical management of the patient.

## APPLICATION

As described, the increasing number of patients with multiple and frequently long-term conditions presents a challenge to the existing models of care. Though generic guidance for multimorbidity has been produced recently,34 much of the evidence-based guidance is still focused on the treatment of single conditions. This means clinicians are making decisions for one morbidity without having the tools to identify, illustrate, or understand the clinical consequences on another pre-existing condition. We will initiate a software library for pathway models consisting in the first instance of the modelled pathways constructed as part of this project. These will be freely available as a ‘plug-in’ meaning they can be used with existing software systems.

The methods and tools we are developing have the potential to maximise the well-being and quality of life of patients with chronic disease, and benefit their families and carers. By optimising the patient management, our proposed techniques can help reduce polypharmacy, avoidable hospital admissions and procedures, and allow us to implement and audit the best practice. We will also explore the theoretical and practical considerations of the applicability of constraint solvers to other sources of healthcare information technology systems. The application of such techniques has the potential to create tools and technologies to address the national and global health challenges.

## DISCUSSION

Medical decision-making is a complex and evolving process particularly in the increasingly busy primary care environment where continuity of care is decreasing,34 and clinicians are reliant on generic guidance that in such busy settings can often be ignored.35 The graphical language we are developing can not only be used to model centrally disseminated guidance but also used to create bespoke care pathways reflecting the local needs of local patients, organisations and commissioners presented in a readily interpretable display.

BPMN has become the de facto and ISO36 standard for process modelling. Its intuitive graphical model37 has been found suitable for the capturing of processes by domain experts without specialist modelling skills, and facilitating communication between non-specialists.26,27,38 BPMN has been studied in healthcare and assessed as suitable for modelling and visualising clinical pathways39,40 and is being chosen as the modelling notation an increasing number of studies.26,27,38–40 Critically, for our purpose to specify conflicts arising from interaction with data, it is designed for extension,39,41 and for formal analysis of modelled pathways, semantics can be properly specified for subsets of BPMN.42,43

Other graphical notations have been proposed for modelling processes. Some, such as Petri nets44 and YAWL,45 do have a rigorous semantics, but either do not account for data (Petri nets), or are over-complex for our purposes (YAWL), or are more suitable for expert modellers (both). In the healthcare domain, a number of Computer Interpretable Guidelines (CIGs) have been proposed. However, none of these have yet become standard, and their bespoke process notations and interaction with data are tightly integrated with their broader facilities for decision support, knowledge management and process automation. We therefore base our notation on BPMN for the benefits of widespread acceptance, ease of interpretation and extension, to allow ease of use by non-specialists while being able to formally specify clinical guidelines.

The acknowledged pressure on clinicians analysing large quantities of data to manage increasingly complex care regimes has led to the development of clinical support systems that can provide readily interpretable medical information. Our approach to manage complex patients is based on producing a series of interactive pathways where any combination can be selected, the conflict between them automatically identified, and suggestions for resolution generated to inform the subsequent decisions. There are, however, other approaches and recently the use of fuzzy logic has gained in popularity.46 This also offers the opportunity to integrate complex human knowledge with computer-aided techniques through tools such as CCTool.47 Many of these rely on the Theory of Change methodologies that combine perspectives of a range of stakeholders to meet a pre-specified goal.48 The resultant maps share much in common with the care pathways that we are creating based on NICE guidance.2 However, the benefit of our approach is that the potentially inhibiting complexity of taking a whole systems view is avoided by building a system one pathway at a time. That these pathways can be used to create unambiguous logical models interpretable by constraint solvers means we can identify potential conflicts between pathways and provide targeted support for clinicians managing multimorbid patients. Such tailored support is missing from the clinical support systems currently in use.

None.

I declare on behalf of all authors that we have no significant competing financial, professional or personal interests that might have influenced the content or the work described in this manuscript.

As lead author, I can confirm that this is a technical report and as such only preliminary data are available.

The main funding for this project was provided by Engineering and Physical Sciences Research Council EP/M014401/1

Litchfield and Turner were responsible for the conception of the work and the design of the study. Litchfield led the drafting of the article with input from Weber, Backman. Lee, Turner and Ferreira Filho all provided critical revisions. The final version was drafted by Litchfield and approved by Turner, Backman, Ferreira Filho, Weber and Lee.

*   Received September 1, 2018.


*   © 2018 The Author(s). Published by BCS, The Chartered Institute for IT under Creative

Commons license http://creativecommons.org/licenses/by/4.0/

## References

1.  1.DOH. Long Term Conditions Compendium of Information 2012. 3rdDepartment of Health[Internet]. Available from: [https://www.gov.uk/government/uploads/system/uploads/attachment\_data/file/216528/dh\_134486.pdf/](https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216528/dh_134486.pdf/). Accessed September 2017
    
    

2.  2.NICE. National Institute for Health and Clinical Excellence. [Internet]. Available from: [https://www.nice.org.uk/](https://www.nice.org.uk/). Accessed September 2017
    
    

3.  3.Kenning C Fisher L Bee P, Bower P, Coventry P. Primary care practitioner and patient understanding of the concepts of multimorbidity and self-management: a qualitative study. SAGE Open Medicine 2013doi:10.1177/2050312113510001
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/2050312113510001&link_type=DOI) 

4.  4.Hawley ST Zikmund-Fisher B Ubel P Jancovic A, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Education and Counseling 2008;73(3):448-55. doi:10.1016/j.pec.2008.07.023pmid:18755566
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.pec.2008.07.023&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18755566&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000261630800010&link_type=ISI) 

5.  5.Negrini S Giovannoni S Minozzi S Barneschi G Bonaiuti D Bussotti A, et al. Diagnostic therapeutic flow-charts for low back pain patients: the Italian clinical guidelines. Europa Medicophysica 2006;42(2):151-70
    
    

6.  6.Object Management GroupUML Superstructure Specification, v.2.4.1 [Internet]. OMG 2011. Available from: [http://www.omg.org/spec/UML/2.4.1/Superstructure/PDF](http://www.omg.org/spec/UML/2.4.1/Superstructure/PDF). Accessed September 2017
    
    

7.  7.Object Management GroupBusiness Process Model and Notation, v.2.0 [Internet]. OMG 2011. Available from: [http://www.omg.org/spec/BPMN/2.0/PDF](http://www.omg.org/spec/BPMN/2.0/PDF). Accessed September 2017
    
    

8.  8.Reisig W. Understanding Petri Nets: Modeling Techniques, Analysis Methods, Case Studies. Berlin, Germany: Springer, 2013.doi:10.1007/978-3-642-33278-4
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-642-33278-4&link_type=DOI) 

9.  9.Harel D, Kugler H. Synthesizing state-based object systems from LSC specifications. International Journal of Foundations of Computer Science 2002;13(1):5-51. doi:10.1142/S0129054102000935
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1142/S0129054102000935&link_type=DOI) 

10. 10.Uchitel S, Kramer J, Magee J. Implied scenario detection in the presence of behaviour constraints. Electronic Notes in Theoretical Computer Science 2002;65(7):65-84
    
    

11. 11.Ziadi T, Helouet L, Jezequel JM. Revisiting state chart synthesis with an algebraic approach. Proceedings of 26th International Conference on Software Engineering (ICSE) 2004IEEE Computer SocietyEdinburgh, UK242-51. doi:10.1109/ICSE.2004.1317446
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/ICSE.2004.1317446&link_type=DOI) 

12. 12.Araujo J. Whittle J and Kim D-K. Modelling and composing scenario-based requirements with aspects. Proceedings of 12th IEEE International Conference on Requirements Engineering (RE) 2004IEEE Computer SocietyKyoto, Japan58-67
    
    

13. 13.Bowles JKF, Bordbar B, Alwanain M. Weaving true-concurrent aspects using constraint solvers. Proceedings of 16th International Conference on Application of Concurrency to System Design (ACSD 2016), IEEE 2016JuneTorun, Poland19-24. doi:10.1109/ACSD.2016.19
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/ACSD.2016.19&link_type=DOI) 

14. 14.Prasad M Biere A, Gupta A. A survey of recent advances in SAT-based formal verification. International Journal on Software Tools for Technology Transfer 2005;7(2):156-73. doi:10.1007/s10009-004-0183-4
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s10009-004-0183-4&link_type=DOI) 

15. 15.Nethercote N Stuckey P Becket R Brand S Duck G, Tack G. MiniZinc: towards a standard CP modelling language. Principles and Practice of Constraint Programming—CP 2007. Berlin, Germany: Springer, 2007.529-43. doi:10.1007/978-3-540-74970-7_38
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-540-74970-7_38&link_type=DOI) 

16. 16.Gent IP Jefferson C, Miguel I. Minion: a fast, scalable, constraint solver. Proceedings of 17th European Conference on Artificial Intelligence (ECAI) 2006IOS PressRiva del Garda, Italy98-102
    
    

17. 17.de Moura LM, Bjørner N. Z3: an efficient SMT solver. Proceedings of Tools and Algorithms for the Construction and Analysis of Systems (TACAS) 2008SpringerBudapest, Hungary337-40. doi:10.1007/978-3-540-78800-3_24
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-540-78800-3_24&link_type=DOI) 

18. 18.Rubin J Chechik M, Easterbrook S. Declarative approach for model composition. Proceedings of International Workshop on Modeling in Software Engineering (MISE) 2008ACMLeipzig, Germany7-14. doi:10.1145/1370731.1370734
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1145/1370731.1370734&link_type=DOI) 

19. 19.Bowles JKF, Bordbar B. A formal model for integrating multiple views. Proceedings of 7th International Conference on Application of Concurrency to System Design (ACSD) 2007IEEE Computer SocietyBratislava, Slovakia71-9. doi:10.1109/ACSD.2007.39
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/ACSD.2007.39&link_type=DOI) 

20. 20.Zhang D Li S, Liu X. An approach for model composition and verification. Proceedings of International Conference on Networked Computing and Advanced Information Management (NCM) 2009IEEE Computer SocietySeoul, Korea1102-7. doi:10.1109/NCM.2009.271
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/NCM.2009.271&link_type=DOI) 

21. 21.Bowles JKF, Bordbar B, Alwanain M. A logical approach for behavioural composition of scenario-based models. Formal methods and software engineering. Proceedings of 17th International Conference on Formal Engineering Methods 2015NovemberICFEM 2015Paris, France3-5252-69. doi:10.1007/978-3-319-25423-4_16
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-319-25423-4_16&link_type=DOI) 

22. 22.1.  Redmill F, 
    2.  Anderson T
    
    Baxter GD Kuster-Filipe J, Miguel A, Tan KRedmill F, Anderson T. The effects of timing and collaboration on dependability in the neonatal intensive care unit. Constituents of Modern System-safety Thinking. London, UK: Springer, 2005.195-210. doi:10.1007/1-84628-130-X_13pmid:16137245
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/1-84628-130-X_13&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=16137245&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

23. 23.Burns A, Hayes IJ. A time band framework for modeling real-time systems. Real-Time Systems 2010;45(1):106-42. doi:10.1007/s11241-010-9094-5
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s11241-010-9094-5&link_type=DOI) 

24. 24.Jensen K, Kristensen LM. Coloured Petri Nets—Modelling and Validation of Concurrent Systems. Berlin, Germany: Springer, 2009.doi:10.1007/b95112
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/b95112&link_type=DOI) 

25. 25.Weber P Bordbar B, Tiňo P, Majeed B. A framework for comparing process mining algorithms. 2011 IEEE GCC Conference and Exhibition (GCC) 20112011 IEEE GCC625-8. doi:10.1109/IEEEGCC.2011.5752616
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/IEEEGCC.2011.5752616&link_type=DOI) 

26. 26.Müller R, Rogge-Solti A. BPMN for healthcare processes. Proceedings of 3rd Central-European Workshop on Services and their Composition, Services und ihre Komposition 2011ZEUS 2011Karlsruhe, Germany65-72. CEUR-WS.org
    
    

27. 27.Rojo MG, Rolón E, Calahorra L, García FO, Sánchez RP, Ruiz F, et al. Implementation of the Business Process Modelling Notation (BPMN) in the modelling of anatomic pathology processes. Diagnostic Pathology 2008;3(Suppl 1):S22. doi:10.1186/1746-1596-3-S1-S22pmid:18673511; pmcid:PMC2500103
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1186/1746-1596-3-S1-S22&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18673511&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

28. 28.Casati F Ceri S Pernici B, Pozzi G. Conceptual modeling of workflows. Proceedings of OOER’95: Object-Oriented and Entity-Relationship Modelling, 14th International Conference 1995LNCS, SpringerGold Coast, Australia341-54
    
    

29. 29.van der Aalst WMP. The application of Petri nets to workflow management. Journal of Circuits, Systems and Computers 1998;8(1):21-66. doi:10.1142/S0218126698000043
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1142/S0218126698000043&link_type=DOI) 

30. 30.MicrosoftZ3 Solver [Internet]. Microsoft 2012. Available from: [https://github.com/Z3Prover/z3](https://github.com/Z3Prover/z3). Accessed September 2017
    
    

31. 31.1.  Proença J, 
    2.  Lumpe M
    
    Bowles J, Caminati MBProença J, Lumpe M. Correct composition of dephased behavioural models. Formal Aspects of Component Software. FACS 2017 2017;10487Cham, Switzerland: Springer233-50Lecture Notes in Computer Sciencedoi:10.1007/978-3-319-68034-7_14
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-319-68034-7_14&link_type=DOI) 

32. 32.Gao X Xiao B Tao D, Li X. A survey of graph edit distance. Pattern Analysis and Applications 2010;13(1):113-29. doi:10.1007/s10044-008-0141-y
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s10044-008-0141-y&link_type=DOI) 

33. 33.Boyd C Darer J Boult C Fried L Boult L, Wu A. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005;294(6):716-24. doi:10.1001/jama.294.6.716pmid:16091574
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1001/jama.294.6.716&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=16091574&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000231068600025&link_type=ISI) 

34. 34.Devlin RA Hogg W, Zhong J, Shortt M, Dahrouge S, Russell G. Practice size, financial sharing and quality of care. BMC Health Services Research 2013;13446. doi:10.1186/1472-6963-13-446pmid:24165413; pmcid:PMC3819507
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1186/1472-6963-13-446&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=24165413&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

35. 35.Kaffash J. Seven in ten GPs ignore NICE guidance once a month, Pulse Magazine2015. Available from: [http://www.pulsetoday.co.uk/news/clinical-news/seven-in-ten-gps-ignorenice-guidance-at-least-once-a-month/20009357.article](http://www.pulsetoday.co.uk/news/clinical-news/seven-in-ten-gps-ignorenice-guidance-at-least-once-a-month/20009357.article)
    
    

36. 36.International Organization for Standardization. ISO/IEC 19510:2013 Information technology—Object Management Group Business Process Model and Notation 2013
    
    

37. 37.Scheuerlein H Rauchfuss F Dittmar Y Molle R Lehmann T Pienkos N, et al. New methods for clinical pathways—Business Process Modeling Notation (BPMN) and Tangible Business Process Podeling (t.BPM). Langenbeck’s Archives of Surgery 2012;397(5):755-61. doi:10.1007/s00423-012-0914-zpmid:22362053
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s00423-012-0914-z&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22362053&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

38. 38.Ruiz F Garcia F Calahorra L, Llorente C, Gonçalves L, Daniel C, et al. Business process modeling in healthcare. Studies in Health Technology and Informatics 2012;17975-87
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22925789&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

39. 39.Braun R Schlieter H Burwitz M, Esswein W. BPMN4CP: design and implementation of a BPMN extension for clinical pathways. IEEE International Conference on Bioinformatics and Biomedicine (BIBM) 2014IEEE Computer SocietyBelfast, UK9-16. doi:10.1109/BIBM.2014.6999261
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/BIBM.2014.6999261&link_type=DOI) 

40. 40.Rolón E, Aguilar ER, García F, Ruiz F Piattini M Calahorra L, et al. Process modeling of the health sector using BPMN: a case study. Proceedings of First International Conference on Health Informatics, HEALTHINF 2008 2008;2Funchal, Portugal173-8
    
    

41. 41.Stroppi L Chiotti O, Villareal P. Extending BPMN 2.0: method and tool support. LNBIP 2011;9559-73. doi:10.1007/978-3-642-25160-3_5
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-642-25160-3_5&link_type=DOI) 

42. 42.Kossak F Illibauer C Geist V Kubovy J Natschlager C Ziebermayr T, et al. A Rigorous Semantics for BPMN 2.0 Process Diagrams. Cham; Heidelberg; New York; Dordrecht; London: Springer, 2014.doi:10.1007/978-3-319-09931-6
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/978-3-319-09931-6&link_type=DOI) 

43. 43.Van Gorp P, Dijkman RM. A visual token-based formalization of BPMN, 2.0 based on in-place transformations. Information and Software Technology 2013;55(2):365-94. doi:10.1016/j.infsof.2012.08.014
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.infsof.2012.08.014&link_type=DOI) 

44. 44.Murata T. Petri nets: properties, analysis and applications. Proceedings of the IEEE 1989;77(4):541-80. doi:10.1109/5.24143
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1109/5.24143&link_type=DOI) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=A1989AA09800002&link_type=ISI) 

45. 45.van der Aalst WMP, ter Hofstede AHM. YAWL: yet another workflow language. Information Systems Journal 2005;30(4):245-75. doi:10.1016/j.is.2004.02.002
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.is.2004.02.002&link_type=DOI) 

46. 46.Amirkhani A Papageorgiou E Mohseni A, Moosavi M. A review of fuzzy cognitive maps in medicine: taxonomy, methods and applications. Computer Methods and Programs in Biomedicine 2017;142129-45. doi:10.1016/j.cmpb.2017.02.021pmid:28325441
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.cmpb.2017.02.021&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=28325441&link_type=MED&atom=%2Fbmjhci%2F25%2F3%2F142.atom) 

47. 47.Map and Steer Complex Systems: A project powered by ERIE to aid decision making and intervention design in complex contexts. Available from: [http://cctool.herokuapp.com/](http://cctool.herokuapp.com/). Accessed February 2018
    
    

48. 48.Taplin DH, Clark H. Theory of Change Basics. New York, NY: ACT Knowledge, 2013.