Generating insights in uncharted territories: real-time learning from data in critically ill patients–an implementer report ============================================================================================================================= * Davy van de Sande * Michel E. Van Genderen * Joost Huiskens * Robert E. R. Veen * Yvonne Meijerink * Diederik Gommers * Jasper van Bommel ## Abstract **Introduction** In the current situation, clinical patient data are often siloed in multiple hospital information systems. Especially in the intensive care unit (ICU), large volumes of clinical data are routinely collected through continuous patient monitoring. Although these data often contain useful information for clinical decision making, they are not frequently used to improve quality of care. During, but also after, pressing times, data-driven methods can be used to mine treatment patterns from clinical data to determine the best treatment options from a hospitals own clinical data. **Methods** In this implementer report, we describe how we implemented a data infrastructure that enabled us to learn in real time from consecutive COVID-19 ICU admissions. In addition, we explain our step-by-step multidisciplinary approach to establish such a data infrastructure. **Conclusion** By sharing our steps and approach, we aim to inspire others, in and outside ICU walls, to make more efficient use of data at hand, now and in the future. * data science * machine learning * artificial intelligence * critical care outcomes * COVID-19 ## Introduction The current pandemic demonstrated that healthcare was in uncharted territory. As such, the outbreak of the novel COVID-19 could be a turning point for the initiation of advanced analytics, especially in intensive care medicine. The pandemic emphasized the importance of instant, or even real time, analysis of large volumes of intensive care unit (ICU) data in order to generate new insights to eventually improve quality of care.1 The ICU typically is an environment where clinicians are confronted with large amounts of clinical data siloed in multiple information systems and are often not optimally used to aid clinical decision making.2 Even more, data are often collected when dealing with complex diseases or conditions to improve understanding of the clinical course and to evaluate the effects of therapeutic interventions at a later stage. However, especially in pressing times, advanced data analytics tools (eg, artificial intelligence) can be helpful by mining large volumes of data and discover clinical patterns and best treatment options from clinical data.3 4 Immediately after the outbreak, several initiatives such as the Dutch Data Warehouse and the covidpredict initiative were announced aiming to collect large amounts of data on COVID-19 ICU patients to eventually optimise clinical care and improve outcome.5 However, the collection and organisation of data from multiple ICUs is time consuming and is often obstructed by technical and privacy challenges.6 Furthermore, the agglomeration of data is unresponsive to local variations in population or disease-specific patterns, and different local practices and clinical definitions impede proper comparison between cohorts.7 Given these considerations, we aimed to implement a local data infrastructure that would enable us to learn in real time from our own consecutive COVID-19 ICU admissions by comparing patient characteristics, treatment regimens and clinical outcomes. Here, we present how a structured data approach can help to improve quality of care and can serve as a basis for advanced data analysis. ### Implementing a real-time clinical data infrastructure All consecutive patients with COVID-19 admitted to the adult ICU of the Erasmus University Medical Center, a tertiary referral centre in Rotterdam, The Netherlands, were analysed. The need for written informed consent was waived by the regional medical research ethics committee. Figure 1 provides a week-by-week overview of the data infrastructure development process along with the engaged stakeholders. ![Figure 1](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/28/1/e100447/F1.medium.gif) [Figure 1](http://informatics.bmj.com/content/28/1/e100447/F1) Figure 1 Road towards a real-time clinical data infrastructure. Stakeholders were engaged early in the process and tasks were distributed (dark blue=ICU research team, light blue=ICU physicians, dark grey=data and analytics department and light grey=IT department). After 6 weeks, a data infrastructure was implemented that allows to extract data in real-time from the electronic health record (HIX). *Demographics, medication, lab tests, ventilator settings, vital signs and clinical notes were extracted from the database using structured query language (SQL) in Microsoft SQL management Studio. ¥The analytics platform, SAS Viya (V.8.3) was used to process and further analyse the data. Figure 1 was created by the lead author, and permission to reuse the image was obtained. HIX, Healthcare Information eXchange; ICU, intensive care unit; IT, information technology. From start, a weekly recurring meeting was scheduled with the local information technology (IT) department, the data and analytics department and a third software party (SAS Institute, Cary, North Carolina, USA). Clinical questions and needs were formulated and determined in collaboration with a team of ICU physicians. Physicians were particularly interested in the treatment effect of several interventions, such as the effect of prone positioning, effect of high or low dose steroids, timing of steroids and the influence of body mass index on several outcome parameters, such as survival and mechanical ventilation days, in these critically ill patients. In our hospital, all patient data from the electronic health record (EHR), Healthcare Information eXchange (HIX), are routinely stored in a structured query language (SQL) server database (referred to as ‘database’). Since these data originate from a single source, it was already properly formatted (data engineering) and did need not require additional harmonisation. As such, the data and analytics department established a live connection to the database server (by using Microsoft SQL management studio) for qualified members of the ICU research team with restricted access to ICU patients. Subsequently, the IT department installed a data analytics platform (SAS Viya V.8.3) to ensure data extraction, data management, facilitate data visualisation and advanced analytics, and data and model governance. In the second week, relevant data were identified in the database (figure 1), and SQL queries were written to extract these data, supervised by a senior data analyst. The data and analytics department provided support throughout the data identification and extraction process. Extracted data were continuously verified with the team of ICU physicians to ensure its completeness. From the third to the fourth week tables containing, raw data were joined from separate queries and processed using SAS programming language. To be able to cope with the rapidly growing number of patients with COVID-19, ICU capacity expanded from 45 to 102 beds in a matter of days, mostly towards other parts of the hospital. To oversee patient characteristics in multiple ICU locations, a real-time ‘COVID-19 clinical data dashboard’ was required and was developed in the fifth week. Since then, the data were further processed, and in the sixth week, a research database was constructed that could be used to perform in-depth advanced analysis. ### Analysing clinical data in real time A dashboard was successfully constructed containing information regarding patient demographics (such as gender, age and body mass index), treatment (such as prone positioning, optimal positive end-expiratory pressure titration and steroids (yes/no)), complications (such as pulmonary aspergillosis) and outcome (such as ventilator-free days and ICU mortality). Real-time availability of these data via the dashboard provided us with the opportunity to quickly reflect on treatment regimens and clinical outcomes, without the need to await findings from national and international database studies. To optimise its usefulness to continuously drive care improvement, we implemented a plan–do–check–act procedure. Clinical outcomes of the different treatment protocols were continuously analysed and discussed during a weekly meeting, and amended if necessary (‘plan’). Protocol adjustments (‘do’) were closely monitored using the data infrastructure and the dashboard with a continuous real time data feed (‘check’). Results were evaluated during the weekly meeting or earlier when needed, and outcomes and possible additional actions were discussed (‘act’). This infrastructure enabled us to detect the high incidence of pulmonary embolisms in patients with COVID-19, the first group in the Netherlands, and more recently, it contributed to adaptations in our local COVID-19 ICU treatment protocols leading to implementation of a pulse dose intravenous methylprednisolone.8–10 As such, we believe that structuring and organizing these vast amounts of clinical data on a local level is fundamental to leverage data analytics to improve quality of care. Currently, 19 August 2021, the research database contains information of 546 COVID-19 ICU patients and increasing, with an overall mortality of 22.9% (125 patients), 69% were male (377 patients) and median age is 63 years (IQR 54–69). The data infrastructure is organised in such a way that it is automatically updated (by means of the scheduled SQL jobs in SAS Viya), facilitating continuous real-time data analysis in order to answer urgent clinical questions. To date, the data infrastructure is limited to EHR data and will therefore be enriched with data from multiple information systems (imaging, microbiology and bedside monitors) to warrant future use. ## Conclusion We demonstrate the successful development of a real-time data infrastructure that enabled both data-driven care and decision making and rapid answering of critical clinical questions during pressing times, such as the COVID-19 pandemic. Although this data infrastructure was developed in the ICU, the underlying process could be extrapolated to other specialties to enable real-time data analysis to eventually improve quality of care.2 By sharing our steps and clinical use, we aim to inspire others to make more efficient use of the data at hand, now and in the future. ## Data availability statement No data are available. Not applicable. ## Ethics statements ### Patient consent for publication Not applicable. ### Ethics approval The need for written informed consent was waived by the regional medical research ethics committee of the Erasmus MC University Medical Center, Rotterdam, the Netherlands. ## Footnotes * Contributors DvdS designed and drafted the manuscript. MEVG participated in the design and drafted the manuscript. JH participated in the design and drafted the manuscript. RERV participated in the design and reviewed the manuscript. YM participated in the design and reviewed the manuscript. DG conceived the study, participated in its design and reviewed the manuscript. JvB conceived the study, participated in its design and coordination sign and reviewed the manuscript. All authors read and approved the final manuscript for submission. * Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. * Competing interests DG has received speakers fees and travel expenses from Dräger, GE Healthcare (medical advisory board 2009–12), Maquet and Novalung (medical advisory board 2015–18). JH currently works as industry expert healthcare at SAS Institute. No financial relationships exists that could be construed as a potential conflict of interest. All other authors declare no competing interests. * Provenance and peer review Not commissioned; externally peer reviewed. * Received July 20, 2021. * Accepted September 13, 2021. * © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/). ## References 1. Arabi YM, Azoulay E, Al-Dorzi HM, et al. How the COVID-19 pandemic will change the future of critical care. Intensive Care Med 2021;47:282–91.[doi:10.1007/s00134-021-06352-y](http://dx.doi.org/10.1007/s00134-021-06352-y) 2. van de Sande D, van Genderen ME, Huiskens J, et al. Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit. Intensive Care Med 2021;47:750–60.[doi:10.1007/s00134-021-06446-7](http://dx.doi.org/10.1007/s00134-021-06446-7)pmid:http://www.ncbi.nlm.nih.gov/pubmed/34089064 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 3. Big hopes for big data. Nat Med 2020;26:1. [doi:10.1038/s41591-019-0740-8](http://dx.doi.org/10.1038/s41591-019-0740-8)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31932805 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1038/s41591-020-0883-7&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 4. Guo C, Chen J. Big data analytics in healthcare: data-driven methods for typical treatment pattern mining. J Syst Sci Syst Eng 2019;28:694–714.[doi:10.1007/s11518-019-5437-5](http://dx.doi.org/10.1007/s11518-019-5437-5) 5. Fleuren LM, de Bruin DP, Tonutti M, et al. Large-scale ICU data sharing for global collaboration: the first 1633 critically ill COVID-19 patients in the Dutch data Warehouse. Intensive Care Med 2021;47:478–81.[doi:10.1007/s00134-021-06361-x](http://dx.doi.org/10.1007/s00134-021-06361-x)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33595710 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 6. Thoral PJ, Peppink JM, Driessen RH, et al. Sharing ICU patient data responsibly under the society of critical care Medicine/European Society of intensive care medicine joint data science collaboration: the Amsterdam University medical centers database (AmsterdamUMCdb) example. Crit Care Med 2021;49:e563–77.[doi:10.1097/CCM.0000000000004916](http://dx.doi.org/10.1097/CCM.0000000000004916)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33625129 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 7. Millar JE, Busse R, Fraser JF, et al. Apples and oranges: international comparisons of COVID-19 observational studies in ICUs. Lancet Respir Med 2020;8:952–3.[doi:10.1016/S2213-2600(20)30368-4](http://dx.doi.org/10.1016/S2213-2600(20)30368-4)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32835653 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 8. Endeman H, van der Zee P, van Genderen ME, et al. Progressive respiratory failure in COVID-19: a hypothesis. Lancet Infect Dis 2020;20:1365. [doi:10.1016/S1473-3099(20)30366-2](http://dx.doi.org/10.1016/S1473-3099(20)30366-2)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32530428 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 9. Klok FA, Kruip MJHA, van der Meer NJM, et al. Incidence of thrombotic complications in critically ill ICU patients with COVID-19. Thromb Res 2020;191:145–7.[doi:10.1016/j.thromres.2020.04.013](http://dx.doi.org/10.1016/j.thromres.2020.04.013)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32291094 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.thromres.2020.04.013&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjhci%2F28%2F1%2Fe100447.atom) 10. Endeman H. Immunomodulatie ARDS bij COVID (SARS-CoV-2): erasmus medical center adult intensive care department, 2021. Available: [https://icv-erasmusmc.nl/protocol/immunomodulatie-ards-bij-covid/](https://icv-erasmusmc.nl/protocol/immunomodulatie-ards-bij-covid/)