Article Text

Download PDFPDF

Generating insights in uncharted territories: real-time learning from data in critically ill patients–an implementer report
  1. Davy van de Sande1,
  2. Michel E. Van Genderen1,
  3. Joost Huiskens2,
  4. Robert E. R. Veen3,
  5. Yvonne Meijerink4,
  6. Diederik Gommers1 and
  7. Jasper van Bommel1
  1. 1Department of Adult Intensive Care, Erasmus MC University Medical Center, Rotterdam, The Netherlands
  2. 2Department of Health, SAS Institute Inc, Huizen, The Netherlands
  3. 3Information Technology; project Research Suite, Erasmus MC University Medical Center, Rotterdam, The Netherlands
  4. 4Data and Analytics, Erasmus MC University Medical Center, Rotterdam, The Netherlands
  1. Correspondence to Dr Michel E. Van Genderen; m.vangenderen{at}


Introduction In the current situation, clinical patient data are often siloed in multiple hospital information systems. Especially in the intensive care unit (ICU), large volumes of clinical data are routinely collected through continuous patient monitoring. Although these data often contain useful information for clinical decision making, they are not frequently used to improve quality of care. During, but also after, pressing times, data-driven methods can be used to mine treatment patterns from clinical data to determine the best treatment options from a hospitals own clinical data.

Methods In this implementer report, we describe how we implemented a data infrastructure that enabled us to learn in real time from consecutive COVID-19 ICU admissions. In addition, we explain our step-by-step multidisciplinary approach to establish such a data infrastructure.

Conclusion By sharing our steps and approach, we aim to inspire others, in and outside ICU walls, to make more efficient use of data at hand, now and in the future.

  • data science
  • machine learning
  • artificial intelligence
  • critical care outcomes
  • COVID-19

Data availability statement

No data are available. Not applicable.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from


The current pandemic demonstrated that healthcare was in uncharted territory. As such, the outbreak of the novel COVID-19 could be a turning point for the initiation of advanced analytics, especially in intensive care medicine. The pandemic emphasized the importance of instant, or even real time, analysis of large volumes of intensive care unit (ICU) data in order to generate new insights to eventually improve quality of care.1

The ICU typically is an environment where clinicians are confronted with large amounts of clinical data siloed in multiple information systems and are often not optimally used to aid clinical decision making.2 Even more, data are often collected when dealing with complex diseases or conditions to improve understanding of the clinical course and to evaluate the effects of therapeutic interventions at a later stage. However, especially in pressing times, advanced data analytics tools (eg, artificial intelligence) can be helpful by mining large volumes of data and discover clinical patterns and best treatment options from clinical data.3 4

Immediately after the outbreak, several initiatives such as the Dutch Data Warehouse and the covidpredict initiative were announced aiming to collect large amounts of data on COVID-19 ICU patients to eventually optimise clinical care and improve outcome.5 However, the collection and organisation of data from multiple ICUs is time consuming and is often obstructed by technical and privacy challenges.6 Furthermore, the agglomeration of data is unresponsive to local variations in population or disease-specific patterns, and different local practices and clinical definitions impede proper comparison between cohorts.7

Given these considerations, we aimed to implement a local data infrastructure that would enable us to learn in real time from our own consecutive COVID-19 ICU admissions by comparing patient characteristics, treatment regimens and clinical outcomes. Here, we present how a structured data approach can help to improve quality of care and can serve as a basis for advanced data analysis.

Implementing a real-time clinical data infrastructure

All consecutive patients with COVID-19 admitted to the adult ICU of the Erasmus University Medical Center, a tertiary referral centre in Rotterdam, The Netherlands, were analysed. The need for written informed consent was waived by the regional medical research ethics committee. Figure 1 provides a week-by-week overview of the data infrastructure development process along with the engaged stakeholders.

Figure 1

Road towards a real-time clinical data infrastructure. Stakeholders were engaged early in the process and tasks were distributed (dark blue=ICU research team, light blue=ICU physicians, dark grey=data and analytics department and light grey=IT department). After 6 weeks, a data infrastructure was implemented that allows to extract data in real-time from the electronic health record (HIX). *Demographics, medication, lab tests, ventilator settings, vital signs and clinical notes were extracted from the database using structured query language (SQL) in Microsoft SQL management Studio. ¥The analytics platform, SAS Viya (V.8.3) was used to process and further analyse the data. Figure 1 was created by the lead author, and permission to reuse the image was obtained. HIX, Healthcare Information eXchange; ICU, intensive care unit; IT, information technology.

From start, a weekly recurring meeting was scheduled with the local information technology (IT) department, the data and analytics department and a third software party (SAS Institute, Cary, North Carolina, USA). Clinical questions and needs were formulated and determined in collaboration with a team of ICU physicians. Physicians were particularly interested in the treatment effect of several interventions, such as the effect of prone positioning, effect of high or low dose steroids, timing of steroids and the influence of body mass index on several outcome parameters, such as survival and mechanical ventilation days, in these critically ill patients. In our hospital, all patient data from the electronic health record (EHR), Healthcare Information eXchange (HIX), are routinely stored in a structured query language (SQL) server database (referred to as ‘database’). Since these data originate from a single source, it was already properly formatted (data engineering) and did need not require additional harmonisation. As such, the data and analytics department established a live connection to the database server (by using Microsoft SQL management studio) for qualified members of the ICU research team with restricted access to ICU patients. Subsequently, the IT department installed a data analytics platform (SAS Viya V.8.3) to ensure data extraction, data management, facilitate data visualisation and advanced analytics, and data and model governance.

In the second week, relevant data were identified in the database (figure 1), and SQL queries were written to extract these data, supervised by a senior data analyst. The data and analytics department provided support throughout the data identification and extraction process. Extracted data were continuously verified with the team of ICU physicians to ensure its completeness.

From the third to the fourth week tables containing, raw data were joined from separate queries and processed using SAS programming language. To be able to cope with the rapidly growing number of patients with COVID-19, ICU capacity expanded from 45 to 102 beds in a matter of days, mostly towards other parts of the hospital. To oversee patient characteristics in multiple ICU locations, a real-time ‘COVID-19 clinical data dashboard’ was required and was developed in the fifth week. Since then, the data were further processed, and in the sixth week, a research database was constructed that could be used to perform in-depth advanced analysis.

Analysing clinical data in real time

A dashboard was successfully constructed containing information regarding patient demographics (such as gender, age and body mass index), treatment (such as prone positioning, optimal positive end-expiratory pressure titration and steroids (yes/no)), complications (such as pulmonary aspergillosis) and outcome (such as ventilator-free days and ICU mortality). Real-time availability of these data via the dashboard provided us with the opportunity to quickly reflect on treatment regimens and clinical outcomes, without the need to await findings from national and international database studies. To optimise its usefulness to continuously drive care improvement, we implemented a plan–do–check–act procedure. Clinical outcomes of the different treatment protocols were continuously analysed and discussed during a weekly meeting, and amended if necessary (‘plan’). Protocol adjustments (‘do’) were closely monitored using the data infrastructure and the dashboard with a continuous real time data feed (‘check’). Results were evaluated during the weekly meeting or earlier when needed, and outcomes and possible additional actions were discussed (‘act’). This infrastructure enabled us to detect the high incidence of pulmonary embolisms in patients with COVID-19, the first group in the Netherlands, and more recently, it contributed to adaptations in our local COVID-19 ICU treatment protocols leading to implementation of a pulse dose intravenous methylprednisolone.8–10 As such, we believe that structuring and organizing these vast amounts of clinical data on a local level is fundamental to leverage data analytics to improve quality of care.

Currently, 19 August 2021, the research database contains information of 546 COVID-19 ICU patients and increasing, with an overall mortality of 22.9% (125 patients), 69% were male (377 patients) and median age is 63 years (IQR 54–69). The data infrastructure is organised in such a way that it is automatically updated (by means of the scheduled SQL jobs in SAS Viya), facilitating continuous real-time data analysis in order to answer urgent clinical questions. To date, the data infrastructure is limited to EHR data and will therefore be enriched with data from multiple information systems (imaging, microbiology and bedside monitors) to warrant future use.


We demonstrate the successful development of a real-time data infrastructure that enabled both data-driven care and decision making and rapid answering of critical clinical questions during pressing times, such as the COVID-19 pandemic. Although this data infrastructure was developed in the ICU, the underlying process could be extrapolated to other specialties to enable real-time data analysis to eventually improve quality of care.2 By sharing our steps and clinical use, we aim to inspire others to make more efficient use of the data at hand, now and in the future.

Data availability statement

No data are available. Not applicable.

Ethics statements

Patient consent for publication

Ethics approval

The need for written informed consent was waived by the regional medical research ethics committee of the Erasmus MC University Medical Center, Rotterdam, the Netherlands.



  • Contributors DvdS designed and drafted the manuscript. MEVG participated in the design and drafted the manuscript. JH participated in the design and drafted the manuscript. RERV participated in the design and reviewed the manuscript. YM participated in the design and reviewed the manuscript. DG conceived the study, participated in its design and reviewed the manuscript. JvB conceived the study, participated in its design and coordination sign and reviewed the manuscript. All authors read and approved the final manuscript for submission.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests DG has received speakers fees and travel expenses from Dräger, GE Healthcare (medical advisory board 2009–12), Maquet and Novalung (medical advisory board 2015–18). JH currently works as industry expert healthcare at SAS Institute. No financial relationships exists that could be construed as a potential conflict of interest. All other authors declare no competing interests.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.