Article Text
PDF
Abstract
Objective To define a core dataset for intensive care unit (ICU) patients outcome prediction in Iran. This core data set will lead us to design ICU outcome prediction models with the most effective parameters.
Methods A combination of literature review, national survey and expert consensus meetings were used. First, a literature review was performed by a general search in PubMed to find the most appropriate models for intensive care mortality prediction and their parameters. Second, in a national survey, experts from a couple of medical centres in all parts of Iran were asked to comment on a list of items retrieved from the earlier literature review study. In the next step, a multi-disciplinary committee of experts was installed. In four meetings, each data item was examined separately and included/excluded by committee consensus.
Results The combination of the literature review findings and experts’ consensus resulted in a draft dataset including 26 data items. Ninety-two percent of data items in the draft dataset were retrieved from the literature study and the others were suggested by the experts. The final dataset of 24 data items covers patient history and physical examination, chemistry, vital signs, oxygenations and some more specific parameters.
Conclusions This dataset was designed to develop a nationwide prognostic model for predicting ICU mortality and length of stay. This dataset opens the door for creating standardised approaches in data collection in the Iranian intensive care unit estimation of resource utility.
- data sets
- prognosis
- risk assessment
- intensive care units
- Iran
Commons license http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
INTRODUCTION
The burden of critical illness is high worldwide, including in the developing countries.1,2 Not surprisingly, there has been a resurgence of interest in critical care medicine in these countries.3 However, interventions that have been shown to benefit critically ill patients in the developed countries may not be easily translated into intensive care settings in the developing countries3; moreover, certain interventions may not benefit critically ill patients in these countries or may even cause harm.3
There have been hardly any prospective studies with high quality, testing clinical scoring systems in critically ill patients in the developing countries.4 Those studies reported poor validation of clinical scoring systems, but have not explored a new model or approach. Case-mix of ICU patients, critical care facilities (laboratory tests, monitoring equipment, point of care testing, etc.), as well as staffing resources in the developing countries, are not directly comparable to those used in developed countries.5 ICU patients in the developing countries have a higher proportion of young adults and are more likely to be suffering from trauma and infectious illnesses, such as sepsis. Hence, the data which should be collected to manage the patients and make decision might be different.6
Clinical critical care scoring systems are used widely for categorisation and prognostication of ICU patients, helping clinical decision making and resource planning in individual ICUs, comparing quality of patient care across ICUs, and standardizing research in the field of critical care medicine. Unfortunately, available clinical scoring systems have almost exclusively been designed for, and validated in the developed countries. There is some evidence that the establishment of such a system improves critical care services.7
These scoring systems predict outcomes (e.g., mortality) using values of various parameters which are chosen by diverse approaches. But regarding the type of ICU patients or outcomes to be predicted, the parameters are different.8 There are various types of risk scoring systems used for ICU patients which differ depending on various affecting parameters. Studies showed that the conventional systems [e.g., acute physiology and chronic health evaluation (APACHE) II, IV] were not valid in some countries and it may be because of calculation type or parameter selection.8 Various results have been published for the Iranian population over the last 10 years.9–13 This motivated the authors to inspect for the most related risk factors in Iranian population and inspect for the calculation methods. The objective of this study is to define a core (minimum) data set of parameters tailored to Iranian ICU patients, as the first phase of designing/calibrating a national ICU risk scoring system.
METHODS
The development of this minimum dataset was conducted in two phases. In the first phase to evaluate the existing clinical scoring models, a literature review was performed. In the second phase, experts’ consensus approach was applied to comment on the findings of the literature review and define the final parameters.
Literature review and conventional models study
To retrieve data regarding ICU outcome prediction in the international literature, a systematic PubMed search has been performed until 31 December 2016. Keywords and terms related to ICU mortality prognostic models were used (as more extensively described in Table 1). Citations in-process, which are not indexed with MeSH headings, were also searched. All original articles describing ICU mortality risk models (for adult patients) and their parameters were considered and all data items that describe a parameter of the models were extracted from the relevant articles by two authors. Duplicated studies and studies written in a language other than English were removed. Two authors classified the papers by reading their title and abstract. In this phase, the models that are specifically used for ICU patients were screened. The results were compared and discussed until consensus was reached. References were searched manually to find potentially suitable articles that were missed during the systematic search. Finally, the parameters of the extracted models were aggregated and then the duplicate items were removed.
Consensus of experts
A multidisciplinary committee of five anaesthesiologists, five intensivists and two medical informaticians was installed. At the first committee, meeting experts agreed on the objective and the scope of the core dataset as the smallest set of data and their definitions, necessary for predicting mortality and length of stay of adult patients in ICUs. The committee agreed to define the minimum dataset, allowing experts to add data items which may be needed for more complex and specific cases. The parameters extracted from the existing mortality risk models were listed in a semi-structured questionnaire and were sent to a group of 32 anaesthesiologists and 41 intensivists in 21 centres in all parts of the country (in seven metropolises, six big and four small cities). The clinicians were asked to answer three questions clearly. First, if it is possible to collect these items in their centres, second to mention arguably if each of the parameters are unnecessary for prediction of mortality or length of stay in ICU, and third, if they think there exists other effective parameters which did not mentioned in the designed list. In order to avoid experts to blindly trust the literature, the results of the literature review were provided to the experts only to check their decisions on what data items should be included in the final version of the core dataset. Answers to each question separately were aggregated and summarised and were presented to the multidisciplinary committee at the second meeting. During four other sessions, the committee examined the listed and suggested items one by one. Each item was eliminated if could not been collected in at least one centre. This decision was taken to make the result useful anywhere in the country. All of the newly suggested items and the other parameters, which were known necessary by at least one-third of the clinicians (≤12), were examined separately and included/excluded by committee consensus. Data items were extended or restricted until agreement was reached. Finally, all included parameters (data items) were listed and reported as the core dataset for the national ICU risk scoring system.
RESULTS
Literature review and development process
Our electronic search in Medline resulted in 12,792 unique articles. A total of 258 articles remained after title and abstract screening of which 203 articles were included for review of the full text. Consequently, a total of 24 articles on prediction of ICU mortality were included. Furthermore, two eligible studies were found by searching references of the included studies. By conducting literature review and extracting existing models, 11 models from these articles were included. Figure 1 shows the flow diagram of the systematic search of the study.
The extracted models were approved by the committee and after aggregating and removing duplicate items, 51 distinct data items were extracted and sent to the clinicians. As clinicians emphasized, all of the data items are collectable wherever around the country. Thirty-five of 73 experts (11 anaesthesiologists and 24 intensivists) (48% of all, 34% of anaesthesiologists and 58% of intensivists) responded to questionnaires. Based on the clinicians’ responses, 24 (47%) data items were considered as necessary and therefore was added to the draft of the dataset. The comparison between the results of the literature review study and the data items included in this step showed that in total almost 53% of the data items were considered as unnecessary and then were eliminated. The clinicians also suggested two new parameters which discussed as new data items in the experts meetings. These two data items which were not mentioned in the literature review study were ‘Type of nutrition’ and ‘Drug Addiction’ (Table 2). As the objective was to design a minimum dataset applicable for all ICUs, the suggested data items were added to the dataset, and the new list was presented to the multidisciplinary committee. The draft list of data items presented in Table 2. These data items, which were selected by clinicians, were discussed in the committee. Of the included data items from literature, 29 data items were excluded from the draft of the core dataset (see Table 2). In this step, there were 26 data items in the draft list of prognostic parameters.
Final version of the dataset
After discussion on the data items in the draft dataset in the multidisciplinary committee, 29 data items was deleted and no new data items (24 data items based on literature and two new data items) were added. These data items were categorised into four categories: patient history and physical examination, chemistry, vital signs, oxygenations and the others (Table 2). The highest number of data items was related to the category ‘chemistry’. Table 2 also shows the list of the final dataset. The complete description of the core dataset can be acquired from the authors.
DISCUSSION
The Iranian outcome prognostic core dataset was designed for developing a nationwide prognostic model to predict ICU mortality and length of stay. This dataset includes 24 data items. Information concerning patient laboratory items (chemistry) is an essential component of the ICU outcome prognostic model by experts’ consensus. The majority of data items (20 out of 24) in the core dataset were necessary to consider in the prognostic model and the others would be considered if existed. A survey of clinicians’ opinion in all parts of the country based on a primary literature review helped the committee to select data items by adding disregarded data items and removing unnecessary items.
Nearly all data items which were reported in less than 33% of expert responses were not included in the expert-based draft of the dataset and the residual data items (including the data items that were included by more than 12 experts and the ones that were suggested by them to include) were discussed in the committee meetings. These accomplishments showed the necessity of performing the literature review next to the expert consensus. Since the literatures were to a certain extent diverse, the results of the literature review were provided to the experts only to check their decisions on what data items should be included in the final version of the core dataset. In other words, the experts were asked to score the items without names, which reduced the bias of tendency or interest to some models during item selection. The results of the literature review could have been provided beforehand, but this would have made it impossible to compare the results from the literature with the expert-based dataset.14 This dataset has been designed for general intensive care patients and it may not appropriately cover some patients in specific ICUs like cardiac surgery ICU patients or paediatric ICUs. The current core dataset includes some residual categories such as ‘Metastatic Cancer’ or ‘Ventilated’ which have to be made more explicit when applicable. This flexibility may enable using the dataset for more centres with higher complexity level of surgical ICU cases. To accomplish a whole ICU outcome prediction more data items may be required.
This dataset includes data items that are important for the risk assessment of the patient and items that health care providers would like to know at the first day of ICU admission. Of Course, to use this dataset in the risk model development process, one must use real values of each data item in large amount and check the statistical significance. But here, reviewing literature and a consensus on experts’ opinion led to make a comprehensive appropriate dataset that is collectable in all parts of the country. This study was performed as a part of a larger project in order to calibrate/design appropriate mortality risk scores for Iran.
Comparing our dataset to other sets of famous scoring systems, we found that Iranian experts in both steps selected the items which are exactly chosen for the APACHE IV mortality risk score.15,16 This may be occurred because of the relatively high use and familiarity with this risk scoring system in Iran.17
Similar to this study, Ahmadian et al. also used a combination of literature review and expert consensus to develop a core dataset. Both methods used in the study proved to be valuable and complementary.9,10 Also, Simmons et al.18 designed a national dataset for monitoring diabetes patients by reviewing only three published datasets and applying a survey by recruiting 147 experts. In this study, our consensus approach was a combination of a nationwide survey of clinicians’ opinion and real face-to-face meetings which lead to good insight in required data items to develop a practical prediction model. Considering the results of literature review and the survey as input in experts meetings could optimized discussion on all data items and resulted in overall agreement (consensus). It is doubtful whether this could be reached as easy by using tele conference and emails as used in the study of Campbell et al.19 Also, without a national wide survey we may lose many opinions or could not distinguish collectable items regarding the low resource in some centres.4
These national datasets would improve clearness and uniformity of written communication among clinicians and provide information that is both essential and desirable for patient management in Iran.20 This would be done by developing optimum national calibrated scoring models which help to assess care quality more strongly. Moreover, the implementation of this dataset in the healthcare settings would prevent unnecessary resource allocation. The next step consists of creating a proper data dictionary for the designed set of data to improve overall understanding of data items and to standardize definitions and ensure consistency of use for all data items used in ICUs.20,21
CONCLUSIONS
The combination of literature review and specialists consensus provided a proper foundation for designing the dataset. This approach may be useful for designing datasets in other domains. The diversity in the ICU outcome prediction data collection found by the literature review shows that expert panels are needed to determine the appropriate data items. On the other hand, using only the experts’ consensus would not be sufficient because they may simply overlook some data items. The literature helped our experts to carry out useful modifications in the dataset. This core dataset will enable healthcare settings to evolve towards calibration/designing a proper tool for predicting outcome in ICUs in Iran.