Introduction

This retrospective cohort study described referral patterns from family physicians (FPs) to other medical specialties. Patterns of referrals reflect standards of care, physician practice scope and patient expectations, and are influenced by policy,1^,2 geography,3 physician4 and patient characteristics.3^, 5^, 6 Most variability in referral rates arises from the patient.3^, 5^, 6 Clinical factors such as chronic conditions are of particular importance.4^,7 Primary care electronic medical record (EMR) databases are ideally suited to explore these clinical influences. Unlike registries or health administrative databases, primary care EMRs are clinically comprehensive and contain patient data unavailable elsewhere.

In order to take advantage of the rich data available in EMRs, appropriate statistical modelling must be employed. There are many outcomes in primary care research that take the form of counts; for example, physician visits, referrals to other providers, chronic conditions, medications and diagnostic tests. Logistic regression, where counts are dichotomised, is often used to model these data. While not incorrect, dichotomizing always results in the loss of valuable information.8 An alternative that maintains the variation in the outcome is to use a multi-variable technique such as Poisson regression which can model count data. One assumption of the Poisson distribution is that the mean and variance are the same. However, primary care data are often over-dispersed (with a large number of zero counts in the data), meaning that this assumption is not met. For example, in modelling the number of FP visits made by a population in a year, there will be many people who do not visit at all, some who visit only once or twice, and progressively smaller numbers of people with more visits. Poisson regression is not appropriate in this situation. The negative binomial distribution is more flexible and is well suited to handle over-dispersed data.9

Further complicating the study of primary care count variables is the fact that much of primary care data are collected about patients within practice settings. This is especially true in the growing area of EMR database research, where patient level data are collected for many practices. This clustering of the data must be accounted for, using for example, multi-level modelling techniques which allow for the apportioning of variance between patient and practice levels. Until recently, there was no readily available software that could perform multi-level negative binomial regression, a technique that can both properly model over-dispersed count data and account for the clustering of individual patient level data within practice settings. With the recent inclusion of multi-level negative binomial regression in statistical software packages, its use has grown in popularity.10

This paper provides an illustration of multi-level negative binomial regression which models over-dispersed health care count data (the number of referrals) and accounts for the clustering of patients within practices. The methodologic insights gained from this study have relevance to future studies on many research questions that utilise count data, both within primary care and broader health services research.

Methods

Results

Table 1 shows the location of the eligible practice sites across the provinces and networks and the number of patients within each.

There were 78,731 medical referrals during the study period for 87,258 patients from 28 eligible practice sites. The mean exposure time (first visit to the end of data period) was 39.6 months [Standard Deviation (SD) = 13.5]. Referral rates ranged across practice sites from 0.06 to 0.71 referrals/patient/year. The mean number of referrals was 0.29 referrals/patient/year (SD = 0.46). Table 2 reports the mean referral rates across patient-level variables. Patients had a mean of 14.6 FP visits (SD = 13.3).

Table 3 reports the independent effects of each patient-level variable and the full model. Males were less likely to have as many referrals as females, and the probability of referrals increased with age, the total number of chronic conditions and the number of FP visits.

Discussion

Using multi-level negative binomial regression to account for over-dispersion of data, this study demonstrated statistical modelling that will allow for a more refined understanding of the influence of patient, physician, practice and jurisdictional levels on referrals. While this study extends our statistical modelling for primary care count data, there were limitations in the application of our model.

The number of level 2 groups (practice sites) was not large enough to guarantee unbiased estimates in a multi-level model. Several researchers have used Monte Carlo simulations to investigate the effects of level 1 and level 2 sample sizes on the precision of variance components, estimates and standard errors.15 While there is some disagreement over the robustness of fixed and random effects,16 there are similar findings that the standard errors are generally underestimated when the number of groups is below 50 and the model is complex, with additional parameters and a non-linear distribution (such as the negative binomial distribution we have used).17 For a more thorough explanation of cluster robust inference, the reader is referred to Cameron and Miller.18

Regarding representativeness of the data, the current age and sex of the CPCSSN database have been found to be somewhat representative of the Canadian population as measured by the Canadian census.19 Further, for this study in particular, missing or incomplete referrals data for half of the sites prevented modelling at the regional level in the current study and resulted in over-representation of one region, limiting generalizability. Therefore, improvements in the completeness of EMR data are needed in order to model additional levels in the analysis of count data.

In addition, the distribution of practice locations was unbalanced with over 90% of the sites urban. This may have limited the improvement in the multi-level model when we included this practice-level factor. Other practice-level factors were not measured, including an analysis by province, as several sites within provinces were ineligible for inclusion in the study, and the remaining sample was predominantly from Ontario. Improvements in our ability to capture accurate practice-level characteristics are needed to model levels beyond the individual patient level.

The overall referral rate (0.29 referrals/year/patient) was lower than two regional Canadian studies (0.563 and 0.466). While this may be attributable to time effects, differences in referral practice by jurisdiction, and/or differences in patient populations, it is likely that this reflects the effect of missing and/or incomplete data from some practice sites. Despite the lower referral rate, the model was consistent with previous research,3^–5 where referral rates increased with patient age, female sex and more exposure to the physician through FP visits where a referral might occur. To our knowledge, this is the first referrals study to account for multiple chronic conditions. Unsurprisingly, the number of medical referrals increased with morbidity level.

Despite its limitations, the analysis conducted in this paper is a major step forward in the methods used to understand EMR data reported in the primary care literature.6^,20 This multi-level negative binomial analysis can serve as an illustration for the modelling of myriad count outcomes that are important in primary and other healthcare research. In particular, researchers using pooled EMR data from several practice sites should employ multi-level modelling to account for the clustering of these data. As the accuracy and completeness of EMR data improve over time, the power of these analytic techniques will further increase.