A conceptual framework for the OECD Health Care Quality Indicators Project

Abstract

Issues. The Health Care Quality Indicator (HCQI) Project of the Organization for Economic Cooperation and Development (OECD), which is aimed at developing a set of indicators for comparing the quality of health care across OECD member countries, requires a balanced conceptual framework that outlines the main concepts and domains of performance that should be captured for the current and subsequent phases of the project.

Addressing the issues. This article develops a conceptual framework for the OECD’s HCQI Project. It first argues that developing such a framework should start by addressing the question, ‘performance of what—and to what ends?’ We identify at least two different major classes of frameworks: (i) health and (ii) health care performance frameworks, both of which are in common use. For the HCQI, we suggest a conceptual framework that is largely a purposeful modification of the existing performance frameworks and which is driven by the health determinants model.

Conclusions. The conceptual basis for performance frameworks can be traced back to the health determinants model. A health performance framework takes a broader, societal or public health view of health determination, whereas a health care performance takes a narrower, mostly clinical or technical view of health care in relation to health (needs). This article proposes an HCQI framework that focuses on the quality of health care, maintains a broader perspective on health and its other determinants, and recognizes the key aims of health policy.

It has understandably become commonplace for countries to formally assess and ‘incentivize’ the performance of their health care systems [1–3]. Umbrella organizations such as the World Health Organization (WHO) and the Organization for Economic Cooperation and Development (OECD) have taken an international lead in encouraging health system performance measurement [2,4,5]. The reasons for the increased interests include rising costs, aging populations, market failures, poor quality, medical errors, lack of accountability, and inequalities [1,2,6]. In all these, there are widespread perceptions of poor value for the money and effort spent on health care [1,7].

To manage their perennial problems, many industrialized countries have, among other things, sought to manage ‘health production’ and their health goals through performance measurement. As a result, there has been a proliferation of health and health care indicators. To manage the proliferation of indicators, aid prioritization, and ensure coherence, health care and its influence on health must be adequately conceptualized using conceptual frameworks [3,8–10].

The OECD’s Health Care Quality Indicator (HCQI) Project is an international effort aimed at developing a common set of indicators for raising questions about the quality of health care delivered across OECD member countries. It is foreseen that the project, which builds on existing performance efforts, will also fill an important gap in the well-known OECD Health Data. A sound conceptual framework is necessary to define what is meant by ‘quality of health care’ and to place it within a wider performance framework which acknowledges the key health policy goals adopted by the OECD and its member countries. This article presents a conceptual framework for the HCQI Project.

Methods

To develop such a framework, this article builds on (i) recent reviews of performance measurement systems [3,11,12], (ii) extensive consultation with the national representatives to the HCQI Project, and (iii) conceptual analysis. Owing to feasibility and cost concerns, the consensus and synthesis approach employed here differs from other known structural approaches [13,14]. The presentation is as follows: We first argue that addressing the question, ‘performance of what—and to what ends?’ makes a difference as to what type of framework should be developed. We then go on to outline how an examination of what determine(s) health can provide the first step toward designing a framework for the HCQI. This article draws extensively from the performance experiences of the UK, Canada, Australia, the USA, the European Community Health Indicators Project (ECHI), The Commonwealth Fund’s International Quality Indicator Project, the WHO, and the OECD itself.

Health care and other determinants of health

By ‘health care’ we mean the combined functioning of public health and personal health care services. A health system includes all activities and structures whose primary purpose is to influence health in its broadest sense. This notion is in keeping with WHO’s use of the term health system: ‘all the activities whose primary purpose is to promote, restore or maintain health’ [4].

Health is determined by many interdependent factors, one of which is health care. (See Figure 1, which is illustrative rather than actually representative of possible relationships and divisions.) This multi-determinant approach to health was principally advanced by the Lalonde ‘White Paper’ in Canada in 1974 [15], and subsequently expanded on in many publications, including the seminal paper by Evans and Stoddart [16]. The non-health care determinants were largely grouped into three main fields by the Lalonde paper: environment, lifestyle, and human biology [15].

Environment refers to both physical and social (living) circumstances of human existence, and how these shape or disrupt health. Lifestyle refers to the behavioral or choice-related issues such as nutritional styles, drinking, smoking, and other habits, as well as education or knowledge-based activities that influence health and illness. Human biology refers to the biological or genetic constitution that determines how the human body primarily functions to maintain a healthy state and how it responds to other, mainly exogenous, determinants of health.

In addition to its direct effects on health and illness, health care may act indirectly on the other determinants to maintain or improve health. For instance, public health—through its classical health prevention, promotion, and protection strategies—influences the host constitution, lifestyle, and environment, respectively. Health care influences human biology (for instance, via the pharmacodynamics of anti-hypertensive drugs) and/or the environment–host interaction (such as drug effects on causative agents of diseases in the human body).

In Figure 1, ‘response’ refers to the individual—mostly biological, psychological, or social—reaction to the constellation of health determinants, acting singly or in combination. This response factor represents the final common pathway to health; its final manifestation is seen as health change, illness, or disease.

The interrelationships among the health care system and other determinants of health can be of at least three types, namely:

• linkages exist between health care and health (depicted in Figure 1 by the arrows running between health care and health fields, including those running via the ‘response’ box);

• linkages exist between health care and non-health care determinants (depicted in Figure 1 by the arrows running from the health care space to lifestyle, environment, and host constitution and by arrows running between the latter non-health care determinants); and

• linkages exist between non-health care determinants and health (depicted in Figure 1 by the arrows running from lifestyle, environment, and host constitution to the health field, mainly via the ‘response’ box).

Performance of what—and to what ends?

In trying to measure performance, policymakers and researchers need to form a clearer image of what it is they want to measure and the key goals of health policy. Here, we make a distinction between conceptual frameworks for health care system performance (or health care performance) and those for health system performance (or health performance). Specifications of the key goals of health policy have been set out by many countries and by international organizations such as the WHO and the OECD.

Health care performance refers to the maintenance of an efficient and equitable system of health care without emphasizing an assessment of the non-health care determinants. That is, in an assessment of health care performance, the direct functioning of the delivery system of health care is evaluated vis-à-vis its established public goals for the level and distribution of the benefits and costs of personal and public health care. A health care performance evaluation is, therefore, concerned with linkages between health care and health, mentioned in the Health care and other determinants of health section. However, in many health care systems, clinical preventive services are used to influence clinically relevant lifestyles, for example smoking cessation as part of cardiac care.

Health performance is a much broader conceptual approach to measuring performance by explicitly using non-health care determinants, health care, and contextual information to give a clearer picture of population health. Again, the main policy goals may be efficiency and equity, but a much wider view of the determinants of health and their costs must be adopted. The equitable distribution of health status itself is an important concern, and responsiveness to consumers is augmented by the concern to influence lifestyles. Given that a health performance framework is largely concerned with all the interrelationships among health, health care, and non-health care factors, health performance subsumes health care performance. See Table 1 for an illustration of which countries and international agencies have health or health care performance frameworks.

Goals

In addition to improving health, there seems to be growing agreement that the wider goals of health policy include two key economic and social objectives: efficiency and equity. In the OECD’s approach to classifying these goals, the former is subdivided into (i) macroeconomic efficiency or sustainability (setting the right level for health expenditure—especially public expenditure on health) and (ii) microeconomic efficiency or value for money [1,29]. The concern for equity extends both to the distribution of the payments for health care across the population (fair financing) and to the distribution of access to health services across the population (fair access). In other words, to what extent is payment according to ability to pay and to what extent is access according to need?

Dimensions of health care performance

Dimensions of health care performance are those definable, preferably measurable, attributes of the system that are related to its functioning to maintain, restore, or improve health [30–32]. Table 2 gives an overview of commonly used performance dimensions in selected countries and agencies. The list that follows has been designed to be inclusive. There is undoubtedly some overlap and redundancy in the concepts reported below.

A common key dimension seen in all performance frameworks is effectiveness, which is the degree of achieving desirable outcomes, given the correct provision of evidence-based health care services to all who could benefit but not to those who would not benefit [3,6,31,33,34]. Donabedian stresses that effectiveness is the extent to which attainable improvements in health are, in fact, attained [35]. For a system to deliver effective care effectively, other dimensions of performance must be in place.

Appropriateness, as a performance dimension, is the degree to which provided health care is relevant to the clinical needs, given the current best evidence [32]. Safety is a dimension where the system has the right structures, renders services, and attains results in ways that prevent harm to the user, provider, or environment [31,32].

Efficiency involves finding the right level of resources for the system and ensuring that these resources are used to yield maximum benefits or results [30,31,35]. The OECD framework includes ‘health expenditure’ or cost as part of efficiency: the ‘macroeconomic’ aim is to find the sustainable level of health spending (especially of public spending on health), and the ‘microeconomic’ efficiency goal is to minimize expenditure for any given level of outcomes and responsiveness [29]. Finding the right level of public expenditure on health will be a matter for political judgement. However, a set of good performance indicators for a health system—especially measurement of outcomes—can play a valuable part in informing such judgements.

Continuity addresses the extent to which health care for specified users, over time, is smoothly organized within providers and institutions. Coordination can then be seen as health care being smoothly organized across providers and institutions. Continuity can be measured from the patient’s perspective and can end up as part of patient-centeredness, whereas coordination is difficult to measure from the patient’s perspective because a lot of it goes on ‘behind the scenes’. Accessibility is the ease with which health services are reached. Access can be physical, financial, or psychological and requires that health services are a priori available. A closely related dimension, therefore, is equity, which defines the extent to which a system deals fairly with all concerned. Equity deals both with the distribution of the burden of paying for health care [4,29] and with the distribution of health care and its benefits among a people [33,35].

Responsiveness refers to how a system facilitates people to meet their legitimate non-health expectations [4]. Patient-centeredness captures the degree to which a system actually functions by placing the patient/user at the center of its delivery of health care and is increasingly being measured as patient experiences of health care with emphasis on caring [6,30,31]. Responsiveness and patient-centeredness are often taken to be equivalent. Timeliness is the degree to which health care is provided within the most beneficial or the necessary time window [32,35]. Timeliness may become part of accessibility or responsiveness so as to reflect patient experiences of promptness of health care [4]. Acceptability is conformity to the wishes, desires, and expectations of health care users and their families [35]. As such, acceptability is often presented as a part of or substitute for patient-centeredness, as seen in Canada (see footnote in Table 2).

There can be more dimensions of performance in use, but the ones described here are among the most common ones (see Table 2 for a comparison of dimensions across different frameworks). Clearly, a linkage of various dimensions of performance within a coherent, comprehensive, and parsimonious conceptual structure is what is needed for a performance framework. The list above cannot provide such a framework because it contains overlaps (and perhaps gaps), but it does provide a quarry for a proposal for a more unified, comprehensive, and parsimonious structure.

Developing a conceptual framework for the OECD HCQI Project

On the basis of the foregoing, for the HCQI Project, we need a conceptual framework that both recognizes the project’s focus on quality of health care (while keeping a broader perspective on health and its other determinants) and reflects a coherent set of key goals for health policy which certain member countries and the OECD itself have identified. More specifically, the proposed framework builds on the common dimensions of performance which are incorporated into a model that borrows heavily from the Institute of Medicine’s national health care quality indicator framework, developed for the USA (Figure 2). It also relies on a modification of the Canadian Health Indicator Framework (Figure 3) and its adaptations, seen in Australia and within the ECHI Project, and on the WHO and OECD proposals for identifying key economic and social goals for health policy (Figure 4). The result is a health performance framework that meshes well into the one suggested by the International Organization for Standardization (ISO) for its health informatics network [36].

The shaded area in the resulting framework (Figure 5) represents the current focus of the HCQI Project. Within this area, the Expert Group which advises the OECD on the HCQI project suggested that indicators should be developed in the following priority areas: (i) cardiac care, (ii) mental health, (iii) diabetes mellitus, (iv) patient safety, and (v) primary care and prevention/health promotion. These five areas were chosen on the basis of the high impact of the three health areas on the burden of disease, an informal survey of member countries’ priorities for indicator development in different health service areas, and a discussion about priorities at the first meeting of the HCQI Expert Group in January 2003. The HCQI framework has four interconnected tiers (to denote potential causal pathways shown in Figure 5), which represent:

• health—to capture the broader measures of the health of the society that may be influenced by health care and non-health care factors;

• non-health care determinants of health—to delineate mostly society-wide, non-health care factors that also influence health (as suggested in Figure 1 earlier);

• health care system performance—to capture the processes, inputs, and outcomes of the health care system and its efficiency and equity; and to recognize that these may sometimes influence non-health care determinants; and

• health system design and context—to give pertinent country and health system policy and delivery characteristics which affect costs, expenditure, and utilization patterns and which are necessary for appreciating and contextualizing the findings of the health care performance tier.

For the first, second, and fourth tiers of the framework (Figure 5), many data elements and indicators from the OECD Health Data and OECD Factbook indicators, for example, could be used to fill in the gaps and give a rounded picture of health progress in a country. The third tier in Figure 5 is a matrix of dimensions of health care performance (columns) by health care needs (rows). The represented dimensions are effectiveness, safety, responsiveness/patient-centeredness, access, and cost/expenditure. These map approximately into structure, process, and outcome (from right to left). Effectiveness, safety, and responsiveness/patient-centeredness are taken to be the core quality dimensions. Effectiveness covers the HCQI priority areas of cardiac care, mental health, diabetes, and primary care and prevention/health promotion. The link between the third tier (health care performance) and the second (non-health care determinants) is captured by the priority area primary care and prevention/health promotion. These dimensions are, in essence, those core attributes of health care that increase the likelihood of desired outcomes. Likewise, cost/expenditure is a dimension that is included in the framework, although it is explicitly linked on a performance level to questions of efficiency. Macroeconomic efficiency and microeconomic efficiency (written below the third tier) are concerned, respectively, with finding the right level of health expenditure and maximizing the value for money (or the ratio of quality to costs). Equity is concerned with the fairness of the distribution of health care across populations and also with the fairness of payment for health care. ‘Equity’ can also be estimated for non-health care determinants of health and for health status. Within the fourth tier (‘health system design and context’) will be located the levers that countries try to pull to influence the performance of the health system for the better.

Non-represented dimensions such as appropriateness, continuity, timeliness, and acceptability could easily be accommodated within other dimensions, as was partly done within the Commonwealth Fund’s International Quality Indicators Project [28]. For example, appropriateness based on its conceptual definition could be mapped into effectiveness, whereas continuity and acceptability could be absorbed into patient-centeredness.

Patients’ health care needs are represented by the rows in the health care performance tier of Figure 5 and are defined to reflect evolving needs over the life cycle [25]. Therefore, these patient needs have an underlying ‘clinical logic’ (i.e. a logical pathway from health to health care) and roughly depict how health care is delivered in many systems. Health care needs address issues related to (i) health maintenance, thus disease prevention and health promotion (‘staying healthy’ or prevention); (ii) health improvement/restoration (‘getting better’ or acute care), as well as (iii) continuous and integrated health management (‘living with illness/disability’ or chronic care and ‘coping with end-of-life’).

Discussion and conclusion

We have developed a conceptual framework for the OECD’s HCQI Project by first exploring the place of health care in health system performance and by revisiting health care as one of the determinants of health. We saw that a health performance framework subsumed a health care performance framework when a model of health determinants was taken as a valid basis for performance evaluation.

The proposed HCQI framework highlights two important points. Firstly, it allows for the reality that although high quality health care should produce desired health outcomes and responsiveness to consumers, such outcomes will vary between and across individuals, communities, and health care systems because of varying preferences, structures, and responsibilities. We recognize that OECD health care systems vary in their policies, responsibilities, and structure for managing health care and non-health care determinants of health, an issue that is best highlighted in the fourth tier, ‘health system design and context’, of the HCQI framework (Figure 5). This new framework embeds health care within the broader health system and even within the economic, social, and political context of OECD countries. Secondly, there are factors other than providers and the health care system which will influence health outcomes. A framework that is unduly narrow and clinical in its focus will miss this larger picture and interpretation. The increasing recognition of using a broader framework is also underscored by the UK’s recent revision of its standards framework to include Public Health in addition to safety, clinical, and cost effectiveness, governance, patient focus, accessible and responsive care, and care environment and amenities [20].

Having a complex and integral framework such as the Canadian framework is not just intended to tie up loose theoretical ends—no matter how desirable that might be—but is meant to reflect the broader goals, setup, and nature of the system in question. The fact that the UK, Canada, Australia, the USA, WHO, and now the OECD are all investing in (increasingly complex and integral) performance frameworks lends support to the growing recognition of the need for conceptual clarity in performance measurement [2,3,8].

Beyond conceptual concerns, indicator selection and prioritization of health areas can undermine a performance framework if care is not taken to define the selection and prioritization criteria beforehand. These criteria have been addressed in recent literature, which suggest that the key criteria for selecting indicators are importance (including disease burden) of what is being measured, scientific soundness (i.e. validity, reliability, and explicit evidence) of measures, and their feasibility (i.e. mainly data needs and cost of measurement) [25,37] and that burden of disease, health care utilization rates, and cost of associated health care are useful criteria for prioritization of health areas to be included in a performance framework.

For the HCQI Project, this article has presented a conceptual framework that is based on well-known frameworks from the USA, Canada, and elsewhere, thus yielding an ambitious framework for performance evaluation. It is possible to demonstrate how the resulting framework could be used to integrate and interpret the HCQI within the broader OECD Health Data and related health projects. The Netherlands is already the first country to apply this OECD framework to its biennial health care system performance report [38]. The proposed OECD framework will serve to infuse coherence and balance into current and future work on performance measurement across OECD countries. A good conceptual framework is particularly essential when there are societal requirements for fairness, transparency, accountability, performance attribution, and rewarding of excellence.

Disclosure

The authors declare that they have no conflicts of interest.

Acknowledgements

We acknowledge the insightful comments of Dr Margarita Hurtado of the American Institutes of Research and Dr Edward Kelley, head of the OECD HCQI Project. We also thank representatives of OECD member countries for their helpful comments that helped improve the companion OECD working document based on this article. This paper reflects the opinion of the authors and not an official position of the OECD, its Member countries or institutions participating in the project.

References

Author notes

1Department of Social Medicine, University of Amsterdam, Amsterdam, 2National Institute of Public Health and the Environment, Center for Health Services Research and Prevention, Bilthoven, 3Faculty of Social and Behavioural Sciences, Tilburg University, Tranzo, Tilburg, The Netherlands, 4Organization for Economic Cooperation and Development, The Health Division, Directorate for Employment, Labour and Social Affairs, Paris, France