Quality in the provision of headache care. 1: systematic review of the literature and commentary

Abstract

Widely accepted quality indicators for headache care would provide a basis not only for assessment of care but also, and more importantly, for its improvement. The objective of the study was to identify and summarize existing information on such indicators: specifically, did indicators exist, how had they been developed, what aspects of headache care did they relate to and how and with what utility were they being used? A systematic review of the medical literature was performed. A total of 32 articles met criteria for inclusion. We identified 55 existing headache quality indicators of which 37 evaluated processes of headache care. Most were relevant only to specific populations of patients and to care delivered in high-resource settings. Indicators had been used to describe overall quality of headache care at a national level, but not systematically applied to the evaluation and improvement of headache services in other settings. Some studies had evaluated the use of existing disability and quality of life instruments, but their findings had not been incorporated into quality indicators. Existing headache care quality indicators are incomplete and inadequate for purpose. They emphasize processes of care rather than structure or outcomes, and are not widely applicable to different levels and locations of headache care. Furthermore, they do not fully incorporate accepted evidence regarding optimal methods of care. There is a clear need for consensus-based indicators that fully reflect patients’ and public-health priorities. Ideally, these will be valid across cultures and health-care settings.

Introduction

Despite the high prevalence of recurrent headache disorders—principally migraine and tension-type headache—and the substantial burden of public ill-health they generate [1, 2], there are considerable variations worldwide in the nature, scope, organization, quantity and quality of medical care provided for these illnesses. At the same time, there is good evidence that optimal care is rarely achieved. There are multiple clinical, social and political barriers to both provision of and access to effective headache care, a recent publication by the World Health Organization stated: “The facts and figures presented … illuminate the worldwide neglect of a major cause of public ill-health and reveal the inadequacies of responses to it in countries throughout the world” [2]. In addition, and standing as a barrier to improvement, it is not certain that there is a universal view of what optimal headache services should look like, or, indeed, of the meaning in this context of “optimal”.

It is axiomatic that health-care systems should aspire to high quality of care, but progress toward this requires that quality can be assessed. Before this is possible, “quality” must first be defined. Donabedian [3], in a view now widely endorsed, suggested that quality of care should be considered in each of three domains: “structure” (the attributes of the settings in which care occurs); “process” (the giving and receiving of care); and “outcome” (the effects that care has on health status). Donabedian [4] also described seven attributes that in his view collectively defined health-care quality: efficacy, effectiveness, efficiency, optimality, acceptability, legitimacy and equity. A definition of quality of health care offered by the US Institute of Medicine (IOM) put the emphasis on outcomes: the degree to which health-care services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge [5]. The IOM specified six attributes of quality, differing somewhat from Donabedian’s seven: safety, timeliness, effectiveness, efficiency, equity and patient/family-centerdness.

What are needed for services delivering headache care are agreed and widely accepted quality measures. Their application to existing services might, by establishing standards of care, providing benchmarks and revealing deficiencies, motivate and guide improvements. Their employment will bring clarity of purpose to initiatives aimed at developing services where none exist. As a project within the Global Campaign against Headache [6, 7], which seeks both to develop and improve services, we performed a systematic literature review with the aim of identifying and summarizing existing information on quality indicators for headache care. Specifically, our questions were: did indicators exist, how had they been developed, what aspects of headache care did they relate to and how and with what utility were they being used?

Methods

Search strategy

During 2009 we searched Medline, EMBASE and CINAHL without language restrictions for relevant articles using the search terms “headache disorders” and “health care quality” for both title and text words. We combined the individual search results using the Boolean operator “AND”. We extended the search to articles containing text words “headache care*.tw” or “headache service*.tw” in order to ensure identification of all relevant articles. The thesaurus terms and text words used for these searches are shown in Table 1. The search was limited to articles from 1988 onwards, this being the year in which universally accepted definitions of headache disorders were first published [8]. We updated the search in February 2012 and found no new articles specifically related to the definition of quality or to quality indicators in the context of headache care. All identified articles were transferred to Reference Manager^®, a bibliographic software program.

Table 1 Search terms in the literature review

Inclusion and exclusion criteria

Two of the authors (SP and MP) screened titles and abstracts for relevance using predefined inclusion and exclusion criteria. Studies were included if they reported the development or application of quality indicators for headache care or evaluated other aspects of service provision for headache disorders. Articles unrelated to primary headache disorders, review articles, drug treatment studies, case reports, letters and comments, and papers that were not written in English were excluded. We obtained full-text versions of papers selected for further review, and again evaluated these against our inclusion and exclusion criteria. Any disagreements were resolved by consensus. Figure 1 shows the flow of studies through the review.

Fig. 1

Flow of studies through the review

Data extraction and analysis

A data extraction form was developed to summarize the relevant results of the studies selected. Formal statistical analysis was not possible because of the heterogeneity of evidence and measures in the selected articles, so a narrative synthesis of the findings was prepared.

Results

A total of 32 studies met criteria for inclusion in the review. Four [9–12] reported development of quality indicators for use in headache-service delivery; 28 [13–40] (listed in Table 2) evaluated aspects of headache care quality in a variety of settings, but did not employ formal quality measures to do so.

Table 2 Studies (n = 28) (identified by first author) evaluating headache care components associated with quality

Development of quality indicators

Tables 3 and 4 list the characteristics and results of the four studies that reported on the development of a total of 55 quality indicators. These indicators are categorized according to domain of application: structure, process or outcome of headache care. Within these domains are sub-domains into which they are further categorized. In fact, none of the studies developed indicators of quality of structural aspects of headache care. All, however, selected indicators of process, and 36 of the 37 process measures evaluated diagnostic or treatment procedures (for example, the use of specific diagnostic procedures when certain features were present in history or physical examination, or the prescription of specific medications for certain types of headache). Such process indicators commonly use “if–then” statements, and therefore do not apply to all patients evaluated for headache. Only two studies developed indicators of outcome of headache care: a total of 18 indicators, focusing on frequency and severity of headaches or on uptake of care. None of these outcome indicators evaluated disability, quality of life or patients’ satisfaction with care.

Table 3 Quality domains for which indicators were developed within the four studies

Table 4 Existing headache quality indicators (developed within the four studies)

All four studies were conducted in developed western countries, but they varied in their goals and in their methods of indicator development. Two [9, 10] aimed to evaluate overall quality of care provided at a national level for a number of common conditions, headache among them. They employed elaborate consensus methods for the development of indicators focused on process, since the scope of the projects made it unfeasible to collect outcome information. The other two studies [11, 12] had the more specific goal of evaluating migraine care, either within health plans or within a specialty clinic. They employed less extensive, more practical methods to develop quality indicators, but concentrated on treatment outcomes. These four studies are the key, and we describe them below in more detail.

McGlynn et al. [10] developed quality indicators for 30 acute and chronic conditions; the purpose was to assess overall quality of medical care provided to adults in the United States (US). Marshall et al. [9] developed quality indicators for 18 diseases in a project to assess quality of health care delivered to adults in primary care in the United Kingdom (UK). In both studies, the process of developing quality indicators began with a literature review. McGlynn et al. [10] then used a modified Delphi method to develop a final set of indicators. US experts in each disease area rated putative quality indicators on a 9-point scale from 1 (“not valid”) to 9 (“very valid”). Only those with a median composite score of ≥7 were selected. Similar methods were employed by Marshall et al. [9]. UK primary-care experts rated putative indicators on a 9-point validity scale, and also rated the necessity of including such information in patient records. Only those that achieved mean scores of ≥8 for validity and ≥6 for necessity scale were selected.

Leas et al. [12] developed a set of 20 quality of care measures specific to migraine—the only set of indicators to include both process and outcome measures. Existing measures were identified through a literature review and other candidate measures through telephone interviews with leaders in migraine care, health-care purchasing and managed care. An advisory board of experts then discussed all putative measures and, by consensus, selected a final set of indicators for testing. In contrast to those developed by McGlynn and by Marshall, this set of quality indicators included a number that evaluated the costs and outcomes of treatment.

Ferrari et al. [11] developed a set of quality indicators for headache care in conjunction with the staff of a specialist headache center and a quality assurance office. The goal of these indicators was to ensure the provision of consistent, high-quality care within a specialized headache treatment center. This set of indicators did not include any measures of headache care structure or process but instead focused exclusively on specific aspects of treatment outcome.

Evaluation of specific aspects of headache care

Table 2 lists the 28 studies [13–40] that assessed specific components of headache care associated with quality, without employing formal quality indicators. It also indicates the domains of quality (and sub-domains) addressed by each. All these studies were conducted in highly resourced locations, but evaluated strategies and aspects of care delivered in a range of settings by different practitioners. Four studies [18, 23, 28, 30] were conducted in primary care, three [14, 36, 37] in intermediate care, where treatment was provided by general practitioners with a special interest in headache, and one [20] in a setting where specialist nurses provided care. Six studies [21, 22, 31, 38–40] were conducted in specialty headache clinics, one [35] in an inpatient treatment setting, and five [13, 15, 17, 32, 33] evaluated headache care in emergency departments. Four studies [17, 19, 28, 39] assessed diagnostic accuracy, while eight [16, 18, 23, 26, 27, 31, 34, 35] evaluated the effects of specific strategies of headache management. Only two of the studies [31, 34] included a control group; one of these [34] was a randomized clinical trial. Some of the studies evaluated outcomes using existing general or headache-specific disability or quality of life instruments. Table 5 (available on-line) documents the study characteristics in detail.

We found no studies that reported on the reliability, validity, practicality or implementation of any of the 55 indicators.

Discussion

While many quality indicators have been developed to evaluate headache care, evidence regarding their reliability, validity and practicality is lacking. They emphasize processes of care rather than outcomes, and ignore structure. Most cover areas of routine assessment, but do not clearly specify the tool or process to be used in evaluation. Others describe desirable treatment in broad terms, including diagnosis, management or administration of particular tests or drugs. None of the identified measures report inter-rater reliability or other psychometric properties. They are not clearly applicable to different levels and locations of headache care. There is no evidence that any of them have been used for quality improvement, although this is presumably the purpose for which they were developed.

The process of developing quality indicators was not, in any of the studies, begun with, or therefore informed by, an agreed definition of “quality”. What is surprising is that neither did these studies attempt to construct a definition, in the specific context of headache care, as a prerequisite for developing indicators of it. While quality is important in health care for any condition, and may to that extent have a general definition, there are aspects of it that are specific to or of particular importance in headache care. Furthermore, it is not clear that a universally accepted general definition of quality of care does exist; even its attributes are not wholly agreed [4]. At issue here is whose perspectives matter in the meaning and assessment of health-care quality: patients’, health-care providers’ or payers’? Assuming they all do to an extent, and they are not perfectly aligned, which have priority? Quality is not necessarily coupled to financing: there is no direct relationship between better outcomes and the amount spent on health care [41]. Improving the quality of care for headache disorders goes beyond better diagnosis and good treatment, since large numbers of people with headache do not consult doctors and hence will not benefit from improvements in care processes. There is clear evidence of high barriers to care [2], and the need to dismantle them is high on the agenda for headache-service quality improvement. Sorting out these issues appears to be a prerequisite for developing quality indicators for headache services, but it has not been done.

Our study has strengths: the systematic nature of the literature search and review and the incorporation of information from studies that provided indirect evidence relevant to the development of quality indicators. Its main limitation was that we were able to search only for publicly available quality indicators and implementation studies: it is possible that insurance companies or other health management organizations have developed, validated and implemented proprietary quality indicators that have not been published. Of course, if such indicators exist, it might be asked whose perspective(s) they reflect.

In conclusion, we identified a number of studies providing evidence of the value of specific types, strategies and measures of headache treatment, but much further work is needed to incorporate these findings into the development of valid and practical quality indicators. There is no agreed definition of “quality” of headache care, and no considered view on how the non-aligned perspectives of different stakeholders in headache care should be placed in order of priority. Consensus on these issues is urgently required if health care for headache—clearly suboptimal throughout the world—is to be improved. This is a priority for patients and for public health.