- Research article
- Open Access
Quality indicators in headache care: an implementation study in six Italian specialist-care centres
The Journal of Headache and Pain volume 18, Article number: 55 (2017)
Headache disorders are highly prevalent, and have a substantial and negative impact on health worldwide. They are largely treatable, but differences in structure, objectives, organization and delivery affect the quality of headache care. In order to recognize and remedy deficiencies in care, the Global Campaign against Headache, in collaboration with the European Headache Federation, recently developed a set of quality indicators for headache services. These require further assessment to demonstrate fitness for purpose. This is their first implementation to evaluate quality in headache care as a multicentre national study.
Between September and December 2016, we applied the quality indicators in six Italian specialist headache centres (Bologna, Firenze, Modena, Padova, Roma Campus Bio-Medico and Roma Sapienza). We used five previously developed assessment instruments, translated into Italian according to Lifting The Burden’s translation protocol for hybrid documents. We took data from 360 consecutive patients (60 per centre) by questionnaire and from their medical records, and by different questionnaires from their health-care providers (HCPs), including physicians, nurses, psychologists and nursing assistants.
The findings, comparable between centres, confirmed the feasibility and practicability of using the quality indicators in Italian specialist headache centres. The questionnaires were easily understood by HCPs and patients, and were not unduly time-consuming. Diagnoses were almost all (> 97%) according to ICHD criteria, and routinely (100%) reviewed during follow-up. Diagnostic diaries were regularly used by 96% of physicians. Referral pathways from primary to specialist care existed in five of the six clinics, as did urgent referral pathways. Instruments to assess disability and quality of life were not used regularly, a deficiency that needs to be addressed.
This Italy-wide survey confirmed in six specialist centres that the headache service quality indicators are fit for purpose. By establishing majority practice, identifying commonalities and detecting deficits as a guide to quality improvement, the quality indicators may be used to set benchmarks for quality assessment. The next step is extend use and evaluation of the indicators into non-specialist care.
Headache disorders are highly prevalent, with major negative impacts on health, well-being and productivity worldwide. They are treatable, but differences in structure, objectives, organization and delivery affect the quality of medical care provided for them. While service quality is one of the most important factors influencing outcomes delivered by a health system, “quality” does not have clear meaning in this context, and its measurement is not straightforward. In the field of headache disorders, although systematic classification and diagnostic criteria have been in place since 1988 , no definition of service quality existed until recently . Lacking the grounding of any such definition, quality indicators proposed in the past have been partial and selective, and assessed for neither validity nor usability across headache-care settings .
The Global Campaign against Headache [3, 4], conducted by Lifting the Burden (LTB), a UK-registered non-governmental organization in official relations with the World Health Organization, aims inter alia to remedy deficiencies in headache-care worldwide. This requires a means of identifying such deficiencies. In a collaborative project with the European Headache Federation (EHF), LTB used various qualitative research methods to develop, through focus-group consultations and expert consensus, a definition of quality applicable to headache services.
“Good-quality headache care achieves accurate diagnosis and individualized management, has appropriate referral pathways, educates patients about their headaches and their management, is convenient and comfortable, satisfies patients, is efficient and equitable, assesses outcomes and is safe” .
Along with this definition, LTB proposed 30 indicators in nine quality domains that might be used to evaluate headache service quality, with the objectives not only identifying deficiencies but also of guiding their remedy. Each was associated with a specific assessment instrument.
The indicators were developed to be applicable across countries and cultures, and at all levels within health systems, in primary care and in specialist centres. No external standards were available to assess criterion-validity, which depended therefore on the methodological rigour employed in their development . In a pilot implementation study conducted successfully in two specialist headache centres in Portugal and Germany , the assessment questionnaires were found easy to apply by health-care practitioners (HCPs) and patients, and not unduly time consuming . Despite the differences in language and institutions (a university teaching hospital versus a private hospital), the findings were comparable in the two countries, suggesting operational validity in the evaluation of headache care. A subsequent pan-European study confirmed this in 14 specialist headache centres . Further evaluations are nonetheless needed, and here we conduct a multicentre national study, extending implementation to the quality assessment of six Italian specialist headache centres (level three according to the EHF/LTB standard ) and examining acceptability and ease of use of the assessment instrument in another language and country.
Of more than 80 headache specialist centres in Italy, ten participate in a network (the Young Italian Headache Network) of young Italian researchers on headache; seven of these expressed their willingness to participate in the study. However, one centre discontinued its participation for unknown reasons, without contributing any data. The study was completed over 3 months, between September and December 2016, in the remaining six centres (Table 1). The centre of Roma Sapienza, involved also in the earlier study of Schramm et al. , collected new data in accordance with our study protocol.
In each centre we collected data from service records (via the service manager), from the physicians and other HCPs (nurses, psychologists and/or nursing assistants) involved in outpatient consultations, and from 60 consecutive patients (30 enrolled when making a first-time visit and 30 others undergoing a follow-up visit).
Data collection, and instruments
We employed the five evaluation instruments used by Schramm et al. . Table 2, set out similarly as in the earlier studies [6, 7], provides an overview of these. They included three different questionnaires, one each for the physicians, other HCPs and patients, all completed anonymously. The questionnaire for patients was formulated as an exit questionnaire, which they received from their HCPs at the end of their consultations and completed and returned before leaving. All questionnaires were translated from their English originals into Italian by two authors (LP and SG) according to Lifting The Burden’s translation protocol for hybrid documents , with TJS and physicians and patients not involved in the study providing review and assistance.
In addition, data were extracted into two data-collection forms from the patients’ medical records and from service records. This was done by a local investigator at each centre who, to avoid any bias, did not perform any consultations with patients enrolled in the study.
Data management and analysis
Data were entered into spreadsheets at each centre, and then transferred to the data collection centre (Headache and Drug Abuse Centre, University of Modena and Reggio Emilia).
Descriptive analysis was performed by LP. Demographic and clinical data were provided as numerical values and are summarized as percentages or mean values. No hyphotheses were statistically tested.
The characteristics of the study centres and participants (physicians, other HCPs and patients) are summarized in Table 1. We took data from a total of 360 patients and from two to six physicians and up to six other HCPs per centre. In one centre (Modena), one physician refused to complete the questionnaire; in all other cases, both HCPs and patients reported that the questionnaires were easy to apply, readily understood and not unduly time consuming. None of the indicators caused or led to difficulties on a practical level; the percentage of missing answers was zero.
Evaluation of each service according to the quality indicators is shown in Table 3, with results summarised below by domain.
Domain A: accurate diagnosis
Diagnoses were recorded using ICHD terminology in ≥97% of cases. All diagnoses at every centre were documented after the first visit and reviewed during follow-up using diagnostic diaries. Temporal duration of the presenting headache disorder was reported in ≥88% of patients’ records (Table 3).
Domain B: individualized management
Waiting-list times for first appointment were not documented, but all clinics reported a formal triage system to expedite appointments in case of perceived urgency.
Mean time allocated to patients’ visits ranged, according to patients’ reports, between 26 (Padova) and 49 min (Bologna) for the first visit and between 17 (Firenze) and 29 min (Bologna and Roma Campus Bio-Medico) for follow-up visits. Patients’ and HCPs’ satisfaction with time per visit was ≥83% in all centres.
Frequency of symptoms and treatment plans were properly reported in most cases (respectively >82 and >87%). Psychological therapies were available in four of the six centres: in two of these (Roma Campus Bio-Medico and Roma Sapienza) their prescription was correctly reported in patients’ records, while in the other two (Padova and Firenze) these therapies were not prescribed to any patients enrolled in the study. Disability instruments were available in three centres (Padova, Modena and Firenze), but no HCP utilized them regularly. Follow-up calendars were available and recorded future appointment dates in all centres (Table 3).
Domain C: appropriate referral pathways
Referral pathways were well established in most centres: exceptions were Roma Campus Bio-Medico for primary care and Padova for urgent consultations (Table 3).
Domain D: education and reassurance of patients
Most patients (> 85%) expressed satisfaction with the information given by physicians, and >82% declared they had received appropriate reassurance (Table 3).
Domain E: convenience and comfort
Both patients and HCPs found the service environment clean and comfortable in four centres, with some dissatisfaction expressed in two (Bologna and Firenze). Patients’ satisfaction was generally high with the welcome they received from the services (67–98%) and with waiting times to be seen in the centres (67–82%). Average waiting times varied little, 8–15 min for first visits and 5–25 min for follow-up visits, and were unsatisfactory only for the HCPs in one centre (Bologna) (Table 3).
Domain F: patient satisfaction
Overall satisfaction with their management was expressed by 78–95% of patients. (Table 3).
Domain G: efficiency and equitability
All clinics had a protocol to avoid wastage of resources, and input costs were regularly recorded, but not all physicians were aware of these in one centre (Firenze) while the information was available to senior management only in another (Padova). All centres provided equal access for all who might need it.
Domain H: outcome assessment
Outcome assessments were made routinely only in Padova, and were based on symptom burden. Neither disability nor quality of life were assessed routinely at any centre.
Domain I: safety
All services appeared safe: prescribed drugs were well documented, protocols for reporting serious adverse events (SAEs) existed in each centre and there were no recorded SAEs.
The study was planned as a multicentre national service quality evaluation. It was conducted in six Italian specialist headache centres, and has shown common practices among them. The centres were all university teaching hospitals, with similar settings and levels of care, and operating within a common culture, with one language and in the same health service. The Italian National Health Service (NHS) has some differences between the administrative regions of Italy, but everywhere provides citizens with free access to a general practitioner, a general paediatrician and admission without charge to hospitals. Specialist consultations, as for headache, are provided with a small charge. Specifically in palliative care, Italian law establishes citizens’ right of access to pain care centres for both oncological and non-oncological pain.
However, our objective was not to compare the service quality of these centres but to establish, in Italian headache care, the usability, practicality and acceptability of the quality indicators, and their associated assessment instruments, developed by LTB. The six centres all collected data efficiently, found the questionnaires to be readily understood and easily applied, and, importantly, the whole process was not unduly time consuming. These findings accorded with those of the previous studies with similar purpose [6, 7]. Consequently, we believe the indicators can be used by the Italian headache scientific community to identify majority practices while simultaneously recognizing deficiencies, and therefore to set benchmarks against which quality may be judged in future in this country.
We should nonetheless comment on the specific findings. The results were mostly positive in all 30 quality indicators, but a few deficits were uncovered. In domain A, diagnoses utilized current ICHD criteria, and were routinely reviewed during follow-up; diagnostic diaries were regularly used in each centre. The influence of the continuous efforts of Italian scientific societies to facilitate application of the ICHD criteria is gratifyingly in evidence here. In domain B, triage systems were in place to adjust waiting times to urgency of need, an important consideration. Most patients and physicians were satisfied with the time given in consultations, and with waiting times. Follow-up calendars were available in each centre. On the other hand, no centre recorded in service records the average waiting times for a visit, while HCPs perceived that times were too long for high-quality care, a consequence of the high numbers of patients seeing the relatively few doctors working in each centre. Psychological therapies were provided by four centres, with different modalities, a disparity probably due to the difficulties in recruiting psychologists in the NHS (often they are private specialists). Surprisingly, despite that headache disorders are collectively the third highest cause of years lived with disability (YLDs) worldwide , no centre recorded patients’ disability scores. In domain C, referral pathways were in place in five of six centres. Importantly, most patients were generally satisfied and felt welcomed, informed and reassured (domains D, E and F). Furthermore, in domain G, every centre achieved well in the use of protocols to avoid wastage of resources and in promoting equitable access, while in domain I, every service appeared safe.
A critical issue emerged in the lack of formal outcome assessment (domain H), indicating the need for greater use of, above all, disability and quality of life instruments (HALT  and WHOQoL , or similar). In two services (Firenze and Modena), even though the physicians were well aware of the availability of validated and easy-to-apply instruments, and of their importance, nobody used them during follow-up because of time constraints. All centres complained that their use was not practicable because of lack of time for HCPs to administer them. Nonetheless, without formal and standardized outcomes assessment, it was unclear how the centres established the effectiveness of their interventions against headache. One centre (Padova) routinely used a symptom measure similar to HURT [13, 14], and all others employed some measure of the frequency and severity of headache, such as a headache index and/or numerical rating scales.
Potential for benchmarking
The consensus group developing these indicators foresaw their employment in benchmarking, but deferred exploration of this potential important application . Our findings in these six centres suggest that the indicators may indeed be deployed in setting benchmarks – at least in Italian specialist headache care. They clearly identified majority practice, with some aspects requiring improvement. Majority practice fails to make routine use of formal outcome measures, despite that this is an obvious part good record-keeping, and would appear to be necessary to judge whether best outcomes are achieved. The evident practical (lack-of-time) difficulty in doing this needs to be addressed. It should be recognized that HURT, for example, is intended for self-completion [13, 14], and can be brought by patients to follow-up visits already filled in – and not only used for its purpose of guiding treatment (less necessary in specialist care) but also placed in the patients’ records. The solution may lie in developing an electronic version, which has not so far been done. In the context of benchmarking, there are aspects of care not currently included in the quality indicators that many might think should be, and which at least deserve to be discussed for the future. One that is clearly relevant is whether diagnostic examinations and the prescriptions of treatments follow evidence-based guidelines [15, 16]. The consensus group responsible for developing the indicators acknowledged this, but also recognized that assessment of these was highly problematic . A second aspect, not included in the validated set of quality indicators but perhaps important nonetheless, is the non-medical barriers to care erected, before visits, by various impositions on patients (finding and paying for parking space, for example). We asked patients about their satisfaction with regard to this particular aspect, in a similar manner to the other quality indicators (“yes”/“not yes” responses), obtaining positive replies as follows: Firenze 45%, Roma Sapienza 53%, Modena 68%, Padova 80%, Bologna 82%, Roma Campus Bio-Medico 87%. Although not a validated quality indicator, this (or an adaption of it) might identify additional difficulties encountered by patients in receiving medical care from the specialist centres that are relevant to service quality.
This Italy-wide survey, the first such study to examine headache service quality indicators, showed in six specialist centres that the indicators are fit for purpose, identifying many commonalities and detecting deficits as a guide to quality improvement.
While it questioned more than 40 health-care workers and 360 patients, it was deliberately restricted to specialist services operating in similar settings. There was purpose in this, since it was the indicators themselves, and their application, that were being assessed. On the other hand, it was a limitation. Among the next steps is to expand their utilization into more varied primary- and secondary-care settings; most headache patients look for treatment in non-specialist care . Although quality may be an absolute attribute, it is not necessarily attainable to perfection, or there may be practical or cost constraints that plead for compromise. Accordingly, benchmarks may be different in primary care, and this is a pressing area of enquiry. We expect that the quality indicators can be applied in primary care, perhaps with some refinement, but this is yet to be demonstrated.
Headache Classification Committee of the International Headache Society (1988) Classification and diagnostic criteria for headache disorders, cranial neuralgias and facial pain. Cephalalgia 8(Suppl 7):1–96
Peters M, Perera S, Loder E, Jenkinson C, Gil-Gouveia R, Jensen R, Katsarava Z, Steiner TJ (2012) Quality in the provision of headache care. 1: systematic review of the literature and commentary. J Headache Pain 13:437–447
Steiner TJ (2004) Lifting the burden: the global campaign against headache. Lancet Neurol 3(4):204–205
Steiner TJ, Birbeck GL, Jensen R, Katsarava Z, Martelletti P, Stovner LJ (2010) Lifting The Burden: the first 7 years. J Headache Pain 11(6):451–455
Peters M, Jenkinson C, Perera S, Loder E, Jensen R, Katsarava Z, Gil-Gouveia R, Broner S, Steiner TJ (2012) Quality in the provision of headache care. 2: defining quality and its indicators. J Headache Pain 13:449–457
Katsarava Z, Gil-Gouveia R, Jensen R, Gaul C, Schramm S, Schoppe A, Steiner TJ (2015) Evaluation of headache service quality indicators: pilot implementation in two specialist-care centres. J Headache Pain 16:53
Schramm S, Uluduz D, Gouveia RG et al (2016) Headache service quality: evaluation of quality indicators in 14 specialist-care centers. J Headache Pain 17:111
Steiner TJ, Antonaci F, Jensen R, Lainez JMA, Lantèri-Minet M, Valade D (2011) Recomendations for headache service organisation and delivery in Europe. J Headache Pain 12:419–426
Peters M, Bertolote JM, Houchin C, Kandoura T, Steiner TJ (2007) Translation protocol for hybrid documents. J Headache Pain 8(suppl 1):S45–S47
Steiner TJ, Birbeck GL, Jensen RH, Katsarava Z, Stovner LJ, Martelletti P (2015) Headache disorders are third cause of disability worldwide. J Headache Pain 16:58
Steiner TJ (2007) The HALT and HART indices. J Headache Pain 8(suppl 1):S22–S25
Schmidt S, Mühlan H, Power M (2005) The EUROHIS-QOL 8-item index: psychometric results of a cross-cultural field study. Eur J Pub Health 16:420–428
Buse DC, Sollars CM, Steiner TJ, Jensen RH, Al Jumah MA, Lipton RB (2012) Why HURT? A review of clinical instruments for headache management. Curr Pain Headache Rep 16:237–254
Westergaard MLS, Steiner TJ, MacGregor EA, Antonaci F, Tassorelli C, Buse DC, Lipton RB, Jensen RH (2013) The Headache Under-Response to Treatment (HURT) questionnaire: assessment of utility in headache specialist care. Cephalalgia 33:245–255
Mitsikostas DD, Ashina M, Craven A, Diener HC, Goadsby PJ, Ferrari MD, Lampl C, Paemeleire K, Pascual J, Siva A, Olesen J, Osipova V, Martelletti P, EHF committee (2015) European headache Federation consensus on technical investigation for primary headache disorders. J Headache Pain 17:5
Steiner TJ, Paemeleire K, Jensen R, Valade D, Savi L, Lainez MJ, Diener HC, Martelletti P, Couturier EG (2007) European principles of management of common headache disorders in primary care. J Headache Pain 8(suppl 1):S3–S21
This study was conducted as a project of Young Italian Headache Network. The authors are thankful to Luigi Alberto Pini, for his assistance, support and encouragement.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
LP and SG conceived the study, collected data from their centre, performed the statistical analysis and drafting the manuscript. SB, VF, CL, EM, AN, MP, MU SC collected data from the remaining centres and assisted with drafting the manuscript. TJS participated in the study coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Each participating centre obtained approval from its local ethics review committee, before commencing the study. Informed consent was obtained from all participants. Data were collected anonymously, and held in accordance with European data-protection legislation.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Pellesi, L., Benemei, S., Favoni, V. et al. Quality indicators in headache care: an implementation study in six Italian specialist-care centres. J Headache Pain 18, 55 (2017). https://doi.org/10.1186/s10194-017-0762-x
- Headache disorders
- Headache care
- Service quality evaluation
- Global Campaign against Headache