Are the current IHS guidelines for migraine drug trials being followed?

Abstract

In 2000, the Clinical Trials Subcommittee of the International Headache Society (IHS) published the second edition of its guidelines for controlled trials of drugs in migraine. The purpose of this publication was to improve the quality of such trials by increasing the awareness amongst investigators of the methodological issues specific to this particular illness. Until now the adherence to these guidelines has not been systematically assessed. We reviewed all published controlled trials of drugs in migraine from 2002 to 2008. Eligible trials were scored for compliance with the IHS guidelines by using grading scales based on the most essential recommendations of the guidelines. The primary efficacy measure of each trial was also recorded. A total of 145 trials of acute treatment and 52 trials of prophylactic treatment were eligible for review. Of the randomized, double-blind trials, acute trials scored an average of 4.7 out of 7 while prophylactic trials scored an average of 5.6 out of 9 for compliance. Thirty-one percent of acute trials and 72% of prophylactic trials used the recommended primary efficacy measure. Fourteen percent of the reviewed trials were either not randomized or not double-blinded. Adherence to international guidelines like these of IHS is important to ensure that only high-quality trials are performed, and to provide the consensus that is required for meta analyses. The primary efficacy measure for trials of acute treatment should be “pain free” and not “headache relief”. Open-label or non-randomized trials generally have no place in the study of migraine drugs.

Introduction

The last major breakthrough in acute migraine treatment was the introduction of the triptans in the beginning of the 1990s. The triptans have now become the gold standard for acute migraine therapy [1] although their superiority over analgesics has been questioned [2]. However, not all patients respond to triptans [3] and triptans are used only by a minority (10–15%) of migraine patients [4, 5]. The triptans have, thus, probably not reduced migraine-related abeseteeism and socio-economic costs [6].

No currently marketed drug has been developed specifically for the prevention of migraine. Several drugs have, however, been shown to effectively reduce attack frequency in some patients, but these drugs are often associated with adverse effects that can be intolerable [7]. Thus, there is a large need for developing new therapies for the treatment of migraine.

In recent years, several candidates for pharmacological treatment of migraine has been discovered, such as CGRP-antagonists [8], NOS-inhibitors [9] and tonabersat [10, 11]. As a consequence, a large number of clinical trials in migraine are currently being carried out and many more will be carried out in the years to come. Possibly these studies will show new substances to be more effective and better tolerable in comparison with current treatments.

In order for the multi-million dollar investments going into these trials to truly benefit the migraineurs and in the end reduce the global burden of migraine, it must be ensured that the trials are internally and externally valid and that publication bias [12] does not take place. Also, to allow for comparison of trial outcomes and an international collaboration on drug development and therapy for migraine, there must, at least to some extent, be a global agreement on trial design and the parameters used for the assessment of efficacy, adverse events and safety.

In order to address these specific methodological problems and to generally improve the quality of migraine trials the Clinical Trials Subcommittee of the International Headache Society (IHS) published the first edition of its guidelines for controlled trials of drugs in migraine in 1991 [13].These guidelines consist of a series of recommendations, with comments, for the selection of patients, trial design, evaluation of results and the use of statistics. The second edition of these guidelines was published in 2000 [14], amongst other changes introducing additional recommendations for efficacy measures in acute trials.

In 2007, The European Medicines Agency published Guidelines on Clinical Investigation of Medicinal Products for the Treatment of migraine [15] and these guidelines are mainly based on the IHS guidelines with the same primary efficacy parameters.

Thus, these recommendations have been widely elaborated and disseminated but it has not been systematically assessed to what extent they are actually followed by investigators.

The objective of this review was to assess to what extent clinical drug trials in migraine carried out from 2002 until 2008 followed the 2000 IHS guidelines.

Methods

Criteria for considering studies for this review

Studies were required to be prospective controlled trials of pharmacological interventions for the treatment of migraine attacks, either acute treatment or preventive treatment. Migraine could be with and/or without aura, special types of migraine or unspecified.

Study participants were required to be adults (aged 18 years or older).

The publication dates for the studies were between 2002 and 2008, both years inclusive.

The following were excluded from the review:

• Studies of combined pharmacological and non-pharmacological treatment.

• Studies reported only in languages other than English, German, Swedish, Norwegian or Danish.

• Studies not concerning migraine exclusively (e.g. migraine and tension-type headache or migraine and depression).

• Studies of pharmacokinetics and/or pharmacodynamics exclusively.

• Studies of safety and tolerability exclusively.

• Studies with non-clinical outcome measures only (e.g. blood samples, MRI, EEG).

• Studies of induced migraine.

Search methods

PubMed was searched using the Cochrane Highly Sensitive Search Strategy (HSSS) for PubMed (as revised 2008) [16] and with publication date limits ranging from January 1, 2002 until December 31, 2008. The entire search string thus being: “‘2002/01/01’[Publication Date]: ‘2008/12/31’[Publication Date] AND (migraine AND (randomized controlled trial OR randomized OR placebo OR drug therapy OR randomly OR trial OR groups) NOT animals NOT (humans and animals))”.

The Cochrane Central Register of Controlled Trials (CENTRAL) was searched using the word “migraine”.

Clinicaltrials.gov was searched using the word “migraine” and with the following criteria: “Study type: Interventional studies”, “Age group: Adult (18–65) and Senior (66+)”.

Furthermore, trials were identified by searching reference lists of review articles.

Eligibility

Titles and abstracts of studies identified by the literature search were screened for eligibility. Papers that could not be excluded with certainty on the basis of information contained in the title and/or abstract were retrieved in full for screening.

Data extraction and analysis

Trials were divided into trials dealing with acute treatment and trials dealing with prophylactic treatment.

Of the various points of recommendations listed in the 2000 IHS guidelines some are optional and some only apply to special types of trials. Only a few can be regarded as generally essential.

For the evaluation of the trials in this review simplified version of the guidelines were developed containing only the recommendations which were deemed essential and presented in an unambiguous way which allows for clear judgment of whether the trial follows the guidelines on each point.

Based on this simplified version of the guidelines two schedules for the evaluation of drug trials dealing with acute and prophylactic treatment, respectively, were developed in which a trial is assigned one point for each guideline requirement met and 0 for each of those that are not. The evaluation schedules are shown in Table 1, 2.

Table 1 Schedules for evaluation of clinical trials in migraine

Table 2 Results

In addition, the following were registered for each trial: publication date, size (number of treated patients), “setting” (neurology, emergency medicine, internal medicine, general practice or other), location (US, Europe, other, multinational) and the diagnosis of patients treated in the trial.

For trials not using the recommended primary outcome measure the outcome measure actually used was registered.

For the prophylactic trials the following were also registered: whether the trial design was cross-over or parallel-group, the duration of the treatment period and the attack frequency required for inclusion.

Acute trials could score a maximum of seven points, prophylactic trials a maximum of nine points, items.

To evaluate compliance with the IHS guidelines the following measures were defined a priori: mean score of randomized, double-blind (RDB) trials, mean score of all reviewed trials, percentage of trials either not randomized or not double-blind (non-RDB), percentage of RDB trials using the recommended primary efficacy measure, percentage of RDB trials that are placebo-controlled.

Results

Data collection

The search was performed on August 12, 2009. Searching Pubmed using HSSS retrieved 3,296 items. Searching CENTRAL retrieved 2,169 items and searching clinicaltrials.gov retrieved 233 items.

Of the 5,698 titles and abstracts screened 255 papers were retrieved in full. 12 papers were unretrievable. A total of 184 retrievable papers, containing reports on 145 acute [17–148] and 52 prophylactic trials [149–198], were considered eligible for further review.

Evaluation of trials

The IHS diagnostic criteria were used in by far the most studies (92% of acute trials and 98% of prophylactic trials). The reports of studies that did not use IHS criteria did not mention the diagnostic criteria actually used, simply stated that a physician had diagnosed the patients or used modified IHS criteria (e.g. “at least three items from the list of criteria”[113]).

The median number of treated subjects in the acute trials was 328 (range 12–5,388) while the median number of treated subjects in the prophylactic trials was 88 (range 14–818). 47% of acute trials and 37% of prophylactic trials were carried out in the US. 89% of both acute trials and prophylactic trials were carried out within a neurological setting.

Thirty-one percent of acute RDB trials applied the recommended “pain free at 2 h” as the primary efficacy measure while 49% of acute RDB trials applied this measure as a secondary endpoint. “Headache relief” (a decrease from moderate or severe headache to none or mild) was the primary efficacy measure in 39% of the reviewed acute RDB trials. The “sustained pain free” endpoint was only used by 5 (3%) acute trials. However, 68 (47%) of acute trials used a modified and less rigorous version of this efficacy measure: the percentage of patients that sustained pain-free state at 24 h instead of the recommended 48 h.

More than 86% of prophylactic trials used the recommended treatment duration of at least 3 months. The average treatment duration was 16.0 weeks. Only 3 (6%) of the reviewed prophylactic trials applied stratification for migraine attack frequency. Seven of the prophylactic studies (14%) used the recommended migraine attack frequency of 2–6 as an eligibility criterion. Most studies required an average attack frequency of different intervals between 3 and 12 attacks per month. Eight study reports did not mention an attack frequency requirement. 72% of the prophylactic studies used the recommended “migraine attacks per 4 weeks” as the primary efficacy measure.

Discussion

The IHS guidelines for drug trials in migraine consist of a series of recommendations with comments. Some recommendations in the guidelines are general for randomised clinical trials (RCTs), such as randomisation, double-blinding and placebo-control. These recommendations are followed in 67% of the clinical studies. Other recommendations are migraine-specific, such as operational diagnostic criteria and primary efficacy measure. Amongst these recommendations the use of operational diagnostic criteria is a major contributor to the external validity of the results of the RCT. The IHS diagnostic criteria were used in 94% of the RCTs Fig. 1, 2.

Fig. 1

Average scores for acute trials

Fig. 2

Average scores for prophylactic trials

The choice of a primary efficacy measure is crucial when designing a drug trial. It is also important that the same primary efficacy is used in similar RCTs when meta-analyses are performed. Some consensus internationally is therefore needed and international guidelines like these of IHS are suggested in order to provide this consensus. In the first IHS guidelines ‘complete response’ [13], which was very similar to current “sustained pain-free” [14] was suggested as the primary efficacy measure. At the same time Glaxo used, in the extensive trial programme of sumatriptan, its so-called “Glaxo criterion”: a decrease in headache from severe or moderate to none or mild [199]. This ‘headache relief’ was subsequently used in the extensive trial programs of the triptans, naratriptan, zolmitriptan, rizatriptan, almotriptan, eletriptan and frovatriptan in the 1990s [1] and its use persisted into the 2010s [200]. In 2000, pain-free after 2 h was recommended as the primary efficacy parameter by IHS. Only 31% of the RCTs of acute treatment reviewed here comply with this recommendation. However, this endpoint has been used in an increasing number of RCTs (Fig. 3).

Fig. 3

Number of trials using “2 h pain free” and “2 h pain relief”, respectively, as a primary efficacy endpoint

Studies of patient satisfaction have consistently shown that patients consider complete relief of pain, a fast onset of action and no pain recurrence the most desirable effects of a migraine drug for acute treatment [201, 202]. While the “headache relief” endpoint is an effective measurement of patients’ response to treatment it, thus, does not reflect what patients actually consider important. Furthermore, “headache relief” has a much higher placebo rate than “pain free” [203].

Some of the acute trials apply even more rigorous primary efficacy endpoints, such as “pain free at 30 min” [82] or “total symptom relief at 1 h” [59]. Nearly 50% of the acute trials use the composite endpoint of “sustained pain free” either for 24 or 48 h. “Sustained pain free” is probably the ideal drug response in regard to patients’ requests. It is obtained, however, in only 20–30% of triptan trials [3].

A few of the acute trials do not have the treatment of headache as their primary objective. These trials focus on the accompanying symptoms of nausea, photophobia and phonophobia. This is acceptable and clinically relevant since some migraineurs consider these symptoms to be the main problem.

Several of the acute trials investigate the relationship between the timing of drug intake (in relation to the onset of migraine pain or cutaneous allodynia) and drug efficacy [23, 62, 73, 86, 102, 103, 122, 130]. The results of these trials suggest that “early” triptan administration, while the headache is mild, is more efficient in terms of pain-free outcomes and reduced risk of recurrence when compared to “late” administration, when the headache is moderate to severe.

In acute trials of migraine with aura exclusively, drugs could be taken either during the aura phase or during the headache phase [11]. An efficacy endpoint in such a trial could very well be the duration of the aura, since many patients suffering from migraine with aura are severely disabled by the aura itself. Alternatively, the efficacy measure could be whether the migraine headache occurs. Some of these patients often or even exclusively have a mild tension-type like headache following their aura [7]. These patients obviously regard their aura symptoms and not their headache as the main problem.

The prophylactic trials comply well with using the recommended primary efficacy measure. There are currently no studies of what effects patients specifically request from preventive treatment. Possibilities could be lowering of attack frequency, shortening of attack duration, amelioration of migraine intensity or less days of absence from work. A recent study showed that patients generally prefer treatments highly efficient in reducing attack frequency over treatments causing few side-effects and requiring infrequent dosing [204].

Stratification for frequency of migraine attacks is only used by three of the reviewed prophylactic studies and none of these make use of the baseline stratification in the following statistical analyses. This is probably because attack frequency per se is not thought of as an important prognostic factor for treatment response. This is unfortunate as this has previously been shown to be the case [205] and the knowledge that could be obtained by stratification could be clinically useful and serve as a basis for further studies.

The recommended attack frequency requirement of 2–6 attacks per month for preventive trials has not gained popularity. It should be noted that this requirement is not feasible for some studies of the migraine subtypes which is obvious for chronic migraine and for menstrually related migraine (MRM).

Specific guidelines for trials in chronic migraine have recently been published in 2009 by the IHS [206] in which the recommended primary endpoint depends on the study objective but includes the number of headache days, the number of migraine days or the number of migraine episodes. The four trials reviewed here adhere to this recommendation. Recommendations for trials in MRM specifically are contained in the second edition of the IHS guidelines.

Nearly 14% of the reviewed trials are either not randomized or not double-blind. These studies are especially prevalent amongst the acute trials. Open-label studies in general are often smaller, explorative early phase studies. For the non-RDB acute trials reviewed here, however, the average number of treated patients is 556, i.e. larger than the overall average of the acute trials. Three huge studies of more than 2,300 subjects each contribute to this average [18, 53, 72]. One of these trials scored a total of 0 on the rating scale used in this review. Randomization and the double-blind technique are generally considered to be the cornerstones of quality trials. Because of the above-mentioned subjective nature and large placebo effect, this especially holds true for migraine studies. In addition these features are fairly cheap and easy to apply. In some special cases it is not possible to effectively blind the investigator (e.g. surgical procedures) or the patients (e.g. trials of drugs with characteristic and commonly known side-effects). If this is not the case, performing open-label or non-randomized trials in migraine can hardly be considered anything but unethical to both the patients treated and to the rest of the scientific society.

Investigators should be careful when reporting the details of their studies and avoid using ambiguous or implicit terms or methods. For example, in this study it was found that several studies did not specify that the diagnosis of migraine was made based on the IHS criteria even though this was highly possible as the investigators had used these criteria in all of their previous studies. In some trials the word “headache” is used synonymously with migraine headache even though “headache” of course could refer to all other kinds of head pain. Thus, there is an important difference between having “reduction of migraine days” or “reduction of headache days” as an efficacy endpoint, especially since migraineurs often experience tension-type headache between migraine attacks (“interval headache”) [207]. Many prophylactic trials do not specify the attack frequency required for eligibility even though this is highly relevant. Most prophylactic trials also neglect the important aspect of defining how long a time span between attacks is required for the attacks to be regarded as separate and not a case of recurrence.

This study is limited by its use of a very simplified version of the IHS guidelines for assessing adherence. Other elements of the guidelines could have been assessed as well. Result accuracy could have been increased by employing multiple reviewers. The reports reviewed were published between 2002 and 2008. Some study protocols have necessarily been written before the publication of the 2000 IHS guidelines. One study published in 2006 [145] was actually done in 1988, i.e. before the publication of the first edition of the IHS guidelines.

In conclusion, even though the quality of clinical trials in migraine is generally high and the IHS recommendations are well adhered to, there is still room for improvement. Investigators should be encouraged to report meticulously and to use clinically relevant primary efficacy measures. The once popular “headache response” endpoint for acute trials should now be considered obsolete. Stratification should be applied in future parallel-group prophylactic trials.

Open-label or non-randomized trials generally have no place in the study of migraine drugs.

The IHS should develop specific guidelines for assessment of effects of the timing of drug administration in acute trials, e.g. by providing definitions of the terms “early migraine” and “mild migraine” as well as other relevant phenomena such as cutaneous allodynia.

Furthermore, the IHS should offer specific guidelines for migraine subtypes such as migraine with aura [208] and for special treatments such as botulinum toxin injections and similar complex regimens that could be seen in future migraine trials.