A Prospective Comparison of Four Study Designs Used in Assessing Safety and Effectiveness of Drug Therapy in Hypertension Management

Abstract

The objective of the study was to compare prospectively the impact of study design on drug therapy safety and effectiveness data obtained in hypertension management. The main study was a randomized controlled clinical trial of four different prospective study designs used in postmarketing assessment involving 1008 primary care practices in nine Canadian provinces. Two thousand nine hundred sixty-four patients with mild to moderate hypertension received an angiotensin converting enzyme (ACE) inhibitor daily for 14 weeks in one of four postmarketing studies—a randomized double-blind clinical trial (RCT) (10 to 40 mg fosinopril daily v 5 to 20 mg enalapril daily), two structured open label trials of 10 to 40 mg fosinopril daily (one with free drugs), or an unstructured open label trial of 10 to 40 mg fosinopril daily. Patient demographic and baseline characteristics, systolic and diastolic blood pressures, adverse events reported, and data quality were recorded as the outcome measures. The results showed that the RCT patients were titrated to higher doses of ACE inhibitor than patients in the open studies, P < .008; patients in the open studies were more likely to receive adjuvant diuretic therapy, P < .008. The decrease in blood pressure was similar for patients in all four studies, mean decrease in systolic BP was between 18 and 20 mm Hg, mean decrease in diastolic BP was between 11 and 13 mm Hg. Fewer patients in the unstructured open trial reported adverse events than patients in the RCT; a 55% relative reduction in reported adverse events (P < .008) was associated with the unstructured trial. There were also fewer drug-related adverse events per patient reported in the unstructured study (17 per 100 patients) than in the other studies (27 to 41 per 100 patients), P < .008. Physician preference for rounding off blood pressure measurements to 0 or 5 occurred most often in the unstructured open trial (P < .008). In conclusion, despite differences in dose titration and in the use of adjuvant therapy, antihypertensive drug therapy effectiveness observed in an RCT may be similar to uncontrolled postmarketing studies. Open trials with scheduled follow-up visits are as effective in detecting severe adverse events as RCT, but postmarketing studies with unstructured schedules of follow-up are insufficient in identifying drug-related adverse events and have poorer quality data. Am J Hypertens 1997;10:1191–1200 © 1997 American Journal of Hypertension, Ltd.

Postmarketing surveillance, randomized clinical trial, study design, angiotensin converting enzyme inhibitors, primary care

The majority of clinical studies of drug safety and efficacy before marketing (phase III) are randomized double-blind clinical trials (RCT). The RCT is the preferred design to maximize the internal validity of the study and allow regulatory agencies to draw inferences regarding a drug's safety and efficacy.¹ Due to concerns about the generalizability of phase III RCTs, after regulatory approval to market a new drug product additional open label studies of drug safety and effectiveness are conducted in the hope of expanding external validity.² Although the latter may provide results with greater generalizability, the open label design sacrifices confidence in the internal validity of the results and places some limits on the inferences that one can make from the results obtained. Consequently, there is some uncertainty regarding the optimal source of data to assess the actual health benefits and risks associated with drugs used in the postmarketing environment. This may be particularly relevant in the context of pharmacoeconomic analyses.³

The effects of study design on premarketing clinical trials have been described.^4–7 In some, but not all circumstances, active and new treatments appear to have performed better in uncontrolled and/or unblinded trials than in well-designed RCT. These observations arise by comparing data from different published studies with differing study designs, and may be the result of unknown confounders, as in the comparison of study design, “study design” had not been randomly allocated. Even less is known about the impact of study design in the postmarketing setting, particularly the effects on the type of patients selected and the overall study results of prospective (cohort) studies. Any extrapolation from what is known about differences in premarketing clinical trial studies to postmarketing studies would also be limited by the issue of generalizability.

The RCT is the most rigorous design for establishing efficacy and safety of comparative therapeutic options.⁸ Differences in outcomes between RCT and open studies are usually attributed to biases inherent in open study designs. There are, however, other factors that may be attributed to study design that can influence study outcomes. Patient characteristics may differ because of more restrictive inclusion/exclusion criteria in RCT, or patient compliance may be improved in RCT by frequent and controlled patient follow-up. Effectiveness may also be influenced by compliance related to medication access and affordability, or more directive dosing instructions, the latter being more common in RCT than in open studies.

The frequency of reported adverse events may be higher in RCT where adverse events are actively solicited through frequent follow-up visits. On the other hand, open studies may provide a greater opportunity to detect less frequent events if their patient numbers are larger than in the costlier RCT. Open trials may also be better able to detect drug–drug and drug–disease interactions because they are less restrictive on the use of concomitant therapy and correspond more closely to the real life use of drug therapies.

In this study four study designs were randomly allocated for use in a postmarketing environment where an angiotensin converting enzyme (ACE) inhibitor used in the treatment of mild to moderate hypertension was being monitored. The purpose of this study was to evaluate prospectively the differences in physician and patient characteristics, clinical efficacy and safety results, and data quality, among different designs available for use in postmarketing drug evaluation. A comparison between different prospective designs, using a random allocation of the different designs, has not been reported previously in the literature.

Methods

Study Design

The overall design of this prospective study was comprised of four different postmarketing clinical trials, each with a different study design, evaluating the effects of an ACE inhibitor in hypertensive patients during a 14-week period (Table 1). The four study protocols (A, B, C, D) used within the Monopril Unique Safety and Effectiveness Trial (MUST) were: Study A conformed to standard practices used in phase III randomized clinical trials—double-blind, randomized control arm, strict inclusion/exclusion criteria, washout period, scheduled visits, strict dosing instructions, specific restrictions on concomitant drugs, laboratory tests, on-site monitoring by a clinical trials monitor.⁹ Studies B and C conformed to standard practices used in phase IV postmarketing open clinical trials—large size, open label (drug samples provided in study B, prescription to be filled in study C), looser inclusion/exclusion criteria, no washout period, scheduled visits, looser dosing instructions, looser restrictions on concomitant drugs, no laboratory tests, phone monitoring.^10,11 Study D was conducted as a postmarketing research survey—very little direction given to the physician regarding how the drug of interest should be administered, prescription dispensing of the drug, customary follow-up and dosing, loose instructions on concomitant drugs, no laboratory tests, no monitoring.

Table 1

Variation Among Study Protocols

Study	A	B	C	D
Drug allocation	RCT	Cohort	Cohort	Cohort
Blinding	Double blind	Open	Open	Open
Inclusion and exclusion criteria	Strict	Loose	Loose	Loose
Washout period	2 weeks	None	None	None
Dose titration	Strict	Flexible	Flexible	None
Concomitant drugs	Limited	Flexible	Flexible	No restrictions
Study drug provided free to patient	Yes	Yes	No	No
Mandatory follow-up visits	3	3	3	1
Mandatory laboratory tests	yes	No	No	No
Clinical trials monitor	Site visit	Phone check	Phone check	None

Patients with mild to moderate hypertension (sitting diastolic blood pressure [DBP] > 95 mm Hg and < 115 mm Hg) were eligible for the study. Patients previously diagnosed as hypertensive had to have uncontrolled blood pressure at the time of entry (with or without antihypertensive therapy) to be eligible to participate. Patients had to be ≥ 18 years of age and if female with childbearing potential, they were required to use an effective method of contraception. Patients were required to provide written informed consent before being enrolled in the study. In general, the same inclusion/exclusion criteria applied to each study, except in study A, where criteria were explicitly stated in the protocol. For the three other studies physicians were referred to the product monograph. In addition, for study A, patients were excluded if their serum creatinine was >130 μmol/L or if they had proteinuria >150 mg/day, a customary exclusionary requirement for phase III trials involving ACE inhibitors.⁹

Patients in study A were randomly allocated (and blinded) in a 4:1 ratio to receive either fosinopril or enalapril, provided in identical-appearing tablets. The starting dose for fosinopril was 10 mg once daily and for enalapril 5 mg once daily. Dosage adjustment according to the protocol was to occur if DBP was unsatisfactory after a minimum of 2 weeks at the same dose. In study B, fosinopril was provided to physicians in 10-mg tablets in bottles of 30 tablets each. In studies C and D, the physician prescribed fosinopril according to the protocol (study C) or the product monograph (study D) and then the fosinopril was dispensed from a community pharmacy.

In study A, physicians had an option of adding 25 mg hydrochlorothiazide once daily if, by week 10, the maximum allowable dose of ACE inhibitor had been reached but the patient had not attained the target sitting blood pressure (ie, DBP < 90 mm Hg or decrease in DBP of > 10 mm Hg). Physicians in studies B and C could exercise this option at week 6. Physicians in study D could add concomitant antihypertensive medications at their discretion.

All four studies required a baseline assessment of blood pressure and a follow-up assessment at week 14 (Figure 1). Study A also had laboratory tests (potassium and creatinine levels) performed at baseline, and these patients were required to have a 2-week washout period if their previous antihypertensive therapy was discontinued. Patients in studies A, B, and C were reassessed after 2, 6, and 10 (optional) weeks of therapy. Data collected at each visit included blood pressure, heart rate, adverse events (any symptom or sign that appeared or worsened during the course of the study, regardless of severity and causal relation to the study drugs, although severity and causality were estimated by the physician), concomitant medication, and medication compliance (estimated from patient history). The adverse effects data were collected using standard forms used in typical postmarketing studies. Severity was left to the physician's judgment, by choosing from the classification scheme provided on the form (mild, moderate, severe, or laboratory test abnormality only). In addition, events were reported as serious if they were fatal, life-threatening, or permanently disabling requiring or prolonging in-patient hospitalization.

Patient follow-up schedule.

Figure 1.

Primary care physicians potentially interested in participating in a postmarketing study were identified in 49 geographic regions in Canada (all provinces except Saskatchewan) by pharmaceutical firm representatives. Each region was then randomly assigned to conduct studies B, C, or D. In addition, each region would also conduct study A. Then, in a 1:4 ratio, physicians within each region were randomly assigned to be recruited into either study A or one of study B, C, or D, depending on which of the latter three designs were allocated to the region (Figure 2). Only one physician per practice was enrolled, and during the conduct of the study the physicians were blinded to the presence of other parallel arms in the overall study design. They were provided with this information at the completion of the overall study.

Randomization of geographic regions, study physicians and patients to study designs and treatment groups. R1 = randomization of geographic regions to study pairs AB, AC, and AD. R2 = randomization of physicians in a given region to studies A, B, C, and D. R3 = randomization of patients in study A to fosinopril and enalapril treatment group.

Figure 2.

Physicians randomized to studies B, C, or D were formally recruited by pharmaceutical firm representatives. Physicians assigned to study A were recruited by pharmaceutical firm clinical trials monitors. The clinical monitors provided the follow-up and supervision for studies A, B, and C until completion of the study. All study A physicians had three site visits by a clinical monitor, whereas physicians in studies B and C received three monitoring telephone calls and 10% of these physicians also received a site visit. There was no clinical monitoring in study D.

Analysis

Data collected from patients receiving enalapril were not included in the analysis of the outcome variables, which could be affected by treatment (eg, blood pressure, adverse events), whereas outcome variables that would not be affected by treatment (eg, patient characteristics, data quality) included the enalapril patients. Only study A had some patients receiving enalapril, therefore exclusion of the enalapril data was necessary to make unbiased comparisons between the four studies pertaining to clinical events. Continuous variables were compared using analysis of variance techniques. Continuous demographic variables were compared across studies using multivariate analysis of variance. Analysis of covariance, adjusted for significantly different patient characteristics and baseline blood pressure, was used to compare the four study groups with respect to the mean decrease in blood pressure between baseline and final visit. For study A, the baseline blood pressure used for comparison with other studies was the blood pressure obtained at screening (ie, before washout). For the three pairwise comparisons identified a priori (study A v study B, C, or D), the significance level (α) was adjusted to 0.017 using a Bonferroni adjustment (ie, 0.05/3) to allow for the multiple comparisons. Repeated measures analysis of covariance was performed on sitting and standing blood pressure and heart rate measurements taken during the study, with the appropriate baseline value used as a covariate. Categorical variables were compared between studies using a Pearson χ² statistic.

Adverse events were compared between studies according to the number of patients experiencing adverse events and number of events experienced after week 0 (ie, after initiation of treatment). Data quality was assessed independently using predetermined objective criteria based on data completeness, data accuracy, and adherence to the protocol. These analyses were conducted using the appropriate χ² and analysis of variance techniques.

For post hoc χ² tests, the α level was adjusted to 0.008 to allow for six pairwise comparisons (A v B, A v C, A v D, B v C, B v D, C v D) using a Bonferroni adjustment. For post hoc tests of continuous data, the Student-Newman-Keul statistic was used.

Ethics approval was obtained from both the McMaster University Faculty of Health Sciences Subcommittee on the Ethics of Research in Human Experimentation and the Ethics Committee of Hôtel-Dieu de Montréal.

Results

Physician Recruitment

Thirty-four percent of physicians approached to participate in study A agreed to do so (ie, they were “recruited” into the study). In comparison, physician recruitment was higher in studies B, C, and D, being 61%, 62%, and 68%, respectively. There were no differences detected between studies with respect to the proportion of urban versus rural geographic areas, the type of physician practice, the practice size, the years in practice, or sex distribution. Nor were there any differences between recruited and nonrecruited physicians. Combining data from all physicians recruited into the study, the mean number of years in practice was 16, the mean number of patients per practice was 3050, of which approximately 15% of patients in the practice had hypertension. Ninety-seven percent of physicians were primary care physicians and 59% were in solo practices.

Patient Characteristics

There were no clinically important differences in age, sex, or body mass index of patients between studies A to D (Table 2). There was a greater percent of white patients in study A and a greater percent of Oriental patients in study D (P < .008). Studies C and D had a greater percent of patients covered by private drug benefit plans than studies A and B (P < .008). There were no differences detected between studies with respect to the proportion of patients having government drug benefit plans.

Table 2

Patient Demographic Characteristics

	Study A	Study B	Study C	Study D
Total (n)	322	1002	720	920
Male	153	470	329	426
Female	160	456	344	443
Not recorded	9	76	47	51
Age, years (SEM)*	54 (0.6)	56 (0.4)	55 (0.4)	55 (0.4)
BMI, kg/m2 (SEM)*	29 (0.3)	29 (0.2)	28 (0.2)	28 (0.2)
Race (%)†
White	93	90	90	85
Black	2	3	3	3
Oriental	2	5	5	10
Other	3	1	3	3
Drug benefit program (%)‡
Government plan	32	33	31	28
Private insurance	33	32	43	40
None	33	33	25	31
Unknown	2	1	1	1

*

P < .05 multiple comparison of A v B.

†

P < .008 pairwise comparison of D v A, B, or C.

‡

P < .008 pairwise comparison of B v C or D.

Data represent mean values and standard error of the mean (SEM) unless indicated otherwise.

The percent of newly diagnosed hypertensive patients was different between studies (Table 3). Study B had the lowest percentage of newly diagnosed patients, as compared to the other studies (P < .008). Among patients previously diagnosed as hypertensive, study A patients had a shorter duration of hypertension than study B (P < .05). Patients in study A had lower baseline systolic BP (SBP) than studies B and D (P < .05) and lower DBP than studies B, C, and D (P < .05). No differences were noted between study patients for the presence of cardiovascular, respiratory, gastrointestinal, genitourinary, musculoskeletal, or central nervous system diseases.

Table 3

Patient Baseline Characteristics

	Study A	Study B	Study C	Study D
Patients (n)	322	1002	720	920
Newly diagnosed hypertension (n, %)*	83 (26)	159 (16)	149 (21)	206 (23)
Duration of hypertension (years, SEM)†	6 (0.3)	7 (0.2)	6 (0.3)	6 (0.2)
Systolic BP (mm Hg, SEM)‡	159 (1.0)	164 (0.6)	162 (0.7)	163 (0.6)
Diastolic BP (mm Hg, SEM)‡	98 (0.4)	100 (0.3)	100 (0.3)	100 (0.2)

Data represent mean values and standard error of the mean (SEM) unless indicated otherwise.

*

P < .008 pairwise comparison of B v A, C, or D.

†

P < .05 multiple comparison of A v B.

‡

P < .05 multiple comparison of A v B, C, and D.

Prevalence of any medication use just before entering the study was 49% in study A, 56% in study B, 54% in study C, and 50% in study D (P < .008, B and C v A and D). Previous or current use of antihypertensive drugs was less in study A (61%) than study B (70%) (P < .008), and the use of antihypertensive drugs in studies C and D were 66% and 63%, respectively. Previous ACE inhibitor use was similar across the four studies (21% to 24%), but previous diuretic use was lower in study A than studies B, C, and D (24% v 28% to 33%). Patients in study A compared to studies B, C, and D used more bronchodilators (7% v 3% to 4%) and nonsteroidal antiinflammatory drugs (12% v 6%), (P < .008).

Medication Use and Effect on Blood Pressure

The decrease in SBP and DBP from baseline to final visit was similar across study groups (Figure 3a and b). The mean decrease in SBP was between 18 and 20 mm Hg and the mean decrease in DBP was between 11 and 13 mm Hg. The model used to identify any differences included baseline blood pressure and significantly different patient characteristics as covariates (ie, age, weight, duration of hypertension, newly diagnosed hypertension, race, type of drug plan). The study had greater than 99% power to detect an a priori clinically important 5 mm Hg difference in blood pressure reduction between studies A to D.

Effects of therapies on blood pressure.

Figure 3.

Use of monotherapy was more common in study A, 92%, as compared to the other studies, 67%, 73%, 73%, for studies B, C, D, respectively (P < .008) (Figure 3c). Across all studies, use of combination (diuretic) therapy was more frequent in patients who had not attained goal DBP reduction by final visit (31%) than those who had attained goal blood pressure (26%) (P < .01). The magnitude of the fosinopril dose administered by the end of the study was greatest in study A and least in study D (P < .008) (Figure 3d). Overall the percentage of patients using the three different doses varied significantly between studies A to D (P < .01). Pairwise comparisons resulted in significant differences (P < .008), except for study B versus study C (P = 0.20).

Adverse Event Reporting

The proportion of patients reporting one or more adverse event was 42% in study A, 37% in study B, 32% in study C, and 19% in study D. Reporting was lower in study D than in studies A, B, and C (P < .008). The relative reduction in patients reporting an adverse event in study D compared to study A was 55%. The mean number of adverse events per 100 exposed patients was 72 for study A, 60 for study B, 52 for study C, and 22 for study D; study D was lower than studies A, B, and C (P < .008) (Table 4).

Table 4

Reporting of Adverse Events (Symptoms)

	Study A*	Study B	Study C	Study D
Patients exposed (n)	253	1002	720	920
All adverse events, n (events per 100 patients)
Total†	181 (72)	602 (60)	375 (52)	198 (22)
Severe	18 (7)	71 (7)	56 (8)	24 (3)
Moderate	63 (25)	253 (25)	181 (25)	94 (10)
Mild	100 (40)	278 (28)	138 (19)	79 (9)
Related or possibly related adverse events, n (events per 100 patients)
Total‡	69 (27)	409 (41)	222 (31)	152 (17)
Severe§	11 (4)	57 (6)	38 (5)	19 (2)
Moderate∥	16 (6)	163 (16)	104 (14)	75 (8)
Mild∥	42 (17)	189 (19)	80 (11)	58 (6)

*

Includes only patients receiving fosinopril.

†

P < .008 for all pairwise comparison.

‡

P < .008 for all pairwise comparison except A v C (P > .25).

§

P < .008 pairwise comparison of D v B or C.

∥

P < .008 pairwise comparison of D v A, B, or C.

Combining the four studies, 54% of all adverse events reported were thought to be related or possibly related to the study drug. Of the severe events reported among the four studies, 61% to 80% were thought to be related to the study drug. When assessing only those adverse events thought to be related to the study drug, 83% to 87% of patients reported the events as mild to moderate, and 13% to 17% of patients reported the events as severe.

Study D reported fewer drug-related adverse events per 100 patients (17 patients) than studies A (27), B (41), or C (31 patients) (P < .008) (Table 4). The number of reported severe adverse events (thought to be related or possibly related to the study drug) per exposed patient was less in study D (2 per 100) than in studies B and C (6 and 5 per 100, respectively) (P < .008).

Data Quality

The proportion of patients that had all three required DBP readings recorded on their final visit was 90%, 83%, 84%, and 87% for studies A, B, C, and D, respectively (Table 5) (P < .008 for A versus B). The proportion of patients with the same three values for the blood pressure at the final visit was 42%, 31%, 42%, and 38%, for studies A, B, C, and D, respectively. The percent of DBP recordings where the last digit was 0 or 5 was 67%, 67%, 70%, and 75% for studies A, B, C, and D, respectively. A last digit of 0 or 5 was recorded more frequently for study D than for studies A, B, and C, and more frequently for study C than studies A and B (P < .008).

Table 5

Blood Pressure Recordings as a Reflection of Data Quality

	Study A*	Study B	Study C	Study D
Patients (n)	322	1002	720	920
Total DBP recordings (n)	5031	12,273	8920	6509
Last digit 0 or 5, n (%)†	3389 (67)	8238 (67)	6294 (70)	4861 (75)
Baseline blood pressure protocol violation n (%)‡	24 (7)	180 (18)	119 (17)	115 (12)
Patients with 3 DBP readings at final visit, n (%)§	290 (90)	834 (83)	608 (84)	802 (87)
Patients with the same DBP for all three readings at final visit, n (%)∥	123 (42)	254 (31)	257 (42)	305 (38)

*

Includes enalapril and fosinopril patients.

†

P < .008 pairwise comparison of C v A or B, and D v A, B, or C.

‡

P < .008 pairwise comparison of A v B or C, and B v D.

§

P < .008 pairwise comparison of A v B.

∥

P < .008 pairwise comparison of B v A, C, or D.

Protocol violations for entry criteria occurred in all four studies (Table 5). Patients with DBP < 90 mm Hg or DBP > 115 mm Hg were inadvertently included in study A (7%), study B (18%), study C (17%), and study D (12%). Failure to add concomitant antihypertensive therapy according to the protocol occurred more often in study B (13%) and study C (11%) than in study A (2%) (P < .008). Premature termination was 12% in study A, 19% in study B, 17% in study C, and 21% in study D, P < .008 for study A versus study D. The most common reasons for premature termination were the personal preference of the patient (6%) and an adverse event or intercurrent illness (5%). Patients simply lost to follow-up was minimal for all groups (0.9%).

Discussion

Our a priori hypothesis for this study was that differing methods of clinical trials will lead to different clinical results and conclusions based on differences in physician behavior, patient characteristics, and study protocol (dosing and follow-up requirements). Comparisons have been made between the results of RCT and open trials, but we are unaware of any prospective randomized design used to evaluate such differences.^4–7

In our comparison of the RCT with the open trial, we did not observe any important differences in physician characteristics or type of practice. The generalizability of our finding may be limited by the fact that the targeted physicians in our study were primary care (family) physicians, whereas many phase III RCT studies arise out of specialty practices. Nonetheless, within the stratum of physicians we recruited, we did not demonstrate a bias toward a particular type of physician enrolling in the study based on trial design, although recruitment rates were lower with the RCT.

We anticipated patient characteristics to be different in the RCT than the open trials due to physician biases when enrolling patients and differences in the application of the inclusion and exclusion criteria. Yet, no distinct pattern evolved between the RCT and open trial patients. Patient differences were noted between all study designs, but the differences involved a variety of characteristics and no study design could be considered distinctly different from the others in this regard.

Patients in the RCT arm of the study had slightly lower baseline blood pressure than the patients in the open trials (3 to 5 mm Hg SBP and 2 mm Hg DBP). The clinical importance of this small difference is not clear. This finding might be attributable to physician selection bias for less hypertensive patients in response to study A having a washout period or being a blinded design. The RCT also had fewer patients with DBP greater than 110 mm Hg. The latter, a violation of the inclusion criteria, occurred more frequently in the open arms and may have contributed to the difference in baseline blood pressure compared to the RCT study. The observation that there was greater concomitant nonsteroidal antiinflammatory drug use in study A patients than in the other arms of the study, a potential cause for increased blood pressure,¹² would have the opposite effect from the results observed.

It was also hypothesized that patients might differ between study A or B versus study C or D because free drugs were supplied to the former patients but not the latter. We did observe that there were more patients with privately insured drug benefit programs in studies C and D. This selection bias did not translate into observable differences in patient characteristics such as age, comorbidity, or cardiovascular risk factors.

Differences in blood pressure reduction between studies could be expected from differences in patient characteristics, protocol-driven dosing variations, and drug accessibility. Despite any of the differences in patient baseline characteristics, the decrease in SBP and DBP across study groups was similar. In addition, the differences in dosing strategies used by physicians to obtain a reduction in blood pressure did not result in any difference in blood pressure reduction. Using a higher dose of a single agent (ACE inhibitor) was as effective as a lower dose of ACE inhibitor used in conjunction with a diuretic. These differences in how the patients were treated pharmacologically were to some degree protocol driven, but they were dictated to reflect procedures adopted in previously conducted trials of different designs.^9–11

A conclusion that one might draw from the data obtained in this study is that hypertension management achieved by titration of monotherapy is as effective as the approach of adding a second agent to a low dose of an initial therapy. Whether this applies only to ACE inhibitors and diuretic therapy or also to other combinations of antihypertensive therapy remains to be determined.

Our data do not support the belief that uncontrolled trials are more likely to produce greater clinical benefits than RCT for a given therapy. It is unclear whether this is a peculiarity associated with the management of hypertension, a disorder that has a clearly defined therapeutic target that all physicians pursue with different means, or whether it is the consequence of comparing open trials with an RCT using primary care physicians in a postmarketing study.

Where observable benefit is the same among different dosing strategies, it is also important to assess whether the risks are similar. Phase IV studies are often undertaken to provide information regarding adverse consequences of drug use that might not be detected during premarketing studies. The legitimacy of some postmarketing studies has been called into question as excuses for mass marketing with little true interest in the evaluation of drug safety.¹³ The results of our study indicate that legitimate postmarketing studies must have regularly scheduled visits with a structured protocol if they are to be useful in detecting drug-related adverse events. Study D, which might loosely resemble a “seeding trial,” recorded the fewest overall and severe adverse events per patient. Justification for conducting loosely monitored “real life” studies is therefore lacking.

Comparing the adverse events data between the RCT and the structured open trial (with physician providing the drug), there was no difference in the proportion of patients reporting one or more adverse event. The RCT had more mild events per exposed patient but fewer events that were thought to be drug related. This may indicate that the RCT detects more noise without capturing more useful signals.

There was a greater proportion of patients reporting adverse events in study A than study C (drug not supplied by physician), but there were no differences in reporting of severe adverse events nor the number of adverse events per patients thought to be related to the study drug. The increase in unrelated adverse events in study A compared to studies B and C cannot be explained by a protocol-driven solicitation of events as data collection forms were similar in all studies. Although only speculative, it is possible that patients in studies B and C reported fewer mild adverse events because of a lower level of apprehension associated with being in a unblinded versus blinded study, than experienced by blinded patients in study A.

The quality of the data was inferior in study D compared to the other studies. This is illustrated by the finding that study D had the greatest excess of blood pressure readings with 0 or 5 as the last digit. As there is no 5 marking on a sphygmomanometer, and 0 is one of 5 markings that a physician can observe, on the basis of chance, 0 and 5 should occur as the last digit approximately 20% of the time. In study D, 0 and 5 occurred 2.75 times more often than predicted by chance. Although studies A and B displayed higher precision, they still had an excessive frequency of 0 and 5 as the last digit in the blood pressure readings (2.35 times more often than would be expected by chance). Random zero sphygmomanometers, which have been used in clinical trials to overcome measurement biases, can secondarily redistribute digit preference but they are not readily adopted for postmarketing studies.¹⁴

The fact that the blood pressure recordings in the RCT were not more precise than an open trial may be related to the conduct of the studies by primary care physicians who were not specifically trained to record blood pressure for the study, but who performed in the manner that they were accustomed to do in their practices. The accuracy of the mean blood pressure data obtained during the studies is not known.

As another measure of data quality, we assessed the frequency in which the three required blood pressure readings taken during the final visit were the same number. On the basis of chance, this would be unlikely given the natural variability of blood pressure to change by at least 2 mm Hg during three measurements taken 2 min apart. It is surprising to observe among all four studies that 31% to 42% of the three blood pressure readings for individual patients at a single visit were exactly the same. In this case study D did not perform worse than the others. This would support the opinion that data quality is more easily influenced by the training and motivation of the participating physicians than the design of the trial per se.

We have assessed the effect of study design on patient characteristics, clinical efficacy and safety results, and data quality in a postmarketing study. This is the first study to make this comparison using a prospective random allocation of study design, permitting greater inference about the internal validity of the results. The generalizability of the results would likely apply to other cardiovascular drugs used in the management of mild to moderate hypertension, although it may be limited to studies involving primary care physicians.

We observed that different designs lead to different dosing strategies to obtain a common therapeutic goal, and do so in an equally effective manner. On the other hand, postmarketing studies with unstructured visits are insufficient in identifying severe adverse events and have a trend toward poorer quality data. Open trials with scheduled follow-up visits are as effective in detecting severe adverse events as RCT. Because the phase III RCT will inevitably precede open-label postmarketing trials, the data from the RCT may be useful in predicting the effects that will be observed when the drug is marketed widely, and fears of limited external validity may be unjustified.

Perhaps the greatest potential for this observation of similar benefit and adverse effects among RCT and open trials will be in pharmacoeconomic analyses. Modeling techniques could use premarketing data and avoid protracted delays waiting for postmarketing clinical results, particularly where the data are a condition for market access.¹⁵

Finally, although we have concluded that there is a similarity in the effects of a single drug observed among RCT patients and open trial patients in the postmarketing environment, we do not dispute that the RCT still remains the essential study design when comparing two or more options, as open trials are unable to provide acceptable data to draw comparative inferences.

The authors thank the participating physicians for their cooperation in conducting this study.