Abstract

Background. Mixed method research, the combined use of qualitative and quantitative methods, is increasingly popular in health services research, especially as a way to examine ‘complex interventions’. This paper seeks to provide a case example of the use of mixed methods in the analysis of a complex intervention (a computerized interactive decision aid) to test whether their use affords insights into potential moderators of the intervention (i.e. patient factors that were associated with the impact of the intervention).

Methods. We conducted a pragmatic randomized controlled trial (n = 149) of a decision aid in women attending their primary care practitioners with menorrhagia. Alongside the trial, a qualitative study was conducted with interviews with a sample of women who had received the intervention (n = 18). Hypotheses generated by the qualitative study were used to inform subsequent quantitative subgroup analyses.

Results. The results from both studies showed that the decision aid was broadly beneficial. The qualitative study found that women with less formal education reported greater levels of benefit from the intervention. However, quantitative analyses of formal education as a moderator of treatment effect found that the intervention provided the greatest benefit in women with greater formal education. The findings of the qualitative and quantitative studies thus demonstrated a significant inter-method discrepancy. Possible reasons for this discrepancy are discussed.

Conclusions. Mixed methods can illuminate different aspects of an intervention or provide greater insight into particular issues. Health service researchers need to be aware that the qualitative and quantitative results may be discrepant, and methods need to be developed to deal with such discrepancies. However, exploring seemingly discrepant results can lead to greater insight and the development of new hypotheses and avenues of research.

Background

Mixed method research, the combined use of qualitative and quantitative methods, is increasingly popular in health services research,1–4 especially as a way to examine ‘complex interventions’. Complex interventions are defined as those ‘built up from a number of components, which may act both independently and interdependently’.5,6

Analysis of a complex intervention may involve an analysis of moderators and mediators. A ‘moderator’ specifies for whom or under what conditions the treatment or intervention works and a ‘mediator’ identifies possible mechanisms through which a treatment or intervention might achieve its effects.7,8

Moderators, in specifying for whom or under what conditions the intervention is most successful, help decision making about subsequent trials and routine service delivery, as they suggest to health professionals which patients may benefit most from the intervention. Quantitatively, these are often explored through so called subgroup analyses.

The Medical Research Council has developed a framework for the evaluation of complex interventions, which they suggest requires the use of both qualitative and quantitative methods.5 They describe a sequential, phased approach to evaluation, while others have considered that the evaluation may be much more flexible or iterative in nature.6,9 Regardless of the approach used, it is common to find that the results of mixed method evaluations are published separately.1,10,11

Combining quantitative and qualitative data in the evaluation of a complex intervention has two potential benefits. First, the two methods may complement one another, such that qualitative methods illuminate some aspects of the intervention, while quantitative methods provide insight into others. Second, the use of mixed methods may be multiplicative, in that the combination affords greater insight than either method alone.12 It is the integration of data or findings from the different components of the study that is the key characteristic and unique contribution of mixed method research.12

Decision aids as an exemplar of complex interventions

Decision aids are technologies designed to support the process of decision making, and can range from simple audiotapes and booklets, to complex interactive computer programmes. In the last decade, there has been an increasing number of randomized controlled trials (RCTs), evaluating the use of patient decision aids in enhancing decision making in health care, which show that decision aids consistently improve knowledge, enhance realistic expectations of benefits and harms, reduce decisional conflict and stimulate active participation in decision making.13 However, little is known about how decision aids achieve their effects and what ‘ingredients’ are critical to an intervention.

A patient decision aid is a good exemplar of a complex intervention. In addition to providing information about a condition or treatment, they focus on the benefits and risks of the alternatives, specifically tailored to the individual patient. They provide detailed descriptions of benefits and risks in functional terms including explicit probabilities. Decision aids should then include a values clarifying exercise, such as ranking all the different treatments in order of personal preference.14 Because of this range of potential ‘active ingredients’, the effect of decision aids may be moderated by patient characteristics. For example, certain types of patients may prioritize choice and shared decision making, and thus may benefit more from such interventions than patients who prefer professional-led decision making.

This paper reports on a study to examine the impact of a computerized decision aid on treatment decision making in women with menorrhagia (heavy menstrual bleeding). Menorrhagia is a condition for which there are many equally effective treatments, with different side effect profiles. Patient involvement in choosing between these different treatments is advocated in guidelines from the Royal College of Obstetricians and Gynaecologists,15 making this an ideal condition in which to study the effect of a decision aid.

This paper seeks to provide a case example of the use of quantitative and qualitative methods in the analysis of a complex intervention to test whether the use of both methods achieves the two potential advantages outlined above: the two methods may complement one another, and second, the use of mixed methods may be multiplicative, in that the combination affords insights which the application of either method alone would fail to provide.

Methods

We conducted a pragmatic RCT of a decision aid in women attending their primary care practitioners with menorrhagia. The aim of the trial was to evaluate whether the addition of a computerized decision aid improved decision making in these women compared to written information alone. Alongside the trial, a qualitative study was conducted with interviews with a sample of women who had received the intervention. The aim of this qualitative study was to examine possible moderators of the decision aid with a view to informing subsequent quantitative subgroup analyses.

Intervention

The computerized decision aid was the Clinical Guidance Tree.16 This is an interactive computer programme designed to provide individualized treatment information, including specific risk information, elicit values and preferences and perform a decision analysis which suggests a preferred treatment option, based on individual preferences (see Box 1). In addition, all women received a commercially available printed information leaflet.

BOX 1 The decision aid intervention

The Clinical Guidance Tree takes the patient through three distinct stages:

  • Stage 1: the programme elicits information from the patient allowing the tree to be tailored to her as an individual, ensuring that the treatment information received is appropriate. For example, has she completed her family? Next, information is provided about menorrhagia and all the different treatment options a patient might choose. It gives details about the different options and their possible outcomes, including side effects. It details how likely different outcomes are to happen.

  • Stage 2: the patient is asked to rate different outcomes in order of preference, by ‘dragging and dropping’ them onto a scale from 0 to 100. This is so that her particular values and beliefs can be taken into account, and her utilities can be calculated in order to perform decision analysis.

  • Stage 3: the programme performs a decision analysis and suggests a preferred treatment to the patient based on her individual preferences. The programme also produces a printout of the session, which includes information on treatments, answers she gave to questions, definitions of any of the terms that she looked up in the programme dictionary and the result of her decision analysis.

Quantitative study

In the RCT, individuals were randomly allocated to either the intervention group (information leaflet and an appointment to use the Clinical Guidance Tree) or the control group (information leaflet only). This trial is reported in detail and published separately.17,18

The primary outcome was the decisional conflict score 2 weeks after the intervention.19 This scale measures the degree of uncertainty about which course of action to take and has been used as an outcome measure in many trials of decision aids.20–24 Secondary outcomes included anxiety, menorrhagia quality of life and knowledge (2 weeks and 6 months after the intervention). Process measures, collected at 6 months, included treatment preferences, hospital outpatient appointments and treatments actually received.

Qualitative study

Women who had received the Clinical Guidance Tree intervention were purposively sampled and invited to take part in an interview. The sampling was iterative: initially women were invited to participate in the interviews 4–6 weeks after they had received the intervention. The baseline demographic data from the quantitative study seemed to suggest the recruitment to the trial of a higher proportion of women from lower socio-economic groups (as defined by employment status and home ownership) and with lower levels of formal education than usually seen in trials of this nature.25–27 Further, the theoretical underpinnings of the intervention, the Decision Support Framework,14 included client characteristics such as socio-economic status and education as two of many suggested factors that may be important in the usefulness of the decision aid. Women were therefore sampled purposively to include both higher and lower levels of formal education and women from higher and lower socio-economic groups.

In this way, the sampling was linked to the theoretical and conceptual framework underlying the intervention, the preliminary quantitative data and the developing qualitative themes. Sampling was continued until no new themes were emerging from the data. JP or a research assistant specifically recruited to the study carried out interviews. Analysis was carried out by all three authors, with agreement of themes and actively seeking out disconfirmatory evidence. The interview schedule was modified iteratively in the light of emerging themes. The schedule covered topics such as symptoms, knowledge about treatment options before the intervention, involvement in decision making, the use of the intervention, the effect of this on consultations since the intervention and the effect on decision making.

Results

The main results of the RCT are reported in a separate publication,17 for the purposes of this paper a brief overview of the results is described.

Quantitative study

Fifteen of the 19 general practices taking part in the study had positive Jarman scores denoting low socio-economic status. One hundred and forty-nine women consented to participate in the study. There was no difference between consenting and non-consenting women in terms of age.

The Clinical Guidance Tree significantly reduced decisional conflict over and above the effect of the information leaflet [difference in means −16.6, 95% confidence interval (CI) −21.5 to −11.7; P < 0.001]. Neither the computerized decision aid nor the information leaflet had any effect on patient anxiety scores at the 2-week or 6-month follow-up.

At the 6-month follow-up, the use of the decision aid led to a significant increase in the women's knowledge of menorrhagia and its treatment options (difference in means 9.3, 95% CI 1.9–16.6; P < 0.01). More interestingly, without apparently affecting either anxiety or treatments undertaken, the use of the decision aid had a beneficial effect on Menorrhagia Specific Utility Scale scores28 (difference in means 10.9, 95% CI 0.9–21.0; P < 0.05). Process measures demonstrated that women in the intervention group felt more able to express a treatment preference, but this did not seem to have been carried through to being more likely to have received their preferred treatment, or indeed any treatment.

Qualitative study

Eighteen women who received the Clinical Guidance Tree computerized decision aid were interviewed. These women were all white and ranged in age from 31 to 47 years. This range of attributes was similar to the main sample (age range 30–52, 98% white).

Eleven women owned their own homes and were in employment. Five had achieved educational qualifications of A-Levels or above. This range of attributes was again similar to those in the RCT.

The broad themes to emerge from the data were summarized into six main areas: information seeking, legitimacy of condition, effect on decision making, practicalities of intervention use, information from decision aid and empowerment. It is the final theme which is the focus of the present paper.

Overall, the decision aid had a greater effect on decision making than the information leaflet as it was seen as interactive, personalized and containing specific risk information. By presenting women with detailed risk information, eliciting their treatment preferences and giving them a printout of the result of the decision analysis, they reported greater confidence to participate in decisions about future management. This was labelled empowerment, defined as the feeling that an individual has the power or authority to do something.29

However, this reported empowerment was most noticeable in those women with lower formal education (Table 1). Women with less formal education reported feeling that their future consulting behaviour had been, or will be, directly influenced by having used the decision aid. They reported increased feelings of confidence to participate actively in discussions with health professionals about their future management. Data from interviews with women with greater levels of formal education demonstrate that the intervention was reported as useful in terms of a source of information, but not in terms of how future decision-making behaviour will change as a result of it (see Table 1). This latter group of women had often already expressed a feeling of ownership of their treatment decisions ‘I decided what I decided’ (C35016) and ‘I can make choices’ (118030) or ‘we are trying this’ (B87012) etc. They saw the intervention as beneficial in terms of another supplementary resource for information and possibly as legitimizing their treatment decision.

Table 1

Qualitative quotes

Women—less formal education Women—more formal education 
‘If I can get to sit down with the nurse or GP, I feel that I can talk it over a bit more now, now that I am informed a little bit more.’ Customer advisor in B&Q, O-Levels only; ID 774032 ‘I don't think it changed my view but it did help to enforce why I actually decided what I decided.’ Vicar, first degree, higher degree; ID C35 016 
‘He won't be able to say “Oh no you don't want that. You don't know anything about it.” I'll say “well yeah, I do because I've been through this” … because you can get pushed into something that you don't really want …. “No I don't want to. I've made my mind up. This is what I want”’
Cook in day nursery, O-Levels only; ID 774026 
‘I can make choices, and I feel that to a certain extent, I have already made choices, I have made a choice to go ahead with it, armed with some information …’
Unemployed, disabled, A-Levels; ID 118030 
‘Oh its gonna be a big difference yeah, like I say, I would have walked in, and I wouldn't have known no difference … I would have just gone “yeah, yep” and I wouldn't have probably understood half of it … but now I've got like homework if you see … and I can go in with my eyes open and say, “well, no I'm sorry, I want to try something else”’ Waitress, no qualifications; ID C13012 ‘… because that came up … as my third choice, or second choice … so that is what we are trying now, so yes I have used the information and gone along and we are trying this …’
Sales co-ordinator, A-Levels; ID B87012 
‘I think it made me decide I should go to the doctors and ask for treatment. Cos before I didn't know what treatments was open to me, so I was just letting them suggest things and waiting and everything. But now I know and I've been and asked them about the coil business …. It made me feel more in control of my treatment.’
Unemployed single parent, GCSE only; ID 842003 
‘… now I can go to the doctor with the information that I'd never heard of previously, you know and put forward my case for wanting to go ahead with it.’ Librarian, A-Levels; ID C35010 
Women—less formal education Women—more formal education 
‘If I can get to sit down with the nurse or GP, I feel that I can talk it over a bit more now, now that I am informed a little bit more.’ Customer advisor in B&Q, O-Levels only; ID 774032 ‘I don't think it changed my view but it did help to enforce why I actually decided what I decided.’ Vicar, first degree, higher degree; ID C35 016 
‘He won't be able to say “Oh no you don't want that. You don't know anything about it.” I'll say “well yeah, I do because I've been through this” … because you can get pushed into something that you don't really want …. “No I don't want to. I've made my mind up. This is what I want”’
Cook in day nursery, O-Levels only; ID 774026 
‘I can make choices, and I feel that to a certain extent, I have already made choices, I have made a choice to go ahead with it, armed with some information …’
Unemployed, disabled, A-Levels; ID 118030 
‘Oh its gonna be a big difference yeah, like I say, I would have walked in, and I wouldn't have known no difference … I would have just gone “yeah, yep” and I wouldn't have probably understood half of it … but now I've got like homework if you see … and I can go in with my eyes open and say, “well, no I'm sorry, I want to try something else”’ Waitress, no qualifications; ID C13012 ‘… because that came up … as my third choice, or second choice … so that is what we are trying now, so yes I have used the information and gone along and we are trying this …’
Sales co-ordinator, A-Levels; ID B87012 
‘I think it made me decide I should go to the doctors and ask for treatment. Cos before I didn't know what treatments was open to me, so I was just letting them suggest things and waiting and everything. But now I know and I've been and asked them about the coil business …. It made me feel more in control of my treatment.’
Unemployed single parent, GCSE only; ID 842003 
‘… now I can go to the doctor with the information that I'd never heard of previously, you know and put forward my case for wanting to go ahead with it.’ Librarian, A-Levels; ID C35010 

These results from the qualitative study generated the hypothesis that the intervention was more effective in women who had less formal education. This hypothesis was then examined in quantitative subgroup analysis.

Quantitative subgroup analyses

The qualitative work described above generated the hypotheses that level of formal education may be an important moderator of the Clinical Guidance Tree intervention. Analyses were conducted to see if the change in the primary outcome [Decisional Conflict Scale (DCS)] was moderated (i.e. exaggerated or diminished) according to the educational status. The participants were considered to have higher formal education if they had achieved educational qualifications of A-Levels or above (n = 32, 23%).

The mean DCS scores in the control and intervention groups at baseline and primary follow-up in each education group are shown in Figures 1 and 2.

FIGURE 1

Mean DCS at baseline and follow-up in the control group

FIGURE 1

Mean DCS at baseline and follow-up in the control group

FIGURE 2

Mean DCS at baseline and follow-up in the intervention group

FIGURE 2

Mean DCS at baseline and follow-up in the intervention group

In Figure 1 (the control group), it is apparent that the participants with a higher level of education have greater decisional conflict at baseline. This is reduced by a similar amount at the 2-week follow-up in both the higher and lower educated group. The gradients of the lines on Figure 1 are similar.

At baseline, participants in the intervention group with a higher level of education also had higher levels of decisional conflict. However, Figure 2 shows the intervention successfully reduced decisional conflict in both the higher education and lower education groups, but a greater reduction from baseline decisional conflict has been achieved in the higher education group (signified by the steeper gradient of the line in Fig. 2).

An analysis of covariance was conducted, including a test for interaction between the intervention and educational status which showed that the effect of the intervention is moderated by educational status, but appeared to reduce decisional conflict more successfully in participants with a higher level of education (difference in means −12.9; 95% CI −24.8 to −0.88; P = 0.036).

Discussion

The results showed that the decision aid was broadly beneficial compared to information leaflets alone. The women interviewed were positive about the use of the decision aid and specifically the impact of its use on decision making. An apparent difference in empowerment reported by women with different levels of formal education suggested a hypothesis that this variable would moderate the effect of the intervention.

However, contrary to the hypothesis generated by the qualitative study, quantitative subgroup analyses suggest that, although the intervention successfully reduces decisional conflict in both groups of differing educational status, it reduces DCS to a significantly greater extent in those of higher educational level. Therefore, the findings of the qualitative and quantitative analyses seem to demonstrate a significant inter-method discrepancy.30 The following discussion will consider four possible explanations, concerning limited power in the quantitative study, biased sampling in the qualitative study, differences in interpretation of decisional conflict in the women with different levels of formal education and differences in the meaning of the concepts of empowerment and decisional conflict.

Previous studies have reported discrepancies where qualitative findings have not been statistically significant in the quantitative analysis.1,2 One obvious explanation is that the lack of significant effect reflects insufficient power. In other words, with a greater sample size and power, similar results may have been demonstrated in the quantitative study. As this study demonstrated a significant effect in the quantitative study (albeit opposite to the predicted direction), this is unlikely to be the explanation in this case. To our knowledge studies reporting apparently directly opposing and significant results such as this have not been reported.

Another explanation is that the findings of the qualitative sample are based on a biased subsample of patients, and the levels of empowerment reported did not generalize to the larger quantitative sample. The number of participants in the qualitative subset was small (n = 18), compared to the whole study (n = 149), although the sampling frame was confined to the 72 women who received the intervention. The women were sampled purposively, linked to both the underlying theoretical framework and the developing account as described in the Methods. The use of a theoretically driven sampling frame greatly increases the transferability of the results over and above a convenience sample,31 and we would argue that the methods applied are in line with current qualitative conventions. However, there remains the possibility that the analysis of the qualitative work is merely over-emphasizing a minor difference in a particular subgroup of participants, which may not generalize to the whole sample.

A third explanation is that the marked difference in baseline decisional conflict between the groups is due to the difference between perceived rather than actual decisional conflict in groups of different educational background. This phenomenon has been described before in a previous study in maternity care, where women with lower educational attainment and manual occupational backgrounds were more likely to report exercising informed choice.32 The authors suggested that patients may perceive levels of informed choice in very different ways, based on the meaning they ascribe to informed choice, expectations concerning its availability and their preferences for exercising it.

In the current study, patients who had a lower level of formal education had lower decisional conflict at baseline. This may reflect the fact that these women were less likely to have been aware that participating in decisions about their treatment was an option, resulting in low levels of perceived conflict. For example, one of the items on the DCS was as follows: ‘I am aware of the choices I have to treat my heavy menstrual bleeding’. If the patient felt that their only options were to continue with symptoms or to have a hysterectomy, they might answer ‘strongly agree’, scoring low decisional conflict. Conversely, patients with high levels of education had higher baseline levels of decision conflict, possibly because they were more aware of the availability of possible treatment choices, but felt uninformed about them and unsupported in terms of deciding between them. The inter-method discrepancy may partly reflect the fact that the DCS is interpreted differently by women with different backgrounds.

If the ‘artificial’ difference in DCS at baseline (driven by these different interpretations) is removed, then the benefits of the decision aid would be similar in the two education groups. The inter-method discrepancy would remain in part, because qualitatively, women with lower educational attainment still reported a greater impact, whereas quantitatively there would be no difference. In this case, the more pronounced qualitative impact of the decision aid in women with lower educational attainment may reflect the fact that their interpretation of the meaning of decisional conflict changes before and after the intervention. That is, these women report a larger impact partly because their expectation of choice was initially so much more limited.

Finally, although there is clearly some overlap between the concepts of empowerment identified in the qualitative work and decisional conflict, it is possible that empowerment may not be fully encompassed by the DCS and therefore the quantitative and qualitative studies were measuring different outcomes. The more pronounced benefit in women with lower levels of education, described by the qualitative study, was an increase in perceived empowerment. Empowerment describes the feeling that an individual has the power or authority to do something, whereas decisional conflict is the uncertainty over which course of action to take. These women were describing an effect that appears greater than simple information provision (see Table 1). They describe that not only were they ‘informed a bit more’ (ID774032) but also they were now in a position to discuss treatment options with the health professional and, if desired, challenge their opinion.

We can postulate why the qualitative study found that women with less formal education appeared to be more ‘empowered’ to participate in decision making following using the decision aid. This could simply be because they had the greatest capacity to benefit; women with more formal education may already have been participating in decision making. Empowerment itself may be either an important mediator of the effect of complex interventions such as decision aids or an important outcome in itself, which requires further study and investigations.

Mixed methods can illuminate different aspects of an intervention or provide greater insight into particular issues. Health service researchers need to be aware that the qualitative and quantitative results may be discrepant, and methods need to be developed to deal with such discrepancies. However, even when discrepant results occur, exploring seemingly discrepant findings performs a useful scientific function leading both to further insights into the issue under study (including possible limitations of outcome measures used) and in suggesting new hypotheses and avenues of research.12

Declaration

Funding: The Medical Research Council provided funding for the study and support for JP with a Training Fellowship in Health Services Research.

Ethics approval: Ethical approval was obtained from Manchester, Stockport, Salford and Trafford, Tameside and Glossop, South Cheshire, Bolton, Bury and Rochdale, Sheffield and Wirral Local Research Ethic Committees.

Conflicts of interest: None.

The authors would like to express their thanks to the practices and patients involved in this study; Tom Fahey and Tim Peters for their comments on earlier drafts of this paper and Angela Swallow for her help with patient interviews.

References

1
Moffatt
S
White
M
Mackintosh
J
Howel
D
Using quantitative and qualitative data in health services research—what happens when mixed method findings conflict? [ISRCTN61522618]
BMC Health Serv Res
 , 
2006
, vol. 
6
 pg. 
28
 
2
Cox
K
Assessing the quality of life of patients in phase I and II anti-cancer drug trials: interviews versus questionnaires
Soc Sci Med
 , 
2003
, vol. 
56
 (pg. 
921
-
934
)
3
Oakley
A
Strange
V
Bonell
C
Allen
E
Stephenson
J
Process evaluation in randomised controlled trials of complex interventions
BMJ
 , 
2006
, vol. 
332
 (pg. 
413
-
416
)
4
Stacey
D
Graham
ID
O'Connor
AM
Pomey
MP
Barriers and facilitators influencing call center nurses’ decision support for callers facing values-sensitive decisions: a mixed methods study
Worldviews Evid Based Nurs
 , 
2005
, vol. 
2
 (pg. 
184
-
195
)
5
Medical Research Council
A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health
 , 
2000
London
Medical Research Council
6
Campbell
M
Fitzpatrick
R
Haines
A
, et al.  . 
Framework for design and evaluation of complex interventions to improve health
BMJ
 , 
2000
, vol. 
321
 (pg. 
694
-
696
)
7
Baron
RM
Kenny
DA
The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations
J Pers Soc Psychol
 , 
1986
, vol. 
51
 (pg. 
1173
-
1182
)
8
Kraemer
HC
Wilson
GT
Fairburn
CG
Agras
WS
Mediators and moderators of treatment effects in randomized clinical trials
Arch Gen Psychiatry
 , 
2002
, vol. 
59
 (pg. 
877
-
883
)
9
Campbell
NC
Murray
E
Darbyshire
J
, et al.  . 
Designing and evaluating complex interventions to improve health care
BMJ
 , 
2007
, vol. 
334
 (pg. 
455
-
459
)
10
Montgomery
AA
Fahey
T
Peters
TJ
A factorial randomised controlled trial of decision analysis and an information video plus leaflet for newly diagnosed hypertensive patients
Br J Gen Pract
 , 
2003
, vol. 
53
 (pg. 
446
-
453
)
11
Weiss
MC
Montgomery
AA
Fahey
T
Peters
TJ
Decision analysis for newly diagnosed hypertensive patients: a qualitative investigation
Patient Educ Couns
 , 
2004
, vol. 
53
 (pg. 
197
-
203
)
12
O'Cathain
A
Murphy
E
Nicholl
J
Integration and publications as indicators of “yield” from mixed methods studies
J Mixed Methods Res
 , 
2007
, vol. 
1
 (pg. 
147
-
163
)
13
O'Connor
AM
Stacey
D
Entwistle
V
, et al.  . 
Decision Aids for People Facing Health Treatment or Screening Decisions, [A Cochrane Review]
The Cochrane Library (Issue 2)
 , 
2003
Oxford
Update Software
14
O'Connor
AM
Tugwell
P
Wells
GA
, et al.  . 
A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation
Patient Educ Couns
 , 
1998
, vol. 
33
 (pg. 
267
-
279
)
15
Royal College of Obstetricians and Gynaecologists
The Initial Management of Menorrhagia
 , 
1998
London
RCOG Press
16
Dowding
D
Swanson
V
Bland
R
, et al.  . 
The development and preliminary evaluation of a decision aid based on decision analysis for two treatment conditions: benign prostatic hyperplasia and hypertension
Patient Educ Couns
 , 
2004
, vol. 
52
 (pg. 
209
-
215
)
17
Protheroe
J
Bower
P
Chew-Graham
C
Peters
T
Fahey
T
Effectiveness of a computerized decision aid in primary care on decision making and quality of life in menorrhagia: resutls of the MENTIP randomized controlled trial
Med Decis Making
 , 
2007
, vol. 
27
 (pg. 
575
-
584
)
18
Protheroe
J
Enhancing decision making in menorrhagia in primary care using computerised decision analysis: randomised trial and qualitative evaluation
 , 
2005
 
PhD Thesis, University of Manchester, Manchester, UK
19
O'Connor
A
Validation of a decisional conflict scale
Med Decis Making
 , 
1995
, vol. 
15
 (pg. 
25
-
30
)
20
O'Connor
A
Tugwell
P
Wells
GA
, et al.  . 
Randomized trial of a portable, self-administered decision aid for postmenopausal women considering long-term preventative hormone therapy
Med Decis Making
 , 
1998
, vol. 
18
 (pg. 
295
-
303
)
21
Davison
BJ
Kirk
P
Degner
LF
Hassard
TH
Information and patient participation in screening for prostate cancer
Patient Educ Couns
 , 
1999
, vol. 
37
 (pg. 
255
-
263
)
22
Man-Son-Hing
M
Laupacis
A
O'Connor
AM
, et al.  . 
A patient decision aid regarding antithrombotic therapy for stroke prevention in atrial fibrillation—a randomised controlled trial
JAMA
 , 
1999
, vol. 
282
 (pg. 
737
-
778
)
23
Goel
V
Sawka
CA
Thiel
EC
Gort
EH
O'Connor
AM
Randomized trial of a patient decision aid for choice of surgical treatment for breast cancer
Med Decis Making
 , 
2001
, vol. 
21
 (pg. 
1
-
6
)
24
Murray
E
Davis
H
See Tai
S
Coulter
A
Gray
A
Haines
A
Randomised controlled trial of an interactive multimedia decision aid on benign prostatic hypertrophy in primary care
BMJ
 , 
2001
, vol. 
323
 (pg. 
493
-
496
)
25
Green
MJ
Biesecker
BB
McInerney
AM
Mauger
D
Fost
N
An interactive computer program can effectively educate patients about genetic testing for breast cancer susceptibility
Am J Med Genet
 , 
2001
, vol. 
103
 (pg. 
16
-
23
)
26
Dolan
JG
Frisina
S
Randomized controlled trial of a patient decision aid for colorectal cancer screening
Med Decis Making
 , 
2002
, vol. 
22
 (pg. 
125
-
139
)
27
Rostom
A
O'Connor
A
Tugwell
P
Wells
G
A randomized trial of a computerized versus an audio-booklet decision aid for women considering post-menopausal hormone replacement therapy
Patient Educ Couns
 , 
2002
, vol. 
46
 (pg. 
67
-
74
)
28
Shaw
RW
Brickley
M
Evans
L
Edwards
M
Perceptions of women of the impact of menorrhoea on their health using multi-attribute utility assessment
Br J Obstet Gynaecol
 , 
1998
, vol. 
105
 (pg. 
1155
-
1159
)
29
Christensen
M
Hewitt-Taylor
J
Empowerment in nursing: paternalism or maternalism?
Br J Nurs
 , 
2006
, vol. 
15
 (pg. 
695
-
699
)
30
Fielding
NG
Fielding
JL
Linking Data
 , 
1986
London
Sage
31
Greenhalgh
T
How to read a paper: papers that go beyond numbers (qualitative research)
BMJ
 , 
1997
, vol. 
315
 (pg. 
740
-
743
)
32
O'Cathain
A
Thomas
K
Walters
SJ
Nicholl
J
Kirkham
M
Women's perceptions of informed choice in maternity care
Midwifery
 , 
2002
, vol. 
18
 (pg. 
136
-
144
)

Author notes

Protheroe J, Bower P and Chew-Graham C. The use of mixed methodology in evaluating complex interventions: identifying patient factors that moderate the effects of a decision aid. Family Practice 2007; 24: 594–600.