Abstract

Objectives. The aim of this study was to compare two different approaches to the measurement of quality in general practice: data derived from routine NHS data sets and results from an index derived from patient-collected data.

Methods. A secondary analysis of existing data sets and a cross-sectional survey were carried out in Lothian, Coventry, Oxfordshire and west London. The subjects comprised randomly selected and consenting practices, and a sample of patients within these practices. A National Health Service Practice Performance Index (NHSPPI) was constructed from 16 routinely available NHS performance indicators. The Consultation Quality Index (CQI) combines the Patient Enablement Instrument (PEI) with a measure of how well the patient knew the doctor, and with observed consultation length.

Results. Scores for 12 of the 16 indicators varied significantly across the four regions. Mean practice NHSPPI score overall was 21.6 (SD 4.3), which varied significantly across regions. NHSPPI was predicted by practice list size, weighted deprivation index and proportion of other language patients in the practice, although their effects could not be separated. Overall there was no correlation between NHSPPI and CQI, although the prescribing component of the index was positively correlated to mean consultation length and negatively correlated with how well patients knew their doctors.

Conclusions. Good quality care as assessed by patients on completion of their consultation is independent of good quality care as assessed by best available measures of practice performance. We suggest that the CQI and the NHSPPI are at least as ready for use as other measures of performance in general practice.

Heaney DJ, Walker JJ, Howie JGR, Maxwell M, Freeman GK, Berrey PNE, Jones TG, Stern MC and Campbell SM. The development of a routine NHS data-based index of performance in general practice (NHSPPI). Family Practice 2002; 19: 77–84.

Introduction

Quality has been placed ‘at the heart' of the NHS.1,2 The introduction of clinical governance means performance has become a corporate responsibility: the aim is continual improvement in quality of service.3,4 As such, clinical governance sets a considerable challenge to general practice in the UK, although the measurement of quality is not a new endeavour.5–11 During the last decade, the Department of Health made several attempts to utilize routinely available NHS data in a systematic form at district level. The measures of performance have included prescribing rates, referral rates, preventive care targets and measures of administrative efficiency. These initiatives have laid the foundations for the indicators published in the national framework for assessing performance.12

Recent work has assessed the face validity of quality indicators for use in general practice.13 Although a total of 240 separate quality indicators were identified as proposed for use in general practice by health authorities, none related to effective communication, care of acute illnesses, health outcomes or patient evaluation. Available and valid individual routine measures may be important but they cannot be used in isolation to represent the complexity of general practice. The gap between available data and what might be seen as comprehensive assessment was recognized in the development of a framework for assessing quality in general practice.14 This framework divided measures into three categories (common acute problems, continuing chronic problems and asymptomatic problems/health promotion) and also classified by time (before, immediately after, 14 days after and 6–12 months after the consultation). This approach emphasized the patient, the doctor and the consultation rather than the population as the denominators in the construction of measures of outcome.

This present paper describes work attempting to compare two different approaches to the measurement of quality. Randomly selected and consenting practices from four different parts of the UK participated in a study investigating quality of care in general practice. Sixteen performance indicators derived from routine NHS data sets were collated for each practice in the study and aggregated into a National Health Service Practice Performance Index (NHSPPI). This paper compares results from this routine data collection with results from an index derived from patient-collected data.

Methods

In 1997, our research team approached a set of 140 randomly selected practices in four participating regions (Lothian, Coventry, Oxfordshire and West London) to gain permission to access routinely held NHS data sets. The research team included three health authority medical advisers and they identified a list of routinely available NHS data which were already in use by them or had been suggested as appropriate measures of quality performance in general practice. From their peer group's experience and publications, a short list of relatively robust prescribing performance indicators was identified. Some prescribing data that are routinely available in England and Wales are unavailable in Scotland, and vice versa: this added a constraint to the range of acceptable markers. To this list was added a short set of routinely available data comprising performance towards remunerated targets, eligibility for postgraduate education allowance, remunerated non-general medical services and levels of remuneration claimed for ancillary staff. The main constraint imposed in the production of ‘routine management data' was to produce results without investing large amounts of health authority resource in their computation. The exact definition for each of the selected indicators is included in Appendix 1. The medical advisers subsequently worked closely together to ensure that data collection was consistent across regions.

Once the results for each region were available, the research team met to define thresholds of acceptable practice performance for each indicator. The team devised four categories: ‘fully acceptable' performance, ‘possibly acceptable' performance, ‘probably unacceptable' performance and ‘unacceptable' performance. The thresholds selected for each indicator are presented in Appendix 1. The team assigned values of 2, 1, 0 and –1 to these categories. Two of the original indicators were dropped at this stage: the cost of inhaled steroids (as it was felt this was not a measure of quality) and the number of contraceptive fee claims (although this was a measure of quality, it was felt that achieving it was out with the control of the practice as other agencies provide contraceptive services). For two further indicators (quinolones and appetite suppressants), it was decided that it was too difficult to differentiate between ‘probably unacceptable' and ‘unacceptable', and the lowest (–1) category was not used. Data for registration for child health surveillance, minor surgery and maternity care were aggregated into one indicator. The practice scored 2 if any combination of doctors in the practice were registered for all three, scored 1 if registered for any two out of the three, scored 0 if only registered for one of the three and scored –1 if registered for none of these.

Responses were aggregated across the basket of 16 indicators to devise a ‘National Health Service Practice Performance Index' (NHSPPI) for each practice. Thus a practice could score a maximum of 32 if for each measure their performance fell within the defined accepted range. The theoretical minimum (–14) would be scored if the practice failed on each indicator. The component parts of the NHSPPI were defined as follows: a practice prescribing subindex (PPSI items 1–10), a practice preventive subindex (PRSI items 11–13) and a practice external quality subindex (PQSI items 14–16).

The above exercise was carried out in parallel with a large consultation-based survey in which 53 of the 140 practices (38%) participated (a condition being the agreement of all partners in small practices and of all except one or two in practices of five or more partners). A total of 26 000 patients attending 221 doctors completed the Patient Enablement Instrument (PEI) 9 after consultations during a 2-week period in March/April 1998. The PEI score was combined with a measure of how well the patient knew the doctor, and with observed consultation length to create a single proxy measure of quality of care at the generality of general practice consultations (Consultation Quality Index; CQI). All doctors with at least 50 qualifying consultations were ranked in descending order separately for their mean enablement score, mean consultation length and the percentage of patients consulting who knew them well or very well using all data available. The ranks were divided into sextiles. Doctors in the top sextile for any variable were allocated a score of 6 points, down to a score of 1 point for a place in the sixth (lowest) sextile. A total CQI score was then calculated for each doctor by adding their scores for all three variables. The scores thus ranged from 3 to 18.15 For the purposes of this paper, the CQI, which we have presented as an index of quality of care at doctor level, was aggregated to practice level and compared with practice score on the NHSPPI to test for correlation between these different approaches to the measurement of quality in general practice. The methods and results from the consultation-based study,16 and the development of the CQI15 have been described more fully elsewhere. The hypothesis that practice demography would predict NHSPPI was tested. A multiple regression model was constructed with overall NHSPPI score as the independent variable. Practice list size, a deprivation rank (calculated using enumeration district level UPA scores), proportion of other language patients in the practice (obtained from the patient questionnaire) and region (i.e. Lothian, Coventry, Oxfordshire and West London) were selected as predictors.

Results

Over the four regions of study, 120/140 of the randomly selected practices gave their permission to be included in the analysis of routine data. Table 1 summarizes the results for each individual performance indicator for all practices, and for those participating and not participating in the consultation study. Only two (item 3 at the 5% level, and item 15 at the 1% level) of the 16 indicators were scored significantly differently between practices which did or did not participate in the consultation study.

Table 2 compares the mean results for each indicator in the four regions in the study. Scores for 12 of the 16 indicators varied significantly across the four regions, but relative performance did not always favour the same regions. Some of the differences appeared to be of marginal clinical significance, but four indicators were of particular interest. The number of items per 1000 patients for cough suppressants and nasal decongestants was 159.8 in Coventry and 115.3 in West London, compared with 32.4 in Lothian and 17.4 in Oxford. Average prescribing for cough suppressants and nasal decongestants in Coventry and West London therefore fell into the ‘probably unacceptable' range as defined for individual practices by the research team (100 items per 1000 patients per annum or above). Similarly, average prescribing in Coventry for the ratio of bendrofluazide 5 mg tablets to all bendrofluazide items and for the ratio of compound diuretics plus diuretics with potassium to all diuretic items fell into the ‘probably unacceptable' category. Finally, Oxford practices prescribed double the number of statins per 1000 patients compared with Coventry practices (the Oxford average falling into our ‘acceptable' category whereas the Coventry average was only ‘possibly acceptable').

Table 3 shows mean practice NHSPPI scores across the four regions in the study. The mean score overall was 21.6 (SD 4.3), the median score was 22 and the modal score was 24. The ‘best' performing practice (score 31) had acceptable scores for all individual indicators except one. The ‘worst' performing practice (score 11) had five ‘acceptable' scores, three ‘possibly acceptable' scores, six ‘probably unacceptable' scores and two ‘unacceptable' scores. There was a statistically significant difference in mean NHSPPI across regions. The mean score in Oxford practices was 24.8 compared with 19.9 in Coventry practices (P < 0.001). When two indicators were removed (cough suppressants and diuretics) from the total scores, this reduced the differences although they remained significant (P = 0.03).

The rank correlations between list size, deprivation and proportion of other language patients in the practice were strong, as shown in Table 4. The overall regression was highly significant (P = 0.003), and the proportion of variance explained (adjusted R2) was 0.27. Yet, reference to the parameter estimates showed that none of the predictors were significant at the conventional 5% level (except region = Oxford). These variables and their effects cannot be separated reliably.

Table 5 shows the correlation between the NHSPPI (and its components) and the CQI (and its components) for practices who participated in the consultation-based survey in each region. Overall, there was no correlation between practices' rank ordering for their scores on the NHSPPI and their scores on the CQI. However, rank order for mean consultation length was correlated with rank order for NHSPPI and to the prescribing subindex (PPSI). Also, practices' rank ordering for how well patients knew the doctor was negatively correlated with rank ordering for the PPSI.

Discussion

Strengths and weaknesses of the method

The indicators selected for inclusion in the NHSPPI are intended to reflect both good clinical practice and cost-effectiveness. They were chosen within the constraint that the data had to be reasonably easy to collate in a consistent manner across the four regions under study. Quality is a multifactorial concept, and individual markers derived from existing routine sources do not always place performance in context. We recognize that in some cases there may be legitimate reasons unknown to the research team which explain why a practice has fallen outside the bounds of acceptability for one particular indicator.

The research team agreed the ranges of acceptability for each indicator after examination of the results but before attempting comparisons between regions or practices. This made our decisions considerably easier, although even then some indicators were not easily subjected to the four-level categorization we devised. It may be that some commentators will disagree with the values we have chosen or that some indicators should be excluded or weighted. Our decisions in defining ‘acceptable' practice were arbitrary; it would be easy to re-adjust the NHSPPI accordingly. In a recently published Delphi study17 looking at the validity of prescribing indicators, all but two of the prescribing indicators in the NHSPPI were rated valid. The exceptions were the ratio of steroid to bronchodilator items which was not assessed because data in terms of defined daily doses (DDD) and specific therapeutic groups age sex-related prescribing units (STAR-PU) are not routinely available in Scotland. Secondly, the number of items for statins per 1000 patients was rated neither valid nor invalid. All preventive components of the NHSPPI are already elements of GPs' remuneration.

An equally important question is whether aggregating individual performance indicators is an appropriate and correct way of assessing quality. Does the performance index presented in this paper reflect a cohesive measure of one aspect of quality in general practice? All the routine indicators used in this study have necessarily been aggregated to practice level. This means that the NHSPPI cannot identify difference in performance between individual GPs. In terms of representativeness, it was important to note little difference in the NHSPPI between practices agreeing and declining to participate in the consultation study.

Implications of results

Variation within general practice is a well-established phenomenon. The results presented for each of the performance indicators are worthy of examination in their own right. Some of the variation appears to be statistically rather than clinically significant, but particularly striking were the 5-fold differences in prescribing for cough suppressants and nasal decongestants across regions and 2-fold differences in prescribing of statins across regions. It is possible that the context (which includes expectations and values of patients) predicts the need for different standards and that some of these variations are due to cultural differences between regions rather than differences in competence between practices. Equally it is possible that there are more practices in Oxford than in Coventry which are ‘early adaptors' of change, and that the differences (both to lower use of discredited therapies and to higher use of new ones) do reflect better as opposed to poorer care.

Higher practice-level deprivation is associated with lower scores on the NHSPPI, but a causal relationship cannot be inferred. The effect of deprivation is confounded with smaller practice list sizes and with higher proportions of patients who speak languages other than English.

Clinical governance

Little evidence of correlation was found between the CQI and the NHSPPI or between components of either. However, practices where doctors had longer mean consultation lengths performed better on the prescribing indicators (in two of four regions). In contrast, practices in which the patients knew the doctors better performed less well on prescribing indicators. The first of these results has good face validity, the second suggests that familiarity may compromise prescribing quality standards. This finding may be due to confounding factors, and requires further examination.

The lack of an overall correlation between the CQI and NHSPPI may be due to the problem of extrapolating between measures which use different denominators. Otherwise the measures capture aspects of quality in general practice which are not related: good quality care as assessed by patients on completion of their consultation is independent of good quality care as assessed by best available measures of practice performance.

Nonetheless, it seems reasonable to ask whether practices with a poor performance on the NHSPPI added to a poor performance on the CQI warrant concern. Would a significant absolute improvement in performance as judged by either the CQI or the NHSPPI over time be enough to demonstrate that a practice has improved the quality of care it provides? The complexity of trying to answer such questions demonstrates that there are likely to be few short cuts in the introduction of clinical governance and that the measurement of quality in general practice is only now at the starting line. We would suggest that the CQI and NHSPPI are at least as ready for use as other measures of performance in general practice, and that continued effort is required to develop and promote instruments which will allow a meaningful assessment of whether continual quality improvement is being achieved.

Appendix 1

Definitions of performance indicators included in the study and thresholds for ‘fully acceptable' performance, ‘possibly acceptable' performance, ‘probably unacceptable' performance and ‘unacceptable' performance for each indicator

Indicatora Direction Range of valuesb 
  Fully
 acceptable ‘Near
 miss' Unacceptable Cause for
 concernc 
a Two indicators featured in the original ‘basket'—cost of inhaled steroids and number of contraceptive fee claims—have been dropped. 
b In computing the PPI, each ‘fully acceptable' result is assigned a score of 2; each ‘near miss' result a score of 1; each ‘unacceptable' result a score of 0; and each ‘cause for concern' result a score of –1. 
c An entry ‘N/A' indicates that the full four-band grading system was not considered applicable to the indicator in question, and that the indicator was, in consequence, reduced to only three bands. 
d Indicator 15 consists of three subitems, each of which attracts a score of 1 when the condition is met (e.g. doctor registered for child health surveillance); otherwise zero. An overall score for item 15 is arrived at by summing scores on the three subitems then subtracting 1, the resulting score ranging from –1 (no conditions met) to 2 (all conditions met). This score is interpreted in the same way as scores on the other (non-binary) indicators, i.e. 2 is considered fully acceptable, –1 represents a ‘cause for concern' level of performance. 
1. Percentage of items for 4-quinolones to all antibiotic items in past year Lower is better <3.0 3.0–4.9 ≥5.0 N/A 
2. Percentage of bendrofluazide 5 mg tablets to all bendrofluazide items in past year Lower is better <20.0 20.0–29.9 30.0–39.9 ≥40.0 
3. Percentage of (compound diuretics+diuretics with potassium) to all diuretic items in past year Lower is better <25.0 25.0–34.9 35.0–49.9 ≥50.0 
4. Nunber of items per 1000 patients for cough suppressants and nasal decongestants in past year Lower is better <50.0 50.0–99.9 100.0–299.9 ≥300.0 
5. Number of items for appetite suppressants per 1000 patients in past year Lower is better <2.0 2.0–2.9 ≥3.0 N/A 
6. Percentage of NSAID items comprised by the practice's top four NSAIDs in past year Higher is better ≥80.0 70.0–79.9 50.0–69.9 <50.0 
7. Percentage of antibiotic items comprised by the practice's top ten antibiotic items in past year Higher is better ≥95.0 85.0–94.9 70.0–84.9 <70.0 
8. Ratio of steroid to bronchodilator items in past year Higher is better ≥50.0 40.0–49.9 30.0–39.9 <30.0 
9. Number of items for statins per 1000 patients in past year Higher is better ≥100.0 50.0–99.9 20.0–49.9 <20.0 
10. Number of items for HRT (BNF 4.4.1.1) per 1000 female patients aged 45–65 in past year Higher is better ≥750.0 600.0–749.9 300.0–599.9 <300.0 
11. Uptake rate (percentage): primary immunization course for infants Higher is better ≥95.0 90.0–94.9 70.0–89.9 <70.0 
12. Uptake rate (percentage): pre-school immunization Higher is better ≥95.0 90.0–94.9 70.0–89.9 <70.0 
13. Uptake rate (percentage): cervical cytology screening rate for the target population Higher is better ≥80.0 50.0–79.9 40.0–49.9 <40.0 
14. Sum of PGEA (all partners) as percentage of maximum that could have been obtained Higher is better ≥95.0 90.0–94.9 0.1–89.9 
Are there doctors working regularly in the practice who are registered for:d(a) Child health surveillance
 (b) Minor surgery
 (c) Maternity care Higher is better Registered for all three Registered for two Registered for one Registered for none 
16. Level of reimbursement for staff costs (£ per 1000 patients) Higher is better ≥£14 000 £12 000–£13 999 £5000–£11 999 <£5000 
Indicatora Direction Range of valuesb 
  Fully
 acceptable ‘Near
 miss' Unacceptable Cause for
 concernc 
a Two indicators featured in the original ‘basket'—cost of inhaled steroids and number of contraceptive fee claims—have been dropped. 
b In computing the PPI, each ‘fully acceptable' result is assigned a score of 2; each ‘near miss' result a score of 1; each ‘unacceptable' result a score of 0; and each ‘cause for concern' result a score of –1. 
c An entry ‘N/A' indicates that the full four-band grading system was not considered applicable to the indicator in question, and that the indicator was, in consequence, reduced to only three bands. 
d Indicator 15 consists of three subitems, each of which attracts a score of 1 when the condition is met (e.g. doctor registered for child health surveillance); otherwise zero. An overall score for item 15 is arrived at by summing scores on the three subitems then subtracting 1, the resulting score ranging from –1 (no conditions met) to 2 (all conditions met). This score is interpreted in the same way as scores on the other (non-binary) indicators, i.e. 2 is considered fully acceptable, –1 represents a ‘cause for concern' level of performance. 
1. Percentage of items for 4-quinolones to all antibiotic items in past year Lower is better <3.0 3.0–4.9 ≥5.0 N/A 
2. Percentage of bendrofluazide 5 mg tablets to all bendrofluazide items in past year Lower is better <20.0 20.0–29.9 30.0–39.9 ≥40.0 
3. Percentage of (compound diuretics+diuretics with potassium) to all diuretic items in past year Lower is better <25.0 25.0–34.9 35.0–49.9 ≥50.0 
4. Nunber of items per 1000 patients for cough suppressants and nasal decongestants in past year Lower is better <50.0 50.0–99.9 100.0–299.9 ≥300.0 
5. Number of items for appetite suppressants per 1000 patients in past year Lower is better <2.0 2.0–2.9 ≥3.0 N/A 
6. Percentage of NSAID items comprised by the practice's top four NSAIDs in past year Higher is better ≥80.0 70.0–79.9 50.0–69.9 <50.0 
7. Percentage of antibiotic items comprised by the practice's top ten antibiotic items in past year Higher is better ≥95.0 85.0–94.9 70.0–84.9 <70.0 
8. Ratio of steroid to bronchodilator items in past year Higher is better ≥50.0 40.0–49.9 30.0–39.9 <30.0 
9. Number of items for statins per 1000 patients in past year Higher is better ≥100.0 50.0–99.9 20.0–49.9 <20.0 
10. Number of items for HRT (BNF 4.4.1.1) per 1000 female patients aged 45–65 in past year Higher is better ≥750.0 600.0–749.9 300.0–599.9 <300.0 
11. Uptake rate (percentage): primary immunization course for infants Higher is better ≥95.0 90.0–94.9 70.0–89.9 <70.0 
12. Uptake rate (percentage): pre-school immunization Higher is better ≥95.0 90.0–94.9 70.0–89.9 <70.0 
13. Uptake rate (percentage): cervical cytology screening rate for the target population Higher is better ≥80.0 50.0–79.9 40.0–49.9 <40.0 
14. Sum of PGEA (all partners) as percentage of maximum that could have been obtained Higher is better ≥95.0 90.0–94.9 0.1–89.9 
Are there doctors working regularly in the practice who are registered for:d(a) Child health surveillance
 (b) Minor surgery
 (c) Maternity care Higher is better Registered for all three Registered for two Registered for one Registered for none 
16. Level of reimbursement for staff costs (£ per 1000 patients) Higher is better ≥£14 000 £12 000–£13 999 £5000–£11 999 <£5000 
Table 1

Minimum, maximum and mean practice values (SD) for each indicator included in the NHSPPI, and mean values (SD) for those practices participating and not participating in the consultation-based survey

Indicator n Minimum Maximum Mean
 (SD) Participants
 n Mean SD Non-participants
 n Mean SD Pa 
a For all indicators except number 15, the column headed ‘P' gives the result from a t-test of the null hypothesis of equal mean values between participating and non-participating practices. For indicator 15, the value shown is the result of a Mann–Whitney test. 
b Three values in excess of 100% were restricted to 100% in all analyses. 
1. Quinolones 120  0.0  21.0  3.4 (2.6) 52  3.2  1.9 68  3.5  3.0 0.46 
2. Bendrofluazide 120  0.0  78.3  26.7 (18.3) 52  24.6  16.1 68  28.4  19.7 0.27 
3. Diuretics 120  3.0  80.6  27.7 (14.3) 52  24.6  13.1 68  30.1  14.7 0.04 
4. Cough suppressants 120  2.0  875.0  90.6 (117.4) 52  74.1  101.6 68  103.3  127.4 0.18 
5. Appetite suppressants 120  0.0  9.0  0.4 (1.2) 52  0.3  0.9 68  0.5  1.4 0.37 
6. NSAIDs 120  57.1  99.2  83.8 (7.2) 52  84.7  6.3 68  83.1  7.7 0.24 
7. Antibiotics 120  63.0  100.0  89.3 (5.1) 52  89.1  3.6 68  89.4  6.0 0.74 
8. Steroid/bronchodilator 120  24.1  70.2  47.3 (9.1) 52  47.7  8.7 68  47.0  9.5 0.66 
9. Statins 120  0.0  489.0  87.2 (59.7) 52  92.3  72.3 68  83.3  48.1 0.44 
10. HRT 120  0.0  1790.0  898.9 (286.1) 52  903.0  272.3 68  895.7  298.2 0.89 
11. Primary immunization 120  75.0  100.0  95.9 (4.2) 52  95.4  4.3 68  96.2  4.1 0.31 
12. Pre-school immunization 120  54.2  100.0  91.4 (7.9) 52  90.8  8.0 68  91.8  7.9 0.50 
13. Cervical cytology 120  40.0  98.7  81.5 (9.5) 52  82.9  8.0 68  80.5  10.5 0.15 
14. PGEAb 118  0.0  100.0  95.4 (12.5) 51  97.0  7.4 67  94.2  15.3 0.21 
15. Doctor(s) registered for
 Child health surveillance/
 Minor surgery/
 Maternity care 119  –1  1.8 (0.6) 51  1.9  0.3 68  1.7  0.7 0.13 
16. Staff reimbursement 118 3133 31 982 12 962 (3517) 51 13 554 4185 67 12 511 2862 0.13 
Indicator n Minimum Maximum Mean
 (SD) Participants
 n Mean SD Non-participants
 n Mean SD Pa 
a For all indicators except number 15, the column headed ‘P' gives the result from a t-test of the null hypothesis of equal mean values between participating and non-participating practices. For indicator 15, the value shown is the result of a Mann–Whitney test. 
b Three values in excess of 100% were restricted to 100% in all analyses. 
1. Quinolones 120  0.0  21.0  3.4 (2.6) 52  3.2  1.9 68  3.5  3.0 0.46 
2. Bendrofluazide 120  0.0  78.3  26.7 (18.3) 52  24.6  16.1 68  28.4  19.7 0.27 
3. Diuretics 120  3.0  80.6  27.7 (14.3) 52  24.6  13.1 68  30.1  14.7 0.04 
4. Cough suppressants 120  2.0  875.0  90.6 (117.4) 52  74.1  101.6 68  103.3  127.4 0.18 
5. Appetite suppressants 120  0.0  9.0  0.4 (1.2) 52  0.3  0.9 68  0.5  1.4 0.37 
6. NSAIDs 120  57.1  99.2  83.8 (7.2) 52  84.7  6.3 68  83.1  7.7 0.24 
7. Antibiotics 120  63.0  100.0  89.3 (5.1) 52  89.1  3.6 68  89.4  6.0 0.74 
8. Steroid/bronchodilator 120  24.1  70.2  47.3 (9.1) 52  47.7  8.7 68  47.0  9.5 0.66 
9. Statins 120  0.0  489.0  87.2 (59.7) 52  92.3  72.3 68  83.3  48.1 0.44 
10. HRT 120  0.0  1790.0  898.9 (286.1) 52  903.0  272.3 68  895.7  298.2 0.89 
11. Primary immunization 120  75.0  100.0  95.9 (4.2) 52  95.4  4.3 68  96.2  4.1 0.31 
12. Pre-school immunization 120  54.2  100.0  91.4 (7.9) 52  90.8  8.0 68  91.8  7.9 0.50 
13. Cervical cytology 120  40.0  98.7  81.5 (9.5) 52  82.9  8.0 68  80.5  10.5 0.15 
14. PGEAb 118  0.0  100.0  95.4 (12.5) 51  97.0  7.4 67  94.2  15.3 0.21 
15. Doctor(s) registered for
 Child health surveillance/
 Minor surgery/
 Maternity care 119  –1  1.8 (0.6) 51  1.9  0.3 68  1.7  0.7 0.13 
16. Staff reimbursement 118 3133 31 982 12 962 (3517) 51 13 554 4185 67 12 511 2862 0.13 
Table 2

Mean practice values (SD) by region for each indicator in the NHSPPI

Indicator Summary values for area 
 Oxford Lothian West London Coventry Pa 
 Mean SD Mean SD Mean SD Mean SD  
a The column headed ‘P’ gives the result from a Kruskal–Wallis test (data classified by area). 
b Three values in excess of 100% were restricted to 100% in all analyses. 
1. 4-Quinolones  3.7  2.6  4.1  3.9  3.3  2.0  2.9  1.9 0.26 
2. Bendrofluazide  22.5  13.8  25.6  14.8  24.9  17.5  32.4  22.8 0.44 
3. Diuretics  18.9  10.4  18.2  7.9  33.6  15.4  35.0  11.9 <0.01 
4. Cough suppressants/nasal decongestants  17.4  9.5  32.4  40.1  115.3  100.6  159.8  157.8 <0.01 
5. Appetite suppressants  0.8  1.9  0.1  0.2  0.4  1.1  0.4  1.1 0.37 
6. NSAIDs  87.4  3.4  78.7  4.1  90.0  4.5  78.6  6.6 <0.01 
7. Antibiotics  88.2  2.7  90.9  3.7  91.0  4.5  87.3  6.7 <0.01 
8. Steroids  49.9  5.3  44.6  8.4  44.9  9.6  49.8  10.4 <0.01 
9. Statins  127.7  100.8  89.6  31.9  85.3  43.5  58.6  24.9 <0.01 
10. HRT  1049.9  226.6  723.0  135.0  880.0  347.6  935.5  274.4 <0.01 
11. Primary immunization  96.5  2.8  97.9  1.3  93.7  6.3  96.2  2.8 <0.01 
12. Pre-school immunization  94.1  4.7  93.4  4.8  85.4  11.0  93.9  4.0 <0.01 
13. Cervical screening  80.0  11.2  87.0  6.6  75.9  10.5  84.3  4.9 <0.01 
14. PGEAb  95.5  8.4  98.4  4.3  95.0  18.1  93.7  12.1 <0.01 
15. Doctor(s) registered for
 Child health surveillance/
 Minor surgery/
 Maternity care  2.0  0.0  1.9  0.4  1.6  0.7  1.7  0.7 <0.01 
16. Staff costs (£ per 1000 patients) 12 765 2214 12 469 5197 13 821 4050 12 568 2032 0.25 
Indicator Summary values for area 
 Oxford Lothian West London Coventry Pa 
 Mean SD Mean SD Mean SD Mean SD  
a The column headed ‘P’ gives the result from a Kruskal–Wallis test (data classified by area). 
b Three values in excess of 100% were restricted to 100% in all analyses. 
1. 4-Quinolones  3.7  2.6  4.1  3.9  3.3  2.0  2.9  1.9 0.26 
2. Bendrofluazide  22.5  13.8  25.6  14.8  24.9  17.5  32.4  22.8 0.44 
3. Diuretics  18.9  10.4  18.2  7.9  33.6  15.4  35.0  11.9 <0.01 
4. Cough suppressants/nasal decongestants  17.4  9.5  32.4  40.1  115.3  100.6  159.8  157.8 <0.01 
5. Appetite suppressants  0.8  1.9  0.1  0.2  0.4  1.1  0.4  1.1 0.37 
6. NSAIDs  87.4  3.4  78.7  4.1  90.0  4.5  78.6  6.6 <0.01 
7. Antibiotics  88.2  2.7  90.9  3.7  91.0  4.5  87.3  6.7 <0.01 
8. Steroids  49.9  5.3  44.6  8.4  44.9  9.6  49.8  10.4 <0.01 
9. Statins  127.7  100.8  89.6  31.9  85.3  43.5  58.6  24.9 <0.01 
10. HRT  1049.9  226.6  723.0  135.0  880.0  347.6  935.5  274.4 <0.01 
11. Primary immunization  96.5  2.8  97.9  1.3  93.7  6.3  96.2  2.8 <0.01 
12. Pre-school immunization  94.1  4.7  93.4  4.8  85.4  11.0  93.9  4.0 <0.01 
13. Cervical screening  80.0  11.2  87.0  6.6  75.9  10.5  84.3  4.9 <0.01 
14. PGEAb  95.5  8.4  98.4  4.3  95.0  18.1  93.7  12.1 <0.01 
15. Doctor(s) registered for
 Child health surveillance/
 Minor surgery/
 Maternity care  2.0  0.0  1.9  0.4  1.6  0.7  1.7  0.7 <0.01 
16. Staff costs (£ per 1000 patients) 12 765 2214 12 469 5197 13 821 4050 12 568 2032 0.25 
Table 3

Minimum, maximum and mean NHSPPI scores by region

 Oxford 
 (n = 25) Lothian
 (n = 23) West London
 (n = 35) Coventry
 (n = 34) Pa All 
a Gives the result of a Kruskal–Wallis test, with the data classified by area. The null hypothesis is one of equal median values of NHSPPI across all areas. 
Performance index summary value
 (all indicators included)
     Minimum/maximum
     Mean (SD) 17/31
 24.8 (2.9) 15/29
 23.0 (3.7) 11/26
 20.1 (4.0) 12/27
 19.9 (4.4) <0.001 11/31
 21.6 (4.3) 
Performance index summary value
 (cough suppressants excluded)
     Minimum/maximum
     Mean (SD) 15/29
 22.8 (2.9) 13/27
 21.2 (3.6) 11/24
 19.2 (3.6) 12/25
 19.3 (3.7) <0.001 11/29
 20.4 (3.8) 
Performance index summary value
 (diuretics excluded)
     Minimum/maximum
     Mean (SD) 18/29
 23.1 (2.7) 13/27
 21.3 (3.5) 11/25
 19.3 (3.8) 11/26
 19.3 (4.1) <0.001 11/29
 20.5 (3.9) 
Performance index summary value
 (cough suppressants and diuretics excluded)
     Minimum/maximum
     Mean (SD) 16/27
 21.1 (2.7) 11/25
 19.5 (3.4) 11/25
 18.5 (3.4) 12/24
 18.7 (3.4) 0.03 11/27
 19.3 (3.4) 
 Oxford 
 (n = 25) Lothian
 (n = 23) West London
 (n = 35) Coventry
 (n = 34) Pa All 
a Gives the result of a Kruskal–Wallis test, with the data classified by area. The null hypothesis is one of equal median values of NHSPPI across all areas. 
Performance index summary value
 (all indicators included)
     Minimum/maximum
     Mean (SD) 17/31
 24.8 (2.9) 15/29
 23.0 (3.7) 11/26
 20.1 (4.0) 12/27
 19.9 (4.4) <0.001 11/31
 21.6 (4.3) 
Performance index summary value
 (cough suppressants excluded)
     Minimum/maximum
     Mean (SD) 15/29
 22.8 (2.9) 13/27
 21.2 (3.6) 11/24
 19.2 (3.6) 12/25
 19.3 (3.7) <0.001 11/29
 20.4 (3.8) 
Performance index summary value
 (diuretics excluded)
     Minimum/maximum
     Mean (SD) 18/29
 23.1 (2.7) 13/27
 21.3 (3.5) 11/25
 19.3 (3.8) 11/26
 19.3 (4.1) <0.001 11/29
 20.5 (3.9) 
Performance index summary value
 (cough suppressants and diuretics excluded)
     Minimum/maximum
     Mean (SD) 16/27
 21.1 (2.7) 11/25
 19.5 (3.4) 11/25
 18.5 (3.4) 12/24
 18.7 (3.4) 0.03 11/27
 19.3 (3.4) 
Table 4

Spearman correlation coefficients of NHS Practice Performance Index (NHSPPI), list size, deprivation and the percentage of patients who speak a language other than English in participating practices

 List size Weighted
 deprivation
 index Other
 language
 patients 
NHSPPI 0.25
 (0.07) –0.57
 (0.001) –0.44
 (0.001) 
List size  –0.36
 (0.01) –0.41
 (0.002) 
Weighted deprivation
 index   –0.62
 (0.001) 
 List size Weighted
 deprivation
 index Other
 language
 patients 
NHSPPI 0.25
 (0.07) –0.57
 (0.001) –0.44
 (0.001) 
List size  –0.36
 (0.01) –0.41
 (0.002) 
Weighted deprivation
 index   –0.62
 (0.001) 
Table 5

Rank correlations between NHS Practice Performance Index (NHSPPI) and its components and Consultation Quality Index and its components by region

Measure na Rank correlation with practice level b 
  Mean
 PEI score Mean
 consultation length Mean
 ‘know the
 doctor' score Mean
 CQI
 scorec 
a Data are restricted to practices which returned at least 50 valid enablement scores for English-speaking adult patients. 
b The practice-level values for mean PEI score, mean consultation length and mean ‘know the doctor' score are derived from consultations involving English-speaking adult patients only. 
c Computed at practice level as the mean of the CQI scores of the individual doctors working in the practice. 
Aggregated NHSPPI: participating practices in Oxford 12 NS 0.58 (P = 0.05) NS 0.66 (P = 0.02) 
Practice prescribing subindex (PPSI): participating practices in Oxford 12 NS 0.64 (P = 0.03) NS NS 
Practice preventive subindex (PRSI): participating practices in Oxford 12 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Oxford 12 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in Lothian 13 NS NS NS NS 
Practice prescribing subindex (PPSI): participating practices in Lothian 14 NS NS NS NS 
Practice preventive subindex (PRSI): participating practices in Lothian 14 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Lothian 13 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in West London 14 NS NS NS NS 
Practice prescribing subindex (PPSI): participating practices in West London 14 NS NS NS NS 
Practice preventive subindex (PRSI): participating practices in West London 14 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in West London 14 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in Coventry NS 0.66 (P = 0.05) NS NS 
Practice prescribing subindex (PPSI): participating practices in Coventry NS 0.66 (P = 0.05) NS NS 
Practice preventive subindex (PRSI): participating practices in Coventry NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Coventry NS NS NS NS 
      
Aggregated NHSPPI: all participating practices 48 NS 0.31 (P = 0.03) NS NS 
Practice prescribing subindex (PPSI): all participating practices 49 NS 0.36 (P = 0.01) –0.33 (P = 0.02) NS 
Practice preventive subindex (PRSI): all participating practices 49 NS NS NS NS 
Practice external quality subindex (PQSI): all participating practices 48 NS NS NS NS 
Measure na Rank correlation with practice level b 
  Mean
 PEI score Mean
 consultation length Mean
 ‘know the
 doctor' score Mean
 CQI
 scorec 
a Data are restricted to practices which returned at least 50 valid enablement scores for English-speaking adult patients. 
b The practice-level values for mean PEI score, mean consultation length and mean ‘know the doctor' score are derived from consultations involving English-speaking adult patients only. 
c Computed at practice level as the mean of the CQI scores of the individual doctors working in the practice. 
Aggregated NHSPPI: participating practices in Oxford 12 NS 0.58 (P = 0.05) NS 0.66 (P = 0.02) 
Practice prescribing subindex (PPSI): participating practices in Oxford 12 NS 0.64 (P = 0.03) NS NS 
Practice preventive subindex (PRSI): participating practices in Oxford 12 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Oxford 12 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in Lothian 13 NS NS NS NS 
Practice prescribing subindex (PPSI): participating practices in Lothian 14 NS NS NS NS 
Practice preventive subindex (PRSI): participating practices in Lothian 14 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Lothian 13 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in West London 14 NS NS NS NS 
Practice prescribing subindex (PPSI): participating practices in West London 14 NS NS NS NS 
Practice preventive subindex (PRSI): participating practices in West London 14 NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in West London 14 NS NS NS NS 
      
Aggregated NHSPPI: participating practices in Coventry NS 0.66 (P = 0.05) NS NS 
Practice prescribing subindex (PPSI): participating practices in Coventry NS 0.66 (P = 0.05) NS NS 
Practice preventive subindex (PRSI): participating practices in Coventry NS NS NS NS 
Practice external quality subindex (PQSI): participating practices in Coventry NS NS NS NS 
      
Aggregated NHSPPI: all participating practices 48 NS 0.31 (P = 0.03) NS NS 
Practice prescribing subindex (PPSI): all participating practices 49 NS 0.36 (P = 0.01) –0.33 (P = 0.02) NS 
Practice preventive subindex (PRSI): all participating practices 49 NS NS NS NS 
Practice external quality subindex (PQSI): all participating practices 48 NS NS NS NS 

We thank Dr R Elton (University of Edinburgh), Ms H Rai and Dr M Pierce (Imperial College School of Medicine). We also wish to acknowledge our indebtedness to the 221 doctors and 53 practices that participated, together with their managers and reception staff. The work was supported by grants from: the Chief Scientist's Office at the Scottish Office Home and Health Department; Anglia and Oxford NHS R & D Directorate; West Midlands NHS R & D Directorate; and North Thames NHS R & D Directorate.

References

1
Secretary of State for Health. The New NHS: Modern and Dependable. London: HMSO, 1997.
2
Secretary of State for Health for Scotland. Designed to Care: Renewing the National Health Service in Scotland. Edinburgh: Stationery Office, 1997.
3
Donaldson LJ, Muir Gray JA. Clinical governance: a quality duty for health organisations.
Qual Health Care
 
1998
;
7
(Suppl):
S37
–S44.
4
Roland M, Holden J, Campbell S. Quality Assessment for General Practice: Supporting Clinical Governance in Primary Care Groups. Manchester: University of Manchester, National Primary Care Research and Development Centre, 1998.
5
Wilkin D, Hallam L, Doggett M. Measures of Need and Outcome for Primary Health Care. Oxford: Oxford Medical Publications, 1992.
6
Toon PD. What is Good General Practice? Occasional Paper 65. London: Royal College of General Practitioners, 1994.
7
Baker R, Streatfield J. What types of general practice do patients prefer? Exploration of practice characteristics influencing satisfaction.
Br J Gen Pract
 
1995
;
45
:
654
–659.
8
Majeed F, Voss S. Performance indicators for general practice.
Br Med J
 
1995
;
311
:
209
–210.
9
Howie JGR, Heaney DJ, Maxwell M. Measuring Quality in General Practice. Occasional Paper 75. London: Royal College of General Practitioners, 1997.
10
Lawrence M, Olesen F. Indicators of quality in health care.
Eur J Gen Pract
 
1997
;
3
:
103
–108.
11
Safran DG, Tiara DA, Rogers WH. Linking primary care performance to outcomes of care.
J Fam Pract
 
1998
;
47
:
13
–220.
12
NHS Executive. The New NHS; Modern and Dependable: A National Framework for Assessing Performance. Consultation Document. London: NHS Executive, 1998.
13
Campbell SM, Roland MO, Quale JA, Buetow SA. Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they?
J Publ Health Med
 
1998
;
20
:
414
–421.
14
Stott N, Kinnersley P, Jones Elwyn G. Measuring general practice-based primary care: generic outcomes.
Fam Pract
 
1997
;
14
:
486
–491.
15
Howie JGR, Heaney DJ, Maxwell M, Walker JJ, Freeman GK. Developing a ‘consultation quality index' (CQI) for use in general practice.
Fam Pract
 
2000
;
17
:
455
–461.
16
Howie JGR, Heaney DJ, Maxwell M, Walker JJ, Freeman GK, Rai H. Quality at general practice consultations: cross sectional survey.
Br Med J
 
1999
;
319
:
738
–743.
17
Campbell S, Cantril. Assessing the validity of prescribing indicators for cost and quality.
Br Med J
 
2000
;
321
:
425
–429.