Abstract

Objective

Inconsistent performance measurement schemes hinder attempts to make international comparisons about mental health-care quality. This report describes a project undertaken by an international collaborative group that aims to develop a common framework of measures that will allow for international comparisons of mental health system performance.

Design

Representatives from each country submitted reports of quality measurement initiatives in mental health. Indicators were reviewed, and all measurable indicators were compiled and organized.

Sample

Twenty-nine programs from 11 countries and two cross-national programs submitted reports.

Methods

Indicators were evaluated according to measurable inclusion criteria.

Results

These methods yielded 656 total measures that were organized into 17 domains and 80 subdomains.

Conclusions

No single program contained indicators in all domains, highlighting the need for a comprehensive, shared scheme for international measurement. By collecting and organizing measures through an inductive compilation of existing programs, the present study has generated a maximally inclusive basis for the creation of a common framework of international mental health quality indicators.

Introduction

A report from the US Institute of Medicine described a ‘chasm’ that divides potentially effective, evidence-based health-care practices from the actual care that consumers receive [1], and the need to improve the quality of mental health care is no less urgent, especially considering the large global burdens of mental illness [2] and the relatively high prevalence rates of diagnostic and statistical manual of mental disorders diagnoses (from 12% to over 45% of national populations) [3]. There are significant challenges to improving the quality of medical care. Some commentators have expressed concern about the evidence base supporting quality measures and the existing infrastructure for developing such measures [4]. Other challenges to quality improvement include the amount of mental health services delivered outside of formal medical settings (e.g. the criminal justice system) and the relatively slow adoption of information technology in the mental health sector [5].

Accordingly, many countries and international organizations have attempted to develop formal indicator schemes to measure quality in mental health care, with varying degrees of consensus. Researchers began developing conceptual frameworks for quality measurement as early as 30 years ago, and organizations including the US National Committee on Quality Assurance, the Institute of Medicine and the RAND Corporation have refined several iterations of criteria for selecting quality measures [6]. Recent larger efforts have led to inconsistent results. An expert panel of the Organization for Economic Cooperation and Development (OECD) identified 12 indicators across four main domains: (i) continuity of care, (ii) coordination of care, (iii) treatment and (iv) patient outcomes [7] while a similar European Union initiative developed a different set of 32 indicators across four broad domains: (i) demographic and socioeconomic factors, (ii) health status, (iii) determinants of health and (iv) health systems [8]. We [9] recently reviewed US initiatives, and while we identified several shared foci, we also found a lack of coordination that led to significant gaps in both indicator development and data reporting. Similarly, in another review of 55 literature reports from 12 countries [10], we found that no one program covered every measurement domain.

This inconsistency in performance measurement schemes points toward an overarching methodological need. Although individual quality improvement efforts continue to be refined, broader problems still remain, as indicators and their organizing structures are often too heterogeneous to be compared with other schemes. Sometimes, the proposed indicators themselves are not measurable in practice (i.e. the measure cannot be quantified at all).

Analogous efforts from other branches of medicine provide informative methodological examples for developing quality indicators sets. In particular, a 2006 OECD project collected existing quality indicators from around the world, judged them for importance, scientific soundness and feasibility, and mapped them to an organizing framework they developed to categorize those indicators [11].

This paper describes an initial component of a project to develop a set of mental health quality indicators that would be a reasonably comprehensive and compatible scheme for international comparisons. In 2008, clinical experts from 12 countries (Australia, Canada, England, Germany, Ireland, Japan, the Netherlands, New Zealand, Norway, Scotland, Taiwan and the USA), meeting as part of the Clinical Leaders group of the International Initiative for Mental Health Leadership (IIMHL), initiated this project to develop a consensus framework for reporting on mental health-care quality. In the recently concluded first phase of this project, representatives reported a wide range of mental health performance and outcome measures from their national performance measurement programs. We then collected and analyzed these measures in an effort to identify and organize the most commonly used indicators and domains. The ultimate aim of the IIMHL is to develop and implement a balanced, inclusive and common framework of measures that will allow for international comparisons of system performance, with a long-term goal of informing initiatives that will improve mental health services in these countries.

Method

Representatives from each of the participating countries were contacted to identify peer-reviewed journal articles, government reports, white papers and other ‘gray literature’ on population-based quality or performance measurement initiatives in mental health being developed or implemented in each country at the national or other representative level (e.g. province and state, etc.). Initiatives were extracted into a standard document listing context, mental health indicators and original domains, process of indicator development, intended or actual use and related studies and reports. This initial phase of the overall IIMHL project aimed to provide an overview and assessment of the broader context of mental health quality measurement in each participating country [10].

In a second step, performance measures, performance indicators and outcome measures that met the following three main criteria were collated into a single document and determined for inclusion in the current review: Indicators selected and included based on these criteria represent an inventory of measures currently used in participating countries without validating the measures' clinical importance, validity or feasibility. The measures were then organized and assigned to a list of 16 domains and 77 subdomains initially developed for an international survey. These domains and subdomains were adopted from the National Inventory of Mental Health Quality Measures established by the Center for Quality Assessment and Improvement in Mental Health and recommendations from the IIMHL country experts [12]. Researchers at Columbia University reviewed the list of indicators and identified indicators that were unclear or were classified differently to develop consensus on the classification. Discrepancies were resolved by discussion among the three lead authors to develop consensus on the evaluation and to iteratively modify the framework of domains and subdomains to better reflect the range and purpose of the measures collected. Coding was continued until 100% consensus was achieved. This process reflects the ongoing iterative work to establish an overall indicator framework that will ultimately allow for international benchmarking and comparison of mental health quality indicators. Finally, based on this list of indicators and domains, we identified common (and differing) themes, methods and definitions that are discussed below.

  1. The initiative must describe indicators related to mental health and/or substance use.

  2. The indicators should (i) be precisely defined at the numerator and denominator level, (ii) contain information about data sources and (iii) measure quality (as defined by the six US Institute of Medicine domains of effectiveness, efficiency, equitability, safety, timeliness and patient and/or community centered).

  3. The indicators should have a national- or regional-level focus, or otherwise be used to assess the performance among organizations or providers.

Results

Indicators from 31 programs in 11 countries and two cross-national programs were compiled, yielding 656 total measures. The final framework comprised 17 domains and 80 subdomains. Each indicator specified an objective numerator and denominator drawn from an identifiable dataset—as required by the selection criteria noted above.

Numbers of indicators by country or organization are given in Table 1. There was considerable variability in the numbers and types of indicators, programs and datasets. The numbers of total measurable indicators per country ranged from 3 to 211, which is explained only in part by the range of 1–7 programs per country. Supplementary Table S1 (online) gives lists of the programs from which the indicators were drawn, demonstrating the wide scope of the data sources used. Sources of data included, for example, governmental bureaus, state/territory administrations, surveys, protocols, censuses, standardized patient measures, budget reports, insurance records, patient charts and physician reports.

Table 1

Indicators by country/organization

Country/organization (n = 13)Programs (n = 31)Indicators (n = 656)Domains (n = 17)
Australia 46 
Canada 101 12 
Denmark 19 
England 88 15 
Germany 30 
Japan 
Netherlands 33 
New Zealand 65 10 
Norway 
Scotland 14 
USA 211 15 
OECD 12 
EU 27 
Country/organization (n = 13)Programs (n = 31)Indicators (n = 656)Domains (n = 17)
Australia 46 
Canada 101 12 
Denmark 19 
England 88 15 
Germany 30 
Japan 
Netherlands 33 
New Zealand 65 10 
Norway 
Scotland 14 
USA 211 15 
OECD 12 
EU 27 

The gray literature review did not yield any indicators for Ireland and Taiwan that met the inclusion criteria.

Denmark, although not officially participating in the IIMHL Clinical Leads Project, provided information that was included in the study.

Table 1

Indicators by country/organization

Country/organization (n = 13)Programs (n = 31)Indicators (n = 656)Domains (n = 17)
Australia 46 
Canada 101 12 
Denmark 19 
England 88 15 
Germany 30 
Japan 
Netherlands 33 
New Zealand 65 10 
Norway 
Scotland 14 
USA 211 15 
OECD 12 
EU 27 
Country/organization (n = 13)Programs (n = 31)Indicators (n = 656)Domains (n = 17)
Australia 46 
Canada 101 12 
Denmark 19 
England 88 15 
Germany 30 
Japan 
Netherlands 33 
New Zealand 65 10 
Norway 
Scotland 14 
USA 211 15 
OECD 12 
EU 27 

The gray literature review did not yield any indicators for Ireland and Taiwan that met the inclusion criteria.

Denmark, although not officially participating in the IIMHL Clinical Leads Project, provided information that was included in the study.

Counts of indicators by domain are reported in Table 2 that also lists the subdomains into which redundant or similar indicators were combined. After condensing the indicators in this way, 80 total subdomains were identified. Counts for individual subdomains are also listed in Table 2. The numbers of indicators per domain and subdomain were also highly variable, and we found broad types of measurable indicators within each domain. For example, both adequate medication dosage and medication reconciliation are found in the Evidenced-Based Pharmacotherapy domain; the former measures compliance to clinical standards, whereas the latter is a system-level practice. Similarly, the Outcome Assessment domain (which contains 72 total indicators, the greatest number) contains subdomains, including examples as diverse as mortality rates, employment status and the Global Assessment of Functioning scale. (Examples of each subdomain are listed in the online Supplementary Table S2 for illustrative purposes.)

Table 2

Domains and subdomains (total = 656)a

DomainIndicatorsSubdomainsIndicators/subdomain
Symptom or diagnostic assessment 61 Schizophrenia or other psychotic illness 
Bipolar or depressive disorders 17 
Anxiety disorder 
Suicide risk 
Other 30 
Evidence-based pharmacotherapy 53 Selection of medications 14 
Adequate medication dosage 
Medication adherence 
Polypharmacy 
Occurrence of side effects 
Monitoring 
Medication reconciliation 
Other 13 
Evidence-based psychosocial interventions 57 Psychotherapy 11 
Assertive community treatment 
Case management 
Employment support or assistance 
Family psychoeducation 
Early intervention programs 
Other 18 
Other somatic interventions Electroconvulsive therapy 
Other 
Substance use 37 Assessment/screening 10 
 Quantity/frequency of use 
 Engagement in care 
 Pharmacologic treatment 
 Blood/urine monitoring 
 Outcomes 
 Psychosomatic treatment 
 Coordination with substance abuse treatment 
 Access to/wait times for substance abuse treatment 
 Utilization of substance abuse treatment 
 Integrated dual diagnosis treatment 
 Other 
General medical care Preventive medical care or screening 
Chronic illness medical care 
Other 
Continuity and coordination of care 67 Inpatient discharge planning 
Outpatient follow-up after inpatient discharge 19 
Coordination with outpatient mental health 11 
Coordination with primary care 
Inpatient readmission 15 
Other 18 
Access measures 50 Access to primary care 
Access to emergency mental health care 
Access to social services, housing and foster care 
Access to/wait times for outpatient service 14 
Other 23 
Utilization, cost and efficiency 40 Utilization of outpatient services 
Duration of hospitalization 14 
Other 18 
Patient safety 48 Use of seclusion/restraint 23 
Medication errors or adverse events 
Elopement/drop out 
Non-medication adverse events 
Falls/injuries 
Other 16 
Forensic or legal issues 13 Criminal justice encounters 
Involuntary or compulsory hospitalization 
Involuntary or compulsory community treatment 
Other 
Outcome assessment 72 Change in reported symptoms 
Functioning 23 
General health status 
Mortality 17 
Employment or income 
Housing 
Client or family self-assessment 
Other 13 
Recovery 32 Access to peer or consumer services 
Recovery environment 
Shared decision making 20 
Other 
Cultural or ethnic issues Racial or ethnic disparities in care 
Training in cultural competency 
Access to culturally specific care 
Other 
Population-based resources 50 Total expenditure for mental health services/population 
Mental health workforce/ population 20 
Other 21 
Client/family perceptions of care 18 Not subdivided  
Other types of measures or domains 38 Not subdivided  
DomainIndicatorsSubdomainsIndicators/subdomain
Symptom or diagnostic assessment 61 Schizophrenia or other psychotic illness 
Bipolar or depressive disorders 17 
Anxiety disorder 
Suicide risk 
Other 30 
Evidence-based pharmacotherapy 53 Selection of medications 14 
Adequate medication dosage 
Medication adherence 
Polypharmacy 
Occurrence of side effects 
Monitoring 
Medication reconciliation 
Other 13 
Evidence-based psychosocial interventions 57 Psychotherapy 11 
Assertive community treatment 
Case management 
Employment support or assistance 
Family psychoeducation 
Early intervention programs 
Other 18 
Other somatic interventions Electroconvulsive therapy 
Other 
Substance use 37 Assessment/screening 10 
 Quantity/frequency of use 
 Engagement in care 
 Pharmacologic treatment 
 Blood/urine monitoring 
 Outcomes 
 Psychosomatic treatment 
 Coordination with substance abuse treatment 
 Access to/wait times for substance abuse treatment 
 Utilization of substance abuse treatment 
 Integrated dual diagnosis treatment 
 Other 
General medical care Preventive medical care or screening 
Chronic illness medical care 
Other 
Continuity and coordination of care 67 Inpatient discharge planning 
Outpatient follow-up after inpatient discharge 19 
Coordination with outpatient mental health 11 
Coordination with primary care 
Inpatient readmission 15 
Other 18 
Access measures 50 Access to primary care 
Access to emergency mental health care 
Access to social services, housing and foster care 
Access to/wait times for outpatient service 14 
Other 23 
Utilization, cost and efficiency 40 Utilization of outpatient services 
Duration of hospitalization 14 
Other 18 
Patient safety 48 Use of seclusion/restraint 23 
Medication errors or adverse events 
Elopement/drop out 
Non-medication adverse events 
Falls/injuries 
Other 16 
Forensic or legal issues 13 Criminal justice encounters 
Involuntary or compulsory hospitalization 
Involuntary or compulsory community treatment 
Other 
Outcome assessment 72 Change in reported symptoms 
Functioning 23 
General health status 
Mortality 17 
Employment or income 
Housing 
Client or family self-assessment 
Other 13 
Recovery 32 Access to peer or consumer services 
Recovery environment 
Shared decision making 20 
Other 
Cultural or ethnic issues Racial or ethnic disparities in care 
Training in cultural competency 
Access to culturally specific care 
Other 
Population-based resources 50 Total expenditure for mental health services/population 
Mental health workforce/ population 20 
Other 21 
Client/family perceptions of care 18 Not subdivided  
Other types of measures or domains 38 Not subdivided  

aThe list of 16 domains and 77 subdomains was adopted from the National Inventory of Mental Health Quality Measures established by the Center for Quality Assessment and Improvement in Mental Health and recommendations from IIMHL country experts [12].

Table 2

Domains and subdomains (total = 656)a

DomainIndicatorsSubdomainsIndicators/subdomain
Symptom or diagnostic assessment 61 Schizophrenia or other psychotic illness 
Bipolar or depressive disorders 17 
Anxiety disorder 
Suicide risk 
Other 30 
Evidence-based pharmacotherapy 53 Selection of medications 14 
Adequate medication dosage 
Medication adherence 
Polypharmacy 
Occurrence of side effects 
Monitoring 
Medication reconciliation 
Other 13 
Evidence-based psychosocial interventions 57 Psychotherapy 11 
Assertive community treatment 
Case management 
Employment support or assistance 
Family psychoeducation 
Early intervention programs 
Other 18 
Other somatic interventions Electroconvulsive therapy 
Other 
Substance use 37 Assessment/screening 10 
 Quantity/frequency of use 
 Engagement in care 
 Pharmacologic treatment 
 Blood/urine monitoring 
 Outcomes 
 Psychosomatic treatment 
 Coordination with substance abuse treatment 
 Access to/wait times for substance abuse treatment 
 Utilization of substance abuse treatment 
 Integrated dual diagnosis treatment 
 Other 
General medical care Preventive medical care or screening 
Chronic illness medical care 
Other 
Continuity and coordination of care 67 Inpatient discharge planning 
Outpatient follow-up after inpatient discharge 19 
Coordination with outpatient mental health 11 
Coordination with primary care 
Inpatient readmission 15 
Other 18 
Access measures 50 Access to primary care 
Access to emergency mental health care 
Access to social services, housing and foster care 
Access to/wait times for outpatient service 14 
Other 23 
Utilization, cost and efficiency 40 Utilization of outpatient services 
Duration of hospitalization 14 
Other 18 
Patient safety 48 Use of seclusion/restraint 23 
Medication errors or adverse events 
Elopement/drop out 
Non-medication adverse events 
Falls/injuries 
Other 16 
Forensic or legal issues 13 Criminal justice encounters 
Involuntary or compulsory hospitalization 
Involuntary or compulsory community treatment 
Other 
Outcome assessment 72 Change in reported symptoms 
Functioning 23 
General health status 
Mortality 17 
Employment or income 
Housing 
Client or family self-assessment 
Other 13 
Recovery 32 Access to peer or consumer services 
Recovery environment 
Shared decision making 20 
Other 
Cultural or ethnic issues Racial or ethnic disparities in care 
Training in cultural competency 
Access to culturally specific care 
Other 
Population-based resources 50 Total expenditure for mental health services/population 
Mental health workforce/ population 20 
Other 21 
Client/family perceptions of care 18 Not subdivided  
Other types of measures or domains 38 Not subdivided  
DomainIndicatorsSubdomainsIndicators/subdomain
Symptom or diagnostic assessment 61 Schizophrenia or other psychotic illness 
Bipolar or depressive disorders 17 
Anxiety disorder 
Suicide risk 
Other 30 
Evidence-based pharmacotherapy 53 Selection of medications 14 
Adequate medication dosage 
Medication adherence 
Polypharmacy 
Occurrence of side effects 
Monitoring 
Medication reconciliation 
Other 13 
Evidence-based psychosocial interventions 57 Psychotherapy 11 
Assertive community treatment 
Case management 
Employment support or assistance 
Family psychoeducation 
Early intervention programs 
Other 18 
Other somatic interventions Electroconvulsive therapy 
Other 
Substance use 37 Assessment/screening 10 
 Quantity/frequency of use 
 Engagement in care 
 Pharmacologic treatment 
 Blood/urine monitoring 
 Outcomes 
 Psychosomatic treatment 
 Coordination with substance abuse treatment 
 Access to/wait times for substance abuse treatment 
 Utilization of substance abuse treatment 
 Integrated dual diagnosis treatment 
 Other 
General medical care Preventive medical care or screening 
Chronic illness medical care 
Other 
Continuity and coordination of care 67 Inpatient discharge planning 
Outpatient follow-up after inpatient discharge 19 
Coordination with outpatient mental health 11 
Coordination with primary care 
Inpatient readmission 15 
Other 18 
Access measures 50 Access to primary care 
Access to emergency mental health care 
Access to social services, housing and foster care 
Access to/wait times for outpatient service 14 
Other 23 
Utilization, cost and efficiency 40 Utilization of outpatient services 
Duration of hospitalization 14 
Other 18 
Patient safety 48 Use of seclusion/restraint 23 
Medication errors or adverse events 
Elopement/drop out 
Non-medication adverse events 
Falls/injuries 
Other 16 
Forensic or legal issues 13 Criminal justice encounters 
Involuntary or compulsory hospitalization 
Involuntary or compulsory community treatment 
Other 
Outcome assessment 72 Change in reported symptoms 
Functioning 23 
General health status 
Mortality 17 
Employment or income 
Housing 
Client or family self-assessment 
Other 13 
Recovery 32 Access to peer or consumer services 
Recovery environment 
Shared decision making 20 
Other 
Cultural or ethnic issues Racial or ethnic disparities in care 
Training in cultural competency 
Access to culturally specific care 
Other 
Population-based resources 50 Total expenditure for mental health services/population 
Mental health workforce/ population 20 
Other 21 
Client/family perceptions of care 18 Not subdivided  
Other types of measures or domains 38 Not subdivided  

aThe list of 16 domains and 77 subdomains was adopted from the National Inventory of Mental Health Quality Measures established by the Center for Quality Assessment and Improvement in Mental Health and recommendations from IIMHL country experts [12].

Discussion

Several aspects of this analysis provide important insights into the state of quality measurement worldwide, and the provisional organization of these data represents a meaningful step toward developing an international framework for mental health quality indicators. No single program contained indicators in all domains, further emphasizing the need for a comprehensive, shared scheme for international measurement. This finding echoes the results of two recent literature reviews arising from the IIMHL project, which reviewed numerous US [9] and international [10] mental health quality measurement programs, but still identified significant gaps in the scope of every program.

A significant methodological contribution of this effort is its inductive approach to organize mental health measures. By deriving categories and examples of quality indicator measures through a compilation of the existing programs, this stage of the IIMHL project has generated an organized list of measures that is maximally broad in scope, thus allowing future efforts to proceed from the most inclusive basis possible.

Quality indicators have complicated origins, as each guideline for measurement is a product of the goals, context and time from which it came. The priorities and influences of these performance measurement programs will vary under the influence of broader cultural factors, and the framers, providers and consumers coming from a given culture or health-care system may not be fully aware of how their context impacts the ways they measure quality. One important motivation of a structured framework approach such as this, derived from a comprehensive inventory of international measures, is to attempt to avoid these potential biases by providing an inclusive global perspective on the state of quality measurement.

The relative frequencies of indicators per domain may give a sense of how different measures have been prioritized or an indication of which measures are actually measurable. For example, despite the recent popularity of the recovery movement in mental health [13], the recovery domain in this sample has among the fewest measurable indicators. It is important to note, however, that no strong conclusions can be drawn from these counts. In many cases, these frequencies may simply reflect the feasibility of measurement: outcome assessment and symptom assessment are likely easier to quantify than culture issues or perceptions of care. It is also important to note that high numbers of indicators in particular domains do not necessarily indicate greater coverage of topics within those domains. In many cases, different indicators were capturing quite similar measurement concepts with different approaches or minor variants in numerator or denominator definition (e.g. measuring medication adherence or access/wait times to outpatient services). That said, on a more straightforward level, these frequencies do give an important sense of which areas are in need of further quality measure development.

There are several limitations to this study. The member countries of the IIMHL Clinical Leaders Group are necessarily a ‘convenience sample’, and participation is limited to countries that already have some infrastructure and resources in place to measure mental health quality indicators. Language issues may have hampered the collection of indicators from countries, for example, translations were done by the IIMHL partners who provided the indicators themselves, creating a somewhat onerous step for participating members. More broadly, the first step of obtaining measures from representatives in peer countries depended on those representatives for thoroughness and rigor, and while they were asked to submit a broad collection of measures, individuals may have gone about their selection processes differently. In terms of the organization of the indicators, the system of domains that was created is inherently subjective and rests in part on a priori assumptions about the organization of quality measures. That said, this provisional categorization was not the primary goal of this effort, and the study methodology allowed for flexible adjustments to these categories as the data were analyzed. In addition, it should be noted that these measures have not been subjected to a validation process. In fact, a key component of a research agenda for quality improvement in mental health is evaluation of the reliability, validity and utility of quality indicators. Obviously, evaluating 600+ measures of mental health quality is unrealistic. Ultimately, each country needs to determine its own priorities, amidst the reality of what is feasible, given available data and resources. It is likely that national programs of quality measurement will develop ways to create composite measures or dashboards to compress the information in ways that can be directly acted upon to improve quality.

Several of the next key steps of the Clinical Leaders project in Phase II will proceed from these data. Now that the performance indicators have been collected and grouped, the IIMHL Clinical Leaders Group will rate this sample for validity, importance and feasibility, with a goal to propose a consensus list of mental health quality indicators that could be collected by each of the participating countries. At the same time, country networks are forming to further discuss data infrastructures and explore implementation strategies that will allow for the comparison of performance measures across countries in the future. The ultimate goal is to enable quality measurement initiatives that will help to transform mental health services worldwide.

Supplementary material

Supplementary material is available at International Journal for Quality in Health Care Journal online.

Funding

This work was supported by: the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services; the Irving Institute for Clinical and Translational Research, Columbia University [grant number UL1 RR024156]; the National Center for Research Resources, a component of the National Institutes of Health; the Mental Health Therapeutics CERT at Rutgers, the State University of New Jersey, funded by the Agency for Health Care Research and Quality [AHRQ, grant number 5 U18 HS016097]. Additional funding came from government and non-government organizations of the countries participating in the IIMHL project (Australia, Canada, England, Germany, Ireland, Japan, Netherlands, New Zealand, Norway, Scotland, Taiwan and the USA).

Acknowledgements

Thanks to the members of the IIMHL Clinical Leaders Project. This article represents the collaborative effort of a group of researchers, clinicians and government officials across the participating countries. The views expressed in this article are those of the authors.

References

1
Pincus
HA
Page
AEK
Druss
B
, et al. 
Can psychiatry cross the quality chasm? Improving the quality of health care for mental and substance use conditions
Am J Psychiatry
2007
, vol. 
164
 (pg. 
712
-
9
)
2
World Health Organization (WHO)
The Global Burden of Disease: 2004 Update
2008
WHO
 
3
Kessler
RC
Angermeyer
M
Anthony
JC
, et al. 
Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative
World Psychiatry
2007
, vol. 
3
 (pg. 
168
-
76
)
4
U.S. Institute of Medicine (IOM)
Crossing the Quality Chasm: A New Health System for the 21st Century
2001
Washington, DC: National Academies Press
5
U.S. Institute of Medicine (IOM)
Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders. Improving the Quality of Health Care for Mental and Substance-Use Conditions
2006
Washington, DC: National Academies Press
6
Kilbourne
AM
Keyser
D
Pincus
HA
Challenges and opportunities in measuring the quality of mental health care
Can J Psychiatry
2010
, vol. 
55
 (pg. 
549
-
57
)
7
Organization for Economic Co-operation and Development (OECD)
Information availability for measuring and comparing quality of mental health care across OECD countries
2004
OECD
 
Health Technical Papers No. 20, www.oecd.org/dataoecd/53/47/41243838.pdf (28 July 2011, date last accessed)
8
National Institute for Health and Welfare (STAKES)
Establishment of the Indicators for Mental Health Monitoring in Europe
2006
STAKES
 
9
Herbstman
BJ
Pincus
HA
Measuring mental health care quality in the United States: a review of initiatives
Curr Opin Psychiatry
2009
, vol. 
22
 (pg. 
623
-
30
)
10
Spaeth-Rublee
B
Pincus
HA
Huynh
PT
, et al. 
Mental Health Quality Indicator Project. Measuring quality of mental health care: a review of initiatives and programs in selected countries
Can J Psychiatry
2010
, vol. 
55
 (pg. 
539
-
48
)
11
Marshall
M
Klazinga
N
Leatherman
S
, et al. 
OECD Health Care Quality Indicators Project: the expert panel on primary care prevention and health promotion
Int J Qual Health Care
2006
, vol. 
18
 
Suppl 1
(pg. 
21
-
5
)
12
Hermann
RC
Improving Mental Healthcare: A Guide to Measurement-based Quality Improvement
2005
Washington, DC
American Psychiatric Publishing
13
Hogan
MF
The President's New Freedom Commission: recommendations to transform mental health care in america
Psychiatr Serv
2003
, vol. 
54
 (pg. 
1467
-
74
)

Author notes

Members of the IIMHL Clinical Leaders Project are Peggy Brown, Alan Rosen (Australia), Janet Durbin, David Goldbloom, Elizabeth Lin, Patricica Wiebe (Canada), Hugh Griffiths (England), Wolfgang Gaebel (Germany), Catherine Brogan, Martin Rogan, Ian Daly (Ireland), Hiroto Ito (Japan), Paul Spronken, Jan Tromp, George Witte, Jos de Beer (Netherlands), David Chaplow, Peter McGeorge, Fran Silvestri (New Zealand), Torleif Ruud (Norway), Denise Coia (Scotland), Joseph J. Cheng (Taiwan), Neal Adams, Anita Everett, Joe Parks, Peter Stuart, Harold Pincus, Ken Thompson and Christopher Carroll (USA).

Supplementary data