Results of the first 100 accreditation procedures in France

Abstract

Purpose. To identify the areas of needed improvement that were most frequently identified in the first 100 accredited hospitals by the French Accreditation College (FAC) according to the standards manual. To compare the outcomes of accreditation procedures according to the status and size of the accredited hospitals.

Data sources. We analyzed the first 100 summaries of accreditation reports available on the website of the Agence Nationale d’Accréditation et d’Evaluation en Santé (ANAES).

Data extraction. Data were collected on hospitals, accreditation processes, and outcomes (decisions of the FAC). For each decision, we assessed the relationship with accreditation manual criteria, and analyzed their distribution by chapter.

Results of data synthesis. Among the 100 accredited hospitals (40 public, 43 private, 17 mixed), nine were accredited without recommendations for improvement, 47 with recommendations, 40 with reservations, and four with major reservations. All of them received requests for improvement. No significant difference was found concerning the FAC decisions according to status and size of hospitals, although there was a trend that the larger the hospital, the more numerous and more serious the decisions of the FAC. The main topics addressed by decisions were those given high priority by the FAC (information given to patients and its traceability on patient records, and signing of prescriptions for medication).

Conclusion. Despite wide heterogeneity in the summaries on accreditation and in FAC decision-making, this study provides an initial insight into common quality defects and ANAES priorities for hospitals in France.

accreditation, health care, peer review, quality assurance,, quality of health care

In France, the 1996 law reforming public and private hospitalization stipulates that ‘in order to ensure continuous quality and safety improvement of health care, all public and private hospitals must submit to an external evaluation procedure named accreditation’ [1], which is implemented by the national agency for accreditation and evaluation in health care (ANAES). The French Accreditation College (FAC), a component of the ANAES, is a key element responsible for examining the survey reports, attributing accreditation, and defining recommendations for improvement. The objectives of this procedure are to assess quality and safety of care, to assess a hospital’s ability to ensure continuous improvement in quality of overall patient care, to formulate explicit recommendations, to involve professionals at all stages of the quality initiative, to provide external recognition of the quality of care in hospitals, and to improve public confidence [2]. All French hospitals had to enroll within 5 years of publication of the law on April 25, 1996. The first hospitals enrolled in the procedure in June 1999 and the first summaries of accreditation reports were available in June 2000.

An initial analysis performed on the first 20 summaries of accreditation reports showed that those standards that were most frequently identified by the FAC decisions [3,4] were: requirement for patient’s informed consent and provision to the patient of information about the benefits and risks of the planned procedures; assessment of respect for patient’s rights; prescriptions made by the prescribing practitioner, dated and containing the practitioner’s name and signature; patient care coordination between sectors of clinical activity; continuity of care; production of an annual activity report in each sector; annual assessment of staff; implementation of a system to report undesirable events; implementation of specific safety prevention programs; and all staff involvement in quality policy definition. In its 2000 annual activity report, the FAC confirmed this analysis and highlighted four topics [5]: medical prescriptions; patient information about benefits and risks of the planned procedures (and its traceability in patient records); patient record accessibility and archiving; and specific prevention programs (covering blood derivatives, adverse drug reactions, medical device safety, and risks associated with the use of parts of the human body or products derived from it), which must be operational in a hospital.

The main objective of this study was to identify the areas of needed improvement that were most frequently identified in the first 100 accredited hospitals by the FAC, according to the standards manual. The secondary objective was to compare the outcomes of accreditation procedures according to the status and size of the accredited hospitals.

Table 1

Structure and contents of the accreditation manual

Chapter abbreviation	Chapter topic	No. of standards	No. of criteria
Section 1: patients and patient care
DIP	Patient rights and information	9	36
DPA	Patient records	7	24
OPC	Organization of patient care	15	66
Section 2: management and organization for the patient’s benefit
MEA	Management of the healthcare organization and its activity sectors	9	22
GRH	Management of human resources	11	25
GFL	Management of general support services	10	35
GSI	Management of the information system	4	13
Section 3: quality and prevention
QPR	Quality management and risk prevention	5	23
VST	Specific prevention programs and transfusion safety	5	18
SPI	Monitoring, prevention, and control of the risk of infection	11	36

Materials and methods

Sample

We analyzed the first 100 summaries of accreditation reports made available on 29 January 2002 on the ANAES web site (http://www.anaes.fr).

Principles of the French accreditation procedure

The procedure begins with the official enrollment of the hospital, which signs a contract with the ANAES [6]. A self-assessment is performed, according to the accreditation manual published by the ANAES [2]. The manual is divided into three sections, and each is subdivided into chapters containing a set of standards and criteria ( Table 1). The self-assessment is followed by a survey visit conducted by three to six peers from the ANAES accreditation department, including at least one physician, one member of the allied health professions, and a hospital manager. Then, the FAC, which is composed of health professionals independent from the accreditation department, writes an accreditation report based on the peers’ findings, on the results of the self-assessment, and on comments supplied by the hospital after the visit. In this report, the FAC accredits the hospital according to four levels of decision: requests for improvement only, recommendations, reservations, and major reservations. The level of decision depends on the frequency and the seriousness of the departure from the standards, and on implementation of corrective measures. Finally, the accreditation report is sent to the accredited hospital and a summary is published on the ANAES website.

Data collection

In each summary, we collected data concerning hospital, accreditation processes, and decisions. Each topic addressed by the decisions was compared with the relevant criteria and standards of the accreditation manual. Therefore, the number of decisions was lower than the number of criteria and standards concerned with these decisions.

Analysis

We compared the distribution of the four grades of decision, and the duration of the accreditation process, to the status and size of the hospital (Table 2). For each grade of decision, the most frequently cited standards and criteria are presented. The number of decisions per chapter was analyzed in two ways: by raw distribution (i.e. the number of citations) and by weighted distribution (i.e. weighted by the relative number of criteria in each chapter in order to take into account the relative size of each chapter in the accreditation manual).

Table 2

Decisions of the French Accreditation College by status and size of the hospitals

	Requests for improvement	Recommendations	Reservations	Major reservations	Total
	n (%)	n (%)	n (%)	n (%)	n (%)
Status of hospital
Public	2 (5.0)	17 (42.5)	18 (45.0)	3 (7.5)	40 (40.0)
Mixed	3 (17.6)	5 (29.4)	8 (47.1)	1 (5.9)	17 (17.0)
Private	4 (9.3)	25 (58.1)	14 (32.6)	0 (0.0)	43 (43.0)
Size of hospital
<100 beds	3 (10.4)	17 (58.6)	9 (31.0)	0 (0.0)	29 (29.0)
100–399 beds	5 (12.5)	17 (42.5)	17 (42.5)	1 (2.5)	40 (40.0)
400–799 beds	1 (6.7)	7 (46.7)	5 (33.3)	2 (13.3)	15 (15.0)
>799 beds	0 (0.0)	6 (37.5)	9 (56.3)	1 (6.3)	16 (16.0)
Total	9 (9.0)	47 (47.0)	40 (40.0)	4 (4.0)	100 (100.0)

Statistical tests comprised the Pearson’s chi-square test for categorical variables and the non-parametric Mann–Whitney U-test for continuous variables. Any P-value <0.05 was considered statistically significant.

Results

Description of hospitals

The sample differed from the national distribution by overestimating the number of public hospitals (29% in France versus 40% in our sample; P < 0.03) and underestimating the number of small-sized hospitals (37% of hospitals with <100 beds in France versus 49% in our sample; P < 0.02) [15].

Decisions of the FAC

The decisions of the FAC are described in Table 2. The proportion of hospitals accredited with reservations or major reservations was greater in the public and mixed sectors and in larger hospitals, but the difference did not reach statistical significance.

Duration of the accreditation procedure

The median duration of the accreditation procedure (25th and 75th percentiles) was 7.7 months (range 5.7–10.6 months) between enrollment and survey, 7.3 months (6.0–9.3) between survey and FAC recommendations, and 2.0 months (1.3–3.2) between FAC recommendations and summary publication. The duration did not differ between hospitals with <100 beds, 100–399 beds, and 400–799 beds (15.0–15.4 months), but was longer in those with >799 beds (18.1 months; P < 0.05).

The survey visit lasted from 2 to 12 days. The median number of peer-visit days (number of days multiplied by number of peers) was 12 (range 6–74).

Major reservations formulated by the FAC

Major reservations were applied to four hospitals. There was only one major reservation in three hospitals, and two major reservations in one hospital. These concerned the absence of implementation of the recommendations made by external safety control organizations (in two hospitals), insufficient safety in operating theaters, and the declaration of all computer files with nominative data to the National Commission for Computers and Freedoms (CNIL) (whose role is to protect the private lives and the personal and public freedoms of individuals with regard to electronic data).

Reservations formulated by the FAC

Reservations were applied to 44 hospitals. The number of reservations applied to each hospital varied from one to eight (median of two). A total of 97 reservations were formulated, involving 171 criteria and standards in the accreditation manual.

The raw distribution of reservations showed that the chapters most frequently cited in the reservations were (in descending order): patient records [44 (26%) of reservations]; monitoring, prevention, and control of the risk of infection [28 (17%)]; quality management and risk prevention [27 (16%)]; specific prevention programs and transfusion safety [18 (11%)]; and organization of patient care [18 (11%)] (Figure 1). According to the weighted results, the chapters on patient records, quality management and risk prevention, specific prevention programs and transfusion safety, and monitoring, prevention, and control of the risk of infection were those most frequently cited (Figure 1).

Figure 1

Raw and weighted distributions of reservations by accreditation manual chapter.

‘Clear identification of the prescribing practitioner’ in the patient records chapter was the most cited criterion (DPA4c, 16 hospitals). The next most frequently cited criteria are presented in Table 3.

Table 3

Standards and criteria of the accreditation manual most cited in reservations to hospitals accredited with reservations or major reservations (n = 44)

Standard or criterion reference No.	Standard or criterion topic	No. of reservations recorded
DPA4c	Prescriptions made by the prescribing practitioner, dated, and containing the practitioner’s name and signature	16
DPA7a	Assessment of quality of the patient records	5
OPC9	Operational procedures between the pharmacy and the clinical activity sectors	4
OPC9a	Rules governing the conditions of prescription	4
QPR1b	Goals of quality policy are defined and expressed in the quality management and risk prevention program	4
SPI1	Coordinated infection control policy in existence	4
SPI8a	Sterilization covered by a quality assurance system	4
DIP5a	Patient’s informed consent and provision to the patient of information about the benefits and risks of the planned procedures	3
DPA1c	Confidentiality of patient records and information	3
DPA5e	Documentation of the considerations of benefits and risks in the patient records	3
DPA5g	Written document sent to doctor nominated by the patient allowing continuation of care	3
DPA6b	Conditions for archiving patient records	3
QPR1a	Definition of the quality policy by top management in consultation with the representative bodies and staff	3
QPR3f	Existence of a document management system	3
VST1a	Existence of a structure for specific safety prevention programs and transfusion safety	3
VST2a	Implementation of adequate systems for operating specific prevention and transfusion safety programs	3
SPI9a	Written, validated, and evaluated procedures for control of infection risk	3

Recommendations formulated by the FAC

The number of recommendations related to standards or criteria in the accreditation manual varied from one to 17 in the 91 hospitals that received recommendations (median of five). A total of 499 recommendations were formulated, involving 648 standards and criteria in the accreditation manual.

The raw distribution of recommendations showed that the chapters most frequently cited in the recommendations were (in descending order): organization of patient care [118 (18%) of recommendations]; patient records [84 (13%)]; management of general support services [82 (13%)]; patient rights and information [70 (11%)]; and management of human resources [69 (11%)] (Figure 2). According to the weighted results, the chapters on patient records, quality management and risk prevention, management of human resources, management of general support services, and specific prevention programs and transfusion safety were those most frequently cited (Figure 2).

Figure 2

Raw and weighted distributions of recommendations by accreditation manual chapter.

The criterion most cited in the recommendations was GRH7a, ‘conducting an annual interview between each member of staff and the managerial staff ’, in the chapter on management of human resources (21 hospitals). Other frequently cited criteria are presented in Table 4.

Table 4

Standards and criteria of the accreditation manual most cited in recommendations to hospitals accredited with recommendations, reservations, or major reservations (n = 91)

Standard or criterion reference No.	Standard or criterion topic	No. of reservations recorded
GRH7a	Conducting of an annual interview with the managerial staff for each professional	21
DIP5a	Patient’s informed consent and provision to the patient of information about the benefits and risks of the planned procedures	18
DPA5e	Documentation of the considerations of benefits and risks in patient records	17
QPR4b	Implementation of a system reporting undesirable events	14
DIP6	Respect for patient privacy, personal dignity, and liberty	12
DPA4c	Prescriptions made by the prescribing practitioner, dated, and containing the practitioner’s name and signature	12
OPC9a	Rules governing the conditions of prescription	11
GRH4c	Production of up-to-date job profiles	11
GFL7a	Existence of processes for each category of waste	11
MEA1a	Definition and prioritization of the objectives of the development plan	10
DPA1b	Policy of implementing a system grouping patient information	9
OPC8a	Organization of pre-, per- and post-intervention care	9
MEA1b	Involvement of representatives bodies, activity sectors, and professionals in producing the development plan	9
GRH3a	Involvement of the organization’s Medical Committee in all decisions concerning its tasks	9
VST1a	Existence of a structure for specific safety prevention programs and transfusion safety	9

Requests for improvement formulated by the FAC

The number of requests for improvement related to standards or criteria in the accreditation manual varied from eight to 92 in the 100 hospitals. All were subject to requests for improvement (median 27). A total of 3025 requests for improvement were issued, involving 3317 standards and criteria in the accreditation manual.

The raw distribution of requests for improvement showed that the chapters most frequently cited in requests for improvement were (in descending order): organization of patient care [558 (17%) of the requests for improvement]; management of general support services [368 (11%)]; patient rights and information [355 (11%)]; management of human resources [333 (10%)]; and patient records [324 (10%)] (Figure 3). According to the weighted results, the chapters on management of the information system, patient records, quality management and risk prevention, specific prevention programs and transfusion safety, and management of human resources were those most frequently cited (Figure 3).

Figure 3

Raw and weighted distributions of requests for improvement by accreditation manual chapter.

The standards most often cited were those concerning the policy defined by the hospital and/or activity sector to address the standard (the first standard in each chapter). These represented four of the 18 standards most cited in requests for improvement.

The last standards in each chapter, which concern evaluation of the results concerning continuous quality improvement and the levels of achievement with respect to the objectives set by the hospital, were differentially cited in requests for improvement. These standards were frequently cited only in the chapters on patient rights and information (DIP) and patient records (DPA), and, to a lesser extent, in the chapters on infection risk prevention (SPI), management of general support services (GFL), and information systems (GSI).

The criteria most cited in requests for improvement concerned the assessment of respect for patient rights (DIP9a; 43 hospitals), the involvement of all activities in the information system (GSI1b; 37 hospitals), the assessment of general support services such as catering, laundry, security, and transportation (GFL10; 36 hospitals), the requirement for informed consent and information about the benefits and risks of the planned procedures (DIP5a; 34 hospitals), the assessment of the quality of patient records (DPA7a; 34 hospitals), the up-to-date description of job profiles (GRH4c; 33 hospitals), and the implementation of a document management system (QPR3f; 33 hospitals).

Discussion

This review pinpoints topics frequently cited in accreditation decisions taken by the FAC. Two are close to those defined as priorities by the FAC in its 2000 annual activity report [5]: information to patients about the benefits and risks of the planned procedures, and its traceability in patient records, and clear identification of the practitioner issuing the prescription. The other two priorities defined by the ANAES (patient record accessibility and archiving, and specific prevention programs which must be operational) were less frequently cited. Other topics outside the priorities of the ANAES appeared frequently: the assessment of the quality of patient records; the implementation of operational procedures, notably prescription rules between the pharmacy and the clinical activity sectors; the drafting and up-to-date description of job profiles; the implementation of a system to report undesirable events; the involvement of all activities in the information system (especially patient records); the conducting of an annual interview between each member of staff and the managerial staff; the assessment of the general support services; and the implementation of a document management system.

Final outcomes of accreditation programs have rarely been reported [7–9], but studies on the intermediate outcomes of accreditation have been published in the United States [10,11], Australia [12,13], and Catalonia [14]. By comparing the intermediate outcomes of hospital accreditation in United States with those in France [10], it is striking that the performance areas that received a low score of compliance (3–5) from the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) were frequently cited in the reservations or recommendations made by the FAC. This was notably the case for ‘medication use’ (FAC decisions in about half of the hospitals), ‘availability of patient-specific information’ (FAC decisions in 25% of hospitals), ‘assessing staff competence’ (FAC decisions in 20% of hospitals), ‘implementation of safety plans’ (FAC decisions in nearly 20% of hospitals), ‘infection control’ (FAC decisions in 15% of hospitals), and ‘credentialing’ (FAC decisions in 10% of hospitals). Other performance areas frequently found to be either partially or non-compliant in the United States—‘initial assessment of patients’ and ‘design of the environment’—were rarely cited in the FAC decisions. On the other hand, the areas concerning patient records (archiving, confidentiality, assessment of quality) and the assessment of patient rights, commonly cited in the FAC decisions, were rarely highlighted by the JCAHO.

The main limitations of this review are the analysis of the summary reports and the limited number of accreditation reports as yet available in France. Errors of judgment in the correspondence between decisions found in the reports, and criteria or standards in the accreditation manual may have occurred, since our work was based on the summaries of accreditation reports. The limited sample size hampered the power of statistical analysis. The close distribution of hospital status in our sample differed from national figures by including a larger proportion of public hospitals and the largest hospitals [15], and the sample also comprised volunteer hospitals able to enroll quickly in the accreditation procedure. This may reflect a higher degree of investment in the continuous quality improvement process.

The content of the summaries of accreditation reports was heterogeneous. Firstly, the length of the summaries varied from 20 to 50 pages and was not related to the results. Moreover, the number of decisions varied substantially. For example, the number of requests for improvement varied in the range of one to four between hospitals with an identical number of reservations and recommendations. This range is not likely to reflect true differences between hospitals, i.e. ‘the health care organization’s ability to ensure continuous improvement in quality of overall patient care’, which is the main objective of the accreditation procedure [2]. In some summaries, a decision was formulated for each specific criterion in the accreditation manual, whereas another concerned improvement plans, thus covering a large number of criteria. For reasons of equity, this heterogeneity should be reduced in the future.

The initial trends observed in this study show that the largest hospitals received more high-grade decisions. According to the ANAES statement that the accreditation procedure is a process for assessing ‘quality and safety of care’ [2], this finding calls into question the widespread opinion that the quality and safety of care is higher in reference hospitals, such as teaching hospitals, than in small hospitals [16,17]. Indeed, seven of the nine teaching hospitals in our sample had reservations or major reservations (78%). Do these differences reflect a selection bias or do they result from unfair assessment? Although these findings in part reflect the real situation, we must bear in mind that the French accreditation procedure investigates macro-processes, not outcomes. Until links between clinical processes and outcomes are studied further [18], we lack information about the relationship between these macro-processes and outcomes. Herein lies an area of research that might even question the overall effectiveness and efficiency of the accreditation process [19].