Abstract

Transparency and documentation of the decision process are at the core of a credible risk assessment and, in addition, are essential in the presentation of a weight of evidence (WoE)–based approach. Lack of confidence in the risk assessment process (as the basis for a risk management decision), beginning with evaluation of raw data and continuing through the risk decision process, is largely because of issues surrounding transparency. There is a critical need to implement greater transparency throughout the risk assessment process, and although doing so will not guarantee the correctness of the risk assessment or that all risk assessors come up with the same conclusions, it will provide essential information on how a particular conclusion or decision was made, thereby increasing confidence in the conclusions. Recognizing this issue, the International Life Sciences Institute Health and Environmental Sciences Institute convened a multisector committee tasked with discussing this issue and examining existing guidance and recommendations related to transparency in risk assessment. The committee concluded that transparency is inextricably linked to credibility: credibility of the data, credibility of the risk assessment process, and credibility of the resulting decision making. To increase this credibility, existing guidance concerning criteria elements of transparency related to the risk assessment process must be more widely disseminated and applied, and raw data for studies used in human health and environmental risk assessment must be more widely available. Finally, the decision-making process in risk management must be better documented and a guidance framework established for both the process itself and its communication to the public.

The credibility and ultimate acceptance of any risk management decision are inseparably tied to transparency. Risk management decisions are often criticized on the basis that they are not supported by the underlying scientific information. This perception, warranted or not, is often because of a lack of clarity in what information was (and was not) taken into consideration, limited availability of the underlying scientific data, and too few details on the methodology used to arrive at a particular decision. These elements all must be effectively demonstrated and communicated to stakeholders for credible decision making. The Obama Administration’s call to improve and increase transparency, as evidenced in a March 2009 memo on scientific integrity calling for “ … transparency in the preparation, identification, and use of scientific and technological information in policymaking” underscores the importance of this issue (White House, 2009).

Transparency in risk assessment, as defined in the U.S. Environmental Protection Agency (USEPA) Risk Characterization Handbook (USEPA, 2000), requires clear explanation of the risk assessment process by ensuring that any reader understands the steps, logic, key assumptions, limitations, and decisions in the risk assessment and by ensuring comprehension of the supporting rationale that led to the outcome of the risk assessment.

Transparency and credibility are inextricably linked, most notably with respect to risk assessment and risk management. Transparency alone does not ensure credibility or correctness but is a critical element. Credibility in this context can be described as having three components or steps: credibility of the data, credibility of the risk assessment process, and credibility of the resulting decision making. All three areas are important, build upon each other, and must be balanced. A loss of credibility at any one step in the process cannot be regained in another step and will decrease the overall credibility of any resulting decision. In addition, if the risk assessment process and the underlying data upon which it relies are transparent, any potential interference or external influence in the decision-making process can be more easily identified.

Of the three credibility components listed above, the risk assessment process has perhaps been given the most attention. It is often portrayed as the central and pivotal piece of the overall risk “puzzle.” Risk assessments are often said to use a “weight of evidence (WoE) approach” to reach conclusions based on the integration of multiple sources and types of data, incorporation of variability, and use of uncertainty factors among other considerations. However, the term is rarely defined, and the methodologies used vary (Weed, 2005). Recognizing this issue, the International Life Sciences Institute’s Health and Environmental Sciences Institute (HESI) convened a multisector committee tasked with discussing this issue, recognizing that the need for increased transparency is shared by scientists and regulators from government, academia, and industry. This article reflects the consensus opinion shared by the authors. Following a 2006 workshop on WoE in risk assessment that identified transparency as critical to any WoE approach, this committee examined existing guidance and recommendations related to transparency in risk assessment to understand why transparency is consistently cited as an issue and what recommendations have been made by these groups to improve transparency (EFSA, 2006; NRC, 1989; Rasmussen and Jensen, 2005; USEPA, 2000; USFDA, 2002). These documents cite a number of criteria as important for establishing transparency in risk assessments (Table 1). However, these elements do not appear to be universally applied, and although the need for flexibility must be recognized, the wider application of these criteria should be strongly encouraged. When followed, these criteria will lead to greater clarity, consistency, and reasonableness (USEPA, 2000). The call made by President Obama to increase transparency has already led to significant efforts within several U.S. Government agencies. The USFDA (2009) recently formed a Transparency Task Force, whose mission is to “ … recommend ways to improve the openness and transparency of the agency’s information” and includes two public forums to solicit input on how best to address this mission. In addition, USEPA Administrator Lisa Jackson, in a May 2009 memo (USEPA, 2009a), outlined a new Integrated Risk Information System process that “ … will be more transparent and timely and it will ensure the highest level of scientific integrity.” Continuing this initiative, on 1 October 2009, the USEPA announced a “ … new transparent process that will allow the public to review and comment on risk assessments and proposed registration decisions for pesticides” (USEPA, 2009b). Furthermore, a workgroup developed under the auspices of the USEPA’s Science Policy Council is currently developing a white paper on transparency in the agency’s risk assessment process. The paper seeks to evaluate needs, opportunities, challenges, and recommendations for enhancing transparency and will build upon principles set out in their Risk Characterization Handbook (USEPA, 2000).

TABLE 1

Criteria for Transparency in Risk Assessment

• Employing brevity and plain language 
• Defining technical terms 
• Using understandable tables, graphics, and equations to present technical data 
• Identifying purposes and goals of the risk assessment prior to initiation 
• Identifying other similar risk assessments that have been conducted 
• Identifying assessment approaches, assumptions, and extrapolations and their impacts 
• Evaluating the use of models versus measurements 
• Describing the choices made among plausible alternatives 
• Describing the significant data gaps and their implications 
• Identifying and differentiating scientific conclusions and default assumptions 
• Characterizing and differentiating sources of uncertainty regarding data and assessment processes 
• Identifying the major risk conclusions and the assessor’s confidence in them 
• Employing brevity and plain language 
• Defining technical terms 
• Using understandable tables, graphics, and equations to present technical data 
• Identifying purposes and goals of the risk assessment prior to initiation 
• Identifying other similar risk assessments that have been conducted 
• Identifying assessment approaches, assumptions, and extrapolations and their impacts 
• Evaluating the use of models versus measurements 
• Describing the choices made among plausible alternatives 
• Describing the significant data gaps and their implications 
• Identifying and differentiating scientific conclusions and default assumptions 
• Characterizing and differentiating sources of uncertainty regarding data and assessment processes 
• Identifying the major risk conclusions and the assessor’s confidence in them 

Data integrity is absolutely critical in all scientific endeavors, and credible scientific data are the foundation for risk assessment and risk management. If those data are unavailable, even the most transparent of risk assessment processes cannot ensure credibility. Peer review helps ensure the quality of a study design as well as the logic of the conclusions; however, a peer review of a study for journal acceptance is usually performed based on summary data rather than the full spectrum of raw data. Good Laboratory Practices with Quality Assurance help to ensure the integrity of data but do not address the credibility of the results or conclusions. Greater access for interested stakeholders to the raw data would enhance the credibility of study results and conclusions with those stakeholders. Original data (including negative data) from studies conducted by investigators at academic, government, industrial, and nongovernmental organizations can and should be made available to a significantly wider audience (White House, 2009). Although this wider availability of raw data is recommended, this does not imply that there is a need for reanalysis of that data or the original study author’s conclusions, especially in cases where the study was not intended for use as the basis for a risk assessment. The authors recommend that the data are available as a “safety check” for reanalysis as appropriate.

There are a number of potential challenges that will have to be overcome with this approach. These include the need to preserve the confidentiality of ongoing work related to the published material, the potential for proprietary data being used by competitors, the potential for use of the data as the basis for litigation, and the need to maintain confidentiality of participants in epidemiology studies. The wider availability of raw data, regardless of the intent, could lead to multiple misinterpretations of the data (sometimes intentional) raising the question of how such misinterpretations could be corrected. However, these are not new concerns and have been faced and addressed in other situations, such as in the international compensatory agreements on data sharing for testing of pesticides and pharmaceuticals.

A variety of approaches could promote wider availability of data and bring clarity to the processes used to evaluate and weight data to support risk management decisions. Modern electronic tools make it feasible to make available even large volumes of original data so that they could be examined by journal peer reviewers or risk assessors after publication. Many journals already make supplemental data available via the Internet, and this concept could be extended. Where studies are conducted for regulatory purposes, the data could be made available by the regulated entity at the same time as the release of a draft regulatory decision by the reviewing agency in accordance with statutes protecting proprietary data or the study sponsors could make the data available on their website or in a clearinghouse website.

Risk assessment and risk management are often intertwined. Discussions between risk assessors and risk managers early in the risk assessment process help to focus the overall purpose of the risk assessment, identify information gaps and needs, and establish anticipated risk management needs that will facilitate an informed, transparent, and ultimately more credible risk management decision. However, the risk management/decision-making process is often poorly defined, and guidance frameworks (analogous to those established for the risk assessment process) are rare. As a result, it is often at this critical decision-making (risk management) step where the overall process can deteriorate and lose credibility. In the absence of a clearly understood or communicated risk management process, external opposition to the decision may be displaced to the more visible process, risk assessment. The establishment of guidance methodology for risk management and decision making should be undertaken, with input from all stakeholders. The scientific data, the risk assessment process, the risk assessment conclusions, and the risk management considerations should all be integrated in a well-communicated framework. One such framework has been recently suggested by the National Research Council in their report, “Science and Decisions: Advancing Risk Assessment” (NRC, 2008).

Although transparency informs stakeholders, promotes credibility, and enhances risk communication, it does not imply empowerment to determine risk management decisions. Likewise, although transparency will enhance our understanding of the basis of the conclusions of the risk assessment, these conclusions may not be universally supported.

The following steps would increase the credibility of decisions based on scientific conclusions and represent the consensus opinion of this HESI multisector committee:

  • Existing guidance concerning criteria elements of transparency related to the risk assessment process must be more widely disseminated and applied.

  • Raw data for studies used in human health and environmental risk assessment must be more widely available.

  • The decision-making process in risk management must be better documented and a guidance framework established for both the process itself and its communication to the public.

FUNDING

ILSI Health and Environmental Sciences Institute Risk Assessment Methodologies Technical Committee, Weight of Evidence Project.

This document represents the authors’ views expressed as individual scientists and does not necessarily represent the policies and procedures of their respective institutions.

References

European Food Safety Authority (EFSA)
Transparency in risk assessment carried out by EFSA: guidance document on procedural aspects
EFSA Journal
 , 
2006
, vol. 
353
 (pg. 
1
-
16
Available at: http://www.efsa.europa.eu/en/scdocs/scdoc/353.htm. Accessed January 2, 2010
National Research Council (NRC)
Improving Risk Communication
 , 
1989
 
The National Academies Press, Washington, DC
National Research Council (NRC)
Science and Decisions. Advancing Risk Assessment
 , 
2008
 
The National Academies Press, Washington, DC
Rasmussen
B
Jensen
KK
The Hidden Values. Transparency in Decision-Making Processes Dealing with Hazardous Activities
2005
 
Frederiksberg, Denmark: Danish Centre for Bioethics and Risk Assessment, Project Report 5
U.S. Environmental Protection Agency (USEPA)
Risk Characterization Handbook
 , 
2000
 
Report nr 100-B-00–002
USEPA
Memorandum
New Process for Development of Integrated Risk Information System Health Assessments
 , 
2009
 
Available at: http://www.epa.gov/spc/pdfs/rchandbk.pdf. Accessed January 2, 2010
USEPA
Press Release. EPA Opens Transparency Window into Pesticide Registration Decisions
 , 
2009
 
U.S. Food and Drug Administration (USFDA)
FDA News Release, FDA Forms Transparency Task Force
 , 
2009
 
USFDA
Initiation and Conduct of All ‘Major’ Risk Assessments within a Risk Analysis Framework: A Report by the CFSAN Risk Analysis Working Group
 , 
2002
 
Weed
DL
Weight of evidence: a review of concept and methods
Risk Anal.
 , 
2005
, vol. 
25
 (pg. 
1545
-
1547
)
White House
Presidential Documents, Memorandum for the Heads of Executive Departments and Agencies on Transparency and Open Government
 , 
2009
 
74 Fed. Reg. 4685 (January 26, 2009). Available at: http://www.whitehouse.gov/open/. Accessed January 2, 2010