Enhancing Epidemiology’s Impact on Policy: Mediation by Expert Committees

Abstract

Epidemiologic evidence is often a key source of information used by expert committees to guide policy decisions, yet epidemiologists rarely consider this audience for their research. For a better understanding of the pipeline from epidemiologic research to expert committee assessment to policy, several reports from the National Academies of Sciences, Engineering, and Medicine were reviewed and discussed with staff and committee members. The topics of these consensus committee assessments included health behaviors, medical care, and military exposures. The focus was often on emerging issues of immediate concern for which there was little relevant research available but a need for prompt action. Committees generally sought a comprehensive assessment of potential health effects of a given product or exposure, which often included social and behavioral health outcomes that are rarely addressed by epidemiologists. To enhance epidemiology’s contribution to societal decisions, the choice of research topics should expand to consider emerging societal concerns. Research funding agencies need to be engaged as mediators between committee needs and the research community to stimulate contributory research. Improved communication of research needs to the epidemiology community would be beneficial to researchers aspiring to have an impact and to those who use epidemiologic information to help guide policy decisions.

consequentialist epidemiology, expert committees, health policy

Abbreviations

 
• CDC

  Centers for Disease Control and Prevention

 
• FDA

  Food and Drug Administration

 
• NASEM

  National Academies of Sciences, Engineering, and Medicine

Editor’s note: The opinions expressed in this article are those of the author and do not necessarily reflect the views of the American Journal of Epidemiology.

One of the principal aspirations for epidemiologic research is to generate knowledge that can inform societal and personal decisions to improve health—succinctly encapsulated as “consequentialist epidemiology” (1). While the discipline has scholarly content and appropriately pursues knowledge for its own sake, epidemiologists also conduct research explicitly to inform and advance public health. The path from discovery to action begins with research findings and ends with their application to public policy and individual decisions intended to improve health. The pipeline connecting these may include evidence synthesis in the scientific literature, media reports, public awareness and interest, and finally action by those who formulate public policy and individuals who make decisions that affect their health. This process is haphazard and offers little assurance that important evidence will be given due attention and likewise that misinformation will not be given undue attention, particularly in the current era in which social media is the primary mode of disseminating information. The more effective and reliable pathway from research to public policy and guidance to individuals is through a deliberative process in which the state of the science is carefully evaluated by informed and objective experts. Assigning a prominent role to experts runs counter to the democratization of research, but appropriately constituted teams of experts are the ideal mediators between those who generate the research and those who have the authority and responsibility to formulate policy.

There are a number of organizations that engage in the methodical evaluation of evidence to guide policy, including some with a broad purview in health (the National Academies of Sciences, Engineering, and Medicine (NASEM), the US Preventive Services Task Force, the FDA, and the Centers for Disease Control and Prevention (CDC)) and professional societies (e.g., the American Academy of Pediatrics, the American Psychological Association). Although the details vary, the general model involves identifying and convening a panel of experts chosen for their ability to accurately and independently assess the evidence, free of significant conflicts of interest. In almost all cases, multiple disciplines are represented, given that there are multiple lines of evidence that inform policy. A charge to the group specifies the recommendations that are needed to inform decisions to be made by the sponsoring organization, which may pertain to screening recommendations (US Preventive Services Task Force), vaccine dissemination (CDC advisory groups), or clinical guidance and practices (American Academy of Pediatrics). The panels convened by the CDC to recommend coronavirus disease 2019 (COVID-19) vaccine implementation provide a recent, widely publicized example of this process. Not infrequently, Congress mandates that a review take place, especially for programs of the Veterans Health Administration or Environmental Protection Agency or programs funded directly through Congressional actions.

A key strength of this process is that the questions are coming directly from the organization that needs to act in the interest of public health. If the committee is appropriately constituted, is given a clear assignment based on the needs of decision-makers, and thoroughly evaluates the relevant research, the guidance that emerges is likely to be extremely helpful and influential in forming the final decisions and recommendations.

For many topics, epidemiologic evidence is a critical contributor to the committee evaluations. Yet there is often little direct connection between the questions facing these committees and the epidemiologic research priorities. While epidemiologists may aspire to generate policy-relevant evidence, they often choose topics and conduct studies based on other, more pressing considerations, such as prospects for publication and grant funding, career advancement, and intellectual curiosity. While importance or relevance is among the considerations stimulating epidemiologic research, especially when supported by initiatives from funding agencies, there is not a clear pathway by which the targeted questions posed by expert committees stimulate epidemiologic studies. Unlike peers who judge the merits of research based on scientific excitement, methodological elegance, or publication in the leading journals, such committees are only concerned with how helpful the research is in answering their questions.

Epidemiologists rarely contemplate the prospects for their work’s informing such assessments, and those who serve on review committees rarely have an opportunity to instigate the research that would be helpful. Yet a closer connection between the realms of epidemiologic research and expert committees would benefit both the field of epidemiology and the quality of information available for guiding policy. In this commentary, I consider the pathway connecting epidemiologic research to policy using several examples from NASEM committees and offer recommendations to improve the linkage.

METHODS

Reports were selected from consensus committees convened by NASEM as examples to illustrate a wide range of substantive themes and policy relevance:

• Adverse Effects of Vaccines: Evidence and Causality, 2012 (2) (sponsor: Health Resources and Services Administration)

• The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research, 2017 (3) (sponsors: Arizona Department of Health Services, CDC, Oregon Health Authority, Alaska Mental Health Trust Authority, California Department of Public Health, Colorado Health Foundation, Mat-Su Health Foundation, National Highway Traffic Safety Administration, National Cancer Institute, National Institute on Drug Abuse, Robert W. Woodruff Foundation, Truth Initiative, US Food and Drug Administration (FDA), and Washington State Department of Health)

• Public Health Consequences of E-Cigarettes, 2018 (4) (sponsor: FDA)

• Veterans and Agent Orange: Update 11 (2018), 2018 (5) (sponsor: Department of Veterans Affairs)

• Assessment of Long-Term Health Effects of Antimalarial Drugs When Used for Prophylaxis, 2020 (6) (sponsor: Department of Veterans Affairs)

• Space Radiation and Astronaut Health: Managing and Communicating Cancer Risks, 2021 (7) (sponsor: National Aeronautics and Space Administration)

While multiple organizations engage in this type of assessment, the observations should be broadly generalizable to analogous organizations and processes.

The 6 reports considered include assessments of the health impact of specific consumer products (electronic cigarettes (e-cigarettes), cannabis) (3, 4), evaluation of potential health risks in selected populations with distinctive exposures (military personnel exposed to Agent Orange in Vietnam, space travelers exposed to ionizing radiation) (5–7), and assessment of risks and benefits of health-care interventions (antimalarial prophylaxis among military personnel in Southwest Asia, vaccines) (5, 6). The motivation and sponsors for addressing these topics include public health agencies (CDC, state health departments) (2, 3), health-care organizations (Veterans Health Administration) (5, 6), and federal agencies (National Aeronautics and Space Administration, FDA) (4, 7). In some cases, the pressure to address the issue came from advocates (e.g., exposure to Agent Orange and antimalarial drugs among military veterans) (5, 6). In other cases, those who regulate and advise on product use such as the FDA sought information to guide those decisions (e-cigarettes, cannabis) (3, 4). Common to all of these topics is the goal of isolating and distilling the scientific evidence bearing on policy decisions, free of distortion from political, economic, or ideological influence. While the sponsors in each case ultimately need to consider the full array of relevant issues, not just the scientific evidence, the consensus committees are typically constituted to assess the empirical evidence free of the other considerations the sponsor may face.

The observations regarding the connection between epidemiologic research and consensus committees was drawn from the author’s experience serving on such committees, discussions with experienced project directors who oversee the development of committee reports, and several senior epidemiologists who contributed to the illustrative reports. These discussions focused on which epidemiologic studies were especially beneficial to their deliberations, what features made them particularly useful, and how to strengthen the connection between the epidemiologic research community and committee needs.

FINDINGS

Epidemiology in context

The expert committees that receive information from epidemiologic studies are different in a number of ways from the typical audience of those who evaluate grant applications and publications or make decisions about hiring and promotion. Even if epidemiologists embrace the concept of consequentialist research (1), they may not have the insights needed to directly address the needs of the organizations that shape policy. Arising from the sponsor’s concerns, there is a greater likelihood that epidemiology will contribute “science that is more saliently useful to decision makers with a responsibility to the health of the public” (1, p. 1188), the essence of consequentialist epidemiology.

The committee’s focus is on a specific substantive question, and they are searching for all relevant information, including but not limited to epidemiology. The epidemiologic evidence is assessed in the context of other disciplines, with a particular value on research that is synergistic with other disciplines, such as toxicology or the social sciences. For example, potential health effects of Agent Orange exposure are assessed explicitly based on a combination of epidemiology and toxicology, and many questions about the potential impact of e-cigarettes are informed by both behavioral science and epidemiology. While the rigor of the epidemiologic research is of great importance, the alignment of the research with the committee’s charge is the first consideration.

While committees are likely to include one or more experts in epidemiology, the accessibility of the epidemiologic studies to the entire committee helps to determine its impact. The simpler the study’s hypothesis, methods, results, and interpretation, the greater its impact on the committee’s judgments. Even if studies call for complex analytical methods, for example, to be maximally useful there should be a translation to straightforward, policy-relevant results that can be understood and used by nonepidemiologists.

A frequent problem is not with the quality of the epidemiologic evidence that is available but with the dearth of any relevant epidemiology that directly addresses the topic of interest, particularly pertinent to the assessment of long-term consequences of exposure to antimalarial drugs and e-cigarettes. The popular topics for epidemiologic researchers, often focused on disease etiology, are frequently misaligned with the pressing questions that committees are tackling. Generally, expert committees are not asking “What causes this disease?” but rather “What is the potential health impact of this exposure?”. Even rudimentary epidemiologic studies that are squarely on target can have a substantial impact when there is little else to rely upon, which is often the case.

Topics of concern

The questions that expert committees address are often narrow, leading to a circumscribed set of relevant studies. The committee may focus on a particular population (e.g., Vietnam veterans or space travelers (5, 7)) or on specific exposures and health outcomes (e.g., e-cigarettes and respiratory disease (4)). Frequently, the issue of concern for formulating policy is an emerging question that has not been the focus of research over an extended period of time. The intensity of concern about long-term effects of antimalarial drugs emerged only after some veterans of the wars in Southwest Asia reported persistent neuropsychiatric conditions (6). Prior to this time, there was very little research on this question, only the usual short-term studies of adverse drug events. The information value of studies is determined by a combination of their relevance and quality, and it is often relevance that dominates when the volume of research is limited.

A recurring theme is an interest in long-term health effects when most of the research pertains only to acute effects of exposure, applicable to antimalarial drugs, Agent Orange, cannabis, and e-cigarettes (3–5). Given that is it much easier to conduct studies of short-term effects, there is often a paucity of studies of long-term consequences, especially for relatively new agents such as e-cigarettes and commercial cannabis products.

Finally, the level of evidence that will guide policy decisions may be surprisingly modest, such that a great deal of weight falls on whatever research is available. Because the issues that are brought to such committees call for rapid decisions, it is not helpful to simply note that the research is inconclusive or that more research is needed. Instead, the full spectrum of research from definitive to limited suggestions needs to be characterized as precisely as possible, particularly on the low end of the spectrum. The algorithm for addressing Agent Orange (8) has been widely used, with categories of “inadequate or insufficient evidence,” “limited or suggestive evidence,” or “sufficient evidence.” The threshold of “limited or suggestive evidence” bears directly on decisions made by the Veterans Health Administration, a sharp contrast from most academic research, which typically focuses on making the transition from “suggestive” to “conclusive.”

Methodological considerations

All the usual strategies for avoiding bias in order to identify causal effects are applicable to research that informs expert committees, but there are some specific study features that enhance their value for informing policy. The selection of a comparison group should be as close as possible to the real-world choices that the sponsor needs to make. For example, in the case of e-cigarettes, the evaluation of public health impact involves not just the question “e-cigarettes versus no e-cigarettes” but extends to a range of scenarios that will ultimately determine the product’s net impact (3). The biological impact of e-cigarettes is clearest in comparing users with those free of any such exposure, whereas for policy guidance, the 2 key issues come down to e-cigarettes versus sustained smoking and e-cigarettes as a gateway to tobacco use.

For guiding policy based on public health impact, there is an emphasis on absolute rather than relative effects, leading to more interest in risk differences and attributable risk. There may also be a need to consider tradeoffs on a common scale, such as years of life lost or disability-adjusted life years, to reconcile information regarding multiple causal relationships. Judgments about vaccines, for example, involve just such comparisons, so that it is not sufficient to simply ask whether there is any morbidity resulting from adverse reactions (2).

STRATEGIES FOR INCREASING THE IMPACT OF EPIDEMIOLOGIC RESEARCH IN EXPERT COMMITTEES

Selection of topics

There is a wide range of topics amenable to epidemiologic investigation and varying motives for the choices we make. Some topics are popular topics based on training and funding opportunities, and most are continuations of long-standing concerns. However, review committees are often charged with addressing a new or evolving topic that has received little or no attention from epidemiologists. Neglected topics with policy implications may generate more impactful epidemiologic research than frequently studied topics in need of refinement. In many cases, simple descriptive epidemiology or screening of potential predictors or determinants of disease can carry a great deal of weight in committee evaluations. Topics that have come to attention through case reports or case series or for which there is public interest generated by advocates or media attention may be easily advanced through even rudimentary epidemiology. Initial steps in the evolution of epidemiologic evidence moving from “no evidence” towards “suggestive evidence” are critically important to stimulating further research on topics of policy importance. Suggested topics for consequential epidemiology are more likely to arise from policy debates than from the epidemiologic literature.

Setting priorities based on public health impact is appropriate for epidemiology, of course, but the most pressing public health issues may not correspond to the problems that the public is interested in. The alleged relationship between vaccines and autism is one such example (2), but more broadly, the concern about Agent Orange among Vietnam veterans (5) or radiation exposure among space travelers (7) may not be at the top of the list based on public health burden. However, for certain interest groups, they are significant concerns that call for informed decisions. A pending policy decision can be viewed as sufficient motivation to conduct informative epidemiologic studies, and epidemiologists’ attempts to redirect public interest are rarely effective.

The health outcomes of interest also offer some guidance on what makes a topic more likely to be informative for policy. Epidemiologists gravitate to the study of well-defined diseases, such as cancer or neurodegenerative disease, with a particular interest in biomedical determinants of etiology. However, the driving concerns with regard to health impacts are often behavioral or psychological conditions. A key concern about e-cigarettes was the potential for use of that product to result in an increased likelihood of tobacco smoking (4), clearly an important causal question and one that tends to be addressed only by behavioral scientists and not by epidemiologists. Similarly, a central issue with cannabis was the potential adverse impact on school performance (3)—not the sort of endpoint that typically engages epidemiologists. While there are many advantages to epidemiology’s aligning itself closely with biomedical sciences, there is a pressing need to forge closer connections with behavioral and social sciences to address societal problems of concern. Epidemiology clearly has a role in addressing the opioid epidemic, vaccine hesitancy, and gun violence, none of which are necessarily aligned with the more traditional realms of biomedically driven etiological research.

Research methods

The need for research that can fill knowledge gaps is universal, but for informing policy, the priority among those gaps is determined by a particular need. The guidance goes beyond the cliché that “more research is needed” to indicate what the key deficiencies are that need to be overcome and what would have a substantial bearing on policy implications. In the case of antimalarial drugs (5), there are abundant studies that monitor for adverse events during the period of use but very little research that addresses long-term health effects after cessation of use; studies that could do so would bear critically on the question posed by the Veterans Health Administration regardless of whether they provided evidence supporting or refuting the presence of such effects. Health effects of cannabis use have been studied for some time, but as the potency and availability of cannabis have changed in recent years, new research is needed to inform current and future policy choices. Long-term health effects of Agent Orange are very difficult to assess in US military veterans (5), but studies of highly exposed Vietnamese populations could provide more clarity regarding health impacts that could be extrapolated to US military veterans.

In many cases, the focus of the committee is on a specific population of concern, a real-world pragmatic concern that may run counter to the search for universal truths. When we query the public health consequences of e-cigarettes (4), for example, the impact on young users is very different from the impact on older, recalcitrant tobacco smokers. The health effects of antimalarial drugs (6) may well be distinct for deployed military personnel as compared with those who travel to malaria-endemic areas for enjoyment. Whereas the purpose of epidemiology (and science more broadly) is to produce generalizable knowledge, policy may need to be informed by an understanding of what occurs in specific, real-world populations both to ensure its relevance to the population of interest and to be persuasive to a broad audience. In going from pure biomedical discovery to informing policy, there is a need to pay attention to the context—that is, the history, culture, and sociology of the target population.

Cross-cutting all of these issues is the need for improved methods of evidence integration. Committees increasingly use the evolving technology of systematic evidence reviews. In contrast to the specific, narrow questions that may lend themselves to conventional meta-analyses that generate a pooled estimate of effect, the broader topics limit the value of conventional tools and call for innovative approaches targeted to unify diverse threads of often fragmentary evidence with a wide range of methodological features and applicability to the question of interest.

Guidance from committees

In order to increase the likelihood of stimulating needed research, the committees that offer research recommendations should be as specific as possible but should also consider how best to reach those who could instigate or conduct such studies. In some cases, the entity sponsoring the report is a research funding agency, and the committee’s recommendations may translate directly into funding priorities—for example, a report on funding priorities for new research sponsored by the Environmental Protection Agency based on a mandate from Congress (9). But in most cases, reports are sponsored by agencies that are not major funders of research (e.g., CDC, US Preventive Services Task Force) but funding agencies like the National Institutes of Health need to be informed of and persuaded by committee reports on research needs. More attention should be directed toward reaching those who establish the intramural and extramural research program priorities at the National Institutes of Health and other federal agencies, as well as foundations and other organizations that seek to advance knowledge on the practical issues of concern to review committees. While reports to the sponsors are obligatory, outreach to relevant funding agencies is not necessarily part of the protocol for report release and dissemination. Epidemiologists may not read the committee reports, but many conscientious individuals monitor research funding opportunities.

Whether directed to the researchers or funding agencies, the more specific the recommendations can be, the more likely they are to be addressed. Rather than “improved studies are needed,” the message should be more pointed—for example, “There is a need to improve the accuracy of exposure assessment through the incorporation of biomarkers” or “A critical confounder limiting past studies needs to be addressed in order to clarify whether there is a causal effect.” In some cases, the research platforms capable of addressing the topic more effectively can be pointed to, noting the promise of improved research through specific avenues. The vaccine safety report noted the need to fully exploit administrative health-care data resources that are large enough to detect increases in rare events, identifying a limited pool of potential contributors (2). Targeting such specific research platforms could well motivate people who manage those resources to seek research support and provide ammunition in making the case that such research would be of value.

More broadly, to reach the epidemiology community and the funding agencies that support it, there needs to be more active communication. While some reports are widely read, in many cases the audience for them is small and includes only persons who are already invested in the topic, missing those who could be inspired to address the issue. Committee report summaries and an assessment of the implications for epidemiology could be a regular feature in epidemiology journals or newsletters. Ad hoc presentations at research meetings could also be effective in highlighting the reports and their research needs. The separate worlds of consensus committees and practicing epidemiologists need to be bridged from both sides.

EPIDEMIOLOGISTS’ ENGAGEMENT IN EXPERT COMMITTEES

There are compelling reasons for epidemiologists to be receptive to serving on review committees. Beyond the prestige in the form of peer recognition, the experience has great value in enhancing awareness of epidemiology in context, one of the rare settings in which epidemiologists engage intensively with colleagues from other disciplines around an important policy issue. Observing how the epidemiologic evidence is blended with the other sources of knowledge about the topic to address a specific, often practical question provides a distinct perspective. Recognizing that a small subset of studies often drives the conclusions is a useful reminder that informativeness is based on both the selection of the topic and the quality of the methods, whereas academic epidemiologists tend to be overly focused on the methods and insufficiently thoughtful about the choice of topics. Those who aspire to have their research fall into the highly influential group will need to take a broader view of the policy questions and how best to invest their efforts to address these pressing societal issues.

ACKNOWLEDGMENTS

Author affiliation: Department of Epidemiology, School of Public Health, Brown University, Providence, Rhode Island, United States (David A. Savitz).

I thank Drs. Amy Berrington de Gonzalez, Clay Johnston, Kathleen Stratton, and Robert Wallace and Anne Styka for sharing their thoughts on the relationship between epidemiologic research and consensus committee reports, and Dr. Rose Martinez for facilitating this project. I also extend appreciation to Drs. Stephanie Engel and Andrew Olshan for reviewing an earlier draft of this article.

The views presented in this article are those of the author and not those of NASEM staff or persons who served on consensus committees.

Conflict of interest: none declared.