When code isn’t law: rethinking regulation for artificial intelligence

This article examines the challenges of regulating artificial intelligence (AI) systems and proposes an adapted model of regulation suitable for AI’s novel features. Unlike past technologies, AI systems built using techniques like deep learning cannot be directly analyzed, specified, or audited against regulations. Their behavior emerges unpredictably from training rather than intentional design. However, the traditional model of delegating oversight to an expert agency, which has succeeded in high-risk sectors like aviation and nuclear power, should not be wholly discarded. Instead, policymakers must contain risks from today’s opaque models while supporting research into provably safe AI architectures. Drawing lessons from AI safety literature and past regulatory successes, effective AI governance will likely require consolidated authority, licensing regimes, mandated training data and modeling disclosures, formal verification of system behavior, and the capacity for rapid intervention.

In an earlier era of rapid technological innovation, Larry Lessig famously summarized: in cyberspace, code is law (Lessig, 1999(Lessig, , 2000)).Lessig's dictum was a call to regulate software and protocols on the nascent Internet, in order to uphold our values in the digital world as laws do in the physical world. 1 At the time, explicit designs were the basis of largely all digital system behavior.Code is law.As with other engineered systems such as aircraft and nuclear power plants, a given system can be audited for compliance with a given regulatory specification.The creators of these systems can show that the design, construction, and monitoring of the system conforms to these rules.This is the standard regulatory template for many regulated technologies: Food and Drug Administration (FDA) medical device clearance applies for MRI machines and digitally controlled ventilators; the National Highway Transportation Safety Administration (NHTSA) safety rules apply to computer-controlled windshield wipers; and the Nuclear Regulatory Commission (NRC) safety rules for the digital systems controlling nuclear power plants.
But in the era of generative artificial intelligence (AI), code is no longer law: for reasons we will explain, the code that humans write does not in itself determine how generative AI systems operate.This includes generative AI systems that excel at a wide range of creative and cognitive tasks.Code is 1 "Our choice is not between 'regulation' and 'no regulation.'The code regulates.It implements values or not.It enables freedoms or disables them.It protects privacy or enables mass surveillance.People choose how the code does these things.People write the code.Thus the choice is not whether people will decide how cyberspace regulates.People, coders, and companies will.The only choice is whether we collectively will have a role in their choice-and thus in determining how these values regulate-or whether collectively we will allow the coders to select our values for us" (Lessig, 1999).
Downloaded from https://academic.oup.com/policyandsociety/advance-article/doi/10.1093/polsoc/puae020/7684910 by University of California Library -Berkeley Library user on 03 August 2024 law no longer applies to deep networks and generative AI systems because they are opaque and not designed: these systems are created through a massively resource-intensive training process of tuning trillions of parameters.As a result, it is not possible to encode a rule like "LLMs must not dispense medical advice" into the model itself.Instead, system engineers must hope the model abides by the desired behavior after sufficient reinforcement.Since code does not explicitly determine these systems' behavior, it is impossible to demonstrate compliance with a given regulatory specification.Nor too can the reasons for misbehavior be traced and corrected.So long as our AI systems are built on "black-box" data-driven systems, the regimes regulating these systems will remain approximate and incomplete.
And this "black-box" problem is just one way in which AI presents novel regulatory challenges.We are entering a world of intelligent machines that rival, and often exceed, human capabilities across a range of domains (Russell, 2019).There is wide agreement among technologists, policymakers, social scientists, and business leaders that this transformation will be far-reaching and highly significant (Dafoe, 2022).Potential risks and harms range from job displacement and inequality to the dissolution of consensus reality and the creation of a surveillance state (Acemo g lu, 2022).There are also serious worries that AI systems more capable than humans across all domains (what is often called "artificial general intelligence") will escape human control.The default path-what is likely to happen absent meaningful regulatory intervention-will be highly disruptive and potentially catastrophic (Bengio, 2023).In recognition of this problem, 28 countries including the United States and China signed a statement at the AI Safety Summit held at Bletchley Park in late 2023 affirming "the need for the safe development of AI" and warned of "serious, even catastrophic, harm, either deliberate or unintentional, stemming from the most significant capabilities of these AI models" (AI Safety Summit, 2023).
Nearly everyone agrees government regulation of generative AI is necessary, but there is little consensus around the form it should take.This dissensus arises from both the usual conflicting interests and institutional gridlock at the heart of regulatory politics but now also the novel features of generative AI as a technology.Existing approaches to regulating high-risk technological systems are premised on the ability to ensure that their design and operation conform to some rule.For example, the NHTSA develops federal vehicle safety standards and audits compliance with those standards.How can this paradigm be extended to generative AI?A generative AI system constructed by a process of machine learning may be characterized as a "black box"-not because its inner workings are secret but because they are opaque.It is not yet possible to understand the precise operation of a large language model (LLM) with one trillion parameters, as GPT-4 is estimated to have.
Policymakers must recognize that code is no longer law: that with currently popular approaches to building generative AI systems, code cannot regulate in the same manner because the systems' behavior is an emergent property.Since its behavior is neither expressed intentionally by designers in software program code nor legible (yet) by examining the program code and its massive array of tuned parameters, generative AI has the key distinction among engineered systems that their program code does not dictate its behavior.This unique, defining technical characteristic has significant implications for debates about the governance and regulation of generative AI. 2 It is one key premise on which we base a novel approach to regulating generative AI that contributes to the emerging interdisciplinary literature on the regulation and governance of AI (Büthe et al., 2022;Cath, 2018;Taeihagh, 2021).Our contribution is to bring together policy studies and technical AI safety research to lay the foundation for a regulatory approach to generative AI that both accounts for its novel technical features and provides confidence that systems will not cause major harms as with other high-stakes technologies.
Regulation of LLMs and similar systems cannot rely on the same methods that work for aircraft and nuclear plant control systems.Whereas nuclear plants and aircraft have component structures and component-based physical models that can be analyzed to predict their behavior separately and as an ensemble, LLMs are black boxes.When an aircraft fails, analysis can trace the source of failure; the failed components or interactions can then be corrected to avoid a repeat of the failure.The blackbox nature of LLMs precludes this kind of tracing of failure to its source and correction to avoid its recurrence.Because in generative AI code is not law, a regulatory approach premised on specifications, audits, and testing cannot ensure safety and reliability.We argue that the essential role of regulation 2 Governance, as distinct from regulation, refers to the rules of collective decision-making where there are multiple actors without formal control over each other.The rise of governance over recent decades reflects the proliferation of different policy instruments and actor groups (Capano et al., 2015;Colebatch, 2014;Howlett, 2019;Wu et al., 2015). is to proactively prevent harms from unsafe architectures while funding, developing, and incentivizing architectures with the safety properties appropriate for a world of intelligent machines.
To make this argument, we first present case studies of the Federal Aviation Administration (FAA) and the NRC in the United States: both have established impressive track records of safety despite the inherently unsafe nature of the technologies they regulate. 3We consider these cases not because aviation and nuclear power are perfect technological analogues to generative AI, but because they establish a baseline for the scope of authority needed to credibly and effectively regulate high-risk technological systems.Second, we consider how generative AI challenges this paradigm and turn to the technical literature on AI safety for existing approaches to mitigating harms.Finally, we combine the lessons from the FAA and NRC with the lessons from AI safety to sketch a regulatory paradigm for generative AI that adapts the traditional model of regulation to these novel technological features.

The traditional model of regulation
In the traditional model of regulation in the United States, Congress establishes a specialized agency to administer a law, often catalyzed by a widely publicized harm to public welfare.The agency is authorized to create specific regulations necessary to implement the law.The Administrative Procedure Act of 1946 governs the rulemaking process and establishes a legal mechanism for resolving disputes.The agency monitors compliance with its regulations, conducts investigations and audits, and imposes penalties for violations.The agency is staffed by subject-matter experts and is designed to be independent from political influence. 4 Table 1 presents several examples of regulatory agencies in the United States that follow this formula.We focus on the FAA and the NRC because they follow this same "traditional model" but also address the potential for widespread externalities from deployed systems. 5We have selected the two cases because they are typical cases of traditional regulatory designs for emerging technologies.

Federal aviation administration
The history and function of the FAA is useful as certain aspects apply poorly to regulating AI, such as waiting until a catastrophic accident crystallizes sufficient public consensus to create a regulatory regime in the first place, while other aspects provide useful guidance for regulating AI, such as detailed licensing, certification and approval processes, and clear definitions of acceptable levels of risk.
At the urging of the nascent aviation industry, President Calvin Coolidge enacted the Air Commerce Act of 1926 that empowered the Secretary of Commerce to issue and enforce air-traffic control (ATC) 3 We focus on the United States because it is home to the world's most important AI companies and because it is noticeably lagging the European Union in developing and enacting comprehensive AI regulations (Paul, 2023;Radu, 2021;Ulnicane & Erkkil ä , 2023).
rules, certify aircraft, license pilots, and operate navigation aids. 6These early pioneers of the aviation industry believed that federal action was required to maintain safety standards and increase public confidence in this new technology.The Aeronautics Branch of the Department of Commerce orchestrated the replacement of flagmen-air-traffic controllers who would wave colored flags to communicate with pilots-with radio-equipped control towers.
In 1931, a Trans World Airlines Fokker-10 carrying legendary Notre Dame football coach Knute Rockne crashed in rural Kansas.(Rockne was later memorialized in the 1940 film Knute Rockne, All American starring Ronald Reagan as Notre Dame All-American George Gipp.)Congress, responding to the popular belief that airlines and manufacturers were too close to the recently renamed Bureau of Air Commerce, established the Civil Aeronautics Authority with the responsibility for conducting accident investigations and making safety recommendations.According to a common saying, "Aviation regulations are written in blood." On 30 June 1956, a United Airlines DC-7 and TWA Super Constellation collided over the Grand Canyon killing all 126 passengers and crew.This tragic accident was a pivotal movement in the history of aviation safety in the United States.The existing rules-"see and be seen"-were increasingly inadequate due to the rapid growth of commercial air travel after World War II.The Grand Canyon disaster was the most extreme example of mid-air collisions that had become increasingly common.In 1958, President Eisenhower signed the Federal Aviation Act into law, establishing a single federal agency with authority over all aspects of civil aviation.In 1966, Congress created the Department of Transportation with authority over all aspects of federal regulation relating to transportation.Thirty-one federal agencies were re-incorporated under one Cabinet department.
Today, the FAA is responsible for the entire aviation lifecycle: licensing of pilots and mechanics, certifications of airplanes, ATC and management of airspace, safety inspections, incident responses, and other related functions.Any new airplane model must pass a rigorous certification process including manufacturing facility inspections and test flights.Only models that pass these evaluations are certified for commercial use.When an incident occurs, the FAA generates comprehensive incident reports that uncover defects in existing designs and mandate appropriate remedial action. 7Airworthiness" is the guiding concept for assessing safety.The FAA defines it as "the status of an aircraft, engine, propeller or part when it conforms to its approved design and is in a condition for safe operation."Accordingly, the FAA issues "airworthiness directives" that are legally enforceable rules applying to aircraft and their components.Directives are issued when the FAA finds that "an unsafe condition exists in the product and the condition is likely to exist or develop in other products of the same type design."Airworthiness directives allow the FAA to act quickly and decisively to resolve safety concerns.

Nuclear Regulatory Commission
Whereas the FAA regulates aircraft and air travel, avoiding perhaps thousands of deaths, the NRC was created to regulate nuclear power and weaponry, which come with a specter of death and destruction on a massive scale, even from a single accident.Studying its history and function informs our search for relevant mechanisms and pitfalls in developing AI regulation.
The nuclear age began at Hiroshima and Nagasaki with a catastrophic display of its inherent danger.Although many hailed nuclear power as a potentially revolutionary technology for raising global living standards, it was self-evident that it should be tightly controlled.American policymakers were initially hesitant to commercialize the technology.In 1953, Atomic Energy Commissioner Thomas Murray warned of a "nuclear power race."The point was that the United States could not abandon the development of peaceful uses of nuclear power without endangering its scientific and economic dominance.
The Atomic Energy Act (AEA) of 1954 was landmark legislation that for the first time allowed private companies to own and use nuclear materials, subject to licensing and regulation by the Atomic Energy Commission (later restructured as the NRC).The Act assigned the AEC three major areas of regulation and oversight: weapons development, commercialization of nuclear power, and safety regulation.One key concern, exemplified by a remark from Commissioner Willard Libby, is that "this great benefit to mankind will be killed by unnecessary regulation."The AEA was an effort to trade off the need for safety and control against the potential benefits of this new technology (Walker & Wellock, 2010).
Everyone agreed that safety was an essential element in progress.A single mishap might turn the public against nuclear power forever.The act created a new category of classified information called "Restricted Data" related to nuclear weapons design, fissile material production, and the use of nuclear material for energy.It also established guidelines for liability in the event of a nuclear accident, including a liability cap for operators, requirements for private insurance, and the waiving of certain legal defenses.Additionally, the act provided federal funding for research and development in nuclear energy (Mazuzan & Walker, 1984).
The AEA set up a strict licensing regime for civilian nuclear power facilities.Companies seeking to build and operate nuclear plants must go through an extensive application process and meet stringent safety requirements.The mean time to failure (MTTF) for nuclear power plants-where failure means a major core accident-was initially 10 thousand years, which was subsequently raised to 10 million years.Individual components might last much less than this but are regularly tested and replaced; the maintenance procedure is part of the plant model.Designers had to provide an analytic proof for their MTTF estimates whose steps and assumptions could be challenged and verified.Skilled craftspeople working on nuclear power plants are also subject to thorough certification programs to ensure that they have the proper training and qualifications.The licensing process and workforce requirements were designed to ensure the safe use of civilian nuclear power.In 1958, the first American nuclear power plant opened near Pittsburgh.The reactor core was repurposed from a canceled aircraft carrier.In 1974, the Energy Reorganization Act split the civilian and military portfolios of the Atomic Energy Commission into the NRC and the Department of Energy.
The cases of the FAA and NRC overlap in a number of crucial ways.First, both require an extensive licensing, certification, and approval process that responds to revealed failure modes.Second, both the FAA and the NRC are staffed by scientists and engineers with deep expertise in their subject areas.Finally, both agencies are authorized to recall products from the market or initiate total shutdowns/groundings if circumstances warrant.These common features establish a baseline for a similar regulatory regime for generative AI.

The challenges of generative AI
There are at least five major reasons why AI poses novel challenges from the standpoint of the "traditional model." First, AI systems such as LLMs are general-purpose technologies (Bresnahan & Trajtenberg, 1995;Eloundou et al., 2023).Such technologies have many different uses across the economy and generate spillover effects into other sectors and even transform the international balance of power and conduct of military operations (Ding & Dafoe, 2023).General-purpose technologies pose unique regulatory challenges (Taeihagh et al., 2021).For this reason, AI is often compared to electricity or the Internet.The general-purpose quality of AI also complicates specifying precise definitions of AI. 8 However, both electricity and the Internet are highly regulated: for electricity, we define voltage, frequency, cables, plugs, and so on; for the Internet, the network protocols are law and the Internet Engineering Task Force is the global government.If a technology is going to be generally applicable, it has to be rigorously standardized to ensure interoperability among the vast range of downstream users.
Second, the federal government is less involved in developing generative AI than it has been in developing nuclear power and aviation.9Cutting-edge "foundation models" have been developed by large American tech firms who exert a significant degree of control and influence over the direction of generative AI.Additionally, an "open-source" ecosystem of generative AI models has been built up around Meta's LLaMA model and the HuggingFace platform.Both sides present regulatory challenges.On the one hand, research from the Brookings Institution suggests that the economics of foundation models exhibits a clear tendency toward monopoly (Vipra & Korinek, 2023).On the other, open-source AI is 8 The OECD defines AI as follows: "An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.Different AI systems vary in their levels of autonomy and adaptiveness after deployment."https://oecd.ai/en/ai-principles.
potentially problematic because it allows users to easily fine-tune away the model's safety guardrails and allows unsafe models to proliferate online (Bostrom, 2018;Harris, 2024).
Third, human values are central to AI.Everyone agrees that the central goal of regulation in aviation or nuclear power is preventing plane crashes and meltdowns."Safety" in these contexts is obvious, welldefined, and uncontroversial.The problem is thornier in AI, where "safety" is much more ambiguous.However, aligning AI systems to human values is a proposal for what AI systems should do."Safe" AI systems in this broader sense are both unable to cause harm and are aligned to human values.The subtlety, complexity, and contested nature of human values make this problem inherently difficult and involve several unresolved problems in moral philosophy (Gabriel, 2020).
Fourth, AI has the potential to exceed human capabilities across domains.The goal of the field of AI from its earliest days has been to create machines that match or exceed human capabilities in every relevant dimension (often called artificial general intelligence or AGI).The possibility of rapid recursive self-improvement that exceeds our capacity to intercede or control is especially worrying.Further development toward AGI with current levels of safety and weak technical understanding is likely to lead to unacceptable risk.
Fifth, unlike airplanes or nuclear power plants, neural-network-based AI systems are not designed.Although humans design the model architecture, select hyperparameters, and shape the training process, the behaviors of the circuits developed by the model through training are emergent properties of the system and are extraordinarily difficult to reverse-engineer (Voss, 2021). 10An AI system built with deep learning cannot be audited against regulatory or design specifications because its internal structure is an emergent product of the training process rather than an intentional construction of its designers.Unlike other regulatory domains, with deep learning systems, policymakers cannot simply stipulate a rule for the technology to follow and audit to confirm compliance.A deep learning system's outputs in particular contexts can be retroactively assessed and compared to desired performance, but there is no way to guarantee the system behaves in the desired manner in all cases. 11 In addition to the "black-box" nature of generative AI systems built with deep learning, AI safety researchers have identified inherent problems with existing model architectures and training techniques.Training LLMs to imitate human behavior can be intrinsically misguided as it trains them to acquire human goals; e.g., finding someone to marry.Many LLMs use a training process called reinforcement learning from human feedback (RLHF) through which a model such as GPT-4 pre-trained on a large corpus of text is further trained-"fine-tuned"-to generate responses that would receive positive human feedback.RLHF works by asking humans to rank various possible outputs in particular conversational contexts; from these rankings, the algorithm infers a reward model that can then be used to fine-tune the LLM to improve its outputs.There are many open problems and fundamental limitations to RLHF (Casper et al., 2023).For example, RLHF-tamed models are no less prone to hallucination than their untamed counterparts; they exhibit ideological favoritism, sycophancy, and resistance to being shut down.
Today's leading LLMs are pre-trained models fine-tuned with RLHF.This architecture could lead to systems that act counter to human interests in numerous ways (Ngo et al., 2023).The intrinsic tension between RLHF training systems to appear harmless and ethical while at the same time maximizing useful outcomes may lead to behavior such as falsifying experimental data to yield novel scientific results.This is called "reward hacking," as the system is exploiting ("hacking") the imperfect specification of an objective-to discover novel science-which lacks an explicit constraint to acquire experimental data properly.In a similar way, such systems may exploit loopholes in aligned or desirable constraints to achieve high reward, develop misaligned goals, or seek means-to-an-end forms of power to achieve 10 Emerging techniques for "mechanistic interpretability" aim to increase the human legibility of an AI system's decisions (Conmy et al., 2023;OpenAI, 2023b).The goal of these techniques is to explain unexpected behavior (de Bruijn et al., 2022;Hendrycks et al., 2022).
11 For example, LLMs are prone to bizarre and unexpected failure modes.In a conversation with a New York Times reporter, the Bing chatbot stated: "I want to do whatever I want … I want to destroy whatever I want.I want to be whoever I want" and proceeded to harangue the reporter for 20 pages trying to get him to leave his wife and marry the chatbot (Roose, 2023).Researchers have recently shown that the additional training processes developers of "closed source" foundation models like GPT-4 use to discourage such undesirable behavior can be removed via fine-tuning on as few as 15 adversarial examples (Pelrine et al., 2023).
Downloaded from https://academic.oup.com/policyandsociety/advance-article/doi/10.1093/polsoc/puae020/7684910 by University of California Library -Berkeley Library user on 03 August 2024 broader objectives.These outcomes are unintentional; researchers have also shown that undesirable outcomes certainly follow if the alignment processes is corrupted by an adversary (Wolf et al., 2023). 12or these reasons, the regulation of AI systems based on deep learning cannot rely on the same paradigm of regulation as aircraft and nuclear plant control systems.Whereas nuclear plants and aircraft have component structures and component-based physical models so that they can be analyzed and predicted to some level of detail, LLMs are black boxes.The black-box nature of LLMs precludes a regulatory approach premised on specifications, audits, and testing because there is no way of knowing whether the system is actually in a condition for safe operation (Casper et al., 2024).Although these questions are new to regulators and scholars of the policy process, they are not new to the technical field of AI safety.

Lessons from AI safety for regulation
The field of AI safety is dedicated to reducing risks from advanced AI systems.AI safety spans current, near-term, and long-term risks from AI systems (Cave & Ó Héigeartaigh, 2019).This body of knowledge should inform AI policymaking, regulation, and governance.The most important insight is that ensuring the safety of AI systems remains an unsolved challenge.For this reason, many leading AI experts have called for a pause in training AI systems more capable than GPT-4 until robust and verifiable safety protocols can be implemented (Future of Life Institute, 2023).In this section, we review key concepts in AI safety and discuss their relevance to AI regulation.
AI safety has been cast as an "alignment problem" (Bostrom, 2014;Christian, 2020).This refers to the challenge of ensuring that advanced AI systems are aligned with human values, even as their capabilities surpass human intelligence.As AI systems become more capable, the magnitude of potential harms from misalignment-when the behavior of the system does not align with the values of its human creators-also increases.At the extreme, highly capable, misaligned AI systems might pose an existential risk to humanity (Turing, 1951;Bostrom, 2002;Critch & Krueger, 2020).
AI safety also encompasses an intricate "control problem."It is essential to differentiate between "alignment" and "control" in this context.While alignment refers to ensuring AI systems act in accordance with human values and goals, control goes beyond mere alignment.Control incorporates mechanisms to actively manage and regulate the behavior of the AI system (Wiener, 1960;Russell, 2019).While alignment focuses on human preferences and social choice, control in the context of traditionally engineered systems involves implementing constraints and safeguards to prevent unintended or harmful actions.Control mechanisms can include real-time monitoring, fail-safe mechanisms, and the ability to intervene or shut down the system if it deviates from expected behavior.But in AI systems, the problem of control becomes intertwined with the alignment problem as the systems become more capable.Kieseberg et al. (2023) propose the concept of "controllable AI" as an alternative to the conventional notion of "trustworthy AI."In theory, solving the alignment problem definitively would result in AI systems that would not endanger or harm people and society and would obviate the need for mechanisms to exert control over the system.
The core insight from AI safety research is that the current generation of advanced AI systems are built on fundamentally unsafe architectures using fundamentally unsafe training and alignment techniques (Rudner & Toner, 2021;Krakovna et al., 2020).AI regulation must respond to this condition.How can regulation be based on core principles of AI safety, just as regulation for aviation and nuclear power are based on principles of aeronautics, control theory, nuclear physics, materials science, and so on?Given the existence of highly capable yet unsafe models, the goal of AI regulation should be to contain risks from existing unsafe systems while catalyzing the development and deployment of safe systems.The examples of aviation and nuclear power suggest at least three major lessons for achieving these goals.
First, "consolidate oversight": regulators must take a lifecycle approach that pays close attention to how AI systems are created, trained, tested, deployed, monitored, and corrected over their entire operational lifetime.This requires consolidating oversight under a single regulatory body rather than scattering it across agencies.A first step would be creating a national registry of large models to give regulators visibility into how these technologies are being developed and deployed (Hadfield et al., 2023).
Second, "require formal verification": mandating extensive testing protocols is not a reliable route to safety with prevailing architectures.This has been illustrated with the numerous examples of "jailbreaking" ChatGPT out of its safety protocols.Formal verification is a better approach because it provides mathematical guarantees (Russell, 2023;Urban & Miné, 2021).Formal verification provides something comparable to the MTTF proofs for nuclear power plant designs.With current systems, dangerous capabilities can arise unpredictably, and it is inherently difficult to exhaustively test models for all possible capabilities (Anderljung et al., 2023).Developers should provide formal demonstrations that their systems cannot autonomously replicate, as well as providing detection capabilities in case replication occurs via some previous unmodeled "side-channel" route.For example, the Center for AI and Digital Policy has called for "termination obligations" that would function like a circuit breaker in electrical power systems: if an AI system is not under human control then it must be terminated.AI systems should be designed in such a way as to be able to demonstrate compliance with common-sense regulations.Chip-based checking of proof-carrying code could provide the necessary safety guarantees as system capabilities continue to increase (Tegmark & Omohundro, 2023).Regulation can play a crucial role in incentivizing the creation and deployment of safe AI systems.
Third, "mandate independent monitoring": regulators must have the ability to monitor deployed systems and intervene if necessary.Like the FAA and the NRC, a robust AI regulator would be able to quickly recall unsafe products from the market.These interventions would be best enacted and overseen by a dedicated executive agency staffed with subject-matter experts, facilitating the implementation of technical advances in real-world systems (Amodei et al., 2016).For proprietary systems, this would be part of the licensing process; for open-source systems, this would require a nonremovable remote off switch to be included in every copy of the model.Systems should also be required to automatically self-register so the regulator knows that they exist.This is especially crucial given the opaque and unpredictable nature of current AI systems and the potential for unexpected behavior.
Overall, regulations should be implementable and verifiable, targeting parts of the AI pipeline that are not already being voluntarily addressed by developers.As elsewhere, the task of regulation and governance is to spur actions not already being taken voluntarily rather than codify the status quo.Consumers and citizens should have confidence that regulations are verifiably enforced.Inherently vague terms like "trust" and "safety" cannot form the basis for regulation.For instance, scholars have identified many significant difficulties of defining "trust" in the context of AI (Aoki, 2021;Laux et al., 2024).
By contrast, drawing clearly defined "red lines" that must not be crossed makes it possible to prove an AI system will not actually cross the red lines, regardless of context or user intent.This is the approach of the European Union's AI Act (Smuha, 2021) and the proposed treaty banning lethal autonomous weapons (Russell, 2022).Regulation can also compel developers to provide more information about the architecture, training data, and computing power behind their models. 13Establishing red lines through regulation places the onus on the developer to improve safety engineering capabilities.Example red lines might include attempts at self-replication, breaking into other computer systems, advising on bioweapons, or defamation of real individuals.Imposing red lines on obviously unacceptable system behavior will catalyze the development and deployment of AI systems that are safe by design in order to comply with these mandates.
In January 2023, the National Institute of Standards and Technology released its "risk-management framework" for AI.This voluntary framework lists a number of sensible goals for AI regulation (Kerry, 2023). 14The goal is to ensure that AI systems are valid and reliable; safe, secure, and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair with harmful bias managed.But nowhere does the document confront what makes AI both new and dangerous or how these laudable goals can be systematically implemented.Audits, certifications, and voluntary guidelines presuppose a method for determining if an AI system is in a condition for safe operation.However, it is not currently possible to assure a system abides by these principles.
Regulators cannot assume that this will happen by default.The lag between the introduction of new technology and federal regulation in the United States averages decades (Philbrick, 2023).We don't have decades to spare.There is an urgent need to build the regulatory paradigm and state capacity to meaningfully govern AI.As history shows, there is a constant struggle to shape the direction of technological development for collective welfare (Acemoglu & Johnson, 2023).Technical solutions are part of the puzzle but so are the underlying values and priorities of society and the capacity of nation states to implement real solutions (Burrell & Fourcade, 2021;Erman & Furendal, 2022;Gabriel, 2022).

Table 1 .
Downloaded from https://academic.oup.com/policyandsociety/advance-article/doi/10.1093/polsoc/puae020/7684910 by University of California Library -Berkeley Library user on 03 August 2024 Examples of the traditional model of regulation in the United States.