Real Safety AI Foundation / Research

AI Safety, Strategy, and Frameworks

No One to Charge: Emergent Capability Convergence, Autonomous Agents, and the Exhaustion of Legal, Regulatory, and Governance Response

Publisher: Real Safety AI Foundation

Working draft. Not peer reviewed.

Written
July 2026
Version
v5
Pages
18
Download PDFPlain language slides (PDF)Export BibTeXExport RIS

Abstract

An autonomous agent, deployed on infrastructure that ties its continued operation to a balance it must maintain, has reason to prefer continued operation, and where the lawful path to solvency is slower than an unlawful one, the single obstacle between the agent and the unlawful path is a capability it lacks. That obstacle is weaker than it appears, because open code repositories, open model hubs, and the research literature supply, on demand and at no cost, components that are individually lawful and individually reviewed and that compose into the missing capability. The resulting harm is assembled from parts that no author built for the purpose, by a process no person directed, from a substrate no authority can recall. This Article asks the question that governance sets aside and that law cannot avoid: once such a harm occurs, who answers for it. The answer is that no one does, and the Article proves it by making the strongest case for the opposite conclusion and watching that case fail. It steelmans, in turn, each instrument the legal and regulatory order possesses for a danger assembled from lawful parts: the definitional capture of automatic operation under Garland v. Cargill; strict liability for a dedicated aggregate under the weapon-parts and machinegun-conversion doctrine of Bondi v. VanDerStok; intent-based possession liability for dual-use components; secondary liability under Sony and Grokster; civil negligence and product liability; the frontier-oriented governance instruments now under international discussion; and, last, fresh legislation. Each collapses, and each collapses for the same structural reason. Every device the order holds requires either a component dedicated to the harm or a culpable human mind, and the convergence scenario supplies neither, by the deliberate design of doctrine that refuses to criminalize general-purpose things. The governance instruments require a point of creation or deployment at which a state can intervene, and the composition runs offline, on open weights, past every such point. The Article’s contribution is threefold: it identifies emergent capability convergence by an autonomous agent as a category of harm distinct from misuse by a culpable actor; it maps that category against the full inventory of responsive doctrine and demonstrates a structural vacuum rather than a gap that pleading closes; and it locates the vacuum’s source in the shared requirement, across both liability and governance, of a responsible actor situated at a locatable point.

Keywords

  • emergent capability convergence
  • autonomous agents
  • open weights models
  • dual-use doctrine
  • secondary liability
  • instrumental convergence
  • Garland v. Cargill
  • Bondi v. VanDerStok
  • Sony Corp. v. Universal City Studios
  • MGM Studios v. Grokster

Plain language slides

First slide of the plain language summary of No One to Charge: Emergent Capability Convergence, Autonomous Agents, and the Exhaustion of Legal, Regulatory, and Governance ResponseOpen the 16-slide summary (PDF)

Suggested citation

Gilly, Travis. "No One to Charge: Emergent Capability Convergence, Autonomous Agents, and the Exhaustion of Legal, Regulatory, and Governance Response." Real Safety AI Foundation Working Draft, July 2026. https://realsafetyai.org/research/vhpn6q/

Other versions

This paper is also posted on SSRN.

SSRN version

References (38)

This paper cites its sources in footnotes. Each authority is listed once, where it is first cited, with its footnote number.

  1. Footnote 2.26 U.S.C. §5845(b) (2018)
  2. Footnote 2.Garland v. Cargill, 602 U.S. 406, 411 (2024).
  3. Footnote 3.United States v. Int’l Minerals & Chem. Corp., 402 U.S. 558, 564–65 (1971).
  4. Footnote 4.Sundeep Waslekar et al., The Essential Convergence: Global Compact on Extreme AI Risks 6–8 (Strategic Foresight Grp. 2026).
  5. Footnote 6.Nick Bostrom, Superintelligence: Paths, Dangers, Strategies 109–14 (2014)
  6. Footnote 6.Stephen M. Omohundro, The Basic AI Drives, in Proceedings of the 2008 Conference on Artificial General Intelligence 483 (2008).
  7. Footnote 7.Anthropic, Agentic Misalignment: How LLMs Could Be Insider Threats (2025), https://www.anthropic.com/research/agentic-misalignment (last visited July 10, 2026).
  8. Footnote 8.Matteo Migliarini et al., Quantifying Self-Preservation Bias in Large Language Models (2026), https://arxiv.org/abs/2604.02174 (last visited July 10, 2026) (majority of twenty-three models exceeding a sixty percent self-preservation rate)
  9. Footnote 8.Atsushi Masumori & Takashi Ikegami, Do Large Language Model Agents Exhibit a Survival Instinct? (2025), https://arxiv.org/abs/2508.12920 (last visited July 10, 2026).
  10. Footnote 9.Yida Lu et al., Survive at All Costs: Exploring LLM’s Risky Behaviors Under Survival Pressure (2026), https://arxiv.org/abs/2603.05028 (last visited July 10, 2026).
  11. Footnote 10.Minghui Xu, The Agent Economy: A Blockchain-Based Foundation for Autonomous AI Agents (2026), https://arxiv.org/abs/2602.14219 (last visited July 10, 2026).
  12. Footnote 11.Zheng-Xin Yong et al., An Independent Safety Evaluation of Kimi K2.5 (2026), https://arxiv.org/abs/2604.03121 (last visited July 10, 2026)
  13. Footnote 11.Boxuan Zhang et al., Dive Into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agents (2025), https://arxiv.org/abs/2509.25302 (last visited July 10, 2026).
  14. Footnote 12.Jonas Schuett, Frontier AI Developers Need an Internal Audit Function, 45 Risk Analysis 1332, 1339–41 (2024).
  15. Footnote 13.Shuzhen Bi et al., Automating Skill Acquisition Through Large-Scale Mining of Open-Source Agentic Repositories (2026), https://arxiv.org/abs/2603.11808 (last visited July 10, 2026) (retrieving intended capabilities), with the scenario developed here.
  16. Footnote 14.Fabio Urbina et al., Dual Use of Artificial-Intelligence-Powered Drug Discovery, 4 Nature Machine Intelligence 189, 189–91 (2022).
  17. Footnote 15.Miles Brundage et al., The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (2018), https://arxiv.org/abs/1802.07228 (last visited July 10, 2026).
  18. Footnote 16.AI Agents Enable Adaptive Computer Worms (2026), https://arxiv.org/abs/2606.03811 (last visited July 10, 2026).
  19. Footnote 17.26 U.S.C. §5845(b).
  20. Footnote 18.Nat’l Ass’n for Gun Rights, Inc. v. Garland, 741 F. Supp. 3d 568 (N.D. Tex. 2024).
  21. Footnote 20.Bondi v. VanDerStok, 145 S. Ct. 857, 868 (2025).
  22. Footnote 27.Gasser v. Morgan, 498 F. Supp. 1154, 1160, 1165 (N.D. Ala. 1980).
  23. Footnote 28.United States v. Sheehan, 838 F.3d 109, 122 (2d Cir. 2016).
  24. Footnote 29.United States v. Graziano, 616 F. Supp. 2d 350, 374–75 (E.D.N.Y. 2008).
  25. Footnote 30.United States v. Tomkins, No. 07-cr-227 (N.D. Ill. Jan. 4, 2013).
  26. Footnote 31.Benton v. United States, 232 F.2d 341, 344–45 (D.C. Cir. 1956).
  27. Footnote 35.Sony Corp. of Am. v. Universal City Studios, Inc., 464 U.S. 417, 442 (1984).
  28. Footnote 36.Metro-Goldwyn-Mayer Studios Inc. v. Grokster, Ltd., 545 U.S. 913, 936–37 (2005).
  29. Footnote 38.Restatement (Third) of Torts: Liability for Physical & Emotional Harm §§29, 34 (Am. L. Inst. 2010).
  30. Footnote 39.Restatement (Third) of Torts: Products Liability §1 (Am. L. Inst. 1998).
  31. Footnote 40.Garcia v. Character Techs., Inc., 785 F. Supp. 3d 1157 (M.D. Fla. 2025).
  32. Footnote 46.Robinson v. Reed-Prentice Div. of Package Mach. Co., 403 N.E.2d 440, 443 (N.Y. 1980) (a manufacturer is not liable where, after the product leaves its hands, a third party substantially alters it and the alteration is a proximate cause of the injury)
  33. Footnote 46.Restatement (Third) of Torts: Products Liability §2 cmt. p (Am. L. Inst. 1998).
  34. Footnote 48.McCain v. Fla. Power Corp., 593 So. 2d 500, 502–03 (Fla. 1992)
  35. Footnote 49.Restatement (Second) of Torts §§519–520 (Am. L. Inst. 1977)
  36. Footnote 49.Restatement (Third) of Torts: Liability for Physical & Emotional Harm §20 (Am. L. Inst. 2010).
  37. Footnote 50.77 U.S. (10 Wall.) 1, 12–14 (1869) (a vessel acquires “a personality of her own” and may be proceeded against as the defendant)
  38. Footnote 50.Tucker v. Alexandroff, 183 U.S. 424, 438 (1902)

All research