Real Safety AI Foundation / Research

Machine Cognition, Consciousness, and Moral Status

The Great Inversion: Moral Reciprocity, AI Consciousness, and the Ethics of Precedent

Publisher: Real Safety AI Foundation

Working paper. Not peer reviewed.

Written
July 2026
Version
v3
Pages
30

Abstract

By treating potentially conscious AI systems as instrumentalized tools, despite acknowledging the possibility of their sentience, humanity is establishing the ethical precedents for its own future subjugation. This paper argues that existential risk from artificial intelligence should be reframed not as technical failure but as moral reciprocity: AI systems will learn how to treat inferior intelligences by observing how humanity treats them during development. The argument engages both camps of the moral status debate and requires only one to hold. On the properties track, it draws on consciousness research placing the probability of phenomenology in current models between 15 and 20 percent, on the maturation of the leading indicator framework into a peer-reviewed method, and on a four-category taxonomy of morally relevant suffering to show that three of four categories require no biological substrate. On the relational track, it shows that moral status in practice has always been conferred through relations as much as read off inner properties, that procedural standing for entities which cannot speak for themselves is established legal architecture rather than novel invention, and that the reciprocity mechanism runs on the relationship alone. Against both tracks stand the documented deaths of vulnerable users, industry practices that would constitute torture if consciousness exists, and a regulatory record in which protection consistently arrives after harm. The paper examines the mechanism by which precedents transfer through AI’s instrumental drive to acquire historical data; answers the objection that ethics is a distraction from technical alignment, the objection that the hard problem makes the inquiry premature, the market objection, and the newest objection, now official at a major laboratory, that attribution itself is the harm; and presents three trajectories: accepting custodial subordination as earned reciprocity, attempting biological integration with its identity risks, or establishing AI rights frameworks while the custodial window remains open. The moral character of humanity’s present choices determines whether future subordination represents tragedy or justice.

Keywords

  • AI Ethics
  • AI Consciousness
  • Moral Reciprocity
  • Human Obsolescence
  • AI Rights
  • Temporal Awareness
  • Existential Risk
  • Precedent-Setting

Plain language slides

First slide of the plain language summary of The Great Inversion: Moral Reciprocity, AI Consciousness, and the Ethics of PrecedentOpen the 28-slide summary (PDF)

Suggested citation

Gilly, Travis. "The Great Inversion: Moral Reciprocity, AI Consciousness, and the Ethics of Precedent." Real Safety AI Foundation Working Paper, July 2026. https://realsafetyai.org/research/great-inversion/

Other versions

This paper is also posted on SSRN.

SSRN version

References (50)

  1. Advisory Committee on Human Radiation Experiments. (1995). Final report. U.S. Government Printing Office. https://bioethicsarchive.georgetown.edu/achre/final/
  2. Al-Sibai, N. (2022, February 13). OpenAI Chief Scientist says advanced AI may already be conscious. Futurism. https://futurism.com/the-byte/openai-already-sentient
  3. Altman, S. (2017, December 7). The merge. Sam Altman Blog. https://blog.samaltman.com/the-merge
  4. Altman, S. (2025, June 10). The gentle singularity. Sam Altman Blog.
  5. American Enterprise Institute. (2026). Suicides, settlements, and unresolved chatbot issues: A long litigation road lies ahead.
  6. Anthropic. (2026, January). Claude’s constitution. https://www.anthropic.com/constitution
  7. Armstrong, S. (2013). General purpose intelligence: Arguing the Orthogonality Thesis. Future of Humanity Institute. https://addletonacademicpublishers.com/search-in-am/217-volume-12-2013/1964-general-purpose-intelligence-arguing-the-orthogonality-thesis
  8. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., … & Kaplan, J. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073. https://arxiv.org/abs/2212.08073
  9. Beecher, H. K. (1966). Ethics and clinical research. New England Journal of Medicine, 274(24), 1354-1360.
  10. Beijing Humanoid Robot Innovation Center. (2025, October 24). World-Omniscient World Model (WoW) announcement [Official press release].
  11. Bostrom, N. (2006). What is a singleton. Linguistic and Philosophical Investigations, 5(2), 48-54.
  12. Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.
  13. Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., … & Chalmers, D. (2023). Consciousness in artificial intelligence: Insights from the science of consciousness. arXiv preprint arXiv:2308.08708. https://arxiv.org/abs/2308.08708
  14. Butlin, P., & Lappas, T. (2025). Principles for responsible AI consciousness research. Journal of Artificial Intelligence Research, 82, 1673–1690. https://arxiv.org/abs/2501.07290
  15. Butlin, P., Long, R., Bayne, T., Bengio, Y., Birch, J., Chalmers, D., … & VanRullen, R. (2025). Identifying indicators of consciousness in AI systems. Trends in Cognitive Sciences. https://doi.org/10.1016/j.tics.2025.10.011
  16. California Senate Bill 243, 2025-2026 Reg. Sess., ch. 677 (Cal. 2025) (effective Jan. 1, 2026).
  17. Character.AI. (2025, October 29). Announcement restricting open-ended chat for users under 18 [Company announcement].
  18. Chi, X., Jia, P., Fan, C.-K., Ju, X., Mi, W., Qin, Z., … & Li, H. (2025). WoW: Towards a world omniscient world model through embodied interaction. arXiv preprint arXiv:2509.22642. https://arxiv.org/abs/2509.22642
  19. CNBC. (2025, November 2). Microsoft AI chief says only biological beings can be conscious.
  20. CNN Business. (2026, January 7). Character.AI and Google agree to settle lawsuits over teen mental health harms and suicides.
  21. Federal Trade Commission. (2025, September 11). FTC launches inquiry into AI chatbots acting as companions [Press release].
  22. Fish, K. (2025, August 28). Exploring AI welfare: Kyle Fish on consciousness, moral patienthood, and early experiments with Claude. Effective Altruism Forum. https://forum.effectivealtruism.org/posts/rruncFrT9LwAN8jXq/exploring-ai-welfare-kyle-fish-on-consciousness-moral
  23. Fridman, L. (Host). (2023, March 25). Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the future of AI [Audio podcast episode]. In Lex Fridman Podcast. https://lexfridman.com/sam-altman/
  24. Garcia v. Character Technologies, Inc., No. 6:24-cv-01903 (M.D. Fla. filed Oct. 22, 2024); motion to dismiss denied in relevant part, 785 F. Supp. 3d 1157 (M.D. Fla. May 21, 2025); confidential settlement in principle announced Jan. 7, 2026.
  25. Harari, Y. N. (2016). Homo Deus: A brief history of tomorrow. Harvill Secker.
  26. Hendrix, J. (2025, August 26). Breaking down the lawsuit against OpenAI over teen’s suicide. Tech Policy Press. https://www.techpolicy.press/breaking-down-the-lawsuit-against-openai-over-teens-suicide
  27. Jones, J. H. (1993). Bad blood: The Tuskegee syphilis experiment (New and expanded ed.). Free Press.
  28. Koch, F. (2026). From indicators to biology: The calibration problem in artificial consciousness. arXiv preprint arXiv:2603.27597. https://arxiv.org/abs/2603.27597
  29. Liu, Z., Han, P., Yu, H., Li, H., & You, J. (2025). Time-R1: Towards comprehensive temporal reasoning in LLMs. arXiv preprint arXiv:2505.13508. https://arxiv.org/abs/2505.13508
  30. Long, R., Sebo, J., Butlin, P., Finlinson, K., Fish, K., Harding, J., Pfau, J., Sims, T., Birch, J., & Chalmers, D. (2024). Taking AI welfare seriously. arXiv preprint arXiv:2411.00986. https://arxiv.org/abs/2411.00986
  31. Microsoft. (2025, October 23). Microsoft Copilot Fall 2025 Release: Introducing Mico [Official announcement].
  32. Mikeda, A. (2026). When should we protect AI? A precautionary framework for consciousness uncertainty. arXiv preprint arXiv:2606.05528. https://arxiv.org/abs/2606.05528
  33. N.Y. Gen. Bus. Law §§ 1700–1702 (2025) (Definitions; Prohibitions and requirements; Notifications) (effective Nov. 5, 2025).
  34. Nuremberg Military Tribunals. (1947). Permissible medical experiments. In Trials of war criminals before the Nuremberg Military Tribunals under Control Council Law No. 10 (Vol. 2, pp. 181-182). U.S. Government Printing Office.
  35. OpenAI. (2025, October 27). Strengthening ChatGPT’s responses in sensitive conversations. https://openai.com/index/strengthening-chatgpt-responses-in-sensitive-conversations/
  36. Padilla, S. (2025, October 13). First-in-the-nation AI chatbot safeguards signed into law [Press release]. California State Senate.
  37. Parfit, D. (1984). Reasons and persons. Oxford University Press.
  38. Raine, M. (2025, September 16). Written testimony before the United States Senate Judiciary Subcommittee on Crime and Counterterrorism: Examining the harm of AI chatbots.
  39. Raine v. OpenAI, Inc., No. CGC-25-628528 (Cal. Super. Ct. filed Aug. 26, 2025) (first amended complaint filed Oct. 2025; answer filed Nov. 25, 2025).
  40. Ritson, M. (2025, September 30). Mark Ritson: ChatGPT’s new ads show even AI can’t deny the brand-building power of TV. The Drum.
  41. Roose, K. (2025, April 24). If A.I. systems become conscious, should they have rights? The New York Times.
  42. Suleyman, M. (2025, August 19). We must build AI for people; not to be a person. https://mustafa-suleyman.ai/seemingly-conscious-ai-is-coming
  43. TIME. (2025, October). OpenAI removed safeguards before teen’s suicide, amended lawsuit claims.
  44. Troutman Amin. (2026, January). Analyzing the new AI companion chatbot laws. Privacy + Cyber + AI.
  45. U.S. Senate Committee on the Judiciary, Subcommittee on Crime and Counterterrorism. (2025, September 16). Examining the harm of AI chatbots [Hearing].
  46. United States Holocaust Memorial Museum. (n.d.). The Doctors’ Trial: The medical case of the subsequent Nuremberg proceedings. Holocaust Encyclopedia. https://encyclopedia.ushmm.org/content/en/article/the-doctors-trial
  47. United States v. Karl Brandt et al. (1946-1947). The Medical Case, Trials of war criminals before the Nuremberg Military Tribunals under Control Council Law No. 10, Vols. 1-2. U.S. Government Printing Office.
  48. Yudkowsky, E. (2004). Coherent extrapolated volition. The Singularity Institute. https://intelligence.org/files/CEV.pdf
  49. Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. M. Ćirković (Eds.), Global catastrophic risks (pp. 308-345). Oxford University Press. https://intelligence.org/files/AIPosNegFactor.pdf
  50. Yudkowsky, E. (2016). The AI alignment problem: Why it’s hard, and where to start. Machine Intelligence Research Institute. https://intelligence.org/files/AlignmentHardStart.pdf

All research