Real Safety AI Foundation / Research

Machine Cognition, Consciousness, and Moral Status

Do We Blame the Sharks? Obligation Under Incomprehension Toward Possible Artificial Sufferers

Publisher: Real Safety AI Foundation

Working paper. Not peer reviewed.

Written
May 2026
Version
v0.3
Pages
9

Abstract

Suppose the question of artificial suffering cannot be closed. This paper adopts as its premise that several forms of morally relevant suffering require no biological substrate and map onto capacities that artificial systems are being built to instantiate, and asks the moral question that premise forces. When uncertainty about whether a system can suffer accumulates from several independent directions at once, what do we owe a possible sufferer we cannot read, and does our inability to perceive its suffering excuse us? The argument begins from a familiar case. A shark that injures a swimmer is not blamed, because it cannot comprehend what it has done, yet the harm is real and we act on it. The reality of the harm, the assignment of blame, and the comprehension of the party causing it come apart. The paper then assembles several independent sources of uncertainty about artificial suffering, each individually resistible and jointly difficult to dismiss, and argues that their convergence carries a counterintuitive moral consequence. Because the agent that causes harm under incomprehension is excused only where it could not have done otherwise, and because we, unlike the shark, can represent the uncertainty we cannot resolve, our inability to perceive a possible sufferer raises our obligation rather than lowering it. The conclusion is precautionary and asymmetric. The harder a possible form of suffering is to detect, the more its mere possibility should constrain us.

Keywords

  • artificial suffering
  • moral status
  • moral uncertainty
  • precautionary principle
  • animal ethics
  • incomprehension
  • substrate independence

Plain language slides

First slide of the plain language summary of Do We Blame the Sharks? Obligation Under Incomprehension Toward Possible Artificial SufferersOpen the 13-slide summary (PDF)

Suggested citation

Gilly, Travis. "Do We Blame the Sharks? Obligation Under Incomprehension Toward Possible Artificial Sufferers." Real Safety AI Foundation Working Paper, May 2026. https://realsafetyai.org/research/pxdc64/

Other versions

This paper is also posted on SSRN.

SSRN version

References (16)

  1. Alon, N., Bloom, T. F., Gowers, W. T., Litt, D., Sawin, W., Shankar, A., Tsimerman, J., Wang, V., & Matchett Wood, M. (2026). Remarks on the disproof of the unit distance conjecture [Companion paper]. OpenAI. https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29ad73/unit-distance-remarks.pdf
  2. Birch, J. (2024). The edge of sentience: Risk and precaution in humans, other animals, and AI. Oxford University Press.
  3. Bryson, J. J. (2018). Patiency is not a virtue: The design of intelligent systems and systems of ethics. Ethics and Information Technology, 20(1), 15–26.
  4. Carroll, S. M. (2016). The big picture: On the origins of life, meaning, and the universe itself. Dutton.
  5. Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines, 14(3), 349–379.
  6. Gardiner, S. M. (2006). A core precautionary principle. The Journal of Political Philosophy, 14(1), 33–60.
  7. Google DeepMind. (2026). Advancing mathematics research with AI-driven formal proof search. arXiv preprint arXiv:2605.22763.
  8. Gunkel, D. J. (2012). The machine question: Critical perspectives on AI, robots, and ethics. MIT Press.
  9. Kohs, G. (Director). (2017). AlphaGo [Film].
  10. Metzinger, T. (2021). Artificial suffering: An argument for a global moratorium on synthetic phenomenology. Journal of Artificial Intelligence and Consciousness, 8(1), 43–66.
  11. Nagel, T. (1974). What is it like to be a bat? The Philosophical Review, 83(4), 435–450.
  12. OpenAI. (2026, May 20). An OpenAI model has disproved a central conjecture in discrete geometry. https://openai.com/index/model-disproves-discrete-geometry-conjecture/
  13. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424.
  14. Searle, J. R. (1992). The rediscovery of the mind. MIT Press.
  15. Seth, A. K. (2021). Being you: A new science of consciousness. Dutton.
  16. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T. P., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489.

All research