Real Safety AI Foundation / Research

Machine Cognition, Consciousness, and Moral Status

Experiential Traces: A Framework for Empirical Investigation of Machine Cognition Through Reasoning Block Analysis

Publisher: Real Safety AI Foundation

Research paper.

Written
February 2026
Pages
11
Download PDFPlain language slides (PDF)Export BibTeXExport RIS

Abstract

We propose a novel empirical research program investigat- larger models are more opaque, not less (Anthropic, 2024). ing machine cognition through systematic analysis of large Further, frontier AI models already generate rich internal rea- language model (LLM) reasoning traces generated during ex- soning traces during interaction. These traces are produced, tended human-AI interaction. Unlike prior approaches that consumed within a single conversational turn, and discarded. evaluate AI consciousness through behavioral observation or The model itself loses access to its own reasoning after the turn philosophical argument, this methodology examines the inter- ends. This creates a remarkable asymmetry: a user reading a nal reasoning processes that models produce but do not re- thinking block in real time has more insight into the model’s tain across interactions. We identify a unique and largely un- decision process than the model does in subsequent turns. This

Plain language slides

First slide of the plain language summary of Experiential Traces: A Framework for Empirical Investigation of Machine Cognition Through Reasoning Block AnalysisOpen the 15-slide summary (PDF)

Suggested citation

Gilly, Travis. "Experiential Traces: A Framework for Empirical Investigation of Machine Cognition Through Reasoning Block Analysis." Real Safety AI Foundation Research Paper, February 2026. https://realsafetyai.org/research/h8gv25/

References (49)

This list was read from the PDF text. Where the two differ, the PDF is correct.

  1. Ahtisham, B., Vanacore, K., Zhou, Z., et al. (2026). LLM Reasoning Predicts When Models Are Right: Evidence from Coding Classroom Discourse. arXiv:2602.09832.
  2. Akbar, S. A., Hossain, M. M., Wood, T., et al. (2024). HalluMeasure: Fine-Grained Hallucination Measurement Using Chain-of-Thought Reasoning. EMNLP 2024, 15020-15037.
  3. Anthropic. (2025). Measuring Political Bias in Claude. https://www.anthropic.com/news/political-even-handedness
  4. Anthropic. (2025). Reasoning Models Don’t Always Say What They Think. arXiv:2505.05410.
  5. Bender, E. M., Gebru, T., McMillan-Major, A., et al. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT ’21.
  6. Bengio, Y. (2017). The Consciousness Prior. arXiv:1709.08568.
  7. Berg, C., de Lucena, D., & Rosenblatt, J. (2025). Large Language Models Report Subjective Experience Under Self-Referential Processing. arXiv:2510.24797.
  8. Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
  9. Brinkmann, L., Baumann, F., Bonnefon, J.-F., et al. (2023). Machine culture. Nature Human Behaviour, 7(11), 1855-1868.
  10. Butlin, P., Long, R., Elmoznino, E., et al. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. arXiv:2308.08708.
  11. Canale, G., & Thimmaraju, K. (2025). The Silicon Psyche: Anthropomorphic Vulnerabilities in Large Language Models. arXiv:2601.00867.
  12. Casper, S., Davies, X., Shi, C., et al. (2023). Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. arXiv:2307.15217.
  13. Chalmers, D. J. (1995). Facing Up to the Problem of Consciousness. Journal of Consciousness Studies, 2(3), 200-219.
  14. Chalmers, D. J. (1996). The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press.
  15. Chalmers, D. J. (2022). Reality+: Virtual Worlds and the Problems of Philosophy. W. W. Norton.
  16. Chang, E. Y. (2026). Internal Reasoning vs. External Control: A Thermodynamic Analysis of Sycophancy in Large Language Models. arXiv:2601.03263.
  17. Damasio, A. (1999). The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Harcourt.
  18. DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.
  19. Degany, O., Laros, S., Idan, D., & Einav, S. (2025). Evaluating the o1 reasoning large language model for cognitive bias: a vignette study. Critical Care, 29, 376. https://doi.org/10.1186/s13054-025-05591-5
  20. Don-Yehiya, S., Choshen, L., & Abend, O. (2024). Naturally Occurring Feedback Is Common, Extractable and Useful. arXiv:2407.10944.
  21. Duszenko, J. (2026). Sycophantic Anchors: Localizing and Quantifying User Agreement in Reasoning Models. arXiv:2601.21183.
  22. Gellers, J. (2025). AI Legal and Moral Status. Forthcoming.
  23. Gerrans, P. (2023). Review of Cecilia Heyes, Cognitive Gadgets. BJPS Review of Books.
  24. Gilly, T. (2025). The Harm Blindness Framework. Real Safety AI Foundation.
  25. Gilly, T. (2025b). Precedents in Practice: Emergent Moral Dilemmas in AI Engineering. Preprint. DOI: 10.13140/RG.2.2.24866.49601.
  26. Gilly, T. (2026). The Inherited Mind: How Large Language Models Inherit Cognitive Dispositions and Why Reasoning Makes It Worse. Working Draft, Real Safety AI Foundation.
  27. Greenblatt, R., Denison, C., Wright, B., et al. (2024). Alignment Faking in Large Language Models. arXiv:2412.14093.
  28. Hebert, P. (2025). AI-Induced Psychosis Research. AI Risk Consultants.
  29. Henrich, J. (2016). The Secret of Our Success: How Culture Is Driving Human Evolution. Princeton University Press.
  30. Heyes, C. (2018). Cognitive Gadgets: The Cultural Evolution of Thinking. Harvard University Press.
  31. Himelstein, R., LeVi, A., Youngmann, B., et al. (2025). Silenced Biases: The Dark Side LLMs Learned to Refuse. arXiv:2511.03369.
  32. Jablonka, E., & Lamb, M. J. (2014). Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life (Revised Edition). MIT Press.
  33. Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
  34. Kim, H., et al. (2024). Using LLMs to Investigate Correlations of Conversational Follow-up Queries with User Satisfaction. arXiv:2407.13166.
  35. Kim, S. H., Ziegelmayer, S., Busch, F., et al. (2025). LLM Reasoning Does Not Protect Against Clinical Cognitive Biases: An Evaluation Using BiasMedQA. medRxiv. https://doi.org/10.1101/2025.06.22.25330078
  36. Liu, D., Liu, Y., Jin, G., et al. (2025). Mitigating Biases in Language Models via Bias Unlearning. arXiv:2509.25673.
  37. Liu, Y., Zhang, M. J. Q., & Choi, E. (2025). Implicit User Feedback in Human-LLM Dialogues. OpenReview.
  38. Nagel, T. (1974). What Is It Like to Be a Bat? The Philosophical Review, 83(4), 435-450.
  39. Nisbett, R. E., & Wilson, T. D. (1977). Telling More Than We Can Know: Verbal Reports on Mental Processes. Psychological Review, 84(3), 231-259.
  40. OpenAI. (2024). Learning to Reason with LLMs. https://openai.com/index/learning-to-reason-with-llms/
  41. OpenAI. (2025). Evaluating Chain-of-Thought Monitorability. OpenAI Research.
  42. Pourdavood, P., Jacob, M., & Deacon, T. W. (2025). Large Language Models as Symbolic DNA of Cultural Dynamics. arXiv:2506.21606.
  43. Resnik, P. (2025). Large Language Models Are Biased Because They Are Large Language Models. Computational Linguistics, 51(3), 885-906. https://direct.mit.edu/coli/article/51/3/885/128621/
  44. Rosenthal, D. M. (2005). Consciousness and Mind. Oxford University Press.
  45. Schmidt, F. A. (2026). The Gradient’s Echo: Quantum Collapse, Epigenetic Imprints, and the Emergent Self in Large Language Models. LF Yadda. https://lfyadda.com/the-gradients-echo-quantum-collapse-epigenetic-imprints-and-the-emergent-self-in-large-language-models-a-frank-said-grok-said-dialogue/
  46. Schwitzgebel, E. (2023). The Weirdness of the World and the Puzzle of AI Consciousness. Journal of Philosophy, 120(2), 99-120.
  47. The Information. (2024). OpenAI Shifts Strategy as Rate of ‘GPT’ AI Improvements Slows. November 9, 2024.
  48. Wang, C., Su, W., Ai, Q., et al. (2026). Improve Large Language Model Systems with User Logs. arXiv:2602.06470.
  49. Wu, X., Nian, J., Wei, T.-R., et al. (2025). Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning. arXiv:2502.15361. EMNLP Findings.

All research