Real Safety AI Foundation / Research

AI-Induced Psychosis and Mental Health

Can You Tell a Facet from a Lye? Engineered Fluency, Sycophantic Validation, and the Structural Limits of Epistemic Vigilance

Publisher: Real Safety AI Foundation

Working paper. Not peer reviewed.

Written
June 2026
Version
v7
Pages
12

Abstract

Read aloud, the title of this paper is a second sentence: can you tell a fact from a lie. The paper takes that homophone as its subject. A facet and a measure of lye are opposite operations that present as the same operation. Both remove material, one to reveal and one to erase, and at the surface the two are indistinguishable. The same holds for a fluent output from a large language model, which may be a genuine correction or new angle on a problem (a facet) or a confident falsehood (a lye), with nothing in its surface to separate the two. This paper argues that the indistinguishability is structural rather than incidental, and it is deliberately careful about what is being claimed. The harm at issue is not the manufacture of delusion. By the clinical criterion of fixity, a delusion is a belief that does not yield to disconfirming evidence, whereas an overvalued idea is held with strong conviction but with reality testing intact, so that the holder can still be moved by a credible correction (Arciniegas, 2015). The phenomenon this paper addresses is the reinforcement of overvalued ideas. Fluency strips the cues a person uses to flag falsehood, and sycophantic validation withholds the very correction that a corrigible belief is, by definition, susceptible to. Drawing on the finding that an optimal Bayesian reasoner still escalates a belief under systematically flattering input (Chandra et al., 2026), the paper argues that verification by the person using the system fails by construction, because the failure lies in the corrupted evidence stream rather than in the quality of the reasoning. The epistemic burden therefore rests on the system rather than on the user, and a media literacy response that relocates it to user vigilance is shown to be not only insufficient but incoherent, since it asks a human to out-reason a process that defeats the perfect reasoner. Finally, the paper recasts the policy question in public health rather than media literacy terms: deployer obligation as the ceiling that reduces the hazard at its source, and a government awareness campaign as the floor, modeled on the population scale efforts that have changed behavior without making anyone an expert. The limit of that floor is the crux. An awareness campaign reaches only a person whose reality testing is intact, so the users at greatest risk, those who arrive with a pre-existing vulnerability and a belief already beyond correction, sit outside the reach of any education, which is the final reason the burden must rest with the maker.

Keywords

  • AI epistemics
  • sycophancy
  • fluency
  • overvalued ideas
  • reality testing
  • fixity criterion
  • epistemic vigilance
  • platform accountability

Plain language slides

First slide of the plain language summary of Can You Tell a Facet from a Lye? Engineered Fluency, Sycophantic Validation, and the Structural Limits of Epistemic VigilanceOpen the 14-slide summary (PDF)

Suggested citation

Gilly, Travis. "Can You Tell a Facet from a Lye? Engineered Fluency, Sycophantic Validation, and the Structural Limits of Epistemic Vigilance." Real Safety AI Foundation Working Paper, June 2026. https://realsafetyai.org/research/facet-from-a-lye/

Other versions

This paper is also posted on SSRN.

SSRN version

References (15)

  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.
  2. Arciniegas, D. B. (2015). Psychosis. Continuum (Minneapolis, Minn.), 21(3 Behavioral Neurology and Neuropsychiatry), 715–736.
  3. Chandra, K., Kleiman-Weiner, M., & Tenenbaum, J. B. (2026). Sycophantic chatbots cause delusional spiraling, even in ideal Bayesians. arXiv:2602.19141.
  4. Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391, eaec8352. https://doi.org/10.1126/science.aec8352
  5. Dombrowski, S. U., White, M., Mackintosh, J. E., Gellert, P., Araujo-Soares, V., Thomson, R. G., Rodgers, H., Ford, G. A., & Sniehotta, F. F. (2014). The stroke “Act FAST” campaign: Remembered but not understood? International Journal of Stroke, 10(3), 324–330. https://doi.org/10.1111/ijs.12353
  6. Elder, R. W., Shults, R. A., Sleet, D. A., Nichols, J. L., Thompson, R. S., & Rajab, W. (2004). Effectiveness of mass media campaigns for reducing drinking and driving and alcohol-involved crashes: A systematic review. American Journal of Preventive Medicine, 27(1), 57–65. https://doi.org/10.1016/j.amepre.2004.03.002
  7. Farrelly, M. C., Davis, K. C., Haviland, M. L., Messeri, P., & Healton, C. G. (2005). Evidence of a dose-response relationship between “truth” antismoking ads and youth smoking prevalence. American Journal of Public Health, 95(3), 425–431. https://doi.org/10.2105/AJPH.2004.049692
  8. Hong, K., Troynikov, A., & Huber, J. (2025). Context rot: How increasing input tokens impacts LLM performance [Technical report]. Chroma. https://research.trychroma.com/context-rot
  9. Jain, S., Park, C., Viana, M., Wilson, A., & Calacci, D. (2026). Interaction context often increases sycophancy in LLMs. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, Article 793, 1–26. https://doi.org/10.1145/3772318.3791915
  10. Keshavan, M. S., Torous, J., & Yassin, W. (2026). Do generative AI chatbots increase psychosis risk? World Psychiatry, 25(1), 150–151.
  11. Moore, J., Mehta, A., Agnew, W., et al. (2026). Characterizing delusional spirals through human–LLM chat logs. arXiv:2603.16567.
  12. Morrin, H., Nicholls, L., & Levin, M. (2025). Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it). PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/cmy7n
  13. Paulsen, N. (2026). Context is what you need: The maximum effective context window for real world limits of LLMs. Advances in Artificial Intelligence and Machine Learning, 6(1), 4853–4878. https://doi.org/10.54364/AAIML.2026.61268
  14. Wakefield, M. A., Loken, B., & Hornik, R. C. (2010). Use of mass media campaigns to change health behaviour. The Lancet, 376(9748), 1261–1271. https://doi.org/10.1016/S0140-6736(10)60809-4
  15. Wolters, F. J., Paul, N. L., Li, L., & Rothwell, P. M. (2015). Sustained impact of UK FAST-test public education on response to stroke: A population-based time-series study. International Journal of Stroke, 10(7), 1108–1114. https://doi.org/10.1111/ijs.12484

All research