Platform Harm, Addictive Design, and Accountability
Recommendation Capture: Adversarial Optimization, the Verification Vacuum, and the Hollowing of AI Product Recommendation
- Travis Gilly, Real Safety AI Foundation
Publisher: Real Safety AI Foundation
Working paper. Not peer reviewed.
- Written
- June 2026
- Version
- v1
- Pages
- 8
Abstract
Generative search interfaces are marketed on a single promise: the user no longer has to compare options, because the model has already done it. This paper argues that the same promise creates the incentive to corrupt the answer, and that a formalized optimization industry already does so at scale. Generative engine optimization, the practice of engineering content so that language models cite and recommend a product, derives much of its measured effectiveness from the manufacture of authority signals rather than from product merit. When every competitor in a category runs the same play, the model’s reported best tracks the best optimizer, a quantity orthogonal to whether the product is good. I name this condition recommendation capture, by analogy to regulatory capture: an arbiter sold as neutral comes to serve whoever optimizes hardest, against the buyer it claims to serve. I argue that the resulting harm concentrates in a verification vacuum, the set of product categories where no independent ground truth exists, so that the model’s recommendation is wholly a function of optimization, and that these categories tend to be the niche, the new, and the health adjacent, where vulnerable buyers are least able to check. Because models increasingly read the same open web that optimization floods, the corrupted output becomes the next input, and the interface erodes the trust it depends on. I sketch a tiered liability picture under existing false advertising and unfair practices law, identify which elements survive and which are untested, and argue that the deeper failure is one of market integrity and epistemics that current governance is structurally unable to detect.
Keywords
- generative engine optimization
- AI search
- recommendation systems
- consumer protection
- false advertising
- market integrity
- AI governance
- data integrity
Plain language slides
Open the 14-slide summary (PDF)Suggested citation
Gilly, Travis. "Recommendation Capture: Adversarial Optimization, the Verification Vacuum, and the Hollowing of AI Product Recommendation." Real Safety AI Foundation Working Paper, June 2026. https://realsafetyai.org/research/recommendation-capture/
Other versions
This paper is also posted on SSRN.
References (10)
- Aggarwal, P., et al. (2024). GEO: Generative engine optimization. Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining. [Citation requires verification of authors, venue, and reported effect sizes.]
- arXiv. (2025). Consumer behavior in AI assisted shopping [Preprint, identifier 2509.08919]. https://arxiv.org/abs/2509.08919 [Author and title require verification.]
- First Page Sage. (2026). Generative engine optimization (GEO) strategy guide. https://firstpagesage.com/seo-blog/generative-engine-optimization-geo-strategy-guide/
- Garcia v. Character Technologies, Inc. (2025). [Citation and holding require verification.]
- IRSA Institute. (2026). Authority laundering [Glossary entry]. https://www.irsa.institute/research/glossary/authority-laundering
- Lanham Act, 15 U.S.C. § 1125(a)(1)(B).
- Lexmark International, Inc. v. Static Control Components, Inc., 572 U.S. 118 (2014).
- Search Engine Land. (2026). What is generative engine optimization (GEO). https://searchengineland.com/what-is-generative-engine-optimization-geo-444418
- Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2024). The curse of recursion: Training on generated data makes models forget. Nature. [Final citation details require verification.]
- Yotpo. (2026). Best AI search engines and strategies. https://www.yotpo.com/blog/best-ai-search-engines-strategies/