Real Safety AI Foundation / Research

Privacy, Surveillance, and Consent

Outside the Perimeter: Commercial Collection of Crisis Data and the Subpart That Was Never Written

Publisher: Real Safety AI Foundation

Working paper. Not peer reviewed.

Written
August 2026
Version
v1.2
Pages
18
Download PDFPlain language slides (PDF)Export BibTeXExport RIS

Abstract

American human subjects protection was built around institutions, because the abuses that produced it happened inside institutions. The Common Rule places its review requirement at the point where a covered institution touches a person. This paper argues that the location of that requirement, rather than its substance, is now the operative failure. When the regulated boundary is drawn around the institution, collection can be performed outside it by commercial and quasiinstitutional actors and then delivered inward as material that already exists, at which point the consent question that would have governed the collection is never presented to any review body. The paper names this pattern collection displacement and names the device that carries it the intermediary shield. It traces a documented fortyeight year refusal to write a subpart for adults whose decisional capacity is uncertain or impaired, from the National Commission in 1978 through the National Bioethics Advisory Commission in 1998 to the revised Common Rule of 2018, and shows that even the subpart repeatedly asked for would not reach the present problem, because the collection is performed by parties the rule does not cover. Two instances are examined: the Crisis Text Line and Loris.ai arrangement disclosed in 2022, and a current arrangement in which an unincorporated and then newly incorporated organization solicited conversation records from people in acute psychiatric crisis and supplied them to researchers at multiple universities. The conclusion is that the population most likely to be recruited during a mental health emergency is the one population with no dedicated regulatory protection, and that the collection reaching them now sits outside the perimeter the regulations govern.

Keywords

  • research ethics
  • human subjects protection
  • collection displacement
  • intermediary shield
  • institutional review board
  • decisional capacity
  • secondary data use
  • data brokerage
  • psychiatric crisis
  • informed consent
  • digital mental health
  • Common Rule

Plain language slides

First slide of the plain language summary of Outside the Perimeter: Commercial Collection of Crisis Data and the Subpart That Was Never WrittenOpen the 29-slide summary (PDF)

Suggested citation

Gilly, Travis. "Outside the Perimeter: Commercial Collection of Crisis Data and the Subpart That Was Never Written." Real Safety AI Foundation Working Paper, August 2026. https://realsafetyai.org/research/outside-the-perimeter/

Other versions

This paper is also posted on SSRN.

SSRN version

References (28)

  1. Abdillahi, I. (2024). Algorithmic surveillance in the era of the mental health appsphere. American Journal of Community Psychology, 73(1–2), 17–26. https://doi.org/10.1002/ajcp.12743
  2. Beard, G. M. (1881). American nervousness: Its causes and consequences. G. P. Putnam’s Sons.
  3. Beecher, H. K. (1966). Ethical problems and clinical research. New England Journal of Medicine, 274(24), 1354–1360.
  4. Botkin, J. R. (2017). Transparency and choice in learning healthcare systems. Learning Health Systems, 2(1). https://doi.org/10.1002/lrh2.10049
  5. Burris, S. (2008). Regulatory innovation in the governance of human subjects research: A cautionary tale and some modest proposals. Regulation & Governance, 2(1), 65–84. https://doi.org/10.1111/j.1748-5991.2007.00025.x
  6. Cantril, H. (1940). The invasion from Mars: A study in the psychology of panic. Princeton University Press.
  7. Crisis Text Line. (2022, January 31). An update on data privacy, our community and our service. https://www.crisistextline.org/blog/2022/01/31/an-update-on-data-privacy-our-community-and-our-service/
  8. Enos, G. (2023). Duke public policy report suggests federal law restricting data brokers. Mental Health Weekly, 33(10), 1–7. https://doi.org/10.1002/mhw.33562
  9. Eysenbach, G. (2025). Crisis Text Line and Loris.ai controversy highlights the complexity of informed consent on the internet and data-sharing ethics for machine learning and research. Journal of Medical Internet Research, 27, e67878.
  10. Kim, J. (2023). Data brokers and the sale of Americans’ mental health data. Duke University Sanford School of Public Policy, Cyber Policy Program.
  11. Friesen, P., Douglas-Jones, R., Marks, M., Pierce, R., Fletcher, K., Mishra, A., Lorimer, J., Véliz, C., Hallowell, N., Graham, M., Chan, M. S., Davies, H., & Sallamuddin, T. (2021). Governing AI-driven health research: Are IRBs up to the task? Ethics & Human Research, 43(2), 35–42. https://doi.org/10.1002/eahr.500085
  12. Hall, M. A., & King, N. M. (2016). Conscience, courage, and “consent.” Hastings Center Report, 46(2), 30–32. https://doi.org/10.1002/hast.546
  13. Horizon Omega. (2026). Etienne Brisson (The Human Line Project) at the Montreal AI Safety, Ethics and Governance Event Series [Audio recording of public livestream]. Internet Archive. https://archive.org/details/brisson-human-line-project-montreal-ai-safety-2026_202608
  14. Livingstone, S., Orben, A., & Odgers, C. (2022). Debate: Should academics collaborate with digital companies to improve young people’s mental health? Child and Adolescent Mental Health, 28(1), 150–152. https://doi.org/10.1111/camh.12630
  15. Lynch, H. F., Bierer, B. E., & Cohen, I. G. (2016). Confronting biospecimen exceptionalism in proposed revisions to the Common Rule. Hastings Center Report, 46(1), 4–5. https://doi.org/10.1002/hast.528
  16. Moore, J., Haber, N., et al. (2026). Characterizing delusional spirals through human-LLM chat logs. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency.
  17. National Bioethics Advisory Commission. (1998). Research involving persons with mental disorders that may affect decisionmaking capacity.
  18. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979). The Belmont Report: Ethical principles and guidelines for the protection of human subjects of research. Department of Health, Education, and Welfare.
  19. Nock, M. K., Kleiman, E. M., Abraham, M., Bentley, K. H., Brent, D. A., Buonopane, R. J., et al. (2020). Consensus statement on ethical and safety practices for conducting digital monitoring studies with people at risk of suicide and related behaviors. Psychiatric Research and Clinical Practice, 3(2), 57–66. https://doi.org/10.1176/appi.prcp.20200029
  20. Pisani, A. R., Gallo, C., Gould, M. S., Kanuri, N., Marcotte, J. E., Pascal, B., Rousseau, D., Ranney, M. L., Filbin, B., & Turner, S. (2025). Authors’ reply: Commentary on protecting user privacy and rights in academic data-sharing partnerships. Journal of Medical Internet Research, 27, e59734.
  21. Pooley, J., & Socolow, M. (2013). The myth of the War of the Worlds panic. Slate.
  22. Terry, N. (2015). Developments in genetic and epigenetic data protection in behavioral and mental health spaces. Behavioral Sciences & the Law, 33(5), 653–661. https://doi.org/10.1002/bsl.2203
  23. Tilley, C. L. (2012). Seducing the innocent: Fredric Wertham and the falsifications that helped condemn comics. Information and Culture, 47(4), 383–413.
  24. Tovino, S. A. (2013). A “common” proposal. Houston Law Review, 50, 787.
  25. United States Department of Health and Human Services. (2007). Request for information and comments on research that involves adult individuals with impaired decision-making capacity. Federal Register, 72, 50966.
  26. United States Department of Health and Human Services. (2018). Protection of human subjects, 45 C.F.R. pt. 46.
  27. Wertham, F. (1954). Seduction of the innocent. Rinehart.
  28. White, T., Blok, E., & Calhoun, V. D. (2020). Data sharing and privacy issues in neuroimaging research: Opportunities, obstacles, challenges, and monsters under the bed. Human Brain Mapping, 43(1), 278–291. https://doi.org/10.1002/hbm.25120

All research