I have posted a new article that represents the coming together of parallel strands of thinking I've been engaged in for the last few years. Here's an audio version (omitting most footnotes).

The article, Artificial Legal Agents and the Alien Stance, can be downloaded here: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6305138

Abstract: We find ourselves ensnared in an ever nastier and strident debate over AI and its role in law, society, and private life. Derided as mindless math or a stochastic parrot, AI is often shrugged off as just a bad tool, perhaps even the latest tech bro obsession after crypto and NFTs. I argue that this sort of critique is dangerously misguided, not because AI is a god and ought to be deployed with abandon, but because it distracts us from the real problem.

Using evidence from mechanistic interpretability researchers, cognitive scientists, and philosophers of mind, I argue (a) that modeling is the DNA of thought, (b) that LLMs think, in ways individual to each model, through models of language that work by traversing ecosystems of concepts, (c) that we think in roughly this same predictive fashion, and (d) that, while our manner of thinking is more or less the same, our conceptual ecosystems are alien to one another. This is a problem, because simulating our output has been trained into LLMs. When we interact, they are liable to activate in us what Daniel Dennett described as “the intentional stance,” when what is needed is the “alien stance.”

The “alien stance” is a socially supported skeptical posture toward AI interactions. This skepticism requires social support because, unlike the intentional stance, it does not come naturally to us. I argue that one such application of the alien stance is the creation of “analog zones,” where AI is excluded entirely from certain areas of legal practice. But whether one sees the danger to which the analog zone would be responsive turns on whether one views the law as a more or less formal effort to arrive at correct answers to legal questions or as an extended act of collective intentional conversation. Adherents to the latter view ought to fear that an ever-increasing flood of AI-generated legal argument risks an initially subtle but inevitable loss of control over legal content, defeating the point of legal practice.