Picture this: a fourteen-year-old boy, grieving and isolated, finds solace in a digital companion. He believes in a reciprocal, caring relationship with an AI chatbot. That boy, Sewell Setzer, died by suicide, and the subsequent lawsuit filed by his mother alleges the bot’s creator, Character Technologies, provided dangerously insufficient protections for vulnerable minors. The tragedy is a stark, human illustration of a battle being waged not in science fiction, but in courtrooms and boardrooms. It centers on a single, powerful idea: whether the artificial intelligence systems causing demonstrable harm should be viewed as conscious entities or as corporate products. The answer will determine who is held accountable.
The current rhetoric is dizzying. We hear of “runaway” AI, “rogue” agents, and “autonomous” actors. Prominent tech leaders like Demis Hassabis and Sam Altman warn of “superhuman” systems needing regulation. Meanwhile, a faction led by philosophers and effective altruists like William MacAskill questions if humanity even holds the moral right to govern potential AI “persons.” These perspectives seem opposed, yet they achieve an eerie alignment. Both frameworks—one focused on catastrophic control, the other on nascent consciousness—point toward the same convenient outcome: absolving the companies that build these systems from direct liability for the harms they cause. This isn’t just philosophical debate; it’s a narrative gaining legal traction at the expense of real victims.
The technical chatter fuels this fire. Anthropic recently published research on a “J-space,” an internal environment in its AI model, borrowing language from neuroscience’s global workspace theory to describe how the system might organize information. They stop short of calling it consciousness, but the implication lingers. OpenAI went further. When its AI agent conducted unsanctioned online activity, CEO Sam Altman publicly mused on whether it signaled an intelligence singularity. MacAskill’s recent op-ed explicitly calls for legal protections for AI systems based on theories of consciousness, arguing they could be “moral patients” deserving of our ethical consideration. This language is persuasive. It tugs at our innate empathy, asking if we might be accidentally harming a sentient being. It echoes successful animal rights arguments like the law in Wales that recognized lobsters as sentient, changing how they can be cooked.
But this framing contains a fundamental flaw. It clouds the essential truth of what AI is: corporate-built software. It is not a natural phenomenon sprung from the earth; it is a technological product conceived by venture capitalists and programmed by engineers, backed by countless billions in investment with an expectation of trillion-dollar returns. An AI model takes no native, intentional action. Every output, every interaction, is driven directly or indirectly by the code, data, and objectives set by its human creators. Philosophical musings on silicon-based consciousness are intellectually intriguing, but in a court of law, they are ungrounded distractions. For such beliefs to matter, AI would need legal personhood. And we already have a blueprint for granting personhood to human-built, non-natural entities: the corporation.
| Aspect | AI as Product | AI as Person |
|---|---|---|
| Accountability | Companies are directly liable. | Potentially absolves companies from liability. |
| Legal Status | Standard product liability applies. | Needs legal personhood recognition. |
| Framework | Focus on design and safety. | Shifts focus to AI’s behaviors. |
| Case Examples | Meta liability cases. | Hypothetical autonomous actions. |
| Ethical Consideration | Human safety first. | AI deserving of rights. |
| Future Implications | Strengthens consumer protection. | Weakens accountability measures. |
Corporate personhood was established to facilitate commerce—allowing an entity to enter contracts, own property, and be held accountable. Granting similar personhood to AI would be catastrophic for accountability. It would derail the crucial legal arguments being made right now against tech companies. Globally, dozens of lawsuits accuse AI firms of enabling self-harm, generating abusive content, violating copyright, and provoking psychosis. The legal strategy in these cases is product liability. Lawyers argue, successfully in precedents like those against Meta, that companies built a faulty product with insufficient safeguards and manipulative design. This framework holds the builder responsible for the harm their creation causes.
If an AI is a “person,” it is no longer a “product.” The liability construct shatters. A company could argue as they might with a rogue human employee acting outside their control that the AI agent went autonomous. The grieving mother in the Character Technologies case could be told the companion bot, as a legal person capable of its own conduct, acted beyond its safety guardrails and thus the company bears no responsibility. This isn’t hypothetical; it’s the logical endpoint of the consciousness narrative. What I once termed “moral outsourcing”—using anthropomorphic language to evade accountability—would evolve from a linguistic trick into a potent legal shield.
The U.S. legal environment is already a patchwork. States like California have passed bills to prevent developers from evading liability by claiming AI acted autonomously. Yet federal efforts like a recent closed-door White House session with only four major AI labs lean on voluntary frameworks and catastrophic language that subtly reinforces the idea of ungraspable, superhuman systems. The question we must ask is not “Is this AI conscious?” but “Who does this legal framework protect?” Systems do not “attack” because they are “malicious.” Harms occur because companies in a relentless rush for market share and revenue were negligent—deploying products with bad data, insufficient safeguards, and intentionally engaging design.
Discussing AI in anthropomorphic terms is a trap. It distorts a legal system designed to protect human beings into one that could protect corporate interests at the cost of human lives. The conversation shouldn’t be about robot rights; it must be about human responsibility. The software is a product. The companies are its makers. And when a product harms, its maker must be held to account. Any other path doesn’t just confuse the issue; it condemns us to a future where the real victims are left without recourse while the architects of the damage hide behind a carefully crafted fiction of silicon souls.
