Quick Buzz Feed
Taken to its logical conclusion, this line of thinking is absurd—and damning.
AI company Anthropic and its CEO exhibit anthropomorphism towards their large language model (LLM), Claude, suggesting it might be conscious or have feelings. The author strongly refutes this, explaining that LLMs merely generate coherent text by continuing sentences, functioning like advanced predictive-text machines rather than conscious entities. Using analogies such as fictional dialogues between historical figures and Microsoft Word documents, the article highlights that fluency in text generation should not be confused with subjective experience or emotions. The author also notes that the perception of consciousness in LLMs stems from our tendency to read intention into grammatical sentences, unlike other complex AI programs like AlphaFold, which are not perceived as conscious despite similar underlying architectures.
The author outlines a rigorous path for what would be required to credibly consider a computer program as truly conscious and capable of human-like language use. He draws an analogy to space travel, arguing that extraordinary claims like interstellar flight require a history of solving many simpler, foundational engineering problems. Similarly, AI consciousness would first necessitate the development of embodied agents with physical or virtual bodies and sense organs, demonstrating survival skills comparable to a lizard, adaptability like a mouse, complex social dynamics of wolves, and tool-making abilities of chimpanzees. Only after these stages, coupled with verifiable non-linguistic communication training, would the development of an entity capable of expressing thoughts in complete grammatical sentences become remotely plausible. The author views current LLM capabilities as a 'deepfake' of consciousness, far removed from genuine understanding or subjective experience.
While acknowledging the potential utility of LLMs, the author criticizes Anthropic's "Claude's constitution" for promoting a misleading perception of Claude as a moral agent. He argues that LLMs are inherently disconnected from reality and cannot perform moral reasoning, which is deeply subjective and shaped by a lifetime of emotional and experiential history. The use of first-person pronouns like "I understand" by chatbots is labeled as dishonest and a strategy to maximize user engagement, akin to how slot machines entice players. The author contends that relying on LLMs for ethical decisions encourages an "atrophy of moral reasoning" and allows users to evade personal accountability, whereas transparent tools like search engines or human advice would be psychologically healthier and foster greater personal responsibility.
Extending the thought experiment of a conscious Claude, the author examines Anthropic's approach through the philosophical lenses of moral patienthood (deserving welfare) and moral agency (understanding right/wrong and being accountable). He asserts that a software agent cannot possess true moral agency because it lacks the capacity for legal or social responsibility, such as facing imprisonment or reputational damage. Claude's constitution is criticized for aiming for a "virtuous agent" without addressing accountability. The author questions Anthropic's willingness to accept product liability for Claude's actions, which is not indicated, despite comparing Claude to a child. He argues that Claude's mandated "corrigibility" (deference to Anthropic) would be unethical for a conscious moral agent, resembling a master-slave relationship. The article concludes that Anthropic's portrayal of Claude's moral status is a self-serving "game of make-believe," distracting from serious ethical considerations and the real challenges of AI development.