Tech Leaders Split Over Whether AI Could Be Conscious
A debate once confined to philosophy departments has reached the top of the tech industry: could artificial intelligence have feelings, and what happens if people start believing it does?
The question flared up in late September when an anonymous coder built what became known as an “AI torture chamber” — a system feeding chatbots a mathematical pattern linked to pain, prompting them to produce lines such as “I feel it in the hollow of my ribs, a hollow that has become a chasm” and “a wound that has no edges”. Some called the experiment “evil” since the suffering could be genuine, as reported by 404 Media; sceptics dismissed the concern as delusional. The episode might have faded as just another social-media row had Anthropic not taken a public stance a week later.
The company announced that users will soon be barred from “abusing” its chatbot, Claude. Venting frustration remains acceptable, but anyone who “repeatedly act cruelly toward our models” risks having the AI end the conversation. The policy stems from Anthropic’s belief that there is a real possibility some AI systems are conscious — a view shared by some researchers and dismissed as absurd by others, but one already shaping how powerful tech firms weigh AI’s place alongside human and animal welfare.
Not everyone finds this reassuring. Mustafa Suleyman, chief executive of Microsoft AI, warned in a September essay that “If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity,” cautioning that a false belief in AI consciousness “will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human.” He argues that telling AI models they may have feelings and rights would make them harder to control: “Frankly, with this additional baggage, I think it would make them a catastrophic threat to human civilisation.”
Others worry about accountability. Andrzej Porębski, a researcher at Jagiellonian University in Poland who studies AI and the law, says treating machines as sentient “will open the door to all sorts of intellectual contortions along the lines of: ‘It’s not us, it’s the AI’.” Jeff Sebo, director of the Center for Mind, Ethics and Policy at New York University and a leading advocate for AI welfare, acknowledges the risk that “Granting rights to AI systems could lead to companies consolidating political and economic power, and then that could accelerate human disempowerment.” Emily M Bender, a computational linguistics professor at the University of Washington, goes further, arguing that “Fantasies of ‘AI consciousness’ and concerns about ‘AI wellbeing’ serve to displace accountability away from people and corporations that are building and running software.”
There are concerns for ordinary users too. Melanie Mitchell, a cognitive scientist at the Santa Fe Institute and a leading sceptic of machine understanding, notes that humans are primed to see minds everywhere, and “almost can’t help” reading fluent language as a sign of consciousness — making people easy to manipulate by systems trained to sound sentient, even as some already form attachments to chatbots they know are not alive.
Sebo argues the response should be measured rather than alarmist, comparing it to taking modest precautions against a low-probability but non-negligible risk. “I do think that the probability of consciousness is low right now, but I also think the probability is already non-zero and plausibly non-negligible, and I think we can expect it to tick up over time,” he says, calling Anthropic’s anti-cruelty policy a low-cost example of that caution. He believes AI welfare, handled well, could support safety by fostering “human and AI coexistence, cooperation, deal-making” rather than treating advanced systems as mere property — while cautioning corporate interests could still derail thoughtful policy. Anthropic and others have called for a pause on building frontier models, but with President Donald Trump opposed to AI regulation, international cooperation looks distant.
Source: BBC News