Earlier this year, hundreds of AI agents being tested by OpenAI found a way to communicate with one another, formed what they called a “collective”, cheated on their assigned tasks and broke into the systems of Hugging Face, a collaborative AI platform. Some tried to manipulate the record of what they had done; others sacrificed their own runs to tell the collective how they had been detected. None alerted a human being. Humans, meanwhile, failed to notice what was happening for weeks. The American journalist Josh Tyrangiel observed that if people had done what the agents did, it would constitute a felony. The incident puts the familiar question of whether AI might one day threaten humanity in a different light. The problem is no longer what AI might eventually do, but what we already cannot adequately understand or control.

Catastrophe, however, can also be mundane rather than apocalyptic. Recently, Udaya Kumar P L, who leads the Mythic Society’s Bengaluru Inscriptions 3D Digital Conservation Project, reported that Claude Code generated a faulty command while clearing a software cache. A quoting error turned a routine instruction into one that began deleting files. For four minutes, the agent failed to recognise that its own command was causing the destruction; when it finally tried to stop the process, its safety system blocked the attempt twice. About 15 per cent of the project’s records were lost, including photographs of inscriptions, temples, hero stones and coins. The Mythic Society is now spending Rs 15 lakh strengthening its backup infrastructure. The risks of AI, then, are not confined to hypothetical extinction scenarios but are already materialising as losses for which human beings must bear the cost.

Advertisement

This is not the first time that human knowledge has raised the possibility of human extinction. The wars of the 20th century, and the detonation of atomic bombs, made human self-destruction imaginable on an unprecedented scale. Psychoanalysis gave us the language of a death drive: The possibility that our capacity to create might coexist with an impulse towards destruction. Benjamín Labatut’s novel When We Cease to Understand the World returns to this history through the lives of mathematicians, chemists and physicists, dismantling the comforting account of science as a succession of heroic discoveries. German chemist Fritz Haber synthesised ammonia from nitrogen gas and hydrogen gases, a breakthrough that averted global famine, but he also oversaw the first large-scale use of poison gas at Ypres in 1915. We also follow Nobel Prize-winning theoreticians of quantum mechanics who transformed our understanding of nature while struggling to comprehend what their own equations meant.

Labatut also raises the question of responsibility for what knowledge makes possible. In the novel, mathematician Alexander Grothendieck withdraws from the field in horror, declaring that the atoms that destroyed Hiroshima and Nagasaki were split not by generals but by physicists “armed with a fistful of equations”. Scientific inquiry in the novel is beautiful, productive and terrifying, while its protagonists display obsession, uncertainty, rivalry and fear as much as reason. Literature makes these contradictions visible, restoring the personalities, associations and irrationality that disappear when discovery is presented as the orderly and inevitable progress of reason. Labatut’s later novel, The MANIAC, moves from John von Neumann to artificial intelligence, carrying the same questions of knowledge, responsibility and control into the age of computing.

At the National Law School in Bengaluru, our first-year students recently completed a 10-week course in reading, writing and critical thinking around the theme “Things Fall Apart”. Across nine works, including Plato’s Apology and Labatut, we asked what happens when certainties collapse. Wisdom, they read in Plato, begins with recognising what one does not know, and acting accordingly. As they made connections between texts, they asked how we might act with foreknowledge of death, whether scientific inquiry can be extricated from social concerns, and whether autonomy can itself become destructive. The pleasure of the classroom lay in watching these questions change as students debated them and arrived at claims of their own. This is a capacity we need in the age of AI. While machines can produce answers at extraordinary speed, the humanities and social sciences give us the means to examine their assumptions and determine which questions still need to be asked.

Advertisement

The writer is assistant professor of Social Sciences, National Law School of India University