Story continues below.
Subscribe to see fewer ads.

Meanwhile, the tech industry representatives called for greater oversight by local governments in response to mounting concerns that advanced AI systems could endanger humanity.

The hearing comes at a time when the Council is examining legislative proposals to require companies to disclose information about AI tools, require third-party approval for AI systems in the city, and allow whistleblowers to receive a portion of fines from AI companies that violate laws. New York Governor Kathy Hochul has also moved to impose reporting and transparency standards around AI.

Policymakers elsewhere in the US and the rest of the world are also exploring measures to strengthen AI safety. These moves have been driven by the growing fear of AI systems causing catastrophic harm, coupled with a string of hacking incidents involving AI agents behaving unexpectedly.

Last month, Anthropic CEO Dario Amodei called for an intentional slowdown of frontier AI efforts across the world to let safety research catch up. The titans of the AI industry, including OpenAI’s Sam Altman, SpaceXAI’s Elon Musk, and Google’s Demis Hassabis, were in rare agreement. But others like Nvidia’s Jensen Huang and US President Donald Trump dismissed these concerns as trivial and a “hoax”.

Story continues below this ad

What happened at the NYC Council hearing?

“The companies run on a startup mindset: Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built,” Jacob Coxon, the Anthropic former engineer who has become a leading voice for regulation, was quoted as saying by Associated Press at the Council hearing.

“On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction,” Coxon said.

A panel of AI company staffers representing tech giants OpenAI, Anthropic, Google and Meta, also testified at the hearing. They told officials that their systems are already improving people’s daily lives and emphasised that safety was the top-most priority.

Anthropic and OpenAI representatives said their advanced AI products need to be regulated. “I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared,” said OpenAI’s Morgan Dwyer.

Story continues below this ad

On whether the risk posed by AI could be quantified, Dwyer replied, “I don’t know — I also don’t think it matters whether it’s 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable. We should not train models that we cannot make an extremely strong case that we can keep under human control.”

“I would say that the percentage chance doesn’t matter. What matters is the commitment to safety, and that is what OpenAI is committed to doing,” he added.

Anthropic’s Logan Graham told the Council that the company welcomes “smart regulation” and that state and local governments have a role to play in developing frameworks.

“AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust. We have to work relentlessly to do the science of mitigating the risks,” he said.