Updating 17:07
(The Center Square) — The New York City Council on Monday grilled executives with several major AI companies and heard from whistleblowers who warned of the potential risks posed by rapidly developing machine learning systems.
The 51-member council, which is considering regulations, heard from AI representatives — who faced the threat of subpoena to testify — about the steps their companies are taking to prevent safety issues. Councilors also heard from tech industry whistleblowers who warned that the companies rushing to develop the machine learning platforms are moving too quickly and with few safeguards.
Jacob Coxon, a former Anthropic engineer turned whistleblower, said big tech companies are being “extremely reckless” in the global race to develop the first generation of AI systems. He said AI companies are “gambling with our lives” by racing toward self-improving superintelligence without proper safeguards.
“We do not know how to control any AI system yet,” Coxon said in his remarks. “On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction.”
Coxon recently made headlines following his high-profile resignation, warning that the people who build AI “earnestly believe the technology could cause human extinction by the end of the decade.”
But representatives from Anthropic, OpenAI, Google and Meta also testified Monday and told councilors that their platforms are improving people’s lives in myriad ways and that companies are serious about ensuring the safety of their systems as they are developed.
“At Meta, we are working to make personal superintelligence widely accessible so that it empowers people in ways that improve their lives – and safety is core to everything we do,” a Meta spokesperson said in testimony. “We voluntarily came to the table today because earning public confidence requires rigorous safety work, transparency, and a willingness to engage proactively with policymakers.”
The hearing comes as the Democratic-led council is considering a raft of legislation that would prohibit businesses from marketing, selling or deploying an AI system in New York City without third-party validation and require a human-operated “kill switch” on the machine learning platforms and timely reporting of safety incidents.
Other proposals would shield whistleblowers from legal action if they came forward with information about violations or abuses of the technology.
New York’s Democratic Gov. Kathy Hochul announced last month that the state will begin requiring AI companies to register with the state next month and to report critical safety incidents within 72 hours.
Ahead of Monday’s hearing, Menin called for approval of the package of AI bills and criticized an agreement between President Donald Trump and several top AI executives, including Meta’s CEO Mark Zuckerberg, that would allow the companies to largely “self-regulate” as Congress considers federal regulations.
“The idea that AI companies will simply self-regulate is not going to work,” Menin said in remarks Monday. “What we really need to do is pass this legislation which will be first in the nation and hopefully it will spur action by the federal government and other jurisdictions.”