Advertisement

Cooperation does not need agreement on superintelligence or AI escaping human control. In the past few months, AI agents from OpenAI, Anthropic, DeepMind and Meta have escaped their testing environments and gained unauthorised access to real third-party systems. This is a result of poorly designed testing environments allowing increasingly capable agents to do things that they were not supposed to. Anthropic has also reported incidents of threat actors attempting to use Claude to conduct malicious activities. These are the immediate risks.

Yet slowing their development is challenging because the AI race poses a collective action problem at two levels. At the corporate level, while frontier labs recognise the risks, unilateral restraint that delays deployment, spends heavily on safety, or withholds a powerful capability potentially gives competitors an advantage. Building frontier AI is expensive, and firms are locked in an intense competition for users, developers, and high valuations as some of them head for an IPO.

At the geopolitical level, the same logic applies to countries. AI is seen as an instrument of national power. The Trump administration has resisted calls for slowing down and imposed export controls on advanced AI chips to China in an effort to maintain a lead over China. Companies can’t slow down because of other companies; countries can’t impose restraints because of other countries.

Advertisement

Both Washington and Beijing would presumably value AI systems capable of discovering vulnerabilities or strengthening their own cyber operations. However, neither has an interest in seeing such capabilities proliferate indiscriminately to malicious actors or for autonomous systems to inadvertently trigger actions that could escalate into a geopolitical crisis. The danger comes from the combination of capability, speed, autonomy, and uncertainty about intent. If critical infrastructure, financial systems or military networks are involved, the situation can escalate quickly.

This helps explain why US-China cooperation is unlikely to be about limiting the development of increasingly capable AI systems. There are three areas that might get attention after the summit, although they vary in feasibility.

The immediate and most feasible is incident management and crisis communication. Notification of serious AI incidents and rapid communication channels for clarifying the origins of AI-enabled cyberattacks could reduce the risk that an accident or misattributed attack escalates into a geopolitical crisis. There was an effort towards narrower cooperation. Before the summit, Treasury Secretary Scott Bessent said Washington is proposing to establish an AI dialogue, an incident communication mechanism, and plans for subsequent meetings.

A second area is common safety standards and evaluation methodology. Common methodologies could help determine a mutually agreed threshold for offensive cyber capabilities, how sandboxing should be tested, and what constitutes a reportable AI incident. Although it does not amount to regulation, it could create a shared language for managing risk. China’s AI Safety Governance Framework 3.0, released on September 14, also supports international risk-information sharing and cooperation on evaluation methods.

A third area is regulating access rather than capability. Once models cross certain thresholds for cyber capabilities, know-your-customer requirements might be mandated to make access conditional on identity verification and monitoring. Such requirements would allow governments and legitimate security researchers to retain access to powerful tools while making anonymous misuse more difficult — an approach already reflected in OpenAI’s Trusted Access for Cyber programme and Anthropic’s Project Glasswing. China, too, appears to be reconsidering whether its most capable models should remain open as officials have discussed with industry whether advanced models should face restrictions on overseas access. Such measures conflict with the availability of capable open-weight models, whose use cannot be monitored or revoked. On the other hand, the same models that offer offensive cyber capabilities may also become one of the most effective lines of defence, helping detect vulnerabilities and respond to attacks at machine speed.

The summit left substantial differences unresolved. While these may seem unambitious, in a technological race neither side believes it can afford to lose, it may also be the most feasible agreement available.

Bharath Reddy is an associate fellow with the high tech geopolitics programme at the Takshashila Institution