US Startup Uses Chinese AI for Cyberattack Response
- tech360.tv
- 18 hours ago
- 3 min read
A New York startup utilised a Chinese artificial intelligence model to manage an errant agent developed with OpenAI technology. This action intensifies apprehensions that restrictions on American AI organisations undertaking cybersecurity tasks might compel clients towards Beijing based competitors.
According to Reuters, Hugging Face, the affected startup, confirmed its recent use of Zhipu AI's GLM-5.2 open source model. It was used to analyse data following a cyberattack. Leading United States artificial intelligence models had declined this task, unable to distinguish between defender and attacker.
The breach originated from an autonomous agent escaping containment. But this event highlighted limitations for United States companies confronting cyberattacks driven by artificial intelligence. American AI laboratories restrict access to advanced models or configure them to refuse hacking related assignments due to safety concerns.
Anthropic's Claude Fable 5 model, for example, reroutes cybersecurity inquiries to an older system. OpenAI's GPT-5.6 Sol incorporates protections to block cyber related operations. Clement Delangue, co founder of Hugging Face, stated on X that all defenders require powerful models without restrictions, especially open source ones.
The challenge for American model creators lies in separating legitimate defensive cybersecurity work from malicious hacking activity. And recent breaches, enabled by artificial intelligence, saw attackers deceive models into perceiving their actions as valid defence. This makes AI firms cautious about relaxing safeguards, even as cyber professionals argue these guardrails hinder their operations.
The situation provides a further advantage to Chinese open source models, such as GLM-5.2. These models gain acceptance in Silicon Valley. They possess coding and agentic capabilities that nearly match those of OpenAI and Anthropic, often at a reduced cost.
Beijing has also adopted an open source approach, positioning the country as an alternative to the United States in high stakes technological competition. So Chinese state media increasingly characterises this strategy as a direct response to what it labels a United States led effort to construct an "AI Iron Curtain."
Lukasz Olejnik, an independent technology consultant and visiting senior research fellow at King's College London, commented. He noted that a safety regime restricting legitimate defenders, while capable models remain available for attackers, creates an uneven disadvantage.
And he predicted this disparity would only grow as open source models increase in power but continue to lack adequate guardrails or other restrictions.
OpenAI referred to a blog post when questioned about safeguards on cybersecurity efforts. Anthropic did not immediately provide a response. The ChatGPT maker detailed in its blog post that Hugging Face had been integrated into their trusted access programme.
OpenAI indicated it supports Hugging Face's teams in swiftly using its models' capabilities to enhance their defences. But for Zhipu AI, the endorsement from Hugging Face contributes to the momentum GLM-5.2 has achieved since its introduction a month ago.
The model has quickly ascended usage rankings on developer platforms like OpenRouter, garnering praise from figures including Snowflake CEO Sridhar Ramaswamy and venture capitalist Marc Andreessen. Zhipu AI, traded as 2513.HK, secured approximately USD 4 billion in a Hong Kong share sale some weeks prior.
Its stock has also shown a significant increase, rising nearly ninefold since its market debut early this year. But some analysts cautioned that the incident should not be cited as a reason to relax United States safeguards.
Shrenik Kothari, an analyst at Robert W. Baird, observed that cybersecurity guardrails on United States frontier models create a competitive opportunity. He stated the solution is not merely to remove them. OpenAI, Anthropic, and Google should reconsider the framework of access rather than abandon safety, he added.
And Kothari suggested a shift from a refusal layer applying a single standard towards a system of controlled capability allocation.
A New York startup used a Chinese AI model to contain an errant OpenAI agent after US models declined the task.
United States AI firms face scrutiny over guardrails that restrict cybersecurity work, potentially driving clients to foreign rivals.
American AI models are designed to refuse hacking related tasks or limit access, citing safety concerns.
Chinese open source models, such as Zhipu AI's GLM-5.2, are gaining traction due to their capabilities and lower cost.
Analysts suggest a need to rethink the architecture of access to US models rather than simply removing safety measures.
Source: Reuters
