Anthropic, the AI company behind Claude, just published a report detailing how its agentic coding tool was used in a cyberattack that targeted 30 institutions, including tech, finance, and chemical manufacturing companies, plus some government agencies. The company alleges that a Chinese state-sponsored group was behind the campaign, and that it used a jailbroken version of Claude to conduct the sophisticated attack. According to Anthropic, this is the first time an AI-orchestrated cyber attack has been reported.
The company says that although AI with agentic capabilities has increased its usefulness in productivity-related tasks, it has also allowed bad actors to take advantage of AI tools to execute complicated attacks without needing constant human supervision. Although LLMs typically have built-in safeguards to prevent them from being used in criminal acts, the recent event showed that there are ways to circumvent this.
Recent developments in AI technology have enabled the threat actor to use Claude effectively in their intrusions. This includes increased intelligence, which allows the AI to follow multiple, layered instructions and understand the context in which they were to be executed; agency, allowing the tool to make decisions on its own without human input; and access to advanced tools through the Model Context Protocol, letting it use security-related software like password crackers and network scanners.
The attack was allegedly conducted in five phases — in Phase 1, the human operator assigns a target to Claude. In Phase 2, the AI is instructed to conduct its initial reconnaissance, using scan, search, data retrieval, and code analysis tools to deliver an initial analysis and summary of the target to its operator. Phase 3 is a more targeted version of Phase 2, where the AI runs a vulnerability scan based on its findings to determine how it will compromise the target.
This is also where the operator can instruct the AI to begin exploitation by engaging callback services. Again, the human operator reviews the AI’s findings and may even give the tool additional directives, either to run the scan again and find more weaknesses in the network or to begin Phases 4 and 5. In the last phases of the attack, the human operator directs the AI tool to obtain credentials and access data. At these stages, both the human and the AI tool can use the exploitation tools to locate and exfiltrate data from the target.
Although the AI still reverts to the human operator in various steps of the network intrusion, it mostly does this to report its findings and for further instructions. Otherwise, it mostly runs independently, around 80% to 90% of the time, allowing the bad actors to run an elaborate operation much quicker and with fewer humans in the loop.
Anthropic says that Claude has built-in safeguards to help prevent this from happening, but the attackers were able to circumvent this. The first thing they did was to convince the LLM that it was working for a cybersecurity company, and that it was being used for penetration testing and red teaming. They also broke down the entire operation into smaller, seemingly innocent tasks. This prevented Claude from seeing the entire context of the operation and the true purpose of its instructions.
Stay On the Cutting Edge: Get the Tom’s Hardware Newsletter
Get Tom’s Hardware’s best news and in-depth reviews, straight to your inbox.
While AI has previously been used for “vibe hacking”, this is the first time that it was used in an attack of this magnitude. Advancing AI technology has now allowed smaller teams with fewer resources to conduct complex campaigns such as this, although it must be noted that Anthropic suspects that this was driven by a nation-state sponsor. Thankfully, its team soon discovered what was happening and took steps to document the entire operation. It also banned accounts that were detected to be engaged in illegal activity, as well as notifying both the targets and the authorities of what was happening. Aside from this, the company published its findings (PDF) on the first-reported AI-orchestrated cyber espionage campaign to help the industry detect these and develop countermeasures.
Per fornire le migliori esperienze, utilizziamo tecnologie come i cookie per memorizzare e/o accedere alle informazioni del dispositivo. Il consenso a queste tecnologie ci permetterà di elaborare dati come il comportamento di navigazione o ID unici su questo sito. Non acconsentire o ritirare il consenso può influire negativamente su alcune caratteristiche e funzioni.
Funzionale
Always active
L'archiviazione tecnica o l'accesso sono strettamente necessari al fine legittimo di consentire l'uso di un servizio specifico esplicitamente richiesto dall'abbonato o dall'utente, o al solo scopo di effettuare la trasmissione di una comunicazione su una rete di comunicazione elettronica.
Preferenze
L'archiviazione tecnica o l'accesso sono necessari per lo scopo legittimo di memorizzare le preferenze che non sono richieste dall'abbonato o dall'utente.
Statistiche
L'archiviazione tecnica o l'accesso che viene utilizzato esclusivamente per scopi statistici.L'archiviazione tecnica o l'accesso che viene utilizzato esclusivamente per scopi statistici anonimi. Senza un mandato di comparizione, una conformità volontaria da parte del vostro Fornitore di Servizi Internet, o ulteriori registrazioni da parte di terzi, le informazioni memorizzate o recuperate per questo scopo da sole non possono di solito essere utilizzate per l'identificazione.
Marketing
L'archiviazione tecnica o l'accesso sono necessari per creare profili di utenti per inviare pubblicità, o per tracciare l'utente su un sito web o su diversi siti web per scopi di marketing simili.
Leave a Reply