Just like enterprises, cybercriminals are embracing generative AI to shape their attacks, from creating more convincing phishing emails and spreading disinformation to model poisoning, prompt injections, and deepfakes.
Now comes LLMjacking.
Threat researchers with cybersecurity firm Sysdig recently detected bad actors using stolen credentials to target large language models (LLMs) – which are foundational to the myriad generative tools coming onto the market since OpenAI launched ChatGPT 18 months ago – with the eventual goal of selling the access to other hackers.
Access into the compromised LLM accounts could be used for a number of reasons, such as to steal money or LLM training data, according to Alessandro Brucato, senior threat research engineer at Sysdig.
“Once initial access was obtained, they exfiltrated cloud credentials and gained access to the cloud environment, where they attempted to access local LLM models hosted by cloud providers: in this instance, a local Claude (v2/v3) LLM model from Anthropic was targeted,” Brucato wrote in a report.
.ai-rotate {position: relative;}
.ai-rotate-hidden {visibility: hidden;}
.ai-rotate-hidden-2 {position: absolute; top: 0; left: 0; width: 100%; height: 100%;}
.ai-list-data, .ai-ip-data, .ai-filter-check, .ai-fallback, .ai-list-block, .ai-list-block-ip, .ai-list-block-filter {visibility: hidden; position: absolute; width: 50%; height: 1px; top: -1000px; z-index: -9999; margin: 0px!important;}
.ai-list-data, .ai-ip-data, .ai-filter-check, .ai-fallback {min-width: 1px;}
The stolen cloud credentials were obtained through a vulnerable version of Lavarvel, a free and open source PHP-based web framework for creating web applications. The flaw in the vulnerable Laravel version is tracked as CVE-2021-3129 and comes with a critical threat score of 9.8 out of 10. The stolen credentials could be used to target 10 cloud-hosted LLM services.
Tools being used by threat actors were generating requests that could be used to targets models during the attacks and Sysdig researchers also found a script that could check credentials for the 10 AI services so see which were useful to the attackers. The 10 services not only were Anthropic but also Amazon Web Services (AWS) Bedrock, Microsoft Azure, OpenAI, Google Cloud Platform Vertex AI, A121 Labs, ElevenLabs, MakerSuite, Mistral, and OpenRouter.
Easy Access to Services
The services are designed to give developers easy access to models using in LLMs and come with a simple user interface to let them start building applications quickly. However, for a cloud vendor to run a model, it needs to be submitted for approval. That said, access is usually given quickly after the request is made; Brucato called it “more of a speed bump for attackers rather than a blocker, and shouldn’t be considered a security mechanism.”
The process for interacting with the hosted language models also is simple, using CLI commands.
The Sysdig researchers also discovered the use of a reverse proxy for LLMs being used to provide access to the compromised accounts. The checking code used by the bad actors to verify whether credentials can be used to target particular LLMs also references the OAI Reverse Proxy open source project.
“Using software such as this would allow an attacker to centrally manage access to multiple LLM accounts while not exposing the underlying credentials, or in this case, the underlying pool of compromised credentials,” he wrote. “During the attack using the compromised cloud credentials, a user-agent that matches OAI Reverse Proxy was seen attempting to use LLM models.”
Once in the cloud environment, the hackers subtly poked around to see what they could do while not triggering warnings. This included using seemingly legitimate InvokeModel API request logged by Cloudtrail, an AWS service that logs actions taken by users, with Brucato adding that “this subtle probing reveals a calculated approach to uncover what actions their stolen credentials permitted within the cloud account.”
They also worked to see how the service was configured.
An Expensive Situation
“Attacks against LLM-based Artificial Intelligence (AI) systems have been discussed often, but mostly around prompt abuse and altering training data,” he wrote. “In this case, attackers intend to sell LLM access to other cybercriminals while the cloud account owner pays the bill. … It shouldn’t be surprising to learn that using an LLM isn’t cheap and that cost can add up very quickly.”
The researchers calculated the cost of a “worst-case scenario,” where a bad actor abuses Anthropic’s Claude 2.x model and hits the quota limit in multiple regions. At this point, it could cost the victim more than $46,000 per day.
“By maximizing the quota limits, attackers can also block the compromised organization from using models legitimately, disrupting business operations,” he wrote.
Prevention is Important
There are a number of ways to prevent such an attack, including using strong vulnerability management to prevent initial access and secrets management to ensure credentials are not stored in ways that allow them to be easily stolen. Organizations also can use cloud security posture management (CSPM) or cloud infrastructure entitlement management (CIEM) tools to make sure cloud accounts have the fewest number of permissions needed.
Such protections will be needed, according to Brucato.
“Stolen cloud and SaaS credentials continue to be a common attack vector,” he wrote. “This trend will only increase in popularity as attackers learn all of the ways they can leverage their new access for financial gain. The use of LLM services can be expensive, depending on the model and the amount of tokens being fed to it.”
This normally “would cause a developer to try and be efficient — sadly, attackers do not have the same incentive,” Brucato added.
This post was originally published on 3rd party site mentioned in the title of this site