How to detect LLMjacking

Why cost is a bad primary signal

A cost alert answers the wrong question. It tells you the organisation spent money. It does not tell you which access key, which principal, which source address, or whether the spend was yours.

The other three problems are structural. Billing data is aggregated and arrives late, so the quota is already consumed by the time the anomaly is visible. A product launch, a new dataset, a retry loop and a normal batch job all produce the same curve as abuse. And an attacker who resells model access has a financial reason to stay under whatever threshold you picked.

There is also a recovery window your cost platform cannot see. In the LiteLLM compromise on 21 October 2026, credentials stolen out of the poisoned release were validated roughly four and a half hours after the theft, then probed for model capacity with bedrock:ListFoundationModels. The whole reconnaissance phase was over inside a working afternoon.

What is actually distinguishable

Post-theft model abuse is a sequence, not a volume. Stolen credentials are validated, then used to inventory available capacity, then used to spend. Three stages, in that order, against one access key.

Stage one: validation

GetCallerIdentity and GetSessionToken. On its own this is the most boring call an AWS account makes — every SDK, every CI job and every health check does it. What makes it a signal is being the *first* validation ever recorded for that key, inside a recency window that defaults to 24 hours. A key that has validated weekly for a year is not news. A key that validates for the first time an hour after a package release is.

Stage two: enumeration

ListFoundationModels, GetFoundationModel, GetFoundationModelAvailability, ListCustomModels, ListModelInvocationJobs, ListEndpoints. This is the step with no innocent explanation at scale: somebody is working out how much compute is available to them. A build system does not walk a foundation-model inventory.

Stage three: invocation

InvokeModel, InvokeModelWithResponseStream, InvokeEndpoint, InvokeEndpointAsync. At this point the attacker has stopped doing reconnaissance and started spending your quota. In the rule set, reaching this stage is what moves the finding to CRITICAL; the two preceding stages are HIGH.

Why the rule needs two stages, not one

There is a version of this detection that alerts on any new key touching a model endpoint. It is unlivable. Every new deploy role, every rotation, every sandbox notebook and every onboarding lands in the queue, and the rule is off within a month.

The shipped detector requires at least two distinct stages for the same key. A lone validation is how healthy automation behaves; validation followed by model enumeration is not. One extra stage buys back most of the specificity the single-event rule was missing.

The three parameters — recency, stage gap and the tracked event names — are options on a pure function, which is what makes them testable rather than a matter of taste. If you build your own, keep them configurable and keep the function pure: the same events must produce the same chain every time you replay the window.

Catching the case the sequence cannot see

The sequence catches an attacker walking a script. It cannot catch a key that was already making model calls when it was stolen, because there is no fresh validation and no novel enumeration. That is what a second layer is for: anomalies inside the invocation traffic.

• Volume: more than 500 invocations per hour in absolute terms, or more than three times the key's own six-day daily average.
• Origin: an invocation from a source address block this key has never used, compared at /24 granularity against everything older than 24 hours.
• Credential age: a long-term key (AKIA…) whose entire model history is under seven days old, which usually means the key existed for other reasons and this is new behaviour.
• Model choice: the first use of an expensive model family by a key that has only ever used cheap ones.
• Region: an invocation from a region this key has never touched.

Three or more of those together is CRITICAL. A new origin plus a volume spike is HIGH. Anything else is MEDIUM and belongs in a daily review, not a page.

Be honest about the residual case. A key with eleven consumers has no baseline worth the name, and its first-use rules will fire on a perfectly ordinary rollout. That is an ownership problem, not a threshold problem: a key with one owner and one workload has a history, and a key shared across everything has none.

What to log, or the question is unanswerable

This detection is built from audit events. If the events are not collected, no amount of cleverness afterwards helps — the sequence is only visible while it is being written.

# Tracked event names, by stage. Everything else is noise for this question.
CREDENTIAL_VALIDATION   GetCallerIdentity, GetSessionToken
MODEL_ENUMERATION       ListFoundationModels, GetFoundationModel, GetFoundationModelAvailability,
                        ListCustomModels, ListModelInvocationJobs, ListEndpoints
MODEL_INVOCATION        InvokeModel, InvokeModelWithResponseStream, InvokeEndpoint, InvokeEndpointAsync

# Per event, the join needs: access key id, principal ARN, source IP, region,
# event name, timestamp, and for invocations the model id from the request.

Two collection details decide whether you can answer the question at all. The validation and enumeration calls are control-plane events and show up in a standard trail; the invocation path is a data event and is not in a default trail, so either enable model invocation logging or accept that you will see reconnaissance and never the spending. And keep at least thirty days: the volume baseline is a six-day average and the age check has a seven-day floor, so a shorter window makes every key look new.

Collect the neighbouring plane too. The same source address touching S3, Secrets Manager or IAM is what tells you this is credential theft with a model endpoint attached, rather than a workload you have forgotten about.

Where this does not work

• Session credentials (ASIA…) rotate by design, so age-based reasoning about them is meaningless. The sequence still applies; the first-use check does not.
• The shipped detector reads CloudTrail, so it covers Bedrock and the SageMaker endpoint families. The shape generalises to any provider with a management-event audit trail; the event names do not, so a provider without one needs a gateway in front of it and logs from the gateway.
• A stolen key that already had invocation history, from a known address, in a known region, at a normal volume is invisible to everything above. The remedy for that case is upstream: fewer long-lived keys, one workload per key, and immediate rotation when a credential appears somewhere it should not.
• Trusted-IP allowlists do not help. The LiteLLM crew validated stolen keys from ordinary cloud addresses, which is where your own workloads run too.

Questions