How to Vet an AI Agency After Anthropic's September 2026 Threat Report
Anthropic's September 2026 report closes its customer guidance with one sentence:
"Organizations should treat AI keys and agent integrations with the same level of seriousness as they do production credentials - because attackers treat them with the same level of seriousness, too. AI access should be purchased only through authorized channels." [1]
Here is how to vet an AI agency after that report: seven questions and five contract clauses, each mapped to a failure mode Anthropic documented between December 2025 and August 2026.
That sentence was written for the buyer of AI services, and the report documents what happens when those conditions are missing. One stolen AI API key was reused "for roughly three weeks to conduct secondary attacks" against entirely different organizations [1]. One compromise "escalated from a single stolen developer token to full administrative control of a victim's cloud environment in roughly three hours." [1] And one SaaS provider breach reached "roughly 200 of the SaaS company's downstream customer organizations" - the clients, not the platform [1].
None of those are exotic failures. They are custody failures: nobody could say whose name was on the key, who would notice it being used, or how fast it could be switched off. That is what the seven questions below surface.
Why AI agency security vetting changed in September 2026
Two things are new, and both land on the client.
The AI supply chain is now a target in its own right. In the report's GTG-50020 case, an actor injected malicious instructions into an AI vendor's automated evaluation sandbox, and the sandbox handed over "the production AI API keys from multiple providers belonging to that vendor." [1] The same infrastructure then "attacked roughly thirty AI companies in about four days" [1]. The report is explicit about who was hurt: "the keys involved were customers' keys stolen from customers' environments. The actor never compromised Anthropic's own systems." [1] A botched integration leaks your secret, not the vendor's.
Discount AI access is its own risk class. The report documents a fraudulent reseller "offering cheap Claude access - which turned out to be neither cheap nor actually Claude," with customer traffic silently proxied to another model while a credential harvester collected and resold the victims' credentials [1]. Hence "AI access should be purchased only through authorized channels." [1] If an agency quotes AI capacity far below list price, that is a security question, not a procurement win. The live AI vendor risk tracker is worth a bookmark - vendor risk here moves weekly.
Why this belongs in a contract, not a technology review: reconnaissance, exploitation, tool development and data processing are "now delegated to AI models, which run in harnesses at machine speed and in parallel," and "the main distinguishing feature between these classes of actors is no longer sophistication but intent." [1] The barrier is intent and access, not skill.
The 7 questions to ask before you sign
Ask each in writing, and ask for the answer in the contract rather than the pitch deck - a weak answer, then a good one.
1. Whose name is on the AI API keys - ours, yours, or the agency's?
Weak: "It's all under our enterprise account." Good for billing, useless for accountability: if the key is theirs and the integration is yours, you sit in the shape of the ~200 organizations above [1].
Good: A named legal entity, a named technical owner, a list of which integration uses which key, and confirmation the key is scoped to your tenant only.
2. What is the key rotation interval, and who is alerted when a key is used outside normal hours?
Weak: "We rotate quarterly." Rotation without alerting does not help against a key reused for three weeks [1].
Good: A rotation interval stated in days, an alerting owner by name and role, one documented alert that fired, and a retention window for those alerts.
3. What can the agent reach from our integration - egress allowlist, sandbox versus production?
Weak: "The model can't do anything we don't tell it to." Prompt instructions are not a security boundary; GTG-50020 walked past one [1].
Good: An egress allowlist, separate sandbox and production key sets, and a default-deny posture on outbound writes - the report's own line is that "AI API keys and session tokens are targets; the integrations customers build around AI such as sandboxes, proxies, and resellers are part of the attack surface." [1]
4. What happens to our data and our keys if the agency's own vendor is breached?
Weak: "Our vendor is SOC 2, so we're covered." Certification describes their controls, not their subcontractor's blast radius.
Good: A named subcontractor list, the notification path when a supplier is hit, and what the agency would revoke, in what order, on your behalf.
5. Who discloses an incident to us, within what window, in writing?
Weak: "We'd let you know right away." No owner, no clock, no medium.
Good: A named role, a window measured in hours, and a written medium with a fallback contact. Against a three-hour token-to-admin escalation [1], a week-long disclosure window is disclosure after the fact.
6. What do you log, where does it live, and how long is it retained?
Weak: "Standard logging." Unverifiable.
Good: Authentication events, model invocations, outbound writes and key usage from unexpected identities; where the logs live; retention length; and the format you receive at handover. The standard to test against, from Unit 42: hunt for "bursty API requests, rapid 401/200 HTTP state shifts, parallel authentications and sudden model usage from unexpected identities." [2]
7. How do we take the agent offline in under an hour without breaking the workflow?
Weak: "Just call us." You want a procedure, not a person.
Good: A written kill switch with a tested owner, a documented revocation order (keys, then sessions, then integrations), and a fallback mode for the business while the agent is down.
AI agency contract clauses that put the risk on a named owner
Five clauses make those answers enforceable:
- Key ownership and custody. Name the legal entity holding each key, the scope, the rotation interval and the sub-processors - the clause that keeps the ~200-customer scenario [1] from becoming your scenario.
- Incident disclosure. Owner, window, medium, and a definition of "incident" that includes supplier breaches.
- Log retention and handover. What is retained, for how long, and in what format when the engagement ends - logs are the only evidence you will have after an agent-driven event.
- Termination and revocation. A stated period - hours, not weeks - in which all keys, sessions and integrations are revoked, with written confirmation.
- Named individuals, not just an entity. A single contractor holding every credential is a single point of failure; require a second named owner for key custody.
Our longer question set, including the procurement and access-control questions that sit outside this report, is at AI agency security vetting. If an agency pushes back on this section, the technical layer is covered by our AI agent toolchain security audit - the same audit path that finds the credentials, sandboxes and resellers this report is describing.
The machine-speed test: how to evaluate an AI agent
Ask one question that forces the agency to evidence a control rather than a promise: which control would have caught a three-hour escalation from a developer token to cloud admin, or three weeks of reuse of a stolen AI key? [1]
- If the answer is that the vendor handles it, the risk is unassigned - and the vendor is a target too [1].
- If the answer is the monthly report, the clock is wrong: three hours does not survive a monthly cadence.
- A specific alert, a named owner and a tested response time means the agency has done this before.
The independent datapoint to hold them to is Palo Alto Networks' Unit 42, which responded to an intrusion where a human operator drove frontier models to compress "weeks of methodical intrusion tradecraft (using more than 50 MITRE ATT&CK techniques) into less than 10 hours," an outcome "at the scale of a coordinated effort from multiple red teams, which would normally take human operators around two weeks," with no novel zero-day required [2]. Machine speed is why the questions should be about detection and revocation windows, not model preference.
A client-ready version of these checks, written for the business rather than the agency, is at AI cyberattack preparedness checklist - hand it over as proof.
Bottom line on how to vet an AI agency: the one worth hiring can name who holds each key, when it was last rotated, who is alerted when it is used at 3am, and how fast the agent is revoked - and will put all four in writing. Score the answers with our AI vendor risk assessment framework.
Questions people actually ask
Label key: [AC] = the question as phrased is a Google Autocomplete suggestion (probed 2026-09-10); [GAP] = no ranking page answers it. No People-Also-Ask data is claimed.
How do I vet an AI agency?
[GAP] Start with custody and financial questions before capability ones: whose entity holds the AI keys, the rotation interval, the alert owner, the disclosure window, and the revocation procedure. Everything in the seven questions above is answerable in writing in under a week by any competent agency - and the inability to answer is itself the finding.
How do I evaluate an AI agent?
[AC] Evaluate the agent the way you evaluate a new employee with system access, not the way you evaluate software: what can it reach (egress and sandbox boundaries), what identity does it hold and for how long, what does it log, and who can revoke it within an hour. The report's agent-driven cases - a token-to-admin escalation in roughly three hours, and a stolen key used for roughly three weeks - are the two failure modes those four tests are aimed at [1].
What is an AI vendor risk assessment?
[AC] A structured review of the model provider and of the agency that resells it: where the credentials live and who holds them, the rotation and alerting intervals, what the vendor's own breach would expose in your environment, the disclosure track record, and the revocation path. This report's GTG-50020 case is the reason it is now a client-side question: the keys lost were "customers' keys stolen from customers' environments" [1].
Is a cheap AI reseller or API proxy safe?
[GAP] Anthropic's guidance is "AI access should be purchased only through authorized channels," and the report documents a reseller whose cheap Claude access "turned out to be neither cheap nor actually Claude," complete with a credential harvester [1]. If the discount requires routing your traffic through someone else's infrastructure, treat it as a supply-chain risk.
What should our AI vendor contract include?
[GAP] The five clauses above: key ownership and custody, incident disclosure with a window, log retention and handover, termination revocation with a stated period, and named individuals rather than a single contractor. Each maps to a documented September 2026 failure mode [1].
Who should I hire to secure AI agents?
[GAP] Either an agency willing to put key custody and incident response in the contract (this page is the question set for that conversation) or an independent reviewer who reports to you rather than to the agency. Avoid any arrangement where the party deploying the agent is also the only party assessing its security.
Sources
- Anthropic, "Detecting and countering misuse of AI: September 2026" (published 10 September 2026; the 154-page PDF is available from the same page) - https://www.anthropic.com/threat-intelligence-report-september-2026
- Palo Alto Networks Unit 42, "An AI-Assisted Cyber Attack: Inside a Unit 42 Investigation," 2 September 2026 (updated 3 September 2026) - https://unit42.paloaltonetworks.com/ai-assisted-cyber-attack-inside-a-unit-42-investigation
- POLITICO, "Bad actors in China and Russia are already weaponizing Anthropic's AI," 10 September 2026 (background on how broadly AI is now used in cyber operations) - https://www.politico.com/news/2026/09/10/bad-actors-china-russia-weaponizing-anthropic-01070435
- Axios, "Governments use Claude to spy on people, Anthropic warns," 10 September 2026 (Anthropic's lead on automation of existing operations rather than new targets) - https://www.axios.com/2026/09/10/anthropic-claude-government-surveillance-threats
Quotations from [1] and [2] were matched against the fetched source text for this draft. Anthropic's threat intelligence lead is described as "head of threat intelligence" by Axios [4] and "head of threat research" by POLITICO [3] - cited here in context rather than merged.