Secrets Management for AI Agents on Small Teams

TL;DR: The model never needs the key. That one sentence is most of agent secrets management, and almost every setup I see gets it backwards — the API key sits in the agent’s environment or, worse, in its context window, where a single prompt injection turns your assistant into an exfiltration tool. The fix isn’t an identity platform; at small-team scale it’s an architectural habit: credentials live with the tools, tools execute with the narrowest credential that does the job, the model only ever sees results. Add short-lived tokens where your providers make it easy, log what the agent touched, and keep one kill switch you’ve actually tested. The identity-for-agents vendor pitch can wait until you have agents enough to need it.

The model never needed the key

Walk through what actually happens when your agent “uses” a credential. The model emits a tool call — a blob of JSON saying query_database with some arguments. Your code receives that JSON, executes the query, and returns rows. At no point in that loop does the model need the database password. The code that runs the tool needs it. The model needs the result.

This is obvious once you say it out loud, and yet the default wiring of most agent stacks ignores it. The agent process gets the same .env the app gets. Every tool call executes with the full ambient authority of everything in that file — the Postgres superuser, the Stripe live key, the AWS credentials with * on the resource ARN. Not because anyone decided the agent should have all that. Because nobody decided anything; the agent inherited the human-shaped credential model, and the human-shaped credential model assumes the thing holding the keys has judgment.

An agent does not have judgment. It has instructions, and it takes new instructions from anything that lands in its context — a support ticket, a scraped web page, a README in a cloned repo. Prompt injection is precisely the trick of making retrieved content count as instructions, and a credential the model can see is a credential the model can be talked into repeating. The mid-2025 Supabase incident was the canonical version: an agent with privileged service-role access processed support tickets containing attacker-supplied instructions, and integration tokens ended up in a public thread. Excessive privilege plus untrusted input — that’s the whole recipe, and it’s the top entry in OWASP’s MCP Top 10 for a reason.

The scale of the sprawl backs this up. GitGuardian counted 28.6 million new secrets exposed in public GitHub commits in 2025 — a 34% jump, the biggest in the report’s history — including over 24,000 unique secrets sitting in MCP configuration files specifically. Agents didn’t invent secrets sprawl. They industrialized it, because an agent commits, configures, and copies at machine speed, and it never gets the uneasy feeling a human gets pasting a key somewhere it doesn’t belong.

Three ways agents break the human credential model

It’s worth being precise about what’s actually new here, because “treat the agent like an employee” — the framing half the vendor content uses — leads you to the wrong controls.

Speed. A human with an over-scoped key misuses it occasionally, detectably, at human pace. An agent makes hundreds of tool calls an hour. If one of them is wrong — wrong table, wrong environment, wrong recipient — the blast radius is bounded only by what the credential allows and how fast you notice. I wrote about Amazon learning this in public: the mistake wasn’t novel, the amplification was.

Injectability. You cannot socially engineer your deploy script. You can socially engineer your agent, remotely, in writing, at scale, by putting words where it will read them. Every credential in the model’s reach is one crafted document away from disclosure. This is why the autonomy question and the credential question are the same question wearing different hats.

Ambient authority. Humans accumulate scoped logins with MFA and session expiry. Agents accumulate environment variables — long-lived, unscoped, invisible. OWASP’s agentic guidance pushes “least agency” alongside least privilege: don’t just ask what the agent can access, ask how much it can do with that access before a human checks. An env file full of god-keys maximizes both, silently.

The broker pattern: credentials live with tools

The architectural fix is small enough to describe in one paragraph. Between the model and every side effect sits a broker — the code that receives the tool call, decides whether to run it, executes it with a credential the model has never seen, and returns only the result. If you’ve read the 50-line agent loop, you’ve already seen the shape: the loop’s execute_tool/2 function is the broker seam. You just have to treat it as a security boundary instead of a dispatch table.

In Elixir it barely deserves the word “pattern”:

defmodule Agent.Broker do
  # Each tool gets its own credential, fetched at call time —
  # never placed in the prompt, never returned to the model.
  @tools %{
    "search_orders" => {Tools.Orders, :readonly_db_url},
    "refund_order"  => {Tools.Refunds, :stripe_restricted_key},
    "send_email"    => {Tools.Email, :smtp_send_only}
  }

  def call(tool_name, args, ctx) do
    with {:ok, {mod, cred_key}} <- Map.fetch(@tools, tool_name),
         :ok <- Policy.allow?(tool_name, args, ctx) do
      cred = Secrets.fetch!(cred_key)
      result = mod.run(args, cred)
      AuditLog.record(ctx.run_id, tool_name, args, summarize(result))
      {:ok, redact(result)}
    end
  end
end

Three properties matter more than the code. The credential is fetched inside the call, so it exists in memory for the duration of one tool execution, not for the lifetime of the agent process. The result is redacted before it returns — if the tool response could contain secrets (config dumps, user records with tokens), strip them, because whatever goes back to the model becomes context, and context can be exfiltrated by the next injection. And every call is logged with the run ID, because when something goes wrong the first question is “what did the agent touch,” and grepping model transcripts is a miserable way to answer it.

If your agent reaches tools through MCP servers instead of in-process functions, the boundary moves but the rule doesn’t: the MCP server holds the credential, scoped to what that server does, and the config that launches it is a secrets-bearing file — which is exactly where GitGuardian found those 24,000 leaked keys. Treat mcp.json with the paranoia you’d give .env, because it is one.

Scope per tool, not per agent

The unit of least privilege is the tool, not the agent. “The support agent’s key” is already too coarse — the support agent’s order-lookup tool needs a read-only database role limited to two tables; its refund tool needs a Stripe restricted key capped at refunds; its email tool needs send-only SMTP with your domain locked. Each of those is a five-minute setup task in the respective dashboard, and each converts a catastrophic injection outcome into an annoying one. An attacker who fully owns the model in that architecture can issue refunds until your cap trips. They cannot read your user table, mint API keys, or touch the infrastructure, because no tool in reach holds a credential that can.

This is the same argument I make in the AI-native security stack post at the company level; this is what it compiles down to at the code level. Providers have quietly made it easy: Stripe restricted keys, Postgres roles, GitHub fine-grained PATs, AWS IAM with resource conditions. The primitives are all old. What’s new is having a caller that will predictably, mechanically use everything a credential allows — which turns scoping from best practice into load-bearing wall.

Short-lived beats static, where it’s cheap

Static keys leak and stay leaked; the GitHub incident-response literature is a graveyard of keys revoked months after exposure. Every credential your broker fetches should be as short-lived as your provider makes convenient: AWS STS sessions instead of IAM user keys, OIDC-federated tokens in CI, database credentials from Vault-style dynamic issuance if you already run it — and if you don’t, a weekly rotation habit on the handful of static keys you can’t avoid.

The honest small-team version: you will not get everything short-lived, and you don’t need to. Rank by blast radius. Cloud-account credentials and anything that can move money go first. The key to your error tracker can stay static without keeping anyone up at night. OWASP’s framing — task-scoped, time-bound permissions — is the ideal; a five-person team gets most of the value from applying it to the three credentials that matter and consciously deferring the rest.

The five-person reality check

Since “it depends on your stage” is doing a lot of work above, here’s the explicit line I’d draw for a team of five with agents in the development loop and maybe one in production:

Fine: secrets in environment variables on the tool-executing process, loaded from a managed store (Fly secrets, AWS SSM, 1Password CLI — whichever you already have). Per-tool scoped keys. A shared runbook that says which keys exist and who rotates them. Agents in dev running with dev credentials against dev data.

Negligent: any credential in a prompt, system message, or file the agent is instructed to read. One key shared across tools “for now.” Production credentials in the environment of a coding agent that executes arbitrary shell commands — remember that cloning a repo was enough to redirect an API key in CVE-2026-21852; your agent’s environment is attack surface even when the agent behaves. No log of agent tool calls. No tested way to shut it off.

That last pair costs an afternoon, total, and it’s the difference between an incident and an anecdote.

Kill switch, then rotation

When an agent misbehaves — and the base rate says eventually one will — you need two moves you’ve rehearsed. First, stop the agent: a feature flag the broker checks on every call beats hunting for the process at 2am. Second, rotate what it held: which is only tractable if you scoped per tool, because “rotate everything in the env file” is exactly the Saturday you built all this to avoid. This is also where the audit log pays for itself — rotation scope equals the set of credentials behind tools the run actually called, and the log is the only thing that knows.

What not to buy yet

There is a fast-growing category of agent-identity platforms — non-human identity management, agent IAM, credential brokers as a service. The pitch is real for enterprises running fleets of agents across hundreds of systems with compliance reporting on top. At pre-Series-A, with one product and a handful of tools, the platform mostly sells you a hosted version of the broker function above, plus a new vendor in your critical path and a new place your credentials live.

Buy it when you have the problem it solves: many agents, many teams, auditors asking for agent-access reviews, or a credential graph you genuinely can’t hold in your head. Until then the boring stack — scoped keys, a broker seam you own, your platform’s secret store, one log — covers a five-person team with money left over. This is the same prove-the-return discipline that applies to every tool purchase; security tooling doesn’t get an exemption.

The part that doesn’t change

Strip the AI vocabulary and this post says: don’t give programs more authority than their job needs, don’t put secrets where untrusted input can reach them, log privileged actions, and be able to revoke fast. That was true before LLMs. What agents change is the default — for the first time, the path of least resistance hands a credulous, injectable, machine-speed actor the keys to everything, and the ecosystem’s leak statistics show exactly how that’s going.

If you’re standing up agents with real credentials and want a second pair of eyes on the boundary — what’s scoped, what’s brokered, what you can safely defer — that’s the kind of engagement I run.