Where Should an AI Agent's Passwords and API Keys Live?

An AI agent's passwords and API keys should live in a dedicated secrets manager, outside the prompt, the agent's memory, the knowledge base and any code repository. The agent's tool layer fetches a credential only at the moment of a call, and the model itself never reads it. Each credential should be scoped to the smallest access the role needs, have a named owner, and be rotated on a schedule you can verify.

Key takeaways

• Store credentials in a secrets manager with encryption and access logs, never in prompts, memory, chat threads, spreadsheets or shared documents.
• Keep the secret in the tool layer, not in the model's context. The model asks for an action; the tool attaches the credential.
• Prefer narrow, revocable credentials: delegated tokens or scoped service accounts before broad master keys.
• Rotate on a schedule and after any suspected exposure, with an overlap window and a test after each change.
• Ask your provider to show where secrets live, who can read them, how they rotate and what happens when one leaks.

Where should an agent's credentials actually live?

The short answer is a secrets manager: a system built to hold sensitive values, encrypt them, control who and what can read them, and record every read. Major cloud platforms offer one, and there are standalone options. The specific product matters less than the properties: encryption at rest, per-identity access rules, an audit trail, and a way to change a value without redeploying the agent.

It helps to list the places where credentials should not live, because these are where leaks usually start:

• The system prompt or any instruction file the model reads.
• The agent's long-term memory or knowledge base.
• Chat threads, support tickets and email, including messages to the agent itself.
• Spreadsheets, shared documents and project boards.
• Code repositories and configuration files committed alongside code.
• Personal laptops, browser password managers and sticky notes used by whoever built the integration.

Memory deserves special mention. An agent that remembers things is useful, but anything stored there can be retrieved, summarized and repeated later. If you want to think through what belongs in memory and what does not, AI memory vs chat history covers the distinction. Credentials belong in neither.

Why should the model never see the secret?

A language model processes everything in its context as text it can repeat. If a key sits in the prompt, a crafted email, a web page or a document the agent reads can try to persuade the model to print it. This is a recognized risk class: the OWASP guidance on prompt injection and its list of risks for LLM applications both describe attacks where untrusted content steers the model into disclosing data or taking actions it should not.

The safer design separates thinking from doing. The model decides that it wants to create a draft reply or look up an order. It emits a structured request. A tool layer, which is ordinary software with no persuadable judgment, checks that the request is allowed, fetches the credential from the vault, makes the call, and returns only the result. The secret never enters the conversation, so it cannot be repeated, logged as part of a transcript or stored in memory.

This also changes what a good tool looks like. A tool that accepts a free-form field called "api_key" or "password" from the model is a design smell. Credentials should be bound to the tool by configuration, not supplied by the model at runtime. The article on AI tool contracts explains how to define inputs and outputs so that sensitive values are never part of the interface.

The same applies to the protocols that connect agents to tools. If you use a standard such as MCP, the server that exposes the tool is the right place for credentials, not the agent's instructions. What is MCP, and why does it matter for business agents? gives the background.

Which kind of credential should the agent hold?

Not every credential is equal. Before deciding where to store one, decide which type the agent should have at all. A decision guide:

• If the agent acts on behalf of a specific person, such as reading that person's calendar, use a delegated OAuth token with limited scopes. It can be revoked by the user or an administrator without affecting anyone else. Google's OAuth 2.0 documentation describes how scopes and consent work.
• If the agent performs a system task that belongs to the company rather than a person, use a service account with its own identity and only the permissions that task needs. That way a departing employee never takes the integration with them.
• If a vendor only offers an API key, create a dedicated key for the agent, restricted to the narrowest permission the vendor allows, and label it with the agent's name and purpose.
• If the only option is a human's username and password, treat it as a last resort. Shared logins hide who did what and rarely support scoped permissions or clean revocation.
• Never reuse a key that a person, another system or another agent already uses. One credential per agent per system makes revocation and auditing simple.

The trade-offs between the first two options are covered in service accounts or OAuth tokens for AI agents. Whatever you choose, the scope should follow the principle in least privilege for AI agents: start with the smallest access and add only when a real task requires it. It is also worth separating read, draft and send permissions, as described in read, draft and send AI tools, because a leaked read-only key is a much smaller problem than a leaked key that can send or pay.

How should rotation work?

Rotation means replacing a credential with a new one and retiring the old one. It limits how long a leaked value stays useful, and it proves that your team actually knows how to change it. A key nobody has dared to rotate is a key nobody understands.

A practical plan:

• Name an owner for each credential, a person or role, not a team alias that nobody reads.
• Set a rotation schedule based on the risk of the system. Credentials that can move money, send messages or touch customer records deserve a shorter interval than a read-only reporting key.
• Create the new credential in the vault while the old one still works.
• Update the tool layer to use the new value, ideally by reference to the vault entry, so no code change is needed.
• Run a small test that exercises the real action, such as creating a draft or reading a test record, and confirm it succeeds.
• Revoke the old credential only after the test passes and a short overlap window has closed.
• Record the change: who rotated it, when, and what was tested.

Some credentials expire on their own, especially OAuth tokens. That is a feature, but the agent must handle it gracefully rather than failing silently or asking a user to paste a new token into a chat. The article on what an AI agent should do when a connection expires describes a recovery path that keeps secrets out of conversations.

Rotation should also be event-driven. Rotate immediately when someone with access leaves, when a contractor engagement ends, when a vendor reports an incident, or when you see the value anywhere it should not be.

What counts as exposure, and what do you do about it?

Exposure is broader than a public leak. Any place a credential can be read by someone or something that does not need it counts. Common paths include:

• A person pasting a key into a chat with the agent to "make it work".
• Error messages and debug logs that print request headers.
• Tool outputs that echo back the configuration they used.
• Exported transcripts, screenshots and support tickets that include any of the above.
• Setup documents written by whoever built the integration and later shared widely.
• Backups and test environments that copy production secrets.

A good setup reduces these paths by design: logs redact known secret patterns, tool responses never include request headers, and the agent is instructed to refuse and flag any message that contains something resembling a credential.

When exposure is suspected, the order of operations matters:

• Revoke or rotate the credential first. Investigation can wait a few minutes; a live key cannot.
• Check the access log of the vault and of the target system for use you do not recognize.
• Remove the value from the place where it leaked, including copies in transcripts and memory.
• Review whether the credential's scope was wider than needed, and narrow the replacement.
• Write down what happened and what changed, so the next review has evidence.

An illustrative scenario: a company has an agent that drafts supplier emails. During setup, a team member pastes a mailbox access token into the agent's chat because the first connection attempt failed. The thread is later exported for a training session. Nothing bad happens, but the token is now in a transcript, in an export and in the agent's recent context. The right response is to revoke the token, reconnect through the approved authorization flow, delete the transcript copies and add a rule so the agent flags any pasted credential. The incident costs an hour instead of becoming a story. For the email side of this setup, see connect an AI agent to email without risking your inbox.

How does Be Human handle agent credentials?

Be Human designs credentials as part of the agent's operating context, not as an afterthought. During the first engagement, each role is defined with the tools it needs and the permissions each tool requires, and every credential is created for that purpose alone. Secrets are held in a managed store and attached by the tool layer, so the model does not see them. Managed operations include permission review, rotation, monitoring of access and a defined response when a connection fails or a value may be exposed, with the client's approvers deciding what the agent may do.

What should you ask a provider to confirm?

You do not need to inspect the vault yourself. You need clear answers, preferably in writing and preferably demonstrated. A checklist to bring to a provider conversation:

• Where are our credentials stored, and which product or service holds them?
• Can the model read any credential, directly or through a tool response?
• Who at your company, and who at ours, can retrieve a credential value? Is each retrieval logged?
• Is each credential unique to our agent and our system, or shared across clients or tools?
• What scope does each credential have, and why is that the smallest workable scope?
• What is the rotation schedule, who owns it, and how is a rotation tested?
• What happens to credentials when a person on either side leaves the project?
• How do logs and transcripts avoid capturing secrets?
• What is the procedure if a credential is exposed, and how quickly can access be revoked?
• At the end of the engagement, how are credentials revoked and secrets deleted?

Use the same questions when you compare options. The acceptance checklist for your AI agent integration is a good place to record the answers and the evidence behind them, and the grid for comparing two agent proposals helps when more than one provider is in the running.

If an answer is vague, such as "we keep them secure", ask for the specifics. A competent provider can describe the mechanism in a few sentences and show a log entry that proves it works. Sensitive actions deserve extra scrutiny: before an agent gets any credential that can move money, read can an AI agent make payments for limits to set first.

Frequently asked questions

Is it acceptable to keep an API key in an environment variable? It is better than writing it into code, and many small setups start there. But environment variables are easy to leak through logs, crash reports and copied configuration, and they make rotation clumsy. For anything that touches customer data, messages or money, a secrets manager with access logs is the safer home.

Can I give the agent my own login so it can do what I do? You can, but it is rarely wise. The agent would inherit every permission you hold, actions would appear as yours in audit trails, and revoking access would mean changing your own password. A delegated token with limited scopes, or a dedicated service account, gives the agent only what the role needs and can be removed without disturbing you.

How often should agent credentials be rotated? There is no single correct interval. Set it by risk: credentials that can send, write or pay should rotate more often than read-only ones, and every credential should rotate immediately after staff changes or suspected exposure. What matters most is that the schedule exists, has an owner and is tested.

What should I do if someone pastes a password into the agent's chat? Treat it as exposed. Revoke or change the credential, delete the message from transcripts and any memory, and check access logs for unfamiliar use. Then configure the agent to refuse and flag messages that look like credentials, and give people an approved way to connect systems without typing secrets into a conversation.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.