AI Agents for Risk Management: Monitoring With Evidence

AI agents for risk management work best as monitors and preparers, not as decision makers. They read the sources your team defines, notice conditions that match rules your team wrote, and hand a named person a short case file in which every claim points to the record behind it. The risk owner still judges, approves and signs.

Key takeaways

• A useful risk agent has three jobs: watch defined sources, assemble evidence, and route a case to a named owner.
• Every alert should carry its evidence: the source, the exact excerpt, the time it was read, the rule that fired and what could not be checked.
• Decisions such as risk acceptance, limit changes and contact with regulators stay with people, and the agent's access is read-only by default.
• Start with one risk theme, run the agent in shadow mode, and judge it by whether reviewers can verify alerts without opening other systems.
• Replay past cases, log every run and review permissions on a schedule, so the agent stays trustworthy as sources and rules change.

What does a risk monitoring agent actually do?

Risk teams spend much of their week reading and reconciling: registers, incident logs, supplier notices, control test results, exception reports, regulatory updates. The work matters, but a large part of it is repetitive attention. A monitoring agent takes over that attention load. It has a defined list of sources, a defined set of conditions that matter, and a defined recipient for each kind of finding.

In practice the job splits into three parts:

• Watch: read the sources on a schedule or when something changes, and compare what it finds with rules the team wrote in plain language.
• Assemble: when a rule matches, gather the relevant records, the history of the same item and any earlier alerts, and put them in one case file.
• Route: send the case to the named owner with a clear question, then track whether anyone acknowledged it and remind the right person if nobody did.

What it does not do is just as important. It does not rate a risk as accepted, close a finding, change a limit, contact a regulator or tell a customer anything. Those are human acts with human accountability. If your risk work sits inside project delivery, the patterns in AI for a Project Management Office: Status and Risk Reporting overlap with this design, especially around reporting that people can trust.

Why must every alert carry its evidence?

An alert without evidence forces the reviewer to redo the agent's work. At best that saves nothing. At worst the reviewer trusts a plausible sentence that nobody can trace, and a confident error enters the risk record.

So the rule is simple: the agent states only what it can attach. Each case file contains an evidence package:

• The source and its location, such as the document name, the system and the record identifier.
• The exact excerpt or field values the agent relied on, not a paraphrase alone.
• The time the agent read the source, so the reviewer knows how fresh it is.
• The rule that fired, written the way the team wrote it.
• Related earlier cases for the same supplier, site, control or account.
• A short list of what the agent could not check, for example a mailbox it has no access to.

When evidence is missing, the agent marks the statement as unverified and says why. It never fills a gap with a guess. This is also why retrieval from approved sources matters more than a clever model: the agent should answer from your records and show which records, rather than from general knowledge. Reviewers learn quickly that a case file either checks out in a minute or is clearly labeled as incomplete, and that predictability is what makes the alerts usable.

Where should the line between agent and human sit?

Write the boundary down before the agent runs. A decision guide in three tiers works for most risk teams:

• The agent acts alone: reading sources, collecting records, formatting case files, sending reminders, logging what it did.
• The agent proposes and a person approves: severity ratings, escalation to senior leadership, draft wording for anyone outside the team, and changes to the monitoring rules themselves.
• Humans only: risk acceptance, control exceptions, limit changes, contact with regulators, contractual or disciplinary decisions.

Access should match the tiers. Give the agent read-only credentials scoped to each source, and nothing broader than the task needs. The [OWASP authorization guidance](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) on least privilege and deny by default applies directly. Any write action, such as posting to a register, should pass through an approval step that a person completes.

One more boundary deserves attention: text inside monitored documents is data, not instruction. A supplier notice or an email that says to ignore previous rules must not change the agent's behavior. The agent reads it, quotes it as evidence if relevant, and keeps following its own configuration.

How do you build a first risk monitoring agent?

Keep the first version narrow. One risk theme, one owner, a small set of sources. The steps below work as a plan:

• Choose one risk theme with a named owner, such as supplier continuity, access reviews or open audit findings.
• Write the monitoring rules in plain language, each with an example of a case that should fire and one that should not.
• List the sources and grant read-only access to each, with a person responsible for every connection.
• Define the evidence package for this theme, using the list above as a starting point.
• Define the escalation path: who receives a case first, who is next, what counts as acknowledged, and what happens when nobody responds.
• Run in shadow mode. The agent produces case files, but the team continues its normal work and compares the two afterward.
• Go live on one channel, such as a shared inbox or a ticket queue, and review the alerts together every week for a while.
• Set an observable success criterion, for example that reviewers can verify each alert from its attached evidence alone, and that every missed item found later can be traced to a rule or source gap.

Shadow mode is the step teams most often want to skip. It is also where you find that a rule is vague, a source is stale or an owner field is wrong, before anyone is woken up by it.

What does this look like in practice? An illustrative scenario

Illustrative scenario: a distribution company's risk team monitors supplier continuity. The agent reads the supplier folder, the purchasing system's order log and a feed of public notices the team approved.

One morning a rule fires. The insurance certificate on file for a supplier shows an expiry before today's date, a purchase order for that supplier was released after the expiry, and the notice feed mentions an operational interruption at a plant with the same name. The agent sends the supplier risk owner a case file with three excerpts, each linked to its record, the rule that fired, and one line marked unverified: whether a renewed certificate exists outside the shared folder. It also notes that the plant match is by name only and may be a different company.

The owner checks a mailbox, finds the renewed certificate, confirms the plant is unrelated and closes the case with a note. The agent then proposes adding that mailbox as a source for future certificate checks. The owner approves the change. Nothing was decided by the agent, yet the owner reached a conclusion in minutes with the proof in front of them.

For the supplier-side practices that feed such rules, see AI Agents for Procurement: Supplier Memory, Contracts, Limits, and for alerts that start from planning data, AI for a Supply Chain Planning Team.

How do you test the agent and keep it honest over time?

A monitoring agent is only as good as its last review. Three habits keep it reliable.

First, replay history. Collect past cases, both real incidents and items that looked alarming but were fine, and run the agent against them. Check whether it flags the right ones, whether the attached evidence is correct and whether it says unverified when it should. Repeat this whenever rules, sources or the underlying model change. Guidance on building such checks is covered in Anthropic's piece on evaluating AI agents.

Second, make every run observable. Log which sources were read, which rules were evaluated, which cases were sent, who acknowledged them and how long that took. The [OpenTelemetry observability primer](https://opentelemetry.io/docs/concepts/observability-primer/) is a useful reference for thinking in traces and signals. When a reviewer asks why something was or was not flagged, the log should answer.

Third, watch for alert fatigue. If reviewers dismiss many cases as noise, the rules need tightening, and a person should make that change. Track dismissals by rule, and treat a rule that is always dismissed as a defect. The same discipline applies in adjacent functions, as in AI for Quality Management: Nonconformances and Audit Prep.

For governance vocabulary, the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) offers a shared way to describe who is accountable, how risks are mapped, how performance is measured and how issues are managed. It helps when the agent's own risk has to be explained to an audit committee.

How does Be Human run this as a managed service?

Be Human designs, implements and operates this kind of agent for a company. A first engagement defines one useful role, such as a supplier continuity monitor, along with the memory and tools it needs, the approval boundaries and an observable success criterion. After launch, managed operations can include monitoring of the agent itself, recurring evaluations against past cases, integration maintenance, permission review, incident response and controlled improvements. The agent keeps its identity, memory and permissions under your control, and the service is delivered remotely in English and Portuguese.

If you want to find the first task worth handing to an agent, take the free Be Human AI diagnostic.

Frequently asked questions

Can an AI agent make risk decisions? It can prepare them, but it should not make them. The agent gathers evidence, applies the team's rules and proposes a severity, and a named person decides, approves exceptions and owns the outcome. Keeping that line explicit is what makes the agent safe to run.

How do you stop an agent from inventing evidence? Require every statement in a case file to link to a source record and an exact excerpt, and have the agent mark anything it cannot support as unverified. Then test this by replaying past cases and checking the attached evidence by hand. Read-only access to approved sources also limits what it can cite.

Where should a risk team start? Pick one risk theme with a clear owner and a small number of sources, such as expiring documents or open findings. Run the agent in shadow mode beside the normal process, compare results and adjust the rules before it sends anything live. A narrow start gives you evidence about quality before you widen the scope.

Does a monitoring agent replace analysts? No. It removes repeated reading and reconciliation so analysts spend more time on judgment, investigation and conversations with the business. The accountable work, deciding what to accept, change or escalate, stays with them.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.