Skip to content
be-human.ai
← Back to the journal

Testing & reliability

Agent Observability Tools: Build In-House or Buy, and What to Own

Compare building an AI agent observability layer in-house with buying or managing one, and list the minimum fields your company must own.

Be Human typographic cover with the title: Agent Observability Tools: Build In-House or Buy, and What to Own
Conceptual illustration created with AI · Be Human

Agent observability tools record what an AI agent received, which tools it called, what it changed, and why it stopped. For most businesses, the right first step is a thin observability layer built in-house: a complete, readable record of every action that writes, sends or charges, kept in a format the company controls. A managed platform becomes worth buying when the number of agents, the review load or the compliance demands outgrow that layer, and even then the minimum fields described below should stay in the company's hands.

Key takeaways

  • Agent observability answers three questions: what the agent did, why it did it, and whether the result was correct.
  • Log every tool call that writes, sends, refunds or books, with its inputs, outputs, permissions used and any human approval.
  • Build a thin layer in-house while you run a few agents with one accountable owner. Buy or manage a platform when review work and volume outgrow it.
  • Own the minimum fields outright, including a run identifier, the task, the tool calls, the outcome and the cost per completed task.
  • A dashboard only helps if someone reads it on a schedule, tunes the alerts and decides what changes.

What agent observability has to cover

Software teams have used observability for years. The OpenTelemetry documentation describes telemetry as signals such as logs, metrics and traces, which let operators reconstruct what a system did from the evidence it leaves behind. Agents need those same signals, plus a few that ordinary services rarely produce: the model version in use, the instructions and retrieved context that shaped a decision, the set of tools offered at that moment, and the sequence of steps the agent took through a multi-step task.

The central gap is that an agent can be fully available and still wrong. A server health dashboard will show green while an agent quietly books the wrong room, sends a customer the wrong price, or summarizes a contract and misses a clause. Observability for agents therefore has to record outcomes, not only uptime and response times. Two related pieces explain the practical side: AI agent observability: what should you actually log? and the discussion of silent failure detection for AI agent workflows.

A useful way to frame the scope is to ask, for every agent, what a wrong answer would cost and how long it would take to notice. The answer tells you how detailed the record must be and how often someone has to read it.

Building in-house: what you gain and what it costs

Building your own layer gives you control over retention, access and the shape of the data. You decide where traces are stored, who can read prompts that contain customer details, and how long records live before deletion. The start is usually modest: structured logs with a run identifier, a table of tool calls, and a daily summary that a named person reads.

The cost is quieter than the build itself. Someone must keep the schema current as tools change, maintain each connection to the systems the agent uses, tune alerts so they do not flood the inbox, and decide what happens when a trace reveals a problem. The team that builds the layer also becomes its only reader, so the layer succeeds or fails with that team's attention. Logs that nobody opens are a storage bill, not observability.

An in-house build is the right choice when you run a small number of agents, have one owner with time for this work, and already log your core systems well. It becomes fragile when every new agent needs a custom integration, because each one adds a schema to maintain and another place where data can drift from what the business believes happened.

Buying or managing a platform: where it earns its place

A platform earns its place when several conditions hold at once: many agents across more than one model provider, review queues that need assignment and tracking, retention rules that differ by region, or evidence requirements from customers and auditors. A vendor can also bring evaluation tooling and alerting that would take a team months to design well.

Buying carries its own obligations. Traces often contain the most sensitive material in the business, including customer messages, pricing exceptions and internal notes. Before any platform receives that data, the company should know where it is stored, which people at the vendor can access it, how long it is kept, and how deletion works in practice. Brazilian organizations that process personal data should check their obligations with the ANPD, and organizations serving users in the European Union should review how the EU Artificial Intelligence Act applies to their systems. The NIST AI Risk Management Framework gives a neutral structure for deciding which risks to measure and who owns each one.

A practical test: if the vendor disappeared tomorrow, could you still explain what each agent did last month, and why? If the answer depends on the vendor's export tools, the minimum fields have already moved outside your control.

The minimum fields every company should own

Whatever tool you choose, keep the following fields in a store you control and can export without asking anyone for permission:

  • A run identifier that links every message, tool call and output belonging to one task.
  • The task type and the business object it touched, such as an order, a ticket, an invoice or a document.
  • The role version, instruction version and model version active at the time of the run.
  • Each tool call with its inputs, the returned result, and whether it changed state in another system.
  • The permissions the agent used, and the name of the approver if a person signed off.
  • The final outcome, recorded separately from the agent's own claim that the task succeeded.
  • Errors, retries and stop reasons, including any loop that was halted and why.
  • The cost per completed task, not the cost per message, so usage can be judged against results.

Keep the list short enough that someone reviews it every week. Fields that nobody reads add cost without adding control. The operating notes that explain how to use these fields in daily work are worth documenting alongside them, as described in AI agent runbook: the operating notes worth keeping.

Illustrative scenario: a refund agent with no trail

Illustrative: a support team runs an agent that issues refunds up to a set limit. A customer reports being refunded twice. Without a run identifier that ties the refund tool call to the original ticket, the team spends an afternoon matching timestamps across three systems and still cannot say whether the agent retried after a timeout. With the minimum fields in place, the same question takes minutes. The run shows one refund call, a retry that returned a timeout, and a second call that succeeded. The fix belongs in the retry policy and in a check that stops duplicate writes, and the trace is what made the cause visible in the first place.

The lesson is not that the platform was missing. It is that the business needed a record it could read without help from the people who built the agent.

How a managed setup handles it, and where to start

Who reads the trace matters as much as what it contains. In Be Human's managed approach, the observability layer is designed around one useful role, with the fields, owners and review schedule agreed before the agent goes live. Be Human handles monitoring, recurring evaluations, permission reviews and incident response, while the client keeps ownership of the record. Changes to the agent are written to a log that the business owner can read, so the company decides what changes and when.

Start with one agent that already touches real work. Write down its minimum fields, assign one person to read a sample each week, and note every case where the trace did not answer the question you needed answered. Those gaps tell you what to add next. Once that routine is steady, you will know whether the next step is a larger internal build or a managed platform.

Take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

Should a small business build its own agent observability tool? Often, yes, if the business runs only a few agents and can name one owner who will read the records weekly. Start with structured logs, a run identifier and a short list of required fields. Revisit the decision when review work, the number of agents or compliance demands grow faster than that owner can handle.

Is an LLM tracing platform the same as agent observability? No. A tracing platform records calls, prompts and latency, which is an important input. Observability also requires knowing the outcome of each task, the permissions used, the approvals given and the cost per completed task, and someone has to act on what the record shows.

Which data should never be sent to an outside observability vendor without review? Be careful with customer identifiers, payment details, health or financial information, internal pricing exceptions and any credentials that might appear in tool outputs. Before sending traces anywhere, confirm the storage location, vendor access rules, retention period and deletion process. Redact or minimize fields where the task does not need the raw content.

How long should agent traces be kept? Keep them as long as your operational review, contractual obligations and applicable legal requirements need, and no longer than that. Set the period in writing, apply it to every store that holds traces, and test that deletion actually happens. Shorter retention reduces exposure, but the minimum outcome records should survive long enough to investigate incidents and reconcile disputes.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →