Skip to content
be-human.ai
← Back to the journal

Context & memory

Chunking Rules for Policies and Contracts in AI Agents

Chunk policies and contracts by clause, keep conditions and exceptions together, and tag each chunk so an agent never cites half a rule.

Be Human typographic cover with the title: Chunking Rules for Policies and Contracts in AI Agents
Conceptual illustration created with AI · Be Human

Chunking a policy or contract for an AI agent means splitting the text at clause boundaries, not at a fixed number of characters or tokens. Each chunk should hold one complete rule with its conditions, exceptions, and the definitions it depends on, plus an identifier that tells the agent where the rule came from. If a chunk can be quoted correctly on its own, it is the right size.

Key takeaways

  • Split on the document's own structure (sections, numbered clauses, sub-items) and never on length alone.
  • Keep every condition, exception, and limit in the same chunk as the rule it modifies.
  • Attach the document name, version, clause number, and owner to each chunk so citations can be checked.
  • Test retrieval with questions that depend on exceptions, because that is where half-clauses cause errors.
  • Treat each change to a clause as a re-chunking event with a named reviewer.

Why fixed-size chunks break rules

Most default pipelines cut text every few hundred tokens and call it done. That works for a product manual, where each paragraph stands alone. It fails for policies and contracts, where a rule often spans several sentences and the sentence that matters most sits at the end.

Consider this illustrative scenario. A returns policy says that items may be returned within a set window, except for custom orders, which are final sale unless the item arrived damaged. A fixed-size splitter places the window and the general permission in one chunk and pushes the custom-order exception into the next chunk. An agent that retrieves only the first chunk will confidently tell a customer that a custom order can be returned. Nothing in the chunk is false. The rule is simply incomplete, and the agent has no way to know it.

The same pattern appears in service agreements (a liability cap followed by the carve-outs that void it), in internal approval policies (a spending threshold followed by the exception for emergencies), and in hiring or safety procedures. The failure is not in the model's reasoning. It is in the unit of text the model was given.

Split on clause boundaries, not on length

Use the structure the author already created. Headings start a new chunk. A numbered clause is a candidate chunk. Lettered or bulleted sub-items belong to their parent clause when they qualify it, and become separate chunks only when they stand alone as complete rules.

A practical test: read the candidate chunk aloud and ask whether it would be misleading if quoted without the lines before and after it. If the answer is yes, the boundary is in the wrong place.

Definitions need the same care. When a contract defines a term in section one and uses it in section nine, the agent needs the definition available whenever section nine is retrieved. Store defined terms as their own short chunks, linked to every clause that uses them, rather than hoping the retriever finds section one by chance.

Keep conditions, exceptions, and limits together

A rule is only safe to cite when its boundaries travel with it. Before a chunk is accepted into the knowledge base, check that it contains:

  • The rule itself, stated in the document's own words.
  • Every condition that must be true for the rule to apply, including dates, amounts, roles, and regions written in the clause.
  • Every exception, carve-out, or override that points back to this clause.
  • The defined terms the clause relies on, either inline or through a linked definition chunk.
  • A pointer to any cross-referenced clause that changes the outcome, such as "subject to section 7.2."

If a cross-reference cannot be resolved inside the chunk, do not cut it off silently. Either include the referenced clause, or mark the chunk as incomplete so the agent knows to say that the answer depends on another rule and to route the question to a person.

Attach identity to every chunk

A whole clause is still dangerous if nobody can tell which version it came from. Each chunk should carry metadata that the agent can show when it cites a source: the document title, a version identifier, the section or clause number, the document owner, and a status such as current, superseded, or draft. Without these fields, an agent that retrieves a clause from last year's agreement will present it with the same confidence as today's rule.

The metadata also controls who sees what. A procurement clause may be visible to the purchasing team and hidden from frontline staff. Set those permissions at the chunk level, not only at the folder level, because a single shared file often contains both public and restricted rules.

For a deeper treatment of version handling, see Document Version Control for an AI Knowledge Base: Cite the Current One. For how the sizing decision interacts with retrieval, Chunk Size for AI Knowledge Bases: Keep Policies and Procedures Whole covers the parameters in more detail.

A step-by-step plan for chunking a policy library

Start small. Choose one document family that people ask about often, such as refunds, approvals, or standard supplier terms, and work through it before touching the full library.

  • Step 1: List the documents in scope and name one owner for each. The owner answers questions about intent when the text is ambiguous.
  • Step 2: Mark the structural boundaries in each document: headings, numbered clauses, and lettered sub-items.
  • Step 3: Draft chunks along those boundaries, then check each one for conditions, exceptions, and cross-references.
  • Step 4: Extract defined terms into their own linked chunks and confirm that each clause using them points to the right definition.
  • Step 5: Add metadata (title, version, clause number, owner, status, access scope) to every chunk.
  • Step 6: Write ten to twenty test questions, weighted toward exceptions and edge cases, and record the chunk the agent should retrieve for each.
  • Step 7: Run the questions, read the retrieved chunks, and log every answer that cites a partial rule.
  • Step 8: Fix the boundaries that failed, rerun the tests, and only then expand to the next document family.

The discipline of step 6 matters more than the chunking tool. A team that writes the expected chunk before running the test notices quickly when the retriever returns a neighbor instead.

How do you test that an agent never cites half a rule?

The useful test is not whether the agent answers correctly on easy questions. Easy questions are answered correctly by almost any setup. Test the questions where a partial chunk would produce a confident, wrong answer: the rule with an exception, the threshold with an override, the permission with a condition about who is asking.

For each question, check three things. Did the retriever return the complete clause and its exception? Did the answer state the condition and not only the permission? Did the citation point to the current version? A failure in any one of these is a chunking problem to fix, not a prompt problem to patch with extra instructions.

Keep the test set after launch. When a policy changes, add the new edge cases the team discovered. The evaluation approach described in [Anthropic's guidance on demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) is a useful reference for building this kind of repeatable check.

For agents that also need to answer from source records, How to give an AI agent a reliable source of truth explains why the chunk and the record it came from should stay linked.

How Be Human handles clause chunking in managed operations

Be Human treats chunking as an operating discipline, not a one-time import. In a first engagement, the team identifies the one role and the documents it needs, agrees on the clause boundaries and owners with the client, and builds the chunks with metadata and access scopes. Tests are written from the questions staff actually ask and from the exceptions in the documents. In managed operations, the same team reviews retrieval results on a recurring schedule, tracks changes to source documents, and reopens the chunks that a policy update touches. The client keeps control of which rules are authoritative and who approves changes.

Where chunking meets governance

Chunking decides what the agent can see. Governance decides what it is allowed to say and do with that information. The two should be reviewed together. When a clause changes, the owner approves the new text, the chunk is rebuilt with a new version, the old chunk is marked superseded, and the test questions for that clause are rerun. A short change log that records these steps makes it possible to explain later why an answer changed.

For teams that want a formal frame, the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) gives a vocabulary for mapping and measuring risks in AI systems, and a chunking review fits naturally into that work. For retrieval design more broadly, [Google Cloud's guidance on retrieval-augmented generation architecture](https://docs.cloud.google.com/architecture/rag-capable-gen-ai-app-using-vertex-ai) describes the components that a chunking decision affects.

If you are not sure which of your documents should become agent knowledge first, start with the ones where a partial rule would cause the most harm. The free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

How long should a chunk be for a policy or contract? There is no fixed length that works for every document. A chunk should be as long as one complete rule, including its conditions and exceptions, and no longer than necessary to keep that rule intact. Some clauses are two sentences and some span a full page, so length should follow the clause rather than a setting in the tool.

Should I split a clause that is very long? Split it only where the text contains a real boundary, such as a separate rule, a distinct condition, or a sub-item that stands alone. If you must divide a long clause, repeat the heading and the governing condition in each piece so that no part is quoted without its context. Mark the pieces as parts of the same clause so the agent can cite them together.

Do I need a separate chunk for definitions? Yes, in most contracts and many policies. Definitions are short, but they change the meaning of the rules that use them. Store each defined term as its own chunk, link it to every clause that depends on it, and make sure the agent retrieves the definition whenever it retrieves a clause that uses the term.

How do I know my chunking is working? Write test questions before you look at the results, focusing on exceptions, overrides, and conditions. For each question, check whether the retrieved chunk contains the full rule, whether the answer states the condition, and whether the citation points to the current version. If those three checks pass consistently across a set of edge cases, the chunking is doing its job.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →