Superintelligence and the CFO: Budget, Risk and Capital Questions

For a CFO, superintelligence is a planning constraint more than a forecast. Model capability may improve faster than budget cycles, so the sound response is to fund the parts of an intelligence layer that keep their value whichever model comes next: company memory, tools, permissions and evaluations. Those parts can be staged, measured and stopped, which makes them investable under uncertainty.

Key takeaways

• Treat the investment as a set of options, not a bet on one prediction about future models.
• Separate what depreciates with each model release from what compounds, such as memory, tool integrations, permissions and evaluation sets.
• Tie every funding stage to an observable success criterion and a named owner, so each tranche can be released, held or stopped.
• Put dependency, data exposure and unmanaged change on the risk register, next to the cost line.
• Ask for unit economics early: the cost of one completed piece of work, not the cost of access to a model.

Why is superintelligence a finance question and not only a technology one?

Most discussion of superintelligence centers on when, or whether, AI systems will exceed human performance across many kinds of work. A finance leader cannot resolve that debate and does not need to. The practical issue is that the range of plausible outcomes is wide, and capital allocated today will be judged against a world nobody can describe precisely.

That is a familiar problem. Finance teams already fund projects where the demand curve, the regulation or the input costs are uncertain. The tools are the same: stage the commitment, define what must be true before the next tranche, and keep the ability to change direction. What is different here is the pace at which the underlying technology changes, and the fact that much of the spending looks like a subscription while the real value sits in how the company configures and governs what it buys.

If you want the conceptual background first, the superintelligence glossary and superintelligence versus generative AI explain the terms. For budgeting, the useful move is to stop asking how powerful models will become and start asking which of your investments become more valuable if they do.

Which questions should a CFO ask before approving spend?

The questions below are meant to be asked of whoever is proposing the work, whether that is an internal team or an outside partner. They are deliberately plain, and a proposal that cannot answer them is not ready for approval.

• What single role or task is this money for, and who owns the outcome today?
• What is the observable success criterion, and how will we see it without relying on a vendor's report?
• What part of the spend is tied to a specific model, and what part survives if we change models?
• What data, tools and permissions will the system touch, and who approves each one?
• What does it cost to stop, and what do we keep if we do?

The fourth and fifth questions are the ones most often skipped. Permissions determine the size of the downside, and exit cost determines whether a staged commitment is real or only described that way. A plan that cannot say what you keep after stopping is a purchase with a different label.

How do you budget when future model capability is unknown?

The discipline is to separate commitments by how reversible they are. Some spending is cheap to reverse: a short pilot, a limited set of users, a single integration. Other spending is slow to reverse: multi-year contracts, deep dependence on one vendor's proprietary format, retraining a team around one tool. Under uncertainty, front-load the first kind and be sceptical of the second.

A second discipline is to fund the layer that does not depreciate. A new model release can make last year's prompt tricks obsolete. It does not make your cleaned and organized company knowledge obsolete, nor your approval rules, your tool connections, or the test cases that define what good work looks like. Those assets tend to improve the value of any model plugged into them. The company superintelligence layer article describes this structure in detail, and the budget framework for superintelligence readiness walks through sizing it step by step.

A third discipline is to match each stage to evidence. The proof of concept, pilot or production article sets out what each stage must prove. A finance gate can borrow that logic directly: the proof of concept proves feasibility, the pilot proves value under real conditions, and production proves that operations, support and controls hold up.

Illustrative scenario: a distribution company wants an agent to prepare supplier follow-ups. The first tranche funds one role, read access to two systems, and a set of example cases written by the operations lead. The second tranche is released only if the agent's drafts are accepted with light edits in the pilot, and only for the same two systems. If a much more capable model appears midway, the company swaps the model and reruns the same cases. The tranche structure did not change, and nothing built so far was thrown away.

What costs and assets does an intelligence layer create?

It helps to sort the ledger into four groups, because they behave differently over time.

• Model access: variable, falls and shifts with vendor decisions, and should be easy to change.
• Integration and tooling: the connections to your systems of record, which need maintenance when those systems change.
• Knowledge and memory: the organized company context the agent works from, which gains value as it is corrected and extended.
• Operations and oversight: monitoring, evaluations, permission reviews and incident response, which are recurring and should be budgeted as such.

The last group is the one that surprises finance teams. A system that acts on behalf of the company is not finished when it is deployed. It needs people watching it, testing it and adjusting it, and that work belongs in the run-rate from the start. The what managed AI services include article lists the recurring work so that it can be planned for rather than discovered.

The FinOps Foundation's guidance on unit economics offers a useful frame: relate cost to the unit of value delivered, such as one reconciled invoice, one qualified lead summary or one resolved request. A unit cost can be compared with the cost of the same work done today, and it keeps the conversation away from the headline cost of access, which says little about whether the work is worth it.

Ownership matters too. Whether the central budget or individual departments carry the cost changes who feels the savings and who pushes for adoption. The who pays for the AI agent article covers the trade-offs.

Which risks belong on the CFO's register?

A risk register for this kind of investment should be short and specific. The NIST AI Risk Management Framework is a good reference for structure: it organizes the work around governing, mapping, measuring and managing risk, and it can be applied at the scale of one agent as well as a whole program.

The entries that most often matter to a finance leader are these.

• Vendor dependency: the cost and effort of moving if pricing, terms or product direction change. See how to avoid AI vendor lock-in.
• Data exposure: what company knowledge leaves your control, and under which terms it may be retained or used.
• Excess permissions: an agent that can do more than its task requires turns a small error into a large one.
• Unmanaged change: edits to prompts, tools or models that nobody approved or recorded.
• Unobserved quality: output that looks fine in a demo and degrades quietly in daily use.
• Key person risk: knowledge of how the system works held by one individual or one outside builder.

None of these is a reason to wait. Each has a control that is inexpensive compared with the cost of discovering the risk after the fact: approval boundaries, a change process, periodic permission reviews, and logs that someone actually reads. The point for the CFO is to confirm the controls exist and have an owner, then ask to see the evidence on a schedule.

What step-by-step review can a finance team run?

This plan can be run in a few working sessions with the executive sponsor and whoever will operate the system. It produces a decision, not a report.

• Step one: name the role. Write one sentence describing the work the agent will do and the person accountable for its results.
• Step two: set the baseline. Record how that work is done today, how long it takes in effort terms, and how errors are caught.
• Step three: define the success criterion. Choose something observable, such as the share of drafts accepted without rework, and decide who measures it.
• Step four: list the assets that persist. Memory, tool connections, test cases and written procedures should belong to the company and be exportable.
• Step five: set approval boundaries. Decide which actions the agent may take alone, which need a human approval, and which are never allowed.
• Step six: split the commitment into tranches. Attach a release condition to each, and agree in advance what happens if it is not met.
• Step seven: schedule the reviews. Put permission reviews, evaluation reruns and a contract check on the calendar before launch, not after the first incident.
• Step eight: write the exit. Describe how the work is paused or stopped and what the company keeps, drawing on how to stop an AI pilot without wasting the work if needed.

A CFO who has these eight answers on one page can approve the first tranche with confidence and can explain to a board why the commitment is sized the way it is.

How does a managed approach handle this?

Be Human designs, implements and operates personalized AI infrastructure for companies. A first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion, which maps directly onto the first four steps above. Managed operations can then include monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, so that the oversight work is planned and visible rather than left to whoever has time. The agent's identity, memory, channels, tools and permissions stay under human control, and the company can see what changed and why.

If you want to find the first task worth handing to an agent, take the free Be Human AI diagnostic.

Frequently asked questions

Should a CFO wait until the direction of AI capability is clearer? Waiting has its own cost, because the assets that compound, such as organized knowledge, approval rules and test cases, take time to build. A small, staged first commitment gives you learning without locking in a position on the future. The decision to wait is better made role by role than for the company as a whole.

How should the first investment be sized? Size it to one role with a measurable outcome, and fund only as much as that role needs to reach a pilot decision. The size of the later tranches should depend on what the pilot shows, not on the ambition of the overall program. The AI agent statement of work article shows how to write scope so that this works contractually.

What evidence should finance ask for during a pilot? Ask for the success criterion measured the way it was defined at the start, a sample of real outputs with the edits people made, and a record of changes and permission reviews. Independent evaluation sets, such as those described in OpenAI's evals guide, help show whether quality holds when models or prompts change. Be cautious of summary claims you cannot trace to specific cases.

Who should own the budget line? Ownership usually works best when the person accountable for the outcome also sees the cost, with a shared view held centrally for oversight and risk. The right split depends on how many teams will use the system and who captures the benefit. Decide it before launch so that adoption and cost are not argued over later.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.