Who Pays for That Model Call? Credit Metering in Algen Agent Runtime

A chatbot makes one model call per message, so pricing it is arithmetic. An agent doesn't. One request from a user can become twelve model calls: steps, a retry after a timeout, a fallback to a second provider, a repair of a malformed tool call, a summary of a history that grew too long. How many there will be isn't known until the run ends, and the user didn't see most of them.

If your product sells AI credits, or gives each customer a monthly allowance, every one of those calls has to be charged to the right account, and a customer with nothing left has to be stopped before the next call, not invoiced after it. We learned this the expensive way: early on, a single turn of the Studio Assistant used up an organization's credits by asking a model to check on a build again and again.

This post explains how Algen Agent Runtime meters model calls, and why we put metering in the runtime rather than in each agent.

The short version

1. Reserve, then call, then settle. Before any model call reaches a provider, the runtime asks your meter to hold the most that call could cost. Afterwards it charges what the provider reported. A call that fails is released without charge.
2. Every call goes through the meter, including the ones nobody asked for: retries, fallbacks, schema repairs, streamed calls and history compaction.
3. Every call says whose it is. The meter is told the tenant, user, run, agent and model, taken from the saved run, so a run that pauses for approval and resumes tomorrow still charges the same account.
4. A refusal ends the run cleanly. No retry, no fallback to a cheaper model, no blame on the provider, and an event and audit record that say why.
5. The runtime holds no money. It calls your meter; the ledger, the prices and the wallet stay in your product.

Why counting tokens afterwards isn't enough

The obvious approach is to read token usage off each response and subtract it from a balance. It fails in three ways.

It can overspend. With 50 credits left, a call that costs 80 still runs, because you only find out what it cost once it's done. Run ten conversations in parallel against the same balance and the overspend multiplies. Prepaid credits that can go negative aren't prepaid.

It misses calls. If the counting lives in the agent's own loop, it sees the agent's calls. It doesn't see the retry the framework made after a timeout, the second provider a router fell back to, or the summary call that compacted the history. Those are real tokens on your provider bill and nobody was charged for them.

It charges the wrong account. A multi-tenant service runs one model router for everyone. A run that waits two hours for a human approval resumes on a different worker, where whatever "current tenant" variable the host set at the start no longer exists.

Card payments solved the first problem long ago: authorize the most the purchase could cost, capture what it actually cost, and void the hold if it doesn't go through. The runtime does the same for model calls.

Reserve, call, settle

A meter is three async methods. You pass one to build_container(settings, usage_meter=...), and from then on the model router uses it for every call, for every agent:

class UsageMeter(Protocol):
    async def reserve(self, *, max_input_tokens: int, max_output_tokens: int, call: MeterCall) -> object: ...
    async def settle(self, ticket: object, usage: TokenUsage) -> None: ...
    async def release(self, ticket: object) -> None: ...

Reserve happens after the router has picked a model and before the request leaves. It gets the most the call could use: the estimated size of the request, tools and schemas included, and the request's output cap. The meter places a hold for that amount and returns a ticket, or refuses.

Settle happens when the provider answers. It gets the ticket and the usage the provider reported, so the customer pays for what was used, not for the worst case. The difference between the hold and the charge goes back to the balance.

Release happens when the call doesn't complete: the provider errors, the request times out, a stream breaks or the caller stops reading it. The hold is dropped and nothing is charged. Customers don't pay for your provider's outages.

Two details make the reservation honest.

• Estimates run short on some text. The runtime estimates about four characters a token, and code, JSON and non-English text often use fewer characters per token. usage_meter_margin scales up only what is reserved, so a margin of 4/3 reserves as if a token were three characters. Context packing and the run's own token budget keep the unscaled estimate.
• There is always an output cap. The runtime sets one on every agent step and compaction, so the reservation is never open-ended.

The calls nobody sees

Because metering sits in the router, below everything that makes calls, it covers calls the agent author never wrote:

Fallbacks deserve a second look. If a run falls back from an expensive model to a cheaper one, the meter sees the cheaper model on the new reservation, because the router fills in the provider and model for each candidate it actually tries. Weight credits by model and the customer pays the right rate for the model that answered.

Whose call is it?

The call argument is a frozen MeterCall: who the call is for, why it is made and which model is being tried.

The run fields are built from the saved run, not from anything in memory. A run that pauses for an approval, or one recovered after the process crashes, resumes with the same tenant, user and run, so its later calls charge the same account as its first ones. And metadata values, which often hold account ids, never appear in the runtime's events, audits or traces.

settle and release receive only the ticket, so a meter that needs to know the model or the account when it settles keeps the MeterCall inside the ticket.

When the credits run out

To refuse a call, the meter raises MeterRefusedError with a message meant for the user, such as "You have used this month's model allowance." What the runtime does next matters as much as the refusal:

• It doesn't retry. Asking again won't create credits.
• It doesn't fall back to another model. A refusal is a decision about the account, not a problem with the provider. Quietly switching to a cheaper model would spend money the customer said not to spend.
• It doesn't blame the provider. The refusal doesn't count towards the provider's circuit breaker or latency, so one customer running out can't take a healthy provider out of rotation for everyone else.
• It ends the run with a reason. The run is marked failed with the meter's message as its error, a budget.refused event and audit record name the provider and model that were refused, and the run's trace span is marked budget.refused.

Operators can alert on refusals separately from provider failures. "Customer out of credits" is a sales conversation, while "provider down" means someone gets paged.

One rule for meter authors: a refusal must be a MeterRefusedError. Any other exception from reserve is treated as an internal error, as it should be when your ledger database is down.

What the runtime doesn't own

The runtime ships no ledger, no price list and no wallet. Those belong to your product, because every product prices differently: per token, per credit, per seat with an allowance, with prices by model or a flat rate. The runtime's job is to guarantee that every call asks first, says whose it is and reports what it used. The meter decides what that's worth.

This is also different from the runtime's per-run budgets. A budget bounds one run (its steps, model calls and tokens) to stop a single run spinning out of control. A meter rations an account across all of its runs, conversations and agents. A product usually wants both.

In production: the Studio Assistant

The Studio Assistant in Algen Agent Studio is the meter's first real user. Before the runtime had metering, the Studio Assistant had its own, in the same small library inside Studio as its guards against wasted calls. When those guards moved into the runtime, metering moved with them, and Studio now passes a UsageMeter backed by the organization's AI credits:

• Credits are held in millionths, so a short call charges a fraction of a credit rather than rounding up to a whole one.
• It reserves with a 4/3 margin, so a call is never under-reserved, and settles at the real token count.
• Rates are frozen when the hold is placed. A price change in the middle of a call doesn't change what that call costs, and the settled charge never exceeds the hold.
• Cached input tokens are recorded separately, so the platform's own cost can be priced at the provider's lower rate for cached prompts.
• Holds expire. A reservation still open after 15 minutes, for example after a worker crash, is released by a sweep, so lost calls don't lock up credits.
• Each conversation can have its own limit. Beyond the organization's balance, a user can cap one conversation, and the meter refuses once the next call would pass that limit: "This conversation reached its credit limit; raise the limit to continue."

The incident from the start of this post would now end differently. The polling loop would be stopped by the conversation's limit long before the organization's balance, with a message that says why. (The real fix was still a better tool: one that starts the build, follows it and reports back, so the model doesn't poll at all. Metering limits the damage; it doesn't replace good tool design.)

Writing your own meter

Here is a meter that charges a workspace in credits weighted by model, condensed from the runtime's documentation:

from dataclasses import dataclass

from algen_agent_runtime import MeterCall, MeterRefusedError
from algen_agent_runtime.orchestration.container import build_container
from algen_agent_runtime.types.contracts import TokenUsage

MODEL_WEIGHTS = {"gpt-4.1": 4.0, "gpt-4.1-mini": 1.0}  # credits per token


@dataclass(frozen=True)
class Hold:
    account: str
    call: MeterCall  # kept for settle, which receives only the ticket
    hold_id: str


class CreditMeter:
    def __init__(self, wallets) -> None:
        self._wallets = wallets

    async def reserve(self, *, max_input_tokens: int, max_output_tokens: int, call: MeterCall) -> object:
        account = call.metadata.get("account_id") or call.tenant_id or "unattributed"
        weight = MODEL_WEIGHTS.get(call.model, 1.0)
        hold_id = await self._wallets.hold(account, (max_input_tokens + max_output_tokens) * weight)
        if hold_id is None:
            raise MeterRefusedError("You have used this month's model allowance.")
        return Hold(account=account, call=call, hold_id=hold_id)

    async def settle(self, ticket: object, usage: TokenUsage) -> None:
        weight = MODEL_WEIGHTS.get(ticket.call.model, 1.0)
        await self._wallets.charge(ticket.account, ticket.hold_id, (usage.input_tokens + usage.output_tokens) * weight)

    async def release(self, ticket: object) -> None:
        await self._wallets.cancel(ticket.account, ticket.hold_id)


container = build_container(settings, usage_meter=CreditMeter(wallets), usage_meter_margin=4 / 3)

The work is in wallets: a hold must be atomic, so two parallel runs can't both spend the last credit. That is a row lock or a conditional update in your database, not something an agent framework can do for you. The runtime repository's examples/pattern_guarded_tools has a runnable in-memory version that refuses a user with no allowance.

The runtime is still a pre-release, and the metering contract has changed once already: the call argument arrived one release after reserve did. Meters written without it still work, with a deprecation warning, until 0.2.

Five things worth copying

• Hold before you call. Reserve the most a call could cost, settle what it actually cost, release what failed. Counting afterwards lets accounts go negative.
• Meter below the agent, not inside it. Put the meter where every model call passes, or retries, fallbacks and compaction go unbilled.
• Attribute from saved state. Take the tenant and user from the persisted run, never from request-scoped variables that a pause or restart loses.
• Make "no credits" its own outcome. Don't retry it, don't fall back to another model, don't count it against the provider, and tell the user why.
• Keep the money out of the framework. The runtime guarantees every call asks; your product decides what it costs.