Green SARC: Predictive Cost and Carbon Governance for Agentic AI Systems

The state ingests financial and environmental telemetry: . Of these, and are enforced at the gate; is declared for completeness but is not enforced in the Phase-1 implementation (the field exists in the state object and is unused), a divergence we record honestly in §15.

Let be a proposed action in context . A learned estimator predicts expected token cost and carbon before the action fires. The implementation regresses completion tokens on prompt tokens online per key using the Welford-style sufficient statistics [25], exposing the residual standard deviation . The gate admits iff the forecast fits the remaining budget at confidence and the carbon ceiling:

Operationally the first test is a one-sided upper bound . The implementation forms with the normal quantile (the “Normal- gate”); §7 replaces with a distribution-free conformal margin . Rule-based accounting is the special case a constant threshold independent of : the zero-information gate.

The estimator is trained on the Post-Action Auditor’s own output, closing a loop:

At cold start is weak and the gate behaves conservatively (worst-case forecast); it sharpens as the audit log accumulates. §8 shows the forecast MAE collapsing from a cold-start value of 4,000 tokens to the irreducible noise floor within 20 observations.

The estimator is built in two phases. Phase 1, per-action: predicts the next step only — simple to deploy, and it generates the labeled actuals needed for Phase 2. Phase 2, full-trajectory: a planner-level estimator predicts the cost of an entire plan before the agent starts, enabling rejection of expensive plans, not merely expensive steps. Phase 1 is the data engine for Phase 2; only Phase 1 is implemented here (Phase 2 is an interface stub).

We give two safety statements: a pointwise one assuming calibration, and a distribution-free one (proved in §7).

Within the gated action space the agent optimizes a reward that penalizes brute-force inference where deterministic computation suffices:

with task utility , FinOps weight , GreenOps weight . As SARC argues, this finite penalty shapes behavior within the hard budget/carbon constraints; it does not replace them. §13 makes this quantitative: a soft penalty tuned to the budget in expectation still breaches it most of the time.

Budget and carbon are hard constraints enforced at their sites, not merely penalized in .

Proposition 2 is only as good as the calibration premise; a systematically optimistic admits overspend. Theorem 2 discharges the premise without a distributional assumption, at the cost of a held-out calibration set. §7 states the assumptions, proves the bound, and validates coverage empirically.

The split-conformal bound of §7 and the adaptive variant of §10 are no longer only paper-side analyses: v0.3.0 ships them as runtime strategies in green_sarc.calibrator (SplitConformal, ACIConformal, behind a Calibrator protocol). The PreActionGate constructor gains an optional calibrator=... argument; supplying it replaces the Normal- token bound with the conformal one, and omitting it preserves the prior behaviour exactly (all pre-existing tests pass unchanged, and make verify holds).

We validate the runtime path against the paper-side analysis: re-running the §10 ShareGPT study with the runtime SplitConformal calibrator (--use-runtime-conformal) reproduces the held-out coverage of the offline analysis to within percentage points across . Engineering contract: the calibrator is fit once from a residual log (offline) and, for ACIConformal, updated online from realized-vs-predicted cost at the Post-Action Auditor; the protocol lives in green_sarc.calibrator and the public surface follows semver within . The difference between “we prove” (§7) and “we ship” is now a one-argument opt-in.

Three integration patterns, in increasing order of loose coupling:

1. 1.

   In-process — GreenGovernor.with_defaults(...) wraps the agent’s executor directly. Step-level safety, single replica; best for single-agent CLI tools and notebooks (green_sarc.governor).

2. 2.

   PAIS sidecar — ASGI middleware on /v1/chat/completions; returns HTTP 429 on reject, with SSE-aware passthrough for streaming. Best for agents fronted by an OpenAI-compatible API (green_sarc.adapters.pais_sidecar).

3. 3.

   KAOS-managed MCP advisory + OTel observe — register Green SARC as an MCP server for advisory gate/audit tools, and consume actuals cross-process via the OTel SpanProcessor. Loosest coupling, advisory-only safety (green_sarc.adapters.mcp, green_sarc.adapters.otel).

We sweep the token budget over , with (the mean full-snowball cost), seeds each, running the full Green SARC stack at . The estimator is warm-started on an independent stream so we measure steady-state binding-budget behaviour, not the cold-start transient. We report, per budget: admission rate (admitted attempted steps); over-budget incidence (fraction of admitted steps whose realized cost exceeded the budget remaining at the moment of admission — the per-action breach event of Theorem 2); completed-trajectory rate (fraction of SKUs whose every step ran before the budget was exhausted); MAE on admitted actions; and total tokens.

Table 5 reports the sweep. Over-budget incidence is at every budget level — comfortably within the target — confirming Theorem 2 empirically: the calibrated gate does not admit actions it cannot afford. Admission and completion degrade smoothly and monotonically as the budget tightens: at (a budget one-quarter of the naive baseline) the gate still admits of attempted steps and completes of trajectories with zero overspend; from the budget is slack and everything completes. Forecast MAE is stable ( tokens) across budgets.

Figure 6 plots completed-trajectory fraction against over-budget incidence for the gate (sweeping ) and for the §13 soft penalty (sweeping its weight , with over-budget measured against the binding reference ). The gate’s frontier lies along the bottom axis ( over-budget at every completion level it reaches), dominating the soft penalty, which can only complete most trajectories by breaching the budget on every seed. The penalty frontier is essentially bimodal: on this workload its realized spend jumps from “admit-cheap” (little completed, within budget) to “admit-all” (everything completed, over budget), with no intermediate tracking the budget — a direct consequence of its budget-blindness.

The gate’s empirical over-budget incidence tracks across the entire budget grid (it never exceeds it, here attaining ); admission degrades smoothly as tightens rather than collapsing; and the gate’s work-vs-overspend frontier dominates the soft-penalty baseline of §13. This is the binding-budget evidence that §8’s non-binding headline could not provide.

We replay anon8231489123/ShareGPT_Vicuna_unfiltered [14], a public corpus of real ChatGPT/GPT-4 conversations on the Hugging Face Hub, released under a permissive research license. We use ShareGPT because LMSYS-Chat-1M is access-gated and would not reproduce on a clean clone without a credential; ShareGPT is ungated and serves the same purpose. No LLM is called: we use token counts only. Turns are tokenized with tiktoken (cl100k_base); for each assistant turn we form the pair , capped to a realistic deployment window. This yields pairs across conversations, split into calibration () and test () by a conversation-level partition (§10.3).

We fit online OLS on the calibration split. The residuals (Figure 7) have skew and excess kurtosis ; an Anderson–Darling test gives statistic , far above the critical value , and D’Agostino’s test returns : normality is decisively rejected. The Gaussian- assumption underlying the Phase-1 runtime gate does not hold on real traffic.

We split the calibration and test sets by conversation: every turn of a conversation is assigned wholly to one side, so that within-conversation residual correlation never straddles the split (a row-level shuffle would leak it, violating exchangeability; on this corpus that leak inflates the reported conformal coverage at by pp, so the conversation-level number we report below is the honest, slightly looser one). Figure 8 compares empirical coverage to nominal on the conversation-level test split. The Normal- gate is mis-calibrated: it over-covers at loose (e.g. pp at , wasting budget) and, more dangerously, under-covers at the tight one actually deploys — pp at and pp at , i.e. roughly more budget breaches than promised. Split conformal stays within percentage points of nominal across the entire range ( pp at ). On real residuals the distribution-free bound is not a nicety — it is the difference between a gate that keeps its safety promise and one that quietly violates it at the operating point.

The paper now spans a synthetic-residual world (§4–§9), where Gaussian and conformal bounds coincide because the data-generating process is Gaussian by construction, and a real-residual world (§10), where they diverge and only conformal holds nominal coverage. Conformal calibration is the bound that survives both. This is also why §15 lists promoting conformal into the runtime gate as the leading Phase-2 item.

Coverage guarantees assume the deployment distribution matches calibration; real workloads drift. We split the corpus by conversation into a short-context regime (calibration) and a long-context regime (deployment) — classifying each conversation by its own maximum depth so all of its turns land in one regime — then train the conformal quantile on the former and deploy on the latter ( vs. pairs). Figure 9 shows the result. The fixed quantile mis-covers post-shift — it drifts to against a target ( pp off, here over-conservative, needlessly rejecting work). Adaptive conformal inference (ACI [11]), updating the quantile level online at rate , restores empirical coverage to ( pp off target) within the rolling window. Under drift, the static conformal quantile is no longer sufficient; ACI is the runtime mechanism that maintains the guarantee.

We use BurstGPT [15] (BurstGPT_1.csv, CC-BY-4.0), a trace of real Azure OpenAI traffic with schema (Timestamp, Model, Request tokens, Response tokens, Total tokens, Log Type). We take a -request sample (after dropping failed responses, Response tokens); the model mix is GPT-3.5 (“ChatGPT”) and GPT-4 requests, mapped to the benchmark’s efficient and frontier profiles respectively. Request tokens is the prompt and Response tokens the realized completion — the gate’s per-action target. No LLM is called: token counts only, as in §10.

BurstGPT v1.0 carries no session identifier, so we reconstruct trajectories by temporal clustering: consecutive same-model requests within a  s window are grouped, capped at depth (the IBP default); API-log rows are single-step. This yields trajectories with median depth (real serving traffic is dominated by independent single requests) and maximum depth . We set the Adapter-Node scope cap to the median prompt ( tokens) and the circuit breaker to the depth cap, and document both as policy choices. (When BurstGPT v1.1 ships SessionID, the clustering heuristic becomes a one-line group-by.)

Table 6 and Figure 10 report the ablation with paired-bootstrap CIs over trajectories. Scope (Adapter-Node prompt bounding) cuts tokens by and carbon by ; routing of trajectories to the efficient model adds USD savings () and carbon () at no further token cost — exactly the lever decomposition the synthetic ablation predicted (scope drives tokens, routing converts to USD/carbon). The savings ordering is confirmed on real arrivals.

We repeat the §9 sweep on the real trace ( baseline tokens, ; Figure 11). Over-budget incidence is at every budget — the gate’s safety guarantee holds on real arrivals exactly as on synthetic ones — while admission and completion degrade as the budget tightens (: admitted, of trajectories completed; : full completion). The gate frontier again dominates the soft-penalty frontier, which can reach high completion only by breaching the budget on every seed.

The IBP pipeline tightened two claims and the real trace now corroborates them: the lever ordering (scopetokens, routingUSD/carbon) and the gate’s over-budget incidence under binding budgets both reproduce on BurstGPT. Two claims weaken or shift. First, +full’s extra token saving in the IBP ( vs. ) came entirely from the breaker killing injected runaway SKUs; real arrivals have no such storms, so that increment vanishes — consistent with §10’s finding that real cumulative-prompt curvature is negative. Second, on real single-step traffic “scope” is simply context truncation: the Adapter Node caps each prompt at tokens ( the median), so the token reduction is largely the mechanical consequence of that cap, not an intrinsic property of the architecture. The headline percentage should therefore not be read as a free saving Green SARC delivers: it is a tunable policy knob whose realized magnitude scales with the cap, and whose quality cost (truncated context degrading task utility) is deliberately untracked here (§3). A different operator with a less aggressive cap would see a proportionally smaller number. What survives this caveat, and what we therefore present as the load-bearing claims of this section, are the two cap-independent results: the lever ordering reproduces on real arrivals, and the gate’s over-budget incidence is under binding budgets on real data exactly as on synthetic data. The IBP is thus best read as a controlled stress-test of the multi-step regime; BurstGPT confirms the per-request governance levers and the safety property on real distributions, but it is a single-step serving trace and does not exercise the multi-step snowball or breaker dynamics — a real multi-step agent trace remains the natural next validation (§15).

The carbon results so far use a stipulated intensity curve. We re-compute the BurstGPT carbon reductions under measured grid intensity for two zones with contrasting generation mixes.

BurstGPT is single-step; §11 flagged that it cannot exercise the multi-step snowball or breaker. We close that gap on real agent plans.

The paragraph above names a tradeoff; because SWE-rebench records task outcomes, we can bound it. Each trajectory carries the benchmark’s real resolved flag — whether the agent’s patch passed the held-out tests — and of the plans resolved. We sweep the scope cap (, the median per-step prompt) and, for each cap, report tokens saved against an upper bound on quality harm: a worst-case resolution rate that assumes every resolved trajectory whose actually-used context the cap would have truncated flips to unresolved. This is deliberately pessimistic and, crucially, observational — truncation is simulated on logged trajectories, so the agent cannot react to the smaller context. The replay therefore bounds how much resolved work a cap puts at risk; it cannot establish that the work would in fact fail (a live agent might recover by re-fetching). It is a correlational upper bound, not a causal estimate.

The sweep above varies one knob. To check that the headline operating point is not a cherry-pick, we sweep all three: , scope cap the median prompt ( tokens), and routing fraction — cells over seeds. Token/USD/carbon reductions use the benchmark’s native forecast noise; over-budget incidence is measured under a binding budget at the elevated noise of the stress above (Figures 18, 18).

Three findings, two of them null and stated as such. (i) Token reduction is governed almost entirely by the scope cap: at , at , at , at . (ii) Routing fraction does not move the token reduction at all (it reallocates models, changing USD and carbon, not tokens — the heatmap is flat along the routing axis), and (iii) has no measurable effect on either axis in this regime: the four panels of Figure 18 are identical, and over-budget incidence never exceeds across all cells. The last is the empirical face of Theorem 2: the gate admits on its upper bound, so realized over-budget events are vanishingly rare regardless of the operating point; becomes a live throughput-vs-overspend knob only under the higher forecast uncertainty of real residuals (§10). The paper’s headline operating point (cap , routing , ) achieves the maximum token reduction at over-budget and lies on the Pareto frontier ( of cells are non-dominated): it is the most aggressive cap at zero safety cost, not an interior cherry-pick.

The attacker is a prompt author with white-box knowledge of the estimator, the scope cap, and the budget, who observes gate decisions (and, via timing channels, possibly residuals). The attacker cannot modify src/ or mint tokens from nothing; the cost is always realized by the model provider. This is a cost-side adversary, distinct from the safety-side prompt-injection threat studied by [23, 24]: the goal is not to make the agent misbehave but to make it overspend while passing the gate.

We construct three attacks ( seeds, instances each, paired-bootstrap CIs). Continuation inflation: a prompt whose realized completion is the benign law (“continue indefinitely” semantics). Scope-cap-aware padding: a prompt sized to exactly tokens, maximizing admitted work per call. Model-substitution gaming: a prompt that declares the efficient model while the cost is realized at frontier rates (a misreported model id).

Two honest negatives. First, scope-cap-aware padding defeats the gate by staying inside its admission contract: it pads to just under the cap and extracts maximum legitimate work, so the gate admits it () with no over-budget event (, at the noise floor) and realized cost below the bound (). This is a fundamental limitation of bounded-prompt-only governance and is exactly what the Phase-2 trajectory estimator — which reasons about the whole plan, not one padded step — is meant to address. The gate alone is not sufficient here, and we do not claim otherwise.

Second, continuation inflation and model substitution both defeat the forecast (realized cost and the admitted bound), but they are caught post hoc by the Post-Action Auditor, which logs predicted-vs-actual and feeds the discrepancy back to the estimator and the Escalation Router. The architectural response to a forecast-defeating attack is audit-then-revoke at the Auditor, not admission-time rejection at the gate; this is the cost-domain instance of SARC’s predict–act–log–retrain loop. The gate bounds expected cost; it does not bound an adversary who lies about the future, and the four-site architecture — not the gate in isolation — is what makes the residual detectable.