Platform — The router, the policies, the runbooks

THE ROUTER TABLE

Eight rows of policy, and the topology they describe.

Hover a row to draw the route it produces. Click it to open the runbook clause that produced the row — the table is the legend for the diagram, and both are rendered from the same policy.

Routing table
Route	Match	Tier	Fallback	P50
extract.invoice-lines	kind=extraction & schema_bound	SMALL	small-c → mid-a	240ms
`extract.invoice-lines: match: "kind == extraction and schema_bound" tier: small verify: { mode: schema, schema: invoice.v2 } escalate_after: 2 fallback: [small-c, mid-a] # never frontier for this route`
summarize.thread	kind=synthesis & tokens<2k	MID	mid-c → frontier-a	610ms
`summarize.thread: match: "kind == synthesis and input_tokens < 2000" tier: mid verify: { mode: judge, rubric: faithful.v1 } escalate_after: 2 fallback: [mid-c, frontier-a]`
translate.*	kind=translation	SMALL	small-a → mid-b	260ms
`translate.*: match: "kind == translation" tier: small pin: small-a # strongest multilingual target in tier verify: { mode: schema } fallback: [small-a, mid-b]`
classify.intent	kind=classification	SMALL	small-b → small-c	180ms
`classify.intent: match: "kind == classification" tier: small verify: { mode: schema, schema: intent.enum } escalate_after: 3 # cheap to retry, never worth climbing fallback: [small-b, small-c]`
review.contract	risk > 0.60	FRONTIER	frontier-b → human-review	2.1s
`review.contract: match: "risk > 0.60" tier: frontier # risk floors the tier; score cannot lower it verify: { mode: judge, rubric: legal.v3 } escalate_after: 1 fallback: [frontier-b, human-review]`
code.migrate	kind=code-reasoning	MID	mid-a → frontier-a	940ms
`code.migrate: match: "kind == code-reasoning" tier: mid verify: { mode: schema+exec, harness: migrate.dry-run } escalate_after: 2 fallback: [mid-a, frontier-a]`
bulk.rename	kind=mechanical & volume>0.55	SMALL	small-c → small-b	210ms
`bulk.rename: match: "kind == mechanical-transform and volume > 0.55" tier: small lane: batch # deprioritised against interactive traffic escalate_after: 3 fallback: [small-c, small-b]`
agent.plan	kind=reasoning & ambiguity>0.5	FRONTIER	frontier-a → human-review	2.4s
`agent.plan: match: "kind == reasoning and ambiguity > 0.50" tier: frontier verify: { mode: judge, rubric: plan.executable } escalate_after: 1 fallback: [frontier-a, human-review]`

ROW hover a row

Rows are evaluated top to bottom; the first match wins, and an unmatched request falls through to default-v3. Route names are yours — they appear verbatim in the decision payload, the audit log and the route-activity table in your portal.

REQUEST LIFECYCLE

Five steps, every time, in about 180 microseconds of our time.

Scroll the rail. Each step carries the actual payload at that point in the pipeline — not a simplified illustration of one.

STEP 01 · INGEST

The request arrives

Authenticated, attributed to a route name, and stamped with a decision id before anything else happens. Nothing is sent to a provider yet.

{
  "route": "extract.invoice-lines",
  "runbook": "default-v3",
  "input_tokens": 1184,
  "decision_id": "dec_7QF3M0RTY7K2"
}

STEP 02 · CLASSIFY

Eight dimensions, scored

Deterministic and local. The classifier does not call a model to decide which model to call — that would be the same mistake one level up.

{
  "kind": "extraction",
  "reasoning": 0.18, "risk": 0.10,
  "ambiguity": 0.24, "output_len": 0.31,
  "score": 0.21, "eval_us": 42
}

STEP 03 · POLICY EVAL

The runbook decides

Guards are checked before tier mapping, so a risk floor or a budget brake can override the score. The clause that decided is named in the trace.

{
  "matched": "extract.invoice-lines",
  "guards": [],
  "tier": "small",
  "escalate_after": 2,
  "policy_eval_us": 174
}

STEP 04 · ROUTE

A live target is chosen

Within a tier, targets are interchangeable. Selection is by health and observed p50 over the last 60 seconds, so a degrading provider sheds traffic before it fails.

{
  "target": "small-b",
  "selected_by": "health+p50",
  "observed_p50_ms": 231,
  "fallback": ["small-c", "mid-a"]
}

STEP 05 · VERIFY

Checked, then returned or climbed

The output is validated against what the caller asked for. A pass returns. A fail retries in place, then escalates. Either way the attempt is metered.

{
  "verify": { "mode": "schema", "passed": true },
  "attempts": 1,
  "escalations": 0,
  "latency_ms": 238
}

RUNBOOK ANATOMY

Every clause, and what it is responsible for.

runbooks/annotated.yml

# 1 — identity. Runbooks are versioned artefacts, not settings.
name: reporting-pipeline
version: 7
extends: default-v3

# 2 — classification. Deterministic, local, 42µs p50.
classify:
  dimensions: [reasoning, risk, ambiguity, output_len]
  evaluator: heuristic-v2

# 3 — guards run BEFORE tier mapping and can override it.
guards:
  - when: "risk > 0.60"
    floor_tier: mid
  - when: "budget.period_used > 1.00"
    pin_tier: small

# 4 — tiers hold interchangeable targets, not preferences.
tier:
  small:    { when: "score < 0.31", targets: [small-b, small-c] }
  mid:      { when: "score < 0.58", targets: [mid-a, mid-b, mid-c] }
  frontier: { when: "default",      targets: [frontier-a, frontier-b] }

# 5 — verification is part of the route.
verify:
  mode: schema+judge
  on_fail: retry
escalate_after: 2

# 6 — fallback is a path you keep warm, not a plan you write later.
fallback:
  provider_down: next_target
  tier_exhausted: escalate
  exhausted: human-review

# 7 — budget is enforced by the router, not by a dashboard alert.
budget:
  period: month
  guard: soft
  on_brake: pin_tier(small)

extends: stringInherit a base policy and override only what differs. Most teams keep one house runbook and three or four thin children.
classify.dimensions: string[]Which scores the evaluator produces. Adding a dimension changes the schema version, which is why the schema is versioned.
guards[]: rule[]Evaluated in order before tiering. floor_tier raises a tier and pin_tier fixes it; a guard can never be silently skipped.
tier.<name>.targets: string[]Model classes considered interchangeable at that tier. Order is not preference — selection is by health and observed p50.
verify.mode: enumschema, judge, schema+judge, or schema+exec for outputs that can be run against a harness.
escalate_after: integerVerified failures permitted at a tier before climbing exactly one rung. A request never skips a rung.
fallback.*: mapBehaviour per failure class: unreachable provider, exhausted tier, exhausted ladder. The last one is where human-review earns its place.
budget.guard: soft | hardA soft guard warns at 80% and brakes at 100%. A hard guard refuses over-budget requests with 429 budget_exhausted rather than downgrading them.

ROUTE CONSOLE

Run the classifier against your own wording.

Task to route

policy trace · runbook/v3

LATENCY & PLACEMENT

The router's own cost, measured.

These are the stages GhostRouter adds to a request. Provider time is excluded — it is not ours to claim. The router runs in your region, or in your VPC, or as a sidecar next to the caller; the numbers below are for the sidecar placement, which is the one most clients end up on.

Stage	P50	P95	Where
classify	42µs	96µs	in-process
policy eval	174µs	410µs	in-process
target select	21µs	58µs	in-process
dispatch overhead	0.9ms	2.4ms	egress
verify (schema)	3.1ms	7.8ms	in-process
verify (judge)	210ms	480ms	tier-local

Judge verification is the only stage that costs real time, which is why it is opt-in per route rather than global. Schema verification is effectively free and should be on for anything with a shape.

INTEGRATION

Three ways in. All of them are a base URL and a header.

There is no agent to install and no proxy to babysit. $GR_API below resolves to api.ghostrouter.io/v1 over TLS 1.3; plaintext connections are refused.

route.mjs · node 20+

import { GhostRouter } from "@ghostrouter/sdk";

const gr = new GhostRouter({
  apiKey: process.env.GHOSTROUTER_KEY,
  runbook: "reporting-pipeline"
});

const decision = await gr.route({
  route: "extract.invoice-lines",
  input: pdfText,
  verify: { mode: "schema", schema: "invoice.v2" }
});

console.log(decision.tier, decision.route, decision.policy_eval_us);
// small small-b 174

route.py · python 3.10+

from ghostrouter import Router

# reads GHOSTROUTER_KEY and GR_API from the environment
router = Router(runbook="reporting-pipeline")

decision = router.route(
    route="extract.invoice-lines",
    input=pdf_text,
    verify={"mode": "schema", "schema": "invoice.v2"},
)

print(decision.tier, decision.route, decision.policy_eval_us)
# small small-b 174

if decision.escalations:
    metrics.incr("router.escalated", tags=[decision.route])

POST /v1/route

# $GR_API resolves to api.ghostrouter.io/v1 (TLS 1.3 only)
curl -sS "$GR_API/route" \
  -H "Authorization: Bearer $GHOSTROUTER_KEY" \
  -H "Content-Type: application/json" \
  -H "GR-Runbook: reporting-pipeline" \
  -d '{
    "route": "extract.invoice-lines",
    "input": "Invoice 4471 — 12 line items, net 30.",
    "verify": { "mode": "schema", "schema": "invoice.v2" }
  }'

200 OK · decision payload

{
  "decision_id": "dec_7QF3M0RTY7K2",
  "runbook": "reporting-pipeline@7",
  "route": "extract.invoice-lines",
  "tier": "small",
  "target": "small-b",
  "fallback": ["small-c", "mid-a"],
  "attempts": 1,
  "escalations": 0,
  "verify": { "mode": "schema", "passed": true },
  "policy_eval_us": 174,
  "latency_ms": 238,
  "output": { "invoice_no": "4471", "lines": 12 }
}

.env · base-URL swap

# Any model client that lets you set a base URL can be pointed
# at the gateway without touching a single call site. The router
# speaks the same wire shape and reads one extra header.

MODEL_BASE_URL="$GR_API/gateway"
MODEL_API_KEY="$GHOSTROUTER_KEY"
GR_RUNBOOK="reporting-pipeline"

# Requests arriving on /gateway are classified from their payload
# rather than a route name. Attribution comes from GR-Route if you
# send it, and from the calling key if you do not.

response headers

GR-Decision: dec_7QF3M0RTY7K2
GR-Runbook: reporting-pipeline@7
GR-Tier: small
GR-Target: small-b
GR-Attempts: 1
GR-Escalations: 0
GR-Policy-Eval-Us: 174

# Every gateway response carries its decision id. Pull the full
# trace later with GET /v1/decisions/{decision_id} — retained 30
# days by default, longer by agreement.

Read the docs, or talk to the people who wrote them.

Read the docs Talk to routing engineering

A router table for intelligence.