Method · The Crit Path

The Crit Path.

Four stages. One rule: nothing ships until it beats the benchmark on your data. The name plays on the critical path — the shortest sequence that determines the result — and on the design crit: the honest review before anything gets built.

Book The Crit Walk the four stages

The Crit Path is how we ship: Scope the workflow, Prove the model on your data against a frozen eval set, Deploy inside your perimeter, Assure with monitoring and re-training. The Prove step is a gate, not a formality — if the model doesn't clear the pass mark you set, the project stops and you keep the artifacts. Every build, in every vertical, runs on this rail.

Stage 01 · Scope

Scope. Freeze the target before anyone trains anything.

1.1What happens.We instrument the workflow: API logs and invoices, token volumes per task, prompt chains, latency and privacy constraints. We cluster the traffic, rank candidate workflows by spend and feasibility, and set the pass mark — the accuracy number the build must clear. We agree it in writing before any training starts.

1.2What you hold at the end.A workload map ranked by spend. A frozen eval set — 100–300 graded examples built from your real traffic. A cost model with the break-even point. And the pass mark itself, written into the statement of work.

1.3What can go wrong — and what we do.The traffic turns out too varied for one small model, or the volume is too low to justify a build. That's a finding, not a failure: the readout says keep the frontier API — or don't automate at all — and the engagement ends there, with the artifacts in your hands.

Stage 02 · Prove

Prove. Beat the benchmark or stop.

2.1What happens.We fine-tune an open-weights 3–7B model by distilling from your pipeline's outputs, then run it against the frozen eval set. The set doesn't move: no cherry-picked demos, no re-cut test data. The model must match or beat the pass mark from Scope — on your data, not on a public leaderboard.

2.2What you hold at the end.An eval report with scores per candidate model. The training data, validated by humans. A go or no-go decision in writing — measured, not argued.

2.3What can go wrong — and what we do.The model doesn't clear the bar after the agreed evaluation cycles. Then the project stops at the gate: you pay for the phases delivered and keep everything produced — the eval set, the training data, the numbers. No quiet scope drift, no "let's ship it anyway."

Stage 03 · Deploy

Deploy. Inside your perimeter, behind your interface.

3.1What happens.The model ships into your VPC or on-prem behind the same API interface your code already calls. Hitbox routes every request: the small model answers what it has proven it can answer; low-confidence cases escalate to your existing frontier API. We stage the rollout and keep a rollback path live from day one.

3.2What you hold at the end.The deployment as infrastructure-as-code. The routing config with its escalation policy. Runbooks, monitoring dashboards, and a handover your team can operate without us.

3.3What can go wrong — and what we do.Production traffic drifts from the eval sample — new intents, new formats. Hitbox surfaces it as a rising escalation rate, the frontier model absorbs the shock, and we extend the eval set and re-tune. Nothing degrades silently, because everything is measured.

Stage 04 · Assure

Assure. Keep the system honest after launch.

4.1What happens.We re-run the frozen eval monthly, watch drift indicators on inputs, and track cost per call and escalation rate. Re-training triggers when the numbers say it's due — not on a calendar, and not on a hunch.

4.2What you hold at the end.A monthly one-page report: accuracy against the pass mark, cost per call, escalation rate, drift indicators, and what changed since last month. Numbers, not a slide deck.

4.3What can go wrong — and what we do.Accuracy slips below the pass mark. A re-training cycle triggers. If the drop is structural — the workflow itself changed shape — the report says so and we scope the options honestly, including retiring the model. And Assure is optional: every build ships with runbooks, and the handover is complete without us.

The stack · Hitbox

Hitbox. The part that decides who answers.

Hitbox is our eval-and-routing harness, built and operated by Crit Studio. It does two jobs. Before launch, it gates deployment behind measured accuracy: no model ships until it clears the pass mark on the frozen eval set. In production, it decides per request whether the small model answers or the task escalates to a frontier model.

Escalation is a design target, not an embarrassment. We set the expected escalation share at Scope, and the monthly report tracks it against that target. A router that never escalates isn't confident — it's unmeasured.

The Prove gate

The gate is the point. A stopped project with honest measurements beats a deployed one that quietly underperforms.

The pass mark freezes at Scope, before anyone trains anything — so the bar can't quietly move to meet the model. If the model doesn't clear it after the agreed evaluation cycles, the project stops. You pay for the phases delivered and keep everything produced: the eval set, the training data, the numbers.

The eval set outlasts the project. It's the yardstick for any build — by us or by anyone else — and walking away with honest numbers is a legitimate outcome, not a consolation prize. We price the audit so its readout is worth having on its own.

Book The Crit

Questions, answered straight

What happens if the model fails the Prove gate?

The project stops. You pay for the phases delivered and keep everything produced — the frozen eval set, the human-validated training data, and the numbers. A stopped project with honest measurements beats a deployed one that quietly underperforms.

Who sets the pass mark?

You do, with us, at Scope — anchored to your current system's measured accuracy on the frozen eval set. We write it into the statement of work before any training starts, and it doesn't move afterwards. A bar that moves to meet the model isn't a bar.

How is escalation to a frontier model decided?

Hitbox, our eval-and-routing harness, scores each request's confidence; below the threshold, the task escalates to your existing frontier API. We set the expected escalation share as a design target at Scope and track it in the monthly Assure report. Air-gapped deployments can disable escalation entirely.

The compounding stack

One small model. Then a workforce.

Every system we ship shares one core: your model, your router, your eval harness. The first deployment is the expensive one — every workflow after it reuses the same infrastructure, the same monitoring, the same lessons. Owned models compound; rented APIs just meter.

+1 workflow ≈ freeeach new workflow reuses the model, the router, and the evals — built once, reused everywhere

Flat costnew workflows ride the same GPU — the bill doesn't grow with your usage

<100 msanswers at LAN speed — no API round-trips across the internet

One pass marka single eval harness governs every agent built on the core

Scale is a property of ownership. Rented APIs meter every workflow you add — an owned core amortizes them.

Security · by architecture

Custody holds at every stage.

Data stays in your cloud
Access expires with the project
Every deployment logged
Escalation only if you allow it

Architected to deploy inside your

GDPR SOC 2 ISO 27001

Small models · critical hits

Put your workflow on the rail.

One scoping call tells you whether your workflow belongs on the Crit Path — and the Prove gate makes sure you never find out the expensive way.

Book The Crit Or write to hello@crit-studio.com