Latency + throughput
Measure response time and completed work at the team’s expected load.
Private compute / Local models & harnesses
Evaluate local models
on your workload
Scope a local model setup around the team’s real workload, then prepare installation, monitoring, updates and an agreed fallback policy.
Walk through the example ↓An illustrative scenario
A research team wants to assess whether a local model is useful for organising its internal documents
01 / Supplied by your team
A workload and its boundaries
- Representative task
- A small, authorised set of internal-document tasks and review criteria.
- Operating constraints
- Available hardware, concurrency, storage, power and support requirements.
- Data policy
- The team specifies which data may be processed and whether any external fallback is permitted.
02 / Automated preparation
A candidate setup and an evaluation plan
- Sizing + installation
- Prepare a candidate configuration and a repeatable installation plan.
- Evaluation harness
- Compare candidate outputs with a team-approved quality checklist.
- Operations plan
- Specify health checks, update/rollback steps and the permitted fallback behaviour.
Ready for your team to review.
03 / A human decision
A fallback needs an explicit policy
Sample exception
The illustrative local service is unavailable. The agreed sample policy forbids an external fallback, so the task waits for a person.
The sample owner accepts a local-only pilot, leaves purchasing undecided and requires an approved quality review before operational use.
This control demonstrates a sample decision. Nothing is sent or saved.
04 / A clear handoff
An operable pilot, with acceptance criteria
The sample review decision is still pending.
Sample decision applied
Local-only pilot; pause on failure; no purchase or production acceptance.
- Configuration record
- The candidate setup, model version and repeatable installation steps.
- Evaluation report
- Task-level quality review alongside latency and throughput measurements.
- Runbook
- Monitoring, permitted fallback, updates and rollback responsibilities.
Apply the sample review decision to enable this download. Illustrative handoff contents. A real project needs an agreed scope and review.
Explore each stage. Decisions here only change this demonstration.
Decisions about your setup
- Purchasing and budget approval
- Data and fallback policy
- Accepted model quality
What a first pilot would measure
Measure quality and cost
One task, an agreed test set and a bounded local evaluation before purchasing or rollout.
Accepted quality
Review output against the agreed task rubric; speed alone does not pass.
Total cost
Include hardware, energy, maintenance and any permitted external usage.
These are proposed evaluation criteria. No client results or performance gains are claimed.
Discuss a local setup
Tell us how the task works today. We’ll agree the inputs, review steps, scope and price with you.