Vetting
How we vet engineers
The complete process, published. Read it and judge the standard for yourself.
Every engineer passes five stages: a written screen, work sample review, a structured interview against a role-specific rubric, reference verification, and a calibration review by a second reviewer. The scorecard behind each profile is shown to you, including gaps relevant to your brief.
The five stages
Stage 1 — Written screen
A short written response to a real problem in their claimed specialism. Not a puzzle: an ambiguous scenario where the useful signal is which questions they ask before answering. Roughly half of applicants do not pass this stage, most commonly because they answer a question that was not asked.
Stage 2 — Work sample review
We review real shipped code they can share, or a scoped exercise where they cannot. We assess the same things a good reviewer on your team would: boundaries, naming, error handling, test discipline and whether the next engineer could safely change it.
Stage 3 — Structured technical interview
A ninety-minute interview against a written rubric specific to the role, scored independently before discussion. AI agent engineers are asked about tool design and evaluation; Digital Product Passport engineers about identifier granularity and supplier data. Generic interviews are the reason generic platforms cannot assess specialists.
Stage 4 — Reference verification
We contact people who actually worked with them, and ask what changed after they left. Reference calls that only confirm dates are theatre; the useful question is what broke or improved in their absence.
Stage 5 — Calibration review
A second reviewer checks the scorecard against the rubric without seeing the first reviewer conclusion. Where the two disagree materially, the candidate is re-interviewed. This is the stage most platforms skip, and it is the one that keeps standards from drifting.
What each rubric assesses
The rubric is role-specific, and this is the whole point. Assessing an AI agent engineer with a generalist software rubric produces a confident score about the wrong thing.
| Role family | Assessed specifically for |
|---|---|
| AI agent engineering | Tool schema design, idempotency, evaluation harness discipline, guardrail reasoning, cost and latency control |
| Retrieval engineering | Chunking judgement, hybrid search reasoning, reranking under latency budget, groundedness measurement |
| LLMOps | Gateway and failover design, cache safety, cost attribution, incident handling under provider degradation |
| Forward Deployed Engineering | Requirements archaeology, generalisation instinct, written executive communication, shipped integrations |
| Digital Product Passport | Identifier granularity reasoning, GS1 Digital Link, interoperability instinct, supplier data strategy |
| Traceability | EPCIS transformation event modelling, credential design, selective disclosure, interoperability testing |
| Core engineering | Data modelling, idempotency, failure recovery, performance method, testing discipline |
The stage most platforms skip
Calibration review is unglamorous and it is the reason standards hold. A single reviewer drifts — toward candidates who resemble them, toward the last strong candidate they saw, toward whatever they happened to ask about that week. A second reviewer scoring against the same written rubric, blind to the first conclusion, is the cheapest available correction for that.
Frequently asked questions
What percentage of applicants do you accept?
We deliberately do not lead with a percentage. Acceptance rate measures selectivity against an applicant pool of unknown composition, not engineering ability — a platform with weak applicants can post an impressive number. What matters is what is assessed at each stage, which is why the whole process is published here.
How is vetting different for AI-native roles?
The rubric changes entirely. An AI agent engineer is assessed on tool schema design, evaluation harness discipline, guardrail reasoning and cost control. A generalist software rubric does not test any of those, which is why generalist platforms cannot identify these engineers reliably.
Do you use automated coding tests?
Not as a primary filter. Automated tests measure what is easy to measure and miss judgement, design instinct and communication — which are what actually distinguish senior engineers. We use work samples and structured interviews instead, which is slower and does not scale as cleanly. That is the trade-off we chose.
What do you reject candidates for most often?
At stage one, answering a question that was not asked instead of clarifying an ambiguous requirement. At stage two, code that works but that another engineer could not safely modify. At stage three, claimed production experience that does not survive specific questions about failure modes.
Can I see the vetting record for a candidate?
Yes. Every shortlisted profile includes the work sample findings, the interview scorecard against its rubric, and reference notes. Where a candidate has a gap relevant to your brief, it is shown rather than quietly omitted.