Work Sample Tests for Engineers: Design and Scoring

Why work samples beat every other instrument

Across decades of selection research, one finding is unusually stable: the best predictor of performance in a job is a sample of that job. For engineering this means reviewing code the candidate wrote, or having them write something that resembles what they would actually write for you. Everything else — puzzles, trivia, structured conversation — is a proxy, and every proxy loses information.

Despite this, work samples are used less than they should be, for two understandable reasons. They take effort to design well, and they impose cost on candidates, which biases your pipeline toward people with spare time. Both objections are real, and both are manageable if you keep the exercise short and pay for it where you can.

The version that fails is the one most companies default to: a large open-ended project with a vague brief and no rubric. It costs the candidate a weekend, produces output nobody can compare consistently, and often measures how much free time somebody had rather than how well they engineer.

Designing the exercise

1. 01

   Start from a real decision, not a specification

   The most informative exercises require the candidate to choose between defensible options rather than implement a stated design. Give them a genuine ambiguity — an unclear requirement, two viable data models — and the reasoning they show is worth more than the code they produce.

2. 02

   Cap it at three hours and mean it

   State the cap, and design so that a strong candidate finishes comfortably inside it. Then judge what they chose to do with the time, including what they deliberately left out. Candidates who scope well are demonstrating the skill you most want to see.

3. 03

   Use a realistic but sanitised context

   Base it on a problem your team actually solved, stripped of proprietary detail. Realistic context makes the exercise engaging and lets you compare the candidate's approach against what your team did, which is a far better reference point than an abstract ideal.

4. 04

   Ask for a short written rationale

   Two or three paragraphs explaining trade-offs and what they would do with more time. This is where senior candidates separate themselves, and it is the cheapest signal in the whole process to collect.

5. 05

   Never ask for work you could ship

   Beyond being exploitative, it corrupts the assessment, because you start evaluating usefulness to you rather than capability. If the output would be genuinely valuable to your product, the exercise is scoped wrongly.

What to score, in order of weight

• Judgement: did they make defensible choices, and can they explain why they made them?
• Scoping: did they identify what mattered and deliberately leave out what did not?
• Boundaries: are the module and function boundaries sensible for future change?
• Failure handling: is error behaviour deliberate rather than accidental?
• Testing: do the tests describe behaviour rather than assert implementation detail?
• Communication: is the written rationale clear enough for a colleague to act on?
• Correctness: does it work, which matters but is the least discriminating criterion

Scoring without arguments

The rubric must exist before the first candidate submits anything. For each criterion, write what a weak, adequate and strong response looks like in concrete terms. This feels bureaucratic and it is the difference between a repeatable assessment and a series of opinions that happen to be delivered confidently.

Each reviewer scores independently and records their scores before any discussion. This ordering matters more than people expect: discussing first produces convergence on whoever speaks with most authority, not on the evidence. When independent scores diverge materially, that divergence is information — sometimes about the candidate, often about an ambiguous rubric criterion that should be rewritten.

Keep every scored submission. Over a year they become a calibration set, letting you check whether your bar has drifted and letting new reviewers calibrate against past decisions rather than against their own instincts. This is also the cheapest way to onboard a new interviewer: hand them five past submissions with the scores removed and compare their judgement against the recorded ones before they assess anybody real.

Part of the Vetting & Technical Assessment cluster · Read the pillar page