the independence rule
Score Before You Talk
Two free fixes after you structure the interview: use a panel, and make everyone commit their scores before anyone speaks.
Two things fix the interview after you've structured it, and they cost nothing but discipline: use more than one interviewer, and make them commit their scores before they open their mouths to each other.
Start with the panel. A single interviewer, even a good one, is a single noisy instrument. Run the reliability and it comes back around .44 — barely better than a coin you're calling heads. Put a few interviewers on the same candidate, working the same anchored scale, and the reliability jumps to about .74.1 That is the single cheapest quality upgrade in all of hiring: you already have the people in the building; you're just having them evaluate together instead of in sequence, each unaware of what the last one saw. Add anchored scales on top and the reliability climbs again, into the high .70s.2 Nothing about the candidate changed. You just stopped measuring them with one broken ruler.
But the panel has a failure mode, and most companies walk straight into it: they interview separately and then discuss. The room convenes, someone senior speaks first, and the scores drift toward the loudest confident voice in the debrief. What looks like three independent reads collapses into one read with two echoes. You paid for a panel and got a single opinion wearing a quorum.
The fix is a rule, not a tool: every interviewer commits a score before the debrief begins. Written down, submitted, locked. Then you talk. The independent scores are the measurement — three genuine reads you can average, whose disagreement is itself information. The conversation afterward is for surfacing what each person saw, not for revising toward consensus. A panel that scores after it confers isn't a panel. It's the senior person's gut, laundered through a meeting.
Call it the independence rule: the value of a second opinion is exactly the amount it was formed without hearing the first. Break independence — let the scores form in conversation — and you don't get a more reliable decision, you get a more confident one, which is worse, because now the whole room will defend it.
So the whole upgrade, in one line: more than one interviewer, anchored scales, scores in before anyone speaks. It fits on an index card. It roughly doubles your reliability. And when someone challenges the decision, you can show three independent scores instead of one story about how the room felt.
Panel or it didn't happen — and score before you talk, or the panel didn't either.