Build, pivot, or kill: turning findings into a decision

How much research do you need to make a decision?

The amount of research a decision needs is set by its reversibility, not its importance. A decision you can cheaply undo — a pricing-page headline, a feature behind a flag — deserves a low evidence bar: decide on moderate signal, instrument the result, and correct course if you were wrong. A decision that is expensive to unwind — killing a roadmap item, pivoting the product's core workflow, committing two quarters of engineering — deserves a high bar: strong signal, from more than one method, that survived a genuine attempt to disprove it.

Most teams run this backwards. They over-research reversible calls because research feels responsible, and under-research irreversible ones because momentum feels like evidence. Priya's team at Fieldnote had spent three weeks debating smart dispatch in meetings — an effectively irreversible commitment of a full quarter — on the strength of two customer anecdotes and a competitor's press release. Meanwhile a trivially reversible onboarding copy change had been "pending validation" for a month.

Setting the bar first inverts the psychology: research stops being a ritual you perform until confidence arrives, and becomes a question with a defined finish line.

What is the reversibility test?

Reversibility asks: if this decision turns out wrong, what does it cost to walk it back? The vocabulary most teams borrow is Amazon's doors: a two-way door is a reversible decision — walk through, dislike the outcome, walk back cheaply. A one-way door is effectively permanent, because unwinding it costs more than the decision was worth.

Score reversibility on the total cost of reversal, not the engineering cost. Rolling back code is usually easy; rolling back a feature that customers adopted, support documented, and sales demoed is not. This is the framework's most common failure mode, and it fails in one direction: teams systematically label one-way doors as two-way because rollback is technically possible. A useful check is to ask what reversal costs in three currencies — engineering time, customer trust, and internal credibility — and take the most expensive answer.

For Priya, the door question split her decision in two. Shipping smart dispatch to all customers as the default assignment mode: one-way, because dispatchers would rebuild their daily routine around it. Piloting it as an opt-in suggestion for ten design partners: two-way. Same feature, two different evidence bars — which is itself a finding. If your decision demands more evidence than you can afford to gather, look for a version of the decision that fits through a two-way door.

How do you judge signal strength?

Signal strength is a property of the evidence, not of your enthusiasm for it. Three tests separate strong from weak:

Consistency. Does the same pattern appear across many participants, unprompted, in similar form? A theme that 24 of 36 interviewees raise on their own is consistent. A theme assembled from four loosely related complaints is not.

Directness. Was the signal observed or declared? Watching dispatchers manually reassign jobs for forty minutes each morning is direct evidence of the problem. Dispatchers saying they would trust an algorithm to do it is indirect — stated intent, the weakest common evidence type, because people are honest but bad at predicting their own behavior.

Adversarial survival. Did the finding survive an attempt to kill it? Before labeling a signal strong, deliberately hunt for disconfirming evidence: participants who contradict the pattern, an alternate explanation, a segment where it vanishes. A finding nobody tried to break is a hypothesis wearing a conclusion's clothes.

In Priya's study — 36 AI-moderated interviews with dispatchers, run in parallel overnight, plus a clickable prototype test — the strongest signal was behavioral and consistent: 29 of 36 dispatchers walked through a morning routine dominated by manual reassignment, and in the prototype, most accepted algorithmic suggestions for routine jobs. The weak signal hid beside it: dispatchers overwhelmingly said they wanted automation, but 22 of 36 balked in the prototype when the system assigned their messy, high-stakes jobs without asking. Same study, two signals, opposite strengths.

Which decision does your evidence support?

Cross signal strength with reversibility and the call falls out. The table below is the framework's core — read your row, then your column:

Two things about using it honestly. First, "keep researching" is a cell, not a comfort zone — it comes with an obligation to name the one question that blocks the decision and a date by which it will be answered. Fielding that targeted follow-up used to cost weeks of scheduling; the mechanism that changed this is parallelism — an AI-moderated study can be drafted from a prompt or a prototype link in minutes and interview 30-plus participants inside 24 hours, which makes "keep researching" a days-long cell instead of a quarter-long one. The bar for staying undecided got higher because the cost of resolving it got lower.

Second, the table decides when you know enough — it does not tell you which of build, pivot, or kill the evidence points to. That is the next question.

When should you pivot instead of kill?

Pivot when the need is real and the solution is wrong; kill when the need itself fails to show up. The diagnostic is to separate every finding into those two piles: evidence about the problem (do people have it, how often, how painfully, what do they do about it today) and evidence about your solution (do they understand it, trust it, choose it, use it).

A kill verdict is a problem-pile verdict: the pain is rare, mild, or already solved cheaply, and no redesign of your solution changes that. A pivot verdict is a split verdict: strong problem evidence, failing solution evidence.

Priya's study was a textbook split. Problem pile: strong — 29 of 36 dispatchers demonstrably lose their mornings to manual reassignment. Solution pile: failing in a specific, recoverable way — dispatchers rejected full auto-assignment because they didn't trust it with jobs where a bad match has consequences, but happily accepted suggestions they could confirm with one click. The verdict wasn't kill, and it wasn't build-as-conceived. It was a pivot: from "smart dispatch decides" to "smart dispatch drafts, dispatcher approves." Her team shipped the suggestion mode to design partners — the two-way-door version — with the one-way default-mode decision deferred until usage data existed.

Most "failed" studies are actually split verdicts nobody read closely: the problem was confirmed and the solution was rejected, and the team heard only the rejection. Before you kill, check which pile the bad news is in. Killing a real problem because your first solution was wrong is the most expensive misread in product research.

What if the evidence stays mixed?

If the evidence stays mixed after a targeted follow-up, the finding is the mix — usually a segmentation you haven't named. Signal that refuses to converge often means two populations are answering different questions: in Fieldnote's case, dispatchers at small shops (who knew every technician personally) resisted automation far more than dispatchers at 50-technician operations, who were drowning. That is not noise. That is an answer about who the feature is for, and it converts a mixed verdict into a clear one with a narrower scope.

When the mix persists even within segments, fall back to the table's honest default: mixed evidence is moderate-at-best signal, so make the reversible version of the decision and let instrumented reality arbitrate. What you may not do is launder mixed evidence into a confident readout. If your recommendation memo needs the word "clearly" to survive scrutiny, the evidence wasn't clear — and a decision-maker who catches that once will discount every readout you send afterward. (How to write the memo itself — the ask, the ranked findings, the appendix — is its own craft, covered in the readout guide in this library.)

How do you put this into practice this week?

Take the decision your team has debated longest and run it through the framework in one meeting: score its reversibility in all three currencies, sort your existing evidence into problem pile and solution pile, and grade each pile's signal strength against the three tests. You will land in one of two places. Either the table says decide — in which case the meeting ends with a call, not another calendar hold — or it says keep researching, in which case you leave with the one blocking question, the study design that answers it, and a date. Both outcomes beat the third thing, which is what most teams actually do: gathering evidence indefinitely for a decision whose bar nobody ever set.

Frequently asked questions

How much user research is enough to make a product decision?

Enough that the evidence clears the bar the decision's reversibility sets. A reversible decision — a copy change, a feature flag — can move on one fast study with a moderate pattern. A hard-to-reverse decision — killing a product line, a pivot — needs strong signal that repeats across participants and holds up in more than one method.

What is the difference between a pivot and a kill?

A pivot keeps the underlying need and abandons the current solution: research shows people genuinely have the problem, but not in the shape you built for. A kill concludes the need itself is too weak, too rare, or too cheaply solved elsewhere to justify any version. Diagnose which one your evidence describes before choosing.

How many user interviews do I need to justify killing a feature?

There is no magic number, but a kill is a hard-to-reverse call, so treat 25 to 40 interviews as a reasonable range — enough that an absent need is conspicuous, not just unsampled. More important than count: the finding should be behavioral (people do not use or choose it), not just attitudinal (people say it is not for them).

What are one-way door and two-way door decisions?

The terms come from Amazon's decision-making vocabulary. A two-way door decision is reversible — walk through, dislike what you find, walk back cheaply. A one-way door is effectively permanent: unwinding it costs so much that you should treat it as final. The distinction sets how much evidence a decision deserves before you make it.

How do you know when to stop doing user research?

Stop when new sessions repeat what you already know — researchers call it saturation — and the evidence you have clears the bar for your decision's reversibility. A practical test: write the decision memo now. If you can state the call and the evidence for it without flinching, further research is delay dressed as diligence.

What evidence is strong enough for a build decision?

Strong evidence for a build is convergent and behavioral: the same need appears unprompted across a meaningful share of participants, it survives a method beyond interviews (a prototype test, a fake-door signup, current workaround behavior), and you actively looked for disconfirming signal and failed to find it. One enthusiastic quote is an anecdote, not a green light.