Penny-Wise, Pound-Wise
A practical argument for making smart tradeoffs in usability research without trading away the integrity of the evidence.
Every study makes tradeoffs. The important question is not whether to compromise an ideal process, but which compromises preserve credible participants, useful evidence, and team involvement.
Protect the consequential choices
Participant selection, recruiting time, incentives, international context, team politics, and stakeholder involvement directly affect whether a study can answer its question.
Recruiting is not administrative overhead. The quality of the evidence depends on whether participants reflect the people and conditions the product is meant to serve.
Spend less where the risk is lower
Facilities, exhaustive scripts, automatic videotaping, and large participant counts are often negotiable. Remote testing or short sessions can increase speed and audience diversity when matched to the question.
The person responsible for acting on the findings is often more valuable in the room than an elaborate observation setup.
A study should earn its cost
I begin skeptical of expenses that do not improve the decision. Three well-chosen participants can expose a meaningful cluster of issues; more participants are justified when the study needs segmentation, confidence, or coverage—not simply because a process says so.
- Which constraint threatens the integrity of this study?
- Who needs to witness the evidence to act on it?
- What can be simplified without weakening the decision?