Keep changes narrow
Narrow changes make experiments easier to read.
This is the most important rule for experiments. If too many things change at once, the result may still be interesting, but it will be harder to explain.
One change at a time
Pick one main change.
Examples of narrow changes:
| Narrow change | Too broad |
|---|---|
| Add one proof section | Rewrite the full site |
| Change one prompt group | Change all prompts |
| Test one page cluster | Change every product page |
| Schedule one review | Add many overlapping routines |
Narrow change
One clear change is easier to read than many edits that all happen at once.
Narrow change
Narrow does not mean tiny.
It means the change is clear enough to describe in one or two sentences. If you cannot describe it simply, split the experiment.
Change record
Write the exact change.
This protects the decision later. If someone asks why the result moved, the team can inspect the note instead of guessing.