Terminal
Terminal/Skill
Guidance for designing A/B and multivariate content experiments, choosing metrics, sizing samples, and avoiding common statistical mistakes in a CMS.
By Sanity
Works with
npx skills add sanity-io/agent-toolkit --skill content-experimentation-best-practicesPrinciples and patterns for running effective content experiments to improve conversion rates, engagement, and user experience.
Reference these guidelines when:
Comparing two variants (A vs B) to determine which performs better.
Testing multiple variables simultaneously to find optimal combinations.
The confidence level that results aren't due to random chance.
Making decisions based on data rather than opinions (HiPPO avoidance).
Start with the reference that matches the current problem, such as design, statistics, CMS integration, or pitfalls. See references/ for detailed guidance:
references/experiment-design.md — Hypothesis framework, metrics, sample size, and what to testreferences/statistical-foundations.md — p-values, confidence intervals, power analysis, Bayesian methodsreferences/cms-integration.md — CMS-managed variants, field-level variants, external platformsreferences/common-pitfalls.md — 17 common mistakes across statistics, design, execution, and interpretation