A/B test plan template

Plan an experiment before it runs: hypothesis, variants, primary and guardrail metrics, sample size and the decision rules.

Use this templateFree, opens in your browser. No sign-up.

A/B test plan template
Hypothesis
Variants (50 / 50 split, new trials on web)
Decision rules
If we show the annual price first on the upgrade page, trial → paid conversion rises from 8.0% to 8.8%, because the monthly price looks cheaper next to it.
A · Control: monthly price shown first
B · Variant: annual price first with a 'save 20%' badge
Metrics
Metric
Type
Baseline
Minimum detectable effect
Trial → paid conversion
Primary
8.0%
+10% relative (8.8%)
Revenue per visitor
Secondary
$4.10
Report only
Refunds within 14 days
Guardrail
2.1%
Must not rise > 0.3 pt
Upgrade page load p95
Guardrail
1.2 s
Must not rise > 200 ms
Sample: 2 × 23,000 trials (α 0.05, power 80%)
Duration: 3 full weeks to cover weekly cycles
No peeking: decide only at the planned end
Owner: growth PM · analyst: data team
Test ends after 3 weeks
Primary metric significant?
Guardrails OK?
Keep A, write up the learning
Ship B to 100%
Fix the guardrail issue, re-test
A/B test plan
yes
no
yes
no
The A/B test plan template as it opens in Kelvo: 8 boxes, 2 decisions among them, 5 sticky notes, 1 table and 5 links, in 3 sections.

What's on the board

Hypothesis
If we show the annual price first on the upgrade page, trial → paid conversion rises…
Decision rules
Test ends after 3 weeks · Primary metric significant? · Guardrails OK? · Keep A, write up the learning · Fix the guardrail issue, re-test · Ship B to 100%
Variants (50 / 50 split, new trials on web)
A · Control: monthly price shown first · B · Variant: annual price first with a 'save 20%' badge

Table columns: Metric, Type, Baseline, Minimum detectable effect.

Also on the board

  • Metrics
  • Sample: 2 × 23,000 trials (α 0.05, power 80%)
  • Duration: 3 full weeks to cover weekly cycles
  • No peeking: decide only at the planned end
  • Owner: growth PM · analyst: data team

How to use this template

  1. Click Use this template. It opens in the Kelvo app as a new board, saved in your browser.
  2. Double-click any box to change its text. Replace the examples with your own content.
  3. Add notes fast. Press S for a sticky note in the middle of your view. A sticky is a box like any other: link it, recolour it, resize it.
  4. Paste a table. Paste comma- or tab-separated text and it becomes a table. Tab moves to the next cell and adds rows as you go.
  5. Let Smart Assist draft the next step. Turn on Suggest, finish a label and pause: the next steps appear as dashed ghosts. Tab keeps them, Esc dismisses them.

Good for

  • a/b testing
  • experiment
  • product manager
  • growth
  • metrics
  • Planning a quarter or a release
  • Prioritising a long backlog with a shared method
  • Connecting outcomes to the work that should move them

Related templates

  • North Star metric tree template previewNorth Star metric tree

    Break your North Star metric into input metrics a team can move, and the levers behind each one.

  • HEART metrics framework template previewHEART metrics framework

    Google's HEART framework for UX metrics: goals, signals and metrics for happiness, engagement, adoption, retention and task success.

  • Now / Next / Later roadmap template previewNow / Next / Later roadmap

    An outcome-based product roadmap without fake dates: what we're doing now, next, later, and what we're not doing.

  • PRD one-pager template previewPRD one-pager

    A product requirements doc on one board: problem, goals, non-goals, users, requirements, metrics, risks and timeline.

  • Marketing funnel (AARRR) template previewMarketing funnel (AARRR)

    The pirate-metrics funnel: Acquisition, Activation, Retention, Referral, Revenue, each with the metric to watch.

  • Ansoff growth matrix template previewAnsoff growth matrix

    Choose a growth strategy by pairing existing or new products with existing or new markets.

All product management templates · All templates

Use the A/B test plan template