experiment-design-kit

# Experiment Design Kit Skill

## When to Use
- Translating raw ideas into testable hypotheses with clear success metrics.
- Ensuring experiment briefs include guardrails, instrumentation, and rollout details.
- Coaching pods on best practices for multi-variant or multi-surface tests.

## Framework
1. **Problem Framing** – define user problem, business impact, and north-star metric.
2. **Hypothesis Structure** – "If we do X for Y persona, we expect Z change" with assumptions.
3. **Measurement Plan** – primary metric, guardrails, min detectable effect, power calc.
4. **Variant Strategy** – control definition, variant catalog, targeting, and exclusion rules.
5. **Operational Plan** – owners, timeline, dependencies, QA/rollback steps.

## Templates
- Experiment brief (context, hypothesis, design, metrics, launch checklist).
- Guardrail register with thresholds + alerting rules.
- Variant matrix for surfaces, messaging, and states.
- **GTM Agents Growth Backlog Board** – capture idea → sizing → prioritization scoring (ICE/RICE) @puerto/README.md#183-212.
- **Weekly Experiment Packet** – includes KPI guardrails, qualitative notes, and next bets for Marketing Director + Sales Director.
- **Rollback Playbook** – pre-built checklist tied to lifecycle-mapping rip-cord procedures.

## Tips
- Pressure-test hypotheses with counter-metrics to avoid local optima.
- Document data constraints early to avoid rework during build.
- Pair with `guardrail-scorecard` to ensure sign-off before launch.
- Apply GTM Agents cadence: Monday backlog groom, Wednesday build review, Friday learnings sync.
- Require KPI guardrails per stage (activation, engagement, monetization) before authorizing build.
- If a test risks Sales velocity, include Sales Director in approval routing per GTM Agents governance.

## GTM Agents Experiment Operating Model
1. **Backlog Intake** – ideas flow from GTM pods; Growth Marketer tags theme, objective, expected impact.
2. **Prioritization** – score with RICE + quali