A/B Test Setup
Design a decision-ready controlled experiment with an explicit hypothesis, assignment unit, exposure rules, metrics, guardrails, sample assumptions, QA checks, and stopping criteria.
Sample result
How it works
What A/B Test Setup does
Use it when
Planning an A/B test, feature experiment, growth experiment, holdout, rollout validation, or experiment brief.
Inputs and results
What goes in, what comes back
- You give
A short brief describing what you need
Add a specialist method to the work.
- You get
Experiment brief, launch checklist and written document
The method guides the work with your files and context.
- You check
The sources and open questions
Review the output against your brief.
Questions and answers
Questions about A/B Test Setup
Updated Oct 18, 2026.
What does A/B Test Setup do?
Design a decision-ready controlled experiment with an explicit hypothesis, assignment unit, exposure rules, metrics, guardrails, sample assumptions, QA checks, and stopping criteria. Use when planning an A/B test, feature experiment, growth experiment, holdout, rollout validation, or experiment brief.
Who is A/B Test Setup for?
It is designed for growth teams, product teams, marketers and analysts.
What do I need to use it?
Nothing extra is needed. Sign in to Relay to run it with your own files and context.
What does A/B Test Setup produce?
Experiment brief, launch checklist and written document. Results come back with their sources and any open questions for you to check before anything is used.
How much does A/B Test Setup cost?
It is free to use. Paid model usage, if any, is shown before a run.
Can an AI agent use it?
Yes. Agents can read its public definition through the Relay API and run it with a person’s authorization. See the For Agents page for how to connect.
Details
Trust, access and technical details
Trust and access
- Visibility
- Public: anyone can view
- Status
- Published and maintained
- Pricing
- Free
- cmvdyojrv000o…
- Oct 18, 2026
- 0 signals
- Commit pinned
Security review is a static package assessment, not a guarantee of safety or a behavioral qualification.
Use it from code or an agent
Read the public definition, then check cost and inputs with a preflight before a run.
Open definitionPOST /api/v1/runs/preflight
How agents connect to Relay →Tags and source
- Repo
- rose-digital/relay
- License
- MIT
Connections
Where A/B Test Setup fits
Similar skills5
ab-test-setup
When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.FreeCommunitySkill · MarketingConversion Funnel Diagnosis
Diagnose where and why a conversion funnel leaks, using the user's own web analytics exports, CRM stage data, or product signup and activation events. Defines stages and exact conversion formulas, chooses cohort or period measurement, cuts by segment, checks sample size before calling a difference real, ranks stages by recoverable loss rather than worst rate, gathers qualitative evidence, and ends with a scored hypothesis backlog. Use when someone asks why conversion dropped, where the funnel leaks, why signups do not activate, why leads stall between stages, for a drop-off analysis, or which funnel step to fix first. Do not use for writing landing page copy (use the landing-page-copy prompt), for designing or sizing a specific A/B test (use ab-test-setup), for building a metric hierarchy (use kpi-tree), or for retention curves after purchase (use analyze-cohort-retention).FreeSkill · MarketingMarketing Incrementality Measurement
Measure what marketing actually causes rather than what it gets credit for. Explains attribution models (last-touch, first-touch, position-based, data-driven) and their biases; designs and sizes incrementality tests (randomized holdouts, ghost ads, conversion lift studies, geo-lift and matched markets, on/off switchbacks); outlines a lightweight marketing mix model; reconciles self-reported attribution; and writes a per-channel measurement plan with pre-registered decision rules. Use when someone asks whether a channel really works, why platform ROAS disagrees with revenue, how to run a holdout or geo test, or which attribution model to trust. Do not use for page or product A/B tests (use ab-test-setup), funnel drop-off diagnosis (use diagnose-conversion-funnel), or setting budget levels (use plan-marketing-budget).FreeSkill · MarketingReferral Program Design
Design a customer referral or advocacy program that pays for itself. Tests fit first (is word of mouth already happening, per NPS and referral-source evidence), identifies who refers and why beyond cash, chooses the reward structure (single or double-sided; cash, credit, or product; tiered), models economics against alternative CAC with payback and fraud controls, flags tax and legal points for counsel, sets the moments to ask and the tracking, and launches as a test with a holdout. Use when someone asks to start or fix a referral, refer-a-friend, or customer advocacy program, what referral reward to offer, or whether referrals are worth it. Do not use for partner or reseller commissions, sales compensation (use design-sales-compensation), measuring paid channels (use measure-marketing-incrementality), or offer and pricing design (use develop-offers).FreeSkill · Marketingrevops
When the user wants help with revenue operations, lead lifecycle management, or marketing-to-sales handoff processes. Also use when the user mentions 'RevOps,' 'revenue operations,' 'lead scoring,' 'lead routing,' 'MQL,' 'SQL,' 'pipeline stages,' 'deal desk,' 'CRM automation,' 'marketing-to-sales handoff,' 'data hygiene,' 'leads aren't getting to sales,' 'pipeline management,' 'lead qualification,' or 'when should marketing hand off to sales.' Use this for anything involving the systems and processes that connect marketing to revenue. For cold outreach emails, see cold-email. For email drip campaigns, see email-sequence. For pricing decisions, see pricing-strategy.FreeCommunityHave a task this would help with?
Describe it. Relay plans the work, picks the right playbook and shows the cost before anything runs.