A/B Test Setup Task
Design a decision-ready controlled experiment with an explicit hypothesis, assignment unit, exposure rules, metrics, guardrails, sample assumptions, QA checks, and stopping criteria.
Sample result
How it works
What A/B Test Setup Task does
Use it when
Planning an A/B test, feature experiment, growth experiment, holdout, rollout validation, or experiment brief.
Inputs and results
What goes in, what comes back
- You give
A short brief describing what you need
Pick the job and describe your specifics.
- You get
Experiment brief, launch checklist and written document
It follows the task’s plan with your files and context.
- You check
The sources and open questions
Review the result, the sources and the cost.
Questions and answers
Questions about A/B Test Setup Task
Updated Oct 18, 2026.
What does A/B Test Setup Task do?
Design a decision-ready controlled experiment with an explicit hypothesis, assignment unit, exposure rules, metrics, guardrails, sample assumptions, QA checks, and stopping criteria. Use when planning an A/B test, feature experiment, growth experiment, holdout, rollout validation, or experiment brief.
Who is A/B Test Setup Task for?
It is designed for growth teams, product teams, marketers and analysts.
What do I need to use it?
Nothing extra is needed. Sign in to Relay to run it with your own files and context.
What does A/B Test Setup Task produce?
Experiment brief, launch checklist and written document. Results come back with their sources and any open questions for you to check before anything is used.
How much does A/B Test Setup Task cost?
It is free to use. Paid model usage, if any, is shown before a run.
Can an AI agent use it?
Yes. Agents can read its public definition through the Relay API and run it with a person’s authorization. See the For Agents page for how to connect.
Details
Trust, access and technical details
Trust and access
- Visibility
- Public: anyone can view
- Status
- Published and maintained
- Risk
- Low
- Pricing
- Free
Use it from code or an agent
Read the public definition, then check cost and inputs with a preflight before a run.
Open definitionPOST /api/v1/runs/preflight
How agents connect to Relay →Tags and source
- Repo
- rose-digital/relay
- License
- MIT
Connections
Where A/B Test Setup Task fits
Similar tasks3
Conversion Funnel Diagnosis Task
Diagnose where and why a conversion funnel leaks, using the user's own web analytics exports, CRM stage data, or product signup and activation events. Defines stages and exact conversion formulas, chooses cohort or period measurement, cuts by segment, checks sample size before calling a difference real, ranks stages by recoverable loss rather than worst rate, gathers qualitative evidence, and ends with a scored hypothesis backlog. Use when someone asks why conversion dropped, where the funnel leaks, why signups do not activate, why leads stall between stages, for a drop-off analysis, or which funnel step to fix first. Do not use for writing landing page copy (use the landing-page-copy prompt), for designing or sizing a specific A/B test (use ab-test-setup), for building a metric hierarchy (use kpi-tree), or for retention curves after purchase (use analyze-cohort-retention).FreeTask · MarketingMarketing Incrementality Measurement Task
Measure what marketing actually causes rather than what it gets credit for. Explains attribution models (last-touch, first-touch, position-based, data-driven) and their biases; designs and sizes incrementality tests (randomized holdouts, ghost ads, conversion lift studies, geo-lift and matched markets, on/off switchbacks); outlines a lightweight marketing mix model; reconciles self-reported attribution; and writes a per-channel measurement plan with pre-registered decision rules. Use when someone asks whether a channel really works, why platform ROAS disagrees with revenue, how to run a holdout or geo test, or which attribution model to trust. Do not use for page or product A/B tests (use ab-test-setup), funnel drop-off diagnosis (use diagnose-conversion-funnel), or setting budget levels (use plan-marketing-budget).FreeTask · MarketingReferral Program Design Task
Design a customer referral or advocacy program that pays for itself. Tests fit first (is word of mouth already happening, per NPS and referral-source evidence), identifies who refers and why beyond cash, chooses the reward structure (single or double-sided; cash, credit, or product; tiered), models economics against alternative CAC with payback and fraud controls, flags tax and legal points for counsel, sets the moments to ask and the tracking, and launches as a test with a holdout. Use when someone asks to start or fix a referral, refer-a-friend, or customer advocacy program, what referral reward to offer, or whether referrals are worth it. Do not use for partner or reseller commissions, sales compensation (use design-sales-compensation), measuring paid channels (use measure-marketing-incrementality), or offer and pricing design (use develop-offers).FreeHave a task this would help with?
Describe it. Relay plans the work, picks the right playbook and shows the cost before anything runs.