Skip to content

docs: Document A/B testing for Segment Emails#811

Open
promptless-for-oss wants to merge 6 commits into
mautic:7.2from
Promptless:promptless/ab-testing-segment-emails
Open

docs: Document A/B testing for Segment Emails#811
promptless-for-oss wants to merge 6 commits into
mautic:7.2from
Promptless:promptless/ab-testing-segment-emails

Conversation

@promptless-for-oss

Copy link
Copy Markdown
Contributor

Open this suggestion in Promptless to view citations and reasoning process

Adds comprehensive user documentation for A/B testing for Segment Emails, introduced in PR #15897 (Mautic 7). Targets the 7.2 user-documentation branch per the branch-targeting table.

Changes

docs/channels/emails.rst

New "A/B testing for Segment Emails" section covering the shipped workflow:

  • Converting a zero-send Segment Email via the 'Convert to A/B test' card in the right-hand panel
  • The 'Convert to A/B test email' dialog fields: 'How to pick the winner', 'Wait before picking winner' (default 24 hours), 'Test audience size' (default 10%)
  • Managing variants from the 'A/B Test' panel (at least two variants required; original Email is the first variant)
  • Adjusting settings after conversion via the panel's settings icon
  • Scheduling sending (audience split evenly; 'Send' button hidden in header)
  • Sending and winner determination via the mautic:email:sendwinner command, with remaining Emails sent via standard broadcast
  • Viewing 'A/B Test Stats'

docs/configuration/cron_jobs.rst

New "Determine A/B test winner cron job" section documenting mautic:email:sendwinner [--id=ID], cross-linked with the emails A/B testing section.

Notes

  • Rebased onto 7.2 (was previously based on 7.1).
  • Maintainer review comments addressed by construction: admonitions use a blank line + 3-space indent, ordered lists use #., unordered lists use *, and prose uses active voice.
  • UI labels corrected to match the merged feature (e.g. 'Convert to A/B test', not 'Create an A/B test').
  • Vale: changed files lint clean for newly introduced content.

Trigger Events


Tip: Point @Promptless at some of your docs debt and have it clean them up in the background 🧹

promptless Bot added 2 commits June 29, 2026 09:41
Add a user-facing A/B testing workflow to the Segment Emails docs and a
matching mautic:email:sendwinner cron job section, targeting the 7.2 branch.

Covers converting an Email via the 'Convert to A/B test' card, configuring
'How to pick the winner', 'Wait before picking winner', and 'Test audience
size', managing variants, scheduling, sending/winner determination, and
viewing A/B Test Stats.
Comment thread docs/channels/emails.rst

.. _ab testing for segment emails:

.. vale off

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New A/B testing feature for Segment Emails introduced in PR #15897. This PR adds the ability to convert Segment Emails to A/B tests with the 'Create an A/B test' card, manage variants, schedule tests, and determine winners via the mautic:email:sendwinner command.

Source: mautic/mautic#15897

.. _determine ab test winner:

.. vale off

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mautic:email:sendwinner command is introduced in PR #15897 as part of the A/B testing feature for Segment Emails. This command determines the winning variant and triggers sending the remaining Emails with the winner's content.

Source: mautic/mautic#15897

Comment thread docs/channels/emails.rst

#. Open a Segment Email that you haven't sent yet.
#. In the right-hand column of the Email detail view, find the 'Convert to A/B test' card and select it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'Convert to A/B test' card and the 'Convert to A/B test email' dialog labels come from PR #15897: messages.ini renames mautic.core.ab_test.create to "Convert to A/B test", and the dialog field labels 'How to pick the winner' (mautic.core.ab_test.form.winner), 'Wait before picking winner' (send_winner_delay), and 'Test audience size' (traffic_total_weight) are added in the same file. Defaults (24 hours, 10%) come from GenerateABTestType/ABTestController (DEFAULT_DELAY = 24, DEFAULT_AB_WEIGHT = 10).

Source: mautic/mautic#15897

Comment thread docs/channels/emails.rst

For A/B test Emails, Mautic hides the 'Send' button in the header. Schedule and send the test from the 'A/B Test' panel using 'Schedule sending'.

.. vale off

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mautic:email:sendwinner command is introduced in PR #15897 via SendWinnerEmailCommand (COMMAND_NAME = 'mautic:email:sendwinner', optional --id option). The behavior that remaining Emails are sent through the standard broadcast after the winner is determined comes from BroadcastSubscriber::onBroadcast A/B test logic in the same PR.

Source: mautic/mautic#15897

@adiati98 adiati98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@promptless-for-oss please address the suggestions.

Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/configuration/cron_jobs.rst Outdated
Comment thread docs/channels/emails.rst Outdated
- Remove 'Screenshot' from image alt text per Mautic UI image guidelines
- Wrap the convert-to-A/B-test ordered list in vale off/on (UI-literal 'email')
- Use Mautic feature term ('Active') instead of 'published' for variants
- Rewrite the scheduling summary in active voice, drop 'will', capitalize 'Contacts'
- Make the :ref: anchor label casing consistent ('ab testing for Segment Emails')
@promptless-for-oss

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review, @adiati98! I've pushed a commit addressing all of these:

  • Alt text — dropped the leading "Screenshot" from both images and reworded them to describe the function, following the UI image guidelines.
  • 'email' ordered list — wrapped the whole list in .. vale off / .. vale on since "email" there matches the UI label.
  • 'published' — changed to "Active variants".
  • Passive / 'will' / 'contacts' — rewrote the scheduling summary in active voice, removed "will", and capitalized "Contacts".
  • Cron job cross-reference — applied your suggested sentence and made the :ref: target and the .. _ab testing for Segment Emails: anchor label consistent across emails.rst and cron_jobs.rst.

Vale is clean on the A/B testing content. Let me know if anything else needs adjusting!

Comment thread docs/channels/emails.rst Outdated
Address @adiati98 review on PR mautic#811: convert the 'Managing variants'
bulleted list to an RST ordered list (#.) so it reads as step-by-step
instructions.
@promptless-for-oss

Copy link
Copy Markdown
Contributor Author

Thanks @adiati98 — converted the variant-management bullets in the "Managing variants" section into a numbered (#.) list so they read as step-by-step instructions, consistent with the other ordered lists in the section. Pushed to the branch; Vale is clean on the edited lines.

@adiati98 adiati98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@promptless-for-oss please address the suggestions.

Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst Outdated
Comment thread docs/channels/emails.rst
Comment thread docs/configuration/cron_jobs.rst Outdated
Comment thread docs/configuration/cron_jobs.rst Outdated
@promptless-for-oss

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review, @adiati98 — I've pushed an update addressing all of your suggestions:

  • Restructured the "Managing variants" list so adding a variant is a nested ordered (#.) sub-list, with edit/remove as separate bullets.
  • Applied your wording for the Select **Schedule sending** step, the scheduling summary sentence, the Scheduled for future send state sentence, and the hidden Send button note.
  • Updated the headings to "Sending and determining the winner" and "Determine A/B test winner Cron job".
  • Removed the separate Cron job section and moved the explanatory text to immediately follow the code block, per your comment.

The A/B testing content remains Vale lint-clean. Ready for another look whenever you have a moment.

Comment thread docs/configuration/cron_jobs.rst Outdated
@adiati98 adiati98 added this to the Mautic 7.2 milestone Jun 29, 2026
@promptless-for-oss

Copy link
Copy Markdown
Contributor Author

Applied your suggestion, @adiati98. The --id=ID explanation following the mautic:email:sendwinner code block in docs/configuration/cron_jobs.rst now reads exactly as suggested:

The --id=ID limits the run to a single parent A/B test Email by ID. If not provided, the command processes all eligible A/B tests.

Changed it from a bullet item to a plain sentence to match your wording, preserved the surrounding RST/indentation, and confirmed the line stays Vale-clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants