How long to run an A/B test

A/B test results are generated by the responses of email recipients to the several email variants that you are comparing. To generate meaningful results, give recipients sufficient time to receive and respond to the email messages before the system declares a winner and completes the mailing. By default, the system evaluates recipient response 72 hours after the test begins.

You can configure a different test duration or specify a specific date and time to evaluate recipient responses. You can update the duration or date at any time up to when the system begins to send the winning email variant.

Consider your audience and the nature of the design or content variation that you are testing. If you end the test too soon by setting a short test duration, you might get too few responses to generate an accurate result. Waiting too long to evaluate results might mean that you introduce an unnecessary delay long after the winning variant is obvious. Examine previous response rates and times for your target audience to better determine how long to run the A/B test.

If the mailing must be sent by a specific date, schedule the A/B test accordingly. For example, if you conduct an A/B test for a holiday mailing, allow enough time to generate meaningful results and specify an evaluation date that sends the final mailing in time for the holiday. Similarly, if a holiday occurs during the test, and you anticipate a corresponding drop in email response rates, you might consider starting the test earlier to allow enough time to generate an accurate test.