Blog article

How long should you run A/B testing?

It is impossible today to imagine internet marketing without tools to conduct its optimization. Such methods are A / B tests. It is thanks to them that we can improve the operation of our website so that it meets selected user criteria. Knowledge about user preferences is necessary to check whether they have used the services offered on the site or made a purchase. When implementing A/B tests, you can learn not only what content users prefer, but also what style draws their attention and influences the decision to take action. You can also test the appearance of the page, landing page, advertising banners, advertising texts, mailings and more.

The subject of our study, as well as the initial psychological state ought to be determined before the start of the test, but it is worth getting acquainted with the basic A/B testing principles even earlier. Otherwise it is very likely that we will receive distorted results, and instead of helping to improve the standard of the website, the test results will reduce it. Here are the following guidelines that you should keep in mind.

Test variants at the same time

It is important to remember about the important factor – seasonality. The same internet user will pay attention to other elements and content depending on whether he is looking on weekdays or during leisure (weekends, holidays, holidays). Better results will be obtained by presenting two prepared variants of the page to users at the same time. However, to have the ultimate success it is extremely important to know your customer persona. That is the most important step in conversion optimisation and it could save you a lot of time in A/B testing.

Test only one element at a time

Many people say that this is the biggest drawback of A / B testing – the need to test only one selected element, not several at a time. This requires more time, but it also gives better results. For example, when at the same time we change the title of the article and the photo to the article, we do not know which of the elements influenced the interest of the recipient in the text. Thanks to the precise examination of each detail, we gain a greater impact on customer satisfaction, knowing exactly what goes into it.

Test from general to specific

Regardless of point 2, preparing the landing page for the test, it is best to start by preparing two (or more) very different versions. Pages should vary throughout the layout. Only later, after users indicate which option attracts their attention more, should begin to do tests on selected elements – experiment with changing headers, contact forms, photos. It may be important to even detail such as how to present the most important call-to-action – be it a simple text link or a button. Finally, remember that the smaller the difference between the tested versions, the longer we wait for a reliable result.

Use one source of traffic

Depending on what marketing goal we want to gain, e.g. increasing the number of conversions, we can use various traffic sources, such as affiliate networks, banner campaigns. When performing A / B tests, however, it is worth focusing on one source of traffic. Otherwise, users coming to the page from the search engine campaign, or the people from the mailing, may behave differently. It is important that the source provides stable traffic and is reliable. It means a lot of users, thanks to which we will be able to balance the test results and draw reliable conclusions.

Test on a homogeneous group of recipients

Testing is aimed at understanding the direction of a given group of recipients, that is why it is so important that it is a homogeneous group of users, aimed at searching for similar content and having similar goals. Otherwise, we get very divergent test results, from which it will be difficult to draw one common guideline.

What is the appropriate length of an A/B test?

When starting testing, you must set yourself up for a long-term action. Only this action will allow you to get optimal results and draw appropriate conclusions about the client’s expectations. Optimal results will be obtained by testing at least 10-14 days. Too quick to perform the test will provide unreliable results. However, generally speaking the length of an A/B test depends on the traffic amouns and conversion rates, so the best thing to do would be to wait until the test is statistically significant and provides us with at least 95% usable data.

Last, but not least: Never stop testing!

Testing is a non-single-use tool. After one or more tests, draw conclusions and use to create the page. And then proceed to the next test! You have to make as many tests as you can. One successful A/B test does not mean that you should stop using them as a way to collect information on your customer persona. Take your time for each test and run it as long as necessary, but keep in mind that these test should keep occuring. This is due to the fact that the tastes and needs of customers change during the course of time, depending on the trends generated by the Internet. It is also worth thinking about carrying out tests over time on many groups of clients, because the habits of one group may distort the results.

The tests should enter permanently into the website development strategy, thanks to which you will gain amazing results, and their implementation will be an interesting experiment. Take care, however, that the obtained results should be of the highest quality, which will help you with the above principles.