Multivariate Testing: If you are currently working on improving conversion in your critical pages then you might have come across the term “Multivariate Testing’. A/B Testing is a simple concept. You test for two versions A and B and find out which version converts better. The variation can be any aspect of the page – design, colour of the button, headline, background colour or Call to Action. But you can only test two variables at a time.
The problem arises when you are not sure which component of the page should be optimized for maximizing the conversion. Multivariate testing works for such scenarios. With multivariate testing, you can test any combination of variations. The catch is that if you don’t have enough page views per month then the test would take anywhere between three to five months to get statistical significance.
Types of Multivariate testing
Multivariate testing is done through proxy servers or through JavaScript codes. Most popular tools like Google Website Optimizer and Visual Website Optimizer uses JavaScript to insert page variations. Depending on the traffic available in your website and the available time for the tests, you should either pick Full Factorial or Fractional Factorial Multivariate testing. Full Factorial Multivariate testing: When you start with Multivariate testing, the basic thing that you have to understand is that this is a statistical experiment. The objective of the experiment is to find out which page view provides the maximum return in terms of dollars. In a page there are many elements. The elements that you pick for the experiment and the variations would impact the results of the experiment.
Let us dive into an example: If you are testing two elements (call to action) in two buttons:
Call to Action 1: Learn More
Call to Action 2: Learn More, Now!
Button 1: Green
Button 2: Red
How many combinations of the elements should be tested before a conclusion can be reached?
1) Learn More - Green Button
2) Learn More - Red Button
3) Learn More, Now! - Green Button
4)Learn More, Now! - Red Button
Four This is easy, right! But the problem is that with the increase in elements in the test, the combination increases drastically. Take for example if you are testing one more call to action on the same two button variations. How many more combinations you have to test?
2^3 = 8
For Smaller Business (low traffic) and shorter campaigns, full factorial multivariate testing might not work out. But if you want to figure out the best possible combinations of page elements, full factorial method might be a good fit for you.
Fractional Factorial Multivariate testing: This method of testing only takes a subset of all possible combinations of elements and variations. The disadvantage is that the successful implementation of this testing depends on the accuracy of the assumptions that the team make. The advantage of such test is that you can pick a winner in the shortest period of time.
If you look at the above example, in Fractional Factorial Multivariate testing the testing team makes an assumption like “In our previous tests, Green Buttons have given us 10x times more click rate than red buttons. Let us test only the Green Buttons”
Call to Action 1: Learn More
Call to Action 2: Learn More, Now!
Button 1: Green
How many combinations you need now? Only 2
Learn More – Green
Learn More, Now! - Green
But can you say that the tests are complete? No!
Maybe Red Button with the call to Action – “Learn More” gives the best click rate. Even if you have conducted the test previously with the red button, the day, month and time of the day produces different results.
What is the Solution?
The solution is to switch between Full Factorial and Fractional Factorial multivariate testing in a single day. Monitor the time of the day when the traffic in your website peaks. Enable Full Factorial multivariate testing during that time period. Use the results from that time period to make assumptions about Fractional Factorial testing. If you want expert help in Creating Multivariate Test, contact us or you can call us at: India (Ph): +91 9497189032, UK (Ph): +44 (0)20-3371-9976 or US (Ph): +1 650-491-0004
ByteFive provides Internet Marketing Services (Search Engine Optimization (SEO), Search Engine Marketing (SEM), Develop processes and best practices for Internet Marketing, Manage and Analyze Web Analytics packages, Perform Landing page Optimization, Manage E-Mail Newsletter, Perform Competitor Research, Content Creation and Marketing) and Training to help you achieve your revenue and business goals.