Before you decide and pick the testing tool, consensus should be reached on the goal(s) that you are trying to achieve. Preferably, have one major goal.
How do you set a goal?
1) Duration
Define the duration of the test before setting the goal. If the test is run for 2 weeks, the numbers associated with the goal should be based on that.
2) Specific
It should be specific and tied with a number like Increase Sign-up by 54%, Increase sales by $10,000 or Increase Scroll rate to 95%.
3) Measure Progress
Although experts’ advice website owner not to tamper with variation pages until statistical significance is reached, measuring progress every day is a critical step in the testing process. You might not be required to tamper with the variation page but based on the progress, you can create another variation page and direct some percentage of traffic to the new page.
4) Business Value
Although most Testing tools use multi-armed Bandit algorithm that automatically allocates traffic from less performing variations to better performing ones, it is important to understand the trade off between testing vs. resuming original page.
Are you losing customers because of the variation pages?
How many customers are you losing because of the tests?
Does the test impact the perception about your brand?
The answer to the above questions will help you choose between picking the best practices for tests, and picking the winner on the go, and implementing the changes at the earliest.
Once you are clear about your goals, it is time to create a hypothesis. Please remember that:
a) Hypothesis can go wrong
Like failing to reach the goal, formulating hypothesis can also go wrong if the formulation is based on wrong data, faulty assumptions, and knowledge based on previous tests in different conditions or season.
b) Mix Hypothesis
If in one test, your hypothesis was “placing orange button increased sign-up by 45%”, and in another test the hypothesis was “placing a disclaimer like leads will never be sold or shared with third-party increased conversion by 50%”, try mixing both and create a new hypothesis.
c) Update Knowledgebase
Create an internal knowledgebase to note the results of each A/B and Multivariate tests. Even results of manual testing should be documented, and shared with your team members. With this practice, team can easily mix and match hypothesis, and make decisions about test duration, variations, and effectively perform cost-benefit analysis.
If you want expert help in creating A/B and Multivariate tests, contact us or you can call us at: India (Ph): +91 9497189032, UK (Ph): +44 (0)20-3371-9976 or US (Ph): +1 650-491-0004
ByteFive provides Internet Marketing Services (Search Engine Optimization (SEO), Search Engine Marketing (SEM), Develop processes and best practices for Internet Marketing, Manage and Analyze Web Analytics packages, Perform Landing page Optimization, Manage E-Mail Newsletter, Perform Competitor Research, Create & market Content, and provide customized training to help you achieve your revenue and business goals.