On August 9th 2012, Google released the guidelines for conducting A/B and multivariate tests. The guidelines were released in response to the commonly asked question – “How will testing impact search ranking?” Google has warned marketers from running the experiments for too long.
Below are some guidelines for running an effective test with minimal impact on your site’s search performance.
1) Don’t Cloak
Cloaking is the process of showing different set of contents to bots and humans based on the user-agent value.
When you visit a site, your browser sends an HTTP request
HTTP Request Header
GET: /
HOST: www.bytefive.com
USER-AGENT=Googlebot
REFERRER=http://www.google.com/search?hl=en&q=bytefive
Server sends an HTTP response.
HTTP Response Header
Status: HTTP/1.1 200 OK
Date: Sat, 11 Aug 2012 09:23:46 GMT
In cloaking, the website checks the USER-AGENT and if it is a search bot like Googlebot, then it will show the SEO optimized page and for other user-agents, it will show a different page.
2) Use rel=”canonical”
Add rel=”canonical” link in the head section for all your variation pages like:
Variation 1
<head>
<link rel="canonical" href="http://www.example.com/control "/>
</head>
Variation 2
<head>
<link rel="canonical" href="http://www.example.com/control "/>
</head>
The Google team prefers rel=”canonical” to NOINDEX for testing. If you don’t mention rel=”canonical” and just use NOINDEX, there is a possibility that Google Bot will ignore the “NOINDEX” directive and randomly pick one of the variations as the main page. When the bot sees the actual main page, it will consider it as a duplicate and might de-index the page.
3) Use 302 redirects instead of 301s
Google guidelines recommend use of 302 redirects instead of 301s for all A/B tests. There is a reason behind this guideline. 301 redirects passes nearly 90 to 99 percent of link value (studies by SEOMoz) while 302 passes zero value. When you are running your A/B tests, you don’t want your control pages to lose link juice.
302 redirect is a temporary redirect and search engines will ignore the variation pages as long as you are running the experiments. Once your experiments are complete, 301 redirect all your variation pages. Visitors to your site might have reached the variation page and would have bookmarked it or shared it with their friends. Don’t lose that traffic.
4) Don’t run the experiments for too long
This is a controversial guideline. The Conversion rate optimization evangelists recommend testing the page until you reach statistical significance. Google recommends the same
“A good testing tool should tell you when you’ve gathered enough data to draw a reliable conclusion”
That is great; but here comes the warning:
“If we discover a site running an experiment for an unnecessarily long time, we may interpret this as an attempt to deceive search engines and take action accordingly”
Google does not have access to the visitor data for each variation, unless you have integrated the testing tool with Google Analytics. For low to medium traffic sites, it might take considerable number of months before reaching any statistical significance. The “unnecessarily long time” in the guideline is ambiguous and might scare marketers from more flexible tools like VWO and Optimizely, and force them to use GA Content Experiments (which honestly require lot of work to set up a simple A/B test).
P.S: Don’t forget to remove the scripts and markups after you have found the winner.