Wednesday, August 20, 2008

A/B and Multivariate Testing Landscape

In our most recent whitepaper we investigated the current state of A/B and Multivariate testing, focusing on how the practice is currently used and the obstacles companies face in implementing this type of an online strategy.

Our analysis, taken from surveys given at the eMetrics Marking Optimization Summit, found that 52% of online marketing managers are currently engaged in A/B or Multivariate testing and that an even greater percentage plan to begin testing within the next year.

A high-level overview of the findings:

  • A variety of testing platforms are considered before making a final decision

  • Experiments vary in frequency, page type and page element

  • Practitioners often use more than one platform to meet their needs

  • Pretest hypothesis often fail

  • Lack of Budget was cited as the main reason by those who currently do not do any testing

  • Complexity of Tools and lack of best practices were two top challenges faced by those engaged in Testing

  • A/B and Multivariate testing is “worth it”

  • Behavioral targeting is a small but growing practice

Surprisingly lack of budget was one of the top reasons why companies were not involved in A/B or Multivariate Testing. But if you look at the chart below you will see that companies that do conduct A/B and Multivariate testing find it that it’s worth it. Majority of the respondents who engage in testing said that either it is worth it or can’t live without it. So if you are one of those who think because of lack of budget you can’t do A/B MVT, think again? Your Return will make it worthwhile to do testing. You will be able to move budget from other areas such as Ad spend to A/B testing. Do your ROI calculations. You will be surprised with the positive results you will get with developing a culture of continuous testing.

You can download the whitepaper from ZeroDash1 website.


Looking to fill your Web Analytics or Online Marketing position? Try WebAnalytics Job Board

New Position

Web Data Analyst at Alzheimer's Association (Chicago, IL)