Wednesday, August 19, 2009

Is CAPTCHA Eating Up Your Conversions?

CAPTCHA acronym for “Completely Automated Public Turing test to tell Computers and Humans Apart” is a way for sites to block spam. According to Wikipedia it is
“A CAPTCHA or Captcha (pronounced /ˈkæptʃə/) is a type of challenge-response test used in computing to ensure that the response is not generated by a computer. The process usually involves one computer (a server) asking a user to complete a simple test which the computer is able to generate and grade. Because other computers are unable to solve the CAPTCHA, any user entering a correct solution is presumed to be human. Thus, it is sometimes described as a reverse Turing test, because it is administered by a machine and targeted to a human, in contrast to the standard Turing test that is typically administered by a human and targeted to a machine. A common type of CAPTCHA requires that the user type letters or digits from a distorted image that appears on the screen.”


Below is an example of a CAPTCHA on craigslist.



Though CAPTCHA is a great tool for blocking spam it could be coming in the way of user experience and resulting in a lower conversion than you would have had without it.

I have come across many sites where CAPTCHAs are totally illegible. Such CAPTCHAs not only deter SPAM but also valid human visitors. Even if the CAPTCHA is totally legible it adds one extra step between a visitor and the conversion. Sometimes it takes few refreshes of the CAPTCHA before a visitors gets it right, resulting in a very frustrating experience.

You can spend all your time doing A/B and Multivariate testing the form layout, images, text etc. but probably won’t move the needle if your CAPTCHA is the culprit.

If you have CAPTCHA on your site then I suggest following 5 checks to ensure you have a good CAPTCHA on your site.

  • Clear - Is it clear to your visitors that you have to enter the CAPTCHA before the form can be submitted? Some sites don’t make it clear and leave visitors wondering why their form is not getting submitted. Make sure there is help available on CAPTCH if the visitors get stuck. Also make sure that there is a refresh button to refresh the CAPTCHA image incase visitors can't read it.

  • Readbility - Check all your CAPTCHA images. Can you read them? Will you visitors be able to figure out what your CAPTCHA reads?

  • Accessibility - Visually impaired visitors should be able to fill the form else you will loose them at CAPTCHA.

  • Time – How fast is your CAPTCHA? If it is slow to load or validate you might be loosing conversions.

  • Protection - Do you have a huge SPAM problem (that you need a CAPTCHA or did you put it because everybody else is putting them too? Keep in mind that event a CAPTCHA might not completely protect you from SPAM.

Is CAPTCHA hurting your conversions?

I suggest you conduct A/B testing to understand how CAPTCHA might be affecting your conversions. Create a version of the page that does not have CAPTCHA and test it against the control version (your current version with CAPTCHA).

Analyze the results. You should analyze the conversions you get from each version. Deduct any SPAM when calculating the conversion. Calculate true conversions per month/year from both versions. Make sure your results are statistically significant. Most likely you will see lower conversion from the version with CAPTCHA and higher SPAM from the version without CAPTCHA. Considering the impact of SPAM on your form, determine if the efficiency (clean data) gained by having CAPTCHA on your form outweighs the gain of extra conversion when you remove CAPTCHA. If CAPTCHA is doing more harm than good then remove it.

Examples of CAPTCHA


This CAPTCHA is hard to read but has help and accessibility built into it



This CAPTCHA is hard to read and does not have accessibility




This CAPTCHA is easy to read and has help and accessibility built into it


Have you seen a bad CAPTCHA that caused you to leave the site? Send me the link.

----------------------------------------------------------------------------
Looking to fill your Web Analytics or Online Marketing position?
Post your open jobs on http://www.web-analytics-jobs.com/
----------------------------------------------------------------------------

6 comments:

  1. Interesting.

    I think that the most difficult CAPTCHAs that I know are on Google. It is amazing that sometimes I have to try up to 5 times before I succeed!

    Another point: besides losing conversions, you are also showing a lack of respect to the time of your loyal customers.

    ReplyDelete
  2. Also, reCaptcha looks butt ugly, requires usability and only usually has readable text.

    ReplyDelete
  3. Interesting post.

    I recently read a post regarding test results with and without CAPTCHA (but I can't find the link... :/)

    The outcome was that while it reduced spam - it caused a significant decrease of conversion. I am not totally surprised.

    There are alternatives to CAPTCHA to reduce spams - maybe not 100% bulletproof - but still interesting and not impacting the user experience.

    One of these is to have an "hidden" field that is left empty. If it is filled in (by a spam robot) then the data should be discarded.

    As you pointed out, it happens to often that these are hard to read - requiring several tries before succeeding.

    Once, it happened to me that the CAPTCHA image failed to be generated (so no image displayed) - I had no other choice to quit (after spending time filling the form for nothing). Never went back :-).

    Cheers,

    Michael
    www.kaizen-analytics.com

    ReplyDelete
  4. Agree that Google's captcha is torture.

    Not sure how many people they tested it on before introducing it but they must be losing many comments as well as blocking spammers.

    ReplyDelete
  5. Great post Anil.

    There is also some good data on this subject in this recent post on SEOMoz: http://www.seomoz.org/blog/captchas-affect-on-conversion-rates

    Definitely seems that CAPTCHA inhibits conversions on most sites.

    ReplyDelete
  6. reCaptcha is an interesting project. It uses user input to help transcribe public domain literature.

    ReplyDelete

I would like to hear your comments and questions.