Friday, March 16, 2007

Referring Domains Demystified - Part II

In part I discussed how the referring domain and pages are reported by a web analytics tool. In this part I will discuss why your own domain shows up as the referring domain.

There are following three main reasons why your own domain name shows up as the referring domain.

1. If a user waits for 30 min (or whatever your session time out is) before clicking on the next link on your site.

It is a standard practice to use 30 min session time out. This means that if a visitor waits more than 30 mins to click on a link on the website, the click constitutes a new visit.

As in my last post, let’s take an example of visits for one visitor. For this example I am only showing 5 fields (s-ip, data, time, URI stem, cs(referrer) )

Below is the data for a visitor:



The visit started with a referral from http://www.google.com/?q=seattleindian. The referring domain in your web analytics tool will be Google.com

Let’s assume, this visitor goes on a lunch break leaving the site open in her browser. Come back after an hour and clicks on the home page links, here is how the log file will look like as




This constitutes a second visit (I am assuming a 30 min session time out). The referring page will be http://www.seattleindian.com/seattle/advetise.asp and the referring domain will be SeattleIndian.com for this second visit.

If you are a content site that has long articles or have downloads that takes more than 30 mins to complete, chances are you will see your own domain as the number one referring domain.

2. If you intentionally or un-intentionally exclude one or more of your pages from analysis either by not including javascript tracking (tag-based solutions) or specific exclusions that does not allow that page request to be tracked(this applies to both log file-based and tag based solutions)

Let’s assume http://www.seattleindian.com/seattle/default.asp, the home page of seattleIndian.com is not tagged with the web analytics JavaScript code or for some reason is omitted from the analysis (hard exclude either intentionally or unintentionally).

Taking the same example as above, the log file will look like the following



Note that the first log line


is no longer there. The log file won’t even contain Google.com as the referrer because the visit did not begin at http://www.seattleindian.com/seattle/default.asp (since it was not tagged or was excluded). In fact, according to the analytics tool, the visit began at /seattle/bollywood.asp and was referred by the non-tagged (or excluded) page, the home page of SeattleIndian.com. In this case /seattle/default.asp, the page which is not tagged will show up as the referrer and the referring domain will be the domain itself SeattleIndian.com

Note: I have seen a lot of unintentional excludes that affect the reporting. It is highly recommended to use a third party accuracy audit to make sure your reports are configured properly. Contact me if you need more details or help with this. We do this all the time.

3. If you have sub domains that have their own reporting profiles or suites (or whatever you call them) they could cause your own site to show up as referring domain.

Let’s take an example of http://www.usaindian.net which has several city-specific subdomains e.g. seattle.usaindian.net, ortland.usaindian.net etc. Any reporting that excludes http://www.usaindian.net home page will show a lot of referrers from its own domain i.e. usaindian.net

Here is the log file of a user who searches seattleIndian on Google and then clicks on the link to seattle support page (http://seattle.usaindian.net/seattle/support/asp) from USAIndian.net home page.



Say you want to create a profile for Seattle area only i.e. exclude everything else and only report on traffic to seattle.usaindian.net domain. If you only include traffic from seattle.usaindian.net (or s-ip of 1.2.3.5 in the example above) in your reports then the referring domain will be http://www.usaindian.net, i.e. your own domain.

I hope this was helpful. This concludes my two part series on Referring domains and pages. As always send me your comments and questions.