Saturday, February 17, 2007

Search for a Mertics to Compare Web Sites

Steve Rubel wrote:
"The page view is on life support. It fails to capture all of the myriad of ways consumers engage in online activities without ever leaving a web page. To get a feel for this, spend some time playing with Yourminis. So what will replace it and when will that happen? Let's handicap the field." On his blog What Will Replace the Almighty Page View and he thinks it willbe events or time spent.

Eric Peterson voiced his opinions on this subject on his blog
Worried about page views dying? Don't be.

I however, have a different opinion than Steve and Eric. I think Unique Users make the most sense of all the different metrics that are discussed in these two articles. I also think that maybe we should not just rely on a single metrics such as page views, unique users, no. of events, sessions etc? Maybe it is time to find a new metrics combining some (or all) of these above metrics to compare web sites?

I am going to list my reasons why I think, time spent, event, pages views and session by themselves don’t make sense as measure to rank one site against another.

Time Spent on Site: I am not going to go into detail on this one, you can read my blog article that explains why I am not a big fan of "Time Spent on Site".

Page Views:

Page views were not the right metrics to compare web properties to begin with. Why? Because they can be manipulated very easily. Say it t takes 2+ pages on site A to do anything compared to 1 page on site B, is site A really doing better than site B? Additionally you can split your content in as many pages as you want, there is no min standard page size, thus inflating page views.

Events: I think events will have the same issues as page views, plus everything in flash or AJAX interaction could be an event, where do you draw the line? What count’s as a valid event?

Session: I agree with Eric that this is a relatively stable metrics and agree with all the things he listed out for session. However, I don’t think sessions (alone) make sense as measure of measuring relative value of web properties.
I agree that Unique users have issues but those issues affect every web property, most likely in similar fashion. For example, if I delete my cookies, most likely I will delete for both myspace and yahoo.

Here is an example to make my point:
I go to myspace and read 2 pages in 1 min, wait 31 mins and then go back and read 2 more pages in 1 min. So here is what the web analytics reports will look like

2 sessions (visits)
1 unique users
4 page views
2 mins.

Now I go to yahoo spend 2 mins reading 4 pages in 1 session. Here is what the web analytics report will look like
1 session
1 unique users
4 page views
2 mins

What about the following scenario

Which property is number 1? Aren’t they both the same? If you use Session myspace appear to be number 1. But if you look at Unique Users and rest everything too, they both are equal.

What about the following scenario

I go to myspace site and read 2 pages in 1 min, wait 31 mins and then go back and read 2 more pages in 1 min then come back after 2 hours and read 2 more pages for 1 min. So here is what the web analytics report will look like

3 sessions
1 unique users
6 page views
3 mins.

Now I go to Yahoo spend 2 mins reading 4 pages in 1 session. My friend goes and reads 2 pages in 1 mins in one session. Here is what report will look like

2 sessions
2 unique users
6 page views
3 mins

Which property is number 1? Session will say myspace, even tough yahoo is getting more users?.

Will an advertiser be happy by showing same ad 10 times to one user in (more session but only 1 user) or they will be happier by showing the 5 times to 2 users (fewer sessions but more users)? So shouldn’t yahoo be number one in this scenario?

Let’s face it, it is about unique users. But other metrics do play a role in determining the value of a website

So do you agree that it is time to find a new metrics combining some (or all) of these above metrics? Comments/Thoughts?

3 comments:

  1. Anil,
    you may be forgetting that for advertisers it is about both reach (number of people) and frequency (number of impressions/views per person per time frame) so an advertiser might, under the right circumstances, be perfectly happy with less reach and higher frequency.

    Visitors as a measurement are just too unreliable. The definition for visit is fairly simple (and industry accepted) and much less susceptible to gaming while unique visitor are not. Please see the IAB definitions and notice that the definition for visit is about a half a page while the unique definition takes about about a page and half of guidance.

    IMHO, in addition to needing something that is stable and accepted, it needs to be easy. A page view, an ad impression, these are easy concepts and make for simple comparisons.

    Finally, page views are typically used in conjunction with reach and other metrics for ad planning but one of the reasons that we hear about page views so often (other than it's usually the biggest number available) is that they bear the closest relationship to the ad impression.

    ReplyDelete
  2. Clint, I agree with you that both reach and frequency matter to advertisers. Behavioral Targeting is perfect example of where less reach and higher frequency might make sense.
    I don't understand how page view can be a measure of reach? Visits are better than page views to measure reach. Unique Visitors will be even better. I agree that Unique User measure is not as reliable as Visits.

    ReplyDelete
  3. Anil/Clint,

    The quest for a common currency in terms of web metrics is an interesting one.

    I'm not sure I agree that a "unique user" measure is less reliable than a "visit" one just because the definition for the latter is more concise/simple.

    You'll note in the IAB definitions (and the ABCe/JICWEBS ones) that visits are expressed in terms of actions by "unique browsers", which is then wholly dependent on the 1.5 pages of guidance in clarifying that metric.

    Get your unique user criteria wrong, and your visit figures are no more reliable. And (back to the common currency theme) there are still 3 different "accepted" methodologies for measuring unique users that will give wildly different results under the same name that will project through to visit figures as well.

    IMHO page/event based measures are always going to be very specific to the content, structure and purpose of a particular site and thus of limited value for comparison with other sites.

    However, I appreciate the advertising perspective where the event "how many times has my ad been seen" has more value to it. I've seen some models where a target number of unique users attached to a distinct content group (implied relevance) is used as a measure, and perhaps that might be a good one to develop (arguably closer to a newspaper model).

    Ultimately, it looks like the general tide towards metrics more focused around unique users is the order of the day. Then we can start worrying about users switching devices (home/work/mobile)...

    ReplyDelete

I would like to hear your comments and questions.