Web Analytics Applied

Web Analytics Overkill

When people ask me “what should I track?” my usual philosophy is that tracking should be implemented in such a way as to record in detail anything that is analytically valuable, but also in a way as to make reporting as easy and digestible as possible.  In a forthcoming article, I’ll be discussing the last piece of this statement; but I’d like to say a few words about the first piece – the key phrase is “analytically valuable”.

There is such a thing as “overkill” in Web Analytics.  It often occurs when there is poor communication between the business requirements group or product owner, and the team responsible for designing and implementing the tags, who decide on a policy of “cover our bases” and tag everything they can think of.  This means scores of success events on the site, meaningless variable roll-ups, over-complex campaign tracking codes, and custom link-tracking server requests which slow down performance and result in huge amounts of data within the web analytics tool, the analysis of which would not be worth the effort (literally).

What does “analytically valuable” mean, exactly?  It does not refer to data for reporting or KPI’s (that’s what variable structure is for); rather, it refers to a data-set whose comprehensive analysis would be worth the time and resources, producing valuable, actionable recommendations about website design, user behavior, or marketing effectiveness.  A deep-dive analysis on usage of the “close” button from different popup windows is probably not worth the money.  A team implementing web analytics on a site has to not only ask, “can we capture this behavior?”, but more importantly “can I foresee someone analyzing this data to potentially produce actionable results that would be worth the resources expended?”

This last element is why the overall budget allocated to a website – now and in the future -- should be taken into account when designing a WA implementation.  As a web analyst, this may sound like heresy, but in the real world, it’s why mom-and-pop shops choose Google Analytics (if anything) instead of Omniture.  Alexander’s Pizza Shop on Main Street might have a website with a menu, directions, and pictures, but obviously would waste its money by implementing NetInsight.  It would be dishonest to recommend a state-of-the-art, “measure everything” implementation when it is clear that devoting resources to executing it and analyzing it subsequently would be a waste of money because the website is small-scale and not a significant part of business success.  That’s overkill.  If it becomes a bigger piece of the business, then a more robust implementation might be warranted.

Even within robust, enterprise-level implementations, measurement overkill is possible.  Here are some examples:

·         extensive tagging of the footer on a site.  “Terms of Use”, “Privacy Policy”, “Corporate Info” – these pages on a website usually exist for legal or compliance purposes, and exhaustive measurement of their usage would make no difference.

·         an extensive analysis of a Site Map would probably make no difference either, because they often exist more for SEO than for anything else (though if it’s used more than your navigation, you have a problem!).

·         Over-use of campaign classifications: channel, creative, ad type, adgroup, keyword, date-stamp, banner size, link within an email, landing page à all these can be legitimate and useful pieces of a campaign tracking code.   But requirements and resources will dictate what is more important; thousands of permutations are possible with all these being tracked in combination, and before implementation, questions should be asked as to whether all of these are useful.  A walk-then-run approach might be more appropriate.

·         Over-Use of success events: while websites typically can have multiple success events, some implementers take advantage of the availability of dozens of success events to tag almost any action as a separate success event.  This leads to opaque reporting because it becomes unclear which of these success events gets included in an overall total effectiveness calculation. 

·         Over-redundancy in page-naming: some sites record pages as the URL, then pass a user-friendly name into one or two variables, then pass a hierarchically-defined page name into another variable, then pass another variable in an onclick handler recording another version of the page-name of the link clicked on, while also populating site section and hierarchy variables.  When a manager wants to see a pages report, they don’t know which one to use: “www.mysite.com”, “homepage”, “hp_”, “mycompany|mainsite|homepage” – you get the picture.  And chances are, the numbers for these won’t match up.

There are many other examples, and I’m probably guilty of a few. 

I’m not saying that a “measure everything” approach is bad – on the contrary, with analytical resources available it can be a vast asset for value-optimization of the online channel.  Rather, I’m saying that a “measure everything” approach has to take into consideration the analytical value of the data, and the resources that would be required to implement and take advantage of it. 

What Visitors are Thinking and the Dangers of the Standard Design

In recent weeks, I’ve been in meetings and in conference huddles where “hard numbers,” quantitative, web analytics behavioral analysis was pitched against qualitative studies from surveys and usability teams.  The mantra goes, “web analytics tells you what a visitor did, and usability/survey data tells you why they did it.”  I’m not going to argue for or against this mantra right now, but I’d like to raise a slightly theoretical point at this “why they did it” question – and a concern for web analytics.

Qualitative analyses. where subjects are asked to test the usability or satisfaction of a website, have the usual challenges of bias, sampling, people lying, but there’s a deeper bias which could be more insidious: people could behave on (and evaluate) a website based on how they are culturally conditioned to behave on a website, by the universe and culture of website architecture and navigation with which they have been familiar for years.   There are whole theories around this kind of social behavior (Bourdieu called it habitus, Giddens called it structuralization).  Basically, surveying a heavy web-user about an unconventional website would be like plopping an ancient Greek in front of the Forbidden Palace in Beijing and asking him what he thinks of the architecture.

Let me give an example: usability studies on subjects in the early 2000’s revealed that people’s eyes are drawn to the top-left corner of a website page.  So many major portals – Yahoo, AOL, MSN, WebMD, Wikipedia, Martha Stewart, You Tube – put important navigation or important information on the upper-left.  Five years later, people surfing the web are used to important information being in the upper-left, reinforcing this paradigm.  If asked to comment on a site where important information was on the upper-right, I’m sure they’d complain.

Another example is conversion funnels.   Almost every purchase-conversion funnel I can think of asks for payment information at the very end of the process.  7 years ago, this was probably a statistically significant increase in satisfaction; now, I’m sure every survey study would say unanimously that’s where payment information should go. 

Other examples: Search (why always top-to-bottom?), Videos (why is it always a box with a >?), Products (always a thumbnail to be enlarged), header/footer (why not lefter-righter?), tabs (why always at the top?), drill-downs (why not drill-rights?).

The danger, I think, is that website designers looking at usability and satisfaction surveys are going to drive the web to a point where every website basically looks the same, much like it’s hard to tell the difference now between a BMW, Corvette, or KIA from afar.  In the last week I’ve been asked three times if there’s an “industry-standard” or “universal best practices” for website architecture or design.  But if such a “Standard Design” does emerge, my fear is that web analytics for the purposes of website optimization will be obsolete down the road. 

The classic studies of module- or link-placement, real estate analysis, funnels, forms, navigation, home page optimization, perhaps even information architecture, will simply become redundant, because there will develop a standard way people expect websites to be organized.  All web analytics will be about reporting, content analysis, and marketing, and not site-architecture or page optimization. 

Maybe it’s too late to do anything about these ur-templates – this may all have been decided in the 90’s.  But what about global websites in Asia, South America, or the Middle East designed by American firms, whose audience has not been similarly preconditioned to the Standard Design and who may have other cultural dispositions against the American or Western paradigm (recall that much of the world reads right-to-left)?

Why not build a site adventurously and creatively, and then use – you knew it was coming – testing on behavioral data to see if the real population respond significantly to these changes?  I’d love to see a website with a bottom-nav, tabs on the side, internal search where you scroll right-to-left (think of the advertising potential), new ideas for navigation (I’ve seen some good Web2.0 examples).  Let such a site launch in Qatar, and rather than sending surveys to focus groups there, use cheaper behavioral data to see if it rivals the Standard Design?

I’m not a web designer, nor do I have that background.   But as a web analyst, I have seen a certain standardization in many site templates which often are driven by focus-groups and qualitative analysis, after which traditional web analytics applied to site or page optimization is reduced to miniscule changes in the ordering of top-nav elements or placement of banner ads.  And thus loses its value-proposition for us.