methodologies and tools

This page considers methodologies for the measurement of traffic, e-commerce or demographics. It also highlights research about online audience analysis.

It covers -

• measuring the traffic
• methodologies
• terminology
• advertising
• media analysis

The history and economics of the internet measurement industry and the shape of visualisation tools are discussed in more detail on later pages of this guide. Background is provided by a profile that explores opinion polls, audience ratings and research about the consumption of goods and services.

     measuring the traffic

In addition to the Internet Indicators site we recommend the invaluable Cyberatlas, a site that collects information from metrics companies.  

Unfortunately, when it comes to gauging e-commerce - and more broadly making sense of traffic flows - few can agree on what to measure and how to measure, let alone provide hard and intelligible analysis.  Figures from the US counter-dudes at Media Metrix, Forrester, NextCard differ remarkably, eg by >100%. There are similar variations within Australia.

We have highlighted Nua because its weekly reports aggregate national and international surveys. Some findings by US-based web metrics company Statmarket about online retailing, software and hardware are counter-intuitive. We question some of the analysis but it is valuable in encouraging Australian and overseas experts to examine their assumptions.

Just as importantly, much of their research is publicly available - significant when many competitors zealously protect facts, factoids and foggy figures.

The International Journal of Scientometrics, Infometrics & Bibliometrics (CMetrics) points to sources of information and some of the more useful methodological papers.

     methodologies

Whether it's via Javascript or something else, I have seen Websites launch Port 443 (ie, SSL) sessions in a redirect when you visit. Roughly it went like this:
User > visits site foobar.com
Foobar.com > redirects user to hitcounter site on Port 443
Hitcounter site > redirects user back to Foobar.com
User sees Foobar.com load in the browser.
Because no page loads, the redirect isn't visible to the user unless:
a) you're running the sort of firewall that has a live monitor (Tiny Personal Firewall was nice for this); or
b) the browser settings pop up a "you are about to enter / leave a secure site" each time SSL is invoked.

Caching distorts raw data; each site attracts different audiences, with different demographics; each survey uses different methodologies; many surveys mis-identify certain browsers; short reporting periods and small sample sizes exaggerate fluctuations; and stats don't count those who stay away because their browsers are not supported

From a methodological perspective you can't go past the research from the eLab, in particular the detailed 1996 paper by Donna Hoffman & Tom Novak on New Metrics for New Media: Towards the Development of Web Measurement Standards. The shorter 1998 Methodology for Sampling the World Wide Web paper by Edward O'Neill, Patrick McClain & Brian Lavoie is also recommended.  

Two technical reports are provided by Albert-Lazlo Barabasi & associates in their 1999 studies of the Diameter of the World-Wide Web (here) and Growth Dynamics of the World-Wide Web (here) which supplement the NEC How Big is the Web study noted on the preceding page.

The Cyberatlas site tends to report rather than evaluate but is otherwise invaluable.  MIT offers a brief list of pointers to internet survey methodologies.

One of the more interesting academic studies, Framing Empirical Research on the Evolving Structure of Commercial Internet Markets (PDF), can be found at the homepage of Prof Shane Greenstein at Northwestern University.

Measuring the Pharoah's Arm: Web Traffic as Audience Measurement, an article by Jim Conaghan, provides an amusing and plain-English introduction to traffic measurement concepts. Larry Press's ongoing project on Tracking the Global Diffusion of the Internet offers another perspective.

For site operators we recommend the lucid Measuring the Impact of Your Web Site (New York: Wiley 1997) by Robert Buchanan & Charles Lukaszewski.

A 2001 survey by US researcher ComScore suggests that consumers significantly overestimate how much they spent at ecommerce sites, arguing that surveys are useful for identifying attitudes rather than specific online purchasing behavior.

The brief on Measuring Web Site Usage: Log File Analysis by Susan Haigh & Janette Megarity notes particular issues, complemented by Kim Bartlett's 1999 discussion How Server Statistics Undercount Text Browsers and Dan Tobias' comparison of results from three counters.

     terminology

The World Wide Web Consortium continues to work on proposals for a standard terminology and broad agreement about methodologies. 

Within industry there is considerable disagreement about what and how to count ... and what the counts mean, particularly in relation to advertising. Donna Hoffman & Thomas Novak's paper on Metrics Terminology is a characteristically incisive analysis of those disagreements. 

A brief explanation of some of the jargon features later in this guide.

     advertising

The marketing guide elsewhere on this site offers pointers to industry studies, academic research and other information about online advertising. 

     media analysis

Among audience tracking companies Arbitron Internet Information Services (AIIS) is of interest for its reports on audiences for internet radio and other new media.

Points of entry into the literature about audience measurement are Philip Napoli's Audience Economics: Media Institutions & the Audience Marketplace (New York: Columbia Uni Press 2003), Consuming Audiences? Production & Reception in Media Research (New Hampton: Creskill 2000) edited by Ingunn Hagen & Janet Wasko, Hugh Beville's Audience Ratings: Radio, Television & Cable (Hillsdale: Erlbaum 1988), Ratings Analysis: The Theory & Practice of Audience Research (Mahwah: Erlbaum 2000) edited by James Webster & Patricial Phalen and Interpreting Audiences: The Ethnography of Media Consumption (Thousand Oaks: Sage 1993) by Shaun Moores.

Karen Buzzard's Chains of Gold: Marketing the Ratings and Rating the Markets (Metuchen: Scarecrow 1990), Media Economics: Understanding Markets, Industries & Concepts (Ames: Iowa State Uni Press 1996) by Alan Albarran and Measuring Media Audiences (London: Routledge 1994) edited by Raymond Kent offer insights about broadcast rating businesses and their impact.

More detailed pointers are supplied in the supplementary profile on audience measurement, opinon polling and ratings.




   next part (teledensity)