lies & spin
& Audience Research
methodologies and tools
This page considers methodologies for the measurement
of traffic, e-commerce or demographics. It also highlights
research about online audience analysis.
It covers -
history and economics of the internet measurement industry
and the shape of visualisation tools are discussed in
more detail on later pages of this guide. Background is
provided by a profile
that explores opinion polls, audience ratings and research
about the consumption of goods and services.
measuring the traffic
In addition to the Internet
Indicators site we recommend the invaluable Cyberatlas,
a site that collects information from metrics companies.
Unfortunately, when it comes to gauging e-commerce - and
more broadly making sense of traffic flows - few can agree
on what to measure and how to measure, let alone provide
hard and intelligible analysis. Figures from the
US counter-dudes at Media Metrix, Forrester, NextCard
differ remarkably, eg by >100%. There are similar variations
We have highlighted Nua
because its weekly reports aggregate national and international
surveys. Some findings by US-based web metrics company
about online retailing, software and hardware are counter-intuitive.
We question some of the analysis but it is valuable in
encouraging Australian and overseas experts to examine
Just as importantly, much of their research is publicly
available - significant when many competitors zealously
protect facts, factoids and foggy figures.
The International Journal of Scientometrics, Infometrics
& Bibliometrics (CMetrics)
points to sources of information and some of the more
useful methodological papers.
seen Websites launch Port 443 (ie, SSL) sessions in a
redirect when you visit. Roughly it went like this:
User > visits site foobar.com
Foobar.com > redirects user to hitcounter site on Port
Hitcounter site > redirects user back to Foobar.com
User sees Foobar.com load in the browser.
Because no page loads, the redirect isn't visible to the
a) you're running the sort of firewall that has a live
monitor (Tiny Personal Firewall was nice for this); or
b) the browser settings pop up a "you are about to
enter / leave a secure site" each time SSL is invoked.
Caching distorts raw data; each site attracts different
audiences, with different demographics; each survey uses
different methodologies; many surveys mis-identify certain
browsers; short reporting periods and small sample sizes
exaggerate fluctuations; and stats don't count those who
stay away because their browsers are not supported
From a methodological perspective you can't go past the
research from the eLab,
in particular the detailed 1996 paper
by Donna Hoffman & Tom Novak on New Metrics for
New Media: Towards the Development of Web Measurement
Standards. The shorter 1998 Methodology for Sampling
the World Wide Web
paper by Edward O'Neill, Patrick McClain & Brian
Lavoie is also recommended.
Two technical reports are provided by Albert-Lazlo Barabasi
& associates in their 1999 studies of the Diameter
of the World-Wide Web (here)
and Growth Dynamics of the World-Wide Web (here)
which supplement the NEC How Big is the Web study
noted on the preceding page.
site tends to report rather than evaluate but is otherwise
invaluable. MIT offers a brief list
of pointers to internet survey methodologies.
One of the more interesting academic studies, Framing
Empirical Research on the Evolving Structure of Commercial
Internet Markets (PDF),
can be found at the homepage
of Prof Shane Greenstein at Northwestern University.
Measuring the Pharoah's Arm: Web Traffic as Audience Measurement,
by Jim Conaghan, provides an amusing and plain-English
introduction to traffic measurement concepts. Larry Press's
on Tracking the Global Diffusion of the Internet offers
For site operators we recommend the lucid Measuring
the Impact of Your Web Site (New York: Wiley 1997)
by Robert Buchanan & Charles Lukaszewski.
A 2001 survey
by US researcher ComScore suggests that consumers significantly
overestimate how much they spent at ecommerce sites, arguing
that surveys are useful for identifying attitudes rather
than specific online purchasing behavior.
on Measuring Web Site Usage: Log File Analysis
by Susan Haigh & Janette Megarity notes particular
issues, complemented by Kim Bartlett's 1999 discussion
How Server Statistics Undercount Text Browsers
and Dan Tobias' comparison
of results from three counters.
The World Wide Web Consortium continues to work on proposals
for a standard terminology and broad agreement about methodologies.
Within industry there is considerable disagreement about
what and how to count ... and what the counts mean, particularly
in relation to advertising. Donna Hoffman & Thomas
on Metrics Terminology is a characteristically
incisive analysis of those disagreements.
A brief explanation of some of the jargon features later
in this guide.
The marketing guide
elsewhere on this site offers pointers to industry studies,
academic research and other information about online advertising.
Among audience tracking companies Arbitron Internet
Information Services (AIIS)
is of interest for its reports on audiences for internet
radio and other new media.
Points of entry into the literature about audience measurement
are Philip Napoli's Audience Economics: Media Institutions
& the Audience Marketplace (New York: Columbia
Uni Press 2003), Consuming Audiences? Production &
Reception in Media Research (New Hampton: Creskill
2000) edited by Ingunn Hagen & Janet Wasko, Hugh Beville's
Audience Ratings: Radio, Television & Cable
(Hillsdale: Erlbaum 1988), Ratings Analysis: The Theory
& Practice of Audience Research (Mahwah: Erlbaum
2000) edited by James Webster & Patricial Phalen and
Interpreting Audiences: The Ethnography of Media Consumption
(Thousand Oaks: Sage 1993) by Shaun Moores.
Karen Buzzard's Chains of Gold: Marketing the Ratings
and Rating the Markets (Metuchen: Scarecrow 1990),
Media Economics: Understanding Markets, Industries
& Concepts (Ames: Iowa State Uni Press 1996)
by Alan Albarran and Measuring Media Audiences
(London: Routledge 1994) edited by Raymond Kent offer
insights about broadcast rating businesses and their impact.
More detailed pointers are supplied in the supplementary
profile on audience
measurement, opinon polling and ratings.
next part (teledensity)