lies & spin
sizing the web: domains, sites, hosts
page examines the size and shape of the web: estimates
of the number and growth of domains and hosts.
It covers -
Identification of the total number of sites on the web
is problematical, one reason for the inconsistency of
some figures cited in this guide.
Figures about registrations (live or otherwise) are more
certain than those for the number of sites online at any
one time. The implications of those figures are even more
problematical, as Andrew Odlyzko notes in a cogent paper
on Internet Growth: Myth & Reality, Use & Abuse
in the November 2000 issue of Information Impacts
Figures for domains are more certain than those for the
number of pages or the number of links. Disagreement about
terminology, data collection challenges and the commercial
incentive to look on the bright side - or simply add a
digit or two - mean that figures on number of viewers,
audience demographics or the consequences of their visits
are yet more contentious.
We have highlighted particular data collection and analysis
challenges later in this guide.
In terms of number of domains the web has experienced
- explosive growth, at a rate of between 1,600% and
850% per year
- rapid growth, at a rate of 150% per year
2002-2006 - maturing growth, at a rate of 25% per year.
the peak in 1994-5 the web grew from 700 sites to 12,000
current growth continues the web will increase to 200
million sites by 2010.
how many domains and servers
The October 2001 Web Characterisation report from
the OCLC claimed
that there were 8,745,000 unique sites, of which 3,119,000
are 'public', 2,078,000 are 'private' and 3,246,000 are
As of 10 June 2000 one global figure for active domains
- ie those that were live - was 17.75 million, including
9.48 million dotcom domains. Those figures come
from the DomainStats
site. The OCLC had estimated
that there were 7.12 million unique sites as of June 2000,
up from 4.6 million the preceding year.
that there were over 15 million domains in 2000, with
slower growth of registrations in that year (around 10%
per month) and the disappearance of 330,000 domains. By
late 2006 Netcraft identified 101,435,253 sites, including
'parked' domains and abandoned blogs or other sites that
had not been updated for two years.
One 2008 study
analysed distribution of IP address allocation in 238
countries. As of 2007 the US held 37.73% of IP addresses
worldwide, followed by the UK (12.83%), Japan (7.64%),
China (5.74%), Germany (3.81%), France (3.65%), Canada
(2.81%), S Korea (2.74%), Netherlands (2.00%), Italy (1.67%)
and Australia (1.62%). Those nations had over 80% of total
allocated IP address ranges.
We have highlighted some domain registration figures in
a supplementary profile
on Domains & the DNS.
As of October 2002 there were over 21.2 million dot-com
registrations, for example, along with 3.6 million dot-net
registrations, 740,000 dot-biz, 80,000 dot-name, 920,000
dot-info and around 2.3 million dot-orgs.
Not all of those registrations were active. Some had gone
offline forever, others were temporarily unavailable,
others had been registered by domain name speculators
in the hope of making a profit from transfer to a new
name holder, others were being used by registrars and
other entities engaged in domain name tasting.
As at December 2002 there were 310,733 registrations in
the overall dot-au space (analysed in our auDA profile),
with over 390,000 by December 2003 and 500,000 by early
2005. The number of registrations for some dot-au 2LDs
at August 2002 there were around 113,781 active domains
in the dot-nz space (discussed
in a more detailed profile).
Estimates by Domains Worldwide of other ccTLD registrations
as of October 2002 were -
We have supplied some other figures in a more detailed
note on domain name sizes.
how many registrations are live?
There is considerable industry and academic disagreement
about how many of the millions of domain registrations
are live at any one time and are associated with unique
sites, ie visitors to the address find a distinct site
rather than a 'coming soon' marker, an automated redirection
to the 'real' site at another address or simply nothing
That disagreement is evident in skepticism about "domain
name exhaustion" (claims that the supply of names
is about to run out, so ICANN or ccTLD administrators
must mint new TLDs or 2LDs) and suggestions that several
million names in popular TLDs such as dot-com or dot-info
have been registered by speculators or on a defensive
basis by major corporate interests.
On the basis of a small-scale sample an October 2002 report
by State of the Domain (SoTD),
a subsidiary of a major domain registrar, offered indicative
figures for some gTLDs -
similar report from registry operator Afilias in July
2002 suggested that 24% of dot-info sites were live, 9%
were redirected, 12% were parked and 35% were inactive.
service claimed that a total of 18,395,664 dot-com, dot-net
and dot-org registrations had not been renewed by registrants,
of which 13,783,454 were dot-coms. In October 2005 Netcraft
reported 74.4 million 'sites', including single-page sites
used by domain registrars to promote individual unsold
domain names. Reports on domain name tasting (aka domain
kiting) indicate that around
90% of registrations in some gTLDs were evanescent, exploiting
loopholes in registration rules that allow registrars
to register and then delete names within short grace period
(typically five to 30 days).
Figures for the number of blogs and other types of sites
are problematical. In mid-2003 Blogcount
estimated that there were between 2.4 million to 2.9 million
active blogs - discussed here
- although other estimates were significantly lower.
number of hosts
The January 2000 Domain Survey by the Internet Software
Consortium (ISC) suggested
that there were upwards of 88 million hosts on the net,
expected to increase to around 500 million by early 2003.
The ITU reports (PDF)
that in 2001 there were 141 million hosts across the globe,
of which 2.28 million were in Australia, 0.4 million were
in New Zealand, 2.89 million were in Canada and 106.2
million were in the US. Africa had 0.274 million (of which
0.238 million were in South Africa).
The OECD believes
that there are around 52 thousand secure servers - tools
for electronic commerce - in the USA, and upwards of 74
thousand in the OECD as a whole (a growth of 95% over
the preceding year).
Thoughtful comments on host versus domain counts are made
in Matthew Zook's 2000 article
Internet Metrics: Using Hosts & Domain Counts To Map
The Internet, complemented by the 1998 OECD report
on Internet Infrastructure Indicators (PDF).
The World Internetworking Alliance, a now-defunct advocacy
group, suggested that the number of hosts almost doubled
each year to 1997 -
OECD reports that the regional distribution of hosts per
1,000 inhabitants was as follows -
we have noted in discussing digital divides, such 'regionalisation'
can produce substantial distortions: much of Oceania for
example is comparable to Africa when figures for Australia
and New Zealand are removed.
Early growth of the web is highlighted by another estimate
of the number of sites -
where is the growth occurring?
The October 2001 Web Characterisation report from
the OCLC claimed
that growth of the web was slowing, with an estimated 8.7
million unique sites.
The Mosaic Group at the University of Omaha has a project
on Global Diffusion of the Internet, measuring
growth of the net on a global and nation by nation basis.
The Internet Geography Project (IGP)
at the University of California, under Matthew Zook,
offers authoritative maps and a number of excellent papers.
The UN Development Program 1999 Human Development Report
includes statistics about internet diffusion in the third
and fourth worlds.
The 2003 paper
Trends in the Evolution of the Public Web 1998-2002
by Edward O'Neill, Brian Lavoie & Rick Bennett suggested
that the publicly-accessible web, as of June 2002, contained
3,080,000 sites, an estimated 35% of the overall web.
Growth had continued to slow since 2001.
They commented that
of year-on-year growth rates (measured in terms of the
number of Web sites) for the period 1998 - 2002 reveals
this decline: between 1998 and 1999, the public Web
expanded by more than 50 percent; between 2000 and 2001,
the growth rate had dropped to only 6 percent, and between
2001 and 2002, the public Web actually shrank slightly
in size. The slowdown in growth of the public Web is
even more dramatically evident in absolute terms. Between
1998 and 1999, the public Web exhibited a net growth
of 772,000 sites; a similar number (713,000) were added
between 1999 and 2000. After this point, however, absolute
growth dropped off precipitously: between 2000 and 2001,
only 177,000 new public Web sites were added, and between
2001 and 2002, the public Web shrank by 39,000 sites.
next page (content)