official secrets

This page offers another perspective on information flows by examining official secrets legislation, government secrecy regimes and protection of private sector information. 

It covers -

• secrecy in the digital age
• secrecy legislation
• the ancien regime
• western regimes
• other climes
• Bans, leaks and whistles
• the war on terror

Later pages of this guide consider whistleblowing, media privilege and limits on the power of the state in confessional secrecy.

     regimes

Max Weber's 1922 Economy & Society (Berkeley: Uni of California Press 1979) acerbically commented that

every bureaucracy seeks to increase the superiority of the professionally informed by keeping their knowledge and intentions secret. Bureaucratic administration always tends to be an administration of 'secret sessions': in so far as it can, it hides its knowledge and action from criticism ...

The pure interest of the bureaucracy in power, however,is efficacious far beyond those areas where purely functional interests make for secrecy. The concept of the 'official secret' is the specific invention of the bureaucracy, and nothing is so fanatically defended by the bureaucracy as this attitude ... In facing a parliament the bureaucracy, out of a sure power instinct, fights every attempt of the parliament to gain knowledge by means of its own experts or interest groups. Bureaucracy naturally welcomes a poorly informed and hence a powerless parliament - at least in so far as ignorance somehow agrees with the bureaucracy's interests.

He had elsewhere written that

Bureaucratic administration means fundamentally the exercise of control on the basis of knowledge. This is the feature of it which makes it specifically rational.

This knowledge consists on the one hand of technical knowledge which, by itself, is sufficient to ensure it a position of extraordinary power. But in addition to this, bureaucratic organizations, or the holders of power who make use of them, have the tendency to increase their power still further by the knowledge growing out of experience in the service. For they acquire through the conduct of office a special knowledge of facts and have available a store of documentary material peculiar to themselves. While not peculiar to bureaucratic organizations, the concept of 'official secrets' is certainly typical of them. It stands in relation to technical knowledge in somewhat the same position as commercial secrets do to technological training.

Weber's pessimism concerned government versus parliament but is broadly applicable to the private sector. David Brin's thoughtful The Transparent Society (Reading: Perseus Books 1998) is somewhat idealistic but highlights the notion of reciprocal transparency, ie government and business sharing with citizens the information collected about them: 'they' know a lot about you, you may know very little about 'them'.

For background to openness and restrictions on information access see Sisela Bok's Secrets: On the Ethics of Concealment & Revelation (Oxford: Oxford Uni Press 1985), Russell Stevenson's Corporations & Information: Secrecy, Access & Disclosure (Baltimore: Johns Hopkins Uni Press 1980) and John Baxter's State Security, Privacy & Information (New York: St Martins 1990).

     legislation

     bibliographies

Ralph McCoy's online Freedom of the Press: An Annotated Bibliography is an authoritative and comprehensive guide to several thousand books and articles on freedom of the press. 

Among comparative studies Kenneth Robertson's Public Secrets: A Study In The Development Of Government Secrecy (London: Macmillan 1982) examines the UK, US and Sweden but should be used with caution because of the pace of change. It for example does not include the Ponting and Tisdall cases in the UK or the 1989 UK Official Secrets Act. Administrative Secrecy in Developed Countries (New York: Columbia Uni Press 1979) edited by Donald Rowat is also of value.

     UK

The detailed Espionage & Secrecy: The Official Secrets Act 1911-1989 of the United Kingdom (London: Routledge 1991) by Rosamund Thomas and Secrecy & Power in the British State: A History of the Official Secrets Acts (London: Pluto 1997) by Ann Rogers are studies of the UK experience. 

David Vincent's The Culture of Secrecy: Britain 1832-1998 (Oxford: Oxford Uni Press 2000) is a more nuanced and comprehensive study. Patrick Birkinshaw's Freedom of Information: The Law, the Practice & the Ideal (London: Butterworth 1996) is a definitive study of UK law and practice. There is a more caustic account in Tom Cornford's 2001 paper The Freedom of Information Act 2000: Genuine or Sham?

David Hooper's Official Secrets: The Use & Abuse of the Act (London: Secker & Warburg 1987) is an anecdotal - and entertaining - treatment. Hugo Young's The Crossman Affair (London: Hamilton 1976) retains its status as a major study of changes to UK Cabinet secrecy. Anthony Howard commented

Any back-bench MP is perfectly entitled to record his daily observations and life in parliament and then, if he is lucky enough to find a publisher, to communicate them to a wider public. The problem with Crossman, so far as the Cabinet Office was concerned, arose from his determination to give what Sir John Hunt, the cabinet secretary of the time, described as 'blow-by-blow' accounts of what actually went on within the Wilson cabinet.

The authorities of the period were probably right in regarding this as setting a most disagreeable precedent—a precedent, incidentally, that was soon to be followed by two of his cabinet colleagues, Barbara Castle and Tony Benn (and much later, from within the ranks of the Conservative Party, by the junior minister Alan Clark). Where the guardians of tradition erred was in lacking the nerve to reach for the ultimate Domesday weapon, the then still fully extant Official Secrets Act. Instead, they sought to extend the law of confidentiality (with a rather arcane pedigree reaching back to some Victorian etchings) to cover the content of cabinet discussions.

Judith Cook's The Price Of Freedom (London: NEL 1985) considers application of the British Official Secrets Act to non-defense data. On the Record: Computers, Surveillance & Privacy - The Inside Story (London: Michael Joseph 1986) is another warning by Duncan Campbell & Steve Connor.

     EU

For a perspective on citizen access to EU government information we recommend visiting Statewatch's page tracking implementation of Article 255 of the Amsterdam Treaty to "enshrine" a right of access to documents from the Council of the European Union, the European Commission and the European Parliament.

A broader perspective's provided by Alasdair Davidson's 2001 Supranational Governance & the Right to Information: Experience in the EU (PDF).

     US

Among the extensive literature on US secrecy legislation and policy we recommend Daniel Moynihan's Secrecy: The American Experience (New Haven, Yale Uni Press 1999) and FOI Advocate, an online newsletter covering federal and state developments.

The Torment of Secrecy: The Background & Consequences Of American Security Policies (Chicago: Dee 1996) by sociologist Edward Shils is a classic, complemented by Douglas Stuart's Creating the National Security State: A History of the Law that Transformed America (Princeton: Princeton Uni Press 2008). The Federation of American Scientists 1998 project on Government Secrecy, covered the CIA's pre-publication review process, cold war documentation, declassification policy, freedom of information, secret government spending, and international relations. 

A Culture Of Secrecy: The Government Versus The People's Right To Know (Lawrence: Uni of Kansas Press 1998) is a useful collection of essays edited by Athan Theoharis. Charles Davis & Sigman Splichal edited the broader Access Denied: Freedom of Information in the Information Age (Ames: Iowa State Uni Press 2000). For the 'state secrets' pribilege in modern US law see Claim of Privilege: A Mysterious Plane Crash, a Landmark Supreme Court Case, and the Rise of State Secrets (New York: Harper 2008) by Barry Siegel

The National FOI Coalition (NFOIC) is an alliance of nonprofit state FOI and First Amendment organizations and academic centers.

     Bans, Leaks and Whistles

Liberal democratic states embody a tension between openness and confidentiality, with politicians, advisers and officials recurrently commenting that effective policymaking and administration is dependent on some degree of confidentiality.

Expectations about the extent of that confidentiality differ widely, as do expectations about the shape of restrictions through secrecy law, contract or merely custom. Some states, including the UK, have sought to preserve cabinet solidarity and secrecy through restrictions on the publication of memoirs and letters by serving/past ministers and advisers. Such bans are explored in more detail in a note elsewhere on this site.

UK politician James Callaghan quipped

You know the difference between leaking and briefing. Leaking is what you do and briefing is what I do

An associate less tartly commented that

The Cabinet is at the very centre of national affairs, and must be in possession at all times of information which is secret or confidential. Secrets relating to national security may require to be preserved indefinitely. Secrets relating to new taxation proposals may be of the highest importance until Budget day, but public knowledge thereafter. To leak a Cabinet decision a day or so before it is officially announced is an accepted exercise in public relations, but to identify the Ministers who voted one way or another is objectionable because it undermines the doctrine of joint responsibility

A later page of this guide considers whistleblowing, ie disclosure of information by public/private sector employees in the public interest, despite contract, copyright or secrecy restrictions. Unauthorised release of restricted official information - or strategic leaking - is a feature of recent Western intelligence history, explored in our profile on surveillance and the 'security state'.

In 1986 Mordecai Vanunu provided the London Sunday Times with information about alleged nuclear weapon development activity at Dimona, subsequently being abstracted from Sydney and imprisoned in Israel. An account is provided by Seymour Hersh's The Samson Option: Israel, America & the Bomb (London: Faber 1997).

Disgruntled UK agent Peter Wright divulged information in 1986 through his book Spycatcher, published in Australia despite legal action in Australia and the UK. That fiasco features in Malcolm Turnbull's The Spycatcher Trial (London: Heinemann 1988), Molehunt: Searching for Spies in MI5 (London: Weidenfeld & Nicolson 1987) by Nigel West and A Web of Deception: The Spycatcher Affair (London: Sidgwick & Jackson 1987) by Chapman Pincher. A decade later former MI5 operative David Shayler provided information to the Mail on Sunday in breach of the Official Secrets Act, fled to France and was arrested on his return to the UK in 2000.

     the war on terror

Following the 11 September 2001 incidents a range of governments moved to restrict public access to information, in some instances by summarily removing thousands of documents from official web sites on the basis that the content might be useful in preparing an attack. Such 'sanitisation' of the web - and of libraries and official offline access points - was unprecedented, although similar steps had been taken during the early 1940s and mid 1950s (and most Australian and overseas governments delete pages published by outgoing administrations).

In the US a March 2002 White House memo (later underpinned by the Homeland Security Act (PDF)) ordered federal agencies to "safeguard" information that is "sensitive but unclassified" (aka Sensitive Homeland Security Information or SHSI), a catch-all category encompassing "information that could be misused to harm the security of our nation and the safety of our people", including previously published information about terrorist threats, potential vulnerabilities and disaster response.

Critics have suggested that sanitisation potentially extends to removal of online reports, plans and other material such as -

• Plant Siting and infrastructure planning information
• Chemical toxicity studies (of concern given community interest in understanding risks associated with chemicals and enhanced health protection)
• Accident and transportation safety reports
• photographic or other information from satellites | example
• Contact information about key services (eg fire, police) given its potential value for useful to terrorists
• other policies | example

with associated proposals in the US to establish a new exemption to the Freedom of Information Act for certain "critical infrastructure information".

Genevieve Knezo's 2003 report for the Congressional Research Service on 'Sensitive But Unclassified' and Other Federal Security Controls on Scientific & Technical Information: History & Current Controversy (PDF) offers an outstanding introduction to past US legislation and practice.

In February 2006 US historians reported restoration of classified status of over 55,000 previously declassified pages of documents in the US national archives, a reclassification program itself "shrouded in secrecy" because under a still-classified memorandum the archives is forbidden from even from identifying which agencies are involved. They have noted that many of the withdrawn documents are innocuous; some indeed had been previously published in Foreign Relations of the United States, the State Department's official history series.

In 2008 the Observer revealed disagreement between UK flood risk experts and MI5 over whether to publish inundation maps highlighting areas under threat if any UK dams were to collapse. MI5 reportedly argued that the information could show terrorists where an attack on a dam might have the most impact, dismissing an obligation to publish under the Water Act 2003.

  next page  (secrecy in Australia)