secrecy principles and challenges

This page considers the shape of secrecy and accountability in digital and pre-digital environments. 

It covers -

• introduction
• bureaucratic imperatives
• accountability
• private and public spheres
• information assets
• secrecy in the digital age
• necessary tensions

Later pages of this guide consider whistleblowing, media privilege and limits on the power of the state in confessional secrecy.

     secrets

Max Weber's 1922 Economy & Society (Berkeley: Uni of California Press 1979) acerbically commented that

every bureaucracy seeks to increase the superiority of the professionally informed by keeping their knowledge and intentions secret. Bureaucratic administration always tends to be an administration of 'secret sessions': in so far as it can, it hides its knowledge and action from criticism ...

The pure interest of the bureaucracy in power, however,is efficacious far beyond those areas where purely functional interests make for secrecy. The concept of the 'official secret' is the specific invention of the bureaucracy, and nothing is so fanatically defended by the bureaucracy as this attitude ... In facing a parliament the bureaucracy, out of a sure power instinct, fights every attempt of the parliament to gain knowledge by means of its own experts or interest groups. Bureaucracy naturally welcomes a poorly informed and hence a powerless parliament - at least in so far as ignorance somehow agrees with the bureaucracy's interests.

He had elsewhere written that

Bureaucratic administration means fundamentally the exercise of control on the basis of knowledge. This is the feature of it which makes it specifically rational.

This knowledge consists on the one hand of technical knowledge which, by itself, is sufficient to ensure it a position of extraordinary power. But in addition to this, bureaucratic organizations, or the holders of power who make use of them, have the tendency to increase their power still further by the knowledge growing out of experience in the service. For they acquire through the conduct of office a special knowledge of facts and have available a store of documentary material peculiar to themselves. While not peculiar to bureaucratic organizations, the concept of 'official secrets' is certainly typical of them. It stands in relation to technical knowledge in somewhat the same position as commercial secrets do to technological training.

Weber's pessimism concerned government versus parliament but is broadly applicable to the private sector. David Brin's thoughtful The Transparent Society (Reading: Perseus Books 1998) is somewhat idealistic but highlights the notion of reciprocal transparency, ie government and business sharing with citizens the information collected about them: 'they' know a lot about you, you may know very little about 'them'.

The High Court in Commonwealth v John Fairfax & Sons Ltd & Ors (the first major Australian case seeking prevention of publication of sensitive government documents relating to Australia's national security and foreign policy) commented that

it can scarcely be a relevant detriment to the government that publication of material concerning its actions will merely expose it to public discussion and criticism. It is unacceptable in our democratic society that there should be are restraint on the publication of information relating to government when the only vice of that information is that it enables the public to discuss, review and criticize government action ...

If, however, it appears that disclosure will be inimical to the public interest because national security, relations with foreign countries or the ordinary business of government will be prejudiced, disclosure will be restrained. There will be cases in which the conflicting considerations will be finely balanced, where it is difficult to decide whether the public's interest in knowing and in expressing its opinion, outweighs the need to protect confidentiality.

     UK

The detailed Espionage & Secrecy: The Official Secrets Act 1911-1989 of the United Kingdom (London: Routledge 1991) by Rosamund Thomas and Secrecy & Power in the British State: A History of the Official Secrets Acts (London: Pluto 1997) by Ann Rogers are studies of the UK experience. 

David Vincent's The Culture of Secrecy: Britain 1832-1998 (Oxford: Oxford Uni Press 2000) is a more nuanced and comprehensive study. Patrick Birkinshaw's Freedom of Information: The Law, the Practice & the Ideal (London: Butterworth 1996) is a definitive study of UK law and practice. There is a more caustic account in Tom Cornford's 2001 paper The Freedom of Information Act 2000: Genuine or Sham?

David Hooper's Official Secrets: The Use & Abuse of the Act (London: Secker & Warburg 1987) is an anecdotal - and entertaining - treatment. Hugo Young's The Crossman Affair (London: Hamilton 1976) retains its status as a major study of changes to UK Cabinet secrecy. Anthony Howard commented

Any back-bench MP is perfectly entitled to record his daily observations and life in parliament and then, if he is lucky enough to find a publisher, to communicate them to a wider public. The problem with Crossman, so far as the Cabinet Office was concerned, arose from his determination to give what Sir John Hunt, the cabinet secretary of the time, described as 'blow-by-blow' accounts of what actually went on within the Wilson cabinet.

The authorities of the period were probably right in regarding this as setting a most disagreeable precedent—a precedent, incidentally, that was soon to be followed by two of his cabinet colleagues, Barbara Castle and Tony Benn (and much later, from within the ranks of the Conservative Party, by the junior minister Alan Clark). Where the guardians of tradition erred was in lacking the nerve to reach for the ultimate Domesday weapon, the then still fully extant Official Secrets Act. Instead, they sought to extend the law of confidentiality (with a rather arcane pedigree reaching back to some Victorian etchings) to cover the content of cabinet discussions.

Judith Cook's The Price Of Freedom (London: NEL 1985) considers application of the British Official Secrets Act to non-defense data. On the Record: Computers, Surveillance & Privacy - The Inside Story (London: Michael Joseph 1986) is another warning by Duncan Campbell & Steve Connor.

     EU

For a perspective on citizen access to EU government information we recommend visiting Statewatch's page tracking implementation of Article 255 of the Amsterdam Treaty to "enshrine" a right of access to documents from the Council of the European Union, the European Commission and the European Parliament.

A broader perspective's provided by Alasdair Davidson's 2001 Supranational Governance & the Right to Information: Experience in the EU (PDF).

     US

Among the extensive literature on US secrecy legislation and policy we recommend Daniel Moynihan's Secrecy: The American Experience (New Haven, Yale Uni Press 1999) and FOI Advocate, an online newsletter covering federal and state developments.

The Torment of Secrecy: The Background & Consequences Of American Security Policies (Chicago: Dee 1996) by sociologist Edward Shils is a classic. The Federation of American Scientists 1998 project on Government Secrecy, covered the CIA's pre-publication review process, cold war documentation, declassification policy, freedom of information, secret government spending, and international relations. 

A Culture Of Secrecy: The Government Versus The People's Right To Know (Lawrence: Uni of Kansas Press 1998) is a useful collection of essays edited by Athan Theoharis. Charles Davis & Sigman Splichal edited the broader Access Denied: Freedom of Information in the Information Age (Ames: Iowa State Uni Press 2000).  

The National FOI Coalition (NFOIC) is an alliance of nonprofit state FOI and First Amendment organizations and academic centers.

     and the internet

Nadezhda Mandelstam, memoirist and wife of the great poet, noted that a Soviet official of

exalted rank never commits anything to paper, thus leaving no material evidence ... but only a momentary disturbance of the air which leaves no trace

Issues relating to the longevity of online publications are explored in a more detailed profile elsewhere on this site

     Bans, Leaks and Whistles

Liberal democratic states embody a tension between openness and confidentiality, with politicians, advisers and officials recurrently commenting that effective policymaking and administration is dependent on some degree of confidentiality.

Expectations about the extent of that confidentiality differ widely, as do expectations about the shape of restrictions through secrecy law, contract or merely custom. Some states, including the UK, have sought to preserve cabinet solidarity and secrecy through restrictions on the publication of memoirs and letters by serving/past ministers and advisers.

A later page of this guide considers whistleblowing, ie disclosure of information by public/private sector employees in the public interest, despite contract, copyright or secrecy restrictions. Unauthorised release of restricted official information - or strategic leaking - is a feature of recent Western intelligence history, explored in the profile on surveillance and the 'security state'.

In 1986 Mordecai Vanunu provided the London Sunday Times with information about alleged nuclear weapon development activity at Dimona, subsequently being abstracted from Sydney and imprisoned in Israel. An account is provided by Seymour Hersh's The Samson Option: Israel, America & the Bomb (London: Faber 1997).

Disgruntled UK agent Peter Wright divulged information in 1986 through his book Spycatcher, published in Australia despite legal action in Australia and the UK. That fiasco features in Malcolm Turnbull's The Spycatcher Trial (London: Heinemann 1988), Molehunt: Searching for Spies in MI5 (London: Weidenfeld & Nicolson 1987) by Nigel West and A Web of Deception: The Spycatcher Affair (London: Sidgwick & Jackson 1987) by Chapman Pincher. A decade later former MI5 operative David Shayler provided information to the Mail on Sunday in breach of the Official Secrets Act, fled to France and was arrested on his return to the UK in 2000.

     the war on terror

Following the 11 September 2001 incidents a range of governments moved to restrict public access to information, in some instances by summarily removing thousands of documents from official web sites on the basis that the content might be useful in preparing an attack. Such 'sanitisation' of the web - and of libraries and official offline access points - was unprecedented, although similar steps had been taken during the early 1940s and mid 1950s (and most Australian and overseas governments delete pages published by outgoing administrations).

In the US a March 2002 White House memo (later underpinned by the Homeland Security Act (PDF)) ordered federal agencies to "safeguard" information that is "sensitive but unclassified" (aka Sensitive Homeland Security Information or SHSI), a catch-all category encompassing "information that could be misused to harm the security of our nation and the safety of our people", including previously published information about terrorist threats, potential vulnerabilities and disaster response.

Critics have suggested that sanitisation potentially extends to removal of online reports, plans and other material such as -

• Plant Siting and infrastructure planning information
• Chemical toxicity studies (of concern given community interest in understanding risks associated with chemicals and enhanced health protection)
• Accident and transportation safety reports
• photographic or other information from satellites | example
• Contact information about key services (eg fire, police) given its potential value for useful to terrorists
• other policies | example

with associated proposals in the US to establish a new exemption to the Freedom of Information Act for certain "critical infrastructure information".

Genevieve Knezo's 2003 report for the Congressional Research Service on 'Sensitive But Unclassified' and Other Federal Security Controls on Scientific & Technical Information: History & Current Controversy (PDF) offers an outstanding introduction to past US legislation and practice.

In February 2006 US historians reported restoration of classified status of over 55,000 previously declassified pages of documents in the US national archives, a reclassification program itself "shrouded in secrecy" because under a still-classified memorandum the archives is forbidden from even from identifying which agencies are involved. They have noted that many of the withdrawn documents are innocuous; some indeed had been previously published in Foreign Relations of the United States, the State Department's official history series.

  next page  (studies)