The story that the Financial Times  and other newspapers ran last week about the Cabinet Office 90 day email deletion practice is not new. It was reported on 18 December 2004 by the BBC, two days before the deletions were about to start. Here is the first part of their report 
I tweeted a link to the BBC report on Saturday . I received this tweet in reply
Imagine yourself back on the 18 December 2004, a fortnight before the UK’s Freedom of Information Act comes into force. It comes to light that the Cabinet Office, at the very heart of Government, is planning to have all emails that are over 90 days old deleted from their email servers. In other words they are deleting about ten years worth of correspondence – a portion of which will have been captured elsewhere, but much of which will only have existed on those email servers. The opposition leader has raised objections. Why did archivists and records managers not raise this as an issue?
There are four main reasons why archivists/records managers did not object at the time:
This is an important case with significant implications for records management and archival practice. We might usefully debate the following questions:
 Pickard, Jim and Stacey, Kiran 16 June , 2015 8:17 pm Freedom of information is Mission Impossible for Downing St emails. Financial Times, , available from http://on.ft.com/1QCbS8a (it requires a log-in). Accessed 17 June 2015 OR see see Morris, Nigel 17 June 2015. Government faces call to review self destruct email policy http://www.independent.co.uk/news/uk/politics/government-faces-call-to-review-selfdestruct-email-policy-10327202.html (accessed 20 June 2015).
 BBC News, 18 December 2004, 14:56 GMT. Howard condemns email deletion. Available at http://news.bbc.co.uk/1/hi/uk_politics/4107563.stm (accessed 20 June 2015).
The Financial Times  reported this week that the Cabinet Office auto-deletes their e-mails after 90 days.
In the article several officials and aides are quoted describing day to day problems that Cabinet Office staff have experienced in the absence of an adequate correspondence record:
The Cabinet Office’s 90 day deletion policy was introduced in 2004. The Freedom of Information Act for England and Wales came into force on the 1 January 2005. The article quotes Maurice Frankel of the Campaign for Freedom of Information as stating that the proximity of those two dates ‘was not a co-incidence’. However the article also reports:
An official said the system had been recommended by the National Archives for ‘best records management’ and it was a coincidence that it began at the same time as Freedom of Information
On the one hand I do not believe that the Cabinet Office’s 90 day deletion policy was introduced in 2004 purely as a conspiracy:
On the other hand :
Experience has taught us that asking users to declare important emails as records into any sort of record system does not result in the capture of an adequate correspondence record
The experience we have had in the UK, in continental Europe, in Australia and in the US is that asking colleagues to choose which of their e-mails are significant, declare them as a record, and move them to some sort of electronic file (or paper file, or SharePoint document library) does not capture an even vaguely adequate record of correspondence.
It is interesting to read the quotes of the special advisors/aides/officials in the FT article. One presumes that some emails have been captured into the Cabinet Office’s ERM system (for nearly a decade they had a system provided by Meridio, but have recently replaced it with Google Docs). And yet the officials quoted by the FT talk as though there was no correspondence record at all. My guess is that this is because individuals saved so little correspondence into the system, and saved into the system so selectively, that to all intents and purposes there was and is no Cabinet correspondence record outside of e-mail accounts.
This is only a guess – we would need to know the figures. The most relevant figures being:
We do have these figures for the US State Department, thanks to a report  published by the Office of Inspector General in March 2015. The report states that in 2009 the State Department implemented a record system that was built into the Outlook email client, whereby individuals could declare important emails as records. This was backed up by a policy instruction that individuals were responsible for preserving important emails as records, and by training.
The Office of Inspector General’s report sates that in 2011 the State Department sent 1 billion emails, but only 61,556 were declared as records. Robert Smallwood  calculates that the emails captured into the State Department’s record system constitute only .0061% of their total email communications. In 2013 the situation was even worse – only 41,749 State Department emails were declared as records. Interestingly the report states that one of the reasons for the lack of the declaration of emails as records was a fear that they would show up in searches (which I am interpreting to mean that they would show up in searches conducted in response to Freedom of Information requests).
If the Cabinet Office no longer knows how many emails it received in a given year then the headcount of the Cabinet Office in that year would be the next best figure. Chris Prom  quotes statistics from Radicati  stating that on average a business user sends 33 emails a day.
I recently spoke to an organisation who had a reasonably good electronic records management system, no fear of Freedom of Information, and a strong commitment to record keeping. They told me that the number of items contributed to their records system equated to one item per member of staff every two days. If the figures Prom quotes are correct that would give a ration of around 1 email saved to their records system for every emails 66 sent. That is several orders of magnitutude better than the State Department’s ratio, but for an important organisation that ratio is still nowhere near high enough to function as a useable and defensible correspondence record.
Most UK government bodies operate routine deletion policies, albeit significantly less drastic than the Cabinet Office’s 90 day deletion. Rather than a 90 day deletion it is more common for the deletion to occur 1 or 2 years after the email is sent, or six months to a year after the individual email account holder has left employment. Alternatively or additionally some departments operate a limit on email box size rather than an automatic deletion.
Individuals in such departments do not experience the inconveniences reported by Cabinet Office staff. They are cushioned from the impact of the deletions, but the net result is the same. The correspondence record being kept is inadequate and cannot support basic succession planning/staff handover in the short term, corporate memory in the medium term and historical accountability in the long term.
 Pickard, Jim and Stacey, Kiran 2015. Freedom of information is Mission Impossible for Downing St emails. Financial Times, June 16, 2015 8:17 pm, available from http://on.ft.com/1QCbS8a (it requires a log-in). Accessed 17 June 2015
 The National Archives (TNA) (date unknown) Managing emails. Available from http://www.nationalarchives.gov.uk/information-management/manage-information/policy-process/managing-email/ (Accessed 18 June 2015). TNA’s policy advice to Government departments is that
”You will need to…..limit what users can keep in personal email accounts by the use of:
– email account quotas OR
– automatic deletion after a set period of time”
 Gibbons Paul (FOI man) 17 June 2015. FOI avoidance or good records management? Cabinet Office Email Policy. Available from http://www.foiman.com/archives/1584 accessed 18 June 2015
 Office of Inspector General, March 2015. Review of State Messaging and Archive Retrieval Toolset and Record Email (ISP-I-15-15) Available from https://oig.state.gov/reports/inspection?page=1 or access the pdf directly from https://oig.state.gov/system/files/isp-i-15-15.pdf
 Smallwood, R, 2015. Clinton Email Kerfuffle: Call for Cabinet-level Information Governance. Available at https://www.linkedin.com/pulse/clinton-email-kerfuffle-call-cabinet-level-governance-smallwood(accessed 17 June 2015)
 Prom, Christopher J, 2011. Preserving email (Digital Preservation Coalition Technology Watch Report) pdf available from http://www.dpconline.org/newsroom/latest-news/805-email-tomorrow-and-next-year-and-forever-preserving-email-report-published On page three of the report Prom writes ‘Over 3.1 billion email accounts currently exist, and the average business user sends 33 email messages per day (Radicati Group, Inc. 2011a). ‘
 Radicati Group, Inc., 2011a. Email Statistics Report, 2011–2015, Executive Summary. Available at: http://www.radicati.com/?p=7261 [Accessed July 17, 2011]. (Quoted by Prom)
I attended the Information Governance and eDiscovery summit in London last week.
Two friends, both stalwarts of the records management world in the UK, separately came up to me and said ‘you know what I think Records management is dead now James’
Records management isn’t dead. It is a contestable space at the heart of information governance.
In the e-Discovery stream of the summit there was no mention of any type of document management or records system (whether that be an electronic records management system, or SharePoint). This is because these systems are often not relevant to the preparation of lawsuits, where the lawyers go straight to the e-mail record and/or text messages to establish the chain of events and discussions pertinent to the case.
In effect organisations are operating two parallel spheres of recordkeeping.
The key difference between the correspondence/communication sphere and the document sphere is that:
The paradoxical relationship between the two spheres of recordkeeping
This split between e-mail servers/archives on the one hand and corporate ERM systems/SharePoint implementations on the other creates the following paradoxes:
The task of information governance
Information governance is often described as an umbrella term covering the separate disciplines of eDiscovery, records management, access to information and privacy.
The inclusion of both eDiscovery and records management under this umbrella means that information governance spans both:
One of two things could happen here.
One possibility is that each of the component disciplines of information governance stay entirely separate. Records management carries on asking people to move important correspondence to an ERM/SharePoint or whatever tool they designate as their main repository. Lawyers perfect their eDiscovery/analytics tools which allow them (and only them) to search across the email server/archive and across all repositories in the organisation.
The other possibility is that information governance acts to influence these disciplines, to create synergies and to enable them to become more than the sum of their parts. To an extent this is happening already as analytics tools honed on eDiscovery cases cross over into the records management space to enable organisations to apply disposition decisions to hitherto intractable repositories such as shared drives.
Information Governance should work towards becoming the arena in which the tensions and contradictions between accountability, risk, and privacy can be resolved or managed in relation to both the main correspondence system (which at the time of writing happens to be e-mail) and the main document management applications and repositories.
The task of records management
I would like to see records management use information governance to blur the boundaries between the systems they designate as records systems, and the other repositories in the organisation including, but not limited to, e-mail servers/archives. Lawyers use their eDiscovery tools to search across all repositories in the organisations. As records managers we should be seeking to stretch their retention remit to cover all repositories. We should be seeking to establish relationships between content held in whatever we designate as our main records repository and related content held on shared drives, in e-mail accounts etc.
Cutting through the e-mail paradoxes
Organisations would like to be in a position where they can dispense with the complete record of e-mail communications after as small a time interval as possible, and rely instead on the filtered, lower risk records that exists in their document management sphere. However the swift deletion of e-mails from their email server/archive adversely impacts their ability to account for their actions.
The irony is that external stakeholders/litigants/regulators/hostile FOI requesters are not interested in trivial e-mails, and they are not interested in personal e-mails. External stakeholders would be happy for organisations to delete or redact them.
The problem is that organisations cannot currently defend or explain how they got from the complete correspondence record on the e-mail server/archive, to the filtered record on a corporate electronic records management system/SharePoint/shared drive.
Records management should work towards providing organisations with a defensible consistent, routine and transparent way of distinguishing trivial, sensitive and personal e-mails from typical business correspondence, with a view to enabling organisations to have a record of each individual’s business correspondence that is both filtered and comprehensive; defensible and accessible; to which defensible retention rules can be applied, and which can be linked to related collections of documents and communications held in other applications/repositories, including the designated records repository (ERM/SharePoint or similar).
NARA (2015), GRS 6.1 Email Managed Under a Capstone Approach http://blogs.archives.gov/records-express/files/2015/04/FINAL-GRS-6.1-Review-Package-FR-Posting-03.30.15.pdf (accessed 19 May 2015)
Ravanbakhsh, Arian (2015) DRAFT Capstone GRS Available http://blogs.archives.gov/records-express/2015/04/02/draft-capstone-grs-available/ (accesssed 20 May 2015)
The view that a legal team takes on how long emails/ email accounts should be retained varies from organisation to organisation. Here are two examples I have come across in the past couple of years:
So which legal team is correct? And should records managers be pressing for short or long retention periods on e-mail?
From a lawyer’s point of view it boils down to a simple equation:
Ralph Losey publishes the e-Discovery team blog. He is over in London this week for the Information Governance and e-Discovery summit. He is Chair of the Electronic Discovery Practice Group for the US law firm Jackson Lewis P.C.
Jackson Lewis is a firm that specialises in employment law. Ralph is unequivocal that he wants his clients to retain their e-mail. He told me that most of the employment claims his company defends against are spurious, and that it is easier for him to prove that the claim is spurious if his client has retained their e-mail correspondence from the period in question.
Ralph pointed out that most employees in most companies act legally. The e-mail record is going to defend them more often than it will incriminate them.
This is what he said about email deletion in this recent blogpost (Losey, 2015)
My understanding and experiences with Big Data analytics over the last few years have led me to understand that more data can mean more intelligence, that it does not necessarily mean more trouble and expense. I understand that more and bigger data has its own unique values, so long as it can be analyzed and searched effectively.
This change of position was reinforced by my observing many litigated cases where companies no longer had the documents they needed to prove their case. The documents had short retention spans. They had all been destroyed in the normal course of business before litigation was ever anticipated. I have seen first hand that yesterday’s trash can be tomorrow’s treasure. I will not even go into the other kind of problems that very short retention policies can place upon a company to immediately implement a lit-hold. The time pressures to get a hold in place can be enormous and thus errors become more likely.
There is a definite dark side to data destruction that many do not like to face. No one knows for sure when data has lost its value. The meaningless email of yesterday about lunch at a certain restaurant could well have a surprise value in the future. For instance, a time-line of what happened when, and to whom, is sometimes an important issue in litigation. These stupid lunch emails could help prove where a witness was and when. They could show that a witness was at lunch, out of the office, and not at a meeting as someone else alleges.
For what its worth I think we as records managers should press for e-mails to be kept for as long as we would keep the correspondence of the individual role holder if they kept a correspondence file.
Losey, 2005, Information Governance v Search: The Battle Lines Are Redrawn, e-Discovery team blog, http://e-discoveryteam.com/2015/02/08/information-governance-v-search-the-battle-lines-are-redrawn/ , 8 February 2005 [accessed 13 May 2015]
The main differences between the e-mail policy of the US National Archives (NARA) and those of the national archives of Australia, Canada and the UK, are that:
This is an important development. Not because it solves the challenge of e-mail. It doesn’t. E-mail accounts are still hard to manage because of the undifferentiated presence of sensitive personal data about the account holder and/or the people they correspondence with and/or third parties.
It is important because it gives a green light for archivists and records managers to explore ways of managing e-mail correspondence within an e-mail environment.
Over the past fifteen years the main ambition of records management practice has been to move significant e-mails out of the e-mail environment (typically Microsoft Outlook/Exchange) into a separate ‘record system’ ( paper files/shared drives/electronic records management systems/SharePoint etc.).
The problem with this approach is that an e-mail environment is optimised for people to navigate, browse, sort and read e-mail. In contrast a document environment such as SharePoint is not optimised for e-mail.
SharePoint is a system that is designed to be so flexible that an organisation could, if it so wished, define a different set of metadata columns for every document library in every different SharePoint site. E-mail on the other hand has a fixed set of metadata fields that are common to every e-mail.
We are accustomed to browsing e-mail in a completely different way to browsing documents. We browse e-mails by date, sender or recipient. We browse documents by activity or subject.
Basing the practice of an entire profession (records management) on moving content (e-mail) from an environment that is optimised for it to an environment that is not optimised for it is not a recipe for long term success.
As a profession and as individual practitioners we cannot change this approach overnight. But we can start to explore what policy provisions we would need, and what alterations/additions to e-mail environments and their ecosytsems we would need, in order for e-mails to be manageable over time, and shareable over time, within an e-mail environment.
See this previous post for extracts from the email policies of the four national archives mentioned in this post
This blogpost consists of extracts from the advice on managing e-mail that the national archives of Australia, the UK, Canada and the US currently provide to the agencies/departments/ministries of their respective governments.
I am posting this simply to record the difference in policy between the national archives of the US (NARA) on the one hand and those of the UK, Canada and Australia on the other. My next blogpost will give some background to the difference and on the dilemma that national archives face in relation to e-mail.
The four extracts were taken from policies accessed from the websites of the respective archives on 30 April 2015.
The following extract is taken from ‘Managing e-mail‘ available from the National Archives of Australia website. The navigation pathway is Home/ Records management /Managing your agency records/ Digital records/ Managing email
”You should store business email in a system that can manage it effectively for as long as it is needed. This could be an electronic document and records management system (EDRMS), a case management system or another suitable business system.
If your organisation doesn’t have a more suitable system, it’s better to store your business email in a network or shared drive system than leave it in an email system. Your information will at least be available to other staff to use and it can be stored in context. However, information in shared drives can be altered or deleted without authorisation so this should only be a temporary solution.”
Emails are an important part of the corporate record for all organisations. For public sector bodies they are public records and are subject to the Public Records Act, the Data Protection Act and the Freedom of Information Act. Therefore they need to be managed in a way that meets legislative requirements.
All civil servants have an obligation to keep accurate official records under the Civil Service Code.You will need to:
define clearly which emails need to be kept for business or historical value
communicate simply and often to users the rules for what emails to keep
keep emails with related digital information in a shared corporate information management system
limit what users can keep in personal email accounts by the use of: – email account quotas OR – automatic deletion after a set period of time
The following extract is taken from ‘Email Management Guidelines Roadmap‘ from The National Archives of Canada website. The navigation path is Home/ Services and Program/Managing Government of Canada Information Resource/ Guidelines/ Email Management
3.1. Maintain and use an organized and efficient filing system for email
Email messages, other than non-records or transitory messages, should be moved from the email system to a separate filing system where they should be organized as specified in the classification structure approved by the institution. Messages should be indexed and kept for institution use until their scheduled disposal or until their transfer to archival storage. Archival storage should also be organized and indexed for efficient retrieval.
Canadian courts have followed a well-recognized approach which holds that a document filing system that belongs to a party involved in litigation should be organized and labelled or indexed in such a manner as to facilitate use by the other party.
It can be extremely time-consuming to locate relevant evidence, and a party to litigation might argue that the cost of producing records for discovery is too great. Courts will weigh the cost of discovery against the potential benefits of having the evidence. However, Courts have not been sympathetic when it has been determined that a major portion of the cost is due to the fact that the party’s filing system is poorly organized.
It is important not to underestimate the cost of discovery. On a major case, involving a large institution, the cost can run into the hundreds of thousands, indeed into the millions of dollars. In a number of jurisdictions, some defendants have settled out of court, not as an admission of guilt, but because settlement was less expensive than assembling the evidence required for a defence.
The following extract is taken from NARA Bulletin 2014-06 Guidance on Managing Email
4. What is the role of Federal employees in email management?
Currently, in many agencies, employees manage their own email accounts and apply their own understanding of Federal records management. This means that all employees are required to review each message, identify its value, and either delete it or move it to a recordkeeping system. Some email, such as spam or all-staff announcements, may be deleted immediately. On the other hand, substantive policy discussions conducted in email may be appropriate for preservation for several years or ultimate transfer to NARA.
NARA recognizes that placing the responsibility on employees to make decisions on an email-by-email basis can create a tremendous burden. As a result, NARA recommends that agencies immediately begin to adopt automated or rules-based records management policies for email management, such as the Capstone approach.
5. What is Capstone?
Capstone is an approach to managing email. It is not a type of technology. (See NARA Bulletin 2013-02: Guidance on a New Approach to Managing Email Records.) When adopting the Capstone approach, agencies must identify those email accounts most likely to contain records that should be preserved as permanent. Agencies will determine Capstone accounts based on their business needs. They should identify the accounts of individuals who, by virtue of their work, office, or position, are likely to create or receive permanently valuable Federal records. Capstone officials will generally be the top-level senior officials of an agency, but may also be other key decision makers at lower levels of the agency.
Following this approach, an agency can schedule all of the email in Capstone accounts as permanent records. The agency could then schedule the remaining (non-Capstone)email as temporary and retain all of them for a set period of time based on the agency’s needs. The Capstone Bulletin addresses additional options and best practices.