Solutions to the email problem

Analyticsandemail01Analyticsandemail02

When I started telling people, back in 2015,  that I had started a doctoral research project on email, my records management colleagues would typically smile, pat me on the shoulder and say  ‘great! let us know when you have found the solution’.

Two years later.  There is still no solution.  Solutions do not drop out of the sky.  Solutions emerge from people trying to fix things.  If there is a broken situation and noone tries to fix it, it stays broken.

You do not have to undertake a doctoral research project to work out that organisations are not consistently capturing business email into systems that they designate as ‘records systems’.

Yes there are potential approaches.  Not just one,  many different approaches, suitable in different circumstances, using different technologies or different ways of harnessing human input:

  • you could do what some CRM systems do and match the email addresses of external contacts with locations in your record system (when colleague sends an email to person/organisation  it is about business matter x, when they email person/organisation c it is about business matter y etc.);
  • you could set up one container in your record system to mirror each email account that you provision.  You could give each email account owner a simple means of indicating for every email they send, whether or not they want that email and the thread it is part of to be saved as a record to the mirror container for their account in the record system;
  • you could use analytics just before you run your routine deletion routine on an email account (after someone has left, or after two or three years) to rescue material which is obviously business related (and obviously not personal) from routine deletion.

We could go on and on with this list, you will have ideas of your own. We have not even mentioned machine learning yet.

But a solution for a problem is like a hammer waiting for a nail unless the problem owner wants the problem to be solved.

The problem owner for the email ‘problem’ is the organisation who owns the email accounts and the emails.

The one thing all of the above solutions have in common is that they all result in more emails being captured into record systems.   All of these solutions therefore add to storage costs,  they also add to the cost of servicing access to information requests, and, in highly politicised environments such as many central government departments, they add to the perceived risk of  being forced to disclose potentially embarrassing content.

Email is therefore a problem that the problem owner percieves a benefit from leaving unsolved.  Every conceivable solution to the problem of treating business emails as records would be more expensive, and would be perceived by the problem owner as being more risky,  than the current approach.   The current approach, if we are honest, is for organisations to make a token effort to persuade staff to save important emails as reords, who in turn make a token effort to occasionally place a few emails into a record system, whilst all the rest of the emails are routinely deleted after a designated but essentially arbitrary time period.

Where does this leave archivists and records managers?  Where does it leave records management and where does it leave record keeping?

We are also in a manner of speaking the problem owner.  The problem falls into our professional domain.

We are employed by organisations.   It is not our job to try to pursuade our organisations to do anything that is against their perceived interests (and we would not get very far if we did try).

On one hand we are in a good situation.  Organisations put us under no pressure to change our records management policy telling staff to move emails into the record system, and under no pressure to change the relationship between the email environment and our record system (which is typically a product primarily designed and configured for the capture and management of documents rather than emails).

On the other hand it puts us in a very difficult position. We go to all that trouble to develop corporate wide record systems and corporate wide records retention schedules (both labours of Hercules, demanding all of our professional skills, experience and energy) and yet we are regarded as failures because those systems and those retention schedules fail to embrace the bulk of an organisation’s business correspondence.

It is a problem that in effect we can do little about, because our organisations do not want to do anything about it.  Take any of the ideas listed above to your organisation and you will see what I mean.  Our vendor community cannot help us because our organisations do not want solutions that would result in them keeping significant volumes of email for significantly longer periods of time.

In the short term we can carry on like this.  Our organisations are happy for us to continue to try and continue to fail.  Because (and this is the paradox) our failure to capture business correspondence consistently into systems treated as record systems succeeds in reducing the cost and perceived risk of recordkeeping to our organisations.

But what about the long term?  I fear in the long term that this success-through-failure damages our professional reputation and damages the clarity and moral force of our theory and of our practice.

 

How the Cabinet Office’s 90 day email deletion was reported back in 2004

The story that the Financial Times [1] and other newspapers ran last week about the Cabinet Office 90 day email deletion practice is not new.   It was reported on 18 December 2004 by the BBC, two days before the deletions were about to start.   Here is the first part of their report [2]

Screen Shot 2015-06-20 at 13.51.04

I tweeted a link to the BBC report on Saturday .   I received this tweet in reply

Screen Shot 2015-06-20 at 14.10.02

Good question!

Imagine yourself back on the 18 December 2004, a fortnight before the UK’s Freedom of Information Act comes into force. It comes to light that the Cabinet Office, at the very heart of Government, is planning to have all emails that are over 90 days old deleted from their email servers.    In other words they are deleting about ten years worth of correspondence – a portion of which will have been captured elsewhere, but much of which will only have existed on those email servers.   The opposition leader has raised objections.   Why did archivists and records managers not raise this as an issue?

There are four main reasons why archivists/records managers did not object at the time:

  • We had spent the previous ten years warning our organisations about the inconveniences and dangers of records building up in individual email accounts.  We therefore were not minded to defend those records that had built up in Cabinet Office email accounts
  • We did not know what to do with email accounts.  We did not know how we would manage them over time,  how we would deal with the personal data within them, how we would sensitivity review them, what retention rules we could apply to them, nor how we would appraise them as being worthy or unworthy of permanent preservation.  We could not envisage how or when we could make available to the public an email account selected for permanent preservation.
  • Records management in the UK  had  a great year in 2004,  mainly thanks to the Blair government.  During 2004 central  government threw money at electronic records management systems, created lots of new records management posts, employed lots of records management consultants.  It was exciting.  We were not in a mood to question things too closely.
  • On December 18 2004 we were all so focused on the coming of FOI (and perhaps the coming of the Christmas holiday) that we didn’t notice.   I don’t remember even seeing this article, I can’t remember anyone mentioning it to me.

This is an important case with significant implications for records management and archival practice.   We might usefully debate the following questions:

  • Was the mass deletion of  Cabinet Office emails from email servers in December 2004 carried out for political advantage, administrative expediency or for recordkeeping improvement?   Has the Cabinet Office continued that  90 day auto-deletion policy under successive Labour, Coalition and Conservative administrations for political, administrative or recordkeeping reasons?
  • How serious is the impact of these deletions on the historical record?  Is the important correspondence largely captured elsewhere? Or is this 90 day auto-deletion of email by one of the most important Whitehall Departments going to create a significant and irreplaceable gap in our nation’s historical record?
  • Are we as a profession – records managers and archivists – any better equipped to manage email accounts over time now than we were back in 2004?  Is there a feasible alternative to auto-deletion?
  • Should the UK National Archives follow the example of its counterpart in the US and step in to prevent the auto-deletion of significant email accounts by declaring that it requires UK Government departments to select the email accounts of  important civil servants for permanent preservation?

References

[1] Pickard, Jim and Stacey, Kiran  16 June , 2015 8:17 pm  Freedom of information is Mission Impossible for Downing St emails.   Financial Times, ,   available from http://on.ft.com/1QCbS8a (it requires a log-in).  Accessed 17 June 2015    OR see see Morris, Nigel  17 June 2015.  Government faces call to review self destruct email policy http://www.independent.co.uk/news/uk/politics/government-faces-call-to-review-selfdestruct-email-policy-10327202.html (accessed 20 June 2015).

[2]  BBC News, 18 December 2004, 14:56 GMT.   Howard condemns email deletion. Available at http://news.bbc.co.uk/1/hi/uk_politics/4107563.stm  (accessed 20 June 2015).