Preserving e-mail – records management perspectives

On July 29 I attended the Digital Preservation Coalition event Preserving e-mail – directions and perspectives . The event brought together records managers, archivists, cultural heritage institutions and digital preservation experts.  For a summary of the event see Chris Prom’s Practical e-records blog (starting here).  In this post I will give some thoughts on the records management perspective at the event

Three approaches to managing e-mail

In the afternoon tea break Stephen Howard   gave me his take on the three different approaches records managers could take to e-mail

  • the message-by-message approach – where users are encouraged to move significant e-mails out of their e-mail client and put them together with other documents arising from the same work (this is the traditional records management approach).
  • the e-mail account by e-mail account approach – where some individuals within the organisation are selected as having particularly important roles, and their entire e-mail account is preserved
  • the whole e-mail system approach – where the organisation treats its entire e-mail system as one aggregation and applies one retention or preservation rule to the entire system

In his current organisation Stephen is thinking of applying the in-box by in-box approach. It would be relatively easy to identify people in key roles whose e-mail was worth preserving. Those individuals could be told of the organisation’s intention to preserve the contents of their e-mail account after they had left. They could be given ways of filtering out personal e-mail so that the personal stuff did not enter the archive.

Earlier in the day Stephen had given a presentation in which he reflected candidly on advice he had given back in 2005 to a local authority he worked for at the time. The head of IT in the authority was concerned about the e-mail servers, and their lack of resilience in the face of mounting volumes of traffic and e-mail storage. They wanted to buy an e-mail archiving tool, to remove stored e-mails from the production e-mail servers.  Stephen at the time advised them not to.

The authority decided against an e-mail archive.  Instead they adopted the intention of implementing an electronic document and management system (EDRMS) to manage records in all formats, including e-mail.   In the meantime they  used an array of methods to encourage colleagues to adopt better e-mail practice.  The authority :

  • asked colleagues to save significant e-mails into shared drive folders
  • put quotas on e-mail in-box sizes to encourage staff to weed out ephemeral e-mails
  • encouraged people to avoid sending attachments where alternatives existed
  • gave advice and training on good use of e-mail

None of these measures did any harm, but the overall approach did not work. Few colleagues saved e-mails into the shared drives. The bottom fell out of the EDRM market and the EDRM never came. Stephen wondered whether the IT manager was right after all – maybe the e-mail archiving tool would have been the least-worse option.

Records management concerns about e-mail archiving tools

Records managers have had philosophical concerns about e-mail archiving tools.

A standard definition of a record is that it consists of all documentation regardless of format needed as evidence of a piece of work.  The idea of treating one set of documentation (e-mail) differently purely because of its format was anathema to us records managers.

There are practical as well as philosophical concerns.  In particular the concern that an e-mail archive operates as a ‘black hole’ .   Such an archive may well have a great search engine, but how could the organisation allow people to use that search engine given the vast amounts of personal information buried in every e-mail account?  The fundamental problem is that a typical e-mail account  makes no differentiation between innocuous e-mails, and e-mails containing sensitive personal information about the e-mail account holder or the people they correspond with.

In practice an organisation could allow:

  • individuals to access e-mails in the archive that were sent to or received by themselves
  • central administrators to search the entire archive for e-mails that fall within the scope of a legitimate e-discovery request, data protection subject access request or Freedom of Information request

But I don’t see how an organisation could allow staff to search across an e-mail archive  on a day to day basis, to answer mundane business questions, because it would then also be possible for them to  search for personal information on particular colleagues.

Yes you could tell staff that if they send or receive e-mails containing sensitive personal information about themselves or third parties then they should delete it from the e-mail archive, or flag it up with an access restriction.   But could you ever be confident enough that this has been acted upon to widen up access to the e-mail archive?

The access permission problem means that  organisations will not want to give up totally on the idea of having e-mails aggregated in some ways, other than simply lumped together in an e-mail archive divided into individual e-mail accounts.  One of the aims of Customer Relations Management (CRM) implementations  is to ensure that e-mails to/from customers are aggregated by customer rather than by the e-mail accounts of the members of staff that sent/received them.  An EDRMS implementation aims at aggregating e-mails and other documentation according to the piece of work that they arose from.   Both approaches offer the advantage that access permissions can be ascribed that fit the nature of those e-mails.

We need to have our cake and eat it.  The advantage of e-mail archiving tools is that they give you the security of  knowing that you have a record of everything that has come in or out.  But what we also need is the ability to apply frameworks that enable those e-mails to be understood, managed and accessed according to different criteria than the name of the individual who sent or received them.

What impact will MoReq 2010 have on the e-mail archiving tool market?

Until recently the records management world treated the existence of multiple applications within an organisation (e-mail clients, line of business applications, CRM systems, an HR system etc.)  as a problem that could best be mitigated by implementing a single electronic records management system and endeavouring to get all documents or e-mails needed as records saved into it.

The recent MoReq2010 specification takes a different approach. It attempts to boil down a core set of records management requirements with a view to making it feasible for any and every business application to have enough records management functionality to manage its own records, and to export those records and accompanying metadata, rules and classifications at the end of the useful life of the application.   This is hugely ambitious, as line of business application developers and vendors rarely take notice of records management specifications.

The first output of  MoReq2010 – the Core services and Plug-in Modules published in June of this year, does not specifically mention e-mail.   This is because the Core services cover only that minimum set of requirements that every records system  should possess, and it is possible to envisage a records systems that is not intended to hold e-mails.  But an extension module of MoReq2010 is planned, to specifically outline MoReq 2010 records management requirements for e-mail.

It will be interesting to see what effect that MoReq2010 e-mail module, when it appears, will have  on the e-mail archiving tool market.  A MoReq 2010 compliant e-mail archiving system would be an interesting proposition for records managers – I wonder if any of the big players in the market will rise to the challenge.

This 2009 Gartner magic quadrant report on e-mail active archiving tools shows that many such tools are branching out from simply archiving e-mails and now claim to be able to archive and manage material in shared drives and in SharePoint sites.  All the more reason for such products to go for MoReq 2010 certification.

Whether they do go for it depends in part upon their willingness to re-architect the way their systems maintain metadata and event histories.  MoReq 2010 is much more prescriptive on these fronts than previous standards, and established players with set architectures may be reluctant to change.

Advertisements

3 thoughts on “Preserving e-mail – records management perspectives

  1. James – great article as always. I would say us recrods managers have more than just philosophical concerens about email management and archiving. In all the discussions about managing in place, search tools etc business continuity seems to be ignored ie if I’ve been working on something major and haven’t bothered to save the business critical emails into the appropriate shared area (which takes – let’s face it – about 10 seconds to do if you do it immediately, so why all the fuss about doing it?) and then swan off on holiday or break a leg no-one in my team will be able to locate those vital emails unless there’s a crisis of such major propertions that the compliance team is allowed to search my account. But even then they may not know what to search for – or my team may not even know that the crucial information exists in the first place. So off everyone goes on expensive and time-consuming tangents, reinventing the wheel, losing the court case, getting fined by the ICO, running yet another project etc. In the name of efficiency, isn’t it time we stopped pandering to what the “user” (they aren’t users, they are employees)wants to do and telling them that, as employees of our organisation, they must abide by our best practice rules and manage emails properly. And tell them to get on with it! Clare

    Clare

  2. Very interesting reflections James. I recently saw the trend towards “every business application to have enough records management functionality to manage its own records”, exemplified by the new MoReq standard, being enacted in practice, at the IRMS site visit to Dyson’s R&D headquarters.
    I’ve written about it in more detail on my blog (http://thelibrarycareer.blogspot.com/2011/08/event-irms-site-visit-to-dyson-r-centre.html), but in summary they rejected the option of an EDRM (despite having a £4m budget available to implement one) and instead are building RM functionality into their line of business systems.
    Clare – there was certainly also an element there of ’employees not users’ and ‘just get on with following the procedures’ there as well! Perhaps it helps them that their users are largely engineers and programmers, who are used to structured procedures?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s