On July 29 I attended the Digital Preservation Coalition event Preserving e-mail – directions and perspectives . The event brought together records managers, archivists, cultural heritage institutions and digital preservation experts. For a summary of the event see Chris Prom’s Practical e-records blog (starting here). In this post I will give some thoughts on the records management perspective at the event
Three approaches to managing e-mail
In the afternoon tea break Stephen Howard gave me his take on the three different approaches records managers could take to e-mail
- the message-by-message approach – where users are encouraged to move significant e-mails out of their e-mail client and put them together with other documents arising from the same work (this is the traditional records management approach).
- the e-mail account by e-mail account approach – where some individuals within the organisation are selected as having particularly important roles, and their entire e-mail account is preserved
- the whole e-mail system approach – where the organisation treats its entire e-mail system as one aggregation and applies one retention or preservation rule to the entire system
In his current organisation Stephen is thinking of applying the in-box by in-box approach. It would be relatively easy to identify people in key roles whose e-mail was worth preserving. Those individuals could be told of the organisation’s intention to preserve the contents of their e-mail account after they had left. They could be given ways of filtering out personal e-mail so that the personal stuff did not enter the archive.
Earlier in the day Stephen had given a presentation in which he reflected candidly on advice he had given back in 2005 to a local authority he worked for at the time. The head of IT in the authority was concerned about the e-mail servers, and their lack of resilience in the face of mounting volumes of traffic and e-mail storage. They wanted to buy an e-mail archiving tool, to remove stored e-mails from the production e-mail servers. Stephen at the time advised them not to.
The authority decided against an e-mail archive. Instead they adopted the intention of implementing an electronic document and management system (EDRMS) to manage records in all formats, including e-mail. In the meantime they used an array of methods to encourage colleagues to adopt better e-mail practice. The authority :
- asked colleagues to save significant e-mails into shared drive folders
- put quotas on e-mail in-box sizes to encourage staff to weed out ephemeral e-mails
- encouraged people to avoid sending attachments where alternatives existed
- gave advice and training on good use of e-mail
None of these measures did any harm, but the overall approach did not work. Few colleagues saved e-mails into the shared drives. The bottom fell out of the EDRM market and the EDRM never came. Stephen wondered whether the IT manager was right after all – maybe the e-mail archiving tool would have been the least-worse option.
Records management concerns about e-mail archiving tools
Records managers have had philosophical concerns about e-mail archiving tools.
A standard definition of a record is that it consists of all documentation regardless of format needed as evidence of a piece of work. The idea of treating one set of documentation (e-mail) differently purely because of its format was anathema to us records managers.
There are practical as well as philosophical concerns. In particular the concern that an e-mail archive operates as a ‘black hole’ . Such an archive may well have a great search engine, but how could the organisation allow people to use that search engine given the vast amounts of personal information buried in every e-mail account? The fundamental problem is that a typical e-mail account makes no differentiation between innocuous e-mails, and e-mails containing sensitive personal information about the e-mail account holder or the people they correspond with.
In practice an organisation could allow:
- individuals to access e-mails in the archive that were sent to or received by themselves
- central administrators to search the entire archive for e-mails that fall within the scope of a legitimate e-discovery request, data protection subject access request or Freedom of Information request
But I don’t see how an organisation could allow staff to search across an e-mail archive on a day to day basis, to answer mundane business questions, because it would then also be possible for them to search for personal information on particular colleagues.
Yes you could tell staff that if they send or receive e-mails containing sensitive personal information about themselves or third parties then they should delete it from the e-mail archive, or flag it up with an access restriction. But could you ever be confident enough that this has been acted upon to widen up access to the e-mail archive?
The access permission problem means that organisations will not want to give up totally on the idea of having e-mails aggregated in some ways, other than simply lumped together in an e-mail archive divided into individual e-mail accounts. One of the aims of Customer Relations Management (CRM) implementations is to ensure that e-mails to/from customers are aggregated by customer rather than by the e-mail accounts of the members of staff that sent/received them. An EDRMS implementation aims at aggregating e-mails and other documentation according to the piece of work that they arose from. Both approaches offer the advantage that access permissions can be ascribed that fit the nature of those e-mails.
We need to have our cake and eat it. The advantage of e-mail archiving tools is that they give you the security of knowing that you have a record of everything that has come in or out. But what we also need is the ability to apply frameworks that enable those e-mails to be understood, managed and accessed according to different criteria than the name of the individual who sent or received them.
What impact will MoReq 2010 have on the e-mail archiving tool market?
Until recently the records management world treated the existence of multiple applications within an organisation (e-mail clients, line of business applications, CRM systems, an HR system etc.) as a problem that could best be mitigated by implementing a single electronic records management system and endeavouring to get all documents or e-mails needed as records saved into it.
The recent MoReq2010 specification takes a different approach. It attempts to boil down a core set of records management requirements with a view to making it feasible for any and every business application to have enough records management functionality to manage its own records, and to export those records and accompanying metadata, rules and classifications at the end of the useful life of the application. This is hugely ambitious, as line of business application developers and vendors rarely take notice of records management specifications.
The first output of MoReq2010 – the Core services and Plug-in Modules published in June of this year, does not specifically mention e-mail. This is because the Core services cover only that minimum set of requirements that every records system should possess, and it is possible to envisage a records systems that is not intended to hold e-mails. But an extension module of MoReq2010 is planned, to specifically outline MoReq 2010 records management requirements for e-mail.
It will be interesting to see what effect that MoReq2010 e-mail module, when it appears, will have on the e-mail archiving tool market. A MoReq 2010 compliant e-mail archiving system would be an interesting proposition for records managers – I wonder if any of the big players in the market will rise to the challenge.
This 2009 Gartner magic quadrant report on e-mail active archiving tools shows that many such tools are branching out from simply archiving e-mails and now claim to be able to archive and manage material in shared drives and in SharePoint sites. All the more reason for such products to go for MoReq 2010 certification.
Whether they do go for it depends in part upon their willingness to re-architect the way their systems maintain metadata and event histories. MoReq 2010 is much more prescriptive on these fronts than previous standards, and established players with set architectures may be reluctant to change.