NARA seeks industry help to automate records management

On the morning of Tuesday September 10 the US National Archives and Records Administration (NARA) hosted an event at which the vendor community were able to listen to NARA explain their core attitudes and beliefs about automation and records management.   NARA used the morning to invite vendors to submit information about their existing products, as well as ideas and suggestions for viable automated solutions.

Meg Phillips is External Affairs Liaison at the US National Archives and Records Administration (NARA) and she gave an introductory speech setting the goals and context for the event.

Here are the main points I noted from the speech:

  • Many federal agencies are still using ‘paper-inspired’ records management processes.
    The bigger agencies are finding such processes hard to scale
  • Federal agencies have relied heavily on electronic records management applications certified against standard DoD 5015.2.   NARA still believes that this is a viable approach.
    However the problem is that the penetration of these systems is not very deep.  Even when agencies have implemented such electronic records management systems, there are still considerable quantities of information that are falling outside the scope of the electronic records management systems and that are going unmanaged
  • NARA believes that with types of electronic records such as e-mail and social media there may be other ways of managing them that will work just as well or better than the electronic records management system approach
  • NARA is working on the assumption that automation can improve the consistency of records management outcomes, and reduce the burden on end users
  • NARA are looking to ‘change the conversation’ about records management.  In particular they are looking for more consistent and more compliant record keeping in federal agencies
  • NARA believes that the consistency and compliance of record keeping is directly linked to the burden on end users.  If an Agency  imposes a significant burden on busy staff by asking them to make decisions on every individual e-mail that they receive (should it be treated as a record? if it does need to be treated as a record how should it be filed/categorised?)  then this is likely to have a negative impact on consistency and compliance
  • NARA believes that automating records management decisions to the maximum extent possible will lead to greater consistency and compliance, will result in better records, and more accessible records
  • NARA are looking for solutions that will both scale up to the needs of big agencies but would also scale down and be affordable for smaller agencies with small IT budgets
  • NARA will start by establishing the state of the market in terms of what products are already out there for automating records management processes.
    This would enable NARA both to increase awareness within federal government of what is already available,  and to identify any market needs that need to be addressed
  • NARA wants to identify solutions that can automate records management processes, including auto-classification,  and innovative approaches coming out of the e-discovery vendor community
  • NARA does not assume that open source is the only way to go, but it is one avenue that they need to explore
  • NARA actively supports the automation of RM tasks.  They believe this may be the only truly scaleable way to consistently and compliantly manage electronic records in high volume environments
  • NARA believes this is a sea-change.  They will actively support agencies who are willing to innovate and try approaches that are not yet tried and tested
  • There will be no obligation on Agencies to introduce automated records management processes
  • As well as reducing the records management burden on individuals, NARA are also looking to ease the passage of electronic records through their lifecycle, by looking at tackling ‘moments of risk’ in the life of records.  One of these moments of risk is when records transit from one application to another.  NARA wants to find ways of making the movement of records from one system to another less onerous and risky
  • NARA are looking to standardise the way electronic records of permanent value are transferred to them from Federal Agencies.  NARA has a digital repository, which accepts ‘submission information packages’  (the term used in the OAIS model for a new accession of records to the repository).  NARA will draw up a standard for how such submission information packages should be put together

You can see a video recording of the whole speech on the NARA usenet site (Meg’s speech starts 13 minutes into video number 1).

Background to the event

NARA’s call to industry comes in the context of the  ‘ Managing Government Records Directive
issued in August 2012,  which sets two central goals to  US Federal Government .   One of the goals commits Federal Agencies to manage records in a manner consistent with Federal statutes and regulations and professional standards.   The other goal states that:

  • all e-mail must be managed electronically  by 31 December 2016
  • all permanently valuable electronic records must be managed electronically by end 2019

The goals  themselves are neither particularly interesting nor at first sight, particularly demanding.
What is interesting is the manner in which NARA  is committed to achieve it.  The Directive commits NARA , among other things to:

  • issue new advice on the management of e-mail  (this prompted NARA’s Capstone advice that I blogged about recently)
  • investigate the embedding of records management into commercial cloud services
  • investigate the feasibility of establishing central cloud service for the management of unclassified cloud records
  • investigate the possibilities for reducing the burden of records management on agencies through automation
  • revise its guidance on the transfer of electronic records to the National Archives to ensure it stays current with technology trends.

NARA are committed to reporting by 31 December 2013, on the two commitments relating to the cloud, and the commitment relating to automation.

Sub-goal A3 of the directive commits NARA to  ‘investigate and stimulate applied research in automated technologies to reduce the burden of records management responsibilities’.   In particular it commits NARA to:

  • work with private industry and other stakeholder to produce economically viable automated records management solutions
  • produce, by December 31, 2013,  a comprehensive plan to describe suitable approaches for the automated management of e-mail, social media and other types of digital records content
  • obtain, by December 31 2014, external involvement for the development of open source records management solutions


NARA have published a blogpost about the event.    Cheryl McKinnon attended the event and posted this blogpost 

E-mail and succession planning

One of the core problems with standard e-mail accounts is that business communications accumulate in individual accounts in a way that ensures they are not routinely accessible even to close colleagues.

This was brought home to me when I worked with an organisation that had shared drives, a collaboration tool,  and an electronic records system, but who conducted most of its business through e-mail.  A key part of their work was influencing and managing relations with external stakeholders, and the communications with those external stakeholders were for the most part only captured in e-mail.

A practice arose that when an individual left to take up another post elsewhere in the organisation, then the internal person replacing them would ask that individual to make an archive ( .pst) file of their e-mail account.  Sometimes the individual agreed, and left them a copy of their e-mails on the shared drive.  Sometimes they said no.

The practice was not officially supported or condoned (though it was not outlawed either).

This is a succession planning issue.   Staff starting a new role often need to see the e-mail of there predecessor, in order to carry on with the working relationships of there predecessor.  One person told me that she had transferred roles several times, and she had found it significantly more difficult to get to grips with a role when her predecessor had not provided her with an an e-mail .pst archive.

There is an element of absurdity to the situation.  There was no benefit to the organisation for  e-mail to be duplicated to a flakey .pst file sitting precariously on a shared drive.  The organisation already held all of each outgoing post-holder’s e-mails, it just had no way of letting their successors see them.

The presence of personal communications mixed in with business communications means there is no purely technical solution to this problem.

E-mail and succession planning-1of3 E-mail and succession planning-2of3 E-mail and succession planning-3of3