How will the 21st century keep records?

We are only a decade into the 21st century, but it is already apparent that there will be something different about the records produced by this century, as compared to those of any other time period.  It is not just that they are going to be in electronic rather than hard-copy form, it is more fundamental than that.

Records in previous centuries

The United Kingdom’s National Archives in Kew houses an unbroken eight hundred years worth of records. In every previous one of those 8 centuries of English/British record keeping, there has been a common method of keeping records.  A ubiquitous method that was used for record keeping by any and every organisation.

  • In the 13 and 14th centuries the ubiquitous method of record keeping was the roll.   Manorial courts, ecclesiastical courts and the royal administration all kept records in the same way. They would issue a communication (a charter or a judgement for example). Their clerks would copy the communication out onto a strip of parchment and sew it onto the end of the copy of the previous communication, creating a roll
  • By the nineteenth century the ubiquitous method of record keeping was the register and the book. Government ministries, branches of the armed forces, companies and trading houses all kept records the same way.  Clerks would keep a register of all incoming and outgoing letters, with one line of description for each letter. Outgoing letters would be copied into a letter book before they were sent out.  Incoming letters would be bound into books, or gathered together in boxes or files.
  • In the twentieth century the ubiquitous method of record keeping was the file. Organisations had grown in size, and now performed a great diversity of different types of work.  It was the age of the bureaucracy.  It was no longer possible for a small group of clerks to capture all incoming and outgoing communications within a series of books.   Files were a more flexible record keeping format, a file could be created whenever a piece of work started and closed when it finished

Was there a single organisation, operating in the second half of the twentieth century, that did not create hard copy files to record their work?

Records in our century

In the 21st century there is no such ubiquitous method of record keeping.

Records are still being kept.  But they are all over the place.   When you work with an organisation now you find different teams making different choices about how to record their work, and some teams not making a choice but leaving it down to individuals.   There is a safety net of sorts in the shape of the e-mail inbox, the place of first and last resort to find information in the enterprise.

The range of different ways in which people choose to keep records is dazzling.   I recently visited a compliance team in an organisation.   The team were conscientious, for each area of their work they had made a rational choice of how they keep their records:

  • they kept records of their audit work as shared Outlook folders into which they dragged relevant incoming and outgoing e-mails
  • they kept records of their policy work, and their approval work, as folders on the shared drive
  • they kept records of their inspections on a dedicated line of business system
  • they kept records of their project work as folders on the corporate document management system

The organisation concerned, like many others, is intending to roll out SharePoint over the coming year. That will provide yet more choices in terms of where and how teams store the records of their work.  They might use a SharePoint document library for each piece of work.  Or they might set up a SharePoint team site, or sub-site, for each piece of work.

It is significant that SharePoint is likely to come in alongside, rather than instead of, the systems that are already available to the team.

The implications of the lack of a standard method of keeping records for the records management profession

In such a situation we as a records management profession have three options:

  • We can try to pursuade the 21st century to adopt a particular application or fomat for record keeping
  • We can wait until the 21st century settles on a method of record keeping and in the meantime manage the vast array of different formats as best we can
  • We can try to find a way of managing records that allows people to work (and store records) in any application that they choose, whilst at the same time allowing the organisation to control and manage those records over time

Each of these options has its pitfalls

The records management profession will not be able to dictate to the 21st century how it should keep records

The records management/archives profession has never been able, at any point in history, to dictate the way records were kept. In the 20th century records managers did not create the hard copy file.  It was the other way round – the hard copy file created records managers. Organisations were creating hard copy files anyway. Records management came into existence to solve the problem of how large 20th century organisations managed hard copy files at a corporate scale – to help them manage the thousands of files that were being created across a huge variety of different types of work.

In the first years of the 21st century the archives and records management profession tried to create a ubiquitous method of record keeping for the digital age.  It was to have been the ‘record folder’.  Note that we couldn’t take the word ‘file’ with us into the digital age because it had already been commandeered by Microsoft and others to denote a single digital object, such as a document produced in  MS Word.

Electronic records management systems (EDRMS) were specified by various national and pan national government bodies.   Record folders would be the equivalent of hard copy files.  They would be created when a piece of work started and closed when a piece of work finished.  They would be classified within a corporate fileplan.  They would receive the appropriate retention and access rule from the fileplan.  They would protect any document declared to that folder from amendment and from premature deletion.

Records folders have not become ubiquitous.  Some organisations use systems that support them, most do not. The phrase ‘record folder’ has not entered common parlance, it is a piece of jargon that any organisation implementing an electronic records management system has to establish, define, explain and promote.

The 21st century may never settle on a uniform way of keeping records

There is no guarantee that the 21st century will settle on a uniform way of keeping records. The competition between technology companies is driving constant development of new formats for the communication and storage of information. This competition is fuelled by the fact that the world wide web acts as a giant laboratory in which new collaboration/communication formats can be developed, tried and tested. When these formats are successful on the web they are inevitably brought into the enterprise after a time lag.

In the 20th century there was competition between companies supplying organisations with paper, filing stationery and filing cabinets. But none of those companies were trying to disrupt the way individuals communicated or stored information. No company was trying to come up with an alternative to the hard copy document, or an alternative to the practice of gathering hard copy documents together into files. They could not have been able to disrupt these practices even if they had wanted to.

In the 21st century there has been a constant stream of new formats and applications disrupting the way people record and communicate information. Documents are still important. But successive waves of new formats (e-mail, instant message, blogs, discussion boards, wikis, status updates) have come in alongside the document.  

Microsoft still dominate the communication and storage of information in the enterprise. The two dominant business formats are still the document and the e-mail. They are created, stored and managed with Microsoft tools: Word, Outlook and SharePoint. In 2009 Google launched a beta version of Google Wave, an innovative format for collaborating on outputs that explicitly tried to re-magine both e-mail and the document. Google wants to muscle into the enterprise, and would find it easier to do if they could move people onto a different format, a format invented by Google. We do not yet know whether or not Google Wave will be a success, so far it has not lived up to the hype. We do know that whenever a status quo is established there will always be another tech company trying to disrupt it.

This disruption is bound to continue: market dynamics will see to that. Even if the 21st century did settle on one method/application for keeping records, it would be disrupted within a decade.

We have no experience of implementing records systems that manage content stored in various different applications

We can characterise the situation at the end of the first decade of the 21st century as follows:

  • Records can be, and are, generated in any application and any format that is generally available to knowledge workers
  • More and more different applications and formats are available to 21st century knowledge workers
  • More and more different formats and applications will continue to be made available
  • The attempt to persuade/ cajole/ force colleagues to save all content needed as records into one application (the electronic records management system) looks doomed to only partial success

Lets make three assumptions

  • organisations will continue to exist
  • organisations will continue to want to have a corporate method of managing records
  • organisations will continue to look to vendors of electronic content management (ECM) systems for solutions to the problem of managing records

If you can’t predict what format or what applications people are going to use within your organisation then your only option is to have a system that will manage records produced in any format, in any application.

The challenge for ECM vendors is :

  • Make us a tool that can manage records kept in any format, in any application.  Convince us that it could be used to capture and manage records created in formats and applications that have not been invented yet

The challenge for us as a records management profession is even harder:

  • If and when vendors did provide us with such a tool, how would we deploy it?

There is a vendor term for this model: ‘manage-in-place’. Vendors describe these systems as enabling you to hold a fileplan and retention rules within a manage-in-place Enterprise Content Management System (ECM), but apply them to content held elsewhere. For example you could apply a retention rule to content held in shared drives.

The rise of SharePoint has pushed ECM vendors out of the collaboration space, and sent them scrambling to find a new unique selling proposition. The ‘manage in place’ model seems to provide ECM vendors with a way of staying relevant to document and records management within the enterprise. Their new USP of the ECM is – ‘we can help you govern the content that your users create in SharePoint, shared drives, line of business systems etc.’

However there are a great many unanswered questions about the model. Even if the functionality is there, no-one knows how organisations could make it work. There are no case studies out there that I have heard of, and no guidance notes from any national archive or professional society.

Here are some questions about the implementation of a ‘manage-in-place’ records management system:

  • Would all colleagues interact with it? Or would it sit in the background, used only by administrators and records managers?
  • How would colleagues activate it when they wanted something to be captured as a record? For example lets imagine a team are using a shared drive to keep records of the audits that they carry out. They want those records to be protected and governed by the ‘manage in place’ ECM system. How do they go about that making that happen?
  • How would colleagues know something had been captured as a record?
  • Do records stay in the original application, or do they move to the records management tool?

The main question records managers ask me about SharePoint is ‘do I need to plug an ECM system into the back of it to manage records?’. Explaining the weaknesses in records management in SharePoint 2007 and 2010 is not enough to persuade SharePoint customers to buy ECM systems to manage records. The price tag of ECM licences, and the added complexity of implementing an additional application are strong barriers to overcome. The attitude of people seems to be ‘we are getting SharePoint anyway, we will see how we go with it, and if we need to plug an ECM in to govern the content we will do so at a later date’.

If the ECM vendors seriously want to win market share for their ‘manage-in-place’ records systems then they need to get serious about promoting the concept, explaining the model and sparking a debate about it. They would also need to slash their prices so they get some take-up, some case studies, and some momentum.