Update on MoReq 2010 from the DLM Forum meeting in Budapest

I attended the DLM Forum meeting in Budapest last week (May 12 and 13) at which Jon Garde announced that the core requirements of the MoReq 2010 specification had been finalised and would be published as a PDF on the DLM forum website within the fortnight following the meeting. It was possible that it might also be issued as a hard copy publication later in the year.

How the final version of the MoReq 2010 core requirements differs from the consultation version issued late last year

Jon Garde described the changes to the core requirements of MoReq 2010 since the
consultation version was released late in 2010. These changes included the adoption of a service orientated architecture for MoReq 2010, the dropping of the notion of a primary classification, and a reduction in the number of requirements.

Adoption of a service orientated architecture model
All the requirement in the MoReq 2010 core requirements have been bundled into ten services. A MoReq 2010 compliant system will be capable of offering up its functionality as services, that could be consumed by one or more other information systems within the organisation.

For example several records systems within an organisation could all consume the classification service of one MoReq 2010 compliant system, enabling the organisation to hold its fileplan in one place whilst having it used by several systems.

A MoReq 2010 compliant system must possess the capability to provide ten services:

  • a records service (the capability to hold aggregations of records)
  • a metadata service (the capability to maintain metadata about objects within the system)
  • a classification service (the capability to hold a classification, to apply it to aggregations of records, and to link headings within the classification to retention rules)
  • a disposal service (the capability to hold retention rules, and to dispose of records in accordance with retention rules)
  • a disposal hold service (the capability to prevent the application of a retention rule to a record, for example because the record is required in a legal case)
  • a search and report service (the capability to retrieve and present records and metadata in response to queries)
  • a user and groups service (the ability to maintain information about people and groups that have permissions to use the system)
  • a role service (the ability to assign roles to people and groups to determine what those people and groups can and can’t do within the system)
  • system services (the capability to maintain event histories in relation to objects held within the system)
  • an export service (the capability to export records together with their metadata and event histories in a form that another MoReq 2010 compliant system could understand)

Abandoment of the notion of a ‘primary classification’

The notion of a ‘primary classification’ for records (see my previous post) had been dropped. Instead a record will be assigned a classification, from which it would by default inherit a retention rule. It would be possible though for a person with appropriate permissions to override that inherited retention rule, and instead assign to the record a different retention rule, or to get the record to receive a retention rule from a different part of the classification scheme to the one it has been assigned to.

Reduction in the number of requirements
The number of requirements had been significantly reduced. The consultation draft had contained 436 requirements, these have now been consoldated into 170 requirements. But the final core requirements document would be longer than the consultation draft, because the introductory explanations had been increased to 90 pages.

Plans for the future development of MoReq 2010

The MoReq Governance Board has ambitious plans for the development of MoReq 2010, and regarded the publication of the core requirements as only the beginning. MoReq 2010 has a modular structure, and additional modules are planned that vendors may choose to submit their products for testing against.

The  DLM forum are planning to have a first wave of additional modules for MoReq 2010 available by the time of their triennial conference (due to be held in Brussels in the week of December 12, exact dates/venues yet to be announced).  Unlike the core requirements, the additional modules will be optional rather than mandatory.

Included in the first wave will be:

  • an import service – providing the ability to import records and associated metadata from another MoReq 2010 compliant system. Note that the ability to export records is a core requirement, but the ability to import records is an additional module.  This is because an organisation implementing its first MoReq 2010 compliant system does not need that system to be able to import from another MoReq 2010 compliant system.
  • modules that provide backwards compatibility with MoReq 2

Backwards compatibility with MoReq 2 is important.  One European country (the Czech Republic) has enshrined compatibility with MoReq 2 into records management legislation. The modules that will give backwards compatibility to MoReq 2 will be:

  • a scanning module
  • a file module (MoReq 2010 replaced the concept of the ‘file’ with the broader concept of an ‘aggregation’. The additional module would ensure that a system could enforce MoReq 2 style ‘files’ (which can only be split into volumes and parts). In MoReq 2010 terms a MoReq 2 file is simply one possible means of aggregating records
  • a vital records module
  • an e-mail module (the core requirements of MoReq 2010 itself talks generically about ‘records’ and do not focus specifically on any one particular format)

Note that a system could be MoReq 2010 compliant without being MoReq 2 compliant (because the additional modules that give MoReq 2 compliance are voluntary and not part of the core requirements of MoReq 2010). Any organisation that wanted MoReq 2 compliance as well as MoReq 2010 compliance would be able to specify that a product must be certified against those additional modules.

It is hoped that more additional modules would follow. Jon would like to see MoReq 2010 additional modules that cover records keeping requirements in respect of cloud computing, mobile devices and social software. He urged anyone who feels that there are needs that MoReq 2010 could usefully address to come forward and develop a module to address those needs. For example modules that provide functionality specific to a single sector (health sector, defence sector etc.).

There is also the possibility that modules could be written to specify the functionality required for a MoReq 2010 compliant system to also demonstrate compliance with a different standard or statement of requirements. For example a module could be written to ensure that a MoReq compliant system met all the requirements of the US DoD 5015.2 specification (which raises the interesting possibility of a European testing centre announcing that a system is compliant with the US records management specification).

Development of test centres

;

The MoReq Governance Board plans to accredit an international network of testing centres, to whom vendors can submit products for testing against MoReq 2010. Six organisations have already expressed an interest in becoming testing centres. There is no limit to the number of test centres that may be established. The test centres will use test scripts and templates created by the MoReq Governance Board. Vendors will pay a fee to the test centres to have their products tested, and (assuming they are successful) a fee to the DLM Forum to validate the recommendation of the test centre and to award the certificate.

As well as vendors submitting their products for testing, it would also be possible for an organisation to submit their specific installation of a system for testing.

How MoReq 2010 differs from previous electronic records management (ERM) system specifications

On 5 April this year I sat on a panel at the UK Information and Records Management Society (IRMS) conference. We were each asked for our view of the future of technology. We talked about the cloud, and also of the end of the dominance of ‘the document’ as a format. One member of the audience brought us back down to earth with the comment ‘never mind the future, we don’t know how to manage records in all the many and various systems our organisations have now’.

Ed Fowler of Cap Gemini said that organisations will need to get used to the fact that their records will be created and stored in many different content repositories, and that few if any of these repositories will have records management functionality.  Ed said records managers should abandon any hope that they could specify and implement one corporate electronic document and records management system (EDRMS) and expect all their colleagues to save their records into it. Recently he had only seen one UK organisation tendering for an EDRMS.

Ed was announcing the death of the traditional EDRMS. I would add a caveat here – information systems don’t die, they just lose momentum. EDRMS has lost momentum but most of those organisations that have succeeded in implementing them corporate-wide have felt a benefit and will continue to maintain and develop their systems.

The loss of momentum in EDRM is nowhere better illustrated than with MoReq, the European Union’s specification of electronic records management (ERM) systems. In early summer 2008 MoReq 2 was launched, only to be scuppered by two tornadoes: the global economic downturn, and the rise of Microsoft’s SharePoint (which took the collaboration space out of the reach of EDRMS vendors). By the end of the following year, 2009, only one vendor had submitted a system for testing against the specification, and the decision was taken by its sponsors, the DLM Forum, to commission a radical rewrite to the MoReq specification. The rewritten specification, called MoReq 2010, will be launched at the next meeting of the DLM Forum, in Budapest, next week.

The previous day (April 4) of the IRMS conference had seen a debate between Jon Garde (author of the new MoReq 2010), and Marc Fresko (author of its predecessor, MoReq 2).

Jon Garde introduced MoReq 2010 by arguing that we needed to revisualise the ERM system.  Previous specifications saw an ERM systems as a stand alone content repository that stood alongside the other content repositories within the organisation. MoReq 2010 sees ERM as a capability that could be embodied within each separate application that the organisation uses, or that could sit behind those applications and manage records created within them.

Marc Fresko was critical of MoReq 2010. In his view it introduced too many new concepts too quickly and the timeframe for the publication of the specification has left too little time for debate.

The fact that Jon and Marc differ so sharply in their viewpoints shows how different MoReq 2010 is from previous versions of MoReq.

Previous versions of MoReq aimed to specify systems that could serve all the records management needs of any organisation, in any sector. They created the breed of information system called the EDRMS. The idea was that an EDRM system would be rolled out to all users, and would hold within it a hierarchical classification (called a business classification or fileplan) that covered all the work of the organisation. Users would capture and store documents and e-mails needed as records into the EDRMS, within files (folders) classified against the business classification/fileplan.

The traditional EDRMS could not on its own solve the problem of multiple repositories. In organisations some functions (types of work) are more important than others. The more important functions will usually have a line of business application procured or developed specifically for them. An EDRMS, because it aims to cover all the diverse activities of an organisation, by definition cannot be tailored to any one type of work.  This means even those organisations with corporate-wide EDRM systems still get the problem of managing records in multiple repositories. Once SharePoint came along this problem was made even worse – if an organisation is putting a lot of time and effort implementing SharePoint for collaboration they will probably not have the capacity to implement a corporate EDRMS in tandem with it, however much they recognise the problems of SharePoint sprawl and the weaknesses of SharePoint’s own records management model.

MoReq 2010 has been written to encourage different models of records management system to emerge. It does this by adopting a modular structure. All MoReq 2010 compliant systems have to comply with a core set of requirements. Beyond that there are different modules which vendors can chose whether or not to submit their product for testing against. Some vendors may well decide to submit traditional EDRM-style systems for certification. But they are also able to submit systems for testing that meet a completely different model, for example:

  • Systems that end-users do not interact with directly, but instead capture and store records that users had created in other systems
  • Systems that do not store records, but instead govern and protect records held in other systems
  • Line of business systems or single purpose applications that are not intended to be a general records system but which have have the ability to manage the records that they capture
  • Systems that can fulfill two or more of those roles – for example a system that could be deployed as a traditional EDRM that some end-users would interact with directly, but which also possessed the capability to manage records held in other content repositories

The old aspirations that MoReq 2010 abandons

MoReq 2010 is a major change from all previous specifications of electronic records management systems. It is a break not just with the previous two versions of MoReq issued by the European Union (MoReq and MoReq 2) but also with the UK’s TNA 2002 standard and the US DoD 5015.2 standard.

MoReq 2010 abandons three big aspirations that all those previous specifications shared. It abandons:

  • the idea of the ‘file’ in the electronic records management system being the ERM system equivalent of the traditional hard copy file MoReq 2010 replaces the concept of the ‘file’ with the new concept of an ‘aggregation’ This is a major break with the vocabulary of the hard copy era. In a MoReq 2 compliant system a ‘file’ was limited to two levels of hierarchy beneath it (file/sub-file/parts). In a Moreq 2010 compliant system an aggregation can have any number of levels of hierarchy. The MoReq 2010 ‘aggregation’ has a different relationship to a business classification/fileplan than a MoReq 2 ‘file’. The MoReq 2 file sat at the very bottom of the business classification/fileplan. An aggregation can be a multilevel hierarchy in its own right, so it can sit separate from the business classification, whose role is not so much to act as the only means of navigating around the records system, but more to apply retention rules to records.
  • the idea of one monolithic corporate business classification/fileplan universally applying retention and access rules to all records In previous versions of MoReq only one classification could be linked to retention rules and used to apply retention rules to records (a hierarchical corporate business classification, also called a fileplan). In Moreq 2010 compliant systems there is the possibility of having several classifications, any number of which can be used to apply retention rules to records. If a record is classified against more than one classification then one of them should be nominated as the ‘primary classification’ for that record. The primary classification is the one that the record inherits its retention rule from.
  • the idea of their being a single type of ERM system that can meet the needs of all organisations in all sectors Jon Garde identified that MoReq 2 was twice as long as the first MoReq. Jon says the reason for this is that both MoReq and MoReq 2 tried to specify systems that could meet the records management needs of all organisations, whatever their sector. This meant that if any sector of the economy could prove they had a genuine records keeping requirement then MoReq would try to incorporate it into the specification. The disadvantage of this is that vendors had to configure their systems to cover all sectors and all eventualities, which pushed up the cost for them of developing compliant systems. MoReq 2010 has been written so that the core module contains only requirements that are common to all or most organisations. If a sector has specific requirements they are able to write a separate MoReq 2010 module to capture those needs. Vendors that wished to target that sector could add that functionality to their system and ask for it to be tested and certified against that module. Vendors that didn’t wish to target that sector could ignore it. Jon Garde argues that the MoReq 2010 will be more sustainable over time than previous specifications, as new needs can be incorporated into new modules without having to republish the whole specification. Marc Fresko gave the counter argument that the structure of MoReq 2010 will be more complex for records managers in organisations to work with, because they are going to have to decide which modules are important enough for their organisation to insist upon.

The new aspirations of MoReq 2010

In place of those abandoned aspirations MoReq 2010 brings in three new aspirations:

  • the idea that the electronic records management system will sit behind the existing applications used to create and capture records in the organisation A system can be MoReq 2010 compliant without having any user interface. The core module merely insists that if a system does not have a user interface it must have an API (application programming interface) to enable it to integrate with those systems in the organisation that users do interact with to create records. Note that integrating an electronic records management system behind existing content repositories poses very different challenges to those posed by implementing a stand alone EDRMS that end-users directly engage with to save their records. It is likely that your various content repositories will have been procured from various vendors, written in various programming languages, and have varying quality APIs. Each integration to each content repository is a project in its own right. Aside from the technical challenge there is also the semantic challenge – each of those content repositories will have their own way of organising and describing content.
  • the idea of MoReq compliant systems controlling records held in other systems a system can be MoReq 2010 compliant without having the ability to store records, provided it has the ability to apply classifications, access rules, and retention rules to records stored in other systems; provided it can protect those records from deletion or amendment; and provided it can maintain (and export) an audit trail of events that happen to those records.
  • the idea that when an organisation changes its records system the new records system could understand everything that had happened to those records in the outgoing records system. Record systems capture records and apply retention rules to them. Jon Garde points out that the span of years of many of these retention periods is longer than the life of most records systems. Jon wants a MoReq 2010 compliant system to be able to export a record and its associated metadata at any point during the retention period, so that the receiving system could understand what retention policy has been applied to the record, when that retention period started, and how long it had left to run. The new records system could then apply the remainder of the retention period. MoReq 2010 is much stricter than previous specifications on how a system maintains its audit logs. The system must be able to export an event history for every object in the system.  The event history must record everything that has happened to that object since it was created in or captured by the system. In order for this event history to be understandable by another system MoReq 2010 insists upon the use of unique identifiers. A MoReq compliant system needs to allocate unique identification numbers to every type of object (records, aggregations, retention policies, classification headings etc.); to every actual object; to every type of event in the system (for example ‘the title of a document is changed), and to every actual event.
  • the idea of line of business system vendors incorporating records management functionality into their systems By stripping down the core requirements to a very minimum the DLM-forum hope that providers of line of business systems will see that it is possible for them to seek MoReq 2010 certification.  The hope/dream is that providers of finance systems, human resource management systems, facilities management systems, and customer relationship management systems will add MoReq 2010 functionality to their applications . Note that vendors of those kinds of applications won’t start to think about MoReq 2010 compliance unless and until purchasing organisations start specifying a preference for MoReq 2010 compatible applications. The challenge here is that records managers often do not have a voice in those kind of procurement exercises.

The likely impact of MoReq 2010

MoReq 2010 will bring interesting effects for records managers. The standards we were used to in the first decade of the 21st century (TNA 2002, DoD 50.15, MoReq 2) all ended up certifying a very similar type of system, implemented in a very similar way. Not quite ‘seen one EDRMS seen them all’ but almost.

Out of MoReq 2010 will sprout a variety of different applications, which will need a variety of different implementation methodologies. This is a challenge for the wider records community, not the sponsors of MoReq 2010. All MoReq 2010 can do is to encourage vendors to provide applications that manage records created and/or held in different content repositories (and provide certification to verify that they have that capability).

Whether or not MoReq 2010 ends up being successful will ultimately depend on whether we records managers can come up with viable implementation approaches for the type of application that it will certify.