The mechanics of manage-in-place records management tools

The idea of a manage-in-place records management tool is that it holds your records classification scheme and retention rules, and applies them to content held in a variety of different content repositories/applications (SharePoint, line of business systems, e-mail archives etc.).

At the IRMS conference in Brighton in May I had conversations with several vendors of manage-in-place records management tools about how they went about ensuring that their products could connect with the applications in day-to-day use within organisations

The importance of APIs (application programming interfaces)

In order for the manage-in-place tool to work it needs to have a ‘connector’  to each content repository that it wishes to govern.

The connectors are typically built to use the API (application programming interface) of the content repository.  The API exposes a subset of the content repository’s functionality.   It specifies how any authorised external application (in this case  a manage-in-place tool) can issue commands to the content repository.

Some of the things that a records manager might want their manage-in-place tool to do inside the various content repositories of your organisation include:

  • adding metadata to a document or aggregation of documents
  • linking an aggregation of documents to a node in a records classification
  • preventing editing or deletion of  a document or aggregations of documents
  • linking a retention rule to a piece of content or an aggregation of content

The beauty of the concept of an API is that the two applications can interact with each other without you having to customise either application.  It does not matter if the two applications are written in entirely different programming languages.  Nor does it matter if one or both of the applications are based in the cloud.

In theory:

  • you could replace your manage-in-place tool with a new manage-in-place tool from a different vendor, and none of the content repositories need notice any difference (provided that the new manage-in-place tool carried on issuing the same commands to their API)
  • you could replace a content repository with a successor repository from a different vendor without the manage-in-place tool noticing any difference (provided that the new content repository offered a similar API that enabled them to make the same commands)

In practice each vendor constructs the API for their content repository differently, and this creates two challenges for the makers of manage-in-place tools

1) they have to construct a different connector for each different vendor’s content repository.  Two of the  manage-in-place providers I spoke to at the conference (RSD and IBM) both provided connectors to over 50 different commonly used content repositories.

2)  some APIs are better than others.  Some applications expose more functionality through their API than other applications, and hence let the manage-in-place tool do more things to their content.  One example cited was that the manage-in-place tool can get some document management systems to display the organisation’s records classification (fileplan), so that users of the document management system can link or drag and drop content to the appropriate node in the classification.  Other document management system do not have that functionality exposed in their API.

CMIS (Content management interoperability services)

CMIS  is a specification that aims to overcome the first of these two problems.  The specification was drawn up by a coalition of vendors in the ECM space under the auspices of the OASIS Technical committee.

The idea is that vendors  add a CMIS layer to their applications.  Just like an API, the CMIS layer exposes a subset of the functionality of the native application, so that an external application can make use of that functionality.  The difference is that whereas each vendor’s API is constructed and expressed in a different way, a CMIS layer is standardised. This means that a similar function (for example ‘add a document’) would be expressed in the same way in the CMIS layer of each vendor’s products.

A mange-in-place tool vendor could choose to build connectors to the CMIS layers of content repositories, rather than through the API.  In theory this saves a manage-in-place vendor from building seperate connectors for every different type of content repository they want their product to be able to govern.

Screen Shot 2013-06-25 at 11.18.49

In practice the vendors of the manage-in-place tools that I spoke to told me that they prefer to write connectors that use the API of each application, rather than the CMIS layer.   This is simply because most repositories expose more functionality through their API than through their CMIS layer.
Screen Shot 2013-06-25 at 11.14.02

CMIS and records management

The disadvantage of CMIS being writtten by vendors is that a coalition of vendors have to agree for functionality to be put into the specification. They have tried to capture concepts and functions that are common to all or most existing repositories. Functionality such as records management, which some repositories have and some don’t, has not received prominent treatment in CMIS.  The first version of CMIS had concepts such as a document, and a folder, but it did not support retention rules, nor a records classification/fileplan (although it did have the concept of a folder structure).

The latest version of CMIS (1.1) does have retention functionality in for the first time.  But that has not pleased all of the vendors.  Jeff Potts, of Alfresco wrote this in his blogpost announcing the approval of CMIS 1.1

This new feature allows you to set retention periods for a piece of content or place a legal hold on content through the CMIS 1.1 API. This is useful in compliance solutions like Records Management (RM). Honestly, I am not a big fan of this feature. It seems too specific to a particular domain (RM) and I think CMIS should be more general. If you are going to start adding RM features into the spec, why not add Web Content Management (WCM) features as well? And Digital Asset Management (DAM) and so on? I’m sure it is useful, I just don’t think it belongs in the spec.

This is the dilemma for CMIS:

  • if they do not give full coverage of sets of functionality such as records management then manage-in-place tools will bypass the layer and just use the  APIs of the content repositories.
  • the more detailed and precise their definition of records management functionality is, the harder it is to get the coalition of vendors to agree on it

From a records mangaement point of view what we want out of CMIS (or any other standard in the API space) is to set out a minimum set of records management functionality that the API of every business systems sbould have.

In theory, if CMIS specified a set of API commands that would expose the functionality needed by one or more of the current electronic records management specifications,  then vendors would never have to re-architect their product to meet that electronic records management specification,  All they would need to do is expose the relevant functionality  in their CMIS layers and let the manage-in-place tools use that functionality to govern the content they hold.

Of course this would not solve all of our problems –  one of the biggest content repositories in most organisations are simple shared network drives, that don’t have an API (never mind a CMIS layer!).