Thoughts on the Department for Education’s SharePoint implementation

On 16 February this year I heard Mark Field describe his implementation of SharePoint for collaboration and records management within the UK Department for Education (DfE). Mark’s talk was at a records management update seminar run by Unicom.

Mark’s implementation has the potential to be extremely influential.   It currently has a userbase of 3,000 people within the Department.  DfE are planning to offer their SharePoint facility as a service via the cloud – firstly to the UK education sector, and later to the wider public sector via the Government Application Store on the G-Cloud.  There is currently no timescale as to when it will be available on G-Cloud (when pressed Mark said ‘hopefully within our lifetimes’).

DfE are using SharePoint team sites (rebranded as ‘Information workplaces’). They use the SharePoint records centre for records management, and have implemented SharePoint for their intranet.   They are currently using SharePoint 2007 but will move to SharePoint 2010 once they have transitioned their service to the cloud.

Governance controls

The Department has tight governance controls in place to control the spread of  SharePoint team sites.  Only their Information and records team can create team sites (information workplaces).   Each team site/workplace must have a ‘Workplace owner’ who signs an agreement acknowledging their responsibilities in managing the site.  They have 200 Information workplaces, which tend to be based around a particular work matter or policy issue.

Records management

Regular ‘sweeps’ are undertaken of team sites to suck documents needed as records, and route them to the records centre.  The sweep captures documents that meet all of the following three criteria:

  • at least a year old
  • a major version (1.0, 2.0 etc)
  • checked in

When a document is routed to the records centre it leaves behind a link in the team site document library from which it came.

The Department has decided not to implement a functionally based fileplan. The record centre is structured to mirror the organisational structure.  Retention rules are linked to content types. The Department has limited itself to 11 content types.

There was a mixed reaction by the audience to the model adopted by DfE.  Some people (including ex `Information and Records Management Society Treasurer Alison North) praised its simplicity.   Others were concerned by the Department’s rejection of a functionally based fileplan in favour of the organisational structure.  The question was raised as to whether organisational churn would render the records centre difficult for future users to understand and navigate.

Mark defended the decision to use the organisational structure to organise the SharePoint records centre.  He said that all a searcher would need to know to find documentation is what the relevant organisational unit was called at the time of the document’s creation.

Replacement of the DfE’s Electronic Document and Records Management System (Meridio)

Mark’s lack of faith in a functional fileplan is based on his experience at a previous organisation, where users ‘ran a mile’ from a multi-level fileplan he had constructed with colleagues; and from DfE’s own experience with using a functional fileplan on a  traditional (TNA 2002 compliant) EDRMS system (Meridio).

Mark said Meridio had good functionality but it  was only adopted by 20 to 30% of the Department.   The  Department have ended their EDRMS implementation, and imported all records from Meridio into a SharePoint records centre (a separate records centre from the one that they are using to handle content generated in SharePoint).

DfE’s application of retention rules

The department are choosing to link retention rules to SharePoint content types, and to limit the number of content types to 11.  This is a radical example of the big bucket approach to implementing retention rules.   In most other implementations of ‘big bucket’ retention rules the organisations concerned have applied between 50 and 100 retention rules to their organisation’s information.  Applying only 11 retention categories seems  low to me.

One of the content types Mark mentioned was for human resource documents.  I presume that a ‘human resource’ content type would need to embrace (among other things) both:

  • all the documentation arising from recruitment campaigns
  • documentation of dealings with individual employees

Most organisations keep records of their dealings with individual employees for a considerably longer period than records of their recruitment campaign.

My assumptions as to the nature of the Department for Education’s SharePoint records centre

It is difficult for me to accurately picture Mark’s model without having seen screen shots.  My assumptions are that:

  • the records centre contains one document library for every organisational unit that has existed since the start of their SharePoint implementation. Each library contains all the documents routed to the records centre that originated from colleagues attached to that unit
  • when organisational churn occurs a new records centre document library would be created to accommodate records routed from that organisational unit (although it might be a year before any documents arrive to fill it!)
  • when documents move from a team site to the records centre they bring with them important contextual information such as the details of the pathway/breadcrumb trail in which they had been stored ( team site/document library/folder/sub folder)
  • a visitor to the records centre would be able to sort each document library by any column of metadata, including which of the 11 content types a document belong to, and which team site it came from.

Does the records centre add value?

The big challenge for any implementation of the SharePoint  records centre is the question what value does the record centre add?   What value does it add to have documents moved from one logical structure (Team sites, arranged by policy issue or matter) to a different logical structure (A records centre arranged by organisational unit)?

Is there any reason why anyone would prefer to look at the records centre rather than the original team site? (I can’t think of one).  All the time that he original team sites are maintained that is not a problem.  But how long will the Department retain its  team sites?

I suspect that the Department for Education’s intention would be to apply a shorter retention period to the SharePoint team sites than to the contents in the records centre.  Once the original team sites are deleted you are left only with the content of the records centre, kept in a different arrangement to that known by the people who worked with the documents at the time that the work was carried out.

Take the example of a document library within a team site set up for three different organisational units to collaborate around policy on special need provision in schools.   How easy would it be for a searcher to virtually reconstitute the contents of that team site document library once the contents have been sent to different records centre libraries and after the original team site has been deleted? Does it matter that other information held in the team site, but not captured in a document library, will not have been sent to the records centre?

Practical reasons for moving records from team sites to the SharePoint records centre

Later in the same Unicom event Sharon Richardson talked about the difference in storage capacity of a site collection containing collaborative team sites, and a records centre:

  • Microsoft recommends that site collections containing collaborative team sites are not allowed to grow about 100GB (which Sharon estimates is only 200,000 documents). The constraint is due to the impact of a large site collection on other site collections held on the same SQL Server content database
  • A records centre can be allowed to grow much larger provided that its’ content is stored in a dedicated SQL Server content database which contains no other site collections

The records centre can mitigate problems caused by the quirks of SharePoint’s site collection architecture – but I am still not sure that this counts as adding value!

10 thoughts on “Thoughts on the Department for Education’s SharePoint implementation

  1. “all a searcher would need to know to find documentation is what the relevant organisational unit was called at the time of the document’s creation”. Uh huh. And if the document is 10 years old and there have been 20 organisational changes over the 10 years? we have about 20 organisational changes every year…

    1. Hi Clare,
      DfE are automatically routing documents from team sites to the record centre after a year. The biggest challenge they will have faced is how to make sure all records go to a sensible place in the records centre structure. I think the reason why DfE decided to organise the records centre by organisational unit is that they can easily capture the organisational unit of the contributor of every document to every team site. On the other hand setting up rules to automatically route every document from a SharePoint team site to an appropriate place in a functional fileplan would be no easy task

      1. Hi Clare and James,
        Organisational unit change is rarely that severe. It’s usually a matter of re-arranging the deck chairs. End users identfiy themselves with their organisational location not with some abstracted list of functions. The FAT stranglehold on RM thinking is long overdue for relaxation.

  2. I heard Mark present at an event held by The National Archives. I wasn’t present at the Unicom event.

    The issue Mark has with a functional file plan is valid, but I think he is wrong to blame the tool. A successful file plan has to be a compromise between meeting the needs of the organisation (in terms of security and disposal) and the needs of the user to browse and store information within a context.

    An organisation adopting on 11 content types may perhaps meet the needs of the users, but I doubt it will meet the long term accessibility needs of the organisation.

    Designing IM solutions in SharePoint 2010 is no easy task. SharePoint has lots of functionality, but little “system”. By this I mean the tools to sustain a records management system in the long term.

    Products that extend compliance and provide this long term “system” support to SharePoint are starting to appear on the market. I believe these will provide a much better solution than the approach adopted at DfE.

    In these tough times it is easy to go for a “cheap” solution. If it isn’t sustainable, it will however prove an expensive mistake in the long term.

    1. Hi Richard – in terms of third party tools I would be interested to see whether there is more of a market for a) plug-ins to SharePoint, that apply records management frameworks to content in SharePoint OR b) systems that can apply records management rules to content held in many or all of an organisation’s applications (including SharePoint). I would assume that the plug-ins would be cheaper and easier to implement. It seems that no clear market leader has yet emerged in either category.

  3. I was at the Unicom event too. I think the DfE example is simplistic in terms of the challenge faced by local authorities and other organisations. There is no way any long term structure would last if the organisation structure is used, so a functional model is preferred.

    The 10/11 content types was supported by evidence elesewhere. I’ve always doubted Microsoft’s approach for masses of content types – we are looking at a number of document types – quite how we map to content types or not we haven’t worked out yet. Our experience with HR has showed us that complicated retention/disposal rules don’t work and a much simpler ‘by employee file’ works better.

    I’m not convinced by the record centre. It does seem to be useful for long term archive, but why not just use existing features and site structure!

    1. Andy, it is possible that the wide range of services delivered by local authorities will push them to FAT fileplans, but I suspect there is a FAT/organisational hybrid which would work better – I worked on the DSTL fileplan for some time, and while I left before it was delivered in its final form (although I’m in touch with its architect), it was moving to a hybrid structure as a response to consultation with fileplan users in the business. The key activity in organisational fileplans is mapping structure through time, which is perfectly do-able, and assigning ownership at the right organisational level, which is a policy decision.

  4. Does anyone know if the DfE sharepoint records centre is accessible externally and if so has the dpartment applied an appropriate 2 factor authentication security measure?

    1. Nick (and others, I will respond where I can): the Records Centre is not accessible externally. We can provide two-factor Business Impact Level Three access to our network, using Employeer Authentication Services (EAS), but that is currently only used for projects where DfE staff need to collaborate with external partners. There are no plans to provide external access to the Records Centre. Happy to expand via email.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s