The practicalities of upgrading to SharePoint 2010

In a recent episode of the MOSS Show podcast Hilton Giesenow interviewed two highly experienced SharePoint administrators, Todd Klindt and Shane Young. Todd and Shane described the options for upgrading from SharePoint 2007 to 2010, and why they think the upgrade will be a boon for hardware manufacturers.

If you are running SharePoint 2007 you will only have two options for upgrading to SharePoint 2010:

  • An in-place upgrade: SharePoint goes through all your content databases and converts them to SharePoint 2010. It does it all in one go, on the same hardware, with everything staying at the same url address as previously
  • A database attach upgrade: you keep your SharePoint 2007 farm running, and set up a brand new server farm to run SharePoint 2010 alongside it. You move your content databases over one by one from the SharePoint 2007 farm to the SharePoint 2010 farm

At first sight the in-place upgrade appears the better option. It is quicker, simpler (because you don’t have the complexity of running 2007 and 2010 in parallel) and cheaper (because you can re-use your existing hardware).

However Todd and Shane said that their experiences of running in-place upgrades from SharePoint 2003 to 2007 were so bad that they are wary of trusting it this time around. The in-place option gives you no control over the upgrade process. You have to let the whole upgrade take place over the whole farm. If the upgrade has failed all you can do is revert completely back to SharePoint 2007, diagnose what caused the problem, fix that problem and try the whole thing all over again.

Todd’s experience of running in-place upgrades from SharePoint 2003 to 2007 was that diagnosing the problem that caused your in-place upgrade to fail was like ‘shooting fish in a barrel’ . The problem could have come from anywhere on any content database anywhere on the farm, and the error messages that SharePoint provided were of little or no help.

Todd and Shane will instead be recommending that their clients run a database attach upgrade. They admit that this is a more expensive, time consuming and complex option than the in-place upgrade. It requires you to buy a complete set of new hardware for SharePoint 2010 to run on. It results in have what Todd called ‘URL craziness” as you move the content databases over one by one to run on the SharePoint 2010 farm, whilst still running the remaining databases on SharePoint 2007 from the 2007 farm.

However the advantage of the database attach upgrade is that you have more control over the whole process. You isolate each content database and move them over one by one. If the upgrade to one database fails it doesn’t mean that you have to roll back the whole of the rest of the upgrade. And it is much easier to diagnose problems because you know which content database the problem has occurred in.

Todd and Shane said that if they were a hardware manufacturer like HP or Dell they would be rubbing their hands at the thought of all those upgrades from SharePoint 2007 to 2010, and all the extra hardware that organisations will be buying.

Two conflicting definitions of a ‘record’ in use in organisations today

Organisations today are using two conflicting working definitions of a record. Neither definition is adequate on its own (which is why the two conflicting definitions co-exist with each other!).

The first of the two working definitions states that ‘records’ are those documents/communications that are needed as evidence of a piece of work. This definition comes from records management theory. Records management practice aims to capture these documents in a file/ records folder dedicated to each particular piece of work. The file/ records folder exists in a records system, which could be anything from a shared network drive, to a SharePoint team site, an electronic records management system, a line of business document management system, or a traditional filing cabinet.

Traditionally records management practice aims to ensure that the file/ records folder is the only information about a piece of work that survives over time. This translates into advice to colleagues on handling documents and e-mails such as ‘if it is important put it on the file, it if it isn’t important get rid of it ‘

If a team is confident that the file/records folder tells the whole story of the piece of work, then they can be confident in disposing of all information about that work lying outside of the file. The converse is also true – if you have not got confidence that the files in your records system(s) serves as a sufficient story of your organisation’s activities, then you are not going to be confident that you can delete material from other information stores such as e-mail in-boxes, and shared drives.

However for most pieces of work in an organisation ‘the file’ in a designated record system can no longer be regarded as the single source of evidence on that piece of work. This is down to two factors:

  • it is highly likely that other pieces of evidence about that piece of work persist outside of the file, and that these pieces of evidence are findable or discoverable
  • it is highly unlikely that each colleague has placed all significant documents/communications arising from that piece of work onto the file

I am struggling to think of a single organisation I have worked with that routinely delete e-mails after a period of months, or routinely deletes material on shared drives that is a year or two years old – they simply have not got enough confidence in their ‘records’ to risk losing important material like that.

The second working definition defines ‘records’ as being any information, in whatever form, that is held by that organisation. In other words any piece of information that exists anywhere: on the organisations network, in its filing cabinets, on people’s desktops, in desk drawers. This definition stems from the legal obligations placed on organisations. These obligations vary from jurisdiction to jurisdiction, but include data protection, freedom of information, and e-discovery. Such obligations make no distinction between those documents/communicaions that an organisation intended to keep as records, and those that have accidentally survived.

This definition is not adequate from a practical point of view either. When an organisation receives a data protection subject access request, or a freedom of information request or an e-discovery demand, they have to pull out all the stops and trawl their network. They might ask swathes of the organisation to go through their e-mail and unearth any messages regarding that matter. They might do searches of shared drives and document management systems. They unearth a pile or a mountain of material, that they then spend days or weeks sifting through. That is a massive pain, that an organisation is willing to put up with because that is the law that they are working under. But this approach would be overkill for most internal uses of record, such as the following scenarios

  • You have taken over the management of a contract that has run into problems. You want to understand what has been agreed and discussed with the contractor up to this point.
  • You have been asked to revise the service level agreement you offer to your customers. You want to understand the thinking behind the clauses in the current agreement. You want to read the discussions/correspondence that took place the last time the SLA was drafted
  • A budgeted cost has changed between different versions of a business plan and you are trying to understand why

In each of the above scenarios you would be hoping that the colleagues involved in those pieces of work had kept a good file/records folder of that work, in which they had brought together all the relevant documents/communications relating to the work.

The dilemma for organisations is that they are caught between a rock and a hard place. They are not able to maintain good files across all areas of work. The concept of a file/records folder feels like an old-fashioned concept, a legacy from the hard-copy world of decades gone by. And yet organisations still need some means of relating together the records (however defined!) of a piece of work and managing them over time.

If the file/records folder was invented today, from scratch, with no history or legacy, what features would it have?