The Ontario gas plant cancellation records deletion saga from a recordkeeping perspective

1.     Introduction

The Ontario gas plant cancellation records deletion saga has occupied a considerable amount of column inches and radio and TV time in the province itself, in Canada, and beyond since the spring of 2013.   Little or none of this debate has been informed by a recordkeeping perspective, despite the fact that the deletion controversy started with an allegation that Ontario’s Archives and Recordkeeping Act had been breached by Craig MacLennan, the former Chief of Staff to the Minister of Energy.

This post attempts to provide a record keeping perspective on the saga.

It argues that:

  • the deletions that have caused most of the controversy in the saga are less damaging in recordkeeping terms than a form of deletion that has so far caused little or no controversy – the IT policy of deleting staff email accounts when a member of staff leaves employment.
  • most of the debate in Ontario’s Parliament has concerned the behaviours and motivations of individual political staff who work or who did work in the Office of the Minister of Energy and the Office of the Premier.     Of much more interest from a recordkeeping perspective is the question of why the record keeping systems in place were not robust enough to capture and protect an adequate record of the correspondence of the ministers involved in the gas plant cancellation decisions.

The post goes on to give recommendations as to how the Ontario Government (or any government) could prevent the recurrence  of a similar saga.

To accompany this post I have recorded a records guru podcast in which I discuss the saga with Jon Garde (listen to it here)

2.     The different types of deletion involved in the saga

The saga has involved three different types of deletion:

  • the confession by  Craig MacLennan in April 2013 that he routinely and indiscriminately deleted e-mails from his e-mail account whilst serving as chief of staff to the Ministry of Energy
  • the government’s  IT policy of deleting e-mail accounts of staff members when they leave Ontario’s public service
  • the alleged attempts by staff working in the Office of the Premier, to wipe clean the hard drives of the computers used by colleagues who were leaving their posts during the transition from Premier McGuinty to Premier Wynne  in the autumn of 2012

Which one of these types of deletion is the most important depends upon whether you are looking at this from a political perspective, a police perspective, an access to information perspective, or a recordkeeping perspective.

3.     The routine deletion of e-mails from his  e-mail account by Craig MacLennan

3.1        Craig MacLennan’s confession

In April 2013 Craig MacLennan, former chief of staff to the Minister of Energy, was asked why he had not provided any correspondence to the Estimates Committee of Ontario’s Parliament in response to their request in the spring of 2012 to see correspondence related to a decision to cancel and relocate two gas plants.   He replied that at the time of the request he did not have any such correspondence.   The reason for this was that he kept a clean in-box and routinely deleted e-mails from his in-box and sent items as he went along, in order to keep within limits he thought had been set by IT.

3.2     Did the deletion actually happen?

The first thing to be said about Craig MacLennan’s confession of e-mail deletion is that we do not know whether or not such deletion actually happened!

The policy of the government of Ontario is to delete e-mail accounts when a member of staff leaves.  Craig MacLennan left the public service in June 2012.

The e-mail accounts of most of Ontario’s public servants are split over two tiers of storage.   E-mails less than 30 days old are kept on first tier storage  in Microsoft Exchange.  When an e-mail is thirty days old it moves to the portion of the e-mail account kept on cheaper, slower hardware, within an e-mail archive (Enterprise Vault by Symantec).

The Ministry of Government Services (MGS) stated that the e-mail archive existed purely to save storage costs, not to protect or preserve e-mail.   Whether a particular e-mail was stored in Exchange or in Enterprise Vault should not have made any difference to how long the e-mail was kept or how safe it was from deletion.   An individual member of staff could delete e-mail from either portion of their e-mail account.

In the summer of 2013 MGS forensic staff discovered that for a period of time IT staff had, by an administrative oversight, omitted to delete the Enterprise Vault portion of the e-mail accounts of staff who had left.  During the period when the policy was not applied 30,000 people had left,  so the Ministry was left with 30,000 orphaned accounts (orphaned because the only way of navigating to these accounts was through Microsoft Exchange, but the Exchange portion of the accounts had been deleted, leaving the Enterprise Vault portions of the e-mail accounts invisible and inaccessible).

The forensic staff checked through these 30,000 orphaned accounts and found that one of them belonged to Craig MacLennan.   When they opened MacLennan’s account they discovered it contained 38,000 e-mails including 1,900 relating to the gas plant controversy.

If MacLennan routinely deleted his e-mails as he went along, as he claimed; and if it was possible for an individual to use their Outlook e-mail client to delete an e-mail stored on second tier Enterprise Vault storage; then why were there there 38,000 e-mails left in the Enterprise Vault portion of MacLennan’s account after he left?

There are two alternative possible explanations for this,  either Craig MacLennan routinely deleted e-mails from his Outlook client, but did so in a way that did not cause those e-mails to be deleted from the Enterprise Vault e-mail archive.   Or  Craig MacLennan did not routinely delete e-mails at all.

3.3 Possible explanation 1:  Craig MacLennan deleted the e-mails from his Outlook e-mail client but did so in a way that did not cause them to be deleted from the Enterprise Vault archive

 In July 2012, a month after MacLennan left employment,  the Ontario government upgraded their e-mail server from Microsoft Excange 2003 to Exchange 2010.

  • Prior to the upgrade (and therefore during MacLennan’s spell of employment) it was possible for an individual to use their Outlook e-mail client to delete an e-mail over thirty days old from the Enterprise Vault  archive – but to do so they would have to use the Enterprise Vault tool bar that exists as a plug-in within their Outlook client.
  • Since the upgrade to Exchange 2010 (but after MacLennan left employment) it has been easier for an individual to use their Outlook e-mail client to delete e-mails from the Enterprise Vault archive.  There is still the option of using the Enterprise Vault toolbar within Outlook to delete.  But there is also now the option of simply selecting the e-mail within Outlook and pressing the delete key.

(This information is gleaned from the information given in the table, and in the footnote to the table,  provided on page 11 of the Information and Privacy Commissioner’s ADDENDUM to Deleting Accountability: Records Management Practices of Political Staff A Special Investigation Report)

3.4    Possible explanation 2:  Craig MacLennan did not routinely delete e-mails at all

It is possible that MacLennan might have preferred to be thought guilty of an inadvertent, non-malicious  breach of the Archives and Recordkeeping Act (which carries no penalties) than to be thought guilty of not producing records in response to a request from a Parliamentary Committee.    If MacLennan knew that the Minisitry of Government Services deleted e-mail accounts when staff leave, then he may have supposed that there would be no way of contradicting his claim to have routinely deleted his e-mail.

4.     The deletion of e-mail accounts when staff leave

4.1   The deletion of e-mail accounts when staff leave makes the question of whether or not MacLennan deleted e-mails from his account academic

Craig MacLennan’s reported deletions were:

  • of interest to opposition politicians hoping to prove that there was a conspiracy of political staff serving the Liberal administration to delete records of the gas plant cancellations AND
  • of interest to the Information and Privacy Commissioner from an access to information perspective because the routine deletion was the reason given by MacLennan for his non production of gas plant cancellation correspondence to Parliament.

But from a recordkeeping point of view the question of whether or not MacLennan deleted e-mails from his account  is academic. Even if MacLennan had kept every single one of his e-mails it would not have helped the Archivst of Ontario because the policy of Ontario’s  Ministry of Government Services was to delete staff e-mail accounts when the staff member leaves.

The discovery by forensic staff of the lion’s share of MacLennan’s account in an orphaned Enterprise Vault account does not mean that this correspondence was safe for the duration of the relevant retention rule.   The Ministry of Government Services had decided prior to the investigation to delete the orphaned accounts, in accordance with their policy, once an upgrade to the Enterprise Vault software had taken place.   They have since placed a stay of execution against e-mail accounts relevant to the gas plant saga.

4.2      The contradiction between Ontario’s retention rule on ministerial correspondence and its disposition policy on staff e-mail accounts

The retention schedule for ministerial public records signed by the Archivist of  Ontario states that ministerial correspondence should be transferred to the Archives of Ontario  for permanent preservation after five years (or upon a change of administration).

MacLennan stated that he never filed any e-mails anywhere.     This means that the only place ministerial correspondence of the Minister of Energy would be captured would be in the e-mail accounts of MacLennan and his colleagues (apart from copies scattered amongst the e-mail accounts of senders/recipients).    This in turn means that the e-mail accounts of Craig MacLennan and his colleagues should be of interest to the Archivist of Ontario.

So what policy should Ontario apply to the e-mail accounts of political staff:

  • the retention rule applying to ministerial correspondence that must make up a significant proportion (though not the entirety) of those e-mail accounts?  OR
  • the IT disposition policy of the Ministry of Government Services that e-mail accounts should be deleted when a member of staff leaves?

The only way those two policies could logically co-exist together would be if some sort of filing of e-mails, whether paper or electronic, was taking place.  No such practice existed in the Office of the Minister of Energy.

4.3   The impact of a blanket policy of deleting e-mail accounts when staff leave

From a recordkeeping perspective the policy of deleting e-mail accounts of all staff when they leave employment, however significant their role in public life,  is the most damaging of the three types of deletion described in this post.   It is this policy that most undermines the accountability of political staff and Ministers for their actions, and most undermines the effort to retain a record of the activities of Ministers and their staff over time.

This deletion of e-mail accounts when staff leave was not questioned by the Information and Privacy Commissioner in her report Deleting Accountability.  Nor has it been condemned by the Parliamentary Committees.  The politicians in the Committees have not  criticised this policy because it is a non-partisan policy – the Ministry of Government Services would have applied this policy regardless of the political complexion of the administration.

Political staff work in a high pace, dynamic environment.  Turnover of staff is relatively high.  They neither expect nor receive security of tenure. They are well placed to secure external jobs because of the value of their connections inside Government.   If a member of political staff thinks that the correspondence in their e-mail account would be damaging to themselves and/or the Minister then they can have that record expunged simply by leaving their employment.

5.     Attempts by political staff in the office of the premier to wipe clean hard drives    

The third deletion in the scandal came to light whilst the Information and Privacy Commissioner was conducting her investigation into Craig MacLennan’s reported routine deletion of his e-mail.  The Cabinet Secretary  told her that he had been approached by David Livingston, chief of staff to the  Premier of Ontario, at the time of the change of Premier from Dale McGuinty to Katharine Wynne.   Livingston wanted to know how to get administrator passwords to wipe the hard drives of the computers of departing political staff.

The cabinet secretary referred him onto the Chief Information Officer who was not too concerned about the proposed deletion, on the grounds that it was good practice to wipe clean devices when they were handed on, provided that the Office had complied with its obligations under the Archives and Recordkeeping Act.  He pointed out to Livingston that the Office already possessed the necessary adminstrative passwords.

Early in 2014  Ontario’s Police obtained a search warrant for the off-site storage vendor where the computers in question were being stored.  It is alleged that David Livingston, shortly after his approach to the Cabinet Secretary  had given the administrator passwords to the boyfriend of a member of the political staff, and asked the boyfriend to wipe the hard drives.   The boyfriend was not an Ontarian public servant.

At the time of writing no charges have been laid, no allegations have been proven in court and Livingston’s lawyer has denied any wrongdoing on his client’s behalf.

This deletion is important from a political perspective because it can be interpreted as showing that political staff were prepared to go to considerable lengths to delete records.   It is important to the police because criminal charges may be pressed.  But it is of little or no interest to an archivist or a records manager.     It is hard to believe that political staff were routinely using the hard drive of their computers for record storage.  It is too vulnerable to device failure, and does not generally support any form of mobile or remote access.    One presumes that the wiping of the drives was simply an attempt to defeat any forensic searches that might be made.

6 What should the Government of Ontario do to stop this type of saga recurring?

Most of the public debate on this saga has concentrated on the motivations and behaviour of individual political staff such as Craig MacLennan and David Livingston.  However from a recordkeeping perspective the question of whether or not the actions of these individuals were appropriate is of little importance..

Administrations will come and go, individual political staff will come and go.  Given the confrontational nature of the environment in which they work, we can assume that from time to time it will be in the interests of a member of a political staff to remove correspondence from the record.   Some people will succumb to that temptation, others will not.

From a recordkeeping point of view the most important question is this:

  • how can we best set up systems to routinely capture records of ministerial correspondence in ways that make it difficult for a member of political staff to remove correspondence from the record, or to prevent correspondence being captured onto the record in the first place?

Option 1 – make it as easy as possible for political staff to electronically file e-mails outside of their e-mail account 

One option would be to set up some sort of electronic document management system with e-mail integration so that political staff could simply drag and drop e-mail into folders within their Outlook client.   The folders could either be:

  • big buckets such as ‘ministerial correspondence’, ‘political correspondence’, ‘private and personal’, ‘trivia and ephemera.
  • or a more granular filing structure covering the themes and matters that the staff are dealing with

Such a solution would be an improvement on what they have at the moment (where it appears there is no simple means for staff to file e-mail).   It would have the advantage that once an e-mail had been dragged to a folder linked to a document management system it could be protected from subsequent deletion.

The weakness of  such a solution is that it is too dependent on the motivation and workload of the political staff themselves.  It leaves it down to political staff to decide what goes onto the record and what stays off the record.  This may be acceptable in a high trust environment.   However political staff operate in a low trust environment.   They are accountable to opposition politicians in the Parliament who do not and will never trust them.    Leaving political staff to decide what does and does not go onto the record is problematic, and I would not recommend this option.

The Ontarian government should aim to institute routines for capturing ministerial correspondence that will be trusted by opposition politicians even where they have no trust whatsoever in the individual politcal staff concerned.

Option 2 – automatically archive and preserve all e-mails sent and received by important political staff

The second option is to change the settings on the Enterprise Vault e-mail archive so that for designated members of political staff a copy of all e-mails sent and received is captured in the archive and protected from deletion.  This would mean disabling the ability of such individuals to use their Outlook client to delete an e-mail in the Enterprise Vault archive.

The precedent for this option is the rulings of the US Securities and Exchange Commission that all electronic communications of all broker-traders must be archived and protected.    Barclay T.Blair said in this post that these rulings (SEC 17 a-3 and SEC 17 a-4) ‘single-handedly created the e-mail archiving industry’.

There are strong parallels between the situations of  political staff and of broker-traders.  Both sets of people work in low trust, high scrutiny environments where there might be a powerful incentive to delete a communication from the record, or ensure that a communication did not go on the record.

However there is a key difference.  Unlike the correspondence of traders, the ministerial correspondence of political staff is needed for long term preservation in an historical archive.  But Ontario’s archivist has no legal right to archive the political correspondence of ministers, only their ministerial correspondence:

  • political correspondence is defined as correspondence arising from the political career of the Minister (for example his or her relations with their constituents, with their political party, and their election campaigning).   Ministers are free to dispose of such correspondence as they see fit.
  • ministerial correspondence comprises all correspondence arising from the Minister’s portfolio responsibilities and their role as a member of the cabinet

If Ontario archived and protected every e-mail sent or received by important political staff then they would have to find a way to sift out the political correspondence at the point in time at which the records are due to be transferred to the Archives of Ontario.

The Ontario government should institute a routine, trusted and simple method for political staff to separate out political correspondence from ministerial correspondence.

Recommended option – protect and preserve the e-mail accounts of important political staff –   but give them a means of flagging up  political correspondence and private correspondence

To stop this type of saga recurring I would recommend that the Government of Ontario take the following measures:

  • Designate the roles of certain political staff as being of high importance.  Ensure that all e-mails sent or received by such individuals are captured into an e-mail archive.  Protect the e-mails from deletion or amendment .  Disable the ability of individual staff to use their e-mail client to delete e-mails from the archive, or to amend those e-mails.
  • Find a means by which individuals whose accounts have been designated as of being of high importance can flag certain e-mails as private or personal.   Institute a random system of auditing to ensure that individuals are applying such a flag properly
  • Find a means by which political staff can flag certain e-mails as being political correspondence rather than ministerial correspondence. Institute a random system of auditing to ensure that individuals are applying such a flag properly
  • Retain e-mail accounts designated as being of high importance permanently.   Remove correspondence flagged as personal, and correspondence flagged as being political rather than ministerial, at the point in time at which the account is transferred to the Archives of Ontario.

The core components of the new generation of records management/information governance tools

In my last post I drew a distinction between two generations of records management tools:

  • The first generation of tools are those that hit the market between 1997 and 2009 and we called them electronic document and records management (EDRM) systems
  • The second generation are those that hit the market after 2009 and we seem to be calling them information governance tools

In this post I will look again at this distinction – this time comparing the components and capabilities of the old EDRM systems with the components and capabilities of the newer information governance tools.

The core components of the first generation of records management tools (EDRM systems)

The first generation of tools consisted of six core components/capabilities:

  • an end- user interface  to allow end-users to directly upload documents to the system
  • an integration with the e-mail client (usually Outlook) to allow end-users to drag and drop e-mails into folders within the system
  • document management features:  such as version control, check-in and check out, generic workflows and configurable workflow capabilities
  • a repository:  to store any type of documentation that the organisation might produce
  • classification and retention rules:  capability to hold, link together and apply a records classification (business classification scheme) and a set of retention rules
  • records protection – capability to protect records from amendment and deletion and maintain an audit trail of events in the life of that record

When implementing  such EDRM systems the records managers drew a ‘line in the sand’.  They aimed to implement  a system that would manage records going forward in time.  They did not attempt to deal with legacy content that had already accumulated on shared drives and in email.

The weakness of EDRM systems was that end users did not move all or most significant content into the records system.  Shared drives and e-mails continued to grow and continued to contain important content not captured into the records system.

Added to this a range of disruptions happened:

  • Microsoft’s entry into the content management space with SharePoint 2007 took away the collaboration space from the EDRM systems.   Unless they had complex requirements, organisations with SharePoint no longer needed the version control, check-in check out or workflow capabilities of the EDRM tools.
  • E- discovery/freedom of information/subject access enquiries caused more and more pain to organisations, and tended to focus on material in e-mail and shared drives rather than content in the EDRM
  • The move to smart phones and tablets made the user-interface problematic – smartphones have screens that are too small for the full functionality of an EDRM end-user interface.
  • The move to the cloud made e-mail integration problematic – cloud e-mail services do not allow customisation of their user-interface.

The seven core components of the new generation of records management/information governance tools

The second generation of records management tools, which we are calling information governance tools, consists of seven key capabilities:

  • Indexing engine  the ability to crawl and index content in many different applications and repositories (shared drives, SharePoint, e-mail servers, line of business systems etc)
  • Connectors  a set of connectors to the most common applications and repositories in use in organisations today (SharePoint, Exchange, ECM/EDRM systems etc).   The connectors enable the records system to take action on content in a target repository – for example to delete, move or place a legal hold on it.  They also enable the crawler to extract context to index.
  • Metadata enhancement and auto-classification the ability to add, through the connectors, extra metadata fields to content, and the ability to assign content to a classification either by setting rules based on parameters, or by using auto classification algorithms
  • Analytics dashboard to surface patterns in content repositories, for example to identify duplication, redundancy, trivia and high risk content
  • Classification and retention capability to hold and apply a records classification and a set of retention rules   – this is the main point of continuity between the first and second generation of records management tools.
  • In-place records management  the capability to protect records from amendment and deletion, maintain an audit trail of events in the life of that record, and assign a retention and classification rule to the record, even where the record is held in a different application than the records system itself.  From the end-user point of view this has the advantage that they can stay in the applications they are used to work in – they do not have to learn how to use the records system.
  • Repository  a repository to store any type of documentation that the organisation might produce .   The in-place records management features reduce,  but do not eliminate the need for a records repository.  Records repositories are necessary when an organisation wants to decommission an application, but still wants to retain the content from that application.  In cloud scenarios the repository comes in useful when the organisation wants the content to be available via a cloud application but not stored by the cloud provider

Notice what has been taken away and what has been added:

  • The components that an end-user interacted with – the end-user interface and the document management functionality, have either disappeared entirely or become an optional extra.
  • What comes in their place is the connectors,  indexing engine,  analytics and in-place records management capability necessary in order for a central administrator to understand and act on content held outside of the records system itself


The importance of the analytics dashboard

The key difference between the new generation of information governance tools and the old generation of EDRM systems is that the information governance tools pay as much (often more) attention to existing content as they do to shaping the way future content will accumulate.

The most stark illustration of the change is this:

  • ten years ago if you saw a system demonstration by a vendor at a records management event they would start by showing you their end-user interface for an individual to upload a document.
  • In 2014 a vendor will start by showing you their analytics dashboard

The analytics dashboard is the key to the new generation of  records management/information governance tools

Without the dashboard having an indexing engine crawling across shared drives, e-mail and SharePoint would be useless to the records manager.

The dashboard enables the records manager to actively interrogate the index to hone in on targets for action – information that should be deleted/moved/protected/classified/assigned to a retention rule etc.

392-analytics 1

A typical dashboard shows the records manager  how much content is held. where it is held, what file types there are, what departments it belong to,  what is redundant/outdated/trivial etc.   The dashboard also enables the records manager to use these different dimensions in connection with each other – for example to hone in on content of a particular department in a particular time period.

These are powerful tools in the hands of a central administrator, and it is important that they have workflows and audit trails in them so that:

  • the records manager can get the approval of content owners before making disposal decisions on content
  • the system can record that approval, and record the actioning of the decision

Note however that these tools are more effective at helping records managers make decisions on content that has build up in the shared drive and SharePoint environment than they are at dealing with content that has built up in e-mail accounts.

One of the challenges with EDRM systems was that it was very hard to measure benefit and give a tangible ROI.   The  business case for the new infromation governance tools often arises from savings produced by dealing with legacy data – something that the EDRM systems were not set up to do.  The ROI might come from:

  • savings from storage optimisation (moving less active content to second or third tier storage)
  • savings from reduction of content that has to be reviewed for eDiscovery/access to information requests

The benefits might be

  • capability to move content from legacy applications
  • capability to process the shared drives of functions acquired or divested in mergers and acquisitions

At the ARMA Europe conference last month Richard Hale from Active Navigation and Lee Meyrick from Nuix both gave presentations urging records professionals to be pragmatic and concentrate on targeting particular improvements one at a time.  The dashboard suits that approach – gone is the utopian wish to create a perfect records system, instead we have an incremental approach whereby a central administrators hones in on particular areas of content for protection/enhancement/migration.