Predictions for the application of machine learning to the management of email

Last year I gave two presentations, one to the DLM Forum Triennial and one to the IRMS conference, in which I developed a fictional case study of an organisation that decides to apply machine learning and analytics to email.

In my case study a public sector organisation:

  • is concerned about the low capture of email into its record system (SharePoint) and embarks on a programme to apply machine learning to remedy the shortfall;
  • uses machine learning to apply its existing policy of moving important email into a records system;
  • seeks to apply the machine learning capability on all email accounts on a corporate wide basis.

In reality I think that the attitude of public sector organisations to the application of  analytics and machine learning to email will be rather different to the attitude taken by the organisation in my case study.   My predictions are that public sector organisations in the UK:

  • will be reluctant to apply machine learning to email accounts because of the risks involved;
  • will be just as concerned about the prospect that the application of machine learning might result in very large volumes of email being captured into their record system as they would be about the existing under-capture of emails as records;
  • would use machine learning (or an analytics capability) to look for certain specific types of correspondence that are valuable to the organisation in certain specific accounts rather than applying machine learning/analytics to all accounts across the business;
  • would not move emails identified as important or valuable into their corporate records system, but would instead leave them within email accounts and either place them under a hold to prevent deletion, or move them to an email archive.

Here is the video of my monologue explaining how the organisation applying machine learning to all of its email accounts got on:


Managing email in Office 365

What is an email account in Office 365?   It is a special type of document library, that doesn’t need version control, doesn’t need extra metadata fields and doesn’t live in SharePoint.

It now seems a little incongruous for organisations to ask their staff to move important emails out of email accounts and into a ‘corporate record system’.  If Office 365 is your corporate record system then email accounts are within it already!

One of the things that Microsoft had to do in order to make Office 365 work as a service offering was to get their SharePoint team working with their Exchange team – something that famously never happened whilst both products were predominantly on-premise offerings.  Microsoft customers implementing on-premise SharePoint alongside their on-premise Exchange email system  had to deploy third-party plug-ins if they wanted staff to be able to drag and drop an email into a SharePoint document library without leaving their Outlook email client.

There are two routes Microsoft could have gone with the relationship between Exchange and SharePoint within Office 365:

  • the integration route – building in features that make it easier to move emails from Exchange to SharePoint;
  • the governance route – making common governance features available so that emails in an email account could be governed using the same policies as documents in a document library in SharePoint.

Microsoft’s choice of direction for Office 365 has implications for the policy decisions that organisations need to take on email:

  • If Microsoft were to go down the integration route then it would fit in with the records management belief that an email system is not a ‘record system’ but is instead a ‘communications tool’.  Many organisations over the course of the past decade have designated SharePoint as their corporate records system and asked staff to move important emails into SharePoint.
  • If Microsoft were to go down the governance route then it would fit with the information governance belief that distinctions between record systems and non-records systems are meaningless and unhelpful because organisations are under legal, regulatory and ethical obligations to manage all their business information systems in accordance with information governance principles.

From a marketing point of view, there are clear advantages to Microsoft from going down the governance route rather than the integration route:

  • If Microsoft went down the integration route it would imply that they viewed a SharePoint document library as a better place to store business email than an Exchange email account. This is despite the fact that Exchange was built for and designed around the storage of email messages, whereas SharePoint document libraries were not designed with email in mind.
  • By going down the governance route Microsoft can stay neutral on the question of whether an email is better stored in SharePoint or in Exchange, and can gradually remove any necessity for organisations to move emails out of Exchange and into SharePoint.

It is therefore no surprise to see Microsoft putting their emphasis on the governance route rather than the integration route.

Office 365 comes with a ‘Security & Compliance Centre’ that sits separately from SharePoint or Exchange or any of the other component parts of Office 365.   The Security & Compliance Centre gives you two different means of applying retention rules to content:

  • retention policies which are applied to the containers within which content sits (SharePoint sites, email accounts etc.);
  • retention labels which are applied to individual items of content (emails/documents etc.).

This effectively gives you three alternative options for applying retention to email:

  • apply retention policies to email accounts without applying retention labels; OR
  • ask end users to apply retention labels to emails (or automate the application of labels if and when you develop automation capability), without applying retention policies; OR
  • use a combination approach by applying a default retention policy to email accounts whilst allowing staff (or machines!) to apply a retention label to particular emails that deserve a retention rule that differs from the default.

Note that in applying retention from the Security & Compliance Centre to content in OneDrive, Office 365 groups or SharePoint you will be faced with variations of the three options listed above.   The variation relates to the type of container that you would be applying retention policies to, and the type of content that you would be applying retention labels to.

The fact that Microsoft allows an email account to be treated in the same way as a document library for retention purposes will not stop organisations wanting to apply different retention periods to email accounts than to document libraries even when they arise from a similar business function.  The cost and risk profile of an email account differs significantly from that of a document library.

However Office 365 is a game changer in two ways:

  • it brings the application of retention rules to email in email accounts firmly into the information governance, rather than the IT domain.  The retention policy  and retention label menus in the Office 365 Security & Compliance centre can be used to apply retention policies and/or retention labels to Exchange email accounts and SharePoint sites (as well as other parts of Office 365 including Office 365 groups and OneDrive accounts);
  • it creates the possibility of applying different types of policy towards email. For example if you wanted to apply a Capstone policy towards email you could do so out of the box in Office 365 by simply:
    • setting two retention policies on email; a Capstone retention policy for application to the relatively small number of email accounts that you wish to retain permanently, and a non Capstone retention policy for application to email accounts that you do not wish to retain permanently;
    • deploying  retention labels to enable staff with Capstone email accounts to identify trivial and personal emails so that those emails are exempt from the permanent retention applied to the rest of the correspondence in their email account.