The Information and Records Management Society’s Microsoft 365 Customer Advisory Board Working Group provides a channel through which the records management community can work up feature requests to pass to Microsoft in an effort to influence and inform the company’s future roadmap for the product.
In 2023 the Working Group passed two sets of feature requests to Microsoft, and I reproduce one of them in this post. It is a set of feature requests that aims to make it easier for an organisation to apply a role-based approach to records retention within Microsoft 365.
About role-based retention
Role-based retention has emerged from the Capstone approach to email, developed by the US National Archives and Records Administration (NARA) in 2013.
The working definition of role-based retention used in this set of requirements is that it is an approach to applying retention rules in which:
- Individual accounts are assigned to a short, medium or long term retention rule depending upon the relative importance of the role of the individual to whom they belong;
- Team sites and spaces are assigned to a short, medium or long term retention rule depending on the relative importance of the role of the team to whom they belong.
Like any approach to retention, role-based retention has strengths and weaknesses.
- Its main strength is that it uses the existing way that content is aggregated within the communication and collaboration applications of a cloud suite such as Microsoft 365;
- Its main weakness is that those aggregations (individual accounts and team sites/spaces) tend to cover a broad range of activities. Individual accounts may contain trivial, social and personal material alongside business content.
About the AI models specified in the set of requirements
The set of requirements set out below aims to build on the strengths and mitigate the weaknesses of the role-based retention approach. It does so by requesting AI models that work to:
- Identify trivial, personal, and social items within individual accounts that have no need to be protected by retention policies;
- Cluster together (via a common tag) material within an individual account that arises from the same business activity.
It is likely that social, trivial and personal content will manifest itself in different ways, and have different give-away features, in the different applications of the Microsoft 365 suite. The aggregations in the suite likely to have the most such material are individual accounts within Exchange/Outlook, OneDrive, and Teams Chat. The set of requirements therefore specifies a different model for each of these applications. This gives Microsoft the option of asking the different product groups responsible for these applications to each develop an AI model to identify social, trivial and personal material within individual accounts within that application.
Similarly it is individual accounts that are most likely to benefit from a facility to cluster content arising from the same type of business activity. Again the features of content arising from a similar activity are likely to manifest themselves differently in each different application, so a separate requirement has been arrived at for Exchange/Outlook, for Teams Chat and for OneDrive.
About this set of requirements
What follows is a set of requirements that I drafted, in the context of my work at the UK Central Digital and Data Office, and in discussion with colleagues in the UK National Archives (TNA), the US National Archives and Records Administration (NARA), and the Public Record Office of Victoria (PROV).
The list of requirements reproduced below comes with two caveats:
- It does not constitute a standard. Role based retention is one approach to records retention. Other approaches to retention exist.
- This is a set of feature requests to a technology company. It does not constitute advice, guidance or recommendations to organisations who use Microsoft 365. Organisations need to decide for themselves which approach to retention to apply to which applications within Microsoft 365, on the basis of their business context and on how those applications are used by their staff.
The set of requirements covers the main workloads of Microsoft 365: Exchange/Outlook (email); SharePoint and OneDrive; Teams and Teams Chat. It attempts to answer the question as to what features and AI models would best support the application of a role-based retention policy in each of these applications/workloads. In practice those organisations that apply a role based retention policy are likely to pick and choose which applications they will apply it to. The presence of requirements for functionality to support role-based retention within a particular application/workload should not be interpreted as a recommendation that organisations apply the approach within the workload – that is a decision for each organisation.
The remainder of this post reproduces the set of requirements that was submitted to Microsoft by the IRMS Microsoft 365 CAB working group.
0.Introduction
0.1 About role-based records retention
The role-based approach to records retention is an attempt to generalise out the Capstone approach to email so that it has the potential to be applicable across any system/application that aggregates content by individual or team. The Capstone approach to email was developed by the US National Archives and Records Administration (NARA) in 2013.
A role-based approach to retention assigns individual and/or team based aggregations to a short, medium or long term retention band on the basis of the relative importance of the role of the individual or team that own it.
Role-based retention is one approach to records retention. Other approaches to retention exist. This requirements list does not attempt to recommend or mandate the approach, rather it attempts to list the features and algorithmic models that would make it possible for an organisation, if it so wished, to apply the approach in one, two, several or all of the main workloads of Microsoft 365.
0.2 About this requirements list
This document was developed by the UK Central Digital and Data Office (CDDO), in conversation with the UK National Archives (TNA), the US National Archives and Records Administration (NARA), and the Public Record Office Victoria (PROV). It was developed with a view to feed into discussions of the Information and Records Management Society (IRMS) IM Tech group, and of the group’s working group for the M365 Customer Advisory Board that is run by Microsoft with the IRMS.
This set of requirements seeks to:
- Increase understanding of the role-based approach to retention and what it entails in practice;
- Prompt debate about how role-based retention can best be supported inside Microsoft 365;
- Set out the features and algorithmic models that would support an organisation in applying role-based retention across the major applications within M365;
- Provide a basis for advocacy and dialogue with Microsoft and its ecosystem.
However, this set of requirements:
- Does not represent the policy approach of any actual organisation;
- Is not intended to express the policy position of any of the contributors to this document;
- Does not constitute records management advice to any organisation.
1. Retention bands
One way of streamlining the deployment of a role-based retention approach is to assign each individual (and each team or working group) to a retention band. This is in order to enable each individual account (and/or each team site) to inherit the retention rule applying to the band that the individual (or team) has been assigned to.
Requirements
| 1.1 | Provision of the ability for an administrator to set up three or more retention bands, each with a default retention rule. |
| 1.2a | Provision of the ability for an administrator to allocate any, several or every individual end-user to one of the retention bands. |
| 1.2b | Provision of the ability for an administrator to allocate any, several or every organisational unit/working group to one of the retention bands. |
| 1.3 | Provision of the ability for an administrator to access a central record of which individuals and which organisational units have been allocated to which retention band (and between what dates). |
| 1.4 | Provision of the ability for an administrator to move an individual from one band to another if they take on a new role which merits a different retention band. |
| 1.5 | Provision of a mechanism whereby the change of an individual from one retention band to another (for example after they take on a new role with a very different level of responsibility) results in any content created or received by them in any of their individual accounts from that point in time onwards receiving the default retention rule inherited from their new band. |
| 1.6 | Provision of the ability for an individual end-user to access information about what retention band they have been assigned to, and about what retention band the organisational unit(s) they belong to has been assigned to. |
| 1.7 | Provision of the ability for an administrator to configure the dynamic assignment of individuals to retention bands on the basis of properties in their Active Directory profile. |
2. Allocating default retention policies to containers
Role-based retention involves the assignement of a default retention rule to an aggregation (container) based on the business content within the aggregation. The default rule reflects how long the business content within that aggregation is likely to be of value to the organisation (for operational and/or accountability purposes).
Requirements
| 2.1 | Provision of a mechanism to ensure that each individual account inherits the default retention policy that applies to the band that the individual has been assigned to. |
| 2.2 | Ability for an administrator to exempt a particular individual account (or set of individual accounts) from inheriting its retention policy from the band to which the individual has been allocated. |
| 2.3 | Provision of a mechanism to ensure that each team collaboration site inherits the default retention policy from the band to which the organisational unit mainly responsible for it has been allocated. |
| 2.4 | Provision of the ability for an administrator to exempt a particular collaboration site (or set of collaboration sites) from inheriting its retention policy from the band to which the organisational unit mainly responsible for it has been allocated. |
| 2.5 | Provision of a mechanism to ensure that the retention policy inherited by an individual account (see 2.1) or a collaboration space (see 2.3) from the band that it has been assigned to cannot be overriden by any other retention policy applying to the container unless an administrator first allows the inheritance to be broken. |
| 2.6 | Provision of the ability for an individual end-user to view the retention policy applying to each of their individual accounts. |
| 2.7 | Provision of the ability for an individual end-user to view, for each collaborative site to which they have access, which retention policy applies to that site. |
| 2.8 | Provision of the ability for an administrator to view a list of all individual accounts that have been exempted (under 2.2) from inheriting its retention policy from the band that the individual has been allocated to. |
| 2.9 | Provision of the ability for an administrator to view a list of all collaborative sites that have been exempted (under 2.4) from inheriting a retention policy from the retention band that the individual/organisational unit mainly responsible for it has been allocated to. |
| 2.10 | Provision of the ability for an administrator to prevent any messages within an individual end-user’s Chat account from being acted upon by a retention policy or retention label originating from the account of a party with whom they have been in conversation. |
3 Using retention labels to override default policies
An organisation deploying a role-based retention approach would benefit from the capability to identify material within an individual account (or team site) that does not warrant being retained as long as the business content of the individual (or team/working group). They would benefit from the capability to assign a retention period to such content that is shorter than the default for the account/site, and to have that shorter retention period override the default retention policy for the account/site.
Requirements
| 3.1 | Provision of the ability for an administrator to define, for any particular container, any particular set of containers, and/or any particular type of containers, which retention labels (if any) are permitted to override the retention policy applying to that/those containers. |
| 3.2 | Provision of the ability for an administrator to prevent a retention label from overriding a retention policy set on a container, or on a defined set of containers, or on a type of containers, unless an administrator had configured the container (as per 3.1) to specifically allow its retention policy to be overriden by that label. |
| 3.3 | Provision of the ability for an administrator to configure a container, a set of containers and/or a defined type of containers to permit a particular retention label to always override the default retention policy. |
| 3.4 | Provision of the ability for an individual end-user to view, for each of their individual accounts, which retention labels (if any) can override the retention policy applying to the account. |
| 3.5 | Provision of the ability for an individual end-user to to view, for each collaborative site to which they have access, which retention labels (if any) can override the retention policy on the site. |
4. Identifying trivial content within individual accounts
4.1 Identifying trivial content within email accounts
An organisation applying a role based approach to email would benefit from being able to identify trivial, personal, social and unsolicited emails that do not warrant being retained for as long as the business correspondence within the email account of each individual. For such messages the organisation would benefit from being able to override the default retention policy assigned to the email account with a shorter retention rule applied via a label. Such an organisation might also seek to find a pathway to enable an individual new-to-post (and any generative AI acting on their behalf) to access the business correspondence in their predecessor-in-post’s email account, with the permission of the outgoing post-holder, and without access being provided to any personal or social emails.
Requirements
| 4.1.1 | Provision of the ability for an administrator to automatically assign a retention label to all emails within an email account that the Focused Inbox algorithmic model assigns to the ‘Other’ tab. |
| 4.1.2 | Provision of an algorithmic model that runs over content within an email account that has been assigned to the ‘Focused’ tab, and makes a distinction between: – correspondence that has arisen from the individual’s social/personal life; – correspondence that has arisen from the account holder’s business role. |
| 4.1.3 | Provision of the ability for an administrator to automatically assign a retention label to all messages identified as social or personal by the algorithmic model specified in 4.1.2. |
| 4.1.4 | Provision of the ability for an end-user to correct, reinforce and/or retrain any AI model developed and deployed within their email account in fulfillment of requirement 4.1.2. |
| 4.1.5 | Provision of a means for an individual end-user to proactively flag up content within their account that is personal to them. |
| 4.1.6 | Provision of a means for an individual end-user to indicate that they are happy for a successor-in-post (and any generative AI model acting on their successor-in-post’s behalf) to see correspondence that has been identified (by the model specified in 4.1.2) as arising from their business role. |
4.2 Identifying trivial content within Teams Chat accounts
An organisation applying a role based approach to Teams Chat would benefit from being able to identify conversations that do not warrant being retained for as long as the business conversations within the account of the individual. For such conversations the organisation would also benefit from being able to override the default retention policy assigned to the Chat account with a shorter retention rule applied via a label. Such an organisation might also seek to find a pathway to enable an individual new-to-post (and any Copilot generative AI acting on their behalf) to access the business conversations in their predecessor-in-post’s Chat account, with the permission of the outgoing post-holder, and without access being provided to any personal or social conversations.
Requirements
| 4.2.1 | Provision of a means for an individual end-user to proactively flag conversations within their Teams Chat account that are predominantly personal. |
| 4.2.2 | Provision of an algorithmic model that identifies (and tags or labels) conservations that the individual did not contribute to (for example Chat conversations in meetings that an individual attended). |
| 4.2.3 | Provision of an algorithmic model that labels each conversation within an individual’s Chat account according to whether it is predominantly a business conversation or predominantly non-business [trivial, business, social or personal]. |
| 4.2.4 | Provision of a means for an administrator to ensure that conversations within a Chat account that are flagged as not relating to the individual’s business role are assigned a shorter retention period than the default assigned to the account. |
4.3 Identifying trivial content within OneDrive accounts
An organisation applying a role-based approach to OneDrive would benefit from being able to identify documents that do not warrant being retained for as long as the business documents created by the individual. Such documents include trivial documents and documents that the individual has not shared with any colleagues. Such an organisation might also seek to find a pathway to enable an individual new-to-post (and any Copilot generative AI acting on their behalf) to access the business documents in their predecessor’s OneDrive account, and that have been shared with their predecessor. This would be dependent on the predecessor-in-post granting permission, and on access being denied to personal, social and trivial documents.
Requirements
| 4.3.1 | Provision of the ability to identify documents/content within the OneDrive account that have not been shared with others (by direct sharing with others, or as attachments to messages, or as Loop components, or by any other means). |
| 4.3.2 | Provision of the ability for an administrator to apply a retention rule to documents within a OneDrive account that have not been shared with others that is shorter than the default for the account. |
| 4.3.3 | Provision of the ability for an individual end-user to proactively flag up documents that are predominantly social, personal or trivial. |
| 4.3.4 | Provision of an algorithmic model that labels each document within a OneDrive account as being predominantly business or as predominantly non-business [trivial, social or personal]. |
| 4.3.5 | Provision of a means for an administrator to ensure that documents within a OneDrive account that are flagged as not relating to the individual’s business role are assigned a shorter retention period than the default assigned to the account. |
5 Clustering correspondence within individual accounts
5.1 Clustering correspondence within individual email accounts
An organisation deploying a role based retention approach to email would stand to benefit from an algorithmic capability to cluster together (for example by assigning a common tag) items of correspondence within an email account that have arisen from the same business activity. This would enable an organisation to apply different retention rules to correspondence arising from different activities of the individual. It would also open up the possibility of an end-user allowing their successor (and any Copilot generative AI working on their successor’s behalf ) to access correspondence from some aspects of their activities (but not others).
Requirements
| 5.1.0 | Provision of an algorithmic model to create clusters within each email account of correspondence that has arisen from a similar business activity. The model would identify messages that arose from similar areas of work and assign them a common tag. |
| 5.1.1 | Provision of a means for an end-user to rename a cluster created within their email account. |
| 5.1.2 | Provision of a means for an individual end-user to indicate whether a cluster created within their email account by the algorithmic model specified by 5.1.0 was useful (or not) to them. |
| 5.1.3 | Provision of a means for an individual end-user to re-assign messages from one cluster within their email account to another. |
| 5.1.4 | Provision of a means for an individual end-user to add an item to a cluster within their email account. |
| 5.1.5 | Provision of a means for an individual end-user to remove an item from a cluster within their email account. |
| 5.1.6 | Provision of a means for an individual end-user to indicate whether they are happy for an administrator to know of the existence of a cluster within their email account. |
| 5.1.7 | Provision of a means for an individual end-user to indicate to an administrator whether they are happy for a successor-in-post (and any generative AI model acting on behalf on their successor’s behalf) to access a cluster of correspondence created within their email account. |
| 5.1.8 | Provision of a means for an administrator to provide a person new-in-post (and any generative AI model acting on their behalf) access to a cluster of correspondence from their predecessor, providing that their predecessor had confirmed their consent (as specified by 5.1.7). |
| 5.1.9 | Provision of a means for an administrator to assign a retention label to a cluster created within an email account by the algorithmic model specified by 5.1.0. |
5.2 Clustering correspondence within Teams Chat accounts
An organisation deploying a role based retention approach to Teams Chat would stand to benefit from an algorithmic capability to cluster together (for example by assigning a common tag) conversations within a Chat account that have arisen from the same business activity. This would enable an organisation to apply different retention rules to conversations arising from different activities of the individual. It would also open up the possibility of an end-user allowing their successor-in-post (and any Copilot generative AI working on their behalf) to access conversations from some aspects of their activities (but not others).
Requirements
| 5.2.0 | Provision of an algorithmic model running within each individual Teams Chat account, that could identify conversations that arose from similar areas of work and assign them a common tag or label, to create a ‘cluster’. |
| 5.2.1 | Provision of a means for an individual end-user to rename a cluster that was created within their Teams Chat account. |
| 5.2.2 | Provision of the means for an individual end-user to indicate whether a cluster created in their Teams Chat account was useful (or not) to them. |
| 5.2.3 | Provision of a means for an individual end-user to re-assign conversations within their Teams Chat account from one cluster to another. |
| 5.2.4 | Provision of a means for an individual end-user to add a conversation to a cluster in their Teams Chat account . |
| 5.2.5 | Provision of a means for an individual to remove a conversation from a cluster in their Teams Chat account. |
| 5.2.6 | Provision of a means for an end-user to indicate whether they are happy for an administrator to know of the existence of a cluster within their Teams Chat account and to see the title of that cluster. |
| 5.2.7 | Provision of a means for an individual end-user to indicate whether they are happy for a successor in post (and any generative AI model within Microsoft 365 acting on their successor-in-post’s behalfI) to access a cluster of conversations within their Teams Chat account. |
| 5.2.8 | Provision of a means for an administrator to allow a successor-in-post (and any generative AI acting on their behalf) to have access to a cluster of conversations from their predecessor, providing that their predecessor had confirmed their consent (as specified by 5.2.7). |
| 5.2.9 | Provision of a means for an administrator to assign a retention label to a cluster of conversations created within a Chat account by the algorithmic model specified by 5.2.0. |
5.3 Clustering documents within a OneDrive account
An organisation deploying a role based retention approach to OneDrive may stand to benefit from an algorithmic capability to cluster together (for example by assigning a common tag) documents within a One Drive account that have arisen from the same business activity. This would enable an organisation to apply different retention rules to documents arising from different activities of the individual. It would also open up the possibility of an end-user allowing their successor (and any generative AI model working within Microsoft 365 on their successor’s behalf) to access documents from some aspects of their activities (but not others).
Requirements
| 5.3.0 | Provision of an algorithmic model running within each individual OneDrive account, that could identify documents that arose from similar areas of work and assign them a common tag or label, to create a ‘cluster’. |
| 5.3.1 | Provision of a means for an individual end-user to rename a cluster within their OneDrive account. |
| 5.3.2 | Provision of the means for an individual end-user to indicate whether a cluster within their OneDrive account is useful (or not) to them. |
| 5.3.3 | Provision of a means for an individual end-user to re-assign documents within their OneDrive account from one cluster to another. |
| 5.3.4 | Provision of a means for an individual end-user to remove a document from a cluster within their OneDrive account. |
| 5.3.5 | Number not used. |
| 5.3.6 | Provision of a means for an individual end-user to indicate whether they are happy for an administrator to know of the existence of a cluster within their OneDrive account and to see the title of that cluster. |
| 5.3.7 | Provision of a means for an individual end-user to indicate whether they are happy for a successor-in-post (and any generative AI model acting on their successor-in-post’s behalf) to see a cluster of documents within the OneDrive account. |
| 5.3.8 | Provision of a means for an administrator to allow a successor-in-post (and any generative AI acting on their behalf) to access a cluster of documents from their predecessor, providing that their predecessor had confirmed their consent (see 5.3.7). |
| 5.3.9 | Provision of a means for an administrator to assign a retention label to a cluster of documents within a OneDrive account. |
6 Preserving the connection between messages and any attachments
6.1 Preserving the connection between Teams Chat messages and any attachments
Where a Chat message is sent with a document attachment, the message forms an important part of the metadata of the document and the document forms an important part of the content of the message. An organisation applying a role-based approach to the retention of Teams Chat messages would need a way to ensure that any message stayed linked to a document attached to it, even if the message and document were stored in separate repositories.
Requirements
| 6.1.1 | Provision of a means of ensuring that there is a persistent connection between chat messages and any documents that were exchanged within them. |
| 6.1.2 | Provision of a location for the storage of an individual’s chat correspondence if and when an administrator decided to no longer retain their Exchange email account (whether for reasons of not having to continue to pay the licence fee or other reasons). |
| 6.1.3 | Provision of a means for an administrator to treat an individual’s chat conversations and any attachments posted within them as one aggregation for retention purposes. There must be a way of maintaining the link between chat messages (stored in a hidden folder associated an individual’s email account) and any documents exchanged via Teams Chat, even after an individual has left the organisation and even if an organisation has moved the chat correspondence to a new location and/or disabled the email account it was formerly stored in. |
| 6.1.4 | Provision of a means of ensuring that, if an individual’s Teams Chat account is subject to a longer retention period than their OneDrive account, then the folder of documents within their OneDrive account that contains documents they have sent via Teams Chat is not destroyed for as long as the Chat account is retained. |
6.2 Connection between Teams channel posts and attachments
Where a Teams channel posting contains a document attachment, the post forms an important part of the metadata of the document and the document forms an important part of the content of the post. An organisation applying a role-based approach to the retention of Teams would need a way to ensure that any post stayed linked to a document attached to it, even if the post and document were stored in separate repositories.
Requirements
| 6.2.1 | Provision of a means of ensuring that a persistent connection is maintained between documents (stored in SharePoint) that have been posted through standard channels, private channels and shared channels, and the channels that they were posted through. |
| 6.2.2 | Provision of a means of ensuring that if a SharePoint site is hosting documents that have been posted through Teams channels, that the documents are retained for as long as the channel posts are retained. |
7 Decoupling the storage of email from licence costs
When an individual leaves post the organisation is likely to wish to decommission their email address, but may need to retain their email correspondence.
Requirements
| 7.1.1 | Provision of a way of deactivating an email account that: – Leaves open the possibility that it will be possible to make parts of the content accessible (with the individual’s permission) to their successor-in-post and their successor-in-post’s generative AI; – Does not incur a licence fee; – Does not require content to be manually moved. |
Thinking Records 