Supervisory Review in Office 365

Security and compliance are big concerns for corporate customers moving to Office 365. As such, Microsoft is putting a lot of resources into ensuring that customers have the tools they need to ensure their data is kept securely and compliantly.

Recently, I went through the new Security and Compliance Center in Office 365 and wrote about the features and functionality that existed there. One section of the Security and Compliance Center that I did not dedicate much space to was a new feature called “Supervisory Review.” Today I plan to circle back and take a look at this new functionality in Office 365.

What is Supervisory Review in Office 365?

Supervisory Review is a new set of features in Office 365 that allows administrators to configure Office 365 to capture employee communications that meet specific criteria for examination by designated reviewers. The classic example of the need for this functionality is in financial services organizations. It has long been an industry requirement that an ethical firewall stand between those employees who trade securities and those who recommend securities to customers. Supervisory review policies are a way for management to enforce and monitor that ethical firewall.

At this time, Supervisory Review only captures communications via email. I would expect that this will be expanded in the future. As of this writing, Supervisory Review is still in preview, so the features and functionality may change.

Supervisory Review allows us to configure Supervisory Review Policies (SRP). To create a new SRP, you need to be able to gather the following information:

• Whose email do you want to monitor? You must be able to define the group of individuals you want to review. The easiest way to do this is via a distribution group, but you can also enter induvial user names or use dynamic distribution groups.
• What email do you want to review? You can certainly set a SRP to review all email for a specific set of users, but your reviewers will likely prefer a more limited set of criteria. You can limit your SRP to gather email based on where it is sent or received from (internal/external), or specific terms being found within the message or attachments.
• What sample size will be reviewed? Again, you can set your SRP to gather 100% of the email that meets specified requirements for review, or you can limit your SRP to a lesser percentage of the total messages for review.
• Who will be the designated reviewer?

How to set up a new Supervisory Review Policy

From the Office 365 Security & Compliance Center, navigate to Search & Investigation > Supervisory Review. Click on the + to create a new SRP.

Name your new SRP, and give it a description so anyone working with it later will understand your purpose.

On the next screen, add the users you want to fall under this SRP. I selected four individuals in the screenshot below, but the recommended method would be to use a distribution list.

On the next screen, you can choose conditions to trigger this SRP to capture messages for review. If no conditions are chosen, this SRP will capture all messages.

Here I selected “Message contains these words” and included “profit NEAR(4) guaranteed” in the words and also selected Outbound messages. This means if a message is sent out from my Office 365 tenant to someone, and that message contains the words “profit” and guaranteed” within 4 words of each other, it will be added to the list of messages for potential review.

On the next screen, I defined that I want Office 365 to randomly select 50% of the messages that match the above criteria for review. The idea being that reviewing half of the messages should give me a good idea if there is a problem. You can adjust the percentage of email meeting your query to any number between 1 and 100.

The next screen allows you to specify the reviewer. The reviewer does not have to be an email account in your Office 365 tenant. Here you can see I added myself and Tony Redmond as reviewers for this policy.

After I am happy with the setting for this SRP, hitting finish will create the SRP in my tenant. It does take some time for the new SRP to be provisioned, so don’t expect it to start working immediately. It took about five minutes for this test SRP to provision in my tenant, but your mileage may vary.

Once the new SRP is working, emails that match its criteria will be stored in the default discovery mailbox for your tenant. Messages are stored in a folder named for the policy.

Since it’s not really practical, or advised, to give all your reviewers direct access to the default discovery mailbox, Microsoft has created an app that can be installed in OWA for Supervisory Review. To install the app, run the following command in remote PowerShell connected to your Exchange Online tenant.

In the below screenshot, you can see my mailbox open in OWA to the folders for the SRP we just created. There is no email in there, but if there were I could review them and move them between the folders that were created for me.

There you have it. A rundown of how to create a new SRP and review the email captured by it. I’ll continue to monitor this new feature, and keep you up to date on any changes.