Monitoring the Office 365 End User Experience

With the use of the Office 365 platform, application monitoring requirements have changed. In your on-premises IT infrastructure, you normally monitor the availability of applications by solely checking server reachability. In some cases, you might choose to implement enhanced application monitoring by accessing dedicated protocol endpoints.

Examples of on-premises applications are:

• Exchange Server DAG
  • Monitoring of AutoDiscover, Outlook on the Web, Offline Address Book and Exchange Web Services (Http/Https)
  • Monitoring of message flow
  • Monitoring of database backups
    
    
• SharePoint Server Farm
  • Monitoring of SharePoint Sites (Http/Https)
    
    
• CRM / ERP
  • Monitoring of application web interface (Http/Https)
    
    
• Intranet
  • Monitoring of the intranet web presence (Http/Https)
    
    
• SQL Server
  • Monitoring of TCP port 1433

In addition,  IT Pros monitor default parameters like processor load, memory consumption, available disk space, and more.

The following diagram illustrates the centralized monitoring approach.

However, monitoring the service availability of Office 365 requires a different approach. With Office 365, your users use not only individual products but cloud services based on multiple cloud products. Therefore monitoring individual protocol endpoints is not sufficient.

Operating a hybrid configuration for Exchange Server, SharePoint Server, or Skype for Business with Office 365 comes with complex architecture. Hybrid operation with Office 365 automatically creates dependencies between the individual on-premises and cloud components.

The following simple example depicts a hybrid Exchange Server configuration with Exchange Online, which highlights the increased complexity.

In the cloud world, the combined services and products create new challenges for monitoring. Since you do not monitor individual server systems, but application endpoints, unlike the local IT infrastructure, IT Pros have less visibility and control. Instead, one must rely on the Microsoft-provided status information for the products and services in Office 365. At the same time, your users have expectations of the availability and features of Office 365. The failure of a single service can affect the availability, of other services. This is especially the case when, for example, Azure AD is not available as an authentication instance. Therefore, it is important to discover service disruptions and outages quickly.

What Does Microsoft Offer?

Microsoft provides information about the availability and potential disruption of Office 365 services through the Service Health Dashboard. This dashboard is part of the Microsoft 365 Admin Center and requires administrator permissions and accessibility of the Admin Center using a browser. If the Microsoft 365 Admin Center is not available, you will not be able to access the Service Health Dashboard.

The status information displayed on the Service Health Dashboard is not tenant-specific. So you can not tell from the information, whether or not your users are affected by a reported service degradation. Another shortcoming is that the detailed information always appears with a delay in the dashboard.

The second tool provided by Microsoft is: AAD Connect Health. However, this monitoring component also has limitations. Access to AAD Connect Health requires Global Authorization and Azure AD Premium licensing. The components only monitor Azure AD Connect and, if used, Active Directory Federation Services (AD FS).

Another source is Twitter. The Twitter account @ MSFT365Status provides information on the current operating status of Microsoft 365.

Problems You Should Be Aware Of

Microsoft’s Office 365 has had a rough year or two when it comes to service outages. While every outage is different, one thing remains constant – disruption to your organization.

Just think about the most recent outages. Were you prepared? How were first alerted? How did you respond? Were you able to calculate IT downtime and lost productivity among your end-users?

Read more about Microsoft service outages resources.

Disruptions on the Client-Side

Disruptions in the use of Microsoft's cloud services also result from poor or incorrect implementations in the on-premises IT infrastructure. These problems always occur when not following the recommendations of Microsoft. Approximately 52% of Office 365 relevant Microsoft support cases result from client-side issues. This applies equally to client devices, as well as server components of the hybrid infrastructure.

Common sources of on-premises issues are:

• Network: In the category network, there are numerous possibilities to create sources of error. One of the most common reasons for connectivity problems with Office 365 is the use of proxy servers, as not all protocols allow for proxied connections. Older proxy systems tend to filter protocol packets by mistake, disrupting communication. Even using modern active network components, e.g., Application Delivery Controllers (ADC) or WAN Optimizers can cause the same disruptions. Intrusion Detection Systems (IDS), that break TLS encryption, prevent secure communication with the Office 365 endpoints. The options described apply to client connections to Office 365 as well as server connections to Office 365.
  
  
• Hybrid Exchange Server (coexistence server): The hybrid configuration of Exchange Server and Exchange Online requires the use of at least one Exchange server as a connection point for the coexistence of both “Exchange worlds.” The implementation of the coexistence server often deviates from the clear recommendations of the Exchange product group. It's also easy to forget that coexistence consists of two components: client access (CAS) and message flow (MBX / EDGE). These two components require two different communication paths between the two worlds.
  
  
• Authentication: Authentication errors for accessing cloud services are common when using AD FS. As a result, users can not authenticate at all, or they receive incomplete authentication information for Office 365. This source of error occurs, especially in environments using multiple authentication domains.
  
  
• Configuration:In the field of network configuration, I must mention incorrect DNS entries, especially if you do not work with a split DNS configuration, but use different domain names internally and externally. Expired and incorrect use of SSL / TLS-certificates falls within this category as well.

Do you always follow the implementation recommendations for Office 365 from Microsoft?

Traditional Gaps in Application Monitoring

As previously mentioned, classic application monitoring through on-premises monitoring solutions has its gaps. Monitoring software solutions have been developed to monitor network components and servers. Such solutions use protocols suitable for monitoring, e.g., SNMP, WMI, SSH, or Http(s).

The monitoring of web-based applications is limited to checking the accessibility of a website URL. As long as the tested endpoint reports an OK, the endpoint is considered functional. While some solutions support logging on to the application, e.g., for Outlook on the Web, however, do not perform any further functional tests.

Modern Application Monitoring for Office 365

Advanced application monitoring for Office 365 requires monitoring of Office 365 products and services from the user's perspective, not from a server perspective. This approach is the only way to ensure that you get visibility into the availability and function of cloud services from the user's point of view.

ENow Mailscape 365 helps you to monitor the user experience by:

• Remote Probes for testing of cloud service client endpoints, connection status, and latencies
• Compliance with service level agreements for user services
• Location-based troubleshooting for cloud access taking into account local network infrastructure and hybrid components
• Interactive latency reports as a global performance summary for access to Office 365 services
• Synthetic tests to simulate user access to Exchange Online, SharePoint Online, Microsoft Teams, Skype for Business Online, and other workloads

The following diagram illustrates the use of ENow's Remote Probes in the branch offices. Each probe tests access to Office 365 workloads through the company's central data center in Vienna.

ENow also supports you with:

• Accelerate migration to Office 365 through migration reporting, network availability monitoring in sites, and feature testing of hybrid components (AAD Connect / AD FS)
• With the ENow Dashboard, you can keep track of the health of your on-premises, hybrid and Office 365 components, and quickly and easily identify malfunctions and find their causes
• Detailed evaluations help you to optimize the use of the licensed cloud services
• Optimize your Office 365 license management by cleaning up unneeded licenses and tracking license purchases

Summary

Using Microsoft 365 brings new challenges to service endpoint monitoring. Traditional monitoring solutions have natural limitations, as they can not perform detailed functional testing to simulate user actions. But it is these functional tests that enable you to make a meaningful assessment as to whether a service is functionally available or not. The technical monitoring of a protocol endpoint is not sufficient.

Microsoft provides basic information for service disruptions. However, communication takes place with a time delay and via interfaces that require special authorization to access. These interfaces are unsuitable for ensuring service level agreements or for service reporting.

With ENow, you get full visibility into the service availability of Office 365 workloads. The dashboard view helps to identify problems early and to identify faults and their respective causes. With detailed reporting, you can optimize license usage for better cost control on your acquired Office 365 licenses.

Office 365 End User Experience Monitoring and Reporting

To ensure your Office 365 collaboration goals are met, monitoring usage and adoption is just the start. Put your users at the heart of your monitoring strategy by measuring their ability to perform essential tasks within key Office 365 applications.

ENow’s Office 365 End User Experience Monitoring and Reporting:

• Mimics end-user behavior with synthetic transactions by proactively testing common actions
• Visually maps service status for a specific user in a specific remote location
• Pinpoints issues down to the root cause, decreasing MTTR

Gain true end-to-end visibility into how users experience Office 365 applications with ENow’s Office 365 End User Experience Monitoring and Reporting solution. Access your free 14-day trial today!