The Autodiscover Dilemma: Steps to Overcome It

Autodiscover was first introduced in Exchange 2007 and Outlook 2007 to quickly configure Outlook profiles, based on only the username and password. Outlook connects to the Exchange server, you enter your email address and password, and the Exchange server returns an XML package that Outlook uses to create or change its profile. The first implementation of Autodiscover was in Exchange 2007, but it is still used in Exchange 2019. Of course, there have been some improvements over the years, both in Exchange server as well as in Outlook, but overall the mechanism is the same.

There are several ways for Outlook to find which Exchange server it must contact, and that’s in this particular order:

1. SCP (Service Connection Point) in Active Directory.
2. Root Domain check.
3. Autodiscover DNS check.
4. Local Autodiscover.xml file.
5. HTTP Redirect.
6. SRV DNS records check.
7. Cached URL in the Outlook profile.
8. Direct connect to Office 365.

The first option, the SCP in Active Directory is most used for Active Directory domain joined clients. Outlook clients retrieve this record from Active Directory and in this record they find the location of the Exchange server they have to connect to, in order to use Autodiscover.

Non-domain joined clients, or domain joined clients use typically option three, which is a DNS lookup to a record like Autodiscover.contoso.com. the FQDN is constructed by the Outlook client, based on the hostname Autodiscover, followed by the domain part of the user’s Email address.

Local Autodiscover.xml files can be found in fancy interorg Exchange migration, HTTP Redirect and SRV DNS records are typically seen in organization that offer local Exchange hosting.

So far everything looks good, and everybody was pretty happy with this solution. Of course there were issues, but they were mostly related to DNS queries (Autodiscover record not available) or incorrect SSL certificates on the Exchange server (autodiscover.contoso.com not available on the SSL certificate). Aside from that, everybody was happy.

Office 365

Exchange Online is built on top of Exchange server 2016 and Exchange server 2019 and as such also uses the Autodiscover mechanism to configure Outlook clients. Of course, the first option is not possible, you cannot have an Office 365 domain joined client.

In an Exchange hybrid scenario, Autodiscover points to your on-premises Exchange environment (i.e. Autodiscover.contoso.com) and when a mailbox is moved to Exchange Online the Outlook client will find a target address pointing to Exchange Online. The Outlook client will follow this and will retrieve the Exchange Online configuration for the mailbox and configure the Outlook profile accordingly.

If you didn’t have any on-premises Exchange and only Exchange Online mailboxes, your Autodiscover DNS record would be a CNAME record and point directly to Exchange Online (i.e. Autodiscover.outlook.com).

Office 2016 Click-2-Run (C2R)

Office 2016 C2R is the Office version that is available in the Microsoft cloud. It is available for users with business subscriptions, home subscriptions, personal subscriptions or Office 365 ProPlus. Microsoft assumes that everyone with some kind of Office 365 subscription that is using Outlook, is using Exchange Online as well. And that’s a reasonable assumption, provided Outlook is following the normal Autodiscover process.  If it is, then everybody should still be happy.

But . . . . after the release of Outlook 2016 version 16.0.6741.2017 (C2R) things have changed.  Microsoft has decided that, since everybody using C2R is using Exchange Online anyway, the order of Autodiscover must be changed. The order is changed in such a way that the Outlook 2016 client starts looking for a Mailbox in Exchange Online, regardless if there is a Mailbox or not.

So, the order for Office 2016 C2R is now suddenly:

1. Check for restart scenarios (rare).
2. Local Autodiscover.xml file by using GPO
3. Last Known Good (LKG) data.
4. Check for Office 365 as priority.
5. SCP record in Active Directory.
6. Root Domain check.
7. Autodiscover DNS check.
8. Local Autodiscover.xml file (when GPO in step 2 fails)
9. HTTP Redirect.
10. SRV DNS records check.
11. Check for O365 as Failsafe (when all other options fail)

Now, where does it hurt?

The first three steps seldom give any results, especially when one does not expect this behavior change. But then Outlook starts checking Office 365 for existence of a Mailbox for this user by hitting the Exchange Online Autodiscover service. The Autodiscover service is always running, so the Outlook client automatically presents a pop-up requesting for credentials. If there’s a Mailbox in Exchange Online then all is well and Outlook is configured, but if there isn’t (and still a whopping 75% or so do not have a Mailbox in Office 365) the authentication requests fail and the pop-up is presented again. And again, and again, until the user starts calling the helpdesk that his/her Outlook is not functioning anymore.

Except for the annoyance of the user and the helpdesk, think about the compliance impact of this change. Suppose you are in an organization that’s not allowed to do anything, or store anything in the cloud and your credentials are suddenly exposed to Exchange Online. I won’t say this is a directory harvesting issue, but there are certainly Security Officers out there in the field that say that this is a security breach. Personally, I am currently working with a customer who is in a Proof of Concept with Office 365 and hitting the authentication pop-ups. Luckily, I knew that this behavior has changed recently, but my colleagues did not know and were looking for a cause for a couple of days. That’s horrible.

Another nasty example is when you are testing Office 365 next to your existing on-premises Exchange environment. Imagine a scenario where Outlook presents an authentication pop-up, and the user enters his credentials. Next Outlook unexpectedly connects to a test Mailbox in Office 365 instead of their on-premises Exchange mailbox. The user doesn’t notice the difference immediately and starts working and sending out emails from the Office 365 mailbox. Again, besides the annoyance, think about possible legal implications when this happens. And the financial cost to fix this when it happens to several users and helpdesk staff needs days to get everything back to its original state.

One last example pertains to organizations that offer local hosted Exchange solutions. These typically depend on the HTTP redirect or SRV DNS records for Autodiscover. Suddenly, without anyone knowing, this behavior changes and users are presented a pop-up without the possibility to logon anywhere. A nightmare for the hoster’s helpdesk!

How to Overcome this Challenge

To avoid this you can use registry entries to control the behavior of Outlook as outlined in the article Unexpected Autodiscover behavior when you have registry settings under the \Autodiscover key on the Microsoft support site. If you add the ExcludeExplicitO365Endpoint DWORD under the HKEY_CURRENT_USER\Software\Microsoft\Office\16.0\Outlook\AutoDiscover subkey and give it a value of “1”, the Outlook client will skip the Office 365 endpoint check and continue with the regular Autodiscover process.

This works fine for domain joined clients or when using SSCM managed clients, but Intune managed client do not have this functionality (to my knowledge) and for non-domain joined clients this key should be added manually through the use of a .reg file.

Summary

Needless to say that this change was badly received by IT Admins and caused a lot of resistance. One customer submitted an idea on UserVoice to change this behavior, but the request was not fulfilled and Microsoft will continue to optimize for Office 365. This in turn caused more frustration for customers and MVPs.  After raising this to the appropriate Program Managers for Outlook and Exchange, it is again under investigation.

To be continued . . . .  but for now, the only thing you can do is workaround these issues using the discussed registry keys. Please note, this only occurs with the Click-2-Run version of Outlook 2016; regular versions are not impacted.

Exchange Hybrid and Office 365 Monitoring and Reporting

On-premises components, such as AD FS, PTA, and Exchange Hybrid are critical for Office 365 end user experience. In addition, something as trivial as expiring Exchange or AD FS certificates can certainly lead to unexpected outages By proactively monitoring hybrid components, ENow gives you early warnings where hybrid components are reaching a critical state, or even for an upcoming expiring certificate. Knowing immediately when a problem happens, where the fault lies, and why the issue has occurred, ensures that any outages are detected and solved as quickly as possible.

Access your free 14-day trial of ENow’s Exchange Hybrid and Office 365 Monitoring and Reporting today!