Hybrid Exchange deployments: Highlights from Ignite

Last week during Microsoft's Ignite conference, I had the pleasure to co-present a session with Timothy Heeney on hybrid Exchange deployments. For those who weren't able to attend Ignite, the recording of that session is available here. During our session, Tim spoke about how Microsoft tests hybrid deployments and the tools it (recently) released to help you troubleshoot hybrid deployments. He also announced some pending changes to the face of the Hybrid Configuration Wizard.

Last year, Microsoft broke cross-premises free/busy information twice by changes made to the MFG infrastructure, which prevented customers from requesting a token because the customer federation metadata was invalidated. To solve the problem, customers had to manually run a cmdlet to update their federation information with the MFG before free/busy started working again. Microsoft also had some “quality” issues with Exchange 2013: A Cumulative Update impacted hybrid deployments or the ability to run the Hybrid Configuration Wizard in multiple ways.

The point here is that the speed at which Office 365 evolves is visibly interfering with the customer experience because the on-premises components are not updated at the same rate. For instance, when the issue with Cumulative Update 5 was detected, it took Microsoft several days to come up with an interim update to fix the issue. Cumulative Update 6 fixed the issue for all customers but introduced another glitch, which prevented the HCW to run under certain circumstances. The underlying problem is that Microsoft can only release an update to the Hybrid Configuration Wizard in a Cumulative Update or as an interim update (hotfix) if the issue is really disruptive. Needless to say, this hurts Microsoft’s ability to evolve hybrid deployments because it must factor in the update cadence for on-premises deployments.

As a “countermeasure,” Microsoft announced it would replace the existing Hybrid Configuration Wizard with a standalone version. The experience from running the standalone version would be very similar to the experience today: An administrator would still launch the wizard from within the Exchange Admin Center, but instead of loading the wizard from the locally installed Exchange server, the wizard would be loaded from Configure.Office.com — very similar to how the oAuth configuration wizard runs now. The wizard itself would not change much either. It's basically the same wizard run from a remote platform.

Image (mock-up) courtesy of Microsoft. Source: BRK3139.

The benefit of this approach is that Microsoft can make as many updates to the "cloud-based" Hybrid Configuration Wizard as it likes because it's now running from a system it owns and controls. As a result, customers always get the latest version and best experience possible, which, of course, should also minimize the problems caused by mismatched versions used in Office 365 and on-premises. The new experience will be made available in the future for Exchange 2016 and Exchange 2013 customers. Customers still running Exchange 2010 hybrid deployments will continue to use the Exchange 2010 Hybrid Configuration Wizard. As Tim pointed out, this is still very early information and details are subject to change between now and when the standalone wizard is released to the public.

I'm very excited about this change. I've lived through the Cumulative Update pain myself and applaud Microsoft's efforts to improve the customer experience in case problems arise. Having the standalone wizard will greatly reduce the time between an issue being discovered and effectively resolved for customers trying to run the HCW.

This brings me to another point from Tim's talk that triggered my attention: the way Microsoft currently tests hybrid deployments. As a result of last year's issues in the hybrid Exchange deployment space, Microsoft now conducts fully automated tests of the Hybrid Configuration Wizard, referred to as "Hybrid Validation" in the aforementioned session. As Tim explained, automatic systems build a new hybrid environment from scratch every day to test all aspects of a hybrid deployment. One of the advantages of running this test daily is the constant updates/upgrades that happen in Office 365. To be honest, the automated testing process is as genius as it is simple. Because all aspects of a hybrid deployment are tested, Microsoft was able to catch and solve a problem long before customers reported it. And that's a good thing, isn't it?

Despite all efforts to prevent problems, you’ll undoubtedly hit the proverbial wall every once in a while. Most problems people run into when deploying a hybrid configuration occur because they don’t fully understand what can (and cannot) be done in a hybrid deployment or as a result of (unforeseen) problems with on-premises components like network or server failures, for example.

To help you better understand what is going on in these cases, Microsoft has developed two troubleshooting tools:

1. The Hybrid Configuration Troubleshooter (http://aka.ms/hcwcheck)
2. The Hybrid Migration Troubleshooter (http://aka.ms/HMTSIgnite)

The first tool proves useful when you’re trying to run the Hybrid Configuration Wizard but are presented with an error. For instance, this could be the case when Autodiscover hasn't been setup correctly on-premises. The tool, which is also run from the Office.com website (you see a pattern develop here?) will check various components of your configuration and then provide you with targeted information about what’s causing the problem and links to articles that explain how to solve it.

The latter tool is targeted toward failed mailbox moves. If you’ve tried moving mailboxes to Office 365 in a hybrid deployment, but one (or more) moves have failed, this tool can help you determine the cause of the failure. When running the wizard, you will have to log into your Office 365 tenant and provide the email address of the mailbox you’re trying to move to Office 365. The tool will then try and fetch relevant information from items like the move request to help you determine the root cause. I should also point out one caveat: Do not remove the migration batch (or move request) if you intend to run the troubleshooter; if you do, it won’t have any information to troubleshoot in the first place.

Tim pointed out that the experience will only improve over time. Each time the tool is run and a new scenario is found that the tool doesn’t cover, checks for that scenario will be added to the tool. The team has already added more than 60 additional tests to the troubleshooter(s). This means that the more people use the tool, the better it will get.

Of course, none of these tools negate the need for proper, proactive monitoring. Even if all your mailboxes are hosted in Office 365, there are many intricacies to deal with. Whether you’re hybrid or not, it's no longer about Exchange alone. Many other components like AD FS and DirSync have their role in your infrastructure and need to be managed properly to ensure your users have the best possible experience in Office 365. After all: Did you know most of the issues with Office 365 are self-inflicted (i.e. caused by a failing or poorly configured on-premises component)?Feel free to reach out to ENow and learn more about what we can do for you!

Michael