Exchange Monitoring: What is Mail Flow?

Exchange Server has two core components. First, there is the mailbox component, with all the required client access protocols and well-known mailbox functions. The second major component is the message flow, with receiving, processing, and sending email messages.

What sounds so trivial and straightforward is a central and complex building block in the Exchange architecture. In this article, I try to shed some light on the Exchange Server message flow's darkness and the overall importance of proper Exchange Monitoring.

The Exchange Server Architecture

Anyone looking at Exchange Server's message flow must always remember an Exchange Server's functional structure and the recommended implementation following the Preferred Architecture (PA). In the PA, Exchange Server is operated in a configuration made up of multiple Exchange Servers, configured as a database availability group (DAG). It is thereby providing highly available mailbox functionality and message transport.

Every Exchange Server consists of a front-end and a back-end component. The Exchange front-end components handle incoming connections from third-party systems. Communication with the transport service components is usually internal to Exchange Server only.

The following diagram shows the high-level transport architecture of Exchange Server in a three server DAG for incoming connections.

Exchange differentiates message transport settings into organization-wide and server-related settings. As the name suggests, the organization-wide settings apply for the entire Exchange organization, regardless of how many Exchange servers or DAGs you operate. Whether and how an Exchange server can process these settings depends on the product version. Exchange stores these settings in the Active Directory configuration partition. An example of these transport settings is the send connector configuration.

Server-related settings apply only to a specific Exchange Server system. These settings are mainly also stored in the Active Directory configuration partition. The configurations of the receive connectors are an example of message transport. However, Exchange Server stores a few settings in the local Windows System Registry.

For the safe and stable operation of Exchange Server, a functioning Active Directory replication is essential. This requirement includes the proper configuration of Active Directory sites. It is vital, especially in large Exchange organizations.

Equally important is the correct and consistent configuration of the TLS settings on all Exchange servers. Without such a configuration, there is no guarantee that receive and send connectors can establish encrypted connections. Exchange Server relies on the so-called SCHANNEL configuration of the local Windows Server operating system and locally installed TLS certificates.

Now, let's get to the real issue, the Exchange Server message flow, and we start with receiving email messages.

Receiving mail messages

Exchange servers accept incoming email messages in different ways. The best-known option is the protocol SMTP, which you might be familiar with already. An Exchange Server distinguishes between different types of sources that connect to front-end transport. In front-end transport accepts connections via the standard TCP port 25. The servers that connect via this standard SMTP port by default include:

These systems connect via the default front-end connector, which in its standard configuration only accepts messages to known recipients of accepted domains. The delivery of application messages to external recipients requires that you create a separate front-end receive connector.

Exchange Server provides two additional ways to get email messages into the transport component, the pickup and replay directories. Both directories are permanently monitored by the Transport Service, which you probably know from earlier Exchange versions as the Hub Transport Service. The service processes a file as soon as you store it in one of the directories. You use the Pickup directory for new email messages created by applications. You use the Replay directory for messages you exported from a message queue for resubmitting to the Transport Service.

Exchange Server uses different forms of authentication to protect against unauthorized SMTP connections. Also, the header firewall filters unauthorized information from email messages, which protects against malicious SMTP headers.

The front-end transport has no complex business logic for email processing. Therefore, it is vital to understand the interaction between front-end transport and transport services in the back-end. You need to keep this in mind when you need to troubleshoot message flow. A front-end receive connector accepts, authenticates the connection, and establishes a proxy connection to the transport service via TCP port 2525. In a single server setup, the front-end establishes a proxy connection to the same server's local Transport Service. In a DAG configuration, the front-end transport selects the DAG member server's Transport Service on which the database copy withthe recipient's mailbox is currently actively mounted.

The following diagram shows the interaction between the Front-end Transport Service and Transport Service in a DAG.

An Exchange Server receiving a message via SMTP wants to protect the message against Exchange transport failures. For this purpose, Exchange Server creates a copy of the received message and transfers a shadow copy of the message to another Exchange Server's transport service. After the Exchange Server processed the shadow copy successfully, the Exchange server sends an OK message to the sending server. Full transport high availability requires using a DAG with member servers in more than one Active Directory site.

There are two transport services in the back-end of the Exchange server. The most important service is the Transport Service, which some of you may remember as the Hub Transport Service from older Exchange Server versions. We'll take a closer look at this service in the next section. The second service is the Mailbox Transport Service. This service is responsible for storing incoming messages in mailboxes of locally mounted active database copies. The service retrieves outgoing messages from mailboxes of active database copies and transmits the messages to the Transport Service for further processing. I will describe the Mailbox Transport Service in a future article.

Receiving an email message appears simple, but message security and the protection against system failures introduce complexity. After successfully receiving an email message, Exchange Server must process it.

Processing mail messages

The Exchange Transport Service does all the processing of email messages. The processing of emails is queue-based, with the emphasis on "queuing." The service stores a received email message in the submission queue, where it remains until the service picks it up for a process called categorization. The main work of the Transport Service takes place in the Categorizer component. The Categorizer tasks include: