MNGIN301

Troubleshooting Hybrid
Mailflow
Vincent Yim
Premier Field Engineer
Microsoft Services

Agenda
Refresher/Overview of Hybrid Routing
Mailflow Options
EOP in Hybrid
Review tools to assist in mail flow
troubleshooting
Issues
Other fun stuf
Questions

Refresher/Overview of Hybrid
2
Distinct Exchange organizations
Routing
HCW creates connectors in each Exchange
org. # of connectors vary based on
Exchange version
Secure Mail

Refresher/Overview of Hybrid
All
messages that are sent between on-premises
Routing

and ExO are sent over a secure connection using

TLS

The Hybrid Configuration wizard creates a dedicated send connector on-premises

scoped to the coexistence domain (tenant.mail.microsoftonline.com)
An outbound connector in EOP is also created and is scoped to the default SMTP
domain (contoso.com)

Each organization is configured to treat messages

sent from the other organization as internal

This allows messages to bypass anti-spam settings and other services

The TLS connection for on-prem server must be a

minimum of Exchange 2010 SP1

Refresher/Overview of Hybrid
E-mail
domain sharing
Routing
Both orgs will accept contoso.com authoritative

How do we prevent mail loops?

Actually, its all about how addressing works

Requires a coexistence domain for

Backboning mailflow

Refresher/Overview of Hybrid
Coexistence
Routing Domain
Based of of the Microsoft Online Default Routing Domain
The coexistence domain is a domain created for each Office 365 tenant

in the format of
<your tenant>.mail.onmicrosoft.com domain
For example, if your Default Routing domain is tenant.onmicrosoft.com
then your coexistence domain would be tenant.mail.onmicrosoft.com
Created when you activate DirSync in your Office 365 tenant
AutoDiscover and MX records created automatically for this domain
Provides the backbone of all coexistence features
Added as an on-premises email address policy when the HCW is run
Mailboxes moved to Exchange Online will have the coexistence domain
stamped on their user object as a target address

Demo
DirsyncStates Pre/Post Migration

Third Party
Email
Security
System

External User

Internet

Mailflow
Options

MX resolves
MX is
Outbound
toYou
on-canto
switched
Exchange
premises
Exchange
choose to
Online
traffic
gateway
Online
route
isProtection
delivered
outbound
ondirect
premises
mail via EOP

Secure Mail
Encrypted & Authenticated Mail Flow
David
Exchange
On-premises
Mailbox
On-Premises
Organization

Exchange Online
Protection
Chris
Cloud
Mailbox

10

Exchange Online

Mail Flow Options

In addition to choosing how inbound
messages are routed, you can also choose
how outbound messages sent from Exchange
Online recipients are routed. The following
describes the available options:

Centralized mail control: This option routes outbound messages sent from the
Exchange Online users through on-premises
This enables you to apply compliance rules to these messages that must be applied
to all of your recipients, regardless of whether they're located in Exchange Online or
on-premises
Decentralized mail control: This option routes outbound messages sent from Exchange

Third Party
Email
Security
System

External User

Internet

Mailflow
Options

Secure Mail
Encrypted & Authenticated Mail Flow
David
Exchange
On-premises
Mailbox
On-Premises
Organization

MXAllresolves
email in
MX is
toout
onand
of the
switched
premises
Exchange to
Exchange
gateway
Online
tenant
mustOnline
go via
Protection
on-premises

Exchange Online
Protection

Chris
Cloud
Mailbox

Exchange Online

12

EOP
When you create
inbound/outbound
connectors in Exchange
Online Admin Center,
these are sitting at the
edge (EOP)
SPAM Filtering
Bypassed

Review Tools for Troubleshooting

Delivery reports
End user can run. Eliminates some helpdesk calls
Somewhat useless to Admin

Message Trace
Loops
NDRs
Messages dropped due to virus
Export to CSV

Use the protocol log

Set to verbose

Review Tools for Troubleshooting

Analyze Headers
ExRCA has Message Header Analyzer
OWA MHA App

Telnet
(your Exchange server might be using IP that's been blacklisted by SPAMHAUS or one of
other RBL services in use by EOP)

DLP policy rule

Hits found through message trace
Or EAC
Or (delayed) Mail Protection Reports for Exchange

Demo
Mail Protection Reports for Exchange

Other Fun stuf

Testing and Tracing Malware Filters
Create a file called EICAR.txt with the

following text: X5O!P

%@AP[4\PZX54(P^)7CC)7}$EICARSTANDARD-ANTIVIRUS-TEST-FILE!$H+H*
Attach EICAR.TXT to a new mail message, and send it through the service.
Confirm your antimalware filter settings have taken afect (policy changes can take up

to an hour to replicate across datacenters)

This EICAR test attachment will cause the message to be treated as malicious
antivirus/antimalware engines

Other Fun stuf

Testing and Tracing Content Filter
A GTUBE message should always be

detected as spam by the content filter, and

the actions that are performed upon the
message should match your configured
settings. Include the following GTUBE text in
a mail message on a single line, without
any spaces or line breaks:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

Other fun stuf

On prem senders to internet recipients will
get SPAM filtering
Demo

Other fun stuf

Outbound SPAM filter
Why did the on-prem message route through
high risk delivery pool?
Outbound spam filtering is needed because
malicious programmers and their malware
are out there taking over computers inside
corporate networks every day. This means
that users in your organization can be
sending large amounts of outbound spam

Issues
Running a Hybrid server from home?
ISPs using dynamic IP ranges will connect, but sessions will then be dropped by EOP.

"454 4.7.5 Certificate validation failure."

CRL check from hybrid server

SMTP fixup/mailguard
220 ****************************************************************************
***********************************

The above is a tell-tale sign that mailguard is enabled on a firewall appliance (most likely
Cisco PIX), and it prevents either side from seeing the STARTTLS verb.
Cannot perform secure mail flow without StartTLS verb

Issues
Changing datacenter IP ranges?

Quite possibly need to re-run HCW if datacenter IP changes

With Exchange 2010 HCW, point-in-time list is copied

Issues
With Exchange 2010 HCW, you may need to
adjust the EHLO response guessed by HCW

Issues

Missing header?
X-MS-Exhange-Organization-AuthAs =
Internal or Anonymous
If anonymous, your message took another path

 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be
interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR
STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.