Scan Sequence and Action in Microsoft Forefront

Protection 2010 for Exchange Server

Published: October, 2009

Software version: Forefront Protection 2010 for Exchange Server
Carolyn Liu
Introduction ............................................................................................................................................................................ 1
Exchange Mailbox and Forefront hook .................................................................................................................................... 3
Scan Processes ........................................................................................................................................................................ 4

Scan Process Type ..................................................................................................................................... 5

Actions for Malware Scans and Filters ...................................................................................................... 6
Action Table .............................................................................................................................................. 7
Scan Job and Filter Types .......................................................................................................................... 9
Scan Sequence .......................................................................................................................................................................10

Message Header Scan and Action Sequence .......................................................................................... 10

Message Scan and Action Sequence ....................................................................................................... 10
Summary ................................................................................................................................................................................12
Introduction
Microsoft Forefront Protection for Exchange Server (FPE) is a leading solution for securing your
messaging environment. Its multi-engine antimalware solution is a proven security product that
has helped many customers to secure their e-mail system. With the introduction of a Premium
Antispam solution and seamless integration with Exchange Hosted Filtering, FPE will bring pro-
tection for Exchange to the next level.
Users familiar with FPE know that besides malware scanning, there are various filtering options.
This article provides insight into the scanning options, as well as the FPE process sequence for
malware scanning and filtering. Administrators can leverage this knowledge to maintain a se-
cure and sophisticated messaging system.
The concept of server roles was introduced in Exchange Server 2007. Server roles enable Ex-
change to clearly classify different functionalities within Exchange and enable administrators to
categorize one or more roles on different servers and locations in the organization.
Exchange Server 2007 introduced the following five roles: Edge Transport, Hub Transport, Client
Access, Mailbox, and Unified Messaging. There is also a combined Hub Transport/Mailbox role.
For more detail about these server roles, see the following article:
http://www.microsoft.com/exchange/evaluation/features/serverroles.mspx
On Edge and Hub Transport roles, Microsoft Exchange provides a Transport Agent framework.
This is a plug-in architecture that enables Exchange e-mail message security vendors to supply
their own agent to process messages passing through the transport pipeline. An agent processes
messages based on SMTP events and communicates to the Exchange Transport pipeline for
processing results and actions, such as discarding a spam message or adding a legal disclaimer
footer when a message leaves an organization. The SMTP events processing sequence is shown
in the diagram below:
n
io

d
an
at

rs
d
d
d

d
d

an
an
an

d
an

an

an
an

t
ic

a
de
m

ec

an
at
ct

m
m
m

nt
m

m
m

t
ea

nn

m
ec
D
ne

m
om
om
om

om

om
om
he

om
fH

o
oC

ej

co
on

pC
ut

aC
hC
oC

oC

tC
lC

nd

nR
O

pC
tT

is
nC

fA

ai

se
nd

oo
at
ut
hl

el

nE

nD
O
cp
O

el
nM
O

nD
nH

nR
nE

nA

nE

nN
nd

nH
nR

O
O

O
O

O
O

O
nE

O
O
O

Figure 1 SMTP Events Processing in Exchange Transport

The processing sequence moves from left to right.

Based on different mail processing requests and the mail delivery status, each agent may inter-
cept different SMTP events. For example, the OnConnect event is often processed by the anti-
spam agent.
For more information about the Exchange Transport architecture and detailed SMTP events, see
the following article:
http://technet.microsoft.com/en-us/library/aa996349.aspx
In the Categorizer (see Figure 2), the routing agent processes the routing events and categorizes
and routes messages already received by the organization to proper mail store(s) or other or-
ganization(s).
On the Edge and Hub Transport roles, Forefront provides real-time protection via the Exchange
Transport framework. This is processed in several stages. First, Forefront Antispam agents
process e-mails at the Edge role via comprehensive mechanisms (IP block list, Sender ID, SMTP
filtering, Content Filtering), stopping spam e-mails before they enter an organization. Next, the
Forefront Antimalware routing agent passes the e-mail messages to Forefront scanning proces-
es for Malware and filtering processing. The Forefront routing agent in the Categorizer inter-
cepts messages that are passing through in real-time and routes the data to one of the Forefront
scanning processes using an Inter-Process Communication mechanism for malware scanning
and various filtering operations.
Figure 2, below, describes the SMTP events going through an Exchange Edge role and different
process points by Transport agents.

EdgeTransportSvc.exe

Smtp Receive
Connector IP Connection MEx Event
Tarpitting Inbound TLS Inbound MLS
Selection throttling Dispatch

Transport
SMTP Receive Agents
n

ll
io

wa
d

d
an
at

rs
d
d
d

d
d

an
an
an
an

an

an

an
an

t
c

a
de
m

nd Fire

ec
i

at
ct

m
m
m

nt
m

m
m

om

t
ea

nn
ec
fD
ne

om
om
om
om

om

om

om
om
he

fH
oC

ej

co
on

pC
ut

nE r
aC
hC
oC

oC

tC

pC
lC

nd

nR
O de

O
tT

is
nC

fA

ai

se

oo
at
ut
hl

el

el
nE

nD
O
cp
O

a
nM
O

nD
nH

nR

nH
nE

nA

nN
He
nd

O
nR

O
O

O
O

O
O

O
nE

O
O

Mex Event Dispatch

Connection Filtering Agent

AddressRewritingInbound Agent

Edge Rule Agent

Sender ID Agent
priority

Recipient Filter Agent

Sender Filter Agent

on restart fork/create
Stranded Mail
Content Filtering Agent
Scanner

Protocol Analysis Agent

create
Attachment Filtering Agent
Messages
Jet

Figure 2 Exchange Transport

Exchange Mailbox and Forefront hook
On the Exchange Mailbox role, Exchange provides a virus scanning API (VSAPI) that enables anti-
virus vendors to scan messages passing through the Exchange Mail Store (mailbox databases).
When a mail client such as Outlook accesses mail, FPE provides real-time protection via the Ex-
change VSAPI plug-in to intercept messages and route the data to one of the FPE scanning
processes for malware scanning and filtering.
This is an additional layer of protection. Because the Mail Store can be very heavily loaded, we
advise customers to deploy their messaging system and protection solution carefully. For exam-
ple, FPE has a virus stamp feature that stamps a message when it is scanned on the Edge or Hub
role so that a redundant scan is not performed when the message is stored in the mailbox.

Inbound
Internet Inbound

Outbound Outbound

FSE-protectedInbound
Edge

Outbound

Inbound

Inbound

Mailbox FSE-protected Hub

Outbound

Inbound
Inbound

Outbound

FSE-protected Hub
Mailbox

Outbound

Figure 3 Exchange and Forefront Topology

Scan Processes
For all Exchange roles that have FPE installed, FPE uses a similar common entity to perform
malware scanning and filtering: a scan process that communicates to the hook agent and works
independently to avoid disruption of any Exchange processes.

A scan process analyzes messages and applies appropriate file navigation, filters, and malware
scans for each part of a message.

There are multiple scanning processes per scan job type (default number is 4), configurable by
the administrator, which enable concurrent processing of multiple messages and reduce the
direct impact of the scanning process on the core Exchange process (preventing, for example,
the possibility of crashing due to the deep content inspection of potentially malicious code).

Currently, the FPE scan process encompasses the following scanning technologies:

 Malware scan (viruses, spyware, and worms)

 Filters, which include:
o Sender-domain: This filter examines an e-mail from particular senders or do-
mains.
o Subject line: This filter examines the subject line of e-mails.
o File: This filter examines file names, file size, file types, or file extensions based
on file content.
o Keyword: This filter compares words and phrases in the message body of an e-
mail.
o Allowed senders: This filter is similar to the sender-domain filter but allows the
administrator to bypass any content protection filters.

Figure 4 Forefront Security for Exchange Server Transport Scan Process

Figure 4 describes the Forefront scan process basic diagram in Exchange Edge and Hub roles.
 Exchange Transport

Forefront
Antispam Agents Antimalware Agent Other Agents

ScanProcess
Scan
Scan Process
Process
Scan Process
AntimalwareEngine
File Navigators Adapters Keyword and Quarantine and
Filtering Engines Actions

Figure 5 Forefront Security for Exchange Server Scan Process on Mailbox Role

Figure 5 describes the Forefront scan process basic diagram on Exchange Mailbox role

Exchange VSAPI Framework

Forefront VSAPI
hook agent

ScanProcess
Scan
Scan Process
Process
Scan Process
AntimalwareEngine
File Navigators Adapters Keyword and Quarantine and
Filtering Engines Actions

SCAN PROCESS TYPE

There are four scan process types: Transport, Realtime, Scheduled, and On-demand.
 Transport Scan Job
The Transport Scan process (FSCTransportScanner.exe) is installed on the Exchange Edge/Hub
Transport role, and scans messages as they arrive from the Exchange Transport Service (Edge-
Transport.exe) and are intercepted by the FPE transport routing agent (FSEAgent.dll).

Realtime Scan Job

The Realtime Scan process (FSCRealtimeScanner.exe) is installed on the Exchange Mailbox role
and scans messages when a user accesses mail via the mail client (such as Outlook or Outlook
Web Access Client). The messages are intercepted by the FPE VSAPI hook agent.

Scheduled Scan Job

The Scheduled Scan process (FSCScheduledScanner.exe) is architecturally the same as the Rea-
time Scan Job, except the trigger is different. The Scheduled scan job is scheduled via the Win-
dows Task Scheduler and leverages Exchange background scanning – a separate task thread that
traverses through items in the Exchange store database looking for instances of items that have
not been scanned.

On-Demand Scan Job

The On-Demand Scan process has been architecturally redesigned for the this release due to
Exchange Server 2010 architecture changes. For Exchange Server 2010, the on-demand scan
leverages EWS (Exchange Web Services) from the Exchange Client Access Server (CAS) Role. On-
demand scanning in Exchange Server 2007 installations will still use the older design (ADO).

ACTIONS FOR MALWARE SCANS AND FILTERS

When malware is found or a filter is matched, the FPE scan process will take necessary actions
on the relevant message part. It is necessary to have a clear understanding of each action taken
by each FPE scan process. The action definitions are:

Clean
A message part (which could be a message body or an attachment) is cleaned. This option only
applies to virus scans. If cleaning is successful, the original part will be replaced by the cleaned
part and reassembled into the original format of the message. For example, an e-mail contains
the attachment a.zip. This zip file contains two files: b1.doc and b2.exe. If b1.doc is infected but
cleaned by FPE and b2.exe is clean, a modified a.zip that contains the cleaned b1.doc and the
original b2.exe will arrive in the user’s inbox.

Delete
A message part is deleted and replaced with custom defined deletion text. For example, an e-
mail contains the attachment a.zip. This zip file contains two files, b1.doc and b2.exe. If b1.doc is
infected, it will be deleted, and a modified a.zip that contains the deletion text b1.txt and the
original b2.exe will arrive at the user’s inbox.
Deletion Text b1.txt contains the following text by default:
“Forefront Security for Exchange Server detected b1.doc to be infected.”
 The FPE administrator can customizethe Deletion Text. For more information on customizing
Deletetion Text, refer the FPE Operations Guide.

Purge
The entire message is deleted and will not be delivered to the recipient(s). This option always
applies to worms (a special virus type). This option is supported in realtime (Exchange Mailbox)
scanning as well. In VSAPI 2.6, the VIRSCAN_DELETE_MESSAGE error code will indicate that the
top level message is deleted, effectively purging the message.
See Table 1 and Table 2 for what this action applies to.

Identify
A user-defined word or phase will be pre-pended to the e-mail subject line. No other action is
taken on the message. This is supported in filtering. It is available for keyword filtering, file filter-
ing, subject line filtering, and sender-domain filtering.
For example, if a keyword is matched within an e-mail message body, text defined by the FPE
administrator will be pre-pended to the e-mail subject line, indicating that a matching keyword
was found. The default pre-pended-text is “SUSPECT:”
FPE administrators can also use this option to add a MIME message header so that it can be
identified later for processing into folders at a user’s inbox or for other purposes identified by
the FPE administrator. By default, X-Junk-Mail is written to the header.

Skip (detect only)

When the Skip (detect only) option is selected, an incident log entry will be created indicating
the infection and filtering information, and the rest of the scanning and filtering process contin-
ues.

ACTION TABLE

The following table shows the action options within FPE filters and default actions among vari-
ous scan job types.
Filter Type
File Filter Keyword Allowed Subject Line Sender-
Scan Job Type Filter Sender Domain
1
Hub Transport Skip (detect Skip (detect N/A Skip (detect Skip (detect
or only) only) only) only)
Edge Transport Purge Purge Purge Purge
Delete Identify Identify Identify
Identify

Default: Default: Default: Identi- Default: Identi-

 Delete Identify fy fy
1
Mailbox Skip (detect N/A N/A Skip (detect Skip (detect
Realtime only) only) only)
Purge Purge Purge
Delete

Default: Default: Skip Default: Skip

Delete (detect only) (detect only)
1
Mailbox Skip (detect N/A N/A Skip (detect Skip (detect
Scheduled only) only) only)
Purge Purge Purge
Delete

Default: Default: Skip Default: Skip

Delete (detect only) (detect only)
1
Mailbox Skip (detect N/A N/A Skip (detect Skip (detect
On-Demand only) only) only)
Purge
Delete

Default: Default: Skip Default: Skip

Delete (detect only) (detect only)

Table 1
Note:

1. The Allowed Sender List is used to identify sender address/domains that are allowed to by-
pass the configured filters (File Filter, Keyword Filter, Subject Line Filter, Sender-Domain Filter).

The following table shows the action choices in FPE among various scan job types for malware
scans.

Malware Type

Virus Spyware

Scan Job Type

Edge Transport Skip (detect only) Skip (detect only)

or Clean Purge
Hub Transport Delete Delete

Default: Clean Default: Delete

Mailbox Skip (detect only) Skip (detect only)
Clean Purge
Realtime Delete Delete

Default: Clean Default: Delete

Mailbox Skip (detect only) Skip (detect only)
Scheduled Clean Purge
Delete Delete

Default: Clean Default: Delete

Mailbox Skip (detect only) (2)
On-Demand Clean
Delete

Default: Skip (detect

only)

Table 2

SCAN JOB AND FILTER TYPES

The following table shows correlation between the scan job and filter types.
Filter Type

File Keyword Allowed Subject Sender-Domain

Senders Lines
Scan Job Type

Hub Transport Yes Yes Yes Yes Yes

or
Edge Transport

Mailbox Yes No No Yes Yes

Realtime

Mailbox Yes No No Yes Yes

Scheduled

Mailbox Yes No No Yes Yes

On-Demand

Table 3
 Scan Sequence
When a message is scanned by an FPE scan process, it is processed by antimalware engines and
filtering engines in one pass. This is done by navigating each part of the encoded message or
compressed files in a recursive manner. This maximizes the performance and increases the
complexity of the process. The following diagrams depict the logic flow of the scan and action
sequence for the scan process.

MESSAGE HEADER SCAN AND ACTION SEQUENCE

Process message headers

Does message match an allowed sender list

for subject or sender filtering?
Antimalware/Filtering Agent

No
Yes Yes
Message Header Scanning

Does message match a Is the action purge? Message removed from pipeline
sender/domain filter? No

Yes
[Transport] Is the action identify? Tag(s) added to header(s)
No
No

Yes
Yes Yes
Does message header Is the action purge? Message removed from pipeline
match a subject filter No

Yes
No
[Transport] Is the action identify? Tag(s) added to header(s)

No

MESSAGE SCAN AND ACTION SEQUENCE

The following diagrams depict the logic flow of the scan and action sequence for the message
body and attachments.
Note:
The scan sequence is a recursive operation based on file navigation flow.
“End of execution” means to go back to the last level of execution of the recursive action. For
example, a message contains a.zip as an attachment, and a.zip contains b.exe and c.doc. If b.exe
is spyware but not a virus, and the spyware scan action is “Delete”, file b.exe will be replaced
with Deletion Text “b.txt”, and the execution will end for b.exe and the flow will go back to the
scan of the next container subpart, c.doc.

Process all file parts from message

Check if is container
No
Yes No Yes Yes
Keyword Filtering

[Transport] Is this Does sender match Does message body Is the action purge? Message removed from pipeline
No file a message Yes an allowed sender list No match a keyword filter? No
body? for keyword filtering?
Yes
Yes [Transport] Is the action identify? Tag(s) added to header(s)

No; action is skip

Worm

Yes
Does file contain a worm? Message removed from pipeline
No
No Yes Yes
Does sender match an allowed sender list Is the action purge? Message removed from pipeline
Does file name or type
Yes for file filtering? No match a file filter?
Yes
Antimalware/Filtering Agent

No
File Filtering

[Transport] Is the action identify? Tag(s) added to header(s)

Yes
If container, have all subparts been scanned yet? Yes
No No
Is the action delete? Deletion text inserted

Process all file parts from container Yes No

Was part of a container? Can file be rebuilt? Treated as corrupted
No; action is skip No
Yes compressed file
New container replaces old

Yes Yes Yes

Does file contain a virus? Is the action clean? Was clean successful?
No
Yes
No
Is the action delete? Deletion text inserted
No
Virus

No; action is skip

No No
Was part of a container?
Yes Treated as corrupted
Can file be rebuilt? compressed file
Yes
New container replaces old
Yes Yes
Does message contain spyware? Is the action purge? Message removed from pipeline
Yes
Spyware

No
No Is the action delete? Deletion text inserted
Yes No
Treated as corrupted
Was part of a container? Can file be rebuilt?
No; action is skipNo compressed file
Yes
New container replaces old

Yes
Was file a subpart of a container? End of execution
No

Continue to workload pipeline

Summary

We summarized some of the core functionalities in Forefront Protection for Exchange Server
and provided detailed views of malware scanning and filtering. This should give you an in-depth
understanding of the product to leverage the superior protection provided by FPE.

The vision behind this product line is to maximize protection by building a solution that is com-
ponentized and is adaptive to current and future scanning technologies. We are working hard
towards that goal.

Your feedback is critical for improving the existing product and building more successful ones in
the future.