Академический Документы
Профессиональный Документы
Культура Документы
Lets begin
There are several components that are involved in the Mail delivery process. Information Store (Store.exe) The Microsoft Exchange Server Information Store (Store.exe) is the end point for e-mails sent to users on this server. It is also the start point for e-mails which are sent by MAPI clients, like Microsoft Outlook 2003, which directly connect to the MSExchangeIS.
Figure 1: MSExchangeIS Exchange InterProcess Communication (EXIPC) EXIPC is responsible for Data Transfer between Internet Information Server 6.0 (IIS) and the Microsoft Exchange Server Information Store (MSExchangeIS). EXIPC provides a layered service between both components to achieve the best possible performance between IIS dependant components and the Exchange databases. As you might know, all Internet Client Access Protocols like HTTP/S, SMTP, POP3 and IMAP4 are configured and managed by IIS with some exceptions.
Figure 2: EXIPC Layer This interaction allows Exchange to be in a FrontEnd, and BackEnd, Server scenario. Through Virtual Servers, multiple configurations of the same protocol can exist on a single Exchange Server. Advanced Queuing Engine (AQE) The Advanced Queuing Engine (AQE) is responsible for creating and managing message queues for e-mail delivery. When AQE receives a Simple Mail Transfer Protocol (SMTP) mailmsg object, this object will be forwarded to the Message Categorizer. The Advanced Queuing Engine then queues the Mailmsg object for message delivery based on the Routing information provided by the Routing Engine process of Exchange Server 2003. The Message Categorizer is part of the Advanced Queuing Engine and is responsible for address resolution on every Mailmsg object that flows through the AQE. The Message Categorizer is implemented as an Event Sink. The Message Categorizer is also responsible for splitting messages into RTF or MAPI.
Routing Engine The Exchange Routing Engine uses Link State information for e-mail routing. The Routing Engine will forward this information to the Advanced Queuing Engine. Please note: The SMTP Stack from Windows Server 2003 will be extended through the Exchange Server installation process with several enhancements. One of these enhancements is the implementation of the XLINKSTATE protocol. The Routing Engine creates and maintains the Link State information for every Exchange Server and is also responsible for routing the messages to inbound or outbound destinations. SMTP Service The SMTP Service processes incoming traffic from any SMTP host. SMTP is also used in most communications between Exchange Servers (except Exchange 5.x Servers which use RPC for message transferring). SMTP is also responsible for some advanced Exchange Server functions like Message Journaling. During the Exchange installation, the built in SMTP Serivce from Windows Server 2003 will be extended with several new functions. Some of the Enhancements are:
Moving the Message Queue Directories to the Exchange installation Directory Providing support for the LSA (Link State Algorithm) in SMTP Moving SMTP Messaging from IIS to the Exchange System Manager
Message Flow
Because understanding the e-mail message flow is important, I will list some high level steps in the message flow:
MAPI client sends a message to a remote recipient Information Store (Store.exe) receives the message The created MailMsg object is forwarded to the Advanced Queue Engine (AQE) The Message Categorizer from the AQE processes the MailMsg object and splits it into MIME or RTF as necessary The Message Categorizer expands groups and checks defined Message limits on Exchange The MailMsg object is then transferred to the Remote Destination Domain within the AQE The AQE passes the destination address to the Exchange Routing Engine SMTP initiates an SMTP session with the remote SMTP host After the SMTP session with the remote host has been established, the information store retrieves the body of the message and converts the message as necessary
SMTP sends the Message from the Queue to the Remote Host
Intra Server Message Delivery Inter Server Message Delivery Message Delivery to the Internet Exchange of Routing Information
Intra Server Message Delivery SMTP will be used for Intra Server Message Delivery for several components like Message Journaling and Message categorization. Exchange Servers in the same Routing Group use SMTP to communicate with each other. Message delivery to the Internet SMTP is often used to deliver e-mail to other exchange organizations or other messaging systems. Exchange Server 2003 can use the Virtual SMTP Server to deliver messages, or one or more Exchange SMTP Connectors or Routing Group Connectors. Exchange of Routing Information SMTP is also used to exchange Link State information between routing groups.
MX Record
A Mail Exchanger Record (MX Record) is a special DNS record specifying how e-mail should be routed. When a message should be sent to that domain, a DNS lookup into the destination DNS domain occurs and will look for an MX record and a responding A Record. The E-Mail will then be sent to the specified Exchange FrontEnd or BackEnd Server for message delivery.
Relaying
SMTP Relaying occurs when one SMTP host forwards e-mail to another SMTP host. Open SMTP relaying occurs when the SMTP host accepts messages from recipients outside the organization and forwards the messages to other recipients that are also outside the organization.
Figure 4: Relaying
If the Exchange Server allows everyone without authentication to deliver messages, the server is called an Open Relay. Open Relays can be used to send UCE (Unsolicited Commercial E-Mail). By default Exchange Server 200x is not an open relay. The following steps describe the process:
The unauthorized user sends an e-mail message to the SMTP Server and addresses multiple recipients in the message. The recipients in the e-mail are in domains external to the Exchange Server's Messaging Organization. The Exchange Server accepts the Message. After Exchange has accepted the message, Exchange delivers this message to an outside SMTP host because there is no match in the recipient policies in the exchange organization.
Routing Groups
Exchange Server 2003 supports the concept of routing groups to control the message flow between Exchange Servers. Routing groups are groups of servers running Exchange Server 2003 that are connected over permanent highspeed network links. Within routing groups, Exchange Server always transfers messages over SMTP. There is one special Server called the Routing Group Master which is responsible for tracking and maintaining the routing information which is necessary for determining the best path for message delivery. The default Routing Group Master is the first server in the routing group. If you wish to transfer the Routing Group Master role you must do so manually in the Exchange System Manager.
If your organization has more than one routing group, you must install a connector between the two or more routing groups. The preferred connector is the Routing Group Connector but you can also use a SMTP, or X.400, Connector. By default, all exchange server organizations include only a single routing group called First Routing Group. All servers in the organization are members of the First Routing Group, unless routing membership is modified by you as an exchange server administrator. You should plan to implement multiple routing groups when one or more of the following conditions occur:
Network connections are slow or not permanent The network is unreliable or unstable Message transmission is complex and indirect, requiring multiple physical network hops Message transmission must be scheduled between different locations The routing group structure is created to prevent users from accessing public folder replicas
Figure 6: Link State Table The Link State Table lists all connectors, and their status, in an Exchange Server 2003 organization. The following information is included in the LST: Link status There are only two states for any given link: up or down. For this reason, connection information, such as whether a link is active or in a retry state, is not propagated between servers running Exchange Server 2003, and it is only available on the server involved in the message transfer. Exchange Server 2003 only considers routing messages by using connectors with a link status of up. Link cost The Link State Table stores costs for each connector. Exchange Server 2003 uses the cost values stored in the link state table to select the least cost route for a message. Costs are configured on each connector, and Exchange Server 2003 records them in the Link State Table.
Conclusion
In this article I tried to show you how Exchange Server 2003 Message flow works. In the second part of this article I will show you how to use Message Tracking, Message Queues and some other tools such as Winroute to troubleshoot Exchange Message flow.
There are several places and tools which can help you find the reason for failed or delayed message delivery. I will go through some basic steps to show you where you should start troubleshooting from. After reading this article and playing with these tools you should be able to troubleshoot e-mail message delivery.
Queues
If you are looking for e-mail messages which were not delivered to their recipients, one of the first places to look to see where the Message has gone is the Queue Viewer. You can find the Queue Viewer in the Exchange System Manager directly under the Server Node. There are several Queues of interest and you should have a look at the state of the Queues and the number of messages in the Queue. If there are any messages in the Queue, you can select the Queue and you will see more information about possible problems in the info pane. If you right click the Queue you can force a connection if the problem is only temporarily.
Messages waiting to be routed Contains outbound SMTP/X400 messages still waiting to be routed to their destination server, when it has been determined the message will be sent.
Figure 1: Queue Viewer For Troubleshooting reasons it is also possible to Stop all Outbound Mail if you click the Symbol in the Queue viewer. Please note that in the picture above Outgoing Mail has already been stopped. Outbound e-mail delivery was stopped for the purposes of this article so that some Messages in the Queues can be easily shown.
Message Tracking
One of the fundamental settings that every Exchange Server should have enabled is the Message Tracking option. The Message Tracking option enables the logging of every email message and, if enabled, for the message subject. You should enable message subject tracking only on low utilized Servers. Message subject logging can also be problematic in Data Security, so please speak with your legal department before implementing this feature.
Figure 2: Enabling Message Tracking After the Message Tracking feature has been enabled, the Message Tracking Feature can be used in the Exchange System Manager to find messages sent to recipients.
Figure 3: Message Tracking Center If an e-mail message is selected, the message can be clicked in order to see the message delivery status details.
Figure 4: Message History As can be seen in the picture above, the message was Submitted from Store, delivered to the AQE, submitted to the Categorizer, Queued for Routing and Queued for Remote Delivery. For an explanation of these terms read the first article about Exchange message flow.
SMTP Logging
With Exchange Server 2003 it is possible to use extended SMTP Logging for troubleshooting purposes. If SMTP Logging is enabled, Exchange will write every outgoing mail through SMTP in a special logfile located by default in \Windows\System32\Logfiles\SMTPSVC1 where SVC1 is the first Virtual SMTP Server. You must enable this feature in the Exchange System Manager under the potocol container from the Exchange Server object.
Figure 5: SMTP Logging After enabling this feature, the generated logfile can be opened and the detailed steps are shown in the SMTP connection process. For better viewing and analyzing, it is possible to export the logfile into Microsoft Excel. With Microsoft Excel the logfile can be formatted so that it is easier to analyze its content.
Diagnostic Logging
One other troubleshooting helper is the Diagnostic Logging of Exchange Server 2003. Diagnostic Logging sets the details that are logged in the Event Viewer for specific
Exchange components to a higher level, so more information will be logged in the Event Viewer Application Log . Diagnostic Logging should only be enabled when troubleshooting specific problems because Diagnostic Logging quickly fills the Event Log. The Logging Level can be set from None to Maximum in the GUI but there is also a Registry Key for setting the Logging Level to Level 7 for SMTP Logging purposes. Diagnostic Logging must be enabled in the Exchange System Manager under the Exchange Server object. After enabling the Diagnostic Logging feature the Event Viewer can be analyzed for specific problems.
Telnet Server.Domaene.TLD 25 The following picture shows the steps which are necessary to establish an SMTP connection and to send an e-mail.
Figure 8: Telnet for SMTP Tests For more information about Telnet and SMTP read my article.
SMTPDIAG
SMTPDIAG is a simple Tool for testing the SMTP Message flow from Exchange Servers to outside SMTP or Exchange Servers. SMTPDIAG can be downloaded from the Microsoft Exchange 2003 Tools Website. After downloading and extracting the SMTPDIAG Tool, open a command prompt and start SMTPDIAG. SMTPDIAG has a very simple Syntax, as you can see in the picture below. SMTPDIAG administrator@mwtraders.msft grotem@it-training-grote.de starts the SMTPDIAG process. SMTPDIAG now checks DNS settings and initiates an SMTP connection to the destination system without sending mail. SMTPDIAG has only two options.
/V = enables Verbose Mode and shows some more details which are hidden in Standard Mode. [-d target DNS] = This parameter is optional. The IP address of the target DNS server can be specified in order to look up remote MX records. This is often configured as an external DNS server in Exchange. An external DNS can be configured at the Exchange virtual server level but not for the Internet Information Services SMTP service.
Conclusion
In this article I tried to show you some troubleshooting tips for problems you may have with e-mail delivery in your Exchange Organization and to external recipients. The first part of this article discussed the basics of Message Flow and Delivery within an Exchange Organization.
network (also known as DMZ) without the need of Active Directory. This server then routes incoming messages into your Exchange Server 2007 organization. Outbound Email Outbound email means messages that are being sent from internal mailbox users to external recipients residing on the Internet. After a Hub Transport server has processed the mail and identified it as outbound mail, the server routes it to the Internet, either directly or again by passing a gateway server. This gateway server can be an Edge Server Transport server. Local Email Local mail flow refers to messages that are processed by a Hub Transport server in an Exchange Server 2007 organization and delivered to a mailbox on the same Active Directory Site. Remote Email Remote Email flow refers to messages that are processed by a Hub Transport server in an Exchange Server 2007 organization and delivered to a mailbox on a different Active Directory site from the source mailbox.
SMTP Connectors
SMTP connectors are Exchange Server 2007 components that support one-way SMTP connections. Due to this new restriction (based on earlier versions of Exchange Server) we need two connectors:
An SMTP Receive connector is required for an Exchange Server 2007 server system to accept any SMTP connection. It is used to enable an Exchange Server Hub Transport role or Edge Transport server role to receive email from any other SMTP server on the Internet, other Exchange Server 2007 Hub Transport server roles, Edge Transport server roles or other Exchange Server 2007 environments. You can configure multiple SMTP Receive connectors with different parameters on a single Exchange Server due to implementation or high availability reasons. You do not have to create SMTP Receive connectors to route mail between Hub Transport server roles within the same forest. An SMTP Send connector is required for an Exchange Server 2007 system to send any SMTP email. It is required to send email to any SMTP server on the internet or to any SMTP server within the same Exchange Server organization.
You can manage each of them using the Exchange Management Console or Exchange Management Shell. To manage connectors using the shell use the Set-ReceiveConnector and Set-SendConnector cmdlets.
Submission Queue Store Driver Microsoft Exchange Mail Submission Service Pickup Directory Categorizer
Messages from outside your Exchange organization enter the transport pipeline through an SMTP Receive Connector. Messages inside enter the pipeline through the Hub Transport server role.
Submission Queue
Each Transport server role (Hub or Edge Transport) has one submission queue that is created by the categorizer when Exchange Transport Service starts. It stores all messages on the local hard disk until they are processed by the categorizer for delivery. They are then finally removed from this queue.
Store Driver
Messages sent by a mailbox user enter the transport pipeline when they reach the senders outbox. The store driver on the Hub Transport retrieves it from the users Outbox and then transfers it to the submission queue. After the message has been successfully added to the submission queue, it is moved from the senders Outbox to the senders Sent Items. Messages are stored in MAPI format and must be converted to Summary Transport Neutral Encapsulation Format (S/TNEF) before being placed in the Submission Queue. This conversion is the job of the store driver, too. If this conversion is unsuccessful, a non-delivery report (NDR) is generated.
Directory site, the Message Exchange Mail Submission service attempts to evenly distribute notifications between each transport role using static load balancing.
Pickup Directory
Each message that is transferred to the pickup directory has been successfully submitted to the submission queue via the categorizer. Messages placed in the Pickup Directory must be in the appropriate format and have read/write permissions configured. It allows you to take a properly formatted text file and have the Hub Transport server role process and deliver it. This can be very helpful when mail flow is being validated in the organization or relaying specific messages or returning to the transport pipeline. Even 3rd party applications may place messages in the Pickup directory rather than communicating directly with the Exchange Server.
Categorizer
The categorizer always picks the oldest message from the Submission queue and checks whether this message has to be routed internally in the Exchange organization or externally. On each Hub Transport server the categorizer performs the following tasks:
Identification and verification of recipients Expansion of distribution lists Determination of routing paths Conversion of content formats Application of message policies
Configure server-specific settings Configure authoritative domains and email address policies Configure a postmaster mailbox Configure Internet message flow Configure messaging policies Configure administrative permissions: o Exchange Organization Administrators o Exchange Server Administrators o Exchange View-Only Administrators
Each of these configuration settings are unique and need to be defined in a design document before the configuration for each company.
Conclusion
As you have seen within this theoretical drilldown, Exchange Server 2007 email routing is a little bit different to earlier versions, but this new release allows a clear and easily understandable way to configure Email transport using role based installation and configuration tasks. In the next part of this article you will see how the tasks described can be configured within Exchange Server 2007 using the GUI administration tools and the Exchange Server Powershell, too. If you still have any further questions, please do not hesitate to contact me. If you missed the first part in this article series please read Exchange Server 2007 Email Routing, Part 1 Theoretical Basics. In the first part of my article we had a close look at Exchange Server 2007 Email Routing theoretical basics. Now we will have a look at how to configure Email routing within Exchange Server 2007 and how we can configure the routing topology to meet our plans. The main Exchange Server 2007 routing topology features are:
No more routing groups No more routing group connectors Uses AD site links instead Uses least cost routing based on network layers OSPF capabilities Queues close to point of failure Improved bifurcation algorithm
This means no link state routing like in Exchange Server 2003 anymore.
your Active Directory Site Links are the basis for your Exchange Routing Topology. This means your site link costs are based on calculating the best way to route messages between sites. If you are installing Exchange Server 2007 in an existing forest, you will be prompted to choose which of your existing routing groups you will connect with. This is because all of your Exchange 2007 servers will exist in a special routing group that should only house Exchange 2007 servers. In an ideal world, your first Exchange 2007 server will be near one of your existing hub routing groups.
Figure 1: Routing between two Sites In an environment with at least three sites in one chain we can see new behavior when an email is sent from the first to the third site. Compared to earlier versions of Exchange Server, Exchange 2007 will now try to route the message directly.
Figure 2: Routing between three Sites Exchange will now directly route the message to the third site, because use of the second site is only an extra cost and does not have any further advantages. The amount of WAN-
Link would not decrease, but the site in between would have to use CPU and other resources for receiving and sending the message. In addition this mail would take more time.
Figure 3: Routing between three Sites in case of failure Now Exchange will queue the mail to site C at the server nearest to its destination server.
Figure 4: Routing between three Sites in case of redundancy In case of redundancy of site links, we always have the topology of routing with least costs. After having understood how to configure intra-organizational email routing, we will now have a look at how to connect Exchange Server 2007 to the internet.
First, we will need to configure Exchange Server 2007 to accept outgoing email messages. This means we will have to create a new SMTP send connector in the organization configuration tab.
In this dialog box you will have to add all address spaces (or SMTP domains) your server should accept and reroute emails.
Figure 7: Configuring the Smarthost In this dialog box you will have to configure the destination server (relay server) of your network environment. It is best to configure Exchange to use DNS MX records. This means that you will just have to change your DNS configuration if you are changing your servers IP addresses.
Figure 8: Configuring the local Hub Transport Server Now we will have to add all source servers (formerly known as local bridgehead servers in Exchange 2003 Server) that are able to use this connector for outgoing email. To finish your configuration you will just have to click NEXT, NEW and FINISH which will create the new connector.
Figure 9: Configuring a new Accepted Domain Exchange will allow handling of three options:
Conclusion
As you have seen above, Exchange Server 2007 allows for a wide variety of configuration options to configure email routing. When looking at the intraorganizational routing topology we can see that Exchange Server 2007 will completely rely upon the Active Directory Site structure. This means that you will have to make sure that your AD structure is clean and correctly configured before you implement Exchange Server 2007. When looking at external email transport scenarios, you will have to configure an SMTP Send Connector from within your administration tool to make sure that outgoing email works properly. To configure incoming email, it's best to set up an Exchange Server 2007 with an Edge Server role. Configuring an Accepted Domains policy will insure that Exchange will only accept emails to domains it is responsible for. Replacing Front end with CAS in Exchange 2007
Introduction
A lot of companies are still running on an Exchange Server 2003 environment (which has been deployed some years ago now) and the design aspects and recommendations that have been issued were only suitable at that time. This means that many may be running an Exchange Server Front-End Solution that has been placed directly into the DMZ. If these companies are planning to migrate to Exchange Server 2007, they need to check whether to leave their front-end server there or replace it with a new machine with Exchange Server 2007 installed on it, or to completely rethink the design of their solution. This article will talk about the pros and cons for the migration and what solution will be best for future requirements.
Replace the existing Exchange 2003 front-end with Exchange 2007 client access service (CAS) role. Replace the existing Exchange 2003 front-end with a reverse proxy server like ISA Server 2006.
So, this solution is quite easy and could be smoothly deployed without any usage interruptions.
Figure 1: ISA Server as Reverse Proxy for OWA and Push Mail Furthermore, this would mean that there are no Exchange Servers anymore in any of your DMZ, leading to a more secure solution (from a reverse proxy server you would only have to open HTTPS to communicate with your Exchange server(s) in the LAN). This, would also lead you to open up to two ports (upon your configuration) from the DMZ to the internal network and not about 8 to 11 ports that need to be opened, but this depends on your design. If you choose this design, you would need to implement a reverse proxy server solution. A lot of firewalls give the possibility to configure a proxy and/or reverse proxy server on them. So in a lot of designs you will not have to choose a new server solution with a new product. If you do not have an existing reverse proxy server, you need to think of a new solution like ISA Server 2006 which is available as software solution or hardware appliance. The decision to choose between software or hardware appliance is up to you, it does not matter when it comes to the functionality we need here.
I would suggest using ISA Server as reverse proxy solution because of the following:
Best integration within your Exchange solution Logon would occur directly on your ISA Server box and not internally; ISA Server would then behave as authentication and authorization in addition, too ISA Server 2006 provides an application filter out of the box that filters the traffic for Outlook Web Access and/or Outlook Mobile Access to make sure that no other unwanted traffic would cross your firewall ISA Server 2006 can act as RADIUS or LDAP proxy to ensure secure authentication with Active Directory Services internally to your LAN As of today ISA Server is the only solution that provides this enhanced functionality
If you choose to implement a reverse proxy server solution, the project itself needs to be planned in more detail due to the fact that interruption from your internet mail solutions (OWA and Active Server Sync) may occur. The migration itself can be prepared well since a lot of things can be prepared before you disable your existing Exchange Server 2003 front-end server and switch to your new server. Here are a couple of points to help you do so:
Installation of operating system and ISA Server on your physical hardware Prepare firewall configuration for ISA Server solution (with new IP address running both solutions at one time is even possible) Configure publishing rules for Outlook Web Access and/or Active Server Sync
It is possible to test the new configuration before you put them into production, this would entail:
using another IP-address and external DNS name using a new digital certificate for the ISA Server
This sounds quite easy, but, it also means that if you are running Active Server Sync, it is only this easy if you use a digital certificate on each mobile device that has a trusted root certificate already installed in its certificate store. Otherwise, you would have to deploy the new root certificated to all of your mobile devices, too.
If you are already running a proxy server in your DMZ that is able to work as reverse proxy server too, you should think about using that one. If you currently do not have any proxies that might act as reverse proxy you should think about implementing ISA Server 2006 on a Windows Server 2003 machine, due to this server does not work with Windows Server 2008 at present. If you want that solution, you should have to wait some more months for the availability of Microsoft Forefront code named Stirling. Exchange 2007 Queue
Brief
This article investigates message queues in Exchange 2007. I begin by highlighting the differences in architecture between Exchange 2003 and 2007 in particular, discussing the fact that Exchange 2007 uses a queue database. I then discuss the new look queue viewer in Exchange 2007 and what it actually does! Finally I take a look at how the queue viewer is built on PowerShell and suggest some ways in which that could be useful!
Figure 1: The location of Exchange 2003 Queues In Exchange 2003 the UI for viewing queues made things fairly easy to find however, it had the drawback of only being able to monitor one servers queues at one time.
Figure 2: Viewing queues in Exchange 2003 In Figure 2 you can easily see the queue type and its status. In this example you can see that there are several mails waiting to be sent which most likely are non-delivery reports (NDRs) from spam mails! With Exchange 2007, the queues are viewed in a new tool called Queue Viewer which you can find alongside other utilities in the new Toolbox area, shown in Figure 3.
When you open Queue Viewer, you can see that it is based on an MMC v3.0 snap-in as shown in Figure 4:
Figure 4: The Queue Viewer UI The major benefit of this is that you can now create your own custom MMCs using the standalone viewer to monitor multiple Exchange 2007 servers at once:
Figure 5: Adding an MMC snap-in for Queue Viewer Having looked at the UI changes, it is nearly time to move on to the fundamental theory of the queues in Exchange 2007; however, before I do there is one other piece of info which you should be aware of. Not all Exchange 2007 servers have queues! This is a big difference to Exchange 2003 where, because of the fact that all servers were involved in message transport, they all had an SMTP queue. In Exchange 2007 only Hub Transport and Edge Transport servers have queues.
Queue Theory
So where does this database fit in? Well as mentioned briefly above, all queue activity now occurs in a new ESE database. The main database file is called mail.que and by default can be found here: C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue
Figure 6: Folder containing queue database files The other files are in the locations as described below:
Trn.chk - The checkpoint file. Trn.log - The current transaction log file. Trntmp.log - The next provisioned transaction log file that is created in advance. Trnnnn.log - Other transaction log files that are created when Trn.log reaches its maximum size. Trnres00001.jrs - The Reserve log file. Trnres00002.jrs - The Second Reserve log file. Temp.edb Temp DB used to verify database schema on start-up.
You might wonder what happens with the logs in this scenario. Well, they are configured for circular logging with transaction logs being deleted after they have been committed. Just before we move on to another area, it is worth stating how to move the queue databases. One important reason for moving the Queue DB and logs is performance. Another slightly less well known reason is that the drive on which the Queue DB and logs are stored must have 4GB or more free space otherwise the server will apply back pressure and start slowing the flow of messages! When moving the DB, the usual rules for splitting transaction logs and DB files apply. To move the databases you must edit the EdgeTransport.exe.config file which by default is located at the location below and then stop and restart the msexchangetransport service: C:\Program Files\Microsoft\Exchange Server\Bin\EdgeTransport.exe.config The key thing to bear in mind before editing the config file is that the parent directory has the correct permissions as set up below; that way the directory will be created for you:
Network Service: Full Control System: Full Control Administrators: Full Control
The relevant lines are shown below. To move the database, you should edit the line containing QueueDatabasePath and to move the logs, you should edit the line containing QueueDatabaseLoggingPath. You can see in Figure 7 that I have moved my DB and logs to H: Figure 7: Editing the EdgeTransport.exe.config file (click to view a larger image) Having looked at the Database it is now time to understand what it contains. There are various different queues:
Submissions: Used by the categorizer to gather all messages that have to be resolved, routed, and processed by Transport agents. Poison Message: The poison message queue is a special queue that is used to isolate messages that are detected to be potentially harmful to the Exchange 2007 system after a server failure. Remote Delivery: Remote delivery queues hold messages that are being delivered to a remote server by using SMTP. Mailbox Delivery: The mailbox delivery queues hold messages that are being delivered to a mailbox server by using encrypted Exchange RPC. Unreachable Destination: Each transport server can have only one Unreachable queue. The Unreachable queue contains messages that cannot be routed to their destinations.
Using Queues
Now that MMC v3 is used for queue viewer the UI is very simple. By default the queue viewer opens and displays the queues for the transport server that you are logged on to. To connect to another server use the Connect to Server task in the right hand action pane:
The main queue viewer window opens with two tabs along the top which show all queues and all messages by default. When you open a queue by double clicking, another tab appears showing only the messages in that queue:
Figure 9: Showing tabs in Queue Viewer To manage the queues all you have to do is highlight the object to operate on and view the actions pane on the right hand side of the window as shown in Figure 9. A key new Exchange 2007 queue viewer feature is message filtering. An example of how to use filtering is in the case of a spam attack. As an administrator you can take advantage of the "Bulk Action" feature, which applies an action to all messages that meet the parameters specified by the filter to remove spam messages with or without NDR.
Figure 11: Some more filtering options The other actions which you can perform in queue viewer are shown below:
Suspend queue This action temporarily prevents delivery of messages that are currently in the queue. Resume queue The opposite of Suspend queue. Retry queue When a connection to the next hop for a queue fails, a retry timer is set. This forces an immediate connection attempt. Suspend message This action temporarily prevents delivery of a single message. Resume message The opposite of Suspend message. Remove message This action permanently prevents delivery of a message. Export message This action copies a message to the file path that you specify. The messages are not deleted from the queue, but a copy of the message is saved
to a file location. Before you export a message, you must suspend the message in the queue.
Figure 12: PowerShell command error It got me thinking about using PowerShell to manipulate the queues. To start off, I used the PowerShell command below to show all commands with Queue in the name: get-command *queue*
Figure 13: get-command *queue* Then I tried the same command this time looking for commands with message in them:
Figure 14: get-command *message* Armed with a knowledge of available commands I began by running a simple get-queue command
Figure 15: get-queue I then moved on to locate any queue with a message count of less than 100. In this example three queues are shown all with 0 messages. All but the submission queue will shortly be removed as their messages have been delivered. The submission queue is persistent and is therefore always present, waiting for mail to be categorized.
Figure 16: get-queue with a filter As you can see, the simplicity of the PowerShell commands make manipulating the queues via script much easier than when using VBScript. Obviously the examples above are simple but they could be taken a lot further. For example, you could run the following to remove messages from queues which come from anything@spammer.com with an SCL rating higher than 5: Remove-Message -Filter {FromAddress -like "*spammer.com*" -and SCL -gt 5} -withNDR $false
Conclusion
Hopefully this has given you an insight into the new way message queues work in Exchange 2007. For more information about the inner workings of queues, check out the Exchange 2007 help file which can be downloaded here:
Introduction
If you one day are faced with a relatively large corrupt Mailbox Store, restoring it can, depending on things such as backup hardware, backup application and network speed, be quite time consuming. Now the last thing you want to deal with in such a situation is frustrated users (or even worse a yelling CEO!). So how can you get your users to calm down (and your CEO to s up) and get back to work while you concentrate on getting the Mailbox Store back to life? Theres one simple answer and that is, you can create a dial-tone database and thereby get message flow and mailbox access recovered almost instantly. By using a dial-tone database your users can start to receive and send mail again, they can even go check out old messages that existed
in their mailbox on the Exchange server (if their Outlook client has been configured to use cached mode that is), bear in mind though they have to switch between Online and Offline mode when prompted with the Outlook 2003 Exchange Recovery Mode dialog box. Ill talk more about Outlook 2003 Recovery mode in Demystifying The Exchange Dial-tone Restore Method (Part 2). Using the dial-tone database restore method means that you, while restoring one or more corrupted Mailbox Stores from the most recent backup, have users connect to a new empty or blank Mailbox Store. The dial-tone restore method is by no means new; its been used with previous versions of Exchange as well, but now that we have the Exchange Server 2003 Recovery Storage Group (RSG) feature, the method becomes even more attractive when restoring Mailbox Stores within your Exchange messaging environment. Note: With previous versions of Exchange a dedicated Exchange recovery server was required. Using a separate Exchange recovery Server meant you first had to restore the required Mailbox Store(s) or database to the recovery server, then either export the data from the restored database(s) to PST files using Exchange Server Mailbox Merge Wizard (ExMerge) or copy the whole Exchange database from the recovery server to the production server. As an Exchange database often is several gigabytes in size, this meant you typically had to copy large amounts of data over the wire which, depending on the network, could add several hours to the total recovery time. Using the Recovery Storage Group feature makes it possible to restore Mailbox Stores without the need to build and use a separate Exchange Recovery Server; instead you can simply restore the Mailbox Store(s) directly to the Recovery Storage Group (RSG) on the respective Exchange Server or any other Exchange 2003 Server in the same Administrative Group. This makes it an easy and painless process to merge data from the restored Mailbox Store(s) to the dial-tone database, or swap the restored database from the Recovery Storage Group (RSG) to the dial-tone database in the original Storage Group, then merge data from the dial-tone database to the restored Mailbox Store. Ill also talk more about swapping databases in Demystifying The Exchange Dial-tone Restore Method (Part 2). Note: If youre not familiar with the Recovery Storage Group (RSG) feature, I recommend you checkout MS KB article: 824126 - How to use Recovery Storage Groups in Exchange Server 2003 which does a great job explaining how you can recover Mailbox Stores or individual mailboxes using by restoring a Mailbox Store to the RSG.
the respective Storage Group. Now right-click the Mailbox Store and select Dismount Store as shown in Figure 1 below.
Figure 1: Dismounting the corrupt Mailbox Store In order to be able to create the dial-tone database the next step is to move the Mailbox Store files (that is Priv1.edb and Priv1.stm) from the MDBDATA folder (by default located under C:\Program Files\ExchSrvr\Mdbdata as shown in Figure 2) to another location on the server.
Figure 2: Copying the Mailbox Store Files (Priv1.edb and Priv1.stm) Note: If you have the disk space available its highly recommended you dont delete but move the Mailbox Store files (Priv1.edb and Priv1.stm) to another location on the server (preferably on the same logical drive), as you never know whether they are needed at a later stage in the recovery process. Also remember to take a copy of any transaction logs contained in the MDBDATA folder; these may very well be needed for transaction log replay after restoring the original database to the Recover Storage Group (RSG). Were now ready to create the dial-tone database; this is done by right-clicking the Mailbox Store we dismounted earlier, then selecting Mount Database as seen in Figure 3.
Figure 3: Creating the Dial-tone Database by mounting the Mailbox Store in Exchange System Manager After a couple of seconds you will be prompted with the dialog box in Figure 4 below.
Figure 4: Creating the Dial-tone database Click Yes and again wait a couple of seconds for the next dialog box to appear then click OK (see Figure 5).
Figure 5: The Dial-tone database was created successfully We have now created the dial-tone database and from this moment on users can once again connect to their mailboxes (although therere still empty). Now that the users can connect to the Exchange Server again its very important you send out an email message informing them whats going on. Such a message could look something like the one shown in Figure 6 below.
That was it for part one, in part two Ill show you what will happen when Outlook 2003 clients tries to connect to the dial-tone database that we created. Ill also show you how to restore the Mailbox Store from backup to the Recovery Storage Group (RSG), and finally show you how to swap the database restored to the Recovery Storage Group (RSG) with the dial-tone database in the original Storage Group then have them merged.
Part 2
Figure 1: Outlook 2003 Exchange Recovery Mode Outlook 2003 Exchange Recovery Mode lets you choose between Connect and Work Offline, if you click Connect youre connected with an empty Exchange Mailbox similar to the one shown in Figure 2, that means email messages, rules, signatures, etc are gone, but youre able to search the Global Address List (GAL) as well as send and receive email messages just like was the case before. Note: Be aware previous Outlook versions dont receive the dialog box shown in Figure 1. Instead a user who chooses to work online will, in most cases, end up with an unreadable OST file, because the encryption data associated with the previous mailbox would be overwritten with a new key from the new empty mailbox. Its therefore recommended to inform any user that accesses his/her mailbox with an earlier version of Outlook to open Outlook in offline mode then export the data to a PST file, which then can be opened or imported when opening Outlook in online mode. For more information I also suggest you read MS KB article: 282496 - Considerations and best practices when resetting an Exchange mailbox database.
Figure 2: Outlook 2003 in Online Mode accessing a Dial-tone database If you click Work Offline the OST file (which is stored locally on the client) is opened, from here you can access any email message which was synchronized between the Exchange mailbox and the OST file prior to the Mailbox Store crash as shown in Figure 3 below.
Figure 3: Outlook 2003 in Offline Mode accessing the local OST file
Figure 4: Creating the Recovery Storage Group Specify the drive you want to restore the Mailbox Store to (see Figure 5). If disk space allows it, its a good idea to restore it to the same logical drive as the dial-tone database is currently located on, as this will improve performance drastically.
Figure 5: Specifying Log and System Path location Click OK. Now that we have the Recovery Storage Group in place, we need to add the database (the one we want to restore from backup) to this Recovery Storage Group. This is done by right-clicking the Recovery Storage Group, then select Add database to recover, which brings us to the screen shown in Figure 6. Here you should highlight the Mailbox Store you want to restore, then click OK.
Figure 6: Adding the database to the Recovery Storage Group Now name the Mailbox Store (see Figure 7) then click the Database tab.
Figure 7: Naming the Recovery Storage Group Mailbox Store Here you should just accept the defaults, but make sure This database can be overwritten by a restore is check marked as shown in Figure 8 below, then click OK.
Figure 8: Specifying the Recovery Storage Group Database location Were now ready to restore the Mailbox Store from backup, in this article were using NTBackup but if you have implemented a third party product such as Veritas Backup Exec thats the one to use. Start NTBackup by clicking Start > Run and type NTBackup, then select the Restore and Manage Media tab as shown in Figure 9.
Figure 9: Restore and Manage Media tab in NTBackup Note: If you dont get the screen shown in Figure 9 when opening NTBackup, its because it starts in Wizard mode. If this is the case remove the checkmark in Always start in wizard mode, exit NTBackup and open it again. Now expand File > Information Store.bkf > Server\Microsoft Information Store\First Storage Group and select the respective Mailbox Store as well as Log Files (see Figure 10).
Figure 10: Expanding and selecting the respective Media item Notice the Restore files to: text box has Original location. Click Start Restore then specify the server to restore to and a temporary location for the log and patch files. Also remember to check mark Last Restore Set (Log file replay will start after this restore completes.) and Mount Database After Restore (see Figure 11), then click Next.
Figure 11: Specifying the server, temporary location for log and patch files The restore will now begin and depending on how large the Mailbox Store is this can take several minutes/hours. When the restore is complete simply click Close (see Figure 12) and exit NTBackup.
Figure 12: Mailbox Store restore complete That was it for Part 2; look forward to Part 3 where Ill show you how to swap the Mailbox Store (we just restored) currently mounted to the Recovery Storage Group with the dial-tone database, which is currently the production database. To end the article, Ill show you how to merge the two databases.
And yes I promise you that the next article will be the last in this series!
Part 3
Figure 1: Restored Mailbox Store under the Recovery Storage Group as seen in System Manager After restoring a Mailbox Store to the Recovery Storage Group its recommended to dismount/mount it once, in order to ensure any required transaction logs have been purged to the database, as well as to make sure the database is in a consistent state. If you belong to the paranoid Exchange Admins you can double check the state of the database by running an ESEUTIL /MH C:\Program Files\Exchsrvr\Recovery Storage Group\database.edb against it (remember to do this while its dismounted). The line State: should be in a Clean Shutdown state, as is the case in Figure 2.
Single Instance Storage (SIS) will be lost meaning the Mailbox Store will become much bigger than was the case prior to the crash.
Original mailbox rules, forms etc. will be kept in the state they were in before the Mailbox Store crash, which mean users wont have to do any modifications to rules that for example move messages to custom folders plus the Outlook offline files (OST files) still will be functioning. The time of the overall merge of data from one database to the other will be greatly reduced. Since the dial-tone database is much smaller in size than the original Mailbox Store. Imagine how long it will take to merge lets say 30GB into a database versus 1GB!
In order to swap the databases the first step is to dismount both of them by right-clicking the Mailbox Stores and select Dismount Store in the System Manager. Note: Theoretically you could swap the databases by changing the logical path of each in the System Manager, but I dont recommend this method and therefore wont go into details on how you accomplish this. Next step is to make sure the .EDB and .STM files associated with the Mailbox Store which were restored to the Recovery Storage Group match the names of the .EDB and .STM files associated with the dial-tone database, if they dont now is a good time to rename them. Important! You should only rename the .EDB and .STM files if you dont need to replay any additional log files into them. Its time to create a folder named NEW (or something else if you prefer) in the folder holding the .EDB and .STM files for the Restored Mailbox Store as well as in the Mailbox Store (Dial-tone database) currently in production, by default the path to each are C:\Program Files\Exchsrvr\Recovery Storage Group and C:\Program Files\Exchsrvr\MDBDATA (shown in Figure 3 below).
Figure 3: Path to the .EDB and .STM file of each Mailbox Store Now move the .EDB and .STM files from the Recovery Storage Group folder to the NEW folder created under the MDBDATA folder. Do the same for the .EDB and .STM files held in the MDBDATA folder; just move them to the NEW folder in the Recovery Storage Group folder instead. When the files have been moved you should move them once again, this time from the NEW folder to the folder above it (that is the Recovery Storage Group and MDBDATA folder). If you get a dialog box asking whether you want to overwrite existing files, its because you did a copy and not a move in the previous step. If this is the case just select Yes. Back in the System Manager you should open the Properties of each Mailbox Store, select the Database tab and check mark This database can be overwritten by a restore (shown in Figure 4 below).
Figure 4: Database tab in the Properties of the Mailbox Store Now mount both Mailbox Stores in the System Manager, when you have done so the users can once again access their original Mailboxes (including rules etc.). In addition the users will only be faced with the Outlook 2003 Exchange Recovery Mode dialog box one more time, and thats the first time they login after the databases have been swapped.
merged, then right-click and select Exchange Tasks in the context menu as shown in Figure 5 below.
Figure 5: Selecting the Mailboxes that need to be merged Now click Next twice (see Figure 6).
Figure 6: Merge or copy mailbox items to selected users current mailbox Make notice of the Destination Mailbox Store and click Next again (see Figure 7).
Figure 7: Destination Mailbox Store Select Merge Data then click Next as shown in Figure 8.
Figure 8: Selecting Merge Data Schedule the processing task or begin the merging immediately, then click Next (see Figure 9).
Figure 9: Schedule the processing task Let the task finish then click Finish (Figure 10 and 11).
We have now restored all mailbox data to the state it was in prior to the Mailbox Store crash, as well as merged any messages that were received while the users were connected to the dial-tone database, and can now call our disaster recovery a success.
Final words
Hopefully these 3 articles inspired you enough to go test out the dial-tone recovery method in your test lab, so that you can make use of its benefits should you one day have to deal with a large corrupt Mailbox Store in your Exchange messaging environment. How does Recovery Storage Group works:
Introduction
Recovery Storage Groups (RSG) are one of the most administrator friendly features of Exchange Server 2003 SP1. Prior to Exchange 2003 SP1, if you needed to restore a mailbox, you were in for a whole pile of work. You would need to configure a completely separate forest with an Exchange recovery server and then restore to that server. Once that was complete you could export to PST and then import that PST into the production mailbox. Not fun and I know of more than one place that had a policy not to restore mail. You can imagine what that led to. One of the features of Exchange Server 2003 SP1 is called Recovery Storage Groups. Fellow MSExchange author Markus Klein wrote on using RSGs in his article titled Recovering Mailboxes with Exchange Server 2003 Service Pack 1. Recovery Storage Groups allow an administrator to mount a restored copy of an Exchange database on any server in the same Administrative Group. Once the database is restored to the RSG, the administrator has a number of options to restore the mailbox(es) to the production server. You can restore a mailbox, a group of mailboxes, or the entire database.
the default locations or specify a different location for the files (see Figure 1). The only way to change this is to delete the RSG and start over.
Figure 1: RSG File Location A few other things that make RSGs different are:
You cannot use RSGs to restore Public Folders Only one RSG is supported per Exchange server
Figure 2: Restore Options One thing to be aware of is that you can only restore backups taken with an Exchange aware backup application. So why isnt the production database overwritten? Simple, the Information Store is smart enough to automatically redirect the restored database to the RSG. When you create an RSG you are prompted to choose a database to recover (see Figure 3).
Figure 3: Database Selection When you start the restore, the Information Store checks to see if the chosen database is in an RSG. If it is, the database is restored to the RSG, if it is not the restore stops. You may also see event ID 9635 in the application event logs. The source of this error is MSExchangeIS and the description will tell you that the database could not be found. Once the RSG is deleted, the recovery process returns to its normal behavior. If you do not want to delete the RSG, but dont want to restore to the RSG, you can modify the behavior in the registry. Open up the registry and drill down to HKLM\System\CurrentControlSet\Services\MSExchangeIS\Parameters\System Create a new DWORD called Recovery SG Override and set the value to 1. This will allow you to keep the RSG, but the restored database will not be redirected. I dont recommend creating this key, and if you do require it, delete it when you are finished. Last thing you want is to forget about it and then wonder why the 3 month old database you just restored to an RSG actually overwrote the production database!!!
How Does the RSG Know Where to Restore the Mail To?
With the RSG created and the database restored, you might now wonder how the mailbox in the RSG connects to the mailbox on the production database. The RSG mailbox and the production mailbox are linked with the following two Active Directory attributes
msExchMailboxGUID msExchOrigMDB
The msExchMailboxGUID is unique for each mailbox and is created during mailbox creation and never changes. The GUID of the mailbox in the RSG must match the GUID of the production mailbox. This leads to a common issue with RSGs; you cannot restore a deleted mailbox. When the mailbox is deleted so is the GUID and when the new mailbox is created a new GUID is also created. Because the GUIDs do not match, a RSG cannot be used to restore the data. Figure 4 demonstrates this link.
Figure 4: GUID Linking The msExchOrigMDB attribute determines the originating database from which the mailbox was a part of. This attribute directs the data to the proper database. The
combination of the msExchMailboxGUID, which determines which mailbox to restore to, and the msExchOrigMDB attribute, which determines which database the mailbox is in, allows the administrator to merge or copy the data into the production mailbox (see Figure 5).
Figure 5: Merge or Copy Data This leads to another common issue, what happens if the mailbox has been moved to a different database since the backup was made? In such a case, the msExchOrigMDB attribute can be edited with ADSIEdit. If the mailbox has been moved to a different database, you must edit the msExchOrigMDB attribute on the RSG so that it points to the database that now contains the mailbox. To do this open ADSIEdit and drill down to CN=Configuration,DC=domainname,DC=tld CN=Services CN=Microsoft Exchange CN=ExchangeOrganizationName CN=Administrative Groups CN=AdministrativeGroupName CN=Servers CN=Servername CN=InformationStore CN=StorageGroupName
From this location, right-click on the database object and select Properties; record the value for the distinguishedName. Next, locate the RSG database under the CN=Configuration,DC=domainname,DC=tld container. Edit the msExchOrigMDB attribute and enter the value you recorded earlier. Close ADSIEdit and you can now restore the mailbox that was moved.
Conclusion
Recovery Storage Groups are a powerful tool available to Exchange administrators that allow you to easily restore mailbox data. This article scratches the surface on how they work, but I hope it answers some of the questions you had on the subject. If you want to know more check out these great articles from other MSExchange authors:
Understanding Information Store: If you dont believe me, stop the Microsoft Exchange Information Store service and count the seconds before your phone starts ringing! The Information Store is made up of a number of components. Figure 1 shows a graphical layout of a typical Exchange server.
Figure 1 Exchange 2000 and 2003 use the same Information Store but there are some differences depending on the version. Table 1 describes these differences.
Store Features
Exchange 2000* or Exchange 2003 Exchange 2003 Standard /w SP2 Standard Pre-SP2
# of Storage Groups 1 + 1 RSG** 1 + 1 RSG** 4 + 1 RSG** # of Stores 1 Mailbox store and 1 1 Mailbox store and 1 5 per Storage Group Public Folder Store Public Folder Store per Storage Group per Storage Group Store Size Limit 16GB per Store 75GB per Store 16TB per Store Table 1 * Any Exchange 2000 service pack level **RSG = Recovery Storage Group
Priv1.edb is a rich-text database file that contains the email messages, text attachments and headers for the users e-mail messages Priv1.stm is a streaming file that contains multi-media data that is formatted as MIME data.
Similarly, each Public Folder Store is made up of a database set that also contains two files:
Pub1.edb is a rich-text database file that contains the messages, text attachments and headers for files stored in the Public Folder tree. Pub1.stm is a streaming file that contains multi-media data that is formatted as MIME data
Exchange utilizes what Microsoft terms a single-instance message store. This singleinstance message store works on a per database basis. What does this mean? If an e-mail message is sent to multiple mailboxes that are all in the same database, the message is stored once and each mailbox has a pointer to the message. The transaction is also logged in the transaction logs for the Storage Group that contains the database. However, if the e-mail message is sent to multiple mailboxes that are located in different databases, the message is copied to each database and written to the transaction logs for each Storage Group that contains the database with a copy of the message. For example, if I send 10 users a 1MB email message and all the mailboxes are located in the same database, one copy of the message is written to the database and each mailbox points to this message which will consume 1MB of disk space in total. If the 10 recipients are located in two different databases, each database will get a copy of this message which will consume 2MB of disk space. As you can see this is a much more efficient use of space as opposed to the alternative of 10 1MB messages using up 10 MB of disk space. Aside from the database files, Storage Groups also contain system files and transaction logs. There are two system files, Tmp.edb which is a temporary database where transactions are processed, and E##.chk. The E##.chk file maintains the checkpoint for the Storage Group. The ## represents the Storage Group number with the First Storage Group file called E00.chk. This checkpoint file keeps track of the last committed transaction. If you are ever forced to perform a recovery, this file contains the point at which the replaying of transaction logs starts.
Transaction Logs
The transaction logs are some of the most crucial files when it comes to a working Exchange server. Microsoft Exchange Server uses transaction logs as a disaster recovery method that can bring a Exchange database back to a consistent state after a crash. Before anything is written to the EDB file, it is first written to a transaction log. Once the transaction has been logged, the data is written to the database when convenient. Until a transaction is committed to the database, it is available from memory and recorded in the transaction logs. This is why you will see store.exe use up to 1GB of memory after the Exchange server has been in use for a while. After an Exchange server is brought back up after a crash, the checkpoint file points to the last committed transaction in the transaction logs which are then replayed from that point on. This form of write-ahead logging is important for you to know. There are four types of transaction logs:
E##.log is the current transaction log for the database. Once the log file reaches 5MB in size it is renamed E#######.log and a new E##.log is created. As with the checkpoint file the ## represents the Storage Group identifier. While the new
E##.log file is being created you will see a file called Edbtmp.log which is a template for Exchange server log files. E#######.log are the secondary transaction logs. They are numbered sequentially starting with E0000001.log using the hexadecimal numbering format and are 5MB in size. Res1.log is a reserved log file that is limited to 5MB in size. When the disk has run out of space, transactions are written to this log file while you work on clearing up space on the disk. Res2.log is another reserved log with the same function as Res1.log.
Transaction logs can grow at a fast pace as each and every transaction is recorded to the log files. There are two ways to manage this growth with the recommended method being a regular full backup of the Information Store. Upon successful backup, the transactions are committed to the database and then purged. The other method is to enable circular logging. Circular logging is disabled by default as it only allows you to recover Exchange data since the last full backup. With circular logging enabled the transaction logs are purged as the transactions are committed to the database. If you have to restore from backup, the transaction logs will not be replayed and all transactions since that backup will be lost. The two reserved log files, Res1.log and Res2.log, are used to save 10MB of space on the disk in case there is no more free space. When the disk runs out of free space, the transactions are logged to the reserve logs as the Information Store shuts down gracefully. You will not be able to restart the Information Store service until you clear up some disk space.
Best Practices
As with anything there are some best practices you can follow in order to maintain a healthy Information Store.
Locating the Exchange program files, SMTP queues, transaction logs and database files on separate disk arrays is ideal. If budget constraints will not allow for this, locating the program files, transaction logs and SMTP queues on separate partitions on one disk array and the database files on a separate disk array will still offer some performance increases at a reduced cost. All files should be located on redundant disk arrays. RAID 1 is the minimum recommended level, with RAID 5 offering an increase in performance and RAID 10 offering the best performance but at an increased cost. Perform regular, full backups of the Information Store to commit the transactions and flush the log files. This can be done with the native Windows backup tool, NTBackup, or a third party solution. Even if you live on the wild side and do not keep backups of your data, it is important to do this to prevent the disk from filling up with log files and running out of space.
Do not use circular logging. As mentioned circular logging will not allow you to replay the transaction logs limiting you to recovering only the data from the latest full backup set.
The Information Store is the most critical component of Exchange Server 2000/2003 and a proper understanding of its structure is important to know for anyone tasked with managing and maintaining an Exchange server.
Using ESEUTIL Tools: Before we start using ESEUTIL and ISINTEG ensure the following:
Make a backup of your Exchange databases even if you think the files are damaged and lost. Use ISINTEG and ESEUTIL with some understanding about what these tools really do. Ensure that you have done all other tests before you use ESEUTIL and ISINTEG. Dismount the store (then it is accessible for offline defrag, tests and more).
ESEUTIL
ESEUTIL is a tool to defragment your exchange databases offline, to check their integrity and to repair a damaged/lost database. ESEUTIL is located in the \EXCHSRVR\BIN directory. This directory is not in the system path so you must open the tool in the BIN directory or enhance the system path with the \EXCHSRVR\BIN directory.
Figure 2: Change the system path to point to the \EXCHSRVR\BIN directory ESEUTIL /D parameters
Defrag
Exchange 2003 defragments the Exchange database every night. But this is only an online defrag of the database. An online defrag doesnt reduce the size of the information store. To reduce the size of the databases, you must use an offline defrag.
Figure 5: ESEUTIL Defrag parameters Depending on the size of the information store and your hardware, the defrag process can consume a lot of time.
Figure 7: ESEUTIL integrity check To start the integrity check for the PRIV1.EDB database, type the following command: ESEUTIL /G C:\Program files\exchsrvr\mdbdata\priv1.edb
Disaster recovery
With a good backup in hand and Exchange databases and logfiles on different hard drives, it is no problem to recover from an Exchange disaster.
Just restore the data from backup and initiate a roll forward of the transaction logs. Well done, the Exchange information store goes online. But what should you do when your backup isn't readable or you don't have a backup? Here's how these tools come to play. Before you start:
Make sure that the databases are really not startable Check the Application log for Exchange events that can tell you the cause of the failure Make a backup of the database Restart the server so that a soft recovery can be done
ESEUTIL /P parameters ESEUTIL /p repairs a corrupted or damaged database. Ensure that you have a minimum of 20% free disc capacity in association to the Exchange database size.
Example: ESEUTIL /P c:\program files\exchsrvr\mdbdata\priv1.edb /Se:\exchsrvr\mdbdata\priv1.stm /Te:\tempdb.edb This command will repair the database PRIV1.EDB. If you have no .STM file, you can create one with ESEUTIL /CREATESTM. Read more about this here. After running ESEUTIL, you can open a detailled logfile called >database<.integ.raw to see the results. As a last Step run ISINTEG fix -test alltests. You can read more about ISINTEG later in this article.
ISINTEG
ISINTEG is used to do some tests on your information store and to fix some detected errors and problems.
Figure 10: ISINTEG parameters ISINTEG is the only repair utility that understands the Exchange database as an Exchange database. What does this mean? ESE is a generic database engine that can be used by different applications (Exchange, Active Directory). ESEUTIL looks into the database as just another ESE database, and can see their tables and indexes. ESEUTIL just fixes the database tables.
Now it is time for ISINTEG. ISINTEG is aware of the relation between database tables and records that turn them into folders and messages. After you run ISINTEG FIX, you will see many warnings but you can safely ignore these messages. You should only pay attendtion to the end of ISINTEG. There should be zero errors reported. If there is an error, run ISINTEG again. This example shows ISINTEG test folder
Conclusion
ESEUTIL and ISINTEG are two powerful tools for ensuring the health of your Exchange information store and a good resource to recover from failures in the store. Use these tools with caution when you want to repair your information store. It is always a good idea to make a backup before you use ESEUTIL to repair your Exchange databases. In this article I have explained only a few features of ESEUTIL and ISINTEG. For a full understanding of these tools, read the following KB articles. Changes of Information store in Exchange 2007
Introduction
During the early stages of the development phase of Exchange Server 2007, rumors about changing the store to SQL circulated the Internet. But these plans were dropped relatively quickly and chances are we wont see an Exchange product where the store is based on SQL before E15 (yes thats the version after E14!). But this doesnt mean that Exchange Server 2007 doesnt introduce any store related changes and improvements, because although the Exchange still is based on a more or less unchanged ESE database a lot of work went into providing a much more scalable, reliable, and optimized product which performs better than was the case with previous versions of Exchange.
Because its a 64-bit application, Exchange 2007 reduces input/output (I/O) requirements up to 75 percent in I/O per second. Exchange Server 2007 also makes better use of existing storage systems and will allow Exchange administrators to use low-cost options like Direct Attached Storage (DAS) in even the most demanding environments. As you can see in Figure 1 the Deleted items and mailbox retention settings also have changed. In Exchange 2003 the default deleted items retention setting was 7 days, but this is 14 days in Exchange 2007. This is also true for Public Folders.
Figure 2: Database Management in Exchange Management Console Like is the case with Exchange 2003 its still ok to keep all Storage Groups on the same spindles, but in terms of performance its better to keep them separated, although this would be quite unrealistic for most organizations that were using, for example, 30 Storage Groups! As I already mentioned databases in Exchange Server 2007 are still based on a more or less unchanged Extensible Storage Engine (ESE). As most of you are aware, ESE relied on two databases files, an .EDB and a .STM file. The purpose of the streaming file (.STM) was to house raw Internet content message streams as defined in Request for Comments (RFC 822). Since the .EDB file isnt very suitable for storing raw Internet content message streams, the idea of introducing the .STM file was understandable, but with Exchange Server 2007 the .STM file has been removed together with the Exchange Installable File System (ExIFS). The reason behind this decision was in order to reduce the overall I/O footprint for Exchange Server 2007. However, unlike previous versions of Exchange, Exchange Server 2007 no longer maintains single-instance storage of message bodies, only for attachments. There is one exception to this rule and that is when a set of mailboxes has been moved from an Exchange 2000 or 2003 Mailbox store to an Exchange 2007 Mailbox database during a transition. In this situation Exchange 2007 maintains single-instance storage for both
attachments as well as message bodies. For additional details see this blog post on the MS Exchange Team blog.
Figure 3: New Transaction Log File Size in Exchange 2007 So whats the reason behind this decision? Well in previous versions of Exchange if a crash destroyed the last few log files that hadnt been committed to the database yet, you would need to restore or repair the database to have it mounted again. Exchange Server 2007 introduces a new feature called Lost Log Resilience (or LLR in short) which will hold the last few log files in memory until the database is shut down. This means that you will never have a case where part of for example log file 5 has been written to the database, but part of log file 4 hasnt. The benefit of this is that if you dont have anything against losing the last few log files, you can tell Exchange to simply throw away the data and mount the database. So the reason why the log files has been reduced to 1MB is to reduce LLR exposure. Now if you lose the last log, it costs up to 1MB of the most recent data instead of 5MB. Another improvement worth mentioning is that the log file sequence numbers now can go above 1 million. As some of you might be aware previous versions of Exchange had a limit of 1 million, so if a database had been running long enough to generate a million logs, you had to shut it down and start over from log #1 ("reset the log sequence"). This would happen every few years for most databases. With the smaller log sizes and the increasing amount of messages passing through most databases, the Exchange Product
group decided 2 billion log files (per storage group!) would be a better maximum log number.
Public Folders
Public Folders are still supported in Exchange server 2007, but bear in mind they have been de-emphasized which means that theres a good chance they wont be included in the next version of Exchange (currently codenamed E14). With this in mind its a good idea to start thinking about migrating to another solution such as SharePoint. Note: Even though Public folders have been de-emphasized in Exchange Server 2007, Microsoft will support them until the end of 2016, so you have got plenty of time. One major drawback is that the Public Folder related administration tasks you can do from within the Exchange Management Console are extremely limited. So if you need to do tasks other than create, delete and move Public Folder databases as well as configure limits, etc. you will, depending on the specific task, need to do so using either the Exchange Management Shell, an Outlook client or System Manager on an Exchange 2003 Server still part of the Exchange organization.
Figure 4: Creating a Public Folder using the Exchange Management Shell So if your organization makes heavy use of Public Folders, I recommend you keep an Exchange 2003 Server running in your organization, until you have migrated away from the Public Folder hierarchy or until Exchange 2007 SP1 is released (which hopefully includes administration tasks in the EMC GUI!).
As you probably have seen in some of my recent Exchange 2007 articles, Exchange 2007 also includes several new high availability features such as Local Continuous Replication (LCR), Cluster Continuous Replication (CCR) and Single Copy Clusters (SCC). I wont dig into those here but instead refer you to the respective articles:
Demystifying the Local Continuous Replication (LCR) feature Installing, Configuring and Testing an Exchange 2007 Cluster Continuous Replication (CCR) Based Mailbox Server (3 part article series) Installing a Two Node Exchange Server 2007 Single Copy Cluster (SCC) in a Virtual Server Test Environment (3 part article series)
Note that LCR are supported with Exchange 2007 Standard edition, but that you need the Exchange 2007 Enterprise edition in order to take advantage of the CCR and SCC features. Since both CCR and SCC are based on the Microsoft Clustering Service (MSCS), you also need to make sure you install Exchange 2007 on a server with Windows 2003 Server Enterprise edition if you want to use these features in your environment. That was all for this time, you should now be even better prepared for the planning phase of Exchange 2007 deployment in your organization. Working of Recovery Storage in Exchange 2007
Introduction
The Recover Storage Group (RSG) feature, which was originally introduced back in Exchange 2003, gives you as the Exchange administrator, the option of mounting a second copy of a mailbox database (typically a mailbox database restored from backup) so that you can extract data from one or more mailboxes in the respective database without affecting the production databases if you need to do so during working hours. Depending on how much you have used the new Exchange 2007 Management Console (EMC), theres a chance you may have noticed you can no longer create an RSG from within the EMC. With Exchange 2007 this is instead done using the Exchange Troubleshooting Assistant (ExTRA) which is launched via the Database Recovery Management tool, which is found under the Exchange Toolbox work center, or by using the Exchange Management Shell (EMS). When mounting a copy of a Mailbox database to an RSG you can extract the data from a mailbox and then merge the data with another mailbox located in a mailbox database in a production Storage Group, but you can also extract the data and then copy it to a specific folder in another mailbox. Note With Exchange 2003 RTM, the data was extracted, copied and merged with another mailbox or mailbox folder using the Microsoft Exchange Server Mailbox Merge Wizard
(ExMerge) tool, but with Exchange 2003 SP1 the process was integrated in the Exchange 2003 System Manager GUI.
Figure 1: Selecting the Storage Group to link with the RSG Now its time to create the RSG, but before you do so you need to give it a name (the default name is Recovery Storage Group which should be ok in most situations). When you have entered an appropriate name, click Create the recovery storage group (Figure 2).
Figure 2: Creating the RSG After a little while you will be presented with a screen similar to the one in Figure 3 and the RSG for the respective Mailbox Database has now been created.
Figure 3: RSG Result With the RSG created we can move, copy or restore database and transaction log files to the recovery storage group paths. To see the path for the recovery storage group log and database files, click Show Create Recovery Storage Group Information. By default the path is C:\Program Files\Microsoft\Exchange Server\Mailbox\<Storage Group>\RSGxxxxxxxxx as you can see in Figure 4. The RSGxxxxxxxxx folder will appear empty in Windows Explorer until you have either moved, copied or restored the database and transaction log files to it.
Figure 4: Storage Group and Recovery Storage Group Paths For the purpose of the example in this article, we will restore a Mailbox Database from a backup using the Windows 2003 Backup tool. So lets launch the Windows 2003 Backup tool by clicking Start | Run and typing cmd.exe followed by hitting Enter. Since we will restore the Mailbox Database by using this tool in Advanced Mode, click Advanced Mode. Now select the Restore and Manage Media tab. Here we need to select the Mailbox Database and Log Files we want to restore, when you have done so click the Start Restore button. Note The Restore files to: drop-down box is set to Original location. Also notice we cannot change this selection. But does this mean the Mailbox Database currently in production will be replaced with the one we restore from backup? No this is not the case, first we havent dismounted the production Mailbox Database and second we havent enabled the This database can be overwritten by a restore option on the Mailbox Database property page. Because of this the Mailbox Database will be restored to the Recovery Storage Group which we just created. Now specify the Exchange Server to which you want to restore the respective Mailbox Database, then enter a temporary location for the log and patch files. Lastly check Last Restore Set (Log file replay will start after this restore completes.) as this is the last restore set. When you have done so, click OK and wait for the restore job to complete then click the Close button. The respective files have now been restored to the RSGxxxxxxxxx folder as you can see in Figure 5.
Figure 5: Restored Mailbox Database in Windows Explorer Since we didnt check the Mount Database After Restore option, the Mailbox Database will now be in a dismounted state, with this in mind lets switch back to the ExTRA Task Center. As shown in Figure 6 we now have several new Recovery Storage Group related tasks available, since the Mailbox Database needs to be mounted before we can extract data from it, we have to click Mount or dismount databases in the recovery storage group.
Figure 6: Selecting Mount or dismount databases in the recovery storage group On the Mount or Dismount Database page, check the Respective Mailbox Database and click Mount selected database (Figure 7).
Figure 7: Mounting the Mailbox Database using the ExTRA Tool When the Mailbox Database has been mounted click Go Back to task center and then select Merge or copy mailbox content. This will bring us to a screen similar to the one shown in Figure 8, here you should just make sure the Mailbox Database you wish to extract data from is selected, and then click Gather merge information.
Figure 8: Selecting a Mounted Database in the Recovery Storage Group We now have the option of swapping the Mailbox Database mounted to the RSG and the linked production Mailbox Database (a recommended step if youre performing a dialtone database restore) by checking Swap database configurations as can be seen in Figure 9. Since this option will swap the two databases, both of them needs to be dismounted, which will affect mail service to the end-users whose mailboxes are stored in the respective database. Since we arent dealing with a dial-tone database restore in this example just click Next.
Figure 9: Database Swap Option On the Select Merge Options page we should click Perform pre-merge tasks (Figure 10). Note that you have the option of clicking Show Advanced Options. Under the Advanced options we can specify different match and filtering options as well as the bad item limit. This is also the place where you specified whether all merge mailbox data should be merged to the respective mailboxes in the production Mailbox Database or should be copied to a single target mailbox.
Figure 10: Specifying Merge Options Final step is to select the mailboxes you want to merge. You do this by checking the box to the left of each user name in the list as shown in Figure 11.
Figure 11: Selecting the Mailboxes to Merge Now wait for the tool to merge the mailbox data from Mailbox Database in the Recovery Storage Group for the selected mailbox. When the mailbox data merge has completed, you should be able to see the content that was deleted from the production Mailbox Database. You dont even need to restart the Outlook or OWA client for the restored data to appear! When you have merged or copied the required Mailbox data, you can use ExTRA to dismount and then remove the Recovery Storage Group. Be sure you remove the files in the RSGxxxxxxxxx folder again after you have removed it, so that the files dont take up valuable disk space.
Working with Recovery Storage Groups using the Exchange Management Shell
As mentioned in the introductory, you can also manage an RSG using the Exchange Management Shell (EMS). If you have a fair amount of experience working with cmdlets, restoring mailbox data from a Mailbox Database in a Recovery Storage Group can be done a lot faster than when using ExTRA.
The first step is to create the RSG. In order to create an RSG via the EMS you need to run the New-StorageGroup cmdlet with the Recovery parameter. So to create a RSG for the First Storage Group on a server named E2K7S04, type: New-StorageGroup Server E2K7S04 LogFolderPath E:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\RSG Name Recovery Storage Group SystemFolderPath E:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\RSG Recovery The LogFolderPath and SystemFolderPath parameters are used to specify where the RSG related files should be located. As you can see, we specified they should be restored to a subfolder called RSG under E:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\RSG. If you intend to do the same please make sure theres sufficient disk space available for the Mailbox Database youre restoring from backup. To see if a respective Storage Group is a Recovery Storage Group (as well as many other types of information) you can use the Get-StorageGroup <storage group name> | FL command. If the Storage Group is a Recovery Storage Group it will say True under Recovery as shown in Figure 12.
Figure 12: Full List of Recovery Storage Group information The next step is to add a recovery database (either moved, copied or restored from backup) to the RSG, this is done by running the New-MailboxDatabase cmdlet with the MailboxDatabaseToRecover parameter. So to add a recovery database to the Recovery
Storage Group on a server named E2KS04 with the edb file path pointing to E:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\RSG, type: New-MailboxDatabase MailboxDatabaseToRecover Mailbox Database StorageGroup E2K7S04\Recovery Storage Group EDBFilePath E:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\RSG\Mailbox Database.edb With the Mailbox Database created in the Recovery Storage Group we now need to configure it to allow overwrites by running the Set-MailboxDatabase cmdlet with the AllowRestore parameter. To allow file restores for the recovery database just created, type: Set-MailboxDatabase -Identity "E2K7S04\Recovery Storage Group\Mailbox Database" -AllowFileR estore $true Now that we have created a recovery database in the Recovery Storage Group and allowed it to be overwritten by a file restore, its time to restore the mailbox database version from which you want to extract and copy or merge data to the mailbox database in production. To do so launch the Windows 2003 Backup tool and restore the respective Mailbox Database version using the same steps as we did when we used the ExTRA to recover Mailbox data. We now need to mount the restore Mailbox Database using the Mount-Database cmdlet. In order to do so type: Mount-Database Identity E2K7S04\Recovery Storage Group\Mailbox Database With the Mailbox Database mounted we can now extract Mailbox data from it. If you for example want to merge the mailbox data of an existing user in the recovery database to the production mailbox database, you need to type: Restore-Mailbox Identity <username> -RSGDatabase servername\RSG name\database name In Figure 13 we recovered mailbox data for a user called Test User 1on a server name E2K7S04.
Figure 13: Restoring Mailbox Data from a Mailbox in a Recovery Storage Group
Note Depending on the size of the mailbox that is to be recovered, this merging process can take a long time. If you need to recover mailbox data for all users in the RSG, you would need to use the following command: Get-MailboxStatistics -Database Recovery Storage Group\Mailbox Database | Restore-Mailbox Lets suppose the mailbox in the recovery database from which you want to recover data in the meanwhile has been deleted from the production mailbox database. In this case you have the option of recovering the mailbox data to a target folder in another mailbox by using the following command: Restore-Mailbox RSGMailbox Test User 1 -RSGDatabase servername\RSG name\database name Identity Test User 2 TargetFolder Test User 1 Recovered data Like is the case when recovering data using the ExTRA tool, you should, when using the Exchange Management Shell, remember to remove the RSG after the required data has been recovered. To do so, first run the command to remove the recovery database: Remove-MailboxDatabase Identity E2K7S04\Recovery Storage Group\Mailbox Database Click Yes to the confirmation warning, then type the following command in order to remove the RSG: Remove-StorageGroup Identity E2K7S04\Recovery Storage Group Finally delete the RSG folder manually using Windows Explorer.
Conclusion
As you have seen throughout this article, the way we work with Recovery Storage Groups (RSGs) has changed quite a lot with Exchange 2007. RSGs can no longer be managed using the Exchange Management Console (formerly known as the Exchange System Manager); instead you need to use the Exchange Troubleshooting Assistant (ExTRA) or the Exchange Management Shell (EMS). But despite the new methods used to manage RSGs, not much has changed when speaking RSG improvements. For example its still not an option to restore a Public Folder database to an RSG. Transaction Log files:
Introduction
One of the most important components of Exchange server is the transaction logs. Exchange server was designed to write all transactions to these log files and commit the changes to the databases when the system allows. Users can send and receive messages without touching the database thanks to this write-ahead method of logging. When a message is sent, the transaction is first recorded in the transaction logs. Until the transaction is committed to the Exchange database (EDB), the only existence of this data is in the system memory and the transaction logs. In the event of a crash, you lose the contents of the memory and all you are left with is the record in the transaction log. These transaction logs are crucial to the recovery of a failed Exchange server, whether it was a minor crash that required a reboot, or a more catastrophic failure requiring the deployment of your disaster recovery plans. The same goes for other transactions such as received messages, deleted items and messages moved to different folders. For this reason, it is recommended to house the transaction files on a redundant storage system, like a RAID 1 array, so that in the event of a hardware failure, no data is lost. Losing a set of transaction logs will not prevent you from restoring from your backups, but you will lose all the messages and changes since the last full backup.
Figure 2: Database Backup Timestamp If ESE detects any inconsistency, it performs what is called a soft recovery. In a soft recovery, the transaction logs are replayed to locate any transaction not yet committed to the database. With multiple databases in a single Storage Group, the processing can be quite complex. All the databases in a storage group use the same set of transaction logs and it is possible that each database is in a different state of consistency. Luckily for administrators, ESE handles this all in the background without any administrator interaction required.
Figure 3: Viewing Transaction Log Not much to see, but that is not to say looking at a transaction log is completely useless as there is some useful information that can be found. If you scroll through a log file you can find header information (see Figure 4) and data, but because the transaction logs are limited to 5MB in size, the data can end up being spread over multiple transaction logs. As an example, lets say that a user sends a 6MB Excel file; the first 500KB maybe written to a transaction log, filling it up and triggering the creation of a new log. This next log could be compromised of the next 5MB of the Excel file, and the rest of the data going into a third transaction log.
Figure 4: Message Headers Along with the header information, you will also be able to see timestamps for when the log was created and a unique signature matching the transaction log to the database. The signature is important as it ensures transactions are committed to the proper database. If you tried to replay transactions to a different database the outcome could be disastrous.
Figure 5: Log Header Information Starting from the top we will see:
Base name The base name will show as e00 as that is the name of the log when it was generated. Once full, it is renamed and a new e00 log is generated. Log file This is the current name and location of the log file. lGeneration This is the generation number. In this example the number is 36307 meaning that there are 36306 previously generated logs. Checkpoint This line specifies which position the checkpoint file (e00.chk) was in when this log was created. This example says NOT AVAILABLE which is not a concern as another log file will have this information. Creation time This is the time when this log was created. Prev gen time This is the time when the previous log was generated. Env Systempath Specifies the location of the checkpoint file. Env LogFilePath Specifies the location of the transaction logs. Signature The ties the transaction log to a specific database. Circular logging This information determines if circular logging is enabled or disabled.
One piece of information to note from this is the time between the Creation time and the Prev gen time. In this example the time difference is 49 minutes and 37 seconds, which tells us that this server is not under much load. If the time stamps were a lot closer together, this would indicate a server that is under load. The second half of this dump file (see Figure 6) contains the database information. Each storage group has its own set of transaction logs and each storage group can contain multiple databases. This section of the log specifies which databases these log files belong too.
Perform regular full backups to commit the transactions and flush the logs. Transaction logs create a high write load and should be moved to a dedicated drive that can support a heavy write load. Protect transaction logs by placing them on a redundant array. RAID 1 is ideal for transaction logs. RAID 5 is not as write friendly and RAID 1+0 is overkill in most instances. Ensure there is enough room on the drive to contain all the transaction logs created between backups. If the drive runs out of room, the MTA will stop and Exchange will stop functioning. Do not place transaction logs on compressed drives as they will need to be decompressed whenever Exchange access them. This will slow the system down. Do not use circular logging except in cases where the Exchange server does not hold any mailboxes (i.e. NNTP servers).
Basics
Windows 2003 has a Built-In Backup program called NTBACKUP which you can use to backup your Windows environment and when you had installed Exchange 2003 on this system, NTBACKUP is enhanced to allow backups of your Exchange Server databases. NTBACKUP features
Local and remote backup of data Exchange Backup ready Scheduled Backups Volume Shadow Copy support Integration with Removable Storgae from Windows 2003
How do you enhance NTBACKUP with the capability to Backup Exchange 2003 without installing Exchange Server? You must install the Exchange System Manager on the Backup Server to backup Exchange Server. It is possible to backup the Exchange Server without Exchange System Manager with the following trick: Copy ESEBCLI2.DLL from the Exchange 2003 CD into the EXCHSRVR\BIN folder Add the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\DLL Paths REG_EXPAND_SZ - esebcli2 - c:\exchsrvr\bin\esebcli2.dll. After modifying the registry you can use NTBACKUP to backup the remote Exchange Server by clicking Tools Remote Store.
Because the NTBACKUP program has no such requestor, organizations must use thirdparty backup applications or implement Exchange 2003 SP1 in its organization. Backup choices
Minimum selection is the storage group (SG) to truncate log files VSC can create a Snapshot from multiple SG at the same time
Restore choices
You can choose the entire storage group or a single database or multiple databases from a single SG Exchange 2003 RTM supports full backups and copy backups All databases must be mounted to purge logfiles
Backup
To start the Backup process click Start Run NTBACKUP.
Figure 1: Start the Backup process During an online backup, the .edb, .stm, and .log files that comprise the Exchange store are being backed up and checked for corruption. The Exchange database store is checked for corruption at file system level. File system level damage may be caused by unreliable hardware, firmware, or disks. This check is done by verifying the checksums on each 4 KB block or page in the database. If there is a checksum failure, backup will terminate (Exchange will not allow you to back up an Exchange store with a wrong checksum in it). This is tpyical for the 1018 error.
Figure 4: The running NTBACKUP process You can see the status of your Exchange Backups when you start the event viewer and select the application log.
Differential
Backs up selected files and marks each Backup Logfiles and delete file as backed up Transaction Logfiles Backs up selected files, but does not Backup Logfiles but doesnt mark any as backed up delete Transaction Logfiles Backs up selected files only if they were Backup only Logfiles but created or modified since the previous cannot be used with enabled backup circular logging Backs up selected files only if they were Backup only Logfiles but created or modified since the previous cannot be used with enabled backup, but does not mark them as circular logging. Logfiles will backed up not be deleted after Backup
The type of Backup depends on the configuration of circular logging. You can specify circular logging settings at the Exchange Storage Group level.
Restore
After a succesful Backup it is possible to do an Exchange Server restore in case of emergency.
You must ensure that the Exchange database store to restore is not mounted. You can dismount a Exchange Database Store in the Exchange System Manager by right clicking the database. Start the NTBACKUP program and select Restore and Manage Media.
Figure 7: NTBACKUP restore process In the following screen you must select the Server to restore the data, a temporary location for log and patch files (this directory must be empty). Click Last Restore Set when this is the last restore device (this is also possible with ESEUTIL) Click Mount Database after Restore if you want to automatically start the restored database.
Figure 8: restore options Depending on the size of the database, the restore process can be very time consuming.
Figure 9: Restore Progress You can read the Logfile after an successful or unsuccessful Exchange restore.
Figure 10: NTBACKUP Logfile The following screenshots shows the Exchange Server MDBDATA directory. As you can see, there are now more Exchange Server Transcation Logfiles except the actual logfile.
Conclusion
The Built-in Windows 2003 NTBACKUP program is suitable for small and medium sized Exchange organizations which don't want to buy an expensive Third Party Backup program.
2.Copy backups:- All files that have been selected are backed up, regardless of the setting of the archive attribute.
3.Differential backups:- Designed to create backup copies of files that have changed since the last normal backup. The presence of the archive attribute indicates that the file has been modified and only files with this attribute are backed up. 4..Incremental backups:- Designed to create backups of files that have changed since the most recent normal or incremental backup. The presence of the archive attribute indicates that the file has been modified and only files with this attribute are backed up. When a file is backed up, the archive attribute is cleared. If the file is later modified, this attribute is set, which indicates that the file needs to be backed up. 5..Daily backups:- Designed to back up files using the modification date on the file itself. If a file has been
No Circular Logging
Log Numbers 1234 12345 123456 Disk Usage 20 MB 25 MB 30 MB
Circular Logging
Log Numbers 1234 2345 3456 Disk Usage 20 MB 20 MB 20 MB
Exchange 2007 extends AD schema Microsoft Active Directory uses the Schema to represent the classes, attributes and objects that are used to display what you can see in the GUI of the Active Directory
Users and Computers Snap In or other Snap Ins. The schema is part of the Schema partition in Active Directory and the Schema partition will be replicated through all Active Directory domain controllers in the Forest. Because Active Directory schema changes are an important part of a healthy Active Directory environment, only members of the Schema Administrators and Enterprise Administrator groups have the right to extend and manage the Active Directory schema.
Requirements
Since Exchange Server 2007 is a 64bit application, you cannot install Exchange Server 2007 on a 32bit Server; but it is possible to use the Exchange 2007 32bit version for Active Directory Schema extension. It is possible to extend the Active Directory schema with a 32bit trial version of Exchange Server 2007 on a 32bit Windows 2003 machine. You should always use the Active Directory Schema Master for expanding the Schema for Exchange Server 2007 because of replication traffic. Exchange Server 2007 prerequisites: A successful Exchange Server 2007 installation depends on a lot of prerequisites. You will need the following updates before installing Exchange Server 2007:
Windows PowerShell 1.0 English-Language Installation Package for Windows Server 2003 (KB926139) Microsoft .NET Framework Version 2.0 .NET Framework Update for .NET Framework Version 2.0 Microsoft Management Console (MMC) 3.0 if Windows Server 2003 R2 is not used
It is one of these parameters that you need to extend the Active Directory schema Setup.com /prepareschema This setup parameter is responsible for adding additional schema attributes to the Active Directory Schema which will be used by Exchange Server 2007 and its subsystems. This Setup parameter is used in conjunction with the Setup.com / PrepareLegacyExchangePermissions parameter, if Exchange Server 2007 is installed into an existing Exchange Server 2003 environment. Setup /PrepareLegacyExchangePermissions This setup parameter prepares Exchange Server 2003 for interoperability between Exchange Server 2003 and Exchange Server 2007. It requires Enterprise Administrator rights and will be executed as part of the /PrepareSchema switch. Read more about this setup switch at http://technet.microsoft.com/en-us/library/bb125224.aspx. You only have to do this if it is a fresh Exchange Server installation.
Figure 1: Schema extension files The following image shows an example of a Schema definition file. The file you will see here is called Schema0.ldf. This file and others will be imported during the Exchange Server 2007 installation or the manual execution of Setup.com /prepareschema.
Use ADSIEDIT to view all Schema extensions during Exchange Server 2007 setup
You can use ADSIEDIT to view all Schema entries in the Schema partition of Active Directory. ADSIEDIT is one of the Windows Server 2003 Support tools which you can find on the Windows Server 2003 installation CD.
Figure 3: Active Directory Schema partition after Schema extension Setup.com /preparedomain If you have other domains in which you would like to install Exchange 2007 Server, execute the following command: setup.com /PrepareAD
Extends the Active Directory schema with new classes and attributes. Creates the property sets for Exchange Server 2007 and Exchange Information and Exchange Personal Information. Adds the appropriate attributes to the Exchange Information and Exchange Personal Information property sets.
Exchange Server 2007 SP1 comes with a lot of additional Schema extensions:
ms-Exch-Foreign-Forest-Public-Folder-Admin-USG-Sid,<SchemaContainerDN> ms-Exch-Internal-NLB-Bypass-Host-Name,<SchemaContainerDN> ms-Exch-Mobile-Additional-Flags,<SchemaContainerDN> ms-Exch-Mobile-Allow-Bluetooth,<SchemaContainerDN> ms-Exch-Mobile-Allow-SMIME-Encryption-AlgorithmNegotiation,<SchemaContainerDN> ms-Exch-Mobile-Approved-Application-List,<SchemaContainerDN> ms-Exch-Mobile-Max-Calendar-Age-Filter,<SchemaContainerDN> ms-Exch-Mobile-Max-Email-Age-Filter,<SchemaContainerDN> ms-Exch-Mobile-Max-Email-Body-Truncation-Size,<SchemaContainerDN> ms-Exch-Mobile-Max-Email-HTML-Body-TruncationSize,<SchemaContainerDN> ms-Exch-Mobile-Min-Device-Password-ComplexCharacters,<SchemaContainerDN> ms-Exch-Mobile-Require-Encryption-SMIMEAlgorithm,<SchemaContainerDN> ms-Exch-Mobile-Require-Signed-SMIME-Algorithm,<SchemaContainerDN> ms-Exch-Mobile-Unapproved-In-ROM-Application-List,<SchemaContainerDN> ms-Exch-Standby-Copy-Machines,<SchemaContainerDN>
Please note: There are many more Schema changes during Exchange Server 2007 SP1 setup but I cannot list all the changes in this article. If you are interested in what changes occur read the following article. Verifying Exchange Server 2007 SP1 schema extensions It is possible to verify the Active Directory schema extensions with ADSIEDIT which is one of the Windows 200x support tools. Navigate to: CN CN=ms-Exch-Schema-Version-Pt,CN=Schema,CN=Configuration,DC=DN-offorest-root-domaincontroller In the Attribute Editor tab, scroll down to the rangeUpper attribute. If Exchange 2007 Service Pack 1 Beta 2 has extended the schema, the value should be 11116. If you are using the RTM version of Exchange 2007 the value should be10637. For Exchange 2003, the value should be 6870 and Exchange 2000 should return a value of 4397.
Conclusion
In this article I showed you how the Exchange Server 2007 setup extends the Microsoft Active Directory schema and why Active Directory schema extensions are necessary. I also showed you how the Exchange Server 2007 SP1 setup adds additional schema changes. RUS in Exchange 2003
The Recipient Update Service (RUS) is a very important component in your Exchange installation, it is RUS that is responsible for updating address lists and email addresses in your Active Directory. Many people ask a simple question, "I just created a new mailbox, but when I look at the users properties in Active Directory Users and Computers, nothing is listed on the Email Address Tab, what did I do wrong?", well the simple answer is nothing, the RUS takes it's time to update all the information in AD, so give it some time and everything will appear. What we will discuss here is how to ensure that the RUS is running correctly and some issue with using RUS in a multiple domain environment. By default your organization will have two RUS objects (Figure 1):
Figure 1 a. The "Enterprise Configuration" Recipient Update Service is responsible for the updating of the email addresses for the system objects such as the Message Transfer Agent (MTA) and System Attendant. The "Domain" Recipient Update Service is responsible for the updating of the address information for recipient objects in the domain that it is responsible for, in Figure 1 our domain is NWTRADERS
b.
To adjust the properties for the Recipient Update Service, right click over the service and then select Properties, the properties for the Recipient Update Service will now be displayed (Figure 2).
Figure 2 Field Domain Exchange Server Description This is the domain that is serviced by this Recipient Update Service. This is the Exchange server responsible for the creation and updating of the address list for the domain specified in the Domain field.
The Windows 2000 Domain Controller that this Recipient Windows 2000 Update Service will connect to when it creates and Domain Controller updates the address list. Update Interval How often the Recipient Update Service will run, if you leave it selected to "Always Run" it will update once every minute.
It is possible to force the Recipient Update Service to start processing, you have two options "Update Now" or "Rebuild", and both of these options are available by right-clicking on the Recipient Update Service. The "Update Now" option will update the address list with changes, the "Rebuild" option as its name implies will completely rebuild the address list. In most cases, having a single Recipient Update Service for each Active Directory domain will be sufficient, but if you have a single AD domain that spans across different physical locations it is recommended that you create a Recipient Update Service in each Active Directory site, and also ensure that you have a Global Catalogue server in each Active Directory site also. OK, so we now understand what the Recipient Update Service does and how to configure it, lets look at a bit of troubleshooting. The first step in troubleshooting the Recipient Update Service, like most other services is to check the Event Log, we are looking for the events that originated from the MSExchangeAL service.
The next step in troubleshooting the Recipient Update Service is to use ADSI Edit to check a mailbox that should appear in the Global Address List. We need to check and see if the "showInAddressBook" attribute is populated (Figure 3)
Figure 3 If the "showInAddressBook" attribute is not populated, the Recipient Update Service may not yet have run, in most cases manually forcing the Recipient Update Service to run will resolve the problems. From time to time organizations have multiple Windows 2000 domains that host user's accounts but may only have one domain that hosts their Exchange 2000 servers. In these scenarios we need to create additional Recipient Update Services, or our Global Address List, will not be populated with the account information from the domains that do not host the Exchange 2000 server. The diagram below shows the scenario that we will use. We have two domains "nwtraders.msft" and "research.nwtraders.msft", our Exchange server (London) is located in the "nwtraders.msft" domain, but we have users in "research.nwtraders.msft" who have mailboxes on our Exchange 2000 server, so we will need to create a Recipient Update Service to keep our Global Address List in order.
The first task that we must perform is to run the Exchange 2000 setup program with the DomainPrep switch in the "research.nwtraders.msft" domain, the person creating the new Recipient Update Service must also have Full Administrative Access on the Exchange 2000 Organization object. The next step is to create the new Recipient Update Service on the Exchange 2000 server (London).
1.
Open the Exchange System Manager, expand the Recipients container, click on the Recipient Update Services container, you should now be shown the existing Recipient Update Services
2.
Right click over Recipient Update Services and then select New > Recipient Update Service from the menu, the "New Object - Recipient Update Service" dialogue box will now be displayed (Figure 4).
Figure 4
3.
In the Domain box, we need to select the Windows 2000 domain that contains the user's accounts that this Recipient Update Service will manage, in our case "research.nwtraders.msft", you can use the Browse button to select the domain, then click Next.
4.
In the next dialogue box (Figure 5) we will select the Exchange 2000 server that will be responsible for updating the Global Address List for this Recipient Update Service, in our case "London", click Next.
Figure 5
5.
You will now be shown a summary of the parameters that you entered (Figure 6), if they all appear to be in order, click Finish to complete the setup of your new Recipient Update Service.
Figure 6
Well that concludes my brief rundown of the Recipient Update Service in Exchange 2000, and hopefully this has given you a better understanding of this important Exchange 2000 service.
Introduction
The Recipient Update Service (RUS) is the component of Exchange that is responsible for generating mail proxy addresses for all mail-enabled objects in an Exchange organization. Normally this component runs in the background and requires little configuration or maintenance. There are times when issues do occur and fixing them should be a top priority in order to keep mail flowing. Lets take a look at what the RUS does before going into some common troubleshooting steps: Description of RUS When you perform the initial install of Exchange, the Recipient Update Service is installed and a default recipient policy is created. This policy is responsible for ensuring that all mail-enabled objects in the Exchange organization have a valid SMTP address following the username@domain.com naming format. You can create a new policy that can be configured to create each SMTP address following a different naming convention such as Firstname.Lastname@domain.com. Microsoft has a list of best practices to follow when creating and/or editing recipient policies.
Create a new recipient policy and assign it a higher precedence rather than editing the default policy Keep the number of recipient policies to a minimum Rebuild the RUS with caution
A lack of understanding of the RUS is the major cause of issues. Often administrators apply a policy without understanding what will be changed. Exchange does not provide much warning about the impact a change will make. On top of that, organizations using a 3rd party application to create and assign SMTP addresses, through MIIS for example, can cause further damage by applying recipient policies blindly. So what do we do when RUS takes a vacation? Troubleshooting Common Issues with RUS As previously mentioned the Recipient Update Service runs quietly in the background and requires little or no maintenance. When issues do occur there are three basic steps to troubleshooting RUS.
Enable Diagnostic Logging Choose an object, or objects to monitor View the Application Log for errors
To begin troubleshooting RUS we first determine if we have more than one recipient policy, if so, set the schedule for all but one to Never Run. In the case of multiple policies, you may be required to go back and enable another policy if you find nothing wrong with the first. Just ensure that only one policy is scheduled to run at a time.
Diagnostic Logging
With the schedule configured, the next step is to enable Diagnostic Logging and set it to log the maximum amount of events on the following services and objects. MSExchangeAL LDAP Operations MSExchangeAL Address List Synchronization MSExchangeSA Proxy Generation To do this, open up Exchange System Manager (ESM) and drill down to Administrative Groups | Servers | Servername, right click the server you wish to configure logging on and select Properties and then go to the Diagnostics Logging tab. Under Services, select MSExchangeAL and set LDAP Operations and Address List Synchronization to maximum (see Figure 1). Next select the MSExchangeSA services and set Proxy Generation to maximum. (Note: Proxy Generation is not available on Exchange 2000 servers). Finally create a test object for the RUS to stamp.
up ADSIEdit and connect to the Domain Controller that RUS is pointing to. Drill down to Domain NC | DC=domain,DC=com | CN=Users (or wherever you created your test object) and right click the test object selecting Properties. Locate the uSNChanged value and record it. Next open up the Event Viewer on the Exchange server you have enabled logging on and search for Event ID: 8011 Description: Base DC When the search is done, open the first event in the list that will include information about the last search completed by the RUS. Locate the line in the event (USNChanged>=########)(uSNChanged<=########) and determine if the value you recorded for the test object falls into this range (see Figure 2).
If the uSNChanged attribute is higher that the range shown in the event, RUS has not queried this object yet. This is common when you have selected to rebuild the RUS which, depending on the size of the domain can take anywhere from a few
minutes to a few days. When you initiate a Rebuild, the RUS starts from scratch with a uSNChanged value of 1 and queries all objects in the domain. If uSNChanged attribute value for the test object falls below the range in the event RUS has passed this object. Open up the next oldest event with ID 8011 until you find the event that contains a range that the value falls into. If you have trouble finding it you can modify the test object and wait for it to be queried again and find the event for it. If the Application log does not contain an event ID 8011 that has "Base 'DC" in the event description, the domain RUS has not started processing yet or the event was over written by more recent events. A rebuild operation can cause this as it can generate a large number of these events.
You can determine if a rebuild operation is taking place by lowering Diagnostics Logging on all items except the Address List Synchronization object and then filter event 8329, which specifies the start of a rebuild operation. At intervals of 10% event ID 8332 will be logged and when the rebuild is complete event ID 8330 will be recorded. If you see event 8329 and 8332, but no 8330 then the rebuild is still running and you should wait until it is complete.
If there is no 8012 event generated after an 8011 event occurs, Exchange did not receive a response to the search. This is usually indicative of a network issue where the Exchange server cannot contact Active Directory. When the search returns zero objects (and you are certain there should be some); this usually indicates a permissions error. In this case the Exchange server computer account does not have the correct permissions required to view user objects. In order to view user objects, the Exchange server needs to be part of the Exchange Enterprise Servers group. If more than 20 objects are found you will see multiple 8012 events, as results are returned in groups of 20. You should see one 8012 event for every 20 objects that the query returned.
Conclusion
The Recipient Update Service is a fundamental component of Exchange and ensuring it is up and running is crucial to a properly functioning Exchange envirmoment. Hopefully these steps help you determine the cause and aid in finding a resolution in the unfortunate event that something does go wrong.
Internet, so it may well be your ISP or Hosting company that would be responsible for the DNS records that your organization needs, so make sure you speak with them about your needs. So back to our message that we need to send to bill@123.com, as we said it is sitting in a queue waiting to get sent, the figure below illustrates the concepts of sending the message.
Server1.abc.com now know what server is responsible for receiving mail for the domain 123.com, because the Public DNS server told it. Server1.abc.com will now establish a TCP session using port 25 (SMTP) with server1.123.com and the message will be transferred, nothing too complicated about that.
The message is going to be received by server1.123.com, and held in a queue, server1.123.com, is now going to use the Active Directory to locate the actual mailbox that the message should be delivered too, in our case the mailbox might be on server1.123.com, in which delivered or the mailbox may reside on another the case the message will be routed by Exchange to (this is shown in the next diagram).
So the question is how does Active Directory know what mailbox belongs to bill@123.com, well when you install Exchange 2000 it creates what is know as a Recipient Policy and the Recipient Policy is used by the Recipient Update Service (RUS) to create the users SMTP email addresses, the Recipient Policy also tells Exchange what SMTP domains it has the responsibility for. You can find your Recipient Policy by performing the following steps: 1. 2. Open the Exchange System Manager Expand Recipients
3.
4. You will now see the Default Recipient Policy listed in the right-hand pane of the screen 5. If you right-click on the Default Recipient Policy and select Properties, you can navigate to the E-Mail Address tab and see the address that will be created for you, see the figure below:
So from the figure above we can see that Exchange will create an SMTP address for our users using the @123.com address, notice the checkbox This Exchange Organization is responsible for all mail delivery to the address, if this was not check Exchange would reject the mail, even though the Public MX record for 123.com pointed to this server. The information I have provided above, summarizes the steps used to send SMTP mail, so back to the real question, how do you get your Exchange environment to host more than one SMTP domain? Our organization currently holds the mail for 123.com by we also want it to host mail for 456.com, as we have seen one of the crucial elements is the MX record, so you will need to ensure that an MX record as been created for 456.com that point to the server who is responsible for his mail. So when a user at abc.com sends some mail to a user at 456.com, our sending SMTP server will query the Public DNS server for an MX record for 456.com, this will be resolved to server1.123.com, which in turn will be resolved to the IP Address 192.168.1.100
But as we know, we have to tell our Exchange organization that it is responsible for handling the mail for 456.com, this is done using our Recipient Policy, so if every user in our organization should have two email addresses one for 123.com and one for 456.com we can edit our Default Recipient Policy and add the extra address as detailed below: 1. 2. Navigate to the Default Recipient Policy and open its properties. Click on the E-Mail Addresses tab and click on New
3.
4. In the Address box enter the SMTP domain name (including the @ sign) that you would like to create, make sure that you also check the This Exchange Organization is responsible for all mail delivery to this address
5. Click OK, and you will now see the E-Mail Address tab as shown below, make sure you check the box next to the new SMTP address you just created. 6. You will notice that one of the SMTP address is listed in bold, this is referred to as the Primary SMTP Address and will be used as the reply address you need to chance this, highlight the SMTP address like to make the Primary and click on the Set as
7. Click OK to exit out of the Default Policy Properties, you will be prompted with a dialogue box asking if you would like to apply the new policy, click Yes to the question. I would suggest you force the Recipient Update Service to run, this will ensure that your users accounts are updated, for more information on the Recipient Update Service look at this article:
not been sent a copy of the hierarchy. In this case, the first thing you need to check is whether the public information store on the new server is correctly configured with an email address. Both public folder hierarchy and content changes are sent between Exchange servers via email messages, so it therefore follows that if the public information stores email each other, theyll need an email address as well as a working email path between them. Its the job of the Recipient Update Service (RUS) to ensure that objects are stamped with correct email addresses, and this includes the public information store. The first thing you should therefore check is the public information stores email address which can be performed with ADSIEdit. Youll find that ADSIEdit is installed as part of the Windows 2003 Support Tools. With the ADSIEdit snap-in open, right-click ADSI Edit and choose the Connect to option. In the resulting Connection Settings window, ensure that the naming context is set to Configuration and then click OK. This will then bind to the configuration naming context allowing you to navigate your way down to the public information store. You will therefore need to work your way down the tree accordingly: CN=Configuration, DC=domain, DC=com CN=Services CN=Microsoft Exchange CN={your Exchange organization name} CN=Administrative Groups CN={relevant administrative group} CN=Servers CN={relevant Exchange server} CN=InformationStore CN={relevant storage group}. In the example in Figure 1 below, you can see that Ive navigated my way down to the First Storage Group on the server DCEXCH1.
Figure 1: Public Store Location Using ADSIEdit Now what you do is to right-click the public folder store shown in the right-hand pane and choose Properties from the context menu. In the resulting property window, find the proxyAddresses attribute and click the Edit button. This will bring up a similar window to the one shown below in Figure 2. Here you can see an example of a public information store that has been correctly stamped with both SMTP and X.400 email addresses; if there were no values in the proxyAddresses attribute, wed instantly know that the RUS has not run against this store and would therefore need to start troubleshooting the RUS. Note that its the enterprise RUS and not the domain RUS that is responsible for stamping email addresses on system objects like the public information store, so make sure youre looking at the correct one. During my early migrations from Exchange 5.5 to Exchange 2000, I remember that one of the most common reasons for the RUS not stamping email addresses on new Exchange 2000 servers was due to a missing proxy address generator, such as those used by fax gateways for example. Other common reasons included invalid domain controllers or Exchange servers listed in the properties of the RUS. Check out the Links section at the end of this document for some useful links for troubleshooting RUS issues. One other thing worth noting is the format of the SMTP address assigned to the public store: servername-IS@domain.com. There will be more on the significance of this later in this article.
Figure 2: Public Store Email Addresses I mentioned earlier that as far as public information stores are concerned, they need both an email address and a valid message path in order for the replication messages to be successfully sent and received. As far as the valid message path is concerned, note that its a good idea to check whether you have any transport links that disallow system messages. This check process can be made really easy by using the Winroute tool. For more information on using Winroute, see the Links section at the end of this article.
Replication Messages
Increasing the diagnostics logging level of your Exchange server is always useful when troubleshooting issues. Diagnostics logging levels can be set for the MSExchangeIS Public Folder object found on the properties of an Exchange server object in Exchange System Manager. In the case of public folder issues, Ive seen many Microsoft PSS professionals in mailing lists and newsgroups recommended to set the diagnostics logging categories shown below in Table 1 on both the source and destination servers involved in the replication process. Doing so will allow you to get a clearer picture of what is happening during the replication process. For this article, were going to concentrate mainly on the replication incoming and outgoing categories. Category Replication AD Updates Replication Incoming Messages Logging Level Maximum Maximum
Replication Outgoing Messages Non-Delivery Reports Replication Backfill Replication General Replication Errors
Table 1: Recommended Diagnostics Logging Settings Once logging has been set, the application event logs on both source and destination servers should then begin to produce more detailed information about the replication messages as and when those messages begin to flow. There are several different types of replication message. Table 2 below shows the message type, purpose, direction of message flow and additionally the associated event ID. Type 0x2 0x4 0x8 Purpose Hierarchy Replication Content Replication Backfill Request Direction Outgoing Incoming Outgoing Incoming Outgoing Incoming Outgoing Event ID 3018 3028 3020 3030 3014 3024 3019
0x80000002 Backfill Response (Hierarchy) 0x80000004 Backfill Response (Content) 0x10 0x20 Status Replication Status Request
Table 2: Message Types Lets take the first message type, the hierarchy replication message, as an example. If you create or delete a public folder, or perhaps change that folders permissions, a hierarchy replication message will be generated from the source server to the destination server. As you might expect, the sending of the hierarchy replication message maps to the Replication Outgoing Messages diagnostics logging category in Table 1 above, whilst the receiving of the hierarchy replication message maps to the Replication Incoming Messages category. It therefore follows that for each outgoing message from the source server, there should be a corresponding incoming message on the destination server.
Consider the example replication message shown below in Figure 3. Here you can see that the event category is shown as Replication Outgoing Message. The reason for this message is clear when you examine the event details. The Type is set to 0x2 (hierarchy) and the affected folder is called New Test Folder; this was a new public folder that I had created.
Figure 3: Outgoing Replication Message Event As I just mentioned, it is normal to expect to see a corresponding incoming replication message on the destination server. Therefore, if you were troubleshooting a situation where Outlook clients connected to the destination server were not displaying the New Test Folder public folder, the next step would be to examine the event log on the destination server for the corresponding incoming message event to make sure it had been received. This event would have a category of Replication Incoming Message and an event ID of 3028. You would expect the event description to again list a message type of 0x2, since this would be a hierarchy message, and also to contain the folder name of New Test Folder. Naturally the same process applies to the other types of replication message. For example, if a user posts a new message to a pubic folder, or perhaps modifies an existing post in some way and saves it back to the public folder, the entire post is replicated and will be seen in the event viewer as a content replication message (type 0x4) with event
IDs 3020 or 3030 depending on whether you are examining the source or destination server. Backfill request and response messages are part of the backfill process which itself is the process whereby public stores that detect they are missing some replication updates rerequest these updates from other public stores. The backfill request is sent as message type 0x8, whilst the response message will be type 0x80000002 for hierarchy messages and 0x80000004 for content messages. Figure 4 below shows an example of a content backfill response message. In this example, server DCEXCH1 is responding to server EXCH3s backfill request for the public folder called Items For Sale.
Figure 4: Content Backfill Response Replication Message Event The final replication message types are status replication, type 0x10, and status request, type 0x20. These are used by the public information stores to allow the receiving store to ascertain as to whether it is synchronized with the sending store.
transport. Probably the first thing to do would be to use the Message Tracking Center in Exchange System Manager in an attempt to see what is happening. You can see from Figure 5 below that Ive tracked messages sent from the public folder store on server DCEXCH1. To do this, I simply typed in dcexch1-is@hobsonlabs.com as the sender of the message. Youll recall from earlier in this article that this is the SMTP address of the sending public information store; you can see that this has been resolved to the friendly display name of Public Folder Store (DCEXCH1).
Figure 5: Tracking Public Folder Replication Messages I personally find it very useful to ensure that subject logging is enabled within message tracking. You can enable this on the General tab of the properties of your Exchange server object in Exchange System Manager. You can see from Figure 5 above that enabling subject logging makes finding your messages much easier, since all messages after 22:33 have the subject field populated after subject logging was enabled. For example, it can clearly be seen that the message sent at 22:45 is a hierarchy replication message. To drill deeper into the events associated with each tracked message, it is simply a case of double-clicking the relevant tracked message. For example, double-clicking the hierarchy message sent at 23:00 reveals the Message History screen as shown below in Figure 6. Here we can see that the last entry shows us that the message has been successfully delivered to the destination server EXCH3. Had the last two entries on this screen not been present, we would have seen that the last line would have stated SMTP:
Message Routed and Queued for Remote Delivery. In this case, we would have known that the message had likely queued on our source server and hence had not been delivered. It would have then been time to check the message queues using the Queue Viewer utility in Exchange System Manager on the source server.
Figure 6: Message Tracking History One last thing to note here is that a single replication message is created even if there are multiple replicas of that public folder. For example, if a public folder is modified on server DCEXCH1 and a replica of that folder exists on both the servers EXCH2 and EXCH3, you will find that only a single replication message is generated and is addressed to both public folder stores at the same time. Naturally when tracking such a message, expect the Message History window to show this as shown below in Figure 7.
Summary
The most important thing to remember about public folder replication is that its message-based replication. Knowing this means that, once youre confident that the public stores have correct email addresses, you can use standard tools in the form of Exchange System Manager and the Event Viewer to start troubleshooting your replication issues. Of course, there are always more complex issues that could arise that are way beyond the scope of this article, but hopefully this article has given you a starting point. PF Part1 the administrator, public folders are a separate database.
Figure 1 This screenshot shows the Exchange databases on a single Exchange 2003 Standard server. The priv1 database, composed of both an EDB and an STM file, contains the user mailboxes. The pub1 database contains the public folders. Both databases in Exchange 2000 and in Exchange 2003 up to SP2 had a limit of 16GB. In the fast moving Internet days, 16GB is not much. However, most mail accumulates in user mailboxes, leaving the public folder database pretty empty. Later on I will show how public folders can be better used to even this out, so you get a smaller mailbox database and more room to grow. Public Folders contain the same type of folders you can access using Outlook, and can hold mail, calendaring and task information. You can set security on these folders so that only specific people will have specific types of access to these folders.
Figure 2 Outlook 2003 sort of hides the public folders, so you first have to access the Folder list, then on the right side, open the Public Folder List, All Public Folders and the select New Folder
Figure 3 Exchange System Manager can only create folders that hold mail items, such as your Inbox and Sent Items folders, while Outlook can also create other types of folders such as Calendar items.
Figure 4 You can also create Public Folders using Outlook Web Access.
Figure 5
Figure 6
Figure 7 In most cases, in a small to medium company you would mostly need to set client permissions and not administrative rights. These can be set by the Outlook client and Exchange System Manager, but not the Outlook Web Access client.
Figure 8
Figure 9 The above screenshots show the default security settings for Public Folders. The owner of a public folder is the user who created it and gets full control of the folder. Authenticated users (designated here as Default) are granted the right to add items and delete their own items and anonymous users can add items but not read them. When creating a new public folder that you want a user to administer, you can simply add the user to the permission list and change the permission level to Owner.
Figure 10 The owner would be able to create subfolders for the folder you created and set further permissions on it.
Figure 11 This way the item becomes available to the relevant department personnel, if you set the right permissions.
Figure 12
If you are worried about mailbox limits you can also create public folders for heavy users and only grant them permissions on those folders. They can move large items to their private public folder, saving room.
Figure 13 The downside of this approach is that each user has to add this folder to his or her own Outlook Address Book so that the contacts will appear there. This is done in the Outlook Address Book tab of each folder in Outlook.
Figure 15
Public Calendar
A public calendar is also a useful tool. It can also save you money, because unlike a dedicated mailbox, a public folder doesnt require a license. You can have as many public folders as you like. So, instead of creating a mailbox enabled user in Active Directory for scheduling meetings in, say, a meeting room, you can simply create an accessible public calendar folder. Unfortunately, public calendar folders are not published in the free/busy folder, so you cant really do advanced scheduling with these folders.
Figure 16 If you use Outlook 2003, after adding a folder to the public folder favorites you will be able to access it using the regular sections without the need to browse all the way to the folder list.
Figure 17 If you have public calendar favorites, you will be able view them side by side to determine which calendar is free and compare them to your own calendar.
Figure 18 You can also modify your Exchange account in Outlook 2003 Cache mode to download public folder favorites so they are automatically available offline. From the Outlook menu choose Tool > E-mail Accounts > Exchange Server Settings > More Settings > Advanced.
Figure 19
Conclusion
Public folders is a handy and powerful tool, not always the easiest to set up, but providing many benefits and granularity. If you know how to use it, it can also save you time and money. This article covered the basic of the basics. Public Folders, like most features in Exchange, are full of hidden surprises. I will try to cover more of that in a future article.
PF Part2 Like mailboxes, public folders can also receive mail from the Internet. By default, public folders do not receive e-mail until you mail enable them. This is how the property pages of a public folder look when they are not mail enabled.
Figure 1 A public folder is mail enabled by using the Exchange System Manager.
Figure 2
Once a public folder is mail enabled we get additional property tabs, similar to those you might recognize from mailbox enabled users.
Figure 3 You can change or add e-mail addresses to suit your needs by editing the SMTP address or adding a new one. By going to the content tab of the public folder and entering an account with access to the folder you get a mini Outlook Web Access interface. I have sent an e-mail from the Internet to the public folder. Notice that instead of the envelope icon you get a post it icon, since Exchange treats e-mails sent to a public folder as posts.
Figure 4 This means you cannot reply or forward this e-mail. However, using the Outlook client, you can overcome this by using the Reply or Forward buttons in the toolbar without opening the post itself.
Figure 5
Folder Automation
While the word "automation" is generally associated with scripts and programming, Outlook and Exchange provide some surprisingly strong automation features that require no scripting at all. The Administration property page of a public folder is where this magic happens. When you press "Folder Assistant" you will be presented with a few automation options that you can create by using rules.
Figure 6
Figure 7 If this looks familiar to you, you might have seen a similar dialog box for it in the Out of Office options.
Figure 8 Don't forget that you also have some Advanced options.
Figure 9 I've used the folder assistant many times in conjunction with other software automation. For example, a service on the Internet would send some data to the public folder and it would then, according to the subject, forward the information to the right person or group. Another such scenario is a public folder which contains information that needs to be accessible to an outside contractor. Sure, you can set up some other forms of Internet access, or instead, simply forward relevant information to your outside contractor or field agents. The "Reply with Template" option can be used in help desk scenarios where the customer needs to know that someone is taking care of him or her. Sure, this can also be done with a mailbox, but remember: public folders are cheaper and more accessible. Also, the rules run server-side even when Outlook is running. Another way that this can be used is to help you better monitor backups. Most backup utilities will e-mail you the results. After a while, if you're a busy administrator, there's a chance you might start ignoring their sometimes cryptic messages which can get lost inside your mailbox, and sometimes even find their way to your Junk e-mail. Instead,
why not set up a public folder for these messages, and have the folder just forward critical errors? On second thought, why stop there? Most applications these days will send you some kind of a notification, and monitoring applications, such as Microsoft Operations Manager, will send you even more. I find that using a public folder is more tidy while leaving you the option to forward important mail to your mailbox, or perhaps an Internet pager. You might have noticed that a folder can also be moderated as well, by using the Moderated Folder button in the Administration tab of the public folder property pages. For more information about this you can read this excellent article.
Shortcuts
What happens if you wish to access a specific public folder? Inside Office you can use a hyperlink to access your folder. The easiest way of constructing such a link is by using the web toolbar in Outlook. To enable this toolbar right click the toolbar area and choose Web.
Figure 10 On the web toolbar you will now see the link to whatever folder you are using at the moment.
Figure 11 You can insert these hyperlinks to an e-mail message or any other office document by using brackets.
Figure 12 After pressing Enter the hyperlink will be created. Such hyperlink will also work in Internet Explorer but you will have to replace Spaces with %2520. In our example the URL will become: outlook://Public%2520Folders/All%2520Public%2520Folders/Test You will probably also get some security warnings too, for trying to launch an application from Internet Explorer. Alternatively, you can use an Outlook Web Access URL such as this: http://exchange/public/test. Unlike the Outlook client URLs, spaces are replaced by %20, like this: http://exchange/public/Internet%20Newsgroups/ Make sure to replace, in the above examples, exchange with the name of your own Exchange server. You can also drag an Outlook folder to your desktop or any other folder on your hard drive to create a shortcut. The shortcut is a special Office file with extension XNK. It can be copied to other machines or moved to one of the shortcut toolbars. I hate to leave without leaving a piece of coding, so here is a code for accessing a folder with VBScript (or VBA): ActiveateTempPublicFolder.vbs Set myOlApp = CreateObject ("Outlook.Application") Set myNameSpace = myOlApp.Application.GetNamespace("MAPI") Set myfolder = myNameSpace.Folders("Public Folders"). Folders("All Public folders").Folders("Test") Myfolder.display
You can use this code to create an Outlook button. First create an Outlook macro.
Figure 13
Figure 14 Then place the code into the Microsft Visual Basic Editor.
Figure 15 Now exit the editor and return to Outlook and choose Tools | Customize. From the list of categories select Macros.
Figure 17 Rename the button to make it more readable by right clicking it and choosing Name:.
Figure 18 Press Close button and now your button is ready to use. Please note that you can create your button in any Office application, not just Outlook.
Conclusion
As you can see the more you dig, the more possibilities you have for using and accessing Public Folders. With very little effort you can maximize the use of your investment in Exchange and while at it, tailor your system for ease of use.
expensive" threshold is 500. Yes, that really needs to be configurable, and we will examine doing that in a future release. We still have the feature where you can specify a specific cost list for a single server. Instead of calling into AD, we'll simply read this cost list and use whatever data it provides. Unlisted servers implicitly have an "infinite" cost, so this provides a simple method for disallowing referrals from a specific server to a set of other servers. This new source of cost information also controls the backfill picker. Previously, the code always queried for cost information once (an hour) and cached it. This cached info was used to prepare client referrals and to pick servers to ask for backfill. Now that we're querying AD instead of the routing engine, all users of that cost information benefit from the new source and all can see the same information. It's important to note that Exchange 2007 exclusively uses the AD site connector cost information, while pre-Exchange 2007 exclusively uses the data from the routing group connectors. Since all Exchange 2007 servers appear to be in a single routing group from pre-Exchange 2007's point of view, without some additional configuration you may experience truly bizarre referrals for users whose default public folder database is on a pre-Exchange 2007 server, but all the replicas live solely on Exchange 2007 servers. In this specific case, the pre-Exchange 2007 server will perceive all the Exchange 2007 servers as having the same cost (because they're all in the same routing group). Clients will get referrals all over the place. To prevent this from happening, you should set all default PFDBs for all mailbox DBs to point to Exchange 2007 as soon as you've replicated content to any Exchange 2007 servers. This will put all clients on the same referral page and you won't end up with clients in Brazil being referred to servers in Belarus. NOTE: If you have Exchange 2003 users that use OWA, you should not point the Exchange 2003 mailbox database to an Exchange 2007 public folder database, until you move all users that need OWA public folder access to Exchange 2007. To read more about this scenario, please see this blog post. - Dave Whitney
object for the public folder that basically tells us the email address and that it's a public folder proxy object. So as a result of these changes we had to change the way transport dealt with SMTP messages destined for public folders. The basic process is (for in-depth information, take a look at http://msexchangeteam.com/archive/2004/09/10/228114.aspx): 1. SMTP mail addressed to Public Folder comes into transport. 2. Transport categorizes the message and obtains a list of all PF stores for that hierarchy. We then pick a store and route the message to it: a. If a PF store is on the same server, we use that. b. If there are no local PF stores, then we'll use a PF store in the same routing group. c. If there are no local PF stores in the routing group, then we use the first entry in the list from msExchOwningPFTreeBL entry. This list always has the most recently added public folder stores to the organization at the top (and as a public folder store is created when a new server is installed, normally this means the most recently installed servers will be at the top of the list). What could happen is we pick a server that has just been added to the org and may not have the full public folder hierarchy yet (and hence not know what to do with the message). To prevent this in Exchange 2003, we don't select any stores younger than two days, unless there is no alternative. See http://support.microsoft.com/kb/328870/en-us for more information on use of PF store Age in routing decisions. 3. If there is a replica on this store, then the message is simply delivered. If not, the store returns the location of the closest replica and hands the message back to transport which then sends it on to the store which actually holds the replica. Exchange 2007 RTM In Exchange 2007 we made some fundamental changes to the way transport works (see http://technet.microsoft.com/en-us/library/aa998825.aspx for more information): No more routing groups. Instead we will use the Active Directory topology for determining the least cost route. Direct Connections are preferred. When possible the source Hub Transport server will establish a direct TCP connection with the destination Hub Transport server that resides in the Active Directory site where the mailbox or replica resides. No more passing it along a chain of Exchange servers if we can avoid it.
So how does that change Public Folder mail routing? Well let's take a look at the stages. Routing Table Calculation When the Microsoft Exchange Transport service starts on a Hub Transport Server (or when a change notification occurs, or after 6 hours), it calculates a set of routing tables that is based on the snapshot of information that is retrieved from Active Directory. The routing component refers to the routing tables to determine how to route messages to recipients. When configuration changes are made, the routing tables are rebuilt. The new routing tables are used to route new incoming messages. Messages in remote delivery
queues are also rerouted if the routing component determines that they are affected by the configuration changes. For more information on the routing table calculation, please see http://technet.microsoft.com/enus/library/aa998825.aspx. Public folder hierarchies are one item that is retrieved during the calculation of the routing tables. Specifically, to ensure that proper delivery can occur to the public folder replica, the "best" public folder store is chosen by reviewing the msExchOwningPFTreeBL directory entry (Note: The msExchOwningPFTreeBL list always has the most recently added public folder stores at the top) using the following criteria (at RTM): 1. First and foremost, if the list contains any legacy Exchange PF stores, they are removed if Exchange 2007 PF stores exist. Age of the store is not considered when building this list. 2. If the choice is between two (or more) Exchange 2007 PF stores, then the following criteria will be used: a. Age Ranking (PF store less than 2 days old by default will not be considered, unless the store's age is less than the threshold or is unknown). b. If there is a local PF store on the same physical machine, we will use that. c. If there is not a local PF store, then we will choose a store within the local Active Directory Site. If there are multiple PF stores, the first server returned is chosen. d. If there are no PF stores within the local Active Directory site, then we will choose a PF store in a remote Active Directory Site, sorted by lowest cost (if the choice is between two or more remote AD sites). If the result is a tie, the first server returned is chosen. 3. In the event that there are no Exchange 2007 PF stores, and if the choice is between two (or more) legacy Exchange PF stores, then the following criteria will be used: a. Age Ranking (PF store less than 2 days old by default will not be considered, unless the store's age is less than the threshold or is unknown). b. If there are multiple PF stores, then the first store is chosen. Transport Steps 1. The Hub Transport server receives the message and categorization is performed. When the categorizer receives a message, it looks up the email address in the directory. If the email address resolves to a public folder directory entry it knows that it has to do something special. 2. The categorizer looks up the homeMDB attribute of the public folder's directory entry. This tells it which PF hierarchy the folder belongs to (usually there's only one PF hierarchy and that's the default "Public Folders" hierarchy associated with the default public stores that get created automatically). 3. Based on the routing table calculations performed by the Transport service, the "best" public folder hierarchy store ("best" TLH) is used to determine which public folder store contains a replica that can be used for delivery. a. If the "best" TLH is located within the same Active Directory site as the Hub Transport server, then we proceed to Step 4.
b. If the "best" TLH is not located within the same AD site, we will route the message to a Hub Transport server in the destination AD site by using the routing table to determine the least cost route. We then proceed at Step 1 with the destination Hub Transport server, the message is categorized (Step 2) and the best TLH store is used to determine the public folder store that contains a replica (Step 3a). c. If the "best" TLH is located in an routing group on a legacy Exchange server, then we will route the message to the legacy Exchange server, and the legacy Exchange server will handle determination of replica based on the steps outlined in http://msexchangeteam.com/archive/2004/09/10/228114.aspx. 4. Instead of transferring the message via SMTP to the hierarchy store for replica lookup, the Hub Transport server will establish a connection to the store service on the mailbox server role that contains the "best" TLH store. The store is then queried to determine if content for the folder (referenced by legacyExchangeDN) is available (IsContentAvailable), which results in a response containing the list of servers hosting the folder if the content is not available locally (store override). The list of servers hosting the folder replica is listed in the same order provided in client folder referrals and the top entry is chosen by transport. This referral tells us to which replica we should route the message, aka, the "best" replica. 5. The Hub Transport server then uses the routing table to determine the least cost route for the server that contains the "best" replica and routes the message accordingly (see http://technet.microsoft.com/enus/library/aa998825.aspx for more information on how least cost routing occurs). 6. The message gets delivered to the public folder store. Exchange 2007 SP1 In Exchange 2007 SP1 we are altering the behavior slightly. In RTM, we prefer an Exchange 2007 PF store over a legacy Exchange PF store, regardless of age. That's right in RTM, we do not consider the age of the public folder store in that case. What does that mean? Well that means we may end up choosing a PF store that doesn't contain the entire hierarchy (as a result of just being created and not having received or processed the hierarchy replication messages). If we choose a PF store that doesn't have the hierarchy we cannot perform a lookup of the replica, and thus we'll NDR the message. That's not the best situation. So in SP1 the decision tree will be: 1. 2. 3. 4. Age Ranking - PF stores less than 2 days old by default will not be considered, unless the all of the stores are less than the threshold or the age is unknown. Proximity - local server is preferred over local AD site over remote AD site over remote RG. Cost - if the choice is between two remote AD site replicas. If still a tie, the first server in the replica list returned by AD is chosen.
Hierarchy Lookup Failure Scenarios If the Hub Transport Server cannot contact the "best" TLH store, then the message will queue on the Hub Transport Server. What are the scenarios here? Well once we choose a "best" TLH store we will stick with it unless one of the following happens: The TLH store can be contacted and we deliver locally in case of a local replica on the TLH store, or we obtain a referral for replica delivery. If the Transport service gets restarted; though this could still result in the same TLH store being chosen.
A different TLH store will only be chosen if an underlying topology change has occurred (routing table recalculated as a result of that change), and a new "best" TLH store is found during route recalculation). Message times out and NDRs.
General Queuing Scenarios So you might be thinking, besides the hierarchy lookup failure scenarios, where might a SMTP based message (and at this point it doesn't even really matter that it is destined to a public folder) queue? So for the purposes of this discussion, let's discuss each based off of the following diagram (please click on the thumbnail to see the full size):
message. If all Hub Transport servers are down within the AD site, the message will be queued on the Mailbox server until a Hub Transport server becomes available. Mailbox servers cannot communicate with Hub Transport servers that are not members of its AD site. Routing the Message to its Destination Once the Hub Transport server receives the message it will perform the steps outlined in the "Exchange 2007 RTM" section. There are two potential scenarios that could result in replica delivery queuing and they depend on the location of the replica: 1. Once the server hosting the best folder replica is chosen, if the Hub Transport server cannot contact the mailbox server that contains the PF replica (intra-AD site), then it will queue on the Hub Transport server. In this scenario, we will queue until the store service is responsive. 2. Once the best replica is chosen, if the Hub Transport server cannot contact the destination Hub Transport Server (inter-AD site), then it will queue on a Hub Transport server. There are a few things with this scenario: a. For one, each AD site that has a Mailbox server, should consider having multiple Hub Transport servers so that the Hub Transport server does not become a single point of failure. b. In the scenario, where all Hub Transport servers are unresponsive (say because of network outage), the source Hub Transport server will not be able to establish a direct connection with the destination Hub Transport servers. So in this case, we will perform a back-off and get it as close as possible to the destination AD site (re-route, i.e. choosing another least cost route, would only occur after a site topology change). Conclusion We hope the above information helps explain how the routing process works for mail delivery to mailenabled public folder replicas