Message Queuing with BizTalk 2006

R2:
MSMQ, IBM WebSphere MQ
and Ordered Delivery
Thomas Abraham
Randall McClure
Enterprise Consultant
BizTalk Consultant
TS: BizTalk 2004 & 2006 TS: BizTalk 2006
blogs.digineer.com/blogs/tabrah
am

blogs.digineer.com/blogs/randal
lm
1

Agenda

Why Message Queuing?

Using MSMQ
Using WebSphere MQ
A Solution for Ordered Delivery
based on MSMQ

Why Message Queuing?

Guaranteed and/or ordered delivery
Cross-platform communication
Transactions
Asynchronous data transmission
(a.k.a. fire and forget)
Buffering/batching
Loose coupling
Load balancing

Queuing Considerations
Dead letter queue management
Poison message management
Consistency of message format per
queue
Latency
Licensing and support costs

Microsoft Message
Queuing
Included with almost every OS
edition
No extra licensing fees
Supports Network Load Balancing
and Cluster Services
Can be bridged to WebSphere MQ
with Host Integration Server 2006
Publish queue locations in Active
Directory and use Windows
authentication

MSMQ Considerations
Limited management tools
4 MB message size limit
Shipping V4.0 with Windows Vista
and Windows Server 2008
Transactional remote receive
Subqueues
Poison message handling

Demo

DEMO
Using MSMQ with the
BTS 2006 R2 WCF
Adapter
7

WebSphere MQ
Formerly MQSeries
Available for dozens of platforms
Highly configurable, widely adopted
Includes extensible management
tools hosted in Eclipse (Java-based)
Two BizTalk adapters:

Server-Based Adapter (w/ BizTalk)

Client-Based (w/ Adapters for Host
Systems)
8

Server-Based Adapter

Client-Based Adapter

10

WebSphere MQ Security
Interacts with Active Directory for
authorization of queue access rights
Domain policy may disallow nondomain users from reading domain
account group memberships
Path of least resistance and hassle:
always set up a domain account for
the MQ service
11

Installing MQAgent
Requires COM+ and DTC running on
WebSphere MQ for Windows server
Enable Network DTC and Network
COM+
Allow In- & Outbound for Transaction
Mgr
Enable Mutual Authentication
Enable XA Transactions
Enable DCOM with Packet Privacy
12

Demo

DEMO
Using WebSphere MQ for
Windows 6.0
13

Ordered Delivery Solution

with MSMQ Queuing

14

Agenda
Patterns, Reloaded
Business Problem
Initial Solution
Implementation Details
Demo

Better Solution
More Details
Demo

Take-aways

15

Patterns,
Reloaded
Uniform Sequential Convoy
Multiple identical messages, over time
First receive initializes a correlation set
Second receive is in a Listen, inside a Loop,
using the same logical receive port and
operation
Later messages that match and follow the
correlation set are received in the loop

Ordered Delivery = uniform seq. convoy

++

Enforce order in orchestration receive shape

and physical ports (limited types, 2006 only)
16

Uniform Sequential
Convoy
First and
followers in
same
receive
shape

Activate,
init
correlation
set

Process
first
message
Listen
shape with
Receive
and
process
branch,
and Delay
Following
branch
correlation
set

Loop until
Delay path
fires before
receiving
following
message

17

Ordered Delivery
Ordered
Delivery
= true
Ordered
delivery on
physical
ports as
appropriate

18

Business
Problem
Consume business events
Filter and transform messages
Deliver to three unique consumers, one of
which affects peoples lives adversely if not

In-order!
Low volume 300/day
6am-6pm operation window

19

Business
Problem

20

Initial Solution
Generate WSE 2.0 web service from
acknowledgement orchestration
Ordered delivery orchestration pattern
Called orchestration chain to preserve order
Web service facades and data access layer to call
SQL Server stored procedures
Oracle adapter to call stored procedures

21

Initial Solution current

mncis

22

Initial Solution MNCIS

XML
<CaseNotification schemaVersion="3:1" >
<NotificationAdminHeader
caseNumberKey="62J007553366"
caseKey="1568853">
<CourtFileNumber>
<ID>62-J0-07-553366-madeup</ID>
</CourtFileNumber>
</NotificationAdminHeader>
<HearingTrialSettingNotification>(0:n)
<\CaseNotification>

23

Initial Solution current

RCCC

24

Initial Solution

DEMO

25

Oops!
What if
Need to one or all consumer applications?
A message is suspended, breaking ordered
delivery?
Find a bug that requires messages to be
replayed, in-order?

Not quite there

Had incoming message file wire-tap, but not in
order, and no means to replay them
Tedious manual process to re-order and replay

26

Better Solution
Solution
Might have implemented a resequencer pattern
at
http://www.microsoft.com/downloads/details.asp
x?FamilyID=02c5fd53-fee9-44fc-a7805d1d34ee8754&DisplayLang=en
Decided to insert MSMQ queues in main path and
in each consumer path
Decided to have the web service write directly to
the main path queue, to have replay capability
separately from BizTalk
Decided to decouple the web service from
BizTalk
27

Queuing Solution planned

mncis

28

Queuing Solution planned

RCCC

For replay

29

Queuing
Solution

DEMO

30

Take-aways
Ordered delivery easy to implement, tougher to
support
MSMQ very helpful for isolation, replay and fault
tolerance
Watch out for subscription loops
Third-party components help but they may not be
robust enough

31

Take-aways
Third-party tools are very helpful
Consider fault tolerance and replay EARLY
File SEND (but not receive) can do ordered delivery;
use %datetime% in filename

32

Resources
DebugView
http://www.microsoft.com/technet/sysinternals/ut
ilities/debugview.mspx

Wfetch (in IIS 6.0 resource kit)

http://www.microsoft.com/downloads/details.asp
x?familyid=B134A806-D50E-4664-8348DA5C17129210&displaylang=en

ContextAdder Jon Flanders

http://www.masteringbiztalk.com/blogs/jon/Com
mentView,guid,25768f43-c0b6-4f9d-bb7e636d52dcd7eb.aspx#commentstart
Modified here by 12/15
http://blogs.digineer.com/blogs/randallm/default.
aspx

33

Resources
Cogin QueueExplorer
http://www.cogin.com/

CreateMSMQQueues.Tools (by 12/15)

http://blogs.digineer.com/blogs/randallm/default.
aspx

SSO tools by BizTalk Samples, Scott

Colestock and others

34

Message Queuing with BizTalk 2006

R2:
MSMQ, IBM WebSphere MQ
and Ordered Delivery

Thank you for attending!

Please visit us at www.digineer.com
35