62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV

BMC.com > All Places > Products > Remedy AR System > Documents

SOAP Web Services Best Practices

Version 23

Created by Sylvain YVON on 24-Oct-2011 01:49. Last modified by Sylvain YVON on 25-Mar-2015 02:40.

<< Other Best Practices

<< AR System APIs and Integration Interfaces Overview

Context and Issue Statement

BMC provides a synthetic and straight-forward description of web services and their implementation in AR System. See the Integration
documentation for your version (7.6.04 or 8.1 ) for :
A short introduction to web services
A description of the web service architecture in AR System
You can also see

this document for a quick dive-in (the document is old but still valid)

Consuming (from a filter set fields action)

Publishing (from Mid-Tier)
Registering (UDDI)

The Pulse: AR server, the SOAP client (really) by Justin Bakker is a great introduction to SOAP web services in the AR System context.

This document adds some practical information gathered from experience and discussions on these boards.

Performance concerns
ITSM Web Services
Designing robust implementations
Complex web services
Web Services and SSL
Debugging tools and techniques
Most common error messages
Known limitations and issues

Consuming vs Publishing

In the following paragraphs, we consider that we are on the AR System side of an interface.
- Consuming means reaching for an external web service from AR System via the Web Service ARF Plugin, in order to push data to or get data from
another application.
- Publishing means exposing methods via the Mid-Tier for external applications to push data to or get data from AR System.

Performance concerns

When allowing 3rd Party consumption of your published Web Service, the Mid Tier is used as the interface platform. As the Mid Tier serves User
access also, dependant on the perceived load by the Web Service, additional dedicated Mid Tiers should be considered for Web Service
integrations. With the standard scalability of AR System, this is a cost effective way to build a robust integration platform.

When consuming a 3rd Party Web Service, the AR System Web Service plugin is utilized. Dependant on the consumption load, additional
performance tuning may be required on the AR side.

ITSM Web Services [publishing]

Using BMC ITSM OOB Web Services : How to test the ITSM Incident Interface based Web Services. There is also some example .NET (VB) code
showing how to Create, Query and Modify using the Incident based Web Services. ( Carl Wilson)

Designing robust web service interfaces

If you are designing a simple query interface to allow your partner to search for items or get some details for them, then you should only need to
publish a web service with a getEntry and a getEntryList methods directly on your form, provided you keep the "performance concerns" in mind.

Now if you are going to design a full AR System WS interface, with bidirectional objects updates, there are a few questions you need to answer and
points of attention :
- consider that the partner application might be unpredictably unavailable, or your request could be rejected for an unanticipated reason : you
should handle the errors without blocking your users
- consider that your application might be unpredictably unavailable (MT or AR System down) : if your partner sends his requests in bulk and
not necessarily in the right order, you should be able to handle them without loosing consistency
- consider that the partner application or yours can have scheduled downtimes : an admin should be able to deactivate the interface
temporarily, and resume it without loosing data

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV

For all these reasons, keeping all transactions in a staging form is strongly recommended.
The diagram below shows a sample design for a bidirectional interface for incidents.

Remember that if the partner web service is unavailable, you will receive an error from the web services plugin. If you consume the WS "on modify"
without handling the errors on the HPD:Help Desk form, the user simply won't be able to save his changes.
You have two options here :
- call the WS synchronously when the staging form entry is created/modified but catch exceptions on your staging form with an error handler
- use an escalation to call the WS asynchronously after an entry in the staging form has been flagged; handle exceptions too.

If you can bear with a few minutes of delay, the second option is the way to go. If you choose the first option the WS call will be done quickly most
of the time, but if the partner is malfunctioning and causes a timeout, your users will be blocked until the timeout is reached.

A staging form comes with a few perks too :

- you keep track of every transactions and their status, see the error numbers and labels
- you can add a mechanism to "replay" an incoming or outgoing transaction after an error
- you can make statistics about your interface
- you can build a user friendly console for the administrators, empowering them with the options listed above

Complex Web Services

A lot of discussions around web services in AR System community are about consuming or publishing complex web services. We call web services
"complex" when they include more than one occurence of an element. For example, you may want to send or receive informations on an update of
an incident or change along with its worklog entries and its related CI.

To either build a complex web service or to consume one, you will need to define the following elements :
The main form on which the web service will be based (ex: HPD:Help Desk)
One or more child forms that contain the multiple elements (ex: HPD:Worklog)
A distinguishing key (DK) : the unique key (for this transaction) that will differentiate the children records (ex: Worklog's ID)
A foreign key (FK) : the child form field that will contain the parent form's Primary Key (ex: Incident's ID)

[publishing] a complex web service

Let's say you have 2 forms :

- P, the parent form (ex: HPD:Help Desk)
- C, the child form (ex: HPD:Worklog)

You want to build a "create" operation.

For P, the Primary Key is the Request ID by default. This primary key is obviously unique.
In C, you need two keys because of the sequence of which AR System creates the records. First, it needs a key to link the records of C to the
records of P. This is the foreign key. Given that you can have multiple records in C for a record in P, it is not unique. When AR System create
the records, it pushes P's primary key into the field you defined as foreign key. You must not map this field in your input mapping.
The second key will identify, uniquely in the context of this request, the record of C. This is the distinguishing Key. This field you have to set
with the input mapping. So this value doesn't have to be unique table-wide, but request-wide.
Here is an example with values. You want to push a request with 1 parent record and 2 child records.

01.
02.
03.
04.
05.
06.
07.

<P>
<...>
<C>
<id>C1</id>
<...>
</C>
<C>

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV
08.
09.
10.
11.

<id>C2</id>
<...>
</C>
</P>

When you send that request to your WS, here's what you get :
- record in P reserves a request ID, for example "1"
- records in C are created
- AR System sets the FK field to "1"
- You mapped the "id" field from the input request to your DK field, so the first one gets C1 and the second gets C2.
You can rerun this multiple times. The only thing that has to be unique in your tables is your request ID, which AR System is taking care of.

Web Services and SSL

This part is focused on the configuration of AR System and its arplugin. The commands for handling certificates and truststores depend on you
server's OS and Java version. This doc by Karl Miller provides a lot of details about how to deal with certificates with Remedy 8.
Please post your question in the AR System community forum if you have one on the subject.

[publishing] a web service with server-side certificate

Server-side certificate, to make people access your web service via https instead of http, can be configured on the web server (Apache, Apache
Tomcat if stand-alone, IIS, Web Logic, ...) or on the reverse proxy gateway.
Either way, this is not an AR System concern.

[consuming] a web service via https

To encrypt the http stream and authenticate the server hosting the WS, if it's private key has not been compromised, you can use https
instead of http. For this, the server needs a valid certificate that has been approved by an authority that is trusted by your server.

Java comes with many world wide trusted authorities, but big IT departments often use their own authority for internal certificates:
it's cheaper, if you have many certificates to handle
it's more secure (search for DigiNotar being compromised for example)
Most of the time, servers are already configured to accept the internal authority's certificates.

To make the plugin validate the server's certificate, you need :

a truststore (if the authority is not already trusted)
to add the authority's certificate to your trustore, or simply the WS server's certificate if it's self-signed
to configure arplugin to use the truststore
edit the armonitor.conf file
locate the line for java arpluginsvr**.jar
add the following parameters to the command line
Djavax.net.ssl.trustStore=<pathtoyourtruststore>\
Djavax.net.ssl.trustStorePassword=<password>\

[consuming] a web service using a client-side certificate

After it has authenticated itself, the server that publishes the web service can require that your server authenticates itself. If your private key
has not been compromised, it pretty much guarantees that no rogue server will access the web service by impersonating your server.
This is a good idea if anonymous login has been configured, for example.

To make the plugin show a client certificate, you need :

if you need a signed certificate :
a .csr file ("Certificate Signing Request") to send to the certification authority
the .cer file the authority will return to you
else :
the .cer of your self-signed certificate
a PKCS keystore
to store your certificate in the keystore
to configure arplugin to use the keystore and to send the client certificate when required
edit the armonitor.conf file
locate the line for java arpluginsvr**.jar
add the following parameters to the command line
Djavax.net.ssl.keyStore=<pathtoyourkeystore>\
Djavax.net.ssl.keyStorePassword=<password>\
Djavax.net.ssl.keyStoreType=pkcs12\

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV

Debugging tools and techniques

Usually, when you develop applications and interfaces based on AR System, your best debugging friend is the server-side logging functionality
(escalations, filters, SQL, arplugin, API). For web services it will help, but sometimes it won't help enough.
Use these log files for debugging:
MT/WebService to see inbound requests to a published AR System WS
Java logging tool like log4j to see outbound requests from AR System Java Plugin to an external WS (See
consuming Web Service? )

How to log soap messages when

AR System/filters
It might be enough for the occasional debugging. For the conception and development of an interface, you will need a SOAP development tool to
forge your sample requests, do some unit testing for both inbound and outbound SOAP requests.
There are a few toolsout there, so take some time to check them out :

SOAP UI , which comes in a free version with all we need for most situations:
Creates sample requests from WSDL file ([publishing] and [consuming])
Creates unit tests ([publishing])
Creates mock web services from a WSDL file ([consuming])

Most common error messages

1. In DevStudio, while loading a WSDL ([consuming])
When an error occurs while loading a WSDL within a "Set Fields" action in a filter, the most obvious symptom is that nothing happens. Clicking
the "reload" button leaves the "ports" and "operations" menu desperatly empty. Remember that only your client is used at this stage, not your AR
System server. The first thing to do is to make sure the "Messages" window of DevStudio is open. The messages displayed there are often
obnoxious, but you need to bite the bullet and take some time to read the details.

PKIX path building failed

ERROR:URInotfoundhttps://mydummyserver:1337/?wsdl
AdditionalDetails:sun.security.validator.ValidatorException:PKIXpathbuildingfailed:
sun.security.provider.certpath.SunCertPathBuilderException:unabletofindvalidcertification
pathtorequestedtarget

You are trying to consume a web service hosted on a server that requires https. The certificate provided by the server is not considered as
trustworthy. What you need is to add the public key of the host of the web service to a local truststore. If the publisher didn't provide it, you
can get it from your browser. On Firefox, click the lock icon next to the URL, then "more informations", "show certificate", "details", "export".
Save it into a crt file. Now you can add it to a local truststore using the java "keytool ", and change your devstudio.ini file to add :
Djavax.net.ssl.trustStore=<pathtoyourtruststore>
Djavax.net.ssl.trustStorePassword=<password>

Restart DevStudio, and if the settings are correct, you should not see this error.
An error occurred trying to resolve schema referenced at [...]
The WSDL references a schema (xsd) that cannot be located/accessed. Make sure the URL is correct, and that your workstation can resolve it.
You can test this with a simple browser. If you can, check if your browser is accessing it via a proxy server. If it does, make sure your DevStudio
is also configured to use this proxy (see below for the "proxySet" option).
If your workstation cannot access to file by any mean, you have to download the file or request it from the publisher. Save the wsdl and the
xsd files in a directory and edit the wsdl file. Replace the "http:// ..." adress with "yourfilename.xsd", save and load this wsdl.

Unable to locate or load the specified file {0} [...]

This one is the most puzzling error you can get. It seems like quite generic and can be seen for a number of reasons. But the most common
cause for it is that the WSDL is not understandable by DevStudio in its current state. See "known limitations and issues" below.

It can also be raised when the WSDL has some references to URL that cannot be resolved, either because they are just dummy URL (like
"http://tempuri.org/ ...") or because your workstation itself cannot resolve them (not on the same network, no access to Internet, or access
through a proxy server). To solve this, there are two solutions :
modify your hosts file to map a dummy IP address with the FQDN (Fully Qualified Domain Name), ie: 127.0.0.1 tempuri.org
if you have access to Internet via a proxy server, modify your devstudio.ini to let it know :

01.
02.
03.

Dhttp.proxySet=true
Dhttp.proxyHost=proxy_address
Dhttp.proxyPort=8080

Known limitations and issues

[Consuming] Documented limitations regarding specific XML schema and WSDL structures render some W3C-compliant web services
impossible to consume by AR System plugin.

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV

The supported structures depend on your version of AR System. See your versions Integration document for more details, or here for
version 8.1 .
So how do you handle this? Well, get ready for some nasty XML tinkering. You are going to forge your own WSDL to make AR System
understand it.
Sometimes it will not be possible. E.g. replacing <s:any> if the sequence never has the same structure.
Sometimes it will be approximate. E.g. replacing an unbound limit to 10, assuming that you will never receive more than 10.
Sometimes you will brilliantly reproduce the original WSDL with a supported structure, after hours of pain. E.g. merging an external
unsupported xsd file into the wsdl.

Either way, you know you will have a lot of work if your interface nemesis changes. This is also a major risk you have to consider when
charging for the conception of a web service interface. Out of 6 web service-based interfaces I created, 4 required a locally edited WSDL.

[Consuming] the appropriate web service URL according to the current environment.

During your development process, chances are you're not consuming the production webservice of your interface mate. So how do you move
to Q&A or production or other environment?
Starting from versions 7.6.04, you can query the Registry to overwrite the WS location at run time, but you can't set the endpoint directly from
a field. See "Querying the registry " form more details.
One solution is to modify the wsdl locally, set an alias for the location tag and configure the hosts file on your server to target to correct
environment. Of course, it supposes that all the URL are the same except for the host name (same port, same path).

Frequent terms and abbreviation

FQDN : Fully Qualified Domain Name

SOAP : Simple Object Access Protocol
WS: Web Service

External links

Jose Huerta posted a good article here

45543Views

Categories: Best Practices, Developer Guides, Tips and Techniques, Utilities

Tags: api, ws, dev, web-service

Average User Rating

(11 ratings)

MOST LIKED

17 Comments
Login or Register to comment
Jason Miller 23-Apr-2013 19:08

Great work

Sylvain YVON. Thanks for updating this document!

Actions

Like (1)

Vinod Gaidhani 24-Apr-2013 05:58

Nice detailed explaination of Web Service Integration in Remedy. I would like to know if BMC has recommended any best practice for Error
Handling while using Web Services.
Actions

Like (3)
Sylvain YVON 25-Apr-2013 01:51 (in response to Vinod Gaidhani)

Hey guys,
Thanks for your feedbacks.
Vinod NameToUpdate, I don't know of any white paper on the subject. The doc even states "In the Error Handler panel, verify
that error handling is disabled".
Actions

Like (0)

Anupam Wagh 15-May-2013 05:21

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV
Thanks

Sylvain YVON for sharing such a great document.

Actions

Like (2)

Pankaj Salpekar 21-May-2013 07:52

Thanks

Sylvain YVON

Actions

Like (1)
Sylvain YVON 21-May-2013 09:44 (in response to Pankaj Salpekar)

Glad this helps, guys.

Feel free to post update requests here, or even update the document yourself if you feel like something is missing.
Actions

Like (3)

Andrs RAMIREZ 14-Nov-2013 09:48

Merci pour la doc et flicitations pour ton papier il est excellent !

Claireetpdagogique.

Actions

Like (1)

Dana Epp 30-Nov-2013 19:04

This article was pretty useful. It would be nice to see more detail on how to consume a web service using client certs though. When creating
a Set Fields web service filter action I don't know where you would define the cert fingerprint that you would be placing in the cert keystore
so you could load it on call to the external web service.

Is that something someone has documented somewhere?

Actions

Like (0)
Carl Wilson 01-Dec-2013 04:22 (in response to Dana Epp)

Hi Dana,
the certificates are all stored in a Keystore for the ARS Web Service Plugin (as the Plugin consumes the external/internal Web
Service). You then define the Plugin (armonitor) with additional parameters where the Keystore is located.
The document that is referenced ( Remedy 8 - Digital Certificates) explains in detail about the Certs and Keystores. Once the
Plugin has been setup to reference the Keystore, you place all certificates required in the Keystore and the system will
automatically pick these up and use them at runtime.
The Keytool and Keystore tools are used to import/export/generate certificates. Typically you will obtain the certificate from the
external/internal source of the Web Service you are consuming.

https://docs.bmc.com/docs/display/public/ars81/Configuring+to+consume+over+SSL+or+with+client+certificates

No additional configuration is required on a Filter or Filter Action.

Let us know if this does not cover your question (and if I completely missed the question) or you require additional information.

Cheers
Carl
Actions

Like (3)

Velan Ramakrishnan 04-Dec-2013 07:24

7KDQNV

Sylvain YVON IRUVKDULQJVXFKDJUHDWGRFXPHQW

Actions

Like (1)

Subhash Sonwane 09-May-2014 00:11

Thanks for sharing document.

Actions

Like (2)

Nancy Ramirez 16-Sep-2014 00:53

Great document Sylvain. Currently I'm working with Complex Webservices in the same way you've explained and it Works as expected for
CREATE operation. However, when using MODIFY operation, it doesn't update the entries in the child form, but seems all entries relate to
the parent form are deleted and it creates again only the information is sent during the MODIFY operation. Do you know how to handle
this?

Thank you
Actions

Like (0)
Sylvain YVON 16-Sep-2014 02:07 (in response to Nancy Ramirez)

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&





62$3:HE6HUYLFHV%HVW3UDFWLFHV_%0&&RPPXQLWLHV
Hi Nancy,
Sorry I never experienced this myself and can't test it right now. Perhaps you could ask this on
the doc with your findings ?

Remedy AR System, then update

Actions

Like (0)

Guillermo Rubio Blanco 16-Sep-2014 02:26

Thanks

Sylvain YVON

Actions

Like (1)

Vinayak Sadashiv Bedake 30-Mar-2015 00:39

What are the possibilities in which WSDL & End point URL's contain different protocols?

any hint?

I observed that they should have same protocol http-http|https-https?

if they are not same in such cases you should go for SSL related troubleshooting.

Regards,
Vinayak
Actions

Like (0)

Leonardo Jose Sebba 31-Mar-2015 13:14

I have problem. When I run the establishment , it's all perfect.

But when I need to perform a SET operation. l get an error , and I can not to identify what the problem.
Example:

Form A to Form B (Operation = SET) i get message : ARERR [ 8940 ] Error parsing XML definition.

If I use directly the SOAP tool, it usually changes the target form , but the call in the form origin through the filter does not .

someone could help me with this problem? or could tell me what could be happening ?? remembering that the operation runs in the same
server.

Edit: after many tests , I managed to perform SET operation . But I found that a type of type selection menu field with 19 options. does not
work.

My configuration in field : Element : true, MaxOccurs : 1, MinOccurs : 0, Nillable : true, Type : String, XMLType : False.

i'm tried other options without success. anyone can help me??
Actions

Like (0)
Sylvain YVON 01-Apr-2015 02:02 (in response to Leonardo Jose Sebba)

Hi Leonardo,
Selection lists are of "string" type by default. Try logging your filters and see what value is sent (string value, alias string, integer
value ?). You can then adapt the field type in the xml properties, or use a temporary field to store the value before it is sent.
If it still doesn't work, please post a question in the AR System community with your wsdl and your filter logs.
HTH
Sylvain
Actions

Find People
Support Login

Like (0)

Community Help
Worldwide

About BMC

BMC.com

Google+

18.4K

Copyright 2005-2016 BMC Software, Inc. Use of this site signifies your acceptance of BMC's Terms of Use, Privacy Policy and Cookie Notice.
BMC, BMC Software, the BMC logos, and other BMC marks are trademarks or registered trademarks of BMC Software, Inc. in the U.S. and/or certain
other countries.
Manage Cookies

KWWSVFRPPXQLWLHVEPFFRPGRFV'2&