Using Oracle ESB as a web service client to pull data periodically

In this article we will cover the detailed steps to create an ESB service that invokes a web service periodically and write the response to a file.  The web service in this case takes an input parameter and returns some response based on the input.  The client is expected to parse the response and take an element from the response and supply that as the input for the next call.

1.    Create a service group. Lets call it EsbWebServiceClient
2.    Create a xsd file to represent the configuration information that we will use to pass to the webservice.
3.    Create a xml file that complies with the schema and the initial parameters for invoking the web service. The ESB adaptor will update this file after each invocation.
4.    Create a file adaptor to read a file.  Don’t check the mark that says delete file after reading.  Specify the schema that you created in step 2 for the schema of the file. When you are done with step you should end up with boxes 1 & 2 shown in the diagram with the line connecting them.
5.    Create a ESB Service  for Soap service (Right click on design pane-> Create ESB Service->Soap Service)  On the pop up window, select the WSDL file representing the service.  For this demo, the web service is a simple service that takes a string as input and returns a string. This creates Box 3.with no lines connected.
6.    Now create a schema that represents the input elements to the webservice and the output of the webservice.  We will use this as the holding area to have both the request and response so we can write the output to a file and update the config file with the updated information.  In the example the schema is called serviceParameterAndResponse.
7.    Create a ESB routing Service and name it ConfigMapper.(Right click on design pane-> Create ESB Service->Routing Service)  On the pop up window, select the serviceParameterAndResponse schema for both request and response formats.. This creates Box 4.with no lines connected. This routing service provides the framework for us to merge the input parameters and the webservice output.
8.    Now create a file adaptor that can write an output file and name it WsResponseWriter. To keep it simple, specify the schema for the file as the same as the response from the webservice  This should create Box 5 with no lines connected.
9.    Now create a file adaptor that can write the config file and name that ConfigFileWriter.. Specify the schema and the location for the file as the same as the schema created in step 2. This should create Box 6 with no connector lines
10.    Double click box 2 and add a routing rule to invoke the method on the webservice.  This should create a reply line.  Now point the reply to the execute method on the routing service created on step 7.  This should create lines 8 & 9 on the esb diagram..
11.    Double click box 4 and add two routing rules.  One pointing to WsResponseWriter and the second one pointing to ConfigFileWriter.  This should create the lines 10 & 11 in the diagram.
12.    Now the diagram is almost complete.  You will see the mappings are still gray. We will add the mappings which will make them yellow.
13.    Double click on the top "X" and create a new mapping.  When the designed is displayed, drag a line from the config parameter to the input parameter of the web service.
14.    Double click of the bottom "X" of box 2.  Create a new mapping.  You will notice there is an option to "include the request in reply play load".  Make sure you select this option.  When the designer panel is displayed you should see a new node called “ESBREQUEST” on the left hand side.  To learn more about the $ESBREQUEST parameter refer to this article (http://www.soastation.org/2007/05/esbrequest-in-oracle-esb-routing.html)
15.    Now select the source view of the xslt and add a function to parse the required fields from the request to map to the output. In the example below, I have mapped the string after the pipe symbol to be stored as the parameter for the next call. And for the response, I have concatenated the input and the output to be written out to the file.  Please note that you will not be able to drat lines to or from the $ESBREQUEST variable on the designer view.

<xsl:param name="ESBREQUEST"/>
 <xsl:template match="/">
   <out1:serviceParameterAndResponse>
     <out1:param1>
       <xsl:value-of select='substring-after(/tns:getElapsedTimeSinceResponse/tns:return,"|")'/>
     </out1:param1>
     <out1:response>
       <xsl:value-of select='concat(substring-after(/tns:getElapsedTimeSinceResponse/tns:return,"|"),substring($ESBREQUEST/tns:lastTime,1.0,2.0))'/>
     </out1:response>
   </out1:serviceParameterAndResponse>
 </xsl:template>

 16.    Now double click on the "X" on box 4 and create new mappings.   These mappings should be straightforward as all the manipulation is done on the previous step.   Just draw a straight line from the corresponding field on the left to right.  Repeat this for both the ‘X” on the box.
17.    Now your diagram should look similar to the one shown above.  You can register the service to your ESB and test it.

If you like see a copy of the project that I used to create the demo, feel free to drop me an email.  My email address is jmeslie at gmail dot com.

Debugging Webservice clients - Log HTTP transport content

Many times it is useful to see the actual content that is send and received through the wire in order to troubleshoot webservice clients. Unfortunately, there does not seem to be a standard way to enable logging across all the webservice clients. Depending on the client you use for accessing the webservice, the debugging method varies. Here are a few tips that will help.

If you are using Axis client to access a webservice and using log4j for logging, you can enable printing of the http request and response contents by setting the "httpclient.wire.content" property to "debug". For example if using the log4j.properties file set the following property:

log4j.category.httpclient.wire.content=debug,stdout

If the webservice is connecting to a secure url using https protocol, setting the java system property "javax.net.debug" to "plaintext" will provide a hex dump of the content send and received.

If you are using the web service proxy provided by oc4j within jdeveloper or the oc4j container, you can enable the java logging property "oracle.webservices.client" in logging.properties file to FINE

oracle.webservices.client.level=FINE

Setting this option will allow a standalone client to log the request and response content to the log file specified by the logging.properties file.

If the client is running within the oc4j container, you can enable the oracle diagnostics logging for "oracle.webservices.client". This can be done by navigating to the <oc4j container> -> Administration --> Logger Configuration page and setting the log level as "FINE" for "oracle.webservices.client" property.
You can view the request and response contents in the log.xml file.

Asynchronous Web Service Calls over HTTP using Oracle ESB and JMS

How to create an asynchronous webservice that supports failover using Oracle ESB?

This article describes a general design for invoking web services in an asynchronous fashion even if the web service provider only supports synchronous invocation. I have used JDeveloper to create this service however the same principles can be used for any development tool.

Use case:

Generally there are 3 different scenarios of using a web services.

1. Request/Response Real Time: .Here the response from the web service is critical for the application to proceed to the next step. If the web service is not available, user needs to know right away. Example: An application depends on a web service to display a list of accounts they can manage. In this case invoking the web service in an asynchronous fashion is not going to help much. A simple synchronous web service call will do the work without adding any complexity.
2. Request only – (delivery critical): Here, the client application needs to send a message to a web service, but it is not expecting any response from the service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation.
3. Request/Response time (not critical, delivery critical): Here, the client application needs to send a message and receive response from the web service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation. The client is not waiting for the response from the service. Instead, the client has provided a callback mechanism to receive the response.

The design described here suites for the third scenario and the second scenario. For the second scenario, only part of the design is needed.

The approach:

The idea here is to use JMS queues to receive the request and response messages to and from the web service. Clients will access the web service through a JMS-WebService adaptor. JMS listeners will invoke the web service and the callback service when messages are received in the request or response queues.

The following diagram describes the approach.

The web service is created and deployed in a remote server. In many cases this might be an external web service provided by a third party hosted outside your firewall. The client provides its callback mechanism as a web service and it is deployed locally. JMS queues are created in the local server for receiving the request and response messages. A JMS adaptor is created that exposes the request queue as a web service. Using JMS adaptor provides the following benefits:

• The client can communicate with the service using SOAP/HTTP(S) which is well supported by most tools than the SOAP/JMS.
• It hides the dirty details of the JMS details from the web service client developer.
• The web service client developer is using the same protocol to access the service if he has to call the service directly. (except for the return value)

A JMS listener is created to monitor the request queue and invoke the actual web service using a SOAP adaptor. The response from the web service will be written to the response queue. The listener and the adaptor can be configured to retry the service invocation if the service is down.

Another JMS listener is created to monitor the response queue and invoke the callback web service using a SOAP adaptor. The listener and the adaptor can be configured to retry the service invocation if the callback service is down when the actual web service responded.

Benefits:

This approach allows you to develop an asynchronous callback mechanism to your web service, even if the original service did not provide one. Most web services that are request/response provides only synchronous calling mechanism only.

It provides a way to decouple the dependency on the external web service without adding any custom client code. These days most application server vendors provide JMS and SOAP adaptor widgets that let you create these without having to write them yourself.

If you are using a web service that does not return and response back (one way request only), this approach can still work just be not creating the response queue and the related components (step 5 and beyond in the above diagram).

Limitation:

This approach assumes that you have enterprise grade tools such as JMS/SOAP Adaptors in your production environment. If such tools are not available, you may have to create these adaptors yourself. If you are developing them yourself, the time spent creating these generic services might be higher than adding failover logic to the application itself.

Creating the service

The instructions assume you are using Oracle application server 10.1.3.x and jDeveloper 10. The exact steps will be different for other application server and IDEs. Please note that these instructions assume that you are familiar with the above tools and does not provide step by step details of each widget.

1. Create and deploy the ‘actual WebService. If the service is provided to you by a third party you can skip this step.
2. Create and deploy the web service response receiver web service to your local server. If you are creating only a one way web service call you can skip this step.
3. Create JMS queues for request and response queues. This can be done through the Enterprise Manager application by navigating to à Administration à Enterprise Messaging Service à Create/delete/edit JMS destinations. Since we are going through this exercise to create a failover mechanism, I would suggest create these queues with File/Database persistence.
4. Create separate JMS Connection factories for each of these queues so we don’t get in to any resource contentions.
5. Create a new jDeveloper ESB project
6. Copy the WSDL for step 1 and 2 in to the project so we can reuse the schema elements.
7. Create JMS adaptor for writing to the request queue (produce message). Check the box that says “Invocable from external service”.

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type and specify a meaningful name (similar to the original service’s method name) for the operation. The WSDL exposed by this adaptor will be later used by the web service client. Select the request queue and the appropriate connection factory. Specify a large time out so if the external service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL.

8. Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the actual web service.
9. Create JMS adaptor for reading from the request queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the request queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL (schema and queue names same as previous step). JDeveloper should have created the adaptor service icon and the routing service.
10. Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 8. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
11. Create JMS adaptor for writing to the response queue (produce message).

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type. Select the response queue and the appropriate connection factory. Specify a large time out so if the callback service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL (or the output message from the external web service).

12. Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the callback web service.
13. Create JMS adaptor for reading from the response queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the response queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL. JDeveloper should have created the adaptor service icon and the routing service.
14. Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 12. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
15. Edit the routing service created in step 10, by double clicking and expand the routing rule for the consume message operation. Wire the response message from the service to point to the produce message operation of the JMS adaptor for the response queue. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous. The following diagram represents the ESB diagram for a project with these settings.
16. 
17. Deploy/Register the ESB project to the OC4J container.
18. Create a web service proxy for the WSDL exposed in step 7. Invoke the service using the generated proxy. If everything went well you should notice the response came back through your callback service.