Asynchronous Web Service Calls over HTTP using Oracle ESB and JMS

How to create an asynchronous webservice that supports failover using Oracle ESB?

This article describes a general design for invoking web services in an asynchronous fashion even if the web service provider only supports synchronous invocation. I have used JDeveloper to create this service however the same principles can be used for any development tool.

Use case:

Generally there are 3 different scenarios of using a web services.

1. Request/Response Real Time: .Here the response from the web service is critical for the application to proceed to the next step. If the web service is not available, user needs to know right away. Example: An application depends on a web service to display a list of accounts they can manage. In this case invoking the web service in an asynchronous fashion is not going to help much. A simple synchronous web service call will do the work without adding any complexity.
2. Request only – (delivery critical): Here, the client application needs to send a message to a web service, but it is not expecting any response from the service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation.
3. Request/Response time (not critical, delivery critical): Here, the client application needs to send a message and receive response from the web service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation. The client is not waiting for the response from the service. Instead, the client has provided a callback mechanism to receive the response.

The design described here suites for the third scenario and the second scenario. For the second scenario, only part of the design is needed.

The approach:

The idea here is to use JMS queues to receive the request and response messages to and from the web service. Clients will access the web service through a JMS-WebService adaptor. JMS listeners will invoke the web service and the callback service when messages are received in the request or response queues.

The following diagram describes the approach.

The web service is created and deployed in a remote server. In many cases this might be an external web service provided by a third party hosted outside your firewall. The client provides its callback mechanism as a web service and it is deployed locally. JMS queues are created in the local server for receiving the request and response messages. A JMS adaptor is created that exposes the request queue as a web service. Using JMS adaptor provides the following benefits:

• The client can communicate with the service using SOAP/HTTP(S) which is well supported by most tools than the SOAP/JMS.
• It hides the dirty details of the JMS details from the web service client developer.
• The web service client developer is using the same protocol to access the service if he has to call the service directly. (except for the return value)

A JMS listener is created to monitor the request queue and invoke the actual web service using a SOAP adaptor. The response from the web service will be written to the response queue. The listener and the adaptor can be configured to retry the service invocation if the service is down.

Another JMS listener is created to monitor the response queue and invoke the callback web service using a SOAP adaptor. The listener and the adaptor can be configured to retry the service invocation if the callback service is down when the actual web service responded.

Benefits:

This approach allows you to develop an asynchronous callback mechanism to your web service, even if the original service did not provide one. Most web services that are request/response provides only synchronous calling mechanism only.

It provides a way to decouple the dependency on the external web service without adding any custom client code. These days most application server vendors provide JMS and SOAP adaptor widgets that let you create these without having to write them yourself.

If you are using a web service that does not return and response back (one way request only), this approach can still work just be not creating the response queue and the related components (step 5 and beyond in the above diagram).

Limitation:

This approach assumes that you have enterprise grade tools such as JMS/SOAP Adaptors in your production environment. If such tools are not available, you may have to create these adaptors yourself. If you are developing them yourself, the time spent creating these generic services might be higher than adding failover logic to the application itself.

Creating the service

The instructions assume you are using Oracle application server 10.1.3.x and jDeveloper 10. The exact steps will be different for other application server and IDEs. Please note that these instructions assume that you are familiar with the above tools and does not provide step by step details of each widget.

1. Create and deploy the ‘actual WebService. If the service is provided to you by a third party you can skip this step.
2. Create and deploy the web service response receiver web service to your local server. If you are creating only a one way web service call you can skip this step.
3. Create JMS queues for request and response queues. This can be done through the Enterprise Manager application by navigating to à Administration à Enterprise Messaging Service à Create/delete/edit JMS destinations. Since we are going through this exercise to create a failover mechanism, I would suggest create these queues with File/Database persistence.
4. Create separate JMS Connection factories for each of these queues so we don’t get in to any resource contentions.
5. Create a new jDeveloper ESB project
6. Copy the WSDL for step 1 and 2 in to the project so we can reuse the schema elements.
7. Create JMS adaptor for writing to the request queue (produce message). Check the box that says “Invocable from external service”.

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type and specify a meaningful name (similar to the original service’s method name) for the operation. The WSDL exposed by this adaptor will be later used by the web service client. Select the request queue and the appropriate connection factory. Specify a large time out so if the external service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL.

8. Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the actual web service.
9. Create JMS adaptor for reading from the request queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the request queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL (schema and queue names same as previous step). JDeveloper should have created the adaptor service icon and the routing service.
10. Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 8. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
11. Create JMS adaptor for writing to the response queue (produce message).

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type. Select the response queue and the appropriate connection factory. Specify a large time out so if the callback service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL (or the output message from the external web service).

12. Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the callback web service.
13. Create JMS adaptor for reading from the response queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the response queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL. JDeveloper should have created the adaptor service icon and the routing service.
14. Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 12. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
15. Edit the routing service created in step 10, by double clicking and expand the routing rule for the consume message operation. Wire the response message from the service to point to the produce message operation of the JMS adaptor for the response queue. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous. The following diagram represents the ESB diagram for a project with these settings.
16. 
17. Deploy/Register the ESB project to the OC4J container.
18. Create a web service proxy for the WSDL exposed in step 7. Invoke the service using the generated proxy. If everything went well you should notice the response came back through your callback service.