Oracle SOA Adapter Tuning Properties

DB Adapter

1. Distributed Polling - While creating DB adapter to poll the records from DB, select Distributed Polling under polling configuration. This property will  automatically uses the syntax SELECT FOR UPDATE SKIP LOCKED that means the same row cannot be processed multiple times.                                                                                                                           
2. Singleton Process:  In case of distributed polling (Cluster having multiple Servers) , all the managed servers try to poll and process the records from the DB.  This will have a performance issue.  The JCA Binding Component supports active fail over of inbound Adapter Services. To enable this fail over feature for a given inbound adapter endpoint, we must add the singleton JCA service binding property in the composite.xml within the <binding.jca> element and set it to a value of true. 

            <binding.jca config="SchedulerPollDBAdapter_db.jca">

            <property name="singleton">true</property>

            </binding.jca>

      The above property will make  one node of the cluster will poll for the record. If the active node goes down then other  node of the cluster will become active node and start  polling for the record.

Above Singleton property can be used for all types of adapters. 

File/FTP Adapter

Use following connection factory for polling the files in distributed configuration

<connection-factory location="eis/HAFileAdapter" adapterRef=""/>

Configure a database table as a coordinator by modifying Oracle File and FTP Adapter deployment descriptor for the connection-instance corresponding to  eis/HAFileAdapter from the Administration Console:

1. controlDir - Set this parameter to the directory structure where you want the control files to be stored. Set it to a shared location if multiple Oracle WebLogic Server instances are running in a cluster.
2. inboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. The pre-created schemas are located under MW_HOME/ASreleaseSOA/rcu/integration/soainfra/sql/adapter/createschema_adapter_oracle.sql. To create the schemas elsewhere, use this script. Set the inboundDataSource property accordingly if you choose a different schema.
3. outboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. See inboundDataSource for schema location.
4. outboundLockTypeForWrite - Set this parameter to oracle if you are using Oracle database. By default Oracle File and FTP Adapters use an in-memory mutex to lock outbound write operations. Choose one of the following values for synchronizing write operations:
5. memory - Oracle File and FTP Adapters use an in-memory mutex to synchronize access to the file system.

• oracle - The adapter uses Oracle database sequence.
• db - The adapter uses a pre-created database table (FILEADAPTER_MUTEX) as the locking mechanism. Use this option only if you are using a schema other than the Oracle database schema
• user-defined - The adapter uses a user-defined mutex. In order to configure the user-defined mutex, implement the mutex interface:  oracle.tip.adapter.file.Mutex and then configure a new binding-property with the name oracle.tip.adapter.file.mutex and value as the fully qualified class name for the mutex for the outbound reference.

Read Payload from AQ Table

Use the following Query to get the USER_DATA

SELECT  USER_DATA.TEXT_LOB
FROM  QUEUE_TABLE ;

SELECT USER_DATA.TEXT_VC
FROM QUEUE_TABLE;

Weblogic JMS Message Bridge

Weblogic JMS bridge can be used to consume and forward the message from remote queues.

In this example I will provide the details  to create a message bridge and foreign servers to connect with TIBCO queue and transfer the messages to internal AQ.

1. Create a User to connect with  TIBCO application. Use security realm to create the user and password.

2. Create a JMS module - tibco_jmsmodule

3.  Create a Subdeployment

Create a Foreign Server to Connect with TIBCO Application

4. Create a Foreign JMS server - tibco_foreignserver  

• Create a file store. 
• Update ContextFactory as com.tibco.tibjms.naming.TibjmsInitialContextFactory
• Update URL as :  tibjmsnaming://hostname:port
• Provide user name and password
• Update the following properties  

                    java.naming.factory.url.pkgs:com.tibco.tibjms.naming,
                    com.tibco.tibjms.naming.security_protocol:ssl
                    com.tibco.tibjms.naming.ssl_enable_verify_host:false,
                    com.tibco.tibjms.naming.ssl_enable_verify_hostname:false

•  Target the Foreign server to appropriate cluster
• Create a Connection Factory  - Provide Connection Factory Name
• Provide local Connection Factory JNDI Name   - Tibco_Local_CF
• Provide the remove Connection Factory JNDI Name  - Tibco_Remote_CF
• Create Queue  - Provide Queue Name
• Provide local Queue JNDI Name  - Tibco_Local_Queue
• Provide Remove Queue JNDI Name - Tibco_Remote_Queue

5. Follow step 4 to connect with Destination

Create an Configure Bridge:

1. Create Source Destination

• Create a Destination and provide valid Name
• Select 
• Provide  TIBCO User Name and Password
• Provide Connection Property Name  - Tibco_Local_CF
• Provide Queue Name   - Tibco_Local_Queue

2. Create Target Destination 

• Create a Destination and provide valid name
• Provide Connection Property Name  - Target_Local_CF
• Provide Queue Name   - Target_Local_Queue

3. Create a Bridge

• Provide a Valid Name
• Select Source Destination
• Select Target Destination
• Target it to Valid Cluster (Same as your Foreign Server Destination)

Known Issues and Solutions

<Mar 23, 2016 2:03:43 PM AEDT> <Info> <MessagingBridge> <BEA-200021> <Bridge "ActiveMeterNotificationInBridge" failed to get one of the adapters from JNDI (javax.naming.NameNotFoundException: Unable to resolve 'eis.jms.WLSConnectionFactoryJNDIXA'. Resolved 'eis.jms'; remaining name 'WLSConnectionFactoryJNDIXA').

javax.naming.NameNotFoundException: Unable to resolve 'eis.jms.WLSConnectionFactoryJNDIXA'. Resolved 'eis.jms'; remaining name 'WLSConnectionFactoryJNDIXA'

        at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1180)

        at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:270)

        at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:187)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:210)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:253)

        at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:426)

        at weblogic.jms.bridge.internal.MessagingBridge.startInternal(MessagingBridge.java:585)

        at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1055)

        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Solution: Deploy jms-xa-adp resource and target it to all the servers.  

<Apr 1, 2016 2:22:29 PM AEDT> <Debug> <MessagingBridgeRuntimeVerbose> <BEA-000000> <Exception:

javax.naming.AuthenticationException: Not permitted: invalid name or password [Root exception is javax.jms.JMSSecurityException: invalid name or password]

        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:670)

        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:491)

        at javax.naming.InitialContext.lookup(InitialContext.java:411)

        at weblogic.deployment.jms.ForeignOpaqueReference.getReferent(ForeignOpaqueReference.java:247)

        at weblogic.jndi.internal.WLNamingManager.getObjectInstance(WLNamingManager.java:92)

        at weblogic.jndi.internal.ServerNamingNode.resolveObject(ServerNamingNode.java:388)

        at weblogic.jndi.internal.BasicNamingNode.resolveObject(BasicNamingNode.java:886)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:219)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:253)

        at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:426)

        at javax.naming.InitialContext.lookup(InitialContext.java:411)

        at weblogic.jms.adapter.JMSBaseConnection$2.run(JMSBaseConnection.java:301)

        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)

        at weblogic.jms.adapter.JMSBaseConnection.startInternal(JMSBaseConnection.java:299)

        at weblogic.jms.adapter.JMSBaseConnection.start(JMSBaseConnection.java:256)

Solution:  Make sure proper Login details are provided in the Foreign server configuration

<Mar 30, 2016 2:54:40 PM AEDT> <Debug> <MessagingBridgeRuntimeVerbose> <BEA-000000> <Exception:

javax.naming.NoInitialContextException: Cannot instantiate class: com.tibco.tibjms.naming.TibjmsInitialContextFactory [Root exception is java.lang.ClassNotFoundException: com.tibco.tibjms.naming.TibjmsInitialContextFactory]

        at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:674)

        at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)

        at javax.naming.InitialContext.init(InitialContext.java:242)

        at javax.naming.InitialContext.<init>(InitialContext.java:216)

        at weblogic.deployment.jms.ForeignOpaqueReference.getReferent(ForeignOpaqueReference.java:190)

        at weblogic.jndi.internal.WLNamingManager.getObjectInstance(WLNamingManager.java:92)

        at weblogic.jndi.internal.ServerNamingNode.resolveObject(ServerNamingNode.java:388)

        at weblogic.jndi.internal.BasicNamingNode.resolveObject(BasicNamingNode.java:886)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:219)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)

        at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:253)

        at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:426)

        at javax.naming.InitialContext.lookup(InitialContext.java:411)

        at weblogic.jms.adapter.JMSBaseConnection$2.run(JMSBaseConnection.java:301)

        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)

        at weblogic.jms.adapter.JMSBaseConnection.startInternal(JMSBaseConnection.java:299)

        at weblogic.jms.adapter.JMSBaseConnection.start(JMSBaseConnection.java:256)

        at weblogic.jms.adapter.JMSManagedConnectionFactory.createManagedConnection(JMSManagedConnectionFactory.java:192)

        at weblogic.connector.security.layer.AdapterLayer.createManagedConnection(AdapterLayer.java:843)

        at weblogic.connector.outbound.ConnectionFactory.createResource(ConnectionFactory.java:91)

        at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1331)

        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:425)

        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:344)

        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:324)

        at weblogic.connector.outbound.ConnectionPool.reserveResource(ConnectionPool.java:705)

        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:318)

        at weblogic.connector.outbound.ConnectionManagerImpl.getConnectionInfo(ConnectionManagerImpl.java:409)

        at weblogic.connector.outbound.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:343)

        at weblogic.connector.outbound.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:134)

        at weblogic.jms.adapter.JMSBaseConnectionFactory.getSourceConnection(JMSBaseConnectionFactory.java:92)

        at weblogic.jms.bridge.internal.MessagingBridge.getConnections(MessagingBridge.java:834)

        at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1079)

        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:553)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Caused By: java.lang.ClassNotFoundException: com.tibco.tibjms.naming.TibjmsInitialContextFactory

        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

Solution: Download the Jars and provide the JAR details in the class path (setDomain.sh)

<Mar 30, 2016 2:18:05 PM AEDT> <Debug> <MessagingBridgeRuntimeVerbose> <BEA-000000> <Exception:

javax.naming.AuthenticationException: [Security:090304]Authentication Failed: User LOGIN_ID javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User LOGIN_ID denied [Root exception is javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User LOGIN_ID javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User LOGIN_ID denied]

        at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:44)

        at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:808)

        at weblogic.jndi.WLInitialContextFactoryDelegate.pushSubject(WLInitialContextFactoryDelegate.java:690)

        at weblogic.jndi.WLInitialContextFactoryDelegate.newContext(WLInitialContextFactoryDelegate.java:485)

        at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:373)

        at weblogic.jndi.Environment.getContext(Environment.java:319)

        at weblogic.jndi.Environment.getContext(Environment.java:288)

        at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)

        at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)

Solution:  Make sure you have created the Logins in security realm to connect with destination. 

Weblogic SAF Configuration

Weblogic SAF agents allow us to store the message  in local persistence , in case the remote server is not available.

Once the remote server is up , the SAF agents will forward the message to destination.

This will overcome the message loss issues when the destination is not available.

 Following configurations needs to be done to achieve SAF mechanism.

At Destination side.
    1. Create a JMS server
    2.  Create a JMS Module
    3. Create a Sub-deployment. Targeting to JMS server
    4. Create a JMS Topic or Queue targeting to Sub-Deployment
    5. Create a Connection Factory.

At Source Side.

   1. Create a JMS server
   2. Create a JMS Module
   3. Create a Subdeployment - Targeting to SAF Agent
   4. Create a Connection Factory
   5. Create a SAF Remote Context  (This will have a remote server URL, Login Id and Password)
   6. Create SAF Imported Destination (Use SAF Remote Context here)
   7. Create a Topic /Queue under SAF Imported Destination.
            a.  Give remote JNDI name - JNDI name of remote destination
            b.  Local JNDI Name - Local name and can be any name Ex: SAFisGood
   8.  Create a SAF Agent
             a. Select a persistence store
             b. Select Agent Type


After Completing the configuration test using following URL.

jms://host:port/SourceCF/Suffix_LocalJNDIName

SOA - FTP Adapter

SOA File / FTP Adapters enable BPEL process to exchange the files on local or remote file systems. 

Steps to Create File Adapter

• Drag and drop the file adapter component  to component pallet.
• Provide the File adapter name under Service Name
• Select Define from operation and schema (specified later) option
• select the Operation
• Read File
• Write File
• Synchronous Read file
• List Files
• Mention the physical or logical path in the next page
• Mention the Archive path if the file needs to the archived after reading 
• Select Delete file after successful retrieval option if we  need to delete the file after successful read
• Mention the type of files which needs to be processed
• Mention the file polling frequency 
• Create a schema file using existing file in the message window by clicking the settings icon
• After creating schema select finish

Difference between Read File and Synchronous Read File Operation 

Synchronous Read :  It is a synchronous operation and follow request - response paradigm.   File read is initiated by a INVOKE process.   If the file is not found then response will have null value. 

Before invoking file read , file should be present. 

Read File :  It is a waiting call and  process will be invoked by a RECEIVE process and will be polling for a file at regular interval. 

File Processing :

If we have many inbound files to process or very large files of more than 1 MB, we may need to increase the config timeout value in the transaction-manager.xml file as shown in the following example:

<transaction-timeout="30000"/>

NOTE:

• The Oracle File and FTP Adapters can also treat file contents as an opaque object and pass the contents in their original format (without performing translation). The opaque option handles binary data such as JPGs and GIFs, whose structure cannot be captured in an XSD or data you do not want to have translated.

• The Oracle File and FTP Adapters support polling multiple directories within a single activation. You can specify multiple directories in J-Developer as opposed to a single directory. This is applicable to both physical and logical directories.

• The Oracle File Adapter provides support for transferring large files as attachments. To use this feature, select the Read File As Attachment check box in the JDeveloper wizard while configuring the "Read operation." This option opaquely transfers a large amount of data from one place to another as attachments. For example, you can transfer large MS Word documents, images, and PDFs without processing their content within the composite application.

SOA Dehydration Process

Oracle BPEL Process Manager uses the dehydration store database to maintain long-running asynchronous processes and their current state information in a database while they wait for asynchronous callbacks. Storing the process in a database preserves the process and prevents any loss of state or reliability if a system shuts down or a network problem occurs. There are two types of processes in Oracle BPEL Process Manager. These processes impact the dehydration store database in different ways.

• Transient processes: this process type does not incur any intermediate dehydration points during process execution. If there are unhandled faults or there is system downtime during process execution, the instances of a transient process do not leave a trace in the system. Instances of transient processes cannot be saved in-flight (whether they complete normally or abnormally). Transient processes are typically short-lived, request-response style processes.

• Durable processes: this process type incurs one or more dehydration points in the database during execution because of the following activities:
  • Receive activity
  • OnMessage branch in a pick activity
  • OnAlarm branch in a pick activity
  • Wait activity
  Instances of durable processes can be saved in-flight (whether they complete normally or abnormally). These processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized. 

• An idempotent activity is an activity that can be retried (for example, an assign activity or an invoke activity). Oracle BPEL Server saves the instance after a nonidempotent activity.

Idempotent BPEL Property

A BPEL invoke activity is by default an idempotent activity, meaning that the BPEL process does not dehydrate instances immediately after invoke activities. Therefore, if idempotent is set to true and Oracle BPEL Server fails right after an invoke activity executes, Oracle BPEL Server performs the invoke again after restarting. This is because no record exists that the invoke activity has executed. This property is applicable to both durable and transient processes.

If idempotent is set to false, the invoke activity is dehydrated immediately after execution and recorded in the dehydration store. If Oracle BPEL Server then fails and is restarted, the invoke activity is not repeated, because Oracle BPEL Process Manager sees that the invoke already executed.

When idempotent is set to false, it provides better failover protection, but at the cost of some performance, since the BPEL process accesses the dehydration store much more frequently. This setting can be configured for each partner link in the bpel.xml file.

Setting this parameter to true can significantly improve throughput. However, as mentioned previously, you must ensure that the partner's service can be safely retried in the case of a server failure. Some examples of where this property can be set to true are read-only services (for example, CreditRatingService) or local EJB/WSIF invocations that share the instance's transaction

This property has the following values:

false: activity is dehydrated immediately after execution and recorded in the dehydration store

true (default): If Oracle BPEL Server fails, it performs the activity again after restarting. This is because the server does not dehydrate immediately after the invoke and no record exists that the activity executed.

inMemoryOptimization BPEL Property

This property indicates to Oracle BPEL Server that this process is a transient process and dehydration of the instance is not required. When set to true, Oracle BPEL Server keeps the instances of this process in memory only during the course of execution. This property can only be set to true for transient processes (that is, those that do not contain any middle process receive, pick, or wait activities).

The default for this property is false, which means that instances are persisted completely and recorded in the dehydration store database for a synchronous BPEL process.

When inMemoryOptimization is set to true, dehydration is deactivated, and Oracle BPEL Process Manager keeps instances in memory only. The settings for the completionPersistPolicy and completionPersistLevel properties are also examined to determine persistence behavior. The inMemoryOptimization property can improve throughput when set to true and, in conjunction with these two other properties, can minimize database growth.

BPEL - CallBack Message Handling

BPEL engine maintains all Async call back messages into database table called dlv_message. Wecan see such all messages in BPEL console call-back manual recovery area.The query being used by bpel console is joined on dlv_message and work_item tables.This query simply picks up all call back messages which are undelivered and have not been modified with in certain threshold time.

Call-back messages are processed in following steps

• BPEL engine assigns the call-back message to delivery service
• Delivery service saves the message into dlv_message table with state 'UNDELIVERED-0'
• Delivery service schedules a dispatcher thread to process message asynchronously
• Dispatcher thread enqueues message into JMS queue
• Message is picked up by MDB
• MDB delivers the message to actual BPEL process  waiting for call-back and changes state to 'HANDLED=2'

So given above steps, there is always possibility that message is available in dlv_message table but MDB is failed in delivering it to BPEL process which keeps message always in state= 0.