SOA Dehydration Process

Oracle BPEL Process Manager uses the dehydration store database to maintain long-running asynchronous processes and their current state information in a database while they wait for asynchronous callbacks. Storing the process in a database preserves the process and prevents any loss of state or reliability if a system shuts down or a network problem occurs. There are two types of processes in Oracle BPEL Process Manager. These processes impact the dehydration store database in different ways.

• Transient processes: this process type does not incur any intermediate dehydration points during process execution. If there are unhandled faults or there is system downtime during process execution, the instances of a transient process do not leave a trace in the system. Instances of transient processes cannot be saved in-flight (whether they complete normally or abnormally). Transient processes are typically short-lived, request-response style processes.

• Durable processes: this process type incurs one or more dehydration points in the database during execution because of the following activities:
  • Receive activity
  • OnMessage branch in a pick activity
  • OnAlarm branch in a pick activity
  • Wait activity
  Instances of durable processes can be saved in-flight (whether they complete normally or abnormally). These processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized. 

• An idempotent activity is an activity that can be retried (for example, an assign activity or an invoke activity). Oracle BPEL Server saves the instance after a nonidempotent activity.

Idempotent BPEL Property

A BPEL invoke activity is by default an idempotent activity, meaning that the BPEL process does not dehydrate instances immediately after invoke activities. Therefore, if idempotent is set to true and Oracle BPEL Server fails right after an invoke activity executes, Oracle BPEL Server performs the invoke again after restarting. This is because no record exists that the invoke activity has executed. This property is applicable to both durable and transient processes.

If idempotent is set to false, the invoke activity is dehydrated immediately after execution and recorded in the dehydration store. If Oracle BPEL Server then fails and is restarted, the invoke activity is not repeated, because Oracle BPEL Process Manager sees that the invoke already executed.

When idempotent is set to false, it provides better failover protection, but at the cost of some performance, since the BPEL process accesses the dehydration store much more frequently. This setting can be configured for each partner link in the bpel.xml file.

Setting this parameter to true can significantly improve throughput. However, as mentioned previously, you must ensure that the partner's service can be safely retried in the case of a server failure. Some examples of where this property can be set to true are read-only services (for example, CreditRatingService) or local EJB/WSIF invocations that share the instance's transaction

This property has the following values:

false: activity is dehydrated immediately after execution and recorded in the dehydration store

true (default): If Oracle BPEL Server fails, it performs the activity again after restarting. This is because the server does not dehydrate immediately after the invoke and no record exists that the activity executed.

inMemoryOptimization BPEL Property

This property indicates to Oracle BPEL Server that this process is a transient process and dehydration of the instance is not required. When set to true, Oracle BPEL Server keeps the instances of this process in memory only during the course of execution. This property can only be set to true for transient processes (that is, those that do not contain any middle process receive, pick, or wait activities).

The default for this property is false, which means that instances are persisted completely and recorded in the dehydration store database for a synchronous BPEL process.

When inMemoryOptimization is set to true, dehydration is deactivated, and Oracle BPEL Process Manager keeps instances in memory only. The settings for the completionPersistPolicy and completionPersistLevel properties are also examined to determine persistence behavior. The inMemoryOptimization property can improve throughput when set to true and, in conjunction with these two other properties, can minimize database growth.

BPEL - CallBack Message Handling

BPEL engine maintains all Async call back messages into database table called dlv_message. Wecan see such all messages in BPEL console call-back manual recovery area.The query being used by bpel console is joined on dlv_message and work_item tables.This query simply picks up all call back messages which are undelivered and have not been modified with in certain threshold time.

Call-back messages are processed in following steps

• BPEL engine assigns the call-back message to delivery service
• Delivery service saves the message into dlv_message table with state 'UNDELIVERED-0'
• Delivery service schedules a dispatcher thread to process message asynchronously
• Dispatcher thread enqueues message into JMS queue
• Message is picked up by MDB
• MDB delivers the message to actual BPEL process  waiting for call-back and changes state to 'HANDLED=2'

So given above steps, there is always possibility that message is available in dlv_message table but MDB is failed in delivering it to BPEL process which keeps message always in state= 0.

Custom XPATH Function in SOA 11g

This post will explain how to create a custom function and configure in SOA 11g

1. Create a JAVA Class which we need to use it as a custom function.

2. The XSLT processor needs the custom Xpath function to be implemented as Static Java Method with a particular  signature as mentioned below.

public class SOAConstants
{
  public static class SOAConstantsXPathFunction
    implements IXPathFunction
  {
    public Object call(IXPathContext paramIXPathContext, List paramList)
      throws XPathFunctionException
    {
      if ((paramList.size() == 2) && ((paramList.get(0) instanceof String)) && ((paramList.get(1) instanceof String))) {
        try
        {
          String str = getPropertyValue(paramList);
          return str;
        }
        catch (Exception localException)
        {
          System.out.println("The Exception : " + localException);
        }
      }
      throw new XPathFunctionException("Pass Only the Service name  and Constant name to look up");
    }
   
    private static String getPropertyValue(List paramList)
      throws XPathFunctionException
    {
      System.out.println("Inside getPropertyValue");
      String str1 = null;
      try
      {
        Document localDocument = SOAConstantsReader.getInstance().getConstantsAsDoc();
        String str2 = paramList.get(0).toString();
        String str3 = paramList.get(1).toString();
        System.out.println("-- payload :" + str2 + " - " + str3);
        NodeList localNodeList1 = localDocument.getElementsByTagName(str2);
        System.out.println("Got Service NodeList");
        Element localElement = (Element)localNodeList1.item(0);
        System.out.println("Got Service Element" + localNodeList1.item(0).getChildNodes().getLength());
        NodeList localNodeList2 = localElement.getElementsByTagName(paramList.get(1).toString());
        localElement = (Element)localNodeList2.item(0);
        str1 = localElement.getChildNodes().item(0).getNodeValue();
        System.out.println("\nThe value of property: " + str1);
      }
      catch (Exception localException)
      {
        throw new XPathFunctionException("The Exception : " + localException);
      }
      return str1;
    }
  }
}

3. The name of the method can be anything but it must be static and take the given parameters.

4. Implement a  function for XSLT using below format.

public Object call(IXPathContext paramIXPathContext, List paramList)
      throws XPathFunctionException

5. Note the method/Function name must be CALL and non static

6. Compile the code and create a JAR file to implement in Domain folder.

7. Now we need to create a descriptor to describe to SOA Suite and XSLT processor that what custom function we have created.

8. We do this by creating or updating the below file in META-INF directory which will create during the installation of SOA suite.

ext-soa-xpath-functions-config.xml

PATH =  $FMW_HOME/Oracle_SOA1/soa/modules/oracle.soa.ext_11.1.1/classes/META-INF

9. Edit the above file to add the function details

<?xml version = '1.0' encoding = 'UTF-8'?>
<soa-xpath-functions xmlns="http://xmlns.oracle.com/soa/config/xpath"
version="11.1.1" resourceBundle="oracle.tip.tools.ide.common.resource.IDEMessageBundle" xmlns:ora="http://schemas.oracle.com/xpath/extension"
xmlns:vha="http://www.oracle.com/XSL/Transform/java/com.vha.util.action.SOAConstants"
xmlns:vha1="http://www.oracle.com/XSL/Transform/java/com.vha.util.action.SetCompositeInstanceIndex"
xmlns:vhadvm="http://www.oracle.com/XSL/Transform/java/com.vha.util.dvm.CustomDVMLookup"
>
  <function name="vha:getPropertyValue">
    <className>com.vha.util.action.SOAConstants$SOAConstantsXPathFunction</className>
    <return type="string"/>
    <params>
      <param name="parentTagName" type="string"/>
      <param name="childTagName" type="string"/>
    </params>
    <desc resourceKey="GET_PROPERTY VALUE"/>
    <detail>
       <![CDATA[This function will get the value of element from SOAConstants.xml file]]>
    </detail>
  </function>



10. The namespace for the function must be “http://www.oracle.com/XSL/Transform/java/” followed by the full classname of the impementing class.  This is required by the XSLT processor.  The namespace prefix can be anything

11. The function name can be anything but it must have a namespace prefix.


12. Before using the custom function in JDeveloper it needs to be registered.  To do this in JDeveloper we go to the “SOA” section in “Preferences” and add the newly created jar file.  To make JDeveloper see the file we then need to restart JDeveloper.  After doing this we will see the XPath function appears in the “User Defined Extension Functions” in the expression builder.

TCP Dump-Linux/Unix OR SNOOP - Solaris

TCP Dump and  Snoop are built in tools in Unix/Linux and Solaris respectively.

These tools are used to capture the network packets and store it in a file for analysis.

TCP Dump or Snoop can be captured for a particular host of source and store it in a file for further analysis.

The captured dump file will contain source and destination address, state and flag information set in the header.

By default tcp dump will capture first 96 bytes of the capture and we can modify the command to capture complete information of the packets.

TCP Dump or Snoop file can be analyzed using wire-shark tool .

commands used to capture the dumps are

tcpdump -w filename.pcap -s0 -i -n src host IPAddress

snoop -i IP

Solaris Memory Utilization

Solaris servers uses certain percentage of free memory for IO cache.

This is based on the ZFS configuration.  This configuration can be checked using root or kernel access.

If there is no memory available, then these cache memory will be used.

If the Application need memory, the I/O cache will release the memory.

The "sr" value on vmstat provide the status of the Memory swap rate.

As long as the "sr" value stays at zero there is no memory issue. 

HTTP - Access Log Entries

c-ip date time cs-method cs-uri cs-status bytes time-taken

Weblogic DataSource

Data Source Connection Pool Sizing

By Steve Felts on Dec 10, 2012

One of the most time-consuming procedures of a database application is establishing a connection. The connection pooling of the data source can be used to minimize this overhead.  That argues for using the data source instead of accessing the database driver directly.

Configuring the size of the pool in the data source is somewhere between an art and science – this article will try to move it closer to science. 

From the beginning, WLS data source has had an initial capacity and a maximum capacity configuration values.  When the system starts up and when it shrinks, initial capacity is used.  The pool can grow to maximum capacity.  Customers found that they might want to set the initial capacity to 0 (more on that later) but didn’t want the pool to shrink to 0.  In WLS 10.3.6, we added minimum capacity to specify the lower limit to which a pool will shrink.  If minimum capacity is not set, it defaults to the initial capacity for upward compatibility.   We also did some work on the shrinking in release 10.3.4 to reduce thrashing; the algorithm that used to shrink to the maximum of the currently used connections or the initial capacity (basically the unused connections were all released) was changed to shrink by half of the unused connections.

The simple approach to sizing the pool is to set the initial/minimum capacity to the maximum capacity.  Doing this creates all connections at startup, avoiding creating connections on demand and the pool is stable.  However, there are a number of reasons not to take this simple approach.

When WLS is booted, the deployment of the data source includes synchronously creating the connections.  The more connections that are configured in initial capacity, the longer the boot time for WLS (there have been several projects for parallel boot in WLS but none that are available).  Related to creating a lot of connections at boot time is the problem of logon storms (the database gets too much work at one time).   WLS has a solution for that by setting the login delay seconds on the pool but that also increases the boot time.

There are a number of cases where it is desirable to set the initial capacity to 0.  By doing that, the overhead of creating connections is deferred out of the boot and the database doesn’t need to be available.  An application may not want WLS to automatically connect to the database until it is actually needed, such as for some code/warm failover configurations.

There are a number of cases where minimum capacity should be less than maximum capacity.  Connections are generally expensive to keep around.  They cause state to be kept on both the client and the server, and the state on the backend may be heavy (for example, a process).  Depending on the vendor, connection usage may cost money.  If work load is not constant, then database connections can be freed up by shrinking the pool when connections are not in use.  When using Active GridLink, connections can be created as needed according to runtime load balancing (RLB) percentages instead of by connection load balancing (CLB) during data source deployment.

Shrinking is an effective technique for clearing the pool when connections are not in use.  In addition to the obvious reason that there times where the workload is lighter,  there are some configurations where the database and/or firewall conspire to make long-unused or too-old connections no longer viable.  There are also some data source features where the connection has state and cannot be used again unless the state matches the request.  Examples of this are identity based pooling where the connection has a particular owner and XA affinity where the connection is associated with a particular RAC node.  At this point, WLS does not re-purpose (discard/replace) connections and shrinking is a way to get rid of the unused existing connection and get a new one with the correct state when needed.

So far, the discussion has focused on the relationship of initial, minimum, and maximum capacity.  Computing the maximum size requires some knowledge about the application and the current number of simultaneously active users, web sessions, batch programs, or whatever access patterns are common.  The applications should be written to only reserve and close connections as needed but multiple statements, if needed, should be done in one reservation (don’t get/close more often than necessary).  This means that the size of the pool is likely to be significantly smaller then the number of users.  

If possible, you can pick a size and see how it performs under simulated or real load.  There is a high-water mark statistic (ActiveConnectionsHighCount) that tracks the maximum connections concurrently used.  In general, you want the size to be big enough so that you never run out of connections but no bigger.   It will need to deal with spikes in usage, which is where shrinking after the spike is important.  Of course, the database capacity also has a big influence on the decision since it’s important not to overload the database machine.  Planning also needs to happen if you are running in a Multi-Data Source or Active GridLink configuration and expect that the remaining nodes will take over the connections when one of the nodes in the cluster goes down.  For XA affinity, additional headroom is also recommended.  

In summary, setting initial and maximum capacity to be the same may be simple but there are many other factors that may be important in making the decision about sizing.