Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Tuesday, October 5, 2010

spring apache commons configuration

I was surprised to find out that integrating the apache commons configuration with the spring framework is not trivial.  I was hoping that there will be one bean that we can use out of the box within the spring framework or the apache commons framework and we set that as a property in the bean and we can go on.  Unfortunately it was not that easy.   It takes a few steps to get this working.  Here are the steps that you need to follow to get this integration working:

1) Define the properties beans in spring xml file (typically ApplicationConfiguration.xml file)
 as follows: 
 <!-- Apache Commons Configuration Composite configuration -->
    <bean id="configurations"
        class="org.springmodules.commons.configuration.CommonsConfigurationFactoryBean">
        <property name="configurations">
            <list>
                <bean class="org.apache.commons.configuration.PropertiesConfiguration">
                    <constructor-arg type="java.net.URL"
                        value="classpath:myconfiguration.properties" /> 
                    <property name="reloadingStrategy">
                        <bean class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/>
                    </property>
                </bean>
                <bean class="org.apache.commons.configuration.SystemConfiguration"/>               
            </list>
        </property>
        <!-- define configuration as a set of spring resources -->
    </bean>
    <bean id="configuration" class="org.apache.commons.configuration.Configuration" factory-bean="&amp;configurations" factory-method="getConfigurations"/>

2) Define a bean with the getter and setter that returns/accepts an array of org.apache.commons.configuration.Configuration class.  To make it easier for the rest of the code to get the configuration easy, you may want to add a utility method in there to return the combined Configutaion.  Here is an example:

private Configuration configs[] ;
private ConfigurationBuilder cfgBuilder ;

    public Configuration[] getConfigs() {
        return configs;
    }

    public void setConfigs(Configuration[] configs) {
        this.configs = configs;
        cfgBuilder = new ConfigurationBuilder();
        for (Configuration cgf: configs) {
            cfgBuilder.addConfiguration(cgf);
        }
    }

    public Configuration getConfig() {
        return cfgBuilder.getConfiguration();
    }
3) Define the bean properties in the spring xml file.  I suggest, that you may want to define the above method in a base class of all your beans and define it as an abstract bean.  This will enable you to use the properties in all your beans without having to define the properties in every bean.  Here is an example:
  <bean id="baseActionBean"  class="com.my.company.BaseAction"  abstract="true" >
            <property name="configs" ref="configuration" />
        </bean>
         <bean id="logonClass" class="com.my.company.UsefulAction"  parent="baseActionBean">
         </bean>
4)  Add the necessary libraries if you do not have them already.  Here is the list of jars you will need.
commons-lang-*.jar
commons-configuration*.jar
spring-modules-jakarta-commons*.jar
 With these changes, you should be able to use the properties in your beans with a code as simple as getConfig().getString("propertyKey")

Wednesday, August 25, 2010

Java URL connection timeout - default timeout might save you from hanging

Many times, when you make a url connection or any connection that works over tcp ip (ftp, http etc) protocol, it is possible that your client just hangs.  It is frustrating to debug this kind of issue because, you do not get any exception from your application except it just hangs.  This can be a problem especially in production environments where there are firewalls between every single component and it is hard to trace network traffic.   

If you are using sun jdk.1.4 or above there is a way you can prevent your code from just hanging.  Best of all, you do not even have to make a code change.  The sun jvm has a way to specify default timeout values for the net client.   All you have to do is just add the following java parameters to the command line that starts your application.  
-Dsun.net.client.defaultConnectTimeout=TimeoutInMiliSec -Dsun.net.client.defaultReadTimeout=TimeoutInMiliSec 
 
This will force the client to timeout and hopefully your application logs the exception that gives enough clue to debug the issue.   For more information about specifying network properties in java check out the Networking Properties guide from sun/oracle.

Monday, April 19, 2010

Pagination using sql

If you have a complex query that returns lot of rows from the database and you are displaying the results in a table,  you may want to add some pagination logic to your application.  However, most people add pagination to the application layer, but retrieve all the rows back from the database.  Java programming frameworks/API such as IBatis, JPA etc provides a mechanism to limit the number of rows retrieved or skip a number of rows.   However, the implementation of those framework is left behind the jdbc driver and often results in the data still retrieved to the application server and then discarded.   You could avoid this resource wastage by changing your query to limit the rows.  The following is an example of how you can do just that.
    select * from ( select rownum rnum, a.* from (
         select columns from your_complex_table order by someField )  a 
            where rownum <  #UPPER_LIMIT )     where rnum >= #LOWER_LIMIT
In this example, you can change the values of #UPPER_LIMIT and #LOWER_LIMIT as parameters to your query limit the results to just the rows you want to deal with.  If you are using a  iBATIS you can make your actual query (the one does fetches all the records) in to an sql fragment and reuse it for the paginated query and get row counts.   It also helps to get all the records in case of exporting all the data to some external format.  The following example shows how you can reuse the sql fragment within your sqlMapping.
<sql id="selectItems">
select columns from your_complex_table order by someField
</sql>
<select id="selectItemCount" resultClass="int">
SELECT COUNT(*) AS total
<include refid="selectItems"/>
</select>
<select id="selectPaginated" resultClass="Item" parameterClass="map">
 select * from ( select rownum rnum, a.* from (
<include refid="selectItems"/>
    ) a  where rownum <  #UPPER_LIMIT )
   where rnum >= #LOWER_LIMIT
</select>

Tuesday, July 1, 2008

Struts/Tiles/oc4j logging

Recently I was troubleshooting an issue with a web application that uses struts/tiles and runs in an oc4j container. I application was getting a exception and oc4j will not print the exception stack trace in the output. It was hard to understand what is going on and it took me a while to figure out. So I am documenting here the process, so it can be a handy reference next time around or for some one who comes across the same issue.

To understand what was going on, the first step was to see what was the exception. To do this I set the development mode to true in the global_application.xml file. The global-web-application.xml file can be found under %ORACLE_HOME%/j2ee/%instance%/config/global-web-application.xml. Update the development attrubute on the orion-web-app tag to true and restart the instance.

<orion-web-app development="true" xsi="http://www.w3.org/2001/XMLSchema-instance" nonamespaceschemalocation="http://xmlns.oracle.com/oracleas/schema/orion-web-10_0.xsd" directory="./persistence" webdir="/servlet" timeout="0" tlds="false" version="0">


Now, if you visit the page on the browser, you will see the stack trace on the browser. This is a handy method, if your log files are cluttered. Alternatively you may view the stacktrace in the log files if you have configured oc4j to print standard out and error to a file. (for more information on how to setup refer to my earlier entry)

In my current situation, the stacktrace was not very useful as it did not point to any application code directly. The following is the stacktrace I received.


[ ERROR] [2008-07-01 09:23:13,500] [AJPRequestHandler-HTTPThreadGroup-5:] {{ServletException in '/StrutsAction2.do?QueryParm=parmvalue': null}} [org.apache.struts.taglib.tiles.InsertTag]
javax.servlet.ServletException: Error in servlet
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:757)
at com.evermind.server.http.ServletRequestDispatcher.unprivileged_include(ServletRequestDispatcher.java:160)
at com.evermind.server.http.ServletRequestDispatcher.access$000(ServletRequestDispatcher.java:50)
at com.evermind.server.http.ServletRequestDispatcher$1.oc4jRun(ServletRequestDispatcher.java:97)
at oracle.oc4j.security.OC4JSecurity.doPrivileged(OC4JSecurity.java:283)
at com.evermind.server.http.ServletRequestDispatcher.include(ServletRequestDispatcher.java:102)
at com.evermind.server.http.EvermindPageContext.include(EvermindPageContext.java:396)
at com.evermind.server.http.EvermindPageContext.include(EvermindPageContext.java:349)
at org.apache.struts.tiles.TilesUtilImpl.doInclude(TilesUtilImpl.java:101)
at org.apache.struts.tiles.TilesUtil.doInclude(TilesUtil.java:137)
at org.apache.struts.taglib.tiles.InsertTag.doInclude(InsertTag.java:758)
at org.apache.struts.taglib.tiles.InsertTag$InsertHandler.doEndTag(InsertTag.java:890)
at org.apache.struts.taglib.tiles.InsertTag.doEndTag(InsertTag.java:460)
at _layouts._template._jspService(_template.java:324)
at com.orionserver.http.OrionHttpJspPage.service(OrionHttpJspPage.java:59)
at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:453)
at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:591)
at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:515)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:711)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368)
at com.evermind.server.http.ServletRequestDispatcher.unprivileged_forward(ServletRequestDispatcher.java:259)
at com.evermind.server.http.ServletRequestDispatcher.access$100(ServletRequestDispatcher.java:50)
at com.evermind.server.http.ServletRequestDispatcher$2.oc4jRun(ServletRequestDispatcher.java:193)
at oracle.oc4j.security.OC4JSecurity.doPrivileged(OC4JSecurity.java:283)
at com.evermind.server.http.ServletRequestDispatcher.forward(ServletRequestDispatcher.java:198)
at org.apache.struts.action.RequestProcessor.doForward(RequestProcessor.java:1056)
at org.apache.struts.tiles.TilesRequestProcessor.doForward(TilesRequestProcessor.java:261)
at org.apache.struts.tiles.TilesRequestProcessor.processTilesDefinition(TilesRequestProcessor.java:237)
at org.apache.struts.tiles.TilesRequestProcessor.processForwardConfig(TilesRequestProcessor.java:300)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:231)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:711)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368)
at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:866)
at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:448)
at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:302)
at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:190)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
at java.lang.Thread.run(Thread.java:595)


As you can see, all I can see is it getting a null exception when the jvm is trying to process the request for StrutsAction2. Unfortunately it was not printing the application class that was causing the exception. The reason for the issue is the exception is happening in a different thread than the one the stacktrace is printed.
Here is what was happening. I was actually calling url accociated with StrutsAction1. The tile definition for the StrutsAction1 includes StrutsAction2 in its response. The stacktrace oc4j was printing was from the thread that was processing StrutsAction2 which is a seperate thread. The only clue I could get from here is that the error is occoring during StrutsAction2. If I try to invoke StrutsAction2 directly it seems to be working fine.

This made things a bit more complex and I started adding more and more debugging statements to the strutsaction2. Then I realized there is a Custom tiles plugin we have defined for the app and it had a debug statement. On the outlook, the debug statement looked harmless to me. However, I was a bit surprised by the fact that I was not even seeing the debugline on the logfile even after turning debug on.

Here is the debug statement.


Logger.debug("Processing Request: " + request.getContextPath() + request.getServletPath() +" Req Detail: " + getLogString(request));

private String getLogString (HttpServletRequest req) {
StringBuffer buf = new StringBuffer("Request Parms:");
Enumeration parms = req.getParameterNames() ;
if (parms != null) {
while ( parms.hasMoreElements() ) {
String parm = parms.nextElement() ;
String pVal = req.getParameter(parm);
buf.append(parm + "=" + pVal + ";");
}
}
buf.append(": Req Headers: ");
Enumeration headers = req.getHeaderNames() ;
if (headers != null ) {
while ( headers.hasMoreElements() ) {
String parm = headers.nextElement() ;
String pVal = req.getHeader(parm);
buf.append(parm + "=" + pVal + ";");
}
}

Enumeration attribs = req.getAttributeNames() ;
if(attribs != null) {
while ( attribs.hasMoreElements() ) {
String parm = attribs.nextElement() ;
Object pVal = req.getAttribute(parm);
buf.append(parm + "=" + pVal.toString() + ";");
}
}
return buf.toString();
}


Not seeing the debug line in the log provided me the next clue. The issue might be somewhere in the debug statement. I came to the conclusion because this debug statement was the first thinging happening in the preprocess method for the plugin.

So my next step is to add a try catch around the debug line and printed the stacktrace in case of ecxeption. This pointed to the bug in the getLogString method. It was the line getting the attribute value "pVal.toString()" as the specific call flow, it was running in to an attribute that had a null value.

So I added a null check to the line as follows to get going. This was interesting beacause the common code that was suposed to be helping with debug was itself the one causing the bug. The leason here is be extra carefull with debug statements especially when adding to comon code.

buf.append(parm + "=" + (pVal==null?"null":pVal.toString()) + ";");"

Tuesday, April 29, 2008

OC4J JDBC Connection Pool issue with ResultSet.getStatement()

Recently one our customers migrated their application from Tomcat to OC4J container and noticed some interesting observations regarding JDBC connection pool. I thought I would share here as it will help others too.

The Issue:
After migrating the application to OC4J, the number of connections to the database held by the enterprise manager did not match with the actual connections held in the database. The customer had Interscope's Wily product and it reported a different number for the number of database connections.

The application's heap usage also goes up significantly as it the conection objects are not released.

This lead us to evaluate the application's code on how they are getting the connection and releasing it. The way they were getting the connection was in line with the typical usage. However, when they are releasing the connection, they are going it by getting the connection object through the reference from the resultset object. They were using a utility method sibilar to the one below.


protected synchronized Connection getConn() {
Connection conn = null;
try {
Context initContext = new InitialContext();

DataSource ds = (DataSource)initContext.looku("dataSourceName");
conn = ds.getConnection();

} catch (Exception e) {
Logger.error("Error in getting a JDBC " + "connection " + e.getMessage());
}
return conn;
}

public synchronized void closeResultSetAndItsAssoc(ResultSet rs) {
if (rs != null) {
try {

Statement stmt = rs.getStatement();
Connection conn = stmt.getConnection();

if (rs != null) {
try { rs.close(); } catch (SQLException ignore) {}
}
if (stmt != null) {
try { stmt.close(); } catch (SQLException ignore) {}
}
if (conn != null) {
try { conn.close(); } catch (SQLException ignore) {}
}
} catch (SQLException ignore) {
appLogger.warn(ignore.getMessage());
}
}

}


This code snippet worked well with Tomcat container. However, with the OC4J container, the actual connection object obtained by the getConn() method above is different from the one obtained by the stmt.getConnection() method. So, it appeared to the connection pool manager that the applicatio never closed the connection. Interestingly the one obtained by the stmt.getConnection() method is the actual physical connection and we would notice the connection is closed at the database rather than going back to inactive state.

Luckily, they were using the getConn() method and the closeResultSetAndItsAssoc() methods from the same object. We were able to make a quick fix to the closeResultSetAndItsAssoc method by closing the connection reference obtained using the getConn() method.

Tuesday, April 22, 2008

What is going on with DATE and TIMESTAMP?

I have ran in to developers using java.sql.Date object in Oracle and wondering why the date object does not contain the time information. The short answer is java.sql.Date and you need to use the java.sql.TimeStamp object that contains both Date and timestamp information.

A detailed explanation of the issue can be found in the Oracle web site . I am copying the text here for a quick reference.

Prior to 9.2, the Oracle JDBC drivers mapped the DATE SQL type to java.sql.Timestamp. This made a certain amount of sense because the Oracle DATE SQL type contains both date and time information as does java.sql.Timestamp. The more obvious mapping to java.sql.Date was somewhat problematic as java.sql.Date does not include time information. It was also the case that the RDBMS did not support the TIMESTAMP SQL type, so there was no problem with mapping DATE to Timestamp.

In 9.2 TIMESTAMP support was added to the RDBMS. The difference between DATE and TIMESTAMP is that TIMESTAMP includes nanoseconds and DATE does not. So, beginning in 9.2, DATE is mapped to Date and TIMESTAMP is mapped to Timestamp. Unfortunately if you were relying on DATE values to contain time information, there is a problem.

There are several ways to address this problem:

Alter your tables to use TIMESTAMP instead of DATE. This is probably rarely possible, but it is the best solution when it is.

Alter your application to use defineColumnType to define the columns as TIMESTAMP rather than DATE. There are problems with this because you really don't want to use defineColumnType unless you have to (see What is defineColumnType and when should I use it?).

Alter you application to use getTimestamp rather than getObject. This is a good solution when possible, however many applications contain generic code that relies on getObject, so it isn't always possible.

Set the V8Compatible connection property. This tells the JDBC drivers to use the old mapping rather than the new one. You can set this flag either as a connection property or a system property. You set the connection property by adding it to the java.util.Properties object passed to DriverManager.getConnection or to OracleDataSource.setConnectionProperties. You set the system property by including a -D option in your java command line.

java -Doracle.jdbc.V8Compatible="true" MyApp

Monday, April 7, 2008

JNDI Lookup for JDBC resources

Recently I came across an issue where the code to perform a JNDI lookup for a JDBC resource that worked fine in a tomcat container did not work when migrated to an OC$J container. Looking at the details of the code and the resource definitions explained why it did not work and the fix needed.

The following is code snippet that is used to get the datasource object by the application.


String dsName = "myDataSource";
initCtx = new InitialContext();
envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup(dsName);



In Tomcat the server.xml file contains the definition for the datasource "myDataSource". The resource ref mapping is done by an entry in the Catalina/hostname/warname.xml file. This setup works fine in a Tomcat container.


When ported to oc4j container, the datasource "myDataSource" is defined in the application's context (in this case it was the default application). However the code is when running within oc4j container, it failed to find the resource. Taking a closer look at the issue helped identify the solution.


The application is looking for the resource name "myDataSource" within its environment. In Tomcat this resource name was made available to it by the context definition within the Catalina/hostname/appname.xml file by having the following entry


In OC4J the datasources.xml file can provide this mapping. However, providing that mapping itself is not sufficient. The web.xml file needs to have a resource-ref definition that the name referenced within its environment is the globalname defined else where. Tomcat container was forgiving for some reason and it lookedup the resource withoout having the resource-ref definition. In order to fix the issue we had tow choice:

1) Change the code to lookup the resource using the global jndi name. ie

String dsName = "myDataSource";
initCtx = new InitialContext();
DataSource ds = (DataSource) initCtx.lookup(dsName);



or
2) Add the resource-ref to the web.xml file so the local name is mapped to the jndi global name.

<resource-ref>
<res-ref-name>myDataSource
<res-type >javax.sql.DataSource
<res-auth >Container
<res-sharing-scope >Shareable< /res-sharing-scope>
</resource-ref>



I personally prefer the later as it gives the option to change DataSource reference/names etc without having to change the code.

Saturday, March 29, 2008

URLConnection is silently retrying POST request

Recently, I was debugging an issue with a webservice that takes long time to respond. I noticed that the client was using the URLconnection object and it was automatically retrying the connection if the server abruptly terminated the connection. I was not expecting the client to reconnect to the server as I did not find any client side code to retry the connection. After googling for similar symptoms I found a known bug (ID 6382788) with the URLConnection object. Below is the explanation of the bug and the work around suggested by sun.
Non-idempotent http POST requests are being automatically resent to the server if the
server does not respond with a valid http response or an IOException occurs. This is
incorrect according to the rfc. Only idempotent requests are supposed to be
automatically resent.

As this has been the behavior of Sun's http client forever, some users may be relying
on this behavior without even knowing it. So the solution is to allow this behavior
to be configured through a system property, sun.net.http.retryPost.

sun.net.http.retryPost determines if the an unsuccessful http POST request will be
automatically resent to the server. Unsuccessful in this case means the server did
not send a valid http response or an IOException occurred. The default value is true
to maintain backward compatibility.
By the way the connection was reset because of a firewall that was terminating the connection as it was taking longer than the defined idle timeout.

Monday, February 11, 2008

HTTPS Connection in Java


Every time I had to create a java client to access a page using a https connection, I come across some issue and I keep googling for the answer. So this time I decided to document this so it will be useful for me the next time. There are few steps that we need to follow to make a https connection in java.

1) Create a trust store
To create a https client you need to provide the trust store with the certificate of the host you are connecting. The trust store is a file where a list of trusted certificates are stored. Thie file need to be created or the certificate of your host need to be manually added to the trust store. I believe there is a way to automatically extract this from the url and dynamically add a handler. (I say this because JMeter does not require you to do this step) This is an isssue when you are connecting to servers in your development environment or servers, whoose certificate is not signed by a valid signing authority.

So, to create the truststore, you need a certificate. You can create a certificate by pointing your browser to the host and use your browser to extract the certificate file. Here are the steps to extract the certificate using IE. Point your browser to the site. When the browser shows the padlock, doubleclick the padlock and bringup the popup window. Go to the details tab and select the options to copy the certificate to a file.
Now, use the keytool that is part of the jdk to create a store file.

<%JAVA_HOME%>\bin\keytool -import -trustcacerts -keystore testkeys -storepass passphrase -noprompt -file mycert.cer

In the above command, testkeys is the name of the truststore file to be created/edited, mycert.cer is the name of the certificate file created by the browser.

If you prefer to install the certificate using a java program you can follow the instructions
from Andreas Sterbenz's Blog.

2) Make truststore available to jvm
Once you have created the truststore file, your code to make normal URL connection should work even for the ssl connection. The only change you need to do is add a JVM argument to the command line. java -Djavax.net.ssl.trustStore="<%PATH_TO_TRUST_STORE%>\testkeys"

If you dont want to add the extra parameter you can update the default trust store found under /lib/security/cacerts file using the above procedure. I suggest that you backup the file before you do anything with the file.

3) Make URL connection as usual

Here is a sample snippet for making a url request.



URL url = new URL("https://someurl");
URLConnection con = url.openConnection();
con.setDoOutput(true);
con.connect();
OutputStreamWriter wr = new OutputStreamWriter(con.getOutputStream());
wr.write("Request data....");
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(con.getInputStream()));
String line;
while ((line = rd.readLine()) != null){
System.out.println(line);
}
wr.close();
rd.close();



4) Create host verifier if your host name does not match with certificate

If your certificate issuing name does not match with the hostname you are connecting to, you may run in to another error. Thie can be solved by adding your own hostverifier to your code. Adding the following snippet will accept any host. Use this with caution in a production environment and allow only known mappings.

HostnameVerifier hostnameVerifier = new HostnameVerifier()
{ public boolean verify(String urlHostName, SSLSession session) { System.out.println("Warning: URL Host: " + urlHostName + " v/s " + session.getPeerHost()); return true; } }; HttpsURLConnection.setDefaultHostnameVerifier(hostnameVerifier);

5) Create authorization handler, if page is password protected

The next challenge you may run in to is if the site you are visiting is protected by username/password. You need to add an authenticator. The easiest way to do this is to add a default authenticator.
The following is an example of a simple authenticator:

static class MyAuthenticator extends Authenticator {
String username , password ;
MyAuthenticator(String user, String pwd ) {

super();
username = user ;
password = pwd;
}
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication (username, password.toCharArray());
}
}

Now to activate this authenticator, you need to add the following code to your source.

MyAuthenticator auth = new MyAuthenticator(user, passwd); Authenticator.setDefault( auth);

Hope fully these steps will get you going with making a url connection to a url that is ssl enabled and password protected.