Embedding HSQLDB server instance in Spring

I was using XAMPP happily for development until I had to host it somewhere accessible via the Internet for the client to test and use. I have a VPS that only has 384 RAM, and needing to find a way fast, I decided to install XAMPP into the VPS. Because of the low RAM, when MySQL was running, Tomcat failed to start, even though the initial Java heap size was set to 64m. I managed to host the site temporarily in Jelastic, before moving to OpenShift.

I toyed with the idea of combining the database and application server instances in 1 JVM, to reduce RAM usage (compared to running MySQL + Tomcat). After searching the Internet, I came across several articles on running HSQL server instances together with Tomcat. No doubt I have to update my site to be compatible with HSQL first, but as a POC (proof-of-concept) attempt, I decided to explore the feasibility of running the HSQL server instance in a Spring container.

There are several reasons to run the HSQL server just like a bean in Spring:
1. All-in-one configuration. Everything that is needed to be configured is done in Spring. There are examples in the Net to run the HSQL instance alongside Tomcat, but this requires adding stuff to Tomcat (see links below).
2. Application server independence. 'Theoretically' (in quotes as I successfully tested this in Tomcat only), since everything is done in Spring, there's no or little that needs to be configured in the appserver.

The HSQL server 'bean' is also meant to launch an instance in network mode (not in-process e.g. mem or file). Some reasons for this:
1. 'mem' in-process access is the fastest, but is not persistent. There are other means to initiate a 'mem' data source using Spring's spring-jdbc tags, which is a better approach.
2. 'file' in-process access is persistent, but like 'mem', it can only be accessed within the Java process.
3. Network mode (hsql) is both persistent and accessible using external JDBC client tools. This is useful for troubleshooting and verification.

After reading HSQLDB's documentation, here's the code that does the HSQL server instance bean lifecycle management:

package org.gizmo.hsql.spring;

import java.io.IOException;
import java.util.Properties;

import org.hsqldb.Server;
import org.hsqldb.persist.HsqlProperties;
import org.hsqldb.server.ServerAcl.AclFormatException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.SmartLifecycle;

public class HyperSqlDbServer implements SmartLifecycle
{
 private final Logger logger = LoggerFactory.getLogger(HyperSqlDbServer.class);
 private HsqlProperties properties;
 private Server server;
 private boolean running = false;
 
 public HyperSqlDbServer(Properties props)
 {
  properties = new HsqlProperties(props);
 }
 
 @Override
 public boolean isRunning()
 {
  if(server != null)
   server.checkRunning(running);
  return running;
 }

 @Override
 public void start()
 {
  if(server == null)
  {
   logger.info("Starting HSQL server...");
   server = new Server();
   try
   {
    server.setProperties(properties);
    server.start();
    running = true;
   }
   catch(AclFormatException afe)
   {
    logger.error("Error starting HSQL server.", afe);
   }
   catch (IOException e)
   {
    logger.error("Error starting HSQL server.", e);
   }
  }
 }

 @Override
 public void stop()
 {
  logger.info("Stopping HSQL server...");
  if(server != null)
  {
   server.stop();
   running = false;
  }
 }

 @Override
 public int getPhase()
 {
  return 0;
 }

 @Override
 public boolean isAutoStartup()
 {
  return true;
 }

 @Override
 public void stop(Runnable runnable)
 {
  stop();
  runnable.run();
 }
}

The abridged Spring configuration:

<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xsi:schemalocation="
   http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">

 <bean class="org.gizmo.hsql.spring.HyperSqlDbServer" id="hsqldb" init-method="start">
  <constructor-arg>
   <value>
    server.database.0=file:d:/hsqldb/demobase
    server.dbname.0=demobase
    server.remote_open=true
    hsqldb.reconfig_logging=false
   </value>
  </constructor-arg>
 </bean>
</beans>

Sample output when starting Spring in Tomcat:
[Server@1e893ae]: [Thread[pool-2-thread-1,5,main]]: checkRunning(false) entered
[Server@1e893ae]: [Thread[pool-2-thread-1,5,main]]: checkRunning(false) exited
[Server@1e893ae]: Initiating startup sequence...
[Server@1e893ae]: Server socket opened successfully in 7 ms.
Sep 27, 2012 9:26:23 AM org.hsqldb.persist.Logger logInfoEvent
INFO: checkpointClose start
Sep 27, 2012 9:26:23 AM org.hsqldb.persist.Logger logInfoEvent
INFO: checkpointClose end
[Server@1e893ae]: Database [index=0, id=0, db=file:d:/hsqldb/demo
base, alias=demobase] opened sucessfully in 442 ms.
[Server@1e893ae]: Startup sequence completed in 451 ms.
[Server@1e893ae]: 2012-09-27 09:26:23.395 HSQLDB server 2.2.8 is online on port 9001
[Server@1e893ae]: To close normally, connect and execute SHUTDOWN SQL
[Server@1e893ae]: From command line, use [Ctrl]+[C] to abort abruptly


References:

• http://hsqldb.org/doc/2.0/guide/index.html
• http://dkuntze.wordpress.com/2009/01/28/hsql-on-tomcat/
• http://www.ibm.com/developerworks/data/library/techarticle/dm-0508bader/

Spring 3.1 - Loading Properties For XML Configuration From Database

Spring makes it easy to inject values obtained from properties files via its PropertyPlaceholderConfigurer and (pre-Spring 3.1) PropertySourcesPlaceholderConfigurer (Spring 3.1). These classes implement the BeanFactoryPostProcessor interface, which enables them to manipulate the values within the Spring XML configuration file before the beans are initialized. So if you specify ${jdbc.driverClassName} to be set to the property 'driverClassName', this variable will be replaced/swapped with the value with the key 'jdbc.driverClassName' in a properties file.

Apart from properties files, the database table can also be a place to get key-value pairs. Great, so just extend the PropertySourcesPlaceholderConfigurer, and have it read a table containing the key-value pairs, populate them and we're done!

However, there's a slight problem. If the DataSource bean also relies on values obtained from a properties file (e.g. JDBC URL, username, password), and being good Springers, inject this bean to the bean class extending PropertySourcesPlaceholderConfigurer, the bean container will fail to startup properly, because the 'jdbc.driverClassName' variable cannot be resolved. Strange, but true.

The reason for this is that any bean injected into a BeanFactoryPostProcessor class will trigger bean initialization BEFORE the BeanFactoryPostProcessor classes are run. You know, dependency injection...all depending beans have to be ready before being injected into the consumer. So this creates a cyclic-dependency kind of thing. All dependencies in the XML configuration are resolved first before the BeanFactoryPostProcessor classes are run.

So, how to go about this? Well, there's a trick you can employ. A BeanFactoryPostProcessor class has access to the ConfigurableListableBeanFactory object via the 'postProcessBeanFactory' method. From this object, you can do a 'getBean' and get a reference of any bean with an id. And guess what, you can get the vaunted DataSource bean without triggering premature bean initialization.

Let's say there's a table 'sys_param' with the following data:

 PARAM_CD        PARAM_VALUE  
 --------------  --------------
 service.charge  1.5          
 rebate.amount   15.99        
 smtp.ip         173.194.79.16

The DbPropertySourcesPlaceholderConfigurer is shown here:

package org.gizmo.labs.utils.spring;

import javax.sql.DataSource;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;

public class DbPropertySourcesPlaceholderConfigurer extends PropertySourcesPlaceholderConfigurer
{
 @Override
 public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException
 {
  DataSource dataSource = beanFactory.getBean(DataSource.class);
  DbProperties dbProps = new DbProperties(dataSource);
  
  setProperties(dbProps);
  super.postProcessBeanFactory(beanFactory);
 }
}

The DbProperties class will make use of the DataSource reference and queries the database to get the key-value pairs:

package org.gizmo.labs.utils.spring;

import java.util.List;
import java.util.Map;
import java.util.Properties;

import javax.sql.DataSource;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.core.JdbcTemplate;

public class DbProperties extends Properties
{
 private final Logger logger = LoggerFactory.getLogger(DbProperties.class);
 private static final long serialVersionUID = 1L;

 public DbProperties(DataSource dataSource)
 {
  super();
  JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); 
  List

To demonstrate that the values from the table are properly injected, here's the class which acts as the consumer:

package org.gizmo.labs.utils.spring;

import java.math.BigDecimal;

import org.apache.commons.lang.builder.ReflectionToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.InitializingBean;

public class DbPropConsumer implements InitializingBean
{
 private final Logger logger = LoggerFactory.getLogger(DbPropConsumer.class);

 private BigDecimal serviceCharge;
 private double rebateAmount;
 private String smtpIp;
 
 @Override
 public void afterPropertiesSet() throws Exception
 {
  logger.debug("I have consumed: {}", this);
 }

 public String toString()
 {
  return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
 } 
 
 public BigDecimal getServiceCharge() {
  return serviceCharge;
 }

 public void setServiceCharge(BigDecimal serviceCharge) {
  this.serviceCharge = serviceCharge;
 }

 public double getRebateAmount() {
  return rebateAmount;
 }

 public void setRebateAmount(double rebateAmount) {
  this.rebateAmount = rebateAmount;
 }

 public String getSmtpIp() {
  return smtpIp;
 }

 public void setSmtpIp(String smtpIp) {
  this.smtpIp = smtpIp;
 }

}

Last but not least, the Spring configuration (DataSource bean not shown, simplified for clarity):

<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xsi:schemalocation="
     http://www.springframework.org/schema/beans 
     http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">

 <bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer">
     <property name="order" value="1">
     <property name="locations">
      <list>
       <value>classpath:system.properties</value>
      </list>
     </property>
 </property></bean>

 <bean class="org.gizmo.labs.utils.spring.DbPropertySourcesPlaceholderConfigurer">
  <property name="order" value="2">
     <property name="placeholderPrefix" value="%{">
     <property name="placeholderSuffix" value="}">
 </property></property></property></bean>

 <bean class="org.gizmo.labs.utils.spring.DbPropConsumer" lazy-init="false">
  <property name="serviceCharge" value="%{service.charge}">
  <property name="rebateAmount" value="%{rebate.amount}">
  <property name="smtpIp" value="%{smtp.ip}">
 </property></property></property></bean>
</beans>

The first 2 bean definitions are the BeanFactoryPostProcessor classes, and to ensure the first one is run first, the 'order' property is set (lower means higher precedence).

For the DbPropertySourcesPlaceholderConfigurer, a different placeholder prefix and suffix is used for clarity (notice the placeholders for DbPropConsumer).

So, upon Spring container startup, you should be able to view a similar output as below:
2012-09-18 00:03:14, DEBUG, org.gizmo.labs.utils.spring.DbProperties, Loading from DB: [service.charge:1.5]
2012-09-18 00:03:14, DEBUG, org.gizmo.labs.utils.spring.DbProperties, Loading from DB: [rebate.amount:15.99]
2012-09-18 00:03:14, DEBUG, org.gizmo.labs.utils.spring.DbProperties, Loading from DB: [smtp.ip:173.194.79.16]

2012-09-18 00:03:14, DEBUG, org.gizmo.labs.utils.spring.DbPropConsumer, I have consumed: org.gizmo.labs.utils.spring.DbPropConsumer@189b939[
  logger=Logger[org.gizmo.labs.utils.spring.DbPropConsumer]
  serviceCharge=1.5
  rebateAmount=15.99
  smtpIp=173.194.79.16
]

Spring 3.1 Bean Profiles

One of the most exciting new features (to me) introduced in Spring 3.1 is the bean definition profiles (http://blog.springsource.com/2011/02/11/spring-framework-3-1-m1-released/). Bean definition profiles are simply bean configurations that are 'activated' based on an the existence of an indicator or marker.

A lot of examples of bean definition profiles uses the JDBC DataSource bean to illustrate its usefulness. Usually for unit testing, a DriverManagerDataSource class is sufficient, as shown below:

<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
 <property name="driverClassName" value="${jdbc.driverClassName}">
 <property name="url" value="${jdbc.url}">
 <property name="username" value="${jdbc.username}">
 <property name="password" value="${jdbc.password}">
</property></property></property></property></bean>

In order to deploy the application (typically a webapp) for active use or production, the DataSource bean is usually configured to get from a JNDI lookup, as shown:

The JNDI lookup way also requires configuring the DataSource connection pool at the application server.

Before Spring 3.1, in order to make one of the bean definitions 'active', the desired bean definition will need to be loaded last, to override the earlier appearing bean definition e.g. the 'dataSource' bean using DriverManagerDataSource will need to be in a Spring XML file that is loaded last. One of the ways to achieve this is to specify the XML file as the last file in a comma-separated value of the context-param 'contextConfigLocation' in web.xml. An Ant task may be invoked to automate a 'deletion' of this file entry so that when deployed to production, the 'dataSource' bean using JNDI lookup will be the 'active' bean. Or, just comment out the undesired bean :). It was clumsy and led to different builds for different environments.

So, how does Spring 3.1 solve this problem? Spring 3.1 allows the incorporation of the 2 different bean definitions with the same id in the same XML file, by having nested 'beans' XML elements, as shown (simplified for clarify):

<beans xmlns="http://www.springframework.org/schema/beans" xsi:schemalocation="
     http://www.springframework.org/schema/beans 
     http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">

 <!-- Generic bean definitions ABOVE -->
  
 <beans profile="openshift,default">
  <bean class="org.springframework.jndi.JndiObjectFactoryBean" id="dataSource">
         <property name="jndiName" value="${jndi.datasource}">
     </property></bean>
 </beans>
 
 <beans profile="testing">
     <bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
         <property name="driverClassName" value="${jdbc.driverClassName}">
         <property name="url" value="${jdbc.url}">
         <property name="username" value="${jdbc.username}">
         <property name="password" value="${jdbc.password}">
     </property></property></property></property></bean>
 </beans>
</beans>

A few things to note:
- The nested beans XML elements must be placed at the end of the configuration XML file
- Ensure that the Spring 3.1 Beans XSD file is used (earlier versions won't work).
- It is recommended to have one bean profile to be the default profile.

To 'activate' a bean profile, one of the ways is to pass in a system property 'spring.profiles.active'. For my testing environment, I use Tomcat, so I placed this property in setEnv.bat (not in Tomcat by default, just create it under ${TOMCAT_HOME}/bin folder, and the file will be processed by Tomcat when starting up). So for testing, the value to pass is: -Dspring.profiles.active=testing

If the 'spring.profiles.active' value is not specified, then the default profile will be the active profile.

To check or test whether which profile is active, you can get the Environment object from an ApplicationContext, as shown:

Environment env = ctx.getEnvironment();
String[] activeProfiles = env.getActiveProfiles();

In my example above, it seems that only 1 profile can be active at a time, but the Spring API allows activation of multiple profiles. Just comma-separate the profiles when specifying the value of 'spring.profiles.active'. But proceed with caution as too many profiles may lead to confusion.

Lazy-initializing beans in Spring: Exceptions

Lazy-initializing beans in a Spring container is generally desirable to allow for faster startup times, especially if there are lots of beans involved. However, there are some beans that must be loaded up or initialized when the application starts e.g. a background Quartz scheduler running jobs, or just doing 1st-level diagnostics or health check that writes results to a log file.

By default, all beans with scope="singleton" are initialized during container startup, but this behaviour can be changed by adding default-lazy-init="true" at the 'beans' top-level element, as shown (simplified for clarity):

To ensure a Spring-managed bean is initialized during container startup regardless of the attribute default-lazy-init="true" at the 'beans' top-level element in Spring configuration XML (or in Java code), you can one of the following:

• Configure your bean definition to force initialization by adding attribute lazy-init="false", as shown in the example below:

• Have your class implement the interface org.springframework.context.SmartLifecycle. This will guarantee the class will be initialized as long as isAutoStartup() is implemented to return 'true'. See the class Javadoc (http://static.springsource.org/spring/docs/3.0.x/javadoc-api/org/springframework/context/SmartLifecycle.html) for more info.

XMLBeans - First Steps

I attempted to revisit XMLBeans to generate XML-Java mapping codes based on the IFX schema. I've tried JAXB v1 but failed miserably (there might be some tweaking needed but I've searched high and low, to no avail...few years ago) The multi-XSD file structure from IFX Forum website was used.

In the XSD folder (all files are from the IFX Forum):
$pain.001.001.01.xsd
$pain.002.001.01.xsd
$pain.004.001.01.xsd
IFX170_XSD.xsd
RemitDetail_Type.xsd

Using XMLBeans 2.5.0 distribution, the command I run:
E:\experiments\exploded-apis\xmlbeans-2.5.0\bin>scomp -mx 1024M -out ifx17.jar E:\yk\xpdesktop\IFX-SIT\IFX1.7_XSD\IFX170_XSD.xsd

IFX is a HUGE schema so the heap size is increased to 1024M in order for the command to complete successfully.

Results:
Time to build schema type system: 3.39 seconds
Time to generate code: 109.938 seconds
Time to compile code: 145.939 seconds
Compiled types to: ifx17.jar

Wokay, this is good. First try without fancy options, and a JAR file is produced. I couldn't achieve the same thing in JAXB v1. Will need to try out using JAXB v2. More on this later.