Month: February 2009

Frozen

February 18, 2009

Imagine what would happen to the earth, if the sun was suddenly shut down for like 10 minutes (by some cosmic force). I guess the earth would be wrapped up in ice within instants, and most forms of life would be extinct within minutes. (Now some intellectually and morally bankrupt Hollywood filmmaker will steal this original imagination of mine from my blog and make a science fiction movie out of it, without any due to me. But thats not what I am worried about right now.)

We normally take the sun for granted.

I did a little research. If we are able to trap all the solar energy falling onto the surface of earth for 3 minutes, it will solve all energy needs of the world for the next 25 years.

If the above is true, then how do you justify the $600 billion and 4000 American lives (Iraqis don’t have a life. So lets not count theirs) lost during the Oil-War in Iraq? I wonder how many barrels of oil on average is a soldier that dies equivalent to. And I wonder how many years of solar energy (or any other alternative energy for that matter) is that $600 billion equivalent to.

Transaction & Batch-processing in Grails

February 1, 2009

Transaction handling is one of the more complex areas of web development. Anytime a user takes any action in the interface which demands a couple of database actions in the backend, then usually you end up having do it as transaction. For a user, everything is either a success or failure. Partial success may be either harmful to the system or doesn’t mean anything to the user. Grails, since it is built on top of Spring and Hibernate, uses their underlying mechanism to deal with transactions. While it may seem confusing in the beginning, Grails actually makes it even more easier.

Lets see it in code.

Suppose we are designing an Offers System, like the one that your bank sends notifying you of some offers and promotions on services. A simplified object structure may be like the following:

class Offer {
     String title
     String content
     Date expirationDate
     Date createdDate = new Date()

     static hasMany = [recipients: Recipient]
}
class Recipient {
     String email

     static belongsTo = [offer: Offer]
}

The relationship is fairly simple. Each Offer can have many Recipients, each Recipient belongs to an Offer.

belongsTo just means the offer Recipient will be deleted whenever the Offer is deleted, which makes sense coz we don’t want to keep the junk if the Offer itself is deleted.

hasMany lets you access to the Recipients from the Offer object. In other words, if you have the Offer object, you can do offer.recipients and get the list of all the recipients for the Offer. Cool.

Now, here is how it will work. We want to add an Offer and some Recipients. Everything should either succeed or fail. Even if only one Recipient out of thousands fail, everything should fail.

There are two ways to do transactions in Grails:

1. Method-level Transaction: By default, each Service class in grails has a transactional property set to true. So if any public method in a Service class throws a RuntimeException or any Error, the transaction will be rolled back. If you do not need this default transaction property, you can explicitly set transactional to false.

In the code below, the OfferService throws a Runtime Exception anytime it isn’t able to add the Fffer or the Recipient, and the transaction will be rolled back.
You wouldn’t get the same transactional behavior if the method throws any Checked Exception or if the transactional property is set to false.

class OfferService {

     boolean transactional = true

     def save(offer, recipients) {
         if(!offer.validate()) {
             throw new RuntimeException("Invalid offer.")
         } else {
             def result = offer.save()
             recipients.each {
            	if(!it.validate()) {
            		throw new RuntimeException("Invalid recipient")
                } else {
                  	offer.addToRecipients(it)
                }
             }
         }
     }
}

You can then catch the exception in the Controller and render a more user-friendly error message.

class OfferController {
     def save = {
    	 def offer = getOffer(params)
    	 def recipients = getRecipientList(params)
   	 try {
   		offerService.save(offer, recipients)
		flash.message = "Successfully added the offer."
   	 } catch (Exception e) {
   		log.error "Exception: " + e
 		flash.message = "An error occured, and the offer was not saved."
   	 }
     }

}

So all you need to do is make sure your Service is transactional, and that it throws a RuntimeException and not a Java CheckedException.

2. Block-level Transaction: The first approach treats everything inside the method as a single transaction. But sometimes your need more control than that. You might want to perform some actions within a transaction and do other operations regardless, or within another block of transaction within a single method.

Each domain class in grails has a static withTransaction method that takes a closure as an argument. Lets see the same thing in code using withTransaction:

class OfferService {

    boolean transactional = false

    def save(offer, recipients) {

    	Offer.withTransaction { status ->
             if(!offer.validate()) {
                  status.setRollbackOnly()
             } else {
         	  def result = offer.save()
                  recipients.each {
           	       if(!it.validate()) {
          	          status.setRollbackOnly()
                       } else {
                  	  offer.addToRecipients(it)
                       }
                  }
              }
         }

    }
}

The status object that we are calling the setRollbackOnly on is actually an instance of org.springframework.transaction.TransactionStatus class. The funny thing is it doesn’t really matter which Domain object you call the withTransaction method on. It just to mark the start and end of a transaction.

As you can guess, with this approach, you don’t need to be in a Service class to use a transaction. You can use it from inside a controller too.

In both of the process above, grails isn’t actually doing a database rollback. It is just saving the changes into the Hibernate session until the transaction is successful, at which point it commits the change to the database. So it is holding onto a lot of objects in session cache.

I have been fighting a situation where I have to batch insert a thousand of objects within a transaction. My common sense tells me I need something like nested transaction, where I could divide a large parent transaction into smaller chunks of transactions, and if one of them fails, it would trigger the parent to fail. But looking at the way Hibernate session works, I was worried that I might have to stick to plain JDBC to do it. But not anymore, coz I found this awesome doc on Hibernate Batch-processing.

Batch Processing: Each action in a Grails Controller is executed within a Hibernate Session. The session is started right before the action starts and is closed once it returns. That explains the reason for the memory issues because all the objects are being cached to the Hibernate session. Hibernate saves it to the database when the session is flushed, thus freeing the memory. The same code as above with the session cleared more frequently:

class OfferService {

    boolean transactional = false
    def sessionFactory

    def save(offer, recipients) {

    	Offer.withTransaction { status ->;
             if(!offer.validate()) {
                  status.setRollbackOnly()
             } else {
         	  def result = offer.save()
                  int i = 0
                  recipients.each {
           	       if(!it.validate()) {
          	          status.setRollbackOnly()
                       } else {
                          i++
                  	  offer.addToRecipients(it)
                          if(i % 100 == 0) {
                               sessionFactory.getCurrentSession().clear();
                          }
                       }
                  }
              }
         }

    }
}

Note: You can get the sessionFactory in a Controller similarly.
This way the Hibernate session, which is the first-level cache, is being cleared every once in a while.
This worked like a charm for me, solving all the memory issues I was having, and also speeding things up significantly.