I just came back from JavaZone 2011, and I have some thoughts.
All in all it was a nice conference. The first day was a little booring. Few really nice talks. However the last day had some really nice talks.
The talks I went to that are worth mentioning are:
CoffeScript
I am not sure what to make of this relatively new scripting language. At first during the presentation, I was very sceptical. The idea is basically to remove/hide out a lot of boiler plate JavaScript code and replace it by magic that adds this at runtime later. It also adds a lot of features on top of JavaScript which made it very cool. That's why I am still on the fence on this one. Can't really decide. However, I don't write that much JavaScript, and thankfully with the help of JQuery, I don't need to. So I don't think I will be looking into this until I really feel that I write a lot of tedious JavaScript.
JavaPosse
Always great! Nuff said!
"Men så hør du da bruker det feil" (dust)
This was a really nice talk from Kåre Nilsen, which basically warned us about Java Frameworks and library hell. He said that we should all think before we add or use a framework, and have a valid reason as to why we use it. We also should know what each library that is bundled with the framework is doing, and we should ask our self if we really need it. Maybe we just can take out the part of the code in the library we are using, and just paste that in our projects.
Play! Framework: to infinity and beyond
I had heard and tested Play! once before this talk, and I found the framework so interesting that I wanted to learn more. And boy am I impressed. This is a framework that you definetly will hear more about in the future. But they did announce that in Play 2, they might break java compatibility and only work with Scala. I don't know what I feel about that, because that kinda forces you to learn "two" things at once. Both Scala and Play!
However, instead of me explaining what Play! is, go check it out!
DISCLAIMER: If your talk is not in this list, its probably just because I didn't attend it. It doesn't mean your session was bad or uninteresting.
This blog is mainly about Java...
Thursday, September 8, 2011
Monday, March 21, 2011
API helper / wrapper for ProcessFinder in Hyperic Sigar PTQL
In my previous blog post I wrote that I was going to implement Hyperic Sigar in JODConverter. Well the alpha is done, and you can go try it out here.
When rewriting the code to use the ProcessFinder in Sigar, I found it very cumbersome to work with PTQL, which is the Process Table Query Language for Sigar.
Sigar describes PTQL as:
Hyperic SIGAR provides a mechanism to identify processes called Process Table Query Language. All operating systems assign a unique id (PID) to each running process. However, the PID is a random number that may also change at any point in time when a process is restarted. PTQL uses process attributes that will persist over time to identify a process.
PTQL is string based, and the processFinder takes the ptql as string as parameter.
I have jboss already running and the ps command locally gives me the pid: 28258
It gives the output (the numbers are arguments)
Now imagine you want to create a more advanced query which takes the one or more of the 19 arguments the java process takes.
The query would look something like this:
I took the liberty to create a helper/wrapper class for ProcessFinder which tries to eliminate some of the errors you can do.
For instance, instead of the code above, you would write the following:
The latest version of SimplePTQL can be found here
And the source (subject to change, see the former link for latest version) is:
At the time of writing, the SimplePTQL class does not support Env and Modules
The unit test for this class can also be found here
Or the source (at the time of writing)
When rewriting the code to use the ProcessFinder in Sigar, I found it very cumbersome to work with PTQL, which is the Process Table Query Language for Sigar.
Sigar describes PTQL as:
Hyperic SIGAR provides a mechanism to identify processes called Process Table Query Language. All operating systems assign a unique id (PID) to each running process. However, the PID is a random number that may also change at any point in time when a process is restarted. PTQL uses process attributes that will persist over time to identify a process.
PTQL is string based, and the processFinder takes the ptql as string as parameter.
processFinder.find("State.Name.eq=java"); //Returns all the process id's named java
I have jboss already running and the ps command locally gives me the pid: 28258
> ps -ef | grep jboss
> shervin 28258 27055 20 13:53 pts/1 00:02:36 /usr/lib/jvm/java-6-sun-1.6.0.24/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:54988 -javaagent:/home/shervin/opt/jrebel/jrebel.jar -noverify -Dprogram.name=JBossTools: JBoss EAP 5.0 Runtime -Xms512m -Xmx768m -XX:MaxPermSize=512m -Djava.net.preferIPv4Stack=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.endorsed.dirs=/home/shervin/opt/jboss/lib/endorsed -Djava.library.path=/usr/lib/jvm/java-6-sun/jre/lib/i386/server:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386:/usr/lib/jvm/jdk1.5.0_22/jre/../lib/i386:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386/client:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386:/usr/lib/xulrunner-addons:/usr/lib/xulrunner-addons:/usr/lib/ure/lib/:/home/shervin/lib/lib-src/hyperic-sigar-1.6.4-src/bindings/java/sigar-bin/lib -Dfile.encoding=UTF-8 -classpath /home/shervin/opt/jboss/bin/run.jar org.jboss.Main --configuration=default -b localhost
When running the sigar shell (java -jar sigar.jar) and writing sigar> pargs 28258
It gives the output (the numbers are arguments)
pid=28258
exe=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/bin/java
cwd=/home/shervin/opt/jboss-eap-5.1/jboss-as/bin
0=>/usr/lib/jvm/java-6-sun-1.6.0.24/bin/java<=
1=>-agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:54988<=
2=>-javaagent:/home/shervin/opt/jrebel/jrebel.jar<=
3=>-noverify<=
4=>-Dprogram.name=JBossTools: JBoss EAP 5.0 Runtime<=
5=>-Xms512m<=
6=>-Xmx768m<=
7=>-XX:MaxPermSize=512m<=
8=>-Djava.net.preferIPv4Stack=true<=
9=>-Dsun.rmi.dgc.client.gcInterval=3600000<=
10=>-Dsun.rmi.dgc.server.gcInterval=3600000<=
11=>-Djava.endorsed.dirs=/home/shervin/opt/jboss/lib/endorsed<=
12=>-Djava.library.path=/usr/lib/jvm/java-6-sun/jre/lib/i386/server:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386:/usr/lib/jvm/jdk1.5.0_22/jre/../lib/i386:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386/client:/usr/lib/jvm/jdk1.5.0_22/jre/lib/i386:/usr/lib/xulrunner-addons:/usr/lib/xulrunner-addons:/usr/lib/ure/lib/:/home/shervin/lib/lib-src/hyperic-sigar-1.6.4-src/bindings/java/sigar-bin/lib<=
13=>-Dfile.encoding=UTF-8<=
14=>-classpath<=
15=>/home/shervin/opt/jboss/bin/run.jar<=
16=>org.jboss.Main<=
17=>--configuration=default<=
18=>-b<=
19=>localhost<=
Now imagine you want to create a more advanced query which takes the one or more of the 19 arguments the java process takes.
The query would look something like this:
processFinder.find("State.Name.ct=java,Args.5.eq=-Xms512m,Args.6.eq=-Xmx768m,Args.7.re=.*MaxPermSize=512m,Args.17.ct=configuration=default,Args.19.eq=localhost");
or you are building the query based on variables which you must retrieve somewhere elsepublic void testArgs() throws Exception {
String state_name= "State.Name.ct=java";
String arg5 = ",Args.5.eq=-Xms512m";
String arg6 = ",Args.6.eq=-Xmx768m";
String arg7 = ",Args.7.re=.*MaxPermSize=512m";
String arg17 = ",Args.17.ct=configuration=default";
String arg19 = ",Args.19.eq=localhost";
ProcessFinder processFinder = new ProcessFinder(new Sigar());
processFinder.find(state_name + arg5 + arg6+ arg7 + arg17 + arg19);
}
It doesn't take a genius to see that it is easy to make mistakes. Hence the more type-safe way of creating PTQL queries.SimplePTQL
I took the liberty to create a helper/wrapper class for ProcessFinder which tries to eliminate some of the errors you can do.
For instance, instead of the code above, you would write the following:
SimplePTQL ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.CT(), "java")
.addArgs(5, SimplePTQL.EQ(), "-Xms512m", Strategy.NOT_ESCAPE)
.addArgs(6, SimplePTQL.EQ(), "-Xmx768m", Strategy.NOT_ESCAPE)
.addArgs(7, SimplePTQL.RE(), ".*MaxPermSize=512m", Strategy.ESCAPE)
.addArgs(17, SimplePTQL.CT(), "configuration=default", Strategy.NOT_ESCAPE)
.addArgs(19, SimplePTQL.EQ(), "localhost", Strategy.NOT_ESCAPE)
.createQuery();
ProcessFinder processFinder = new ProcessFinder(new Sigar());
processFinder.find(ptql.getQuery());
The constructor of the Builder takes the first part of the query as parameters and you can optionally add arguments. You can escape the input which regular expressions should do. It will basically replace all commas (,) with period (.), this because the find method doesn't like commas in the searchValue inside a regular expression./**
* This method will escape Comma (,) to Period (.)
* Because the PTQL cannot escape those correctly, so we will use '.' in regular expression.
* We also have to remove \Q and \E because they are not correctly interpreted as literal characters
*
* @param s - The string you want to espace
*
* NB: Note that you should only espace the value of the PTQL, not the query it self
* ie: State.Name.ct=pipe,name=office1 should be converted to State.Name.ct=pipe.name.office1
* @return - The escaped string
*/
public static String escapePTQLForRegex(String s) {
return s.replaceAll(",", ".").replaceAll("\\\\Q|\\\\E", "");
}
The latest version of SimplePTQL can be found here
And the source (subject to change, see the former link for latest version) is:
At the time of writing, the SimplePTQL class does not support Env and Modules
//
// JODConverter - Java OpenDocument Converter
// Copyright 2011 Art of Solving Ltd
// Copyright 2004-2011 Mirko Nasato
//
// JODConverter is free software: you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public License
// as published by the Free Software Foundation, either version 3 of
// the License, or (at your option) any later version.
//
// JODConverter is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General
// Public License along with JODConverter. If not, see
// <http://www.gnu.org/licenses/>.
//
package org.artofsolving.jodconverter.sigar;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.artofsolving.jodconverter.util.PlatformUtils;
import com.google.common.base.Preconditions;
/**
* A simplified PTQL so to simplify the query building and make it less error prone and typesafe.
*
* See more information http://support.hyperic.com/display/SIGAR/PTQL
*
* @author Shervin Asgari - <a href="mailto:shervin@asgari.no">shervin@asgari.no</a>
*/
public class SimplePTQL {
private final Operator operator;
private final Attribute attribute;
private final String searchValue;
private final String args;
private final Strategy strategy;
public enum Strategy {
ESCAPE, NOT_ESCAPE
}
private SimplePTQL(Attribute attribute, Operator operator, String searchValue, String args, Strategy strategy) {
this.attribute = attribute;
this.operator = operator;
this.searchValue = searchValue;
this.args = args;
this.strategy = strategy;
}
public static class Builder {
private final Operator operator;
private final Attribute attribute;
private final String searchValue;
private Strategy strategy = Strategy.NOT_ESCAPE;
private StringBuilder args = new StringBuilder(""); //Default blank
public Builder(Attribute attribute, Operator operator, String searchValue) {
this.attribute = attribute;
this.operator = operator;
this.searchValue = searchValue;
}
public Builder addArgs(int argument, Operator operator, String searchValue, Strategy strategy) {
Preconditions.checkNotNull(searchValue);
if(strategy == Strategy.ESCAPE) {
args.append(",Args." + String.valueOf(argument) + "." + operator.toString() + PlatformUtils.escapePTQLForRegex(searchValue));
} else {
Pattern pattern = Pattern.compile(",|=");
Matcher matcher = pattern.matcher(searchValue);
Preconditions.checkArgument(!matcher.find(), "searchValue cannot contain comma or equals sign. Either set Strategy.ESCAPE or remove it from the search value");
args.append(",Args." + String.valueOf(argument) + "." + operator.toString() + searchValue);
}
return this;
}
public Builder setStrategy(Strategy strategy) {
this.strategy = strategy;
return this;
}
public SimplePTQL createQuery() {
return new SimplePTQL(attribute, operator, searchValue, args.toString(), strategy);
}
}
/**
* Returns the entiry PTQL query.
* ie:
* <ul>
* <li>State.Name.eq=java</li>
* <li>Pid.Pid.eq=4245</li>
* <li>State.Name.re=^(https?d.*|[Aa]pache2?)$</li>
* </ul>
* @return
*/
public String getQuery() {
return attribute.toString() + operator.toString() + (strategy == Strategy.ESCAPE ? PlatformUtils.escapePTQLForRegex(searchValue): searchValue) + args;
}
private interface Operator {
String toString();
}
private interface Attribute {
String toString();
}
/**
* Equal to value
*/
public static Operator EQ() {
return new Operator() {
@Override
public String toString() {
return "eq=";
}
};
}
/**
* Not Equal to value
*/
public static Operator NE() {
return new Operator() {
@Override
public String toString() {
return "ne";
}
};
}
/**
* Ends with value
*/
public static Operator EW() {
return new Operator() {
@Override
public String toString() {
return "ew=";
}
};
}
/**
* Starts with value
*/
public static Operator SW() {
return new Operator() {
@Override
public String toString() {
return "sw=";
}
};
}
/**
* Contains value (substring)
*/
public static Operator CT() {
return new Operator() {
@Override
public String toString() {
return "ct=";
}
};
}
/**
* Regular expression value matches
*/
public static Operator RE() {
return new Operator() {
@Override
public String toString() {
return "re=";
}
};
}
/**
* <i>Only for numeric value<i> Greater than value
*/
public static Operator GT() {
return new Operator() {
@Override
public String toString() {
return "gt=";
}
};
}
/**
* <i>Only for numeric value<i> Greater than or equal value
*/
public static Operator GE() {
return new Operator() {
@Override
public String toString() {
return "ge=";
}
};
}
/**
* <i>Only for numeric value<i> Less than value
*/
public static Operator LT() {
return new Operator() {
@Override
public String toString() {
return "lt=";
}
};
}
/**
* <i>Only for numeric value<i> Less than value or equal value
*/
public static Operator LE() {
return new Operator() {
@Override
public String toString() {
return "le=";
}
};
}
/**
* The Process ID
*/
public static Attribute PID_PID() {
return new Attribute() {
@Override
public String toString() {
return "Pid.Pid.";
}
};
}
/**
* File containing the process ID
*/
public static Attribute PID_PIDFILE() {
return new Attribute() {
@Override
public String toString() {
return "Pid.PidFile.";
}
};
}
/**
* Windows Service name used to pid from the service manager
*/
public static Attribute PID_SERVICE() {
return new Attribute() {
@Override
public String toString() {
return "Pid.Service.";
}
};
}
/**
* Base name of the process executable
*/
public static Attribute STATE_NAME() {
return new Attribute() {
@Override
public String toString() {
return "State.Name.";
}
};
}
/**
* User Name of the process owner
*/
public static Attribute CREDNAME_USER() {
return new Attribute() {
@Override
public String toString() {
return "CredName.User.";
}
};
}
/**
* Group Name of the process owner
*/
public static Attribute CREDNAME_GROUP() {
return new Attribute() {
@Override
public String toString() {
return "CredName.Group.";
}
};
}
/**
* User ID of the process owner
*/
public static Attribute CRED_UID() {
return new Attribute() {
@Override
public String toString() {
return "Cred.Uid.";
}
};
}
/**
* Group ID of the process owner
*/
public static Attribute CRED_GID() {
return new Attribute() {
@Override
public String toString() {
return "Cred.Gid.";
}
};
}
/**
* Effective User ID of the process owner
*/
public static Attribute CRED_EUID() {
return new Attribute() {
@Override
public String toString() {
return "Cred.Euid.";
}
};
}
/**
* Effective Group ID of the process owner
*/
public static Attribute CRED_EGID() {
return new Attribute() {
@Override
public String toString() {
return "Cred.Egid.";
}
};
}
/**
* Full path name of the process executable
*/
public static Attribute EXE_NAME() {
return new Attribute() {
@Override
public String toString() {
return "Exe.Name.";
}
};
}
/**
* Current Working Directory of the process
*/
public static Attribute EXE_CWD() {
return new Attribute() {
@Override
public String toString() {
return "Exe.Cwd.";
}
};
}
}
The unit test for this class can also be found here
Or the source (at the time of writing)
//
// JODConverter - Java OpenDocument Converter
// Copyright 2011 Art of Solving Ltd
// Copyright 2004-2011 Mirko Nasato
//
// JODConverter is free software: you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public License
// as published by the Free Software Foundation, either version 3 of
// the License, or (at your option) any later version.
//
// JODConverter is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General
// Public License along with JODConverter. If not, see
// <http://www.gnu.org/licenses/>.
//
package org.artofsolving.jodconverter.sigar;
import java.util.List;
import org.artofsolving.jodconverter.process.ProcessManager;
import org.artofsolving.jodconverter.process.SigarProcessManager;
import org.artofsolving.jodconverter.sigar.SimplePTQL.Strategy;
import org.artofsolving.jodconverter.util.PlatformUtils;
import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.Test;
@Test
public class SimplePTQLTest {
public void simpleQuery() throws Exception {
ProcessManager spm = new SigarProcessManager();
SimplePTQL ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.EQ(), "java").createQuery();
Assert.assertEquals(ptql.getQuery(), "State.Name.eq=java");
List<Long> find = spm.find(ptql);
Assert.assertTrue(find.size() > 0);
if(find.size() > 1) {
try {
spm.findSingle(ptql);
Assert.fail("Should not reach here, should get exception");
} catch(NonUniqueResultException nre) {
//More than one results where found
}
}
ptql = new SimplePTQL.Builder(SimplePTQL.PID_PID(), SimplePTQL.EQ(), String.valueOf(find.get(0))).createQuery();
Long findSingle = spm.findSingle(ptql);
Assert.assertEquals(findSingle, find.get(0));
ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.EQ(), "Hopefully There is no Process called this").createQuery();
List<Long> find2 = spm.find(ptql);
Assert.assertTrue(find2.size() == 0);
ptql = new SimplePTQL.Builder(SimplePTQL.PID_PID(), SimplePTQL.GT(), "1").createQuery();
List<Long> find3 = spm.find(ptql);
Assert.assertTrue(find3.size() > 1);
}
public void args() throws Exception {
SimplePTQL ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.RE(), "office.*")
.addArgs(1, SimplePTQL.RE(), "\\Qpipe,name,office1\\E", Strategy.ESCAPE)
.createQuery();
Assert.assertEquals(ptql.getQuery(), "State.Name.re=office.*,Args.1.re=pipe.name.office1");
ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.EQ(), "office.*")
.addArgs(1, SimplePTQL.RE(), "\\Qpipe,name,office1\\E", Strategy.ESCAPE)
.addArgs(2, SimplePTQL.EQ(), "\\Qpipe,name=office2\\E", Strategy.ESCAPE)
.setStrategy(Strategy.ESCAPE)
.createQuery();
Assert.assertEquals(ptql.getQuery(), "State.Name.eq=office.*,Args.1.re=pipe.name.office1,Args.2.eq=pipe.name=office2");
try {
ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.EQ(), "office.*")
.addArgs(1, SimplePTQL.RE(), "\\Qpipe,name,office1\\E", Strategy.ESCAPE)
.addArgs(2, SimplePTQL.EQ(), "\\Qpipe,name,office2\\E", Strategy.NOT_ESCAPE)
.createQuery();
Assert.fail("Method should have thrown IllegalArgumentException");
} catch(IllegalArgumentException ex) {}
}
public void realArguments() throws Exception {
if (PlatformUtils.isWindows()) {
throw new SkipException("Sleep only works on unix");
}
Process process = new ProcessBuilder("sleep", "60s").start();
Assert.assertNotNull(process);
SimplePTQL ptql = new SimplePTQL.Builder(SimplePTQL.STATE_NAME(), SimplePTQL.EQ(), "sleep")
.addArgs(1, SimplePTQL.EQ(), "60s", Strategy.NOT_ESCAPE).createQuery();
ProcessManager spm = new SigarProcessManager();
Long findSingle = spm.findSingle(ptql);
Assert.assertTrue(findSingle > 0L);
spm.kill(findSingle, 9);
Assert.assertEquals(spm.findSingle(ptql).longValue(), 0L);
}
}
Labels:
jodconverter 3,
ptql,
sigar
Testing Prettify
This is a test of Prettify
public class NonUniqueResultException extends Exception {
private static final long serialVersionUID = -7862607833306958651L;
public NonUniqueResultException(String string) {
super(string);
}
}
Labels:
prettify
Monday, March 14, 2011
Commit access to JodConverter
I have gotten commit access to jodconverter 3 so that I can work on some issues.
The first issue I am going to work on is refactoring the code to use Sigar.
Sigar "provides a portable interface for gathering system information" with native libraries for many OSes and it has an Apache License Version 2.
This is my first commit access to an open source project. Looking forward to contribute.
The first issue I am going to work on is refactoring the code to use Sigar.
Sigar "provides a portable interface for gathering system information" with native libraries for many OSes and it has an Apache License Version 2.
This is my first commit access to an open source project. Looking forward to contribute.
Labels:
jodconverter,
jodconverter 3,
OSS,
sigar
Monday, March 7, 2011
Creating an efficient memory based cache
If you find your self in the situation where you are creating global maps as cache, then you have to stop and rethink.
Global maps are prone to memory leaks
You should instead consider using soft reference and WeakHashMap or what I prefer, the MapMaker of Google Guava.
In this blog post, I will describe an efficient way of creating a map based cache. This can can either be stored in the session, application, or your existing cache.
Weak References
- What are weak references?
Weak reference basically means that the garbage collector can come and remove it when it is no longer in use. You have no guarantee that whatever you put in the map, will actually be around when you try to get it.
The reason why that is so useful is if you don't want to (or cannot afford to) retain an object indefinitely in memory.
Consider the following use case: You need to associate information with classes. Now, since you are running in an environment, where classes might get reloaded (say, a Tomcat, or OSGi environment), you want the garbage collector to be able to reclaim old versions of a class as soon as it deems safe to do so.
An initial attempt to implement this, might look like something like this:
The problem here is; this would keep all classes in the cache member forever (or at least, unless they are manually removed), forcing the garbage collector to retain them indefinitely, including everything referenced from the class (static member values, class loader information, etc).
By using weak references, the garbage collector can reclaim old version of the class as soon as no other references to it (usually instances) exist.
On the other hand, as long as such references exist, the value is guaranteed to also be reachable from the weak reference object, and thus, is a valid key in the cache table.
MapMaker FTW!
The thing about MapMaker is that there are many options for the kind of map you build, which enables those maps to serve many different purposes.
With the MapMaker you can choose between weak keys or weak values.
Consider this example:
You have an expensive computation or query which you want to cache for performance gains. You store the value in a map with an id as key which you will use to retrieve your values.
Normally you would store these values in a regular HashMap and store the hashmap in the cache, session or application. Now we have seen that this is generally not a good idea, since it will consume a lot of memory.
It is in these situations the MapMaker shines!
Lets say you have a list of Tasks for each User.
You would normally query the tasks like this:
If you want to rewrite this to use a Computing MapMaker you would write like this:
Here we have created a ConcurrentMap with weak values, which will be garbage collected in two hours. If the tasks have been garbage collected and the user is retrieving the tasks, the function is applied, and you get the tasks automatically, and put it back in the cache for another two hours.
Simple and great!
Global maps are prone to memory leaks
You should instead consider using soft reference and WeakHashMap or what I prefer, the MapMaker of Google Guava.
In this blog post, I will describe an efficient way of creating a map based cache. This can can either be stored in the session, application, or your existing cache.
Weak References
- What are weak references?
Weak reference basically means that the garbage collector can come and remove it when it is no longer in use. You have no guarantee that whatever you put in the map, will actually be around when you try to get it.
The reason why that is so useful is if you don't want to (or cannot afford to) retain an object indefinitely in memory.
Consider the following use case: You need to associate information with classes. Now, since you are running in an environment, where classes might get reloaded (say, a Tomcat, or OSGi environment), you want the garbage collector to be able to reclaim old versions of a class as soon as it deems safe to do so.
An initial attempt to implement this, might look like something like this:
class ClassAssociation {
private final IdentityHashMap<Class<?>,MyMetaData> cache = new ...;
}
The problem here is; this would keep all classes in the cache member forever (or at least, unless they are manually removed), forcing the garbage collector to retain them indefinitely, including everything referenced from the class (static member values, class loader information, etc).
By using weak references, the garbage collector can reclaim old version of the class as soon as no other references to it (usually instances) exist.
On the other hand, as long as such references exist, the value is guaranteed to also be reachable from the weak reference object, and thus, is a valid key in the cache table.
MapMaker FTW!
The thing about MapMaker is that there are many options for the kind of map you build, which enables those maps to serve many different purposes.
With the MapMaker you can choose between weak keys or weak values.
- Soft values are useful for caching, as you can cache values in the map without worrying about running out of memory since the system is free to evict entries from the cache if it needs memory.
- You can choose to have entries expire after a certain amount of time. This is also useful for caching, since you may want certain data cached for a specific period of time before doing an expensive operation to update it.
- One of my favorite things is making a computing map. A computing map uses a Function
to automatically retrieve the value associated with a given key if it isn't already in the map. This combines well with soft values and/or expiration times. After an entry is evicted by the map (due to memory demand or expiration), the next time the value associated with that key is requested it will automatically be retrieved and cached in the map once more.
Map<User,List<Task>> cache = new HashMap<User,List<Task>>(); //the global cache defined somewhere
if(cache.get(user) == null) {
List<Task> userTasks = getTasksForUser(user); // perform an intensive computation/query which we want to cache
cache.put(user, userTasks);
}
return cache.get(user);
If you want to rewrite this to use a Computing MapMaker you would write like this:
ConcurrentMap<String, List<Task>> cache = ...// Get the cache
if(cache != null) {
//If the tasks have been garbage collected, the function is applied, and you get the tasks
return cache.get(user);
} else {
ConcurrentMap<String, List<Task>> cache = new MapMaker().softValues().expireAfterWrite(2L, TimeUnit.HOURS)
.makeComputingMap(new Function<User, List<Task>>() {
@Override
public List<Task> apply(User user) {
return getTasksForUser(user); // perform an intensive computation/query which we want to cache
}
});
cache.put(user, getTasksForUser(user));
return cache.get(user);
}
Here we have created a ConcurrentMap with weak values, which will be garbage collected in two hours. If the tasks have been garbage collected and the user is retrieving the tasks, the function is applied, and you get the tasks automatically, and put it back in the cache for another two hours.
Simple and great!
Tuesday, January 11, 2011
Remember that ordinal parameters are 1-based in hibernate
I got this strange exception java.lang.IndexOutOfBoundsException: Remember that ordinal parameters are 1-based!
When running this query in hibernate/jpa
Which is very strange. The reason why hibernate cannot figure this out is because the ejb 3 form
The message about "1-based" is misleading here. It refers to the fact the the metadata for parameters is actually indexed by their sql positions (which is 1-based).
Good to know!
When running this query in hibernate/jpa
jbpmContext.getSession().createQuery("select pi from org.jbpm.graph.exe.ProcessInstance pi where pi.id = ?1 or pi.id = ?2 or pi.id =?3")
.setParameter(1, 9L).setParameter(2, 10L).setParameter(3, 11L).list();
Which is very strange. The reason why hibernate cannot figure this out is because the ejb 3 form
("?1")
is interpreted as a named parameter! So here we actually have to write it as .setParameter("1", 9L)
instead, or change the query and only type ? like so.jbpmContext.getSession().createQuery("select
pi from org.jbpm.graph.exe.ProcessInstance pi where pi.id = ? or pi.id
= ? or pi.id =?").setParameter(0, 9L).setParameter(1, 10L).setParameter(2, 11L).list();
The message about "1-based" is misleading here. It refers to the fact the the metadata for parameters is actually indexed by their sql positions (which is 1-based).
Good to know!
Subscribe to:
Posts (Atom)
Labels
- Seam (14)
- Java (11)
- Hibernate (5)
- JPA (4)
- Seam 2 (4)
- Seam 2.1.1 (4)
- Seam Series (4)
- Devoxx (3)
- EJB3 (3)
- JavaOne (3)
- MySQL (3)
- PDF (3)
- jodconverter (3)
- jodconverter 3 (3)
- axis2 (2)
- hash password (2)
- pdfa (2)
- sigar (2)
- swing (2)
- testng (2)
- webservice (2)
- AngularJS (1)
- Cache (1)
- Comment (1)
- Dropwizard (1)
- Encryption (1)
- Enum (1)
- Freemarker (1)
- Guava (1)
- IE8 (1)
- IdentityManager (1)
- InnoDB (1)
- Iran (1)
- JOOReport (1)
- JSF (1)
- Jasypt (1)
- JavaPolis (1)
- Memory (1)
- MyISAM (1)
- ODT (1)
- OSS (1)
- OpenOffice (1)
- REST (1)
- Seam book (1)
- Tehran (1)
- UTF-8 (1)
- Ubuntu hardy (1)
- Windows vista (1)
- ant (1)
- chess (1)
- damenes tale (1)
- filters (1)
- firefox (1)
- html cache (1)
- interceptor (1)
- java 5 (1)
- java 6 (1)
- java 7 (1)
- javazone (1)
- javazone 2015 (1)
- jaxb (1)
- jboss (1)
- jboss 4 (1)
- jboss-ws (1)
- jdbc (1)
- jquery (1)
- jrebel (1)
- julebord (1)
- junit (1)
- musehånd (1)
- ojdbc (1)
- oracle (1)
- persistence.xml (1)
- prettify (1)
- ptql (1)
- richfaces (1)
- rollermouse pro (1)
- rpi (1)
- s:cache (1)
- seam-gen (1)
- second level cache (1)
- treecache (1)
- wicket (1)