Thursday, October 28, 2010

Initial thoughts on Fantom

Recently I started playing with Fantom language. So far my reaction is "Wow!". Its a beautiful language with a lot of cool things. I also ran into a few gotchas, but none that falls in the category of "ugly!".

Here are the things I really liked about the language (not necessarily in any order):
  1. Before any technical merits, the first thing I liked is the documentation. Most of the time, when I look for some documentation for a open source project, they notoriously suck. Fantom is the second project whose documentation I really liked and found my way around most of the time. (The first one is SLF4J.)
  2. The language is statically typed, with room for dynamically invoking methods on objects using a mechanism called "trap".
  3. One central theme I observed in the language is being able to reliably create an immutable objects ("const" classes). This simplifies a lot of analysis when it comes to concurrent programming. Also the language supports creating immutable Lists and Maps.
  4. The language has a few forms of literal expressions that the compiler understands and creates objects for you. For e.g. Map, List, Uri and Duration have equivalent literal forms. One of the places where supporting literal forms aces is during serialization. Any object that is serialized is humanly readable (There is provision for overriding this as well!). Likewise you can create objects from string literal forms that can either be read from a file or just constructed by the program on the fly. What that means? One particular case I can think of is ease of creation of test data.
  5. Any field you specify is automatically accessed through getter and setter generated behind the screen. If you want, you can add more checks and behavior to the getters and setters.
  6. For concurrency, the language provides Actors framework. This is one level of abstraction above thread. In short: actors respond to messages and these messages are immutable. In a way, it simplifies the thinking about concurrent programming.
  7. The language supports anonymous function blocks. i.e. functions are objects too! That helps you in having some neat and cool patterns in your code. Like "10.times { ... }" or "aList.each { ... }". That also means the language supports closures.
  8. The language has mixins support. Its a useful concept when you want to assemble behavior on a type.
  9. Java Annotations equivalent is called as facets. You can decorate a field, method or a type with facets. BTW, fields and methods are called as slots in Fantom. I guess that term came from Smalltalk.
  10. You can use any of the Java classes just by adding a "using ..." statement. That gives you enough power and you wouldn't be missing any libraries that you may not find in Fantom.
  11. The code you write runs in JVM, CLR and (almost in) JavaScript. The work to have full support on JavaScript is actively underway.

Okay. Now the gotchas.
  1. You cannot control how many actors you want to be simultaneously active. The language takes care of it for you. Sometimes, you may want to have control on this one.
  2. There is no single queue and multiple consumers paradigm in actors. Either the producer can keep enqueuing the work in the same actor's queue or enqueue work with multiple actors in a round-robin fashion. In a circuitous way, you can implement this.
  3. The way a a collections member field (a List or a Map) is serialized and deserialized, hmm ... still I am not quite clear. I need to read about that section again.

Please understand that these gotchas may not be true, as I am a beginner and may be doing things in a wrong way.

Above everything else, I like the community of Fantom. Those guys are just amazing in terms of responding to my questions and helping to understand the concepts. I hope to write more about my adventures in Fantom-land. Stay tuned.

Tuesday, October 19, 2010

SecretKeyFactory is broken in JDK 1.6 update 22

If you upgrade JDK to 1.6 update 22 (build 04), the SecretKeyFactory is broken. As a result you will not be able to load any PKCS12 key stores. You will get NoSuchAlgorithmException thrown.

I have reported this issue in the bug database. You can view the bug here. I guess it will take upto a day for this bug to be externally visible, if you don't have a SDN account.

Here is the sample program to reproduce the issue:
public static void main(String[] args) {
        SecretKeyFactory instance = SecretKeyFactory.getInstance("PBEWithMD5AndDES");
        System.out.println("Returned instance: " + instance.getAlgorithm());

You will get the exception below:
Exception in thread "main" PBEWithSHA1AndRC2_40 SecretKeyFactory not available
    at javax.crypto.SecretKeyFactory.<init>(DashoA13*..)
    at javax.crypto.SecretKeyFactory.getInstance(DashoA13*..)
    at main.TestClient.main(

Work around: Until the issue is fixed, revert back to JDK 1.6 update 21. I will update my blog once if I find any fix or work around.

Update on 10/25: Actually I figured one thing: I had update 21 as the project's default JRE in my Eclipse, but tried to run the application using update 22. When I changed the project's default JRE to update 22, I didn't get the error anymore. I am guessing that could be the problem. I am not having the issue any more.

Saturday, October 16, 2010

Performance of BigInteger.toString(radix)

You are given a byte array, that represents a number in big endian format (the most significant byte first). You have to convert the byte array to its equivalent hex string. What is the most efficient way to do it?

There are many ways to do it. But let us start with the easiest and correct one and optimize our solution. BigInteger class provides a constructor to convert the byte array to a BigInteger. We can convert that BigInteger to a hex string using the BigInteger.toString(radix) method. The solution is given below:
public static final String toHexStringUsingBigInteger(byte[] input) {
        BigInteger bi = new BigInteger(input);
        return bi.toString(16);

We can also come up with a hand crafted solution that extract nibble by nibble and convert them into their equivalent hex character and finally forming a string. This solution is given below:
public static final String toHexStringUsingCharArray(byte input[]) {
        int i = 0;
        if (input == null || input.length <= 0)
            return null;

        char lookupArray[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f' };
        char[] result = new char[input.length * 2];

        while (i < input.length) {
            result[2*i] = lookupArray[(input[i]>>4) & 0x0F];
            result[2*i+1] = lookupArray[(input[i] & 0x0F)];
        return String.valueOf(result);

For comparison purposes, let us write a test program and see how much time each implementation takes. The test program is given below:
public static void main(String[] args) {
        byte[] input = new byte[]{0x7D, 0x7D, 0x7D, 0x7D, 0x7D, 0x7D, 0x7D, 0x7D};
        long start = System.nanoTime();
        for(int i = 0; i < 100000; i++) toHexStringUsingBigInteger(input);
        long end = System.nanoTime();
        long start2 = System.nanoTime();
        for(int i = 0; i < 100000; i++) toHexStringUsingCharArray(input);
        long end2 = System.nanoTime();
        System.out.println(String.format("Using BigInteger  : %15d", (end-start)));
        System.out.println(String.format("Using char array  : %15d", (end2-start2)));

Here is the output of few runs of the program:
Using BigInteger  :       702013524
Using char array  :        42074621
Using BigInteger  :       711340129
Using char array  :        41369504
Using BigInteger  :       707484052
Using char array  :        41221440

WOW! You see the difference between the BigInteger version and the hand crafted version? Hand crafted version is almost 16 times faster. Why?

You can take a look at how BigInteger.toString(radix) is implemented in the OpenJDK here. The most important point is, it is making use of Conversion.bigInteger2String() method. You can view the source code of that here. Ultimately, bigInteger2String() method is written such a way that it can cater to different radices and different locales. Most of the complexity in that function is around this concern. That is the reason why BigInteger.toString(radix) is terrible in performance.

Moral of the story:
If you are planning to use BigInteger and convert that back and forth to hex string, you are better off writing your own version of toHexString as the one given above, instead of using what you get with the library. You can invoke the BigInteger.toByteArray() and pass the byte array to your toHexString method.

Most important caveats:
Beware of handling of negative numbers between the two approaches. In case of the second approach, you will not get the sign right. You will always get a hex string for the bytes given as argument, without any special interpretation of the bytes. Where as, the BigInteger approach treats the bytes as a signed integer value. Consider the following test program:
public static void main(String[] args) {
        byte[] input = new byte[]{(byte)0x8D};
        System.out.println("From BigInteger : " + toHexStringUsingBigInteger(input));
        System.out.println("From char array : " + toHexStringUsingCharArray(input));

And the output is:
From BigInteger : -73
From char array : 8d

Depending on your application, you will have to choose one usage over the other.

The other important thing is that the code given above doesn't skip the leading zeroes. You can make a small modification and make it to skip leading zero bytes.

Thursday, October 14, 2010

Issue of autoboxing and reflection

Problem: Consider that you want to have a method called "Object invokeAndGet(Object targetObject, String methodName, Object... args)". This method should be able to invoke the given method on the target object and return the value returned by the invocation. If the method has overloaded forms, then depending on the argument types, the invokeAndGet method should invoke the correct form.

Though this problem looks trivial and seems like it can be solved by using reflections APIs, it gets really tricky when you have to deal with primitive types. This is due to the fact that autoboxing converts the primitives into their corresponding wrapper types. For e.g. an integer value is autoboxed into Integer object, etc.

To begin with let us assume that we have the following implementation of the method:

    public static Object invokeAndGet(Object obj, String methodName, Object... args) {
        try {
            Class<?>[] argsTypes = new Class[args.length];
            for(int i = 0; i < argsTypes.length; i++) {
                argsTypes[i] = args[i].getClass();
            Method m = obj.getClass().getMethod(methodName, argsTypes);
            Object retObj = m.invoke(obj, args);
            return retObj;
        } catch (Exception e) {
            System.out.println("**** Exception thrown: " + e);
            return null;

What this code is straight forward. It gets the types of the arguments. Attempts to find out if the target object's class has any method matching the method name and the argument types. If one is found, then that method is invoked. Otherwise null is returned.

The tricky part is this. Assume that the target object's class has a method "void setX(int x)". If you attempt to do invokeAndGet(targetObject, "setX", 1), it will fail. The reason is because of the fact that the argument 1 is converted to its wrapper type, which is Integer. So when you look up, you are looking up for the method with the signature "setX(Integer)", where as the method that is present in the target object has the signature "setX(int)".

There is no bullet proof solution for this problem. One of the ways you can try to solve this issue is by retrying with primitive types when you get a NoSuchMethodException. The modified version of the code is given below. It works for most of the cases:
    public static Object invokeAndGet(Object obj, String methodName, Object... args) {
        try {
            Class<?>[] argsTypes = new Class[args.length];
            for(int i = 0; i < argsTypes.length; i++) {
                argsTypes[i] = args[i].getClass();
            Method m = null;
            try {
                m = obj.getClass().getMethod(methodName, argsTypes);
            } catch (NoSuchMethodException nsme) {
                boolean signatureModified = false;
                for(int i = 0; i < argsTypes.length; i++) {
                    Field field;
                    try {
                        field = argsTypes[i].getField("TYPE");
                        if(Modifier.isStatic(field.getModifiers())) {
                            argsTypes[i] = (Class<?>) field.get(null);
                            signatureModified = true;
                    } catch (Exception e) {
                        // Ignore these exceptions. There is nothing we can do by catching them.
                    m = obj.getClass().getMethod(methodName, argsTypes);
            Object retObj = null;
            if(m != null)
                retObj = m.invoke(obj, args);
            return retObj;
        } catch (Exception e) {
            System.out.println("*** Exception thrown: " + e);
            return null;

What I am doing is very simple. Whenever NoSuchMethodException is thrown, I convert all the wrapper classes to their respective primitive classes by accessing the "TYPE" static field in them. For e.g. Integer.TYPE will give me int.class. Then I retry to get the new method, if I the signature had been modified since the last attempt.

I said that almost works. Here are the corner cases:
1) If your method has mixed type arguments, some of them are primitives and some of them are wrappers, the code given above will not work. For e.g. if you have a method with signature "setXIf(int oldValue, Integer newValue)", then invokeAndGet(targetObject, "setXIf", 1, 2) will fail.
2) There is a remote case when one of the argument class has a field called TYPE which is of type Class. In such cases also the above code will fail.

Please let me know if you know of a better method than the one given above to reflectively invoke a method on an object. I will appreciate that.

Tuesday, October 12, 2010

Creating a HashMap with entries

Problem: Create an instance of HashMap in Java with entries in it. I don't want to create a new HashMap instance and keep adding entries to it. I want to have a concise way of creating a HashMap with entries.

Solution: Guava library comes with factory methods that can create a HashMap without using the verbose form of "Map myMap = new HashMap()". You can simply say "Map myMap = newHashMap()", assuming you have done a static import of the Maps.newHashMap method. But that is not sufficient. It would be better to provide a utility method that looks like this: "Map myMap = newHashMapWithEntries(firstKey, firstValue, secondKey, secondValue)". That way it is easy to create static-final maps instead of writing a separate method to populate them or to populate them from constructor, as given below:
public class MyClass { 
   private static final Map<String, URL> serviceUrls = createServiceUrlsMap(); 
   private static Map<String, String> createServiceUrlsMap() { 
    Map<String, URL> retVal = new HashMap<String, URL>(); 
    retVal.put("google", new URL(""); 
    retVal.put("yahoo", new URL(""); 
    return retVal; 

That is too verbose. What I want is something like:
public class MyClass { 
   private static final Map<String, URL> serviceUrls = newHashMapWithEntries( 
       "google", new URL(""), 
       "yahoo", new URL(""));  

So here is one possible implementation of newHashMapWithEntries:
    public static <K, V> Map<K, V> newHashMapWithEntries(K firstKey, V firstValue, Object... rest) {
        if(rest.length%2 != 0) {
            throw new IllegalArgumentException("Expected the number of args to be even.");
        Class<? extends Object> keyType = firstKey.getClass();
        Class<? extends Object> valueType = firstValue.getClass();
        Map<K, V> retVal = new HashMap<K, V>();
        retVal.put(firstKey, firstValue);
        for(int i = 0; i < rest.length; i += 2) {
            if(!keyType.isAssignableFrom(rest[i].getClass())) {
                throw new IllegalArgumentException("Expected all keys to be of type <? extends " +
                        keyType.getName() + ">, but found " + rest[i].getClass().getName() + ".");
            if(!valueType.isAssignableFrom(rest[i+1].getClass())) {
                throw new IllegalArgumentException("Expected all values to be of type <? extends " +
                        valueType.getName() + ">, but found " + rest[i+1].getClass().getName() + ".");
            retVal.put((K)rest[i], (V)rest[i+1]);
        return retVal;

Saturday, October 09, 2010

Bottle - A simple web framework in Python

I recently came across the Bottle. It is a web framework written in Python. Its very simple to learn and use. It is ideal for prototyping kind of work or you want to run a web application in no time.

SublimeText 3/Anaconda error

When I installed Anaconda manually by downloading and untarring the file (as given in the manual installation instructions here ), I got th...