Monday, July 15, 2013

A Python script to execute Python code like "perl -ne"

I wrote a small utility script that can be used to run a small snippet of Python code like "perl -ne". I find it very useful for my needs. Hope it helps you too. Any suggestions welcome.
You can find the script as a gist here.

Setting the terminal window title - version 2

I had earlier written a small snippet about setting the window title from command line. Based on my experience, I felt that I could make it a lot simpler. Here is version of the same function.

function wtitle {
    if [ -z "$ORIG_PS1" ] ; then
        ORIG_PS1=$PS1
    fi
    export PS1="\[\033]0;$1 - \u@\h:\w\007\]$ORIG_PS1"
}

Wednesday, July 03, 2013

A rarely used but very useful option in grep

I usually do a lot of log analysis. I search for patterns in multiple files in multiple hosts. Then I collate the result and do processing on the result lines. A sample output looks like:
/tmp/input1.txt: line containing pattern
/tmp/input2.txt: another line containing pattern
 As it turns out, I don't need the file names in the grep result. I always used to remove the file name prefixes using a Perl one-liner. Silly me!

I was pretty sure that this was a common problem and it must have been solved already. When I referred to the man page of grep, I came across this gem: -h. If you specify this option, grep will not prefix each line with a the file name.

There is also -H option which will prefix each line with a filename even if you are searching in only one file.

Monday, June 03, 2013

Returning to Python

After a couple of years or so, I starting making use Python as my main programming language for one of the projects. This time I was making use of the virtualenv to install Python with various modules and just tar it up and copy to different machines. virtualenv saved me tons of time.


Tuesday, April 23, 2013

Zig-zag search

This is an interesting problem that I ran into. The problem definition goes like this.
You are given an array of integers. The array is considerably large (say 5 million elements). You are given two inputs: an index i in the array and an integer value v. Start searching the array for value v from the index i, and expand your search towards the two edges of the array. Return the index where the value v occurs closest to index i. If the value v occurs on both sides of index i at an equal distance, return the lower of the two indices. If the value v does not occur at all, return -1.
It was very interesting to solve this problem. Give it a shot, you might also like it.

Saturday, March 23, 2013

Installing Emacs 24.3.1

Earlier I blogged about a little hurdle that I ran into when I was trying to install Emacs 24.1. Since Emacs 24.3.1 was released recently, I thought I would give it a spin.

After downloading the source code, I unzipped the source code and ran configure command. I got a similar error that I was getting before.

 checking for libXaw... configure: error: No X toolkit could be found.  
 If you are sure you want Emacs compiled without an X toolkit, pass  
  --with-x-toolkit=no  
 to configure. Otherwise, install the development libraries for the toolkit  
 that you want to use (e.g. Gtk+) and re-run configure.  

I am trying to install Emacs on Mint 14. So to satisfy all the dependencies, I installed the missing packages.
  sudo apt-get update   
  sudo apt-get install libgtk2.0-dev libtiff4-dev libgif-dev libpng12-dev libxpm-dev libncurses-dev libjpeg-dev libjpeg8-dev libjpeg-turbo8-dev   

After installing the dependencies, the usual three step process worked fine.
 ./configure  
 ./make  
 sudo make install  

Hope this helps if you run into the same problem.

Wednesday, March 20, 2013

A late realization on embedding the code using gist

Off late I started using gist to embed source code in my blog entries. I was thinking about the consequences of doing this (or any form of embedding text using an external site). I realized the following:

  1. First and foremost the search engines don't get to index the page based on the key words in the embedded source code. For e.g. consider an embedded gist that demonstrates ServerSocket. Unless I mention about ServerSocket in my blog somewhere, the serarch engine will not associate the blog entry with the ServerSocket as that word appears only in the embedded source code.
  2. When peopele search for a specific key word in the embedded source code, they will be taken directly to the gist page, instead of the blog entry. This is bad because the user will not get the context and explanation around the source code.
To understand the depth of the problem, please search for alias-to-jumphosts-alist in Google. You will find that the gist is shown first and then the blog entry. Even in this case the blog entry is shown only because it mentions the function in it.

In future I should try to embed the source code in the blog itself and make use of a source code formatter like SyntaxHighligher.

Thursday, March 14, 2013

Using dirtrack-mode inside shell mode

I usually try to automate the things that I repeatedly perform. One the things that I recently automated was:
  • Create a directory with a prefix and today's date or time as suffix.
  • Chdir to the most recently created directory.
The source code for these utility functions is given below:

When I use these utility functions inside the Emacs' shell-mode, the shell-mode was unable to keep track of the current directory. The issue is that I am changing directories without making use of the cd command.

Then came the dirtrack-mode for the rescue. The idea is very simple: you specify the regular expression where the path appears in your shell prompt. The dirtrack-mode figures out the rest of the stuff.

Now my .emacs file contains the following two lines:

Thats it. Works like a charm. Please note that you might have to modify the regular expression to match where the full path appears in your prompt.

Monday, February 11, 2013

One sequence per table or one sequence per database

Let us assume that you are implementing RESTful APIs for your service. Your service has the following two models: users and tweets. While inserting new entities, you should genertate IDs. The IDs for each of these models can be generated in two ways:
  1. You can use a sequence (if you are using Oracle) or autoincrement feature (if you are using MySQL) to have per model IDs.
  2. You can use a global sequence for all the models.
If you are following the first strategy, you will have user IDs 1, 2, 3, 4, etc. and tweet IDs 1, 2, 3, 4, etc. If you are following the second strategy, you will have user IDs 1, 3, 4, 8, etc. and the tweet IDs 2, 5, 6, 7, etc. 

Thinking about it, I felt that using the first strategy will leak information. Consider some one doing a tweet every day at 10:00 AM and get the tweet ID. If first day the tweet ID is 100, and the second day the tweet ID is 200, etc. That person can infer that your system is receiving 100 tweets/day. I don't know how sensitive this information can be.

I would appreciate your thoughts. Please share them in comments.

Saturday, February 02, 2013

The issue with Emacs "cd" function and CDPATH

Recently I wasted sometime trying to figure out an issue with "cd" function and autocompletion in Emacs, while I had a value set for CDPATH environment variable. I hope it saves some time for someone who might bump into the same issue.

I had CDPATH environment variable set and it didn't have "." (current directory) as one of the components. When I tried autocompletion while running "cd" function, it didn't show any of the subdirectories under the current directory. The directories it showed for completion were from somewhere else. As soon as I added ".", I was able to see all the subdirectories of current directory in the autocompletion list.

Hope this saves some time for you as well.

Wednesday, January 23, 2013

Enabling multiple cores in VirtualBox

This post is a follow up to my earlier post Moving from VMWare Player to VirtualBox.

Once you have set up your VirtualBox instance, the first challenge you will face is that the guest OS may possibly be very slow due to the fact that the number of cores allowed is only one. Hence you will have to increase the number of cores that your guest OS can make use of. Open the VirtualBox manager, click on your VM, then click on the Settings button. In the Settings window, click on the System on the left pane and select the Processor tab on the right.
In this page, you can increase the number of processors. If you haven't already enabled the virtualization in your processor, you are likely to get an error when you click on OK. The error message will look like this:


Failed to open a session for the virtual machine Lubuntu.
VD: error VERR_NOT_SUPPORTED opening image file 'C:\VMs\Lubuntu.vmdk' (VERR_NOT_SUPPORTED).
Result Code: E_FAIL (0x80004005)Component: ConsoleInterface: IConsole {db7ab4ca-2a3f-4183-9243-c1208da92392}

If you see this error, you haven't possibly enabled the virtualization in the processor. Virtualization can be enabled only from the BIOS. Please refer to your computer's manufacturer web site to find out how to enable this. For your reference, I am giving a link to how to enable it in ThinkPad W520 (which I own).

Once the virtualization is enabled, you can restart your computer and try to increase the number of cores dedicated your guest OS again. Hopefully, you will not get the same error again.

Thats it. Boot your guest OS and enjoy!

Tuesday, January 08, 2013

Moving from VMWare Player to VirtualBox

This is my first post this year. Wishing you a happy New Year.

Until a week back, I was using VMWare Player for my personal use to learn different OSes. Using VMWare Player, I was trying out various Linux distros. I have been a fan of VMWare Player for six years now. Offlate I started seeing theese limitations:

  • I bought a new i7 quad core laptop. Even though I have 8 cores, my VMWare Player will not allow me to create a VM with more than 2 cores.
  • I used to enjoy the vmnetcfg command line tool which used to come with VMWare Player. But it has been removed from recent installations. Due to this, I was unable to configure NAT port-forwarding.
  • I used to be able perform packet sniffing using vnetsniffer command line tool. That also has been removed in the recent distributions.
  • You cannot run a 64bit guest OS in VMWare Player. Hence I could not run my guest Linux OS distros with more than 3 GB of RAM.
Before I started using VMWare Player primarily, I was evaluating VirtualBox. At that time, it was not mature. It used to be relatively slower than VMWare Player. Hence I gave up my hope on VirtualBox and stuck with VMWare Player. But the recent restrictions in VMWare Player prompted me to evaluate VirtualBox once again.

It is very easy to switch from VMWare Player to VitualBox. You can use the same images that you were using in VMWare Player. VirutalBox offers the following features, which I felt very compelling for me to start using:
  • It allows you to create VMs with as many as 32 cores (as long as the host can support that many cores).
  • You can configure NAT port-forwarding with an easy to use GUI.
  • The tools integration is excellent. VirtualBox supports the same set of facilities found in VMWare Player, like shared clipboard, shared folders, drag-and-drop of objects, etc.
  • It makes the best use of vitualization technology. If you would like to create a VM with more than one core, you must enble virtualization in your processor through BIOS (more on this in a separate post).
  • From the performance point of view, I don't see that much of a difference between VMWare Player and VirtualBox. In fact, I feel VirtualBox is slightly better.
I had a Lubuntu VM running in VMWare Player. Here are the steps I followed to migrate from VMWare Player to VirtualBox.
  • Download and install VirtualBox from here.
  • Shutdown the VM running inside the VMWare Player.
  • Launch VirtualBox Manager. Click on "New" icon to create a new VM.
  • You can accept defaults for most of the settings. When you come to the disk section, instead of specifying a new disk, you can select the existing vmdk file. In my case, I chose the vmdk file corresponding to my Lubuntu VM.
  • Save the settings. 
  • Start the VM from the VirtualBox Manager. It will start the VM in a new window, leaving the VirtualBox manager window intact.
  • Once launched, login to your VM and make sure things are working fine.
The only catch is your VM is running with default number of cores (which is only one) and possibly with low memory. How to increase the number of cores and memory is the discussion of my next post.