Thursday, October 30, 2008

Monitors in Java

Monitors in Java are in reality mutex + conditional variables. Silly note, but I think understanding this is essential.

A theoretical background of monitors could be found in the wiki.

Wednesday, October 01, 2008

Strace and defunct processes

We have a web application running in Tomcat. Today I ran into an issue that the web application started throwing some errors during start up. I wanted to figure what the application reads during the start up. So I started the whole Tomcat under strace, and started to analyze the strace output. Once I figured what was happening, I tried to stop strace with Ctrl-C, but it won't stop. So, I killed it with "kill -9". The strace is dead, but the Java process that was being traced went into a defunct state. This is what I was seeing when I did a "ps aux":
9929 1932 0.0 0.0 0 0 pts/0 Z 16:53 0:03 [java <defunct>]
Hmm ... Not good! I tried to kill this process with "kill -9", but it won't work. If I try to start another instance of Tomcat, it complains that the port 8989, where it was supposed to listen, was already in use. But if I try to do a "netstat -pan", I couldn't find any process listening in that port either. If I try to do a "telnet localhost 8989", I am able to connect. If I try to do a "/usr/sbin/lsof -n -P | grep 8989", I didn't see any process listening in that port! Wierd!

I tried to Google about defunct processes, but didn't find anything that would work.

Upon further investigation, the issue became clear. When I did a "ps auxm", I realized that there were other threds under this PID. And all those threads were under "Tracing" mode. See below (I have modified the output to fit the screen):
9929 1937 0.0 2.3 594924 48764 pts/0 T 16:53 0:00 java ...
... (10 such threads) ...
9929 1932 0.0 0.0 0 0 pts/0 Z 16:53 0:03 [java <defunct>]
The most important column is the "T" column. So, when the strace was killed, it left the process being traced and all the threads under that in being "Traced" state. I observed that any process that is defunct and being "Traced" cannot be killed with "kill -9". So the only way we can bring the process out of this state is by sending a SIGCONT signal. So I tried "kill -CONT". Voila! All those threads were reaped by init process in second.

So next time, if you see a process that is defunct, check if it is having any threads that might be in "T" state. If yes, try to send SIGCONT signal to one of the threads that is in T state. It should bring the process out of that state and the process will immediately be reaped by the init.

This issue happened in kernel version 2.4.21. I don't know if it has been addressed in later versions of kernel.

Tuesday, September 30, 2008

You cannot do SMP in VMWare Player

VMWare player doesn't support SMP, though you might see a processor which is capable of SMP. Here is a quote from the VMWarePlayer manual:
Virtual SMP
VMware Player does not support Virtual SMP™. You cannot use VMware
virtual machine that has more than one virtual processor assigned.

For e.g. when I look at /proc/cpuinfo, I am seeing only one processor though my VMWare Player emulates "Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz". When I inspect the boot messages, I realized that my Linux switch to uniprocessor configuration while booting:
[ 7.135461] SMP alternatives: switching to UP code
[ 7.135523] Freeing SMP alternatives: 11k freed

So if you are trying to test any multithreaded appplication, beware that the results you will get in your VMWare Player's Linux and the real Linux will not be the same. Your VMWare Player's version is likely to give you worse results as its running in uniprocessor mode.

Global Interpreter Lock in Python

Recently we were having an interesting discussion in our team about the threading and performance in Python. One of the key concerns was the Global Interpreter Lock (GIL) in Python. GIL is acquired before any thread accesses any of the Python objects. So if you are writing any C extensions and if you would like to manipulate any Python object, you must first acquire the lock and then only you can read/write/modify any object. Due to this, you will always have considerable of your code sequential.

There is a lot of discussion about GIL on the web. You will particularly find this page useful.

Someone suggested that Lua, by design, doesn't have any such constraints. It performs very well when it comes to multithreading. As I don't have much experience in programming in Lua, I don't know if that claim is true or not.

I am also not sure if Perl has any such global locks issue. I tried to Google it, but didn't find anything useful.

A useful tool to print the signals status

I have written a useful tool that will help in analyzing the signal disposition for a given process. It will take the list of PIDs as input and will print the signal disposition of that process in a readable form. For e.g.
5877: SigQ: 0/4096
5877: SigPnd: None (mask 0000000000000000)
5877: SigBlk: None (mask 0000000000000000)
5877: SigIgn: SIGHUP, SIGUSR2, SIGWINCH, SIGIO (mask 0000000030001002)
5877: SigCgt: SIGALRM (mask 0000000000004001)

The first column is the PID and the second column is the disposition and the the third column is the list of signals in a readable form.

It is a Python script. You can download that from here.

By default, the script doesn't print the real-time signals. To force it, you can specify the -r option in the command line. Sometimes there are signals for which there is no name available (for e.g. Signal 0). If such is the case, it will simply say 'Signal X'. There is no option to print the signal status of all the threads under the given process ID. If you know the thread ID ("ps -Leaf"), you can specify that to get the relevant information.

You can get the help for this script using -h. I have tested it in my Ubuntu box running kernel 2.6.24. Please drop me a word if you feel I can improve anything.

Monday, September 29, 2008

Identifying threads under a given process

This is quick note. The method to identify all the threads under a given process ID differs significantly between Linux kernel versions 2.4.x and 2.6.x.

In case of 2.4.x kernel, you should look for process IDs that are prefixed with dot (".") under the /proc folder. All these PIDs correspond to threads. If you open the /proc/.X/status file , the number appearing against "Tgid:" is the PID of the process that this thread corresponds to. So there is no direct way to tell the list of all the threads undre a given PID. The only way you can get that list is to iterate through all the directories that begin with dot under /proc, and check the "Tgid:" to be matching the PID in question.

In case of 2.6.x kernel, all you need to do is just to get the list of all the directories under /proc/PID/task. Each one of these IDs will correspond to one thread of the process. Also you will exactly know how many threads are there by checking the "Threads:" value in the /proc/PID/status file.

Check if your ps supports the option "-L". If it is supported, you can use that to get better information about the threads under each process.

Monday, September 08, 2008

What is 3G anyway?

I must admit that I didn't know the difference between 2G and 3G wireless technologies until recently. Whenever a discussion comes up among our friends about 3G, everyone seemed to be offering some opinion but me. So I ventured into understanding what is 3G. I know that I don't need to delve deeper to debate about 3G, but to understand it from a layman's perspective. One of the pages that I found was helpful was RadioShack's comparison page that tabulates the difference between different generations. That was a good starting point, from where I went to Wiki to learn more about each generations. I found the following pages to be helpful:

Monday, August 25, 2008

Difference between for (<STDIN>) and while(<STDIN>) in Perl

Can you tell if there is a difference between "for (<STDIN>) { BLOCK}" and "while(<STDIN>) {BLOCK}" in Perl? They both might look like similar. When I used the first construct and gave a huge file as input, my Perl program crashed. When checked the top output after starting the program, I figured that the RSS size of the program kept growing and eventually crashed. Hmm ... when I switched to the second construct, not only the program ran successfully but also used less memory!

The difference is that for construct reads the entire file before it enters into the loop. But the while construct reads only a part of the file and buffers it. I wrote a program to count the number of lines in an input file in both the flavors. while loop version took only 1MB where as for loop version took 149 MB for an input file of 79 MB. On top of it, the while loop version took on an average 1.36 seconds, and the for loop version took around 2 seconds.

So if you are writing any program in Perl that reads files line by line, you must stick to the while() construct if you would like your program not to crash on large file inputs and to run faster.

Virtual IP and timeouts

Let us say you are writing a client application that connects to a server and maintains a persistent connection. Whenever there is some user intervention, for e.g. typing some input, the client application sends the user input to the server. And prints back the output produced by the server. (Sounds like telnet?)

One thing your client program doesn't know about is that the server it is connecting to makes use of a virtual IP! Consider the case when your client program initiates a connection and the connection is in ESTABLISHED state. Right in the middle of the connection, the server drops the virtual IP address and for some strange reason it never gets reassigned to another host. In that case, neither the client program nor the server program will know the event of removal of the IP address. The server application would still be listening in the removed IP address and would still be holding the ESTABLISHED connections through removed IP address. Likewise, the client would still be holding an ESTABLISHED connection to the server.

If you observe the way Linux handles writes on sockets, when the client program writes into the socket, the written bytes will be copied to the kernel buffer and the write call will immediately return. The kernel will perform a best effort delievery attempt to deliver the bytes that you have written. Remember that the packets would never be delivered to the server in case of virtual IP removal. Worst part of this is the client doesn't know about this at all.

If you look at the server part, some of the servers have an inactivity timeout after which the server will close the connection. For e.g. Apache. So the server wouldn't keep these open sockets forever.

Hmm ... Unless the client opens another connection to the server, which would timeout, there is no way the client would know that its unable to reach the server.

So the moral of the story is, if you suspect that the server might be using a virtual IP and it might be dropped and never be reassigned to another server, build both the client and the server programs with proper timeouts for read/write/inactivity. Yes, keeping a time out for inactivity is very important.

Friday, July 25, 2008

Gmail over HTTPS

I had always been wondering what is the problem with these email providers, that they don't let me view my emails over HTTPS. One of the main reasons I could think of was the increased load on the web servers where the HTTP connection terminates. They will have to spend extra CPU cycles to encrypt/decrypt the incoming and outgoing data. Similar load will be incurred on the user's machine as well.

But Google provides mail over HTTPS. And that is seriously a good news. To access your email over HTTPS, instead of typing, type "".

Oh BTW, it is only the traffic comes from Google mail server to your desktop that is encrypted. Still the traffic between the mail servers go in unencrypted form. For e.g., if you are sending an email from your Google mail account to some account in Yahoo!, the mail will be sent in unencrypted form from Google server to the Yahoo! mail server.

Added on 07/29/2008: A friend of mine working in Google told me even better option. Once you login, you go to Settings and make "Browser connection:" to be "Always use https". Thats even better!

Friday, June 27, 2008

Setting the PuTTY window title from command line

I sometimes want to set the title of my PuTTY windows, like "Editor", "Compiler", etc. to identify distinct windows. I found the following script very useful. You can add that to your ~/.bash_profile. Once you login, you can set the title to whatever you want:

function wtitle {
if [ "$TERM" == "xterm" ] ; then
# Remove the old title string in the PS1, if one is already set.
PS1=`echo $PS1 | sed -r 's/^\\\\\[.+\\\\\]//g'`
export PS1="\[\033]0;$1 - \u@\h:\w\007\]$PS1"
echo "You are not working in xterm. I cannot set the title."

The above function will make the window title to be whatever argument you give followed by the usual user@host:workingdirectory. I think this should work with any xterm client. Not just PuTTY. (I haven't tested with any other xterm client.)

For e.g. to set the window title to be Editor, you would give the following command:
wtitle Editor
Update on Jul 15, 2013: I wrote a simpler version of this function which can be found here

Wednesday, June 25, 2008

Changing color schemes in vim

You can change the color scheme of vim by the following command:
The list of available color schemes could be found under /usr/share/vim/vim63/colors/ (If you are using a different version of vim then vim63 might be different for you). If you see file called blue.vim under this directory, to make use of that color scheme, you should give:
:color blue
If you would like to make the change permanent, add this line to your ~/.vimrc file.

Wednesday, June 11, 2008

Configuring the core file name pattern in Linux

First of all, you must make sure that you have set proper ulimit in your shell. You can check this by giving "ulimit -a" command. If the core file size is set to be 0, you can make it unlimited by giving "ulimit -c unlimited". Refer to your shell's man page to know how to set this. The child always inherits the ulimit from its parent process.

You can configure your system such that when an application dumps core the name of the core file has some meaningful name instead of just the bare word core. There are two files you should modify under /proc/sys/kernel configure this.

- This contains pattern of the core file name. The following patterns are allowed:
%% output one '%'
%p pid
%u uid
%g gid
%s signal number
%t UNIX time of dump
%h hostname
%e executable filename

/proc/sys/kernel/core_uses_pid - If this file contains a non-zero value, then the core file name will always be suffixed with the PID of the process.

You must be root to configure these things. Here is an example to make the core file name to contain executable file name and time of the dump:
echo core.%e.%t > /proc/sys/kernel/core_pattern

When a process dumps core, you will see the corefile name like this: core.a.out.1213240387.23989. (I had my core_uses_pid to be 1, so the PID is suffixed.)

A useful one liner to find out the time of the core in a better form is: perl -e 'print scalar(localtime(XXX))' where XXX is the time you are seeing in the core dump file name. Like this:
# perl -e 'print scalar(localtime(1213240387))'
Wed Jun 11 20:13:07 2008

I took the format info from the Linux kernel documentation (Documentation/sysctl/kernel.txt).

Wednesday, May 14, 2008

Finding native-endian in Java

I was getting curious about if its possible to write a Java program that finds if the underlying native platform is little- or big-endian. I guess its not possible to write such a program without having part of the code in C/C++ and using JNI. If anyone reading this blog feels otherwise, please let me know.

There is an API available from NIO to find out the native-endian. I think this API should use some native code underneath.

Wednesday, April 16, 2008

Serious security issue with IRCTC website

The login form of the IRCTC web site is being submitted over HTTP in plain text. This is a very serious issue since both your user ID and password could be sniffed by someone. One thing that I observed was that they have a HTTPS server running and this server is capable of receiving login requests. I think it was a bug in the code that the developer gave the URL as HTTP instead of HTTPS.

How to overcome the issue? This is not a clean approach but works fine. You can copy and paste the URL "" in your browser. Replace the XXX with your user ID and YYY with your password.

I tried to send a feedback about this to the site admin or someone in charge. Pathetic ... I could not find any link/email address in that website to do this. Hopefully someone from IRCTC will read this blog and fix the issue.

Thursday, April 03, 2008

WTH is wrong with Tata Indicom

Tata Indicom has an amazing (?!) web site to manage all your accounts with them online. I don't know WTH is wrong with them, none of the login pages are being submitted over HTTPS. Yes, any n00b running a sniffer can sniff out your password and any other sensitive information you give with a little effort. On top of this, the page was submitted to an IP address, instead of a URL, which was beyond my wild imagination. I had to run a whois query on APNIC server just to confirm if I am talking to one of their servers.

I am surprised how on earth Tata Indicom claims to be the number one (or one of the top) telecom service provider in India, if they don't even know the seriousness of their user's identity.

Wednesday, March 26, 2008

Good tutorials on Emacs Lisp

There are two pages that I would highly recommend to get started with Emacs Lisp.
Of course you should always have the Emacs Lisp reference manual handy.

Customizing colors in comint package in Emacs

The default foreground of prompt (gdb, shell, etc.) in the comint package in Emacs is blue. If you are working in PuTTY, the default background is black. Imagine blue on black background. Looks nasty!

You can customize the color of the prompt by using the comint-highlight-prompt variable. For e.g. I am using yellow foreground in my system. Here is how I have customized it by adding these lines in my ~/.emacs:
(copy-face 'default 'comint-highlight-prompt)
(set-face-foreground 'comint-highlight-prompt "yellow")

In short, the first lines creates a font-face variable by copying the default face and the second line changes the foreground value of that font-face variable to "yellow".

Scott A. Kuhl's .emacs file helped me a lot in understanding how to customize this. Thanks to him.

Tuesday, March 25, 2008

Time saving tip to connect PuTTY in one click

You can create a short cut in your Quick Launch folder that looks like:
\path\to\putty.exe -load "session name"

Session name could be obtained from your PuTTY dialog box when you launch it normally. This saves me a lot of time, since most of the time I connect to the same host.

There are other interesting questions available under PuTTY FAQ.

Customizing the colors in ls output

This posting is specific to customizing the dircolors in bash shell in Linux. It may or may not be applicable to other shells and other platforms.

When the bash shell is started, it executes all the shell scripts under the directory /etc/profile.d/*.sh. You might find other shell scripts under this directory (for e.g. colorls.csh), but those scripts are not to be executed by the bash shell. These files are executed within the current shell's environment, not in a separate execed shell (like a dotshell).

One of these files is A set of rules in search for the user specific dircolors file. This file could be any one of the ~/.dircolors, ~/.dircolors.$TERM, ~/.dir_colors or ~/.dir_colors.$TERM. In case the user has more then one of these files, whichever comes last will be taken.

Let us stick to the convention of using ~/.dircolors file for our customization. It is very easy to create this file by giving the following command:
dircolors --sh > ~/.dircolors
Then you can edit ~/.dircolors using your favorite editor. This file has only one environment variable called LS_COLORS. LS_COLORS variable is ":" separated. Each entry is a name value pair. Name could be the type of the file (for e.g. di is directory, fi is regular file, ln is link, etc.) or a regular expression for file's name (for e.g. *.sh, *.csh, *.gif, etc.). Value is a ";" separated three integers that tell about the attribute, foreground and background respectively. Here is an example (the color rule for a directory):
This says the directory should be displayed with no special attributes and blue foreground. You can easily check the color codes using the following command:
dircolors --print-database
For handy reference, this is what I see in my machine:
# Attribute codes:
# 00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
# Text color codes:
# 30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
# Background color codes:
# 40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white
So let us say I want to make my directories to be yellow and blinking, this is how I would have that entry:
You must remember that not all attributes are supported by all the terminals. Some terminals have limited capabilities.

A tip: In case you are totally fed up with colors (just the way I am), a sane alternative is to use the following alias (like they might not be able to show blinking text).
alias ls='ls --color=none -F'
This will add a "/" after directory, "*" after executable and "@" after a link, etc.

Friday, March 21, 2008

If you have time to waste: C++ or Java which one is faster

Okay. This one is just as useless argument as anyone could imagine: C++ or Java - which one is faster? The long discussion could be read from here. The original poster of the thread (Razii) came up with this idea of reading the whole bible verse by verse and sort it and write it back to disk using programs written in both Java & C++. In doing so he will compare the time taken by both Java & C++ programs. The (seemingly wrong) conclusion he came upto was that Java is faster than C++.

Let me give you my gist of it. I copied and pasted the same piece of code (and fixed the issue of including the iterator header file and dividing by 1000 to get the milliseconds) , and ran the same test in my Linux box. Well, almost always the C++ program was considerably faster than the Java program. Many a times my C++ program took only 50% of the time taken by the Java program. My Java version is "Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)" and my g++ version is "g++ (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)". I am running my experiments on Linux 2.6.9-42.13.ELsmp OS.

Again, this is only if you have some time to waste ... just the way I do ;-)

Have a happy weekend.

Wednesday, March 19, 2008

FTP connections over NATed VMWare virtual machine

I was trying to install the FreeBSD 7.0 in my VMWare virtual machine (VM). After I downloaded the installer disk's ISO image, I connected that ISO file as my CDROM drive in my VM and booted my VM. I should mention that my VM is in NAT mode, as I had decided to install the FreeBSD over the network.

When the installation tries to download the distro files from the FTP servers, it just kept saying it could not connect to the FTP server. I wasted around an hour or so without thinking about the basics (Damn!) that I am behind a NAT and I am trying to download files using active FTP mode. Then I tried to connect in passive mode and everything worked fine.

I don't want to repeat what many people have already clearly explained about passive mode of FTP. Here is an excellent tutorial that talks about active and passive modes of FTP.

Moral of the story is, don't forget that active FTP doesn't work (generally :-) if you are behind a NAT.

I would like to add only one thing: I think it would be a nice idea to have the FreeBSD installer indicate that passive ftp mode could work, in case it fails to open the data connection in active mode.

Friday, March 14, 2008

Event Completion Framework in Solaris 10

Simiar to kqueue framework in FreeBSD, Solaris 10 introduced a framework called Event Completion Framework (ECF). ECF is a very powerful concept that can be used when an application wants to wait for asynchronous events - like read/write events on sockets. Traditionally one would use poll()/select() for these events. There is enough discussion already on how primitive these mechanisms are and how they don't scale well on large number of file descriptors. Another advantage with kqueue and ECF is that you can wait on different kind of activities, not just activities on file descriptors. For e.g. when a process calls fork, or when a process calls exit, etc.

There are a few resources that would be very useful in understanding these frameworks:
  • Robert Benson's article on ECF
  • Sample program given in Bart Smaalder's blog
  • Jonathan Lemon's paper on kqueue
Oh btw, I think Linux is yet to have a mechanism as powerful as these.

Friday, February 01, 2008

Random notes on proc & process status

proc file system gives a view of the processes, including kernel, running in the Linux system. Writing into some of the files under /proc changes the state of the processes in memory. For e.g. you can change how much maximum shared memory you want the kernel to allow by changing the file /proc/sys/kernel/shmmax. This will take effect without rebooting the system, since it directly modifies the value of that particular kernel variable. But most of these changes are transient. Hence you will lose them when you reboot your system.

You can examine the state of the process using proc file system. Each process running in the system will have an entry under /proc/pid, where pid is the process ID. There are good tutorials that explain about each entry under this directory. You can google for them. But I could not find explanation for some entries, particularly status file. Thats the objective of this blog.

For a single-threaded process, you will find all the information about the state of the process under /proc/pid itself. For a multi-threaded process, the /proc/pid will have a subdirectory called task. This directory contains contains one subdirectory per thread, including the main thread: /proc/pid/task/tid1, /proc/pid/task/tid2, etc. The main thread has the thread ID same as the process ID.

The easiest way to find out if a given process is multi threaded or not is to examine the /proc/pid/task directory. If it contains many subdirectories, its a multi-threaded process. Otherwise, it could be a single threaded one.

One particular file that you will find very useful to understand the status of the threads (the only thread, in case of a single threaded application) is /proc/pid/task/tid/status. A sample status file is given below:
Name: a.out
State: S (sleeping)
SleepAVG: 88%
Tgid: 15155
Pid: 15155
PPid: 1
TracerPid: 0
Uid: 9929 9929 9929 9929
Gid: 9929 9929 9929 9929
FDSize: 256
Groups: 10 9929
VmSize: 2792 kB
VmLck: 0 kB
VmRSS: 696 kB
VmData: 40 kB
VmStk: 504 kB
VmExe: 2 kB
VmLib: 2186 kB
StaBrk: 0804a000 kB
Brk: 096fe000 kB
StaStk: bff854b0 kB
ExecLim: ffffffff
Threads: 1
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 000000027f00000e
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000

Let me explain each of the fields in this file.

Name: The name of the binary of this process.
State: The current state of the process. There are totally seven states of a process: running, sleeping, disk sleep, stopped, tracing stopped, zombie and dead. I will explain these states in a separate blog :-)
SleepAVG: I think this is average sleep time. I am not sure.
Tgid: Thread group ID. This is a single value, which implies that a thread cannot be a part of more than one thread group.
Pid: Process ID.
PPid: Parent process ID.
TracerPid: If the process is currently being traced, for e.g. using strace, this will contain the tracer's PID.
Uid: real user ID, effective user ID, saved user ID and the file system user ID.
Gid: real group ID, effective group ID, saved group ID and the file system group ID.
FDSize: What is the size of the file descriptor table for this process.
Groups: List of groups that the owner of the process belongs to. (I am not sure about this info).
VmSize: Total virtual memory size.
VmLck: Locked virtual memory size. (Refer to the man page for mmap and mlockall).
VmRSS: Resident set size. i.e. how much of the total virtual memory is resident in RAM.
VmData: How much of the virtual memory is data.
VmStk: How much of the virtual memory is stack.
VmExe: How much of the virtual memory is executable (a.k.a text).
VmLib: How much of the virtual memory shared object libraries occupy.
StaBrk: The starting address of the data that could be grown using sbrk() call.
Brk: The top of the data that has been reached using sbrk() call.
StaStk: I am not sure about this value.
ExecLim: I am not sure about this value either :-(
Threads: The number of threads this process has.
SigPnd: Signals pending to be delivered to this thread.
ShdPnd: Shared pending signals. I am not sure how this value is used.
SigBlk: Signals that the thread explicitly blocked. These signals will be delivered when the thread unblocks the signal.
SigIgn: Signals that the thread decided to ignore. There is a difference between blocking a signal and ignoring a signal. Pickup APUE and read the chapter on signals.
SigCgt: Signals that the thread decided to catch, by installing a signal handler. This is done through sigaction() function.
CapInh, CapPrm, CapEff: All I know is these are related to capabilities and they stand for capability inheritable, capability permitted and capability effective. I don't have an understanding of what they are.

Okay. Hopefully that helps in understanding the status of a thread when you look at status file next time.

Happy hacking and have a good weekend.

Wednesday, January 23, 2008

Monitoring TCP connection open/close in Ethereal

I found the following filter expression to be useful when I need to monitor only the TCP connection open/close.
(tcp.flags.syn == 1 || tcp.flags.fin == 1 || tcp.flags.reset == 1)

For e.g. when you wish to monitor all the SSL connections opened and closed, the following Filter is good:
(tcp.port == 443) && (tcp.flags.syn == 1 || tcp.flags.fin == 1 || tcp.flags.reset == 1)
You can also combine the IP address of the server you are interested in, like:
(tcp.port == 443) && (tcp.flags.syn == 1 || tcp.flags.fin == 1 || tcp.flags.reset == 1) && (ip.addr == X.X.X.X)
Monitoring the connection open/close activity helps in understanding the client behavior. For e.g. generally Firefox and IE both open multiple (I have seen three or four at the max) https connections and reuse the same SSL session ID. These connections are concurrently open and most of them don't get closed from the browser side until the web server closes them.

Thursday, January 17, 2008

SSL session reuse - how to find if supported?

Before I delve deeper, its a good idea to be clear about SSL session reuse. Every time when a client (browser, curl, etc.) connects to a server over SSL, the server creates a session for that connection. This session ID is sent as a part of the Server Hello message. This is to make things efficient, in case the client has any plans of closing the current connection and reconnect in the near future. Most of the servers have a time out for these sessions (I think 24 hours is a common value, unless pressed for space).

When the client connects to the same server again, it can send the same session ID as a part of the Client Hello. The server will first look up if it can find any sessions with that ID. If found, the same session will be reused. Thus the time spent in verifying the certs and negotiating the keys is saved. If the server cannot find a matching session, then it responds with a new session ID and its certificate in Server Hello message. The client knows that it has to verity the cert and negotiate the key again.

Considerable amount of time is spent in validating server certs. Reusing SSL session will save this time.

But when there is more than one server behind a load balancer, and the client connects only to the load balancer, there is a likelihood that the load balancer forwards second connection to a different server. But luckily, most of the load balancers today can be configured to provide the SSL session stickiness, and hence they will forward the request properly. From my understanding, most of the load balancers achieve SSL session stickiness through IP stickiness.

The question is: how to find out if a server supports SSL session reuse? Or, if you are hitting load balancer, how to find out if the load balancer maintains SSL session stickiness?

OpenSSL has a useful application called "s_client" which can be used to find out if SSL session is supported. The following command will do the magic:
openssl s_client -reconnect -state -prexit -connect ServerURL
The important option here is "-reconnect". It will disconnect and reconnect to the server 5 times using the same SSL session ID, that came in the first Server Hello message. It will print each time if the same session ID is reused or a new session ID is sent in the Server Hello message.

Friday, January 11, 2008

Learning Perl - ebooks & resources

I ventured into brushing up my Perl knowledge, as its really a long time since I read a book on Perl. I found the following resources useful:
  1. Beginning Perl by Simon Cozens. I think this is one of the best books to begin Perl programming.
  2. The perldoc website. This is not really book or a cogent order of tutorials, but an excellent reference once you are familiar with the basics.
  3. A collection of resources to learn Perl from the website.
And there is always perldoc, to refer to topics/functions/modules off line.

Monday, January 07, 2008

Wierd error while setting up SFTP

Recently I had to set up SFTP access in my RedHat Linux box. After uncommenting the "Subsystem sftp ..." in the /etc/ssh/sshd_conf file, I restarted the sshd. When I tried to login, I am prompted for password but immediately I was getting this error message:
Received message too long 1214606444
Hmm ... I couldn't make out what could be the reason. Then when I Googled I cam across this page. I had the same issue in my .bashrc file where I had echoed a welcome message :-(. I confirmed that by taking the hex value of the number above (which is 0x48656C6C) which are the ASCII values of 'H', 'e', 'l' and 'l' respectively. The actual message was "Hello, have a great session!".

Moral of the story: Don't have any unnecessary echo messages in your .bash_profile or .bashrc files.