Mutex Lock

PHP - OpenID library integration

2010-02-28T21:08:00.001-08:00

An excellent introduction to the openID library as a consumer from the PHP here.

Erlang based WebSocket client in place

2010-02-16T12:41:00.000-08:00

An erlang based WebSocket client in place here , for clients that deal with web socket protocol as yet.

Storing Data in a Hash - Erlang

2010-02-16T11:29:00.000-08:00

Came across this nice example in the Erlang mailing list for storing data in an hashmap.

-module(db_server).

-export([start/0, init/1, write/2, read/1, delete/1]).

start() ->
register(server, spawn(db_server, init, [dict:new()])).

init(Records) ->
receive
{add, Pid, Key, Value} ->
RecordsNew = dict:store(Key, Value, Records),
Pid ! {ok, Key, Value},
init(RecordsNew);
{show, Pid, Key} ->
case dict:find(Key, Records) of
{ok, Value} -> Pid ! {ok, Value};
error ->  Pid ! {error, no_such_value}
end,
init(Records);
{delete, Pid, Key} ->
RecordsNew = dict:erase(Key, Records),
Pid ! {ok, ok},
init(RecordsNew)           
end.

write(Key, Value) ->
server ! {add, self(), Key, Value},
receive Res ->
Res
end.

read(Key) ->
server ! {show, self(), Key},
receive Res ->
Res
end.

delete(Key) ->
server ! {delete, self(), Key},
receive Res ->
Res
end.

Deleting tags from remote in git

2010-02-05T12:07:00.000-08:00

Happened to go through a build process , and while flipping around with versions - it created some tags on the remote repository that I wanted to get rid of entirely.

For example - lets assume the tag name is artifactA-0.1.0 .

$ git tag -d artifactA-0.1.0

This deletes the tag locally ( in your local clone )
To push the change to remote and to delete the tag remotely as well - we can give -

$ git push origin :artifactA-0.1.0

where , I assume origin is the name of the remote branch from which I cloned initially.

This should delete the tag remotely as well.

GPG agent

2010-02-04T16:11:00.000-08:00

When preparing some artifacts to be published to a maven repository - needed some help with gpg publishing.

More often that not - when the gpg key verification was happening - it was reporting about a missing file - ~/.gnupg/S.gpg-agent .

'touch'ing would not help because that is not a file , but a socket for the agent to listen on.

$ gpg-agent --use-standard-socket --daemon 2>/dev/null

This makes the agent listen on the socket.

Google Collections SVN repository

2010-01-29T17:18:00.000-08:00

Google Collections 1.0 was released recently towards the end of December 2009. While very useful from an API and performance perspective, the API had quite an amount of surprises / deprecations / removals in its later stages ( 0.7 / 0.8 etc.) .

With 1.0 the API seems to have been stabilized and as an added benefit for those integrating with mvn - it is also available here as the mvn repository - http://google-maven-repository.googlecode.com/svn/repository/com/google/collections/google-collections/1.0/ .

com.google.collections / google-collections / 1.0 should do the trick in ivy.xml / pom.xml as appropriate.

libevent 2.0 released

2010-01-28T11:10:00.000-08:00

As per this update on the google developer blog , libevent 2.0 seems to be released.

For those of you new to the library - libevent provides a platform agnostic event handling library so that the user does deal with the quirks of the operating systems like Linux and Solaris and chooses the best event handling adapter present in the kernel ( Eg: In Linux, from 2.6 - epoll performs much better than poll / select . The former in O(1) in handling of connections whereas the latter group is O(n) in event handling proportional to the number of active connections at that time instant ).

More details are available in the book available here.

Specifically about the update - it seems like the developers on Windows would benefit a lot from the API changes made. I will probably write a more detailed review of the same after playing around with the software / API on various platforms and distributions.

Resetting / Overwriting /etc/resolv.conf in EC2 instance

2010-01-23T00:59:00.000-08:00

Creates a new EC2 instance ( CentOS ) and by default it comes with a dns resolver to the outside world.

For our purposes - we had set up an internal dns server running (bind - process 'named' on a particular host).

When we were launching our pool of servers - we wanted to make sure that the new instances fall under the same domain that we specify it to be.

We also wanted to set the nameserver of the newly created instances pointing to the internal DNS server we have to resolve the ambiguity we have.

Before making the change - the file /etc/resolv.conf was looking as follows.

# cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search some.internal.aws.domain
nameserver 172.x.y.z

where 172.x.y.z was something set by Amazon EC2 by default.

Edit /etc/dhclient.conf ( Create one , if it does not exist )

supersede  domain-name "ec2.mycompany.com" ;
prepend domain-name-servers 10.p.q.r ;my internal company dns

I also found it useful to create an Elastic IP and associate the dns server instance with the elastic ip,
and then have the prepend domain-name-servers refer to the elastic ip , instead of the internal ip.
So - even if the internal dns server fails = we can reconstruct it from another AMI and associate with the elastic ip without affecting the rest of the system.

To see the changes to /etc/dhclient , do the following

$ dhclient -r ; dhclient

This command forcibly renews the dhcp lease that will force the new credentials from /etc/dhclient.conf to be read.

After doing this , verify /etc/resolv.conf

search ec2.mycompany.com
nameserver 10.p.q.r    ; our DNS server
nameserver 172.x.y.z  ; the one set by amzn

So that completes the process and we are good about it.

Zookeeper build

2010-01-17T11:24:00.001-08:00

Was trying to build Zookeeper from trunk ( 3.3.0 ) .

Came across this error -

create-cppunit-configure:
     [exec] configure.ac:33: warning: macro `AM_PATH_CPPUNIT' not found in library
     [exec] libtoolize: putting auxiliary files in `.'.
     [exec] libtoolize: copying file `./config.guess'
     [exec] libtoolize: copying file `./config.sub'
     [exec] libtoolize: copying file `./install-sh'
     [exec] libtoolize: copying file `./ltmain.sh'
     [exec] libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and
     [exec] libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
     [exec] libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
     [exec] configure.ac:33: warning: macro `AM_PATH_CPPUNIT' not found in library
     [exec] configure.ac:33: error: possibly undefined macro: AM_PATH_CPPUNIT
     [exec]       If this token and others are legitimate, please use m4_pattern_allow.
     [exec]       See the Autoconf documentation.
     [exec] autoreconf: /usr/bin/autoconf failed with exit status: 1

Installed -

$ sudo apt-get install libcppunit-dev

Ended up with ..

     [exec] .../zookeeper/src/c/configure: line 5015: syntax error near unexpected token `1.10.2'
     [exec] .../zookeeper/src/c/configure: line 5015: `   AM_PATH_CPPUNIT(1.10.2)'

Hmm.. bad times.

Installing Thrift Continued

2010-01-15T00:20:00.001-08:00

Continuing my previous post of installing Thrift - needed to install the following obvious ones as well.

 sudo apt-get install flex bison

And then , run ./configure once again to make sure all the libraries are linked together.

 ./configure

This should regenerate the makefiles once again, after installing flex and bison.

Installing Thrift

2010-01-15T00:12:00.001-08:00

Checked out thrift from the trunk on my ubuntu box ( 9.10 ) .

The first step on the installation was running the program

$ ./bootstrap.sh

It failed with some errors , some fairly obvious (missing autoconf ) - some not so obvious.
The first step is to get autoconf installed.

$ sudo apt-get install autoconf

Then ran into this error.

$ ./bootstrap.sh
configure.ac:44: error: possibly undefined macro: AC_PROG_LIBTOOL
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:26: installing `./install-sh'
configure.ac:26: installing `./missing'
compiler/cpp/Makefile.am: installing `./depcomp'
configure.ac: installing `./ylwrap'
lib/cpp/Makefile.am:24: Libtool library used but `LIBTOOL' is undefined
lib/cpp/Makefile.am:24:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
lib/cpp/Makefile.am:24:   to `configure.ac' and run `aclocal' and `autoconf' again.
lib/cpp/Makefile.am:24:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
lib/cpp/Makefile.am:24:   its definition is in aclocal's search path.
test/Makefile.am:30: Libtool library used but `LIBTOOL' is undefined
test/Makefile.am:30:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
test/Makefile.am:30:   to `configure.ac' and run `aclocal' and `autoconf' again.
test/Makefile.am:30:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
test/Makefile.am:30:   its definition is in aclocal's search path.

The fix was to install libtool.
$ sudo apt-get install libtool

And then comes the boost libraries -

$ sudo apt-get install libboost1.40-dev libboost1.40-doc

(Your boost library version might be different from mine but you get the idea !!).

HDFS Permissions issues

2009-11-19T14:11:00.001-08:00

If you are running into permissions issue in hdfs installation (Eg: When you try to write to hdfs and that does not seem to work ) - then you may want to *relax* the permissions by the following setting in all hdfs cluster nodes , in hdfs-site.xml ( starting from 0.20 ). Of course - the namednode needs to be restarted for the changes to be effective.

<property>
<name>dfs.permissions</name>
<value>false</value>
</property>

Important - this has a gaping security hole in itself by relaxing the permissions and currently the hdfs team is actively working on enabling better permission based access rules. So - this change is best in the early stages of development to get started and should be revisited once again soon after.

iBator eclipse plugin

2009-07-23T12:43:00.003-07:00

If you are like me and you are into using iBatis eclipse plugin - check out http://ibatis.apache.org/ibator.html. It comes with an eclipse plugin that is quite useful if you are working with the ibatis configuration files.

Handling signals in Java

2009-07-17T09:26:00.001-07:00

Signals by definition are specific to the underlying implementation ( read, operating system) .

Java, being a platform independent language , it is often discouraged to write platform specific stuff except for the rarest cases and when there is a clear business justification for the same.

Java does have an undocumented API that talks about signals where we can override the default signal handler for a given signal. Scenarios where this might be useful and necessary are when we try to release resources ( connections etc.).

Note: When we override default signal handlers, it is important to delegate the behavior to the default handler implementation (Hint: Save the old handler when overriding the same) after we finish the current implementation.


package experiment;

import java.util.HashMap;
import java.util.Map;
import java.util.logging.Logger;

import sun.misc.Signal;
import sun.misc.SignalHandler;

public class CustomSignalHandler implements SignalHandler
{

  private static final Logger               LOGGER   = Logger.getLogger(CustomSignalHandler.class.getName());

  private static Map<Signal, SignalHandler> handlers = new HashMap<Signal, SignalHandler>();


  @Override
  public void handle(Signal signal)
  {
    LOGGER.info("received " + signal);

    // Delegate to the existing handler after handling necessary clean-up.
    handlers.get(signal).handle(signal);
  }


  /**
   * Important: This API is not portable but heavily platform dependent as the signal name depends
   * on the underying operating system. 

   * 
   * @param signalName
   * @param signalHandler
   */
  public static void delegateHandler(final String signalName,
                                     final SignalHandler signalHandler)
  {
    try { 
      Signal signal = new Signal(signalName);
      SignalHandler oldhandler = Signal.handle(signal, signalHandler);
    } finally { 
      handlers.put(signal, oldhandler);
    } 
  }


  public static void main(String[] args)
  {
    final int LONG_TIME = 50000;

    SignalHandler example = new CustomSignalHandler();
    delegateHandler("TERM", example);
    delegateHandler("INT", example);
    delegateHandler("ABRT", example);

    try
    {
      Thread.sleep(LONG_TIME);
    }
    catch (InterruptedException ie)
    {
      ie.printStackTrace();
    }
  }
}

Advanced Linux Programming

2009-07-17T08:35:00.001-07:00

There is a new free e-book on advanced linux programming available here at - http://www.advancedlinuxprogramming.com/alp-folder .

The chapter list is as follows.

Chapter 01 - Advanced Unix Programming with Linux
Chapter 02 - Writing Good GNU/Linux Software
Chapter 03 - Processes
Chapter 04 - Threads
Chapter 05 - Interprocess Communication
Chapter 06 - Mastering Linux
Chapter 07 - The /proc File System
Chapter 08 - Linux System Calls
Chapter 09 - Inline Assembly Code
Chapter 10 - Security
Chapter 11 - A Sample GNU/Linux Application

Among the list of chapters - chapter 7 and chapter 8 to some extent are especially useful and well-written where there are some good hidden treasures even for a seasoned linux professional.

Segmentation Fault - but no core dump ?

2009-07-01T11:48:00.000-07:00

When I was working on a given binary - it gave me a "Segmentation Fault" but no core dump.

Started with the usual suspects .
* Checked the permission of the directory to see if the user has write permission to write the core file. It was ok.
* Checked the /tmp directory just in case.

Then I realized that I was working on bash, that sets core file size to be 0 automatically.

$ ulimit -a
core file size          (blocks, -c)   0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20

..

Reset the same as below

$ ulimit -c unlimited

Now I ran my executable again and yes , this time the core dump is available for processing.

Linux AIO Examples

2009-02-04T11:45:00.001-08:00

Tim Jones gives an excellent introduction and motivation behind aio here. All 2.6 kernels have aio as a standard feature now.

Elastic Block Store

2009-01-29T21:37:00.003-08:00

Amazon comes up with another *potential* low-margin - high volume business by announcing Elastic Block Store today.
The most interesting feature is of course the facility to provide block level storage. The S3 service is already extremely popular - thanks to getting away from the relational model (which sometimes could end up being an overkill ) and reducing the headache of IT management for a potential entrepreneur .

EC2 service, no doubt - is much better to create your own instance of an image from scratch and use it. In spite of having free REST requests to the S3 service , the absence of persistence as such on the image instance was a drawback.

EBS provides us with block-level storage volumes that could be attached to an EC2 instance. As opposed to the other tools that has a much stepper adoption curve ( s3 needs some sort of wrapper around REST - the popular being JetS3t ) - this is probably as simple as it could get and hence it might increase the adoption rate compared to the rest.

I have not got the time to compare the pricing of EBS against the rest, but my guess is that people probably would not mind paying up for this given the level of comfort it gives to making the EC2 instances more usable.

Graphite - Visualization Tool

2009-01-29T21:37:00.001-08:00

Orbitz, the popular travel planning website, had recently brought some of their (previously proprietary) projects into the public domain by making them open source.

Graphite is a scalable, real-time graph visualization tool, released under the Apache License.

Some of the interesting aspects of the same (courtesy, the FAQ of the software):

Written in Python, based on the Django project.
The rendering engine is based on the Cairo framework, the same rendering engine used for the rendering of content in the Firefox 3 browser.
The input data has to be a numeric time series. (This seems intuitive since graph visualization schemes, differences ought to be based on some quantitative measure eventually). And then, of course - any categorical metric could be mapped to preset numerical values to achieve a similar effect.

Graphite, seems to achieve the scalability by storing the entries in a distributed in-memory database, similar to what LiveJournal implements using the memcached service. And more recently, microsoft has started offering Velocity , a competing product in the same space (with subtle differences though- which I will cover later ).

Ct - Programming language for multi-core processor

2009-01-29T21:35:00.001-08:00

With multi-core processors becoming the norm, the responsibility of exploiting the parallelism / improving the performance has increased on the software development rather than the hardware.

Intel has recently come out with a prototype implementation of Ct, a new programming language, for multi-core processors. As per the release notes, the learning curve of Ct is expected to be smoother, as the fundamental language construct seems to be based on the C/C++ programming language, in addition to the language specific features that enable the programmer to refer to parallelism.

A brief introduction to the language construct is available here .

With multi-core systems - it obviously makes more sense to extract / specify data-level parallelism ( + related instructions), as opposed to instruction level parallelism only, to get the best results. New constructs are available in the programming language to specify the same.

Mention of a new Generic Vector Type (TVEC), that exist in the managed space. It is important to note that TVECs could be a flat vector or a multi-dimensional vector.

Restricted operator overloading on TVEC objects, with the important restriction of allowing those with no side-effects.

As a proof of concept, the examples listed in the tutorial talk about the Black-Scholes option pricing model and the Convolution operator (widely applied in Computer Vision / Image processing applications).

I have not been able to confirm if the implementation + runtime is made available to the public yet. One of the components, The Threading building blocks, has been available as a open source project for sometime though.

This interesting release brings some interesting questions.

The last C++ standard (C++03) was written for a single-threaded abstract machine , and threading as yet - is not part of the current C++ standard (current, as supported by the compilers). With fragmented threading libraries across platforms and implementations, portability had always been an issue with threading libraries on C++. But more recently, with Boost Threads providing a nice wrapper over the implementation-specific thread libraries - it is becoming less of a concern. And there is a very good chance that most of these primitives / APIs would be used in the upcoming C++0x standard as well. Given that, the standardization process of introducing thread support into the languages is a little bit late and C++ look-alikes specific to multi-core processors, pushed by the architecture vendor themselves, what would the first choice of technology developers to implement high frequency applications ?

Functions with no-side effects, List Comprehension are all first class citizens, welcome in the Functional Programming world. More specifically, recently , I am fascinated with the Erlang Programming Language with native constructs supporting concurrency (no shared memory, thanks) and based on message passing. So - can the job of extracting better performance from multi-core processors be split between providing a robust interpreter / compiler for the functional programming languages and the functional programming language developer ?

We need to wait and see the way things take shape regarding the above mentioned scenarios.

JMS (ActiveMQ) using Spring

2009-01-27T19:56:00.001-08:00

For a recent project - I wanted to get started with JMS implementations and finally settled on ActiveMQ . I chose the Spring framework because of the range of integration options it gives us with the other parts of the stack.

Here is the sample code fragment using the same. Pre-requisites: Download Apache ActiveMQ 5.2.0 and Spring JMS 2.5.6.A (use ivy from the spring repository to grab the same).

Launch the activemq binary , before running the program below. The binary is usually available in $ACTIVEMQ_HOME/bin/activemq. The binary launches the tcp listening endpoint using the openwire protocol. You will see a line similar to below

INFO  TransportServerThreadSupport   - Listening for connections at: <b>tcp://hostname:61616</b>
INFO  TransportConnector             - Connector <b>openwire</b> Started

package mymq;

import java.io.Serializable;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Session;

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.command.ActiveMQObjectMessage;
import org.springframework.jms.JmsException;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.core.MessageCreator;

public class Producer {

public static class FlyWeight implements Serializable {

public FlyWeight(String _msg) {
msg = _msg;
}

private String msg;

@Override
public String toString() {
return msg;
}
}

/**
* @param args
* @throws JmsException
*/
public static void main(String[] args) {
Producer prod = new Producer();
prod.startProducer();
prod.startConsumer();
}

public void startProducer() {
service.submit(new Runnable() {

public void run() {
try {
JmsTemplate template = new JmsTemplate(getConnectionFactory());
template.afterPropertiesSet();
final DateFormat fmt = new SimpleDateFormat("HH:mm:ss");
while (true) {
Thread.sleep(1000 * 2);
template.send(QUEUE_NAME, new MessageCreator() {

@Override
public Message createMessage(Session session) throws JMSException {
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setObject(new FlyWeight(fmt.format(new Date())));
return msg;
}

});

}
} catch (Exception ex) {

}
}
});
}

public void startConsumer() {
service.submit(new Runnable() {
public void run() {
try {
JmsTemplate template = new JmsTemplate(getConnectionFactory());
template.afterPropertiesSet();
while (true) {
Thread.sleep(1000 * 2);
Message msg = template.receive(QUEUE_NAME);
if (msg instanceof ActiveMQObjectMessage) {
ActiveMQObjectMessage text = (ActiveMQObjectMessage) msg;
System.out.println(text.getObject());
} else {
System.err.println("Message type invalid " + msg.getClass());
}
}
} catch (Exception ex) {

}
}
});
}

static ConnectionFactory getConnectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();
// Default port.
// Important: The script 'activemq' must be launched for this program to
// work
// By default - activemq binds a tcp listener (openwire protocol)
// listening to requests at the same.
factory.setBrokerURL("tcp://localhost:61616");
return factory;
}

static final String QUEUE_NAME = "MyQueue";

static ExecutorService service = Executors.newFixedThreadPool(2);
}

Creating a new user in mysql

2008-12-21T09:09:00.001-08:00

If we want to create a new user in mysql , we can use the following command.



mysql > create database newdb
      > grant all privileges on newdb.* to newuser@"localhost" identified by 'newpassword';

This would create the new user identified by that username and password. This can be validated by logging in again.


mysql > mysql -u newuser -p 
Enter password:

So - now we have created this user and password and tested the same.

Solr - Http Caching enabled by default - how to disable the same.

2008-12-18T15:44:00.001-08:00

Solr is a system , that provides service access (among many other things) to the underlying Lucene implementation and provides a much faster distributed search indexing / retrieval feature.

Much of the configuration in the Solr application is based on solrconfig.xml in the solr.solr.home directory . Among the important set of options - that might be useful for development is ( especially when testing responses over the browser ) could be, disabling http caching so as to not to continue to clear the browser cache before viewing the page.

This is done as follows, by setting the never304 ,attribute to be true for httpCaching as follows.

<httpcaching never304="true" .... />

This should disable httpCaching so that we do not need to refresh the browser. But for the deployment in production - make sure to deploy the same with the property false ,(as it was by default).

JMeter - JUnit Sampler

2008-12-16T11:51:00.001-08:00

JMeter has a JUnit sampler since release . More details are available in the following PDF. I will post an example once I get my first cut example working on the same.

Encryption library for Java - JBCrypt

2008-12-10T20:28:00.001-08:00

I was looking for a good encryption library for Java.

The JBCrypt library (ported from BCrypt, a C++ implementation) is a very useful one for the given purpose.

An introductory article in GWT incubator discusses the plugin in detail.

Lucene 2.4.0 - Hello World

2008-12-09T13:06:00.001-08:00

Lucene 2.4.0 - Hello World application to play around with indexing / searching capabilities of Lucene. The original code is attributed to Lucene tutorial mentioned here.

Some of the API in the code like Hits have been deprecated that creates costly Document objects. The revised code, after addressing compilation warnings is shown herewith.


import java.io.IOException;

import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriter.MaxFieldLength;
import org.apache.lucene.queryParser.ParseException;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.RAMDirectory;

public class HelloLucene {
  public static void main(String[] args) throws IOException, ParseException {
    // 1. create the index
    Directory index = new RAMDirectory();
    IndexWriter w = new IndexWriter(index, new StandardAnalyzer(), true, new MaxFieldLength(25000));
  
    addDoc(w, "Lucene in Action");
    addDoc(w, "Lucene for Dummies");
    addDoc(w, "Managing Gigabytes");
    addDoc(w, "The Art of Computer Science");
    w.close();

    // 2. query
    String querystr = args.length > 0 ? args[0] : "lucene";
    Query q = new QueryParser("title", new StandardAnalyzer()).parse(querystr);

    // 3. search
    IndexSearcher s = new IndexSearcher(index);
    TopDocs docs = s.search(q, null, 100);

    // 4. display results
    System.out.println("Found " + docs.totalHits + " hits.");
    ScoreDoc [] hits = docs.scoreDocs;
    int i = 0;
    for(ScoreDoc scoreDoc : hits) {
      System
.out.println((i + 1) + ". " + s.doc(scoreDoc.doc) );
       ++i;
    }
    s.close();
  }

  private static void addDoc(IndexWriter w, String value) throws IOException {
    Document doc = new Document();
    doc.add(new Field("title", value, Field.Store.YES, Field.Index.ANALYZED));
    w.addDocument(doc);
  }
}

GWT - 1.5.3 - Ubuntu - libstdc++.so.5: cannot open shared object file

2008-12-04T21:53:00.001-08:00

Started my work successfully on GWT 1.5.3 for a test project of mine.

I followed the sample instructions as given in the GWT website.

After using the projectCreator and applicationCreator scripts - when I tried to import the project in eclipse and run it - I encountered the following error trace.
/opt/software/gwt-linux/mozilla-1.7.12/libxpcom.so:libstdc++.so.5: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1751)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1647)
at java.lang.Runtime.load0(Runtime.java:770)
at java.lang.System.load(System.java:1005)
at com.google.gwt.dev.shell.moz.MozillaInstall.load(MozillaInstall.java:190)
at com.google.gwt.dev.BootStrapPlatform.go(BootStrapPlatform.java:40)
at com.google.gwt.dev.GWTShell.main(GWTShell.java:318)

I was working on Ubuntu 8.04.

I did the following

 sudo apt-get install libstdc++5

to get rid of the above mentioned error to install the missing .so files. That fixed the issue.

Boost MPI implementation

2008-02-01T12:10:00.000-08:00

Boost got a new MPI approved in its vast list of libraries -
Boost MPI.

Boost MPI is the way to go if we are planning to use it for high frequency applications.

Alignment

2008-01-29T22:41:00.000-08:00

Sometimes when we are coding on architectures with limited memory footprints , we might end up writing a memory allocator ourselves (more often than not ). Having said that - one of the important design considerations for a memory allocator is getting an address that is probably aligned with the underlying architecture.

Here is the simple code fragment to do the alignment for a given size.
Note that - the align function returns a new address only when the input address is not aligned by itself.


#include <stdlib.h>
#include <stdio.h>

/* align address to be a multiple of n. */
void* align(void* memory, int alignment)
{
 unsigned char* ptr;
 unsigned int offset;

 ptr =(unsigned char*)memory;
 offset =(unsigned int)ptr % alignment;

 return offset ? (ptr + alignment - offset ): ptr;
} 



int main()
{
 void * a;

 a = malloc(sizeof(char) * 10);

 printf("\nAddress: 0x%p ", a ); 
 printf("\nAligned Address:0x%p ",  align(a, sizeof(int) ));
 return EXIT_SUCCESS;
}

C++ STL list with logarithmic lookup ??

2008-01-24T21:17:00.000-08:00

If we talk about standard C++ SQL (C++03, at least) looking for a data structure to simulate a list with logarithmic lookup, the question will appear to be contradictory to begin with.

If we are looking for a data structure whose access pattern is different from that of the underlying storage mechanism - Boost MultiIndex is an option worth spending at.

C++ - atexit implementation

2008-01-23T07:48:00.000-08:00

More than one occasion - when a program exits - we may want to do some cleanup operations and notify other processes ( this becomes extremely important in the case of child-parent processes).

C/C++ does have a function called atexit() that does that.

A sample code fragment to use the same is as follows.


#include <cstdlib>
#include <iostream>

using namespace std;

void do1()
{
 std::cout << "do1 implementation\n";
}

void do2()
{
 std::cout << "do2 implementation\n";
}


int main()
{
 atexit(do1);
 atexit(do2);

 exit(EXIT_SUCCESS);
}

Sample output:


$ g++ -Wall -Werror testexit.cpp
[noname@localhost test]$ ./a.out
do2 implementation
do1 implementation

Tutorial on Boost Threads

2008-01-21T17:54:00.001-08:00

If you are looking for a tutorial on Boost Threads, start here .

Books on Design Patterns for concurrent / networked objects

2008-01-09T15:07:00.000-08:00

One of the interesting books on concurrent / networked objects is -
Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects

Boost ASIO library for high performance networking servers

2008-01-09T12:14:00.000-08:00

If anybody is venturing into developing networking libraries in C++ , the ACE framework is the one that strikes your mind first.

Recently came across the library called boost asio - available here .

Boost ASIO definitely makes things much easy and portable. If the application we are developing needs to be truly portable - the fact that asio is part of the boost framework guarantees portability by definition ( across a large number of compilers across many platforms).

There is also an interesting article on the design patterns to be used by the boost library available here.

That makes interesting reading for sure.

Checking if a process is alive or not.

2007-12-26T17:00:00.000-08:00

In a multi-processing system - if we want to check if a given child process is alive or not (after forking it) for a longer period of time - we could send a signal 0 to the process using the kill API.


   #include <sys/types.h>
    #include <signal.h>

    int kill(pid_t pid, int sig);

Use this API to send a signal of 0. If the errno is set to ESRCH - then the process had most probably exited.

Probability

2007-12-24T00:42:00.000-08:00

Interesting article about probability here.

Want to trace function call stack on Linux

2007-12-21T14:29:00.000-08:00

If you want to trace function call stack on a Linux box - then we can use backtrace family of functions on GNU/Linux.

For more details regarding the API and examples - refer here.

-fpic OR -fPIC

2007-12-19T15:05:00.000-08:00

When preparing dynamic libraries on Linux, there is always a choice about selecting -fpic / -fPIC.

The difference lies mostly on RISC architectures. -fpic seems to take lesser number of instructions as compared to -fPIC .

For more details - refer to Section 2, in the document - "How to write Shared Libraries" by Ulrich Drepper.

How to write shared libraries

2007-12-19T11:30:00.000-08:00

Ulrich Drepper got this nice PDF document, explaining how to write shared libraries on Linux. A must read before anyone ventures onto writing one.

Resolving unresolved symbol error in Linux shared libraries

2007-12-18T19:01:00.000-08:00

Frequently we used to come across a case in which when we create shared libraries on a Linux box and try dynamic loading using 'dlsym' methods - the load fails with unresolved symbols error.

InformIT hosts a wonderful article on the different techniques on resolving these errors.

Option 1:

"To generate an executable that exports all of its functions and variables dynamically, specify the compiler flag -Wl,-export-dynamic when you link your executable"

Option 2:

"Your main executable may load other shared libraries at runtime, using dlopen. By default, they cannot see each other's dynamic symbols. You can make these libraries' symbols visible to each other by specifying the bitwise or of the value RTLD_GLOBAL in the second argument to dlopen."

Specifically I like Option 1 as it makes me aware of the linking going. RTLD_GLOBAL seems too generic to me.

Why gethostbyname is evil

2007-12-01T13:28:00.000-08:00

Ulrich Drepper discusses why gethostbyname is evil in his post.

Moral of the story: Use gethostbyaddr even if we are not talking about IPv6 .

Robust Futex

2007-12-01T13:16:00.000-08:00

A wonderful commentary on robust futexes by Ingo Molnar available here