Resetting / Overwriting /etc/resolv.conf in EC2 instance

Creates a new EC2 instance ( CentOS ) and by default it comes with a dns resolver to the outside world.

For our purposes - we had set up an internal dns server running (bind - process 'named' on a particular host).

When we were launching our pool of servers - we wanted to make sure that the new instances fall under the same domain that we specify it to be.

We also wanted to set the nameserver of the newly created instances pointing to the internal DNS server we have to resolve the ambiguity we have.

Before making the change - the file /etc/resolv.conf was looking as follows.

# cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search some.internal.aws.domain
nameserver 172.x.y.z 

where 172.x.y.z was something set by Amazon EC2 by default.

Edit /etc/dhclient.conf ( Create one , if it does not exist )

supersede  domain-name "ec2.mycompany.com" ;
prepend domain-name-servers 10.p.q.r ;my internal company dns

I also found it useful to create an Elastic IP and associate the dns server instance with the elastic ip,
and then have the prepend domain-name-servers refer to the elastic ip , instead of the internal ip.
So - even if the internal dns server fails = we can reconstruct it from another AMI and associate with the elastic ip without affecting the rest of the system.

To see the changes to /etc/dhclient , do the following

$ dhclient -r ; dhclient 

This command forcibly renews the dhcp lease that will force the new credentials from /etc/dhclient.conf to be read.

After doing this , verify /etc/resolv.conf

search ec2.mycompany.com
nameserver 10.p.q.r    ; our DNS server
nameserver 172.x.y.z  ; the one set by amzn

So that completes the process and we are good about it.

Zookeeper build

Was trying to build Zookeeper from trunk ( 3.3.0 ) .

Came across this error  -

create-cppunit-configure:
     [exec] configure.ac:33: warning: macro `AM_PATH_CPPUNIT' not found in library
     [exec] libtoolize: putting auxiliary files in `.'.
     [exec] libtoolize: copying file `./config.guess'
     [exec] libtoolize: copying file `./config.sub'
     [exec] libtoolize: copying file `./install-sh'
     [exec] libtoolize: copying file `./ltmain.sh'
     [exec] libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and
     [exec] libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
     [exec] libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
     [exec] configure.ac:33: warning: macro `AM_PATH_CPPUNIT' not found in library
     [exec] configure.ac:33: error: possibly undefined macro: AM_PATH_CPPUNIT
     [exec]       If this token and others are legitimate, please use m4_pattern_allow.
     [exec]       See the Autoconf documentation.
     [exec] autoreconf: /usr/bin/autoconf failed with exit status: 1

Installed -

$  sudo apt-get install libcppunit-dev

Ended up with ..

     [exec] .../zookeeper/src/c/configure: line 5015: syntax error near unexpected token `1.10.2'
     [exec] .../zookeeper/src/c/configure: line 5015: `   AM_PATH_CPPUNIT(1.10.2)'

Hmm.. bad times.

Installing Thrift Continued

Continuing my previous post of installing Thrift - needed to install the following obvious ones as well.

 sudo apt-get install flex bison 

And then , run ./configure once again to make sure all the libraries are linked together.

 ./configure

This should regenerate the makefiles once again, after installing flex and bison.

Installing Thrift

Checked out thrift from the trunk on my ubuntu box ( 9.10 ) .

The first step on the installation was running the program

$ ./bootstrap.sh

It failed with some errors , some fairly obvious (missing autoconf ) - some not so obvious.
The first step is to get autoconf installed.

$ sudo apt-get install autoconf

Then ran into this error.

$ ./bootstrap.sh
configure.ac:44: error: possibly undefined macro: AC_PROG_LIBTOOL
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:26: installing `./install-sh'
configure.ac:26: installing `./missing'
compiler/cpp/Makefile.am: installing `./depcomp'
configure.ac: installing `./ylwrap'
lib/cpp/Makefile.am:24: Libtool library used but `LIBTOOL' is undefined
lib/cpp/Makefile.am:24:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
lib/cpp/Makefile.am:24:   to `configure.ac' and run `aclocal' and `autoconf' again.
lib/cpp/Makefile.am:24:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
lib/cpp/Makefile.am:24:   its definition is in aclocal's search path.
test/Makefile.am:30: Libtool library used but `LIBTOOL' is undefined
test/Makefile.am:30:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
test/Makefile.am:30:   to `configure.ac' and run `aclocal' and `autoconf' again.
test/Makefile.am:30:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
test/Makefile.am:30:   its definition is in aclocal's search path.

The fix was to install libtool.
$ sudo apt-get install libtool


And then comes the boost libraries -

$ sudo apt-get install libboost1.40-dev libboost1.40-doc

(Your boost library version might be different from mine but you get the idea !!).

HDFS Permissions issues

If you are running into permissions issue in hdfs installation (Eg: When you try to write to hdfs and that does not seem to work ) - then you may want to *relax* the permissions by the following setting in all hdfs cluster nodes , in hdfs-site.xml ( starting from 0.20 ). Of course - the namednode needs to be restarted for the changes to be effective.

<property>
<name>dfs.permissions</name>
<value>false</value>
</property>

Important - this has a gaping security hole in itself by relaxing the permissions and currently the hdfs team is actively working on enabling better permission based access rules. So - this change is best in the early stages of development to get started and should be revisited once again soon after.

iBator eclipse plugin

If you are like me and you are into using iBatis eclipse plugin - check out http://ibatis.apache.org/ibator.html. It comes with an eclipse plugin that is quite useful if you are working with the ibatis configuration files.

Handling signals in Java

Signals by definition are specific to the underlying implementation ( read, operating system) .

Java, being a platform independent language , it is often discouraged to write platform specific stuff except for the rarest cases and when there is a clear business justification for the same.

Java does have an undocumented API that talks about signals where we can override the default signal handler for a given signal.  Scenarios where this might be useful and necessary are when we try to release resources ( connections etc.).

Note: When we override default signal handlers, it is important to delegate the behavior to the default handler implementation (Hint: Save the old handler when overriding the same) after we finish the current implementation.

package experiment;

import java.util.HashMap;
import java.util.Map;
import java.util.logging.Logger;

import sun.misc.Signal;
import sun.misc.SignalHandler;

public class CustomSignalHandler implements SignalHandler
{

  private static final Logger               LOGGER   = Logger.getLogger(CustomSignalHandler.class.getName());

  private static Map<Signal, SignalHandler> handlers = new HashMap<Signal, SignalHandler>();


  @Override
  public void handle(Signal signal)
  {
    LOGGER.info("received " + signal);

    // Delegate to the existing handler after handling necessary clean-up.
    handlers.get(signal).handle(signal);
  }


  /**
   * Important: This API is not portable but heavily platform dependent as the signal name depends
   * on the underying operating system. 

   * 
   * @param signalName
   * @param signalHandler
   */
  public static void delegateHandler(final String signalName,
                                     final SignalHandler signalHandler)
  {
    try { 
      Signal signal = new Signal(signalName);
      SignalHandler oldhandler = Signal.handle(signal, signalHandler);
    } finally { 
      handlers.put(signal, oldhandler);
    } 
  }


  public static void main(String[] args)
  {
    final int LONG_TIME = 50000;

    SignalHandler example = new CustomSignalHandler();
    delegateHandler("TERM", example);
    delegateHandler("INT", example);
    delegateHandler("ABRT", example);

    try
    {
      Thread.sleep(LONG_TIME);
    }
    catch (InterruptedException ie)
    {
      ie.printStackTrace();
    }
  }
}