Elastic Block Store

Amazon comes up with another *potential* low-margin - high volume business by announcing Elastic Block Store today.
The most interesting feature is of course the facility to provide block level storage. The S3 service is already extremely popular - thanks to getting away from the relational model (which sometimes could end up being an overkill ) and reducing the headache of IT management for a potential entrepreneur .

EC2 service, no doubt - is much better to create your own instance of an image from scratch and use it. In spite of having free REST requests to the S3 service , the absence of persistence as such on the image instance was a drawback.

EBS provides us with block-level storage volumes that could be attached to an EC2 instance. As opposed to the other tools that has a much stepper adoption curve ( s3 needs some sort of wrapper around REST - the popular being JetS3t ) - this is probably as simple as it could get and hence it might increase the adoption rate compared to the rest.

I have not got the time to compare the pricing of EBS against the rest, but my guess is that people probably would not mind paying up for this given the level of comfort it gives to making the EC2 instances more usable.

Graphite - Visualization Tool

Orbitz, the popular travel planning website, had recently brought some of their (previously proprietary) projects into the public domain by making them open source.

Graphite is a scalable, real-time graph visualization tool, released under the Apache License.

Some of the interesting aspects of the same (courtesy, the FAQ of the software):

• Written in Python, based on the Django project.
  
• The rendering engine is based on the Cairo framework, the same rendering engine used for the rendering of content in the Firefox 3 browser.
  
• The input data has to be a numeric time series. (This seems intuitive since graph visualization schemes, differences ought to be based on some quantitative measure eventually). And then, of course - any categorical metric could be mapped to preset numerical values to achieve a similar effect.

Graphite, seems to achieve the scalability by storing the entries in a distributed in-memory database, similar to what LiveJournal implements using the memcached service. And more recently, microsoft has started offering Velocity , a competing product in the same space (with subtle differences though- which I will cover later ).

Ct - Programming language for multi-core processor

With multi-core processors becoming the norm, the responsibility of exploiting the parallelism / improving the performance has increased on the software development rather than the hardware.

Intel has recently come out with a prototype implementation of Ct, a new programming language, for multi-core processors. As per the release notes, the learning curve of Ct is expected to be smoother, as the fundamental language construct seems to be based on the C/C++ programming language, in addition to the language specific features that enable the programmer to refer to parallelism.

A brief introduction to the language construct is available here .

With multi-core systems - it obviously makes more sense to extract / specify data-level parallelism ( + related instructions), as opposed to instruction level parallelism only, to get the best results. New constructs are available in the programming language to specify the same.

• Mention of a new Generic Vector Type (TVEC), that exist in the managed space. It is important to note that TVECs could be a flat vector or a multi-dimensional vector.


• Restricted operator overloading on TVEC objects, with the important restriction of allowing those with no side-effects.

As a proof of concept, the examples listed in the tutorial talk about the Black-Scholes option pricing model and the Convolution operator (widely applied in Computer Vision / Image processing applications).


I have not been able to confirm if the implementation + runtime is made available to the public yet. One of the components, The Threading building blocks, has been available as a open source project for sometime though.


This interesting release brings some interesting questions.

• The last C++ standard (C++03) was written for a single-threaded abstract machine , and threading as yet - is not part of the current C++ standard (current, as supported by the compilers). With fragmented threading libraries across platforms and implementations, portability had always been an issue with threading libraries on C++. But more recently, with Boost Threads providing a nice wrapper over the implementation-specific thread libraries - it is becoming less of a concern. And there is a very good chance that most of these primitives / APIs would be used in the upcoming C++0x standard as well. Given that, the standardization process of introducing thread support into the languages is a little bit late and C++ look-alikes specific to multi-core processors, pushed by the architecture vendor themselves, what would the first choice of technology developers to implement high frequency applications ?


• Functions with no-side effects, List Comprehension are all first class citizens, welcome in the Functional Programming world. More specifically, recently , I am fascinated with the Erlang Programming Language with native constructs supporting concurrency (no shared memory, thanks) and based on message passing. So - can the job of extracting better performance from multi-core processors be split between providing a robust interpreter / compiler for the functional programming languages and the functional programming language developer ?

We need to wait and see the way things take shape regarding the above mentioned scenarios.

JMS (ActiveMQ) using Spring

For a recent project - I wanted to get started with JMS implementations and finally settled on ActiveMQ . I chose the Spring framework because of the range of integration options it gives us with the other parts of the stack.

Here is the sample code fragment using the same. Pre-requisites: Download Apache ActiveMQ 5.2.0 and Spring JMS 2.5.6.A (use ivy from the spring repository to grab the same).

Launch the activemq binary , before running the program below. The binary is usually available in $ACTIVEMQ_HOME/bin/activemq. The binary launches the tcp listening endpoint using the openwire protocol. You will see a line similar to below

INFO  TransportServerThreadSupport   - Listening for connections at: <b>tcp://hostname:61616</b>
INFO  TransportConnector             - Connector <b>openwire</b> Started

package mymq;

import java.io.Serializable;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Session;

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.command.ActiveMQObjectMessage;
import org.springframework.jms.JmsException;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.core.MessageCreator;

public class Producer {

public static class FlyWeight implements Serializable {

public FlyWeight(String _msg) {
msg = _msg;
}

private String msg;

@Override
public String toString() {
return msg;
}
}

/**
* @param args
* @throws JmsException
*/
public static void main(String[] args) {
Producer prod = new Producer();
prod.startProducer();
prod.startConsumer();
}

public void startProducer() {
service.submit(new Runnable() {

public void run() {
try {
JmsTemplate template = new JmsTemplate(getConnectionFactory());
template.afterPropertiesSet();
final DateFormat fmt = new SimpleDateFormat("HH:mm:ss");
while (true) {
Thread.sleep(1000 * 2);
template.send(QUEUE_NAME, new MessageCreator() {

@Override
public Message createMessage(Session session) throws JMSException {
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setObject(new FlyWeight(fmt.format(new Date())));
return msg;
}

});

}
} catch (Exception ex) {

}
}
});
}

public void startConsumer() {
service.submit(new Runnable() {
public void run() {
try {
JmsTemplate template = new JmsTemplate(getConnectionFactory());
template.afterPropertiesSet();
while (true) {
Thread.sleep(1000 * 2);
Message msg = template.receive(QUEUE_NAME);
if (msg instanceof ActiveMQObjectMessage) {
ActiveMQObjectMessage text = (ActiveMQObjectMessage) msg;
System.out.println(text.getObject());
} else {
System.err.println("Message type invalid " + msg.getClass());
}
}
} catch (Exception ex) {

}
}
});
}

static ConnectionFactory getConnectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();
// Default port.
// Important: The script 'activemq' must be launched for this program to
// work
// By default - activemq binds a tcp listener (openwire protocol)
// listening to requests at the same.
factory.setBrokerURL("tcp://localhost:61616");
return factory;
}

static final String QUEUE_NAME = "MyQueue";

static ExecutorService service = Executors.newFixedThreadPool(2);
}