Archive for January, 2010

Building a cluster map using CartographerJS

Behold the Canadian NoProrogue Protests as a Cluster Map:

Today I’m building a cluster map using the way cool CartographerJS thematic JavaScript library. The library greatly simplifies the creation of “heat map” style overlays for Google Maps. In order to follow along, you will need to be familiar with GMap2 and JavaScript.

To create a cluster map, you need to include a few libraries:

  1. Google Maps GMap2 API – Makes maps from divs
  2. RaphaelJS – Library to draw vector graphics
  3. CartographerJS

Download Raphael and Cartographer somewhere webby and then include them in the head of your html page with your GMap API key URL (Google provides this):

10
11
12
  <script src="http://maps.google.com/maps?file=api&amp;v=2&amp;sensor=false&amp;key=ABQIAAAAkWkoIqBrrWjZm_w3j6xq2hSa3-TS0yNMl32jU2eh6AwgDkL60hSCs7jQoaWypWYc5VCS-RKonhKskg" type="text/javascript"></script>
  <script type="text/javascript" language="javascript" src="raphael-min.js"></script>
  <script type="text/javascript" language="javascript" src="cartographer.min.0.3.js"></script>

The data that we are going to plot comes from the NoProrogue.ca protest actions that took place on 23 Jan 2010 across Canada. This dataset was converted from the early estimates produced by Ian Capstick in his Globe and Mail article.

9
<script type="text/javascript" language="javascript" src="http://www.kelvinwong.ca/code/html/protest_maps/protest_data.js"></script>

The data is a simple array packed with object literals. The important parts of the object literals are: the latitude of the point (lat), the longitude of the point (lng), the value to use when plotting (val) and the label to use on the callout (label):

1
2
3
4
var protest_data = [
{lat:43.656076, lng:-79.380279, val:9000, label:"Toronto: 9000, Police place at 7000 - organizers claim 15,000"},
// Other object literals here!!!
];

If you tried out the Hello World demo from the GMap2 documentation, then this code should look familiar. The plan is to place an empty div on the page and fill it with a map and an overlay dynamically written with JavaScript. Here is the target div:

16
17
<body onload="initialize()" onunload="GUnload()">
<div id="map" style="width: 650px; height: 450px"></div>

And here is the code that draws the map then places the map object into the cartographer object:

19
20
21
22
23
24
25
26
27
28
29
30
31
function initialize() {
  if (GBrowserIsCompatible()) {
    var map = new GMap2(document.getElementById("map"));
    map.setCenter(new GLatLng(61.856149,-95.625), 3);
    map.setUIToDefault();
    var mapoptions = { colorize:"#fff",
                    colorizeAlpha:0.1
    };
    var cartographer = Cartographer( map, mapoptions );
    var options = { color:"red", enableGrid:false };
    cartographer.cluster( protest_data, options );
  }
}

The mapoptions object literal specifies the colour of the overlay (colorize) and the opacity of the overlay (colorizeAlpha). If the opacity is set to 1.0 you won’t be able to see the map underneath, so set it to a sane value like 0.1 or 0.3. We pass that object literal and the Google map object as parameters to initialize the cartographer object.

The options object literal is used to construct a cluster map on the overlay. In this case I wanted to specify the colour of the data points (color) and to disable the grid (enableGrid). There are many other settings you can specify based on your own needs. When you are ready to draw your overlay, pass the options object and the data as parameters into the cluster method of the cartographer object. Ecco fatto!

You can view the file without the iframe!

Tags: , , , , , , , ,

Saturday night compilation

It’s another Saturday night at 9pm. Time to compile some code, right? Right?

So, before I start doing that, I’m on IM chatting with an old friend of mine in Toronto. He’s like: “What are you doing sitting in front of your computer on a Saturday night? HUH?”

Without blinking: “I’m building Cassandra this newish database thing…blah blah blah…”

So yeah he’s busy ignoring me by the time I get around to typing about Thrift. His point is that I’m playing with computers on a Saturday night, but on the other hand, he’s working on a press deadline (and it’s midnight in Toronto). What’s worse? I donno.

Tags: , ,

Running Cassandra 0.5.0 on Mac OS X Leopard 10.5.8

This is a short post showing you how to install the superdooper Key-Value Store Cassandra version 0.5.0 on a Macbook running OS X Leopard 10.5.8 (yeah I know, I should upgrade to Snow Leopard sometime). Cassandra 0.5 promises to be peppier than the 0.4 version and at this point it hasn’t been released yet, but I’m impatient, so I’m setting up a ‘one-node cluster’ for fun.

Cassandra 0.5.0 was released on 25 Jan 2010.

You will need to ensure that you are running Java 1.6. Since Java 1.5 is the default on Mac OS X, you need to set up a couple of environment variables. I just put the following into my .bash_profile file using vim.

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export PATH=$JAVA_HOME/bin:$PATH

Now source the file (if you haven’t already done that) and examine your set-up.

Trinity:~ kelvin$ source .bash_profile
Trinity:~ kelvin$ env | grep JAVA_HOME
JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
Trinity:~ kelvin$ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
Trinity:~ kelvin$ xcodebuild -version
Xcode 3.1.2
Component versions: DevToolsCore-1148.0; DevToolsSupport-1102.0
BuildVersion: 9M2621

Yeah, my xcode is a bit old too. So, once that is done, download Cassandra. I am going to use the tag in the Subversion repository for the 0.5.0 release.

Trinity:code kelvin$ svn co https://svn.apache.org/repos/asf/incubator/cassandra/tags/cassandra-0.5.0/ cassandra-0.5.0

...svn key gibberish redacted...

A    cassandra-0.5.0/NOTICE.txt
 cassandra-0.5.0/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
...checkout redacted...
A    cassandra-0.5.0/build.xml
 U   cassandra-0.5.0
Checked out revision 902040.
Trinity:code kelvin$ cd cassandra-0.5.0
Trinity:cassandra-0.5.0 kelvin$ pwd
/Users/kelvin/code/cassandra-0.5.0
Trinity:cassandra-0.5.0 kelvin$

Cassandra expects some folders to be available, so we will need to set them up (per the instructions in the README file). First of all, check the file conf/storage-conf.xml and make sure that these folders exist before you do anything else.


  <CommitLogDirectory>/var/lib/cassandra/commitlog</CommitLogDirectory>
  <DataFileDirectories>
      <DataFileDirectory>/var/lib/cassandra/data</DataFileDirectory>
  </DataFileDirectories>
  <CalloutLocation>/var/lib/cassandra/callouts</CalloutLocation>
  <StagingFileDirectory>/var/lib/cassandra/staging</StagingFileDirectory>

If they do not exist, you will have to make them (careful with the whoami command in backticks):

Trinity:cassandra-0.5.0 kelvin$ sudo mkdir -p /var/log/cassandra
Password:
Trinity:cassandra-0.5.0 kelvin$ sudo chown -R `whoami` /var/log/cassandra
Trinity:cassandra-0.5.0 kelvin$ sudo mkdir -p /var/lib/cassandra
Trinity:cassandra-0.5.0 kelvin$ sudo chown -R `whoami` /var/lib/cassandra

For logging, Cassandra expects that a system.log file is available. Check the conf/log4j.properties file to check where it expects a log file. It should be similar to this:

log4j.appender.R.File=/var/log/cassandra/system.log

When you start Cassandra for the first time it will start the log file.

I used ‘ant‘ to build Cassandra. Make sure that you are in the root of the Cassandra directory. I have a lot of crap in my Java classpath, so I will temporarily exclude it.

Trinity:cassandra-0.5.0 kelvin$ ant -noclasspath
Buildfile: build.xml

build-subprojects:

init:
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/build/classes
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/build/test/classes
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/src/gen-java

check-gen-cli-grammar:

gen-cli-grammar:
     [echo] Building Grammar /Users/kelvin/code/cassandra-0.5.0/src/java/org/apache/cassandra/cli/Cli.g  ....

build-project:
     [echo] apache-cassandra-incubating: /Users/kelvin/code/cassandra-0.5.0/build.xml
    [javac] Compiling 247 source files to /Users/kelvin/code/cassandra-0.5.0/build/classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

build:

BUILD SUCCESSFUL
Total time: 13 seconds
Trinity:cassandra-0.5.0 kelvin$

Now the moment of truth!

Trinity:cassandra-0.5.0 kelvin$ bin/cassandra -f
Listening for transport dt_socket at address: 8888
INFO - Saved Token not found. Using 25907362644306974147256376871662350143
INFO - Starting up server gossip
INFO - Cassandra starting up...

Okay, Cassandra is now waiting for us to do something, so we can open up another Terminal window (Cmd+N) and try the test described in the README file:

Trinity:cassandra-0.5.0 kelvin$ bin/cassandra-cli --host localhost --port 9160
Connected to localhost/9160
Welcome to cassandra CLI.

Type 'help' or '?' for help. Type 'quit' or 'exit' to quit.
cassandra> set Keyspace1.Standard1['jsmith']['first'] = 'John'
Value inserted.
cassandra> set Keyspace1.Standard1['jsmith']['last'] = 'Smith'
Value inserted.
cassandra> set Keyspace1.Standard1['jsmith']['age'] = '42'
Value inserted.
cassandra> get Keyspace1.Standard1['jsmith']
=> (column=last, value=Smith, timestamp=1264159272089)
=> (column=first, value=John, timestamp=1264159260000)
=> (column=age, value=42, timestamp=1264159282593)
Returned 3 results.
cassandra>

Okay, we now have stored and retrieved some data. In the next installment I’ll try to get Thrift running!

Tags: , , , , , ,