[java] archive category

Multi-site Solr for Drupal Notes

Last month I published an article on setting up Solr web search for multiple Drupal web sites. Since then I have received several emails about the set-up. Here are some of my replies:

Can I set up Solr on a separate machine?

Yes. To do that you will have to open the port on your iptables (if it is running). Then configure Tomcat to listen on that address and port in your configuration file. Then you have to set up authentication (probably just basic authentication using a Realm) for your Context.

Why not just use multi-core Solr?

You can and should use multi-core Solr if you are setting up commercial hosting and you have the technical knowledge to set-up separate Solr cores. Multi-core Solr would likely use less resources too. The set-up I describe works fine for a few Drupal web sites and can be maintained using files provided by the project maintainers. If you are only supporting a few sites (like maybe 3) I wouldn’t bother trying anything more complex.

Sun JVM works better

I have heard that Sun’s JVM works better than OpenJDK when running Solr but I haven’t had any problems running OpenJDK. I have been running a couple of small Solr instances for over a year on OpenJDK and Tomcat 5.5 without any issues related to the JVM.

Can I use Solr version X (instead of 1.4.X)?

I have no idea. ApacheSolr recommends Solr 1.4.x in the release notes and unless you need additional Solr features only available in later versions, I recommend that you stick with the supported version.

Will this work on my busy web site?

Define “busy“.

Drupal is very flexible and if you mostly publish content and occasionally your users search for stuff, then the demands on your search service will be minimal and my set-up would probably work fine.

If you run a search service and index other websites by scraping and republishing scraped content in Drupal, then your search is going to get a lot more use since it is the primary feature. In that case, you would probably be better off running Solr on a dedicated machine (or machines).

Why is my Tomcat on port 8983?

Jetty is likely listening on port 8983 (were you following the Solr tutorial and left it running?). Make sure that you are running Tomcat and that it is listening on port 8080.

Drupal: “Your site was unable to contact the Apache Solr server”

Like it says, your Drupal install couldn’t find the Solr server. Check your Drupal settings. If they are not exactly right you will see this error. Check your system using “netstat -anp” and “lsof -i” run as root. Make sure that Tomcat is listening on port 8080. Check your logs.

Why doesn’t my set-up work?

There are a lot of reasons why your Solr set-up isn’t working. UNIX is very good at telling you when things go wrong by writing everything in its logs. Check your SELinux settings using sestatus. Check your audit logs. Check your web server logs. Check your catalina.out log. Try Googling the error codes or the main body of the error message.

Feedback welcome

I’m always glad to hear from readers via comments here on the blog, or by trackbacks or via email.

Tags: , , , ,

Multi-Site Solr for Drupal 6 Search on Tomcat 6 / CentOS 6

ApacheSolr for Drupal 6 improves on the out-of-the-box search experience for Drupal users. The easiest way to get Solr running on your Drupal web site is to use the hosted service provided by Acquia; it is way easier than running your own Solr. You simply point your queries to their Solr server and you’re done.

For various reasons, you might want to run your own Solr web service on your own machine. In this article, I will walk you through setting up a working Solr installation using Tomcat 6 on CentOS 6. The end result of this walkthrough will be two separate Solr indexes (via two separate Solr web apps) for two different web sites running on a single Tomcat. I will assume that you are using Acquia’s Drupal (which ships with SolrPHPClient).

Warning: This article assumes all services are on a single machine (suitable for a small organization). Running Solr on a separate machine is possible but raises security implications that are outside the scope of this article.

These are the tasks that we will work on:

  1. Set-up Solr
  2. Set-up Tomcat
  3. Tweak CentOS security thinger (SELinux)
  4. Configure Acquia Drupal

Prerequisites

The prerequisites are:

  • CentOS 6 Web Server w/ PHP 5.3, MySQL 5, Tomcat 6, Java 6 (all services running w/ no problemos)
  • Acquia Drupal 6 installed
  • Familiarity with Drupal (basic skills – enabling modules, setting permissions on nodes, etc)
  • Familiarity with Java & Tomcat (basic skills)
  • Familiarity working with Linux in a terminal and vi (intermediate skills)

This is my system (a web server set-up with Anaconda):

# uname -a
Linux templeton.localdomain 2.6.32-71.29.1.el6.i686 #1 SMP Mon Jun 27 18:07:00 BST 2011 i686 i686 i386 GNU/Linux
# cat /etc/redhat-release
CentOS Linux release 6.0 (Final)
# yum list installed | grep mysql-server
mysql-server.i686       5.1.52-1.el6_0.1  @updates
# yum list installed | grep php
php.i686                5.3.2-6.el6_0.1   @updates                              
php-cli.i686            5.3.2-6.el6_0.1   @updates                              
php-common.i686         5.3.2-6.el6_0.1   @updates                              
php-gd.i686             5.3.2-6.el6_0.1   @updates                              
php-mysql.i686          5.3.2-6.el6_0.1   @updates                              
php-pdo.i686            5.3.2-6.el6_0.1   @updates                              
php-pear.noarch         1:1.9.0-2.el6     @anaconda-centos-201106051823.i386/6.0
php-xml.i686            5.3.2-6.el6_0.1   @updates
# java -version
java version "1.6.0_17"
OpenJDK Runtime Environment (IcedTea6 1.7.5) (rhel-1.31.b17.el6_0-i386)
OpenJDK Client VM (build 14.0-b16, mixed mode)
# yum list installed | grep tomcat6
tomcat6.noarch          6.0.24-24.el6_0   @updates                              
tomcat6-el-2.1-api.noarch
tomcat6-jsp-2.1-api.noarch
tomcat6-lib.noarch      6.0.24-24.el6_0   @updates                              
tomcat6-servlet-2.5-api.noarch
# /sbin/service tomcat6 status
tomcat6 (pid 1790) is running...                           [  OK  ]
# sestatus
SELinux status:                 enabled
SELinuxfs mount:                /selinux
Current mode:                   enforcing
Mode from config file:          enforcing
Policy version:                 24
Policy from config file:        targeted

Notice the hashmark (#) as my terminal prompt. It denotes that I am executing all these commands as root (use ‘su -‘). You can also prefix the following commands with ‘sudo’.

Download Solr

Obtain a copy of the Solr tarball from a nearby mirror:

http://www.apache.org/dyn/closer.cgi/lucene/solr/

Select Solr 1.4.1 or the latest recommended Solr:

ie. http://apache.sunsite.ualberta.ca//lucene/solr/1.4.1/

I’m using the 54M GZipped Tarball and downloading it using wget:

# wget http://apache.sunsite.ualberta.ca//lucene/solr/1.4.1/apache-solr-1.4.1.tgz
--2011-09-02 02:06:05--  http://apache.sunsite.ualberta.ca//lucene/solr/1.4.1/apache-solr-1.4.1.tgz
Resolving apache.sunsite.ualberta.ca... 129.128.5.190
Connecting to apache.sunsite.ualberta.ca|129.128.5.190|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 56374837 (54M) [application/x-tar]
Saving to: “apache-solr-1.4.1.tgz”

100%[=============================================>] 56,374,837   261K/s   in 6m 20s  

2011-09-02 02:12:42 (145 KB/s) - “apache-solr-1.4.1.tgz” saved [56374837/56374837]

# tar zxvf apache-solr-1.4.1.tgz
apache-solr-1.4.1/client/
...
# pwd
/root

Copy the Solr package somewhere reasonable like in the /opt folder:

# mkdir -p /opt/solr
# cp -r -p /root/apache-solr-1.4.1 /opt/solr

Link it (the Solr WAR file) to the Tomcat library directory:

# ln -s /opt/solr/apache-solr-1.4.1/dist/apache-solr-1.4.1.war /usr/share/tomcat6/lib/solr.war

In the future, when you upgrade your software, install the Solr upgrade and update the symlink.

Create Solr directories

You need to choose where your Solr indexes will be kept. I put them into the /var directory and that’s where I’m assuming that you will put yours:

# mkdir -p /var/solr
# cp -r -p /opt/solr/apache-solr-1.4.1/example/solr/ /var/solr/
# mv /var/solr/solr /var/solr/example.com
# ls -l /var/solr/example.com/
total 12
drwxr-xr-x. 2 root root 4096 Sep  2 02:44 bin
drwxr-xr-x. 3 root root 4096 Sep  2 02:44 conf
-rw-r--r--. 1 root root 2259 Sep  2 02:44 README.txt

Each domain has its own Solr indexes located in ‘data‘ and its own configuration files in ‘conf‘. There are two optional directories: ‘bin‘ (for replication scripts) and ‘lib‘ (for plugins). Unless your other apps use them, chances are they will be missing.

Install Drupal ApacheSolr plugin protwords, schema and solrconfig

You should already have Acquia Drupal 6 running or Drupal 6 with the ApacheSolr plugin installed. You can copy the ‘protwords.txt’, ‘schema.xml’, and ‘solrconfig.xml’ files from the plugin directory in your respective distribution rather than downloading it, but adjust the paths accordingly.

If you don’t already have the ApacheSolr plugin, get it from the Drupal web site.

http://drupal.org/project/apachesolr

Choose the latest Tarball and use wget to download it to your server, then copy the ApacheSolr configuration files (and backup originals using ‘b’ flag):

# wget http://ftp.drupal.org/files/projects/apachesolr-6.x-1.5.tar.gz
# tar zxvf apachesolr-6.x-1.5.tar.gz
...
# echo 'If ur root cp may give u a scary msg next cmd! Ignore it! Y to overwrite!'
If ur root cp may give u a scary msg next cmd! Ignore it! Y to overwrite!
#
# cp -b -p -f apachesolr/protwords.txt /var/solr/example.com/conf
# cp -b -p -f apachesolr/schema.xml /var/solr/example.com/conf
# cp -b -p -f apachesolr/solrconfig.xml /var/solr/example.com/conf
#
# echo 'Fix group so tomcat can use this!'
Fix group so tomcat can use this!
#
# chown -R root:tomcat /var/solr/example.com
# chmod -R 775 /var/solr/

Warning! If you are not using the Acquia distribution and instead installed the ApacheSolr plugin from the main Drupal web site then you should check that you have a copy of the SolrPhpClient (version r22 – see module README for the gory details). The Acquia distribution includes the correct SolrPhpClient (so you might want to use that instead?).

Make the two Solr instances for the two domains

This walkthrough will create two domains, but you can create more. Using the example.com folder as a prototype, just recursively copy it twice to make two domains (use ‘p’ switch to ‘preserve’ the file permissions and settings):

# cp -r -p /var/solr/example.com /var/solr/www1.kelvinwong.ca
# cp -r -p /var/solr/example.com /var/solr/www2.kelvinwong.ca

If the future, to add a new domain, copy the example.com folder you just made and customize it. This will also work for additional domains that you want to support.

Configure Tomcat 6

It’s All About Context: The Context element represents a web application run within a particular Tomcat virtual host. Each web application is based on a Web Application Archive (WAR) file or a corresponding unpacked directory. The web application used to process each web request is determined by matching the request to the path of each Context. You may define as many Context elements as you wish, but each Context MUST have a unique path. More on Context

Contexts are no longer put into Tomcat’s server.xml file since that file is read only at server start-up. Instead Contexts are placed into a folder hierarchy under CATALINA_BASE (on CentOS 6 it is /etc/tomcat6). Create and configure the following files:

# touch /etc/tomcat6/Catalina/localhost/www1.kelvinwong.ca.xml
# touch /etc/tomcat6/Catalina/localhost/www2.kelvinwong.ca.xml
# chown tomcat:root /etc/tomcat6/Catalina/localhost/{www1.kelvinwong.ca.xml,www2.kelvinwong.ca.xml}
# chmod 664 /etc/tomcat6/Catalina/localhost/{www1.kelvinwong.ca.xml,www2.kelvinwong.ca.xml}

Tomcat will use these files to find the WAR and deploy the application using the settings in the Context. Note: Contexts can be overridden (they often are) and there are more than a few in Tomcat. Review Tomcat’s documentation if they give you any trouble.

Make sure your Context fragments have .xml suffixes!

Place the following into /etc/tomcat6/Catalina/localhost/www1.kelvinwong.ca.xml

# vi /etc/tomcat6/Catalina/localhost/www1.kelvinwong.ca.xml
<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/usr/share/tomcat6/lib/solr.war" debug="0" crossContext="true" >
   <Environment name="solr/home" type="java.lang.String" value="/var/solr/www1.kelvinwong.ca" override="true" />
</Context>

The Context fragment is simply telling Tomcat where to find the Context root (document base). It is an absolute path to its web app archive (WAR) file. CrossContext allows Solr to get a request dispatcher from ServletContext.getContext() for access to other web apps on the virtual host. The Environment tag defines the ‘solr/home‘ setting and allows it to be overridden. That’s all you need.

Change the other fragment:

# vi /etc/tomcat6/Catalina/localhost/www2.kelvinwong.ca.xml

Change the paths:

<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/usr/share/tomcat6/lib/solr.war" debug="0" crossContext="true" >
   <Environment name="solr/home" type="java.lang.String" value="/var/solr/www2.kelvinwong.ca" override="true" />
</Context>

Bind Tomcat to Local Port

By default, Tomcat listens on port 8080. The default iptables ruleset in CentOS 6 does not allow remote connections to port 8080. For our purposes this is fine since we want our Drupal sites to connect locally on port 8080. Local good, remote bad.

You can also tell Tomcat to bind to localhost and not any of the other network adapters. Open Tomcat’s server.xml file:

# vi /etc/tomcat6/server.xml

Change Tomcat’s binding address to the localhost address (127.0.0.1) in the Connector tag:

69
70
71
72
73
74
    <Connector port="8080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" 
               URIEncoding="UTF-8"
               maxHttpHeaderSize="65535"
               address="127.0.0.1" />

Solr is a web service that takes many requests from Drupal using the HTTP GET method, similar to you typing into your browser’s web address bar. These requests routinely get very long; you can increase the GET request character limit by increasing the maxHttpHeaderSize attribute (from 8k to 64k as shown). To handle non-English characters, you should also set the request encoding to UTF-8. The Connector as-shown does both.

Restart Tomcat to reload the server.xml file:

# /sbin/service tomcat6 restart
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]

View Solr Admin (optional)

You should now be able to view the Solr administration page if you open a local web browser on the server. If you don’t have a desktop on the server (as should be the case), you can use a text-browser like elinks.

View http://localhost:8080/www1.kelvinwong.ca/admin:

# elinks http://localhost:8080/www1.kelvinwong.ca/admin

You should see the Solr administration page in your browser.

SELinux

“Apache Solr: Your site was unable to contact the Apache Solr server,” reports Drupal; SELinux chuckles.

SELinux is enabled by default on CentOS 6, so you will likely have it running and it will not appreciate Apache trying to talk to Tomcat/Solr on port 8080 (check /var/log/audit/audit.log):

type=AVC msg=audit(1315100262.891:17629): avc:  denied  { name_connect } for  pid=2064 comm="httpd" dest=8080 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:http_cache_port_t:s0 tclass=tcp_socket

type=SYSCALL msg=audit(1315100262.891:17629): arch=40000003 syscall=102 success=no exit=-13 a0=3 a1=bfbe6590 a2=b70426f4 a3=11 items=0 ppid=2060 pid=2064 auid=500 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)

You can either turn off SELinux (not recommended) or fix the attributes so that SELinux allows Apache to talk to Tomcat. The handy tool sealert gives helpful advice:

# sealert -a /var/log/audit/audit.log | less

Summary:

SELinux is preventing the http daemon from connecting to itself or the relay ports

Detailed Description:

SELinux has denied the http daemon from connecting to itself or the relay ports. An httpd script is trying to make a network connection to an http/ftp port. If you did not setup httpd to make network connections, this could signal an intrusion attempt.

Allowing Access:

If you want httpd to connect to httpd/ftp ports you need to turn on the
httpd_can_network_relay boolean: "setsebool -P httpd_can_network_relay=1"

Fix Command:

setsebool -P httpd_can_network_relay=1

Additional Information:

Source Context                unconfined_u:system_r:httpd_t:s0
Target Context                system_u:object_r:http_cache_port_t:s0
Target Objects                None [ tcp_socket ]
Source                        httpd
Source Path                   /usr/sbin/httpd
Port                          8080
Host                          <Unknown>
Source RPM Packages           httpd-2.2.15-5.el6.centos
Target RPM Packages           
Policy RPM                    selinux-policy-3.7.19-54.el6_0.5
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Plugin Name                   httpd_can_network_relay
Host Name                     templeton.localdomainPlatform                      Linux templeton.localdomain                              2.6.32-71.29.1.el6.i686 #1 SMP Mon Jun 27 18:07:00
                              BST 2011 i686 i686
Alert Count                   14First Seen                    Sat Sep  3 18:25:40 2011Last Seen                     Sat Sep  3 18:37:42 2011Local ID                      4b66d238-ddf7-4b74-bbe5-3fb54be5b3e4Line Numbers                  178, 179, 180, 181, 182, 183, 184, 185, 192, 193,
                              194, 195, 196, 197, 198, 199, 200, 201, 202, 203,
                              204, 205, 206, 207, 208, 209, 210, 211

Once upon a time and a very good time it was there was a moocow coming down along the road and this moocow that was coming down along the road met a nicens little boy named baby tuckoo[1]

The quick fix is to set the network relay flag (‘P’ flag makes the change persistent across reboots):

# setsebool -P httpd_can_network_relay=1
# getsebool httpd_can_network_relay
httpd_can_network_relay --> on

You don’t need sealert to use setsebool but it is a useful utility to debug errors with SELinux. If you don’t have sealert installed, it is a simple thing to install it since it is part of the setroubleshoot package:

# yum install setroubleshoot

Configure Drupal to use Solr

Turning now to your Drupal installation…

Enable the Solr Search service module…

Configure the Apache Solr Search module by visiting http://www1.kelvinwong.ca/?q=admin/settings/apachesolr

Solr host name
localhost
Solr port
8080
Solr path
/www1.kelvinwong.ca

The Solr path is the name of your Context fragment minus the xml suffix (ie. /etc/tomcat6/Catalina/localhost/www1.kelvinwong.ca.xml)

The cron job indexes 50 nodes at a time by default. When indexed, you can then search for nodes by keyword.

Save the settings. You should see:

  • The configuration options have been saved.
  • Apache Solr: Your site has contacted the Apache Solr server.
  • Apache Solr PHP Client Library: Correct version “Revision: 22”.

Try a search

You can re-index the site by force or let cron do it gradually. Either way it take a while for Solr to process the data.

http://www1.kelvinwong.ca/?q=admin/settings/apachesolr/index

Once you have indexed your site and adjusted the permissions on the search form (so anonymous users can use the search form), visit it:

http://www1.kelvinwong.ca/?q=search/apachesolr_search

Intentionally misspell something and let Solr give you hints!

What about the other one??? www2?

Ah, yes…the other one is set-up in a similar manner, just use the following configuration in Drupal:

Solr host name
localhost
Solr port
8080
Solr path
/www2.kelvinwong.ca

Tags: , , , , , , , , , , , , , ,

Goodbye XML

I’m leaving you XML. I’m just not into you anymore. I don’t think I ever was…I just used you. Sorry.

Why? You’re just too damn hard to read and you give me a headache. I always have to check if you’re well-formed and, frankly, most of the time, you’re not.

It’s not because you have only one root. No, it’s not. You’re always so damn verbose…and you use tabs everywhere we go and who picks those up? I do. That’s who.

Well, it stops today.

Yes, there is someone else.

Running Cassandra 0.5.0 on Mac OS X Leopard 10.5.8

This is a short post showing you how to install the superdooper Key-Value Store Cassandra version 0.5.0 on a Macbook running OS X Leopard 10.5.8 (yeah I know, I should upgrade to Snow Leopard sometime). Cassandra 0.5 promises to be peppier than the 0.4 version and at this point it hasn’t been released yet, but I’m impatient, so I’m setting up a ‘one-node cluster’ for fun.

Cassandra 0.5.0 was released on 25 Jan 2010.

You will need to ensure that you are running Java 1.6. Since Java 1.5 is the default on Mac OS X, you need to set up a couple of environment variables. I just put the following into my .bash_profile file using vim.

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export PATH=$JAVA_HOME/bin:$PATH

Now source the file (if you haven’t already done that) and examine your set-up.

Trinity:~ kelvin$ source .bash_profile
Trinity:~ kelvin$ env | grep JAVA_HOME
JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
Trinity:~ kelvin$ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
Trinity:~ kelvin$ xcodebuild -version
Xcode 3.1.2
Component versions: DevToolsCore-1148.0; DevToolsSupport-1102.0
BuildVersion: 9M2621

Yeah, my xcode is a bit old too. So, once that is done, download Cassandra. I am going to use the tag in the Subversion repository for the 0.5.0 release.

Trinity:code kelvin$ svn co https://svn.apache.org/repos/asf/incubator/cassandra/tags/cassandra-0.5.0/ cassandra-0.5.0

...svn key gibberish redacted...

A    cassandra-0.5.0/NOTICE.txt
 cassandra-0.5.0/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
...checkout redacted...
A    cassandra-0.5.0/build.xml
 U   cassandra-0.5.0
Checked out revision 902040.
Trinity:code kelvin$ cd cassandra-0.5.0
Trinity:cassandra-0.5.0 kelvin$ pwd
/Users/kelvin/code/cassandra-0.5.0
Trinity:cassandra-0.5.0 kelvin$

Cassandra expects some folders to be available, so we will need to set them up (per the instructions in the README file). First of all, check the file conf/storage-conf.xml and make sure that these folders exist before you do anything else.


  <CommitLogDirectory>/var/lib/cassandra/commitlog</CommitLogDirectory>
  <DataFileDirectories>
      <DataFileDirectory>/var/lib/cassandra/data</DataFileDirectory>
  </DataFileDirectories>
  <CalloutLocation>/var/lib/cassandra/callouts</CalloutLocation>
  <StagingFileDirectory>/var/lib/cassandra/staging</StagingFileDirectory>

If they do not exist, you will have to make them (careful with the whoami command in backticks):

Trinity:cassandra-0.5.0 kelvin$ sudo mkdir -p /var/log/cassandra
Password:
Trinity:cassandra-0.5.0 kelvin$ sudo chown -R `whoami` /var/log/cassandra
Trinity:cassandra-0.5.0 kelvin$ sudo mkdir -p /var/lib/cassandra
Trinity:cassandra-0.5.0 kelvin$ sudo chown -R `whoami` /var/lib/cassandra

For logging, Cassandra expects that a system.log file is available. Check the conf/log4j.properties file to check where it expects a log file. It should be similar to this:

log4j.appender.R.File=/var/log/cassandra/system.log

When you start Cassandra for the first time it will start the log file.

I used ‘ant‘ to build Cassandra. Make sure that you are in the root of the Cassandra directory. I have a lot of crap in my Java classpath, so I will temporarily exclude it.

Trinity:cassandra-0.5.0 kelvin$ ant -noclasspath
Buildfile: build.xml

build-subprojects:

init:
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/build/classes
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/build/test/classes
    [mkdir] Created dir: /Users/kelvin/code/cassandra-0.5.0/src/gen-java

check-gen-cli-grammar:

gen-cli-grammar:
     [echo] Building Grammar /Users/kelvin/code/cassandra-0.5.0/src/java/org/apache/cassandra/cli/Cli.g  ....

build-project:
     [echo] apache-cassandra-incubating: /Users/kelvin/code/cassandra-0.5.0/build.xml
    [javac] Compiling 247 source files to /Users/kelvin/code/cassandra-0.5.0/build/classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

build:

BUILD SUCCESSFUL
Total time: 13 seconds
Trinity:cassandra-0.5.0 kelvin$

Now the moment of truth!

Trinity:cassandra-0.5.0 kelvin$ bin/cassandra -f
Listening for transport dt_socket at address: 8888
INFO - Saved Token not found. Using 25907362644306974147256376871662350143
INFO - Starting up server gossip
INFO - Cassandra starting up...

Okay, Cassandra is now waiting for us to do something, so we can open up another Terminal window (Cmd+N) and try the test described in the README file:

Trinity:cassandra-0.5.0 kelvin$ bin/cassandra-cli --host localhost --port 9160
Connected to localhost/9160
Welcome to cassandra CLI.

Type 'help' or '?' for help. Type 'quit' or 'exit' to quit.
cassandra> set Keyspace1.Standard1['jsmith']['first'] = 'John'
Value inserted.
cassandra> set Keyspace1.Standard1['jsmith']['last'] = 'Smith'
Value inserted.
cassandra> set Keyspace1.Standard1['jsmith']['age'] = '42'
Value inserted.
cassandra> get Keyspace1.Standard1['jsmith']
=> (column=last, value=Smith, timestamp=1264159272089)
=> (column=first, value=John, timestamp=1264159260000)
=> (column=age, value=42, timestamp=1264159282593)
Returned 3 results.
cassandra>

Okay, we now have stored and retrieved some data. In the next installment I’ll try to get Thrift running!

Tags: , , , , , ,

Running Shindig on your Mac

If you run a social network, you really owe it to your users to investigate OpenSocial as a way of allowing your users to add third-party functionality to their profiles. Shindig (available in a PHP and a Java version) allows you to easily add this capability.

Mac OS X Leopard has Java 1.6 and Maven 2.0.6 installed. I had some build issues with the older Maven so I upgraded to 2.0.9. You also need to ensure that your JAVA_HOME variable is set. You can confirm this by typing the following commands in a terminal:

Trinity:shindig kelvin$ echo $JAVA_HOME
/Library/Java/Home
Trinity:shindig kelvin$ mvn -v
Maven version: 2.0.9
Java version: 1.6.0_07
OS name: "mac os x" version: "10.5.5" arch: "x86_64" Family: "mac"

Then download the code from the main repository:

Trinity:code kelvin$ mkdir shindig
Trinity:code kelvin$ svn co http://svn.apache.org/repos/asf/incubator/shindig/trunk/ ./shindig
A    shindig/NOTICE
A    shindig/java
A    shindig/java/social-api
A    shindig/java/social-api/src
A    shindig/java/social-api/src/test

...la dee da...

A    shindig/features/mocks/env.js
A    shindig/features/mocks/xhr.js
 U   shindig
Checked out revision 707938.

Okay, Shindig has now been downloaded. Let’s build it. You can try building it using ‘mvn clean install’ first and if you get any errors, try turning off the tests. Sometimes the tests can get out of sync with the code. I was able to build it without any problems.

Trinity:shindig kelvin$ mvn clean install
[INFO] Scanning for projects...
[INFO] Reactor build order: 
[INFO]   Apache Shindig Project
[INFO]   Apache Shindig Project Parent
[INFO]   Apache Shindig Common Code
[INFO]   Apache Shindig Gadget Renderer
[INFO]   Apache Shindig Social API
[INFO]   Apache Shindig Features
[INFO]   Apache Shindig Web App
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Shindig Project
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean]
[INFO] Deleting directory /Users/kelvin/code/shindig/target
[INFO] [site:attach-descriptor]

...etc etc etc...

[INFO] [install:install]
[INFO] Installing /Users/kelvin/code/shindig/java/server/target/shindig-server-1-SNAPSHOT.war to /Users/kelvin/.m2/repository/org/apache/shindig/shindig-server/1-SNAPSHOT/shindig-server-1-SNAPSHOT.war
[INFO] Installing /Users/kelvin/code/shindig/java/server/target/shindig-server-1-SNAPSHOT-sources.jar to /Users/kelvin/.m2/repository/org/apache/shindig/shindig-server/1-SNAPSHOT/shindig-server-1-SNAPSHOT-sources.jar
[INFO] 
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Apache Shindig Project ................................ SUCCESS [6.196s]
[INFO] Apache Shindig Project Parent ......................... SUCCESS [0.531s]
[INFO] Apache Shindig Common Code ............................ SUCCESS [9.470s]
[INFO] Apache Shindig Gadget Renderer ........................ SUCCESS [25.571s]
[INFO] Apache Shindig Social API ............................. SUCCESS [11.381s]
[INFO] Apache Shindig Features ............................... SUCCESS [7.117s]
[INFO] Apache Shindig Web App ................................ SUCCESS [13.737s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 14 seconds
[INFO] Finished at: Sun Nov 09 10:43:56 PST 2008
[INFO] Final Memory: 51M/80M
[INFO] ------------------------------------------------------------------------

or if you want to skip the tests, try this…

Trinity:shindig kelvin$ mvn clean install -Dmaven.test.skip=true

Woot. Now let’s start it up. By default, the server will run on localhost port 8080.

Trinity:shindig kelvin$ mvn -Prun
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'jetty'.
Downloading: http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.1/maven-antrun-plugin-1.1.pom
1K downloaded
Downloading: http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-plugin-parent/2.0.1/maven-plugin-parent-2.0.1.pom
7K downloaded
Downloading: http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.1/maven-antrun-plugin-1.1.jar
11K downloaded
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Shindig Project
[INFO]    task-segment: [jetty:run-war]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing jetty:run-war


...more more more...

[INFO] Starting jetty 6.1.9 ...
2008-11-09 10:47:45.790::INFO:  jetty-6.1.9
2008-11-09 10:47:45.812::INFO:  Extract jar:file:/Users/kelvin/code/shindig/java/server/target/shindig-server-1-SNAPSHOT.war!/ to /Users/kelvin/code/shindig/target/work/webapp
2008-11-09 10:47:47.372::INFO:  No Transaction manager found - if your webapp requires one, please configure one.
Nov 9, 2008 10:47:48 AM org.apache.shindig.common.JsonContainerConfig loadContainers
INFO: Loading resources from: containers/default/container.js
Nov 9, 2008 10:47:48 AM org.apache.shindig.common.JsonContainerConfig loadResources
INFO: Reading container config: containers/default/container.js
2008-11-09 10:47:48.313::INFO:  Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server

See if you can render anything by opening a web browser and putting the following into the address bar.

http://localhost:8080/gadgets/ifr?url=http://www.labpixies.com/campaigns/todo/todo.xml

If all went well, you should see the To Do list demo.

The To Do Shindig App

The To Do Shindig App

Tags: , ,

Upgrade your Maven on OS X Leopard

As of Mac OS X 10.5.5, Java is now up to 1.6 and Maven comes pre-installed at version 2.0.6. We’re going to upgrade Maven to the latest version 2.0.9. Check your versions!

Trinity:~ kelvin$ java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06-153)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_07-b06-57, mixed mode)
Trinity:~ kelvin$ which mvn
/usr/bin/mvn
Trinity:~ kelvin$ /usr/bin/mvn -v
Maven version: 2.0.6
Trinity:~ kelvin$ 
Trinity:~ kelvin$ 

Visit the Maven web site and download the latest version (currently 2.0.9). It downloads to the Download folder, so you will have to move it. I installed Maven to /usr/local/apache-maven

Trinity:~ kelvin$ sudo sh
Password:
sh-3.2# cd /usr/local
sh-3.2# wget http://gulus.USherbrooke.ca/pub/appl/apache/maven/binaries/apache-maven-2.0.9-bin.zip
--2008-11-09 01:34:39--  http://gulus.usherbrooke.ca/pub/appl/apache/maven/binaries/apache-maven-2.0.9-bin.zip
Resolving gulus.usherbrooke.ca... 206.167.141.10
Connecting to gulus.usherbrooke.ca|206.167.141.10|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2025041 (1.9M) [application/zip]
Saving to: `apache-maven-2.0.9-bin.zip'

100%[======================================================================================>] 2,025,041    456K/s   in 5.1s    

2008-11-09 01:34:45 (389 KB/s) - `apache-maven-2.0.9-bin.zip' saved [2025041/2025041]

sh-3.2# md5 apache-maven-2.0.9-bin.zip 
MD5 (apache-maven-2.0.9-bin.zip) = 5e92d1c5b0a6580127059e28fe2f2b45
sh-3.2# unzip apache-maven-2.0.9-bin.zip 
Archive:  apache-maven-2.0.9-bin.zip
   creating: apache-maven-2.0.9/
   creating: apache-maven-2.0.9/conf/
  inflating: apache-maven-2.0.9/LICENSE.txt  
  inflating: apache-maven-2.0.9/NOTICE.txt  
  inflating: apache-maven-2.0.9/README.txt  
   creating: apache-maven-2.0.9/bin/
  inflating: apache-maven-2.0.9/bin/m2.bat  
  inflating: apache-maven-2.0.9/bin/m2.conf  
  inflating: apache-maven-2.0.9/bin/mvn.bat  
  inflating: apache-maven-2.0.9/bin/mvnDebug.bat  
  inflating: apache-maven-2.0.9/bin/m2  
  inflating: apache-maven-2.0.9/bin/mvn  
  inflating: apache-maven-2.0.9/bin/mvnDebug  
  inflating: apache-maven-2.0.9/conf/settings.xml  
   creating: apache-maven-2.0.9/lib/
  inflating: apache-maven-2.0.9/lib/maven-2.0.9-uber.jar  
   creating: apache-maven-2.0.9/boot/
  inflating: apache-maven-2.0.9/boot/classworlds-1.1.jar  
sh-3.2# exit
Trinity:~ kelvin$

The Maven installation instructions call for the creation of the M2 environment variables, so we need to do that. I put them into the .bash_profile file which is in your home directory.

export M2_HOME=/usr/local/apache-maven-2.0.9
export M2=$M2_HOME/bin
export PATH=$M2:$PATH

Put these somewhere near the top of your .bash_profile file so that the shell will find the mvn 2.0.9 version before it finds the old 2.0.6 version at /usr/bin/mvn.

Trinity:~ kelvin$ source .bash_profile
Trinity:~ kelvin$ mvn -v
Maven version: 2.0.9
Java version: 1.6.0_07
OS name: "mac os x" version: "10.5.5" arch: "x86_64" Family: "mac"
Trinity:~ kelvin$ echo $M2_HOME
/usr/local/apache-maven-2.0.9
Trinity:~ kelvin$ which mvn
/usr/local/apache-maven-2.0.9/bin/mvn
Trinity:~ kelvin$ 

Tags:

jEdit on Vista

I use jEdit to do most of my coding. I had some issues using the “Open with…” feature on Vista Ultimate. For some reason the right-click menu would not launch jEdit 4.3pre12 on Java 1.6.0_03.

I got it working by editing the registry and editing the bat file. For some reason the path to javaw.exe was incorrect. If you fix these paths, everything works as expected.

Fixing the registry

  • Open the registry editor using “regedt32” in the Run dialog (click “the Windows key”-R)
  • Go to: Computer\HKEY_CLASSES_ROOT\*\shell\Open with jEdit\Command
  • Change the key Data: “C:\Program Files (x86)\Java\jre1.6.0_03\bin\javaw.exe” -Xms64M -Xmx192M -jar “C:\Program Files (x86)\jEdit\jedit.jar” -reuseview “%1”
  • Click “OK”

Fixing the bat file

  • Open Windows Explorer and go to: C:\Program Files (x86)\jEdit
  • Edit the jEdit.bat file
  • Modify this line: start “jEdit startup” “C:\Program Files (x86)\Java\jre1.6.0_03\bin\javaw.exe” -Xms64M -Xmx192M -jar “C:\Program Files (x86)\jEdit\jedit.jar” -reuseview %*

ElvisJoke

public class ElvisJoke {
	public static void main(String[] args) {
		ElvisJoke haha = new ElvisJoke();
		TNode theBuilding = haha.new TNode();
		TNode elvis = haha.new TNode(theBuilding,null,"Elvis");
		if (elvis.hasLeft(theBuilding)){
			System.out.println("Thank you very much!");
		}
	}
	private class TNode {
		TNode left;
		TNode right;
		String data;
		TNode(){
			this(null,null,"No Data");
		}
		TNode(TNode left, TNode right, String data){
			this.left = left;
			this.right = right;
			this.data = data;
		}
		boolean hasLeft(TNode node){
			return (left!=null);
		}
		boolean hasRight(TNode node){
			return (right!=null);
		}
		String getData() {
			return this.data;
		}
	}
}

Morse code for Java

I was rummaging through some old files and I found a Morse Code object I wrote when I was playing around with binary trees. I thought that this might be useful to someone, so here it is licensed under the MIT license.