MySQL-Python with Python for Mac and 64-bit MySQL errors

LONG_BIT errors getting you down? If you are building MySQL-Python with the DMG version of Python 2.5.4 then make sure that you have a 32-bit version of MySQL installed or else you will see a variation of this:

(visions)Trinity:envs kelvin$ easy_install -Z mysql_python
Searching for mysql-python
Best match: MySQL-python 1.2.3
Processing MySQL-python-1.2.3.tar.gz
Running MySQL-python-1.2.3/ -q bdist_egg --dist-dir /var/folders/L4/L4q9U6sBHceoN-EDstAkL++++TI/-Tmp-/easy_install-rniNnX/MySQL-python-1.2.3/egg-dist-tmp-4WCvXJ
warning: no files found matching 'MANIFEST'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'GPL'
In file included from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:57,
                 from pymemcompat.h:10,
                 from _mysql.c:29:
/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyport.h:761:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
In file included from _mysql.c:36:
/usr/local/mysql/include/my_config.h:1062:1: warning: "HAVE_WCSCOLL" redefined
In file included from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:8,
                 from pymemcompat.h:10,
                 from _mysql.c:29:
/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyconfig.h:721:1: warning: this is the location of the previous definition
In file included from _mysql.c:36:
/usr/local/mysql/include/my_config.h:1180:1: warning: "SIZEOF_LONG" redefined
In file included from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:8,
                 from pymemcompat.h:10,
                 from _mysql.c:29:
/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyconfig.h:811:1: warning: this is the location of the previous definition
In file included from _mysql.c:36:
/usr/local/mysql/include/my_config.h:1189:1: warning: "SIZEOF_PTHREAD_T" redefined
In file included from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:8,
                 from pymemcompat.h:10,
                 from _mysql.c:29:
/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyconfig.h:820:1: warning: this is the location of the previous definition
error: Setup script exited with error: command 'gcc' failed with exit status 1

If you check, you will find that SIZEOF_LONG is 4 and LONG_BIT is 64 (should be 32) if you are using the 32-bit version of Python 2.5.4 and you have the 64-bit version of MySQL Community Server installed. The pyport.h file will catch it and raise the error — thanks pyport!

The solution for me was to uninstall my 64-bit version of MySQL and install the 32-bit version which was way down on the list of installers. Another solution would be just to use MacPorts to build all your Python bits.

Once the offending version of MySQL was replaced, MySQL-Python installed with only a minor warning. It was immediately usable:

Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>> MySQLdb.__version__

And if you are still running Python 2.5.4 try upgrading to Python 2.6.4

Tags: , ,

Install PHP 5.3.3 on Mac OS Leopard 10.5.8

My attempt at building and installing PHP 5.3.3 crashed and burned with the following error while building the MySQLi extension:

/bin/sh /Users/kelvin/phpsource/php-5.3.3/libtool --silent --preserve-dup-deps --mode=compile gcc  -Iext/mysqli/ -I/Users/kelvin/phpsource/php-5.3.3/ext/mysqli/ -DPHP_ATOM_INC -I/Users/kelvin/phpsource/php-5.3.3/include -I/Users/kelvin/phpsource/php-5.3.3/main -I/Users/kelvin/phpsource/php-5.3.3 -I/Users/kelvin/phpsource/php-5.3.3/ext/date/lib -I/Users/kelvin/phpsource/php-5.3.3/ext/ereg/regex -I/usr/local/php5/include/libxml2 -I/usr/local/php5/include -I/opt/local/include -I/usr/local/php5/include/freetype2 -I/Users/kelvin/phpsource/php-5.3.3/ext/mbstring/oniguruma -I/Users/kelvin/phpsource/php-5.3.3/ext/mbstring/libmbfl -I/Users/kelvin/phpsource/php-5.3.3/ext/mbstring/libmbfl/mbfl -I/usr/local/mysql/include/mysql -I/Users/kelvin/phpsource/php-5.3.3/ext/sqlite3/libsqlite -I/usr/local/pgsql/include -I/Users/kelvin/phpsource/php-5.3.3/TSRM -I/Users/kelvin/phpsource/php-5.3.3/Zend  -no-cpp-precomp  -I/usr/local/php5/include -g -O2 -fvisibility=hidden  -c /Users/kelvin/phpsource/php-5.3.3/ext/mysqli/mysqli.c -o ext/mysqli/mysqli.lo 
 In file included from /Users/kelvin/phpsource/php-5.3.3/ext/mysqli/php_mysqli_structs.h:57,
                  from /Users/kelvin/phpsource/php-5.3.3/ext/mysqli/mysqli.c:33:
 /usr/local/mysql/include/mysql/my_global.h:895: error: duplicate ‘unsigned’
 /usr/local/mysql/include/mysql/my_global.h:895: warning: useless type name in empty declaration
 make: *** [ext/mysqli/mysqli.lo] Error 1
Trinity:~ kelvin$

According to the PHP bug tracker, the problem is a bug that affects Snow Leopard and FreeBSD as well. The fix is pretty simple, either you can build PHP from the head of the trunk from Subversion, or you can replace php-5.3.3/ext/mysqli/php_mysqli_structs.h with this patched version or you can just open one file (php-5.3.3/ext/mysqli/php_mysqli_structs.h) in your text editor and fix it yourself:

Trinity:~ kelvin$ sudo vim /Users/kelvin/phpsource/php-5.3.3/ext/mysqli/php_mysqli_structs.h

Go to line 59 and insert the following:

#if defined(ulong) && !defined(HAVE_ULONG)
#define HAVE_ULONG

The patched section of the file will then read as follows:

  The libmysql headers (a PITA) also define it and there will be an warning.
  Undef it and later we might need to define it again.
#if defined(ulong) && !defined(HAVE_ULONG)
#define HAVE_ULONG
#include <my_global.h>
#if !defined(HAVE_MBRLEN) && defined(WE_HAD_MBRLEN)
#define HAVE_MBRLEN 1
#if !defined(HAVE_MBSTATE_T) && defined(WE_HAD_MBSTATE_T)
#define HAVE_MBSTATE_T 1

The rest of the build and install was uneventful after patching php_mysqli_structs.h.

Tags: , ,

MV Sun Sea in Esquimalt

The migrant/refugee vessel MV Sun Sea arrived in Esquimalt Harbour on Friday.

Tags: , ,

Python 2.7 on Dreamhost

Python 2.7 was released on 3 July 2010 and I wanted to use it on my Dreamhost account, but the usual installation method yields some warnings:

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _tkinter           bsddb185
bz2                dl                 imageop         sunaudiodev
To find the necessary bits, look in in detect_modules() for the module's name.

Three of these modules cannot be installed on Dreamhost’s 64-bit Debian servers anyway and one of them is an older version of a module I am going to install:

Now, if you don’t need any of those remaining modules, then you should be able to just complete the installation and be done with it. If you want all the modules that you can get, you are in for some extra building. This post does a good job of explaining the installation of Python 2.6; mine is based on it. Let’s put the files in the following directories:

Python 2.7
Berkeley DB 4.8
Other executables
Header files
Temporary artifacts

We’ll need to push these values into the UNIX environment by using the export tool under the default bash shell:

$ export LDFLAGS="-L$HOME/local/lib -L$HOME/local/BerkeleyDB.4.8/lib"
$ export CPPFLAGS="-I$HOME/local/include -I$HOME/local/BerkeleyDB.4.8/include"
$ export LD_LIBRARY_PATH=$HOME/local/lib:$HOME/local/BerkeleyDB.4.8/lib

Next make the directories:

$ mkdir ~/temp ~/local

It’s also a good idea to check your machine (note the “x86_64” token). It should look similar to this:

$ uname -a
Linux machine #2 SMP Sat Mar 13 00:42:43 PST 2010 x86_64 GNU/Linux
$ gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.2-1.1' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --enable-cld --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.2 (Debian 4.3.2-1.1) 

Berkeley DB will be hooked into the installed TCL, so you should install TCL before you install Berkeley DB. Install Python last. Other than that, I don’t believe that order of installation should matter. Let’s do it!


The warning “_tkinter” indicates that the _tkinter module was not built. You will need to build both TCL and TK:

$ cd ~/temp
$ pwd
$ wget
$ tar zxvf tcl8.5.8-src.tar.gz
$ cd tcl8.5.8/unix
$ ./configure --prefix=$HOME/local
$ make
$ make install
$ cd ../..
$ wget
$ tar zxvf tk8.5.8-src.tar.gz
$ cd tk8.5.8/unix
$ ./configure --prefix=$HOME/local
$ make
$ make install
$ cd ../..

Berkeley DB 4.8

The warning “_bsddb” will go away when you install version 4.8 of the Oracle Berkeley DB:

$ cd ~/temp
$ wget
$ tar zxvf db-4.8.30.tar.gz
$ cd db-4.8.30/build_unix
$ ../dist/configure --prefix=$HOME/local/BerkeleyDB.4.8 --enable-tcl --with-tcl=$HOME/local/lib
$ make
$ make install
$ cd ../..


Dreamhost has an earlier version of BZip2 (version 1.0.4) and no library (at least I couldn’t find one). To get the latest version:

$ cd ~/temp
$ wget
$ tar zxvf bzip2-1.0.5.tar.gz
$ cd bzip2-1.0.5
$ make -f Makefile-libbz2_so
$ make
$ make install PREFIX=$HOME/local
$ cp ./ $HOME/local/lib
$ ln -s $HOME/local/lib/ $HOME/local/lib/
$ cd ..

Python 2.7

You should be able to build and install Python 2.7 now, less the modules that either cannot be built on a 64-bit platform or cannot coexist with a contemporary version.

$ cd ~/temp
$ wget
$ tar zxvf Python-2.7.tgz
$ cd Python-2.7
$ ./configure --prefix=$HOME/local/Python-2.7
$ make
$ make install

At the end of the make process, you will see this:

Python build finished, but the necessary bits to build these modules were not found:
bsddb185           dl                 imageop         
To find the necessary bits, look in in detect_modules() for the module's name.

As mentioned earlier, these modules are either not buildable on Dreamhost’s 64-bit machines or not compatible with the newer version of Berkeley DB.

Hooking up the new Python

You want to put the new Python 2.7 on your PATH so that bash executes it before the systemwide Python 2.5. If you don’t want to append these export statements, you can also do it via a text editor (vim, emacs, etc).

$ cd ~
$ echo "export PATH=\"$HOME/local/bin:\$PATH\"" >> .bashrc
$ echo "export PATH=\"$HOME/local/Python-2.7/bin:\$PATH\"" >> .bashrc
$ source .bashrc
$ which python
$ python -V
Python 2.7

Try it out!

Try out your new modules…at the shell prompt type “python”!

>>> import bsddb
>>> db = bsddb.btopen('/tmp/spam.db', 'c')
>>> for i in range(10): db['%d'%i] = '%d'% (i*i)
>>> db['3']
>>> db.keys()
['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
>>> db.sync()
>>> db.isOpen()
>>> import bz2
>>> print bz2.__author__
The bz2 python module was written by:
    Gustavo Niemeyer <>
>>> import Tkinter
>>> Tkinter.__version__
'$Revision: 81008 $'

Tags: ,

Shock the monkey!

Some important research from the Institute of Biomedical Engineering, National Taiwan University, Taipei, Taiwan. Drs. Tsai, Chang, Liu, Kuo, Chen, Jaw, & Hsieh (whew…seven researchers) published a study of the electrical properties of the Ding-Dong. No, not that Ding-Dong, the other ding-dong.

Anyway, you can read the shocking news on Disco blog or the abstract on PubMed. This is going to win them seven awards.

USCYBERCOM: 9ec4c12949a4f31474f299058ce2b22a in Python

In Wired, Noah Shachtman pointed out that the US Cyber Command logo has a 32-character code inscribed onto its inner gold ring. It is well-known that MD5 produces 128-bit message digests (often represented as 32-character hexadecimal digests) so it surprised few that the code turned out to be an MD5 hash of the organization’s mission statement.

To recreate this hash in Python, type the following into your Python interpreter:

Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from hashlib import md5
>>> ms = "USCYBERCOM plans, coordinates, integrates, synchronizes and conducts activities to: direct the operations and defense of specified Department of Defense information networks and; prepare to, and when directed, conduct full spectrum military cyberspace operations in order to enable actions in all domains, ensure US/Allied freedom of action in cyberspace and deny the same to our adversaries."
>>> o = md5(ms).hexdigest()
>>> print o
>>> len(o)

Note: The library ‘hashlib’ was added in Python 2.5.

Tags: , , , , ,

Funniest release notes ever? Thank Chaos Tools for Drupal

From the Chaos Tools release notes for version 1.6 for Drupal 6:

The 1.5 release was botched: only some of the changes listed on the release notes were actually included. This release (1.6) includes all changes that were supposed to be in 1.5.

See for yourself and feel your confidence in the codebase rise with each word.

Tags: , , , ,

Installing Webmin on Ubuntu Server 10.04 LTS (Lucid)

Webmin installed on Ubuntu 10.04 LTS Lucid


There is an easier way to install Webmin on Ubuntu 12.04 LTS! This walkthrough shows how to install Webmin 1.580 and upgrade the TLS self-signed certificate to use a 2048-bit key.


I had some trouble installing Webmin 1.510 on Ubuntu 10.04 LTS Server (aka Lucid). The problem is that Webmin uses a deprecated Perl module (a wrapper around Digest::MD5 for users of an ancient MD5 library) and both Debian and Ubuntu refuse to put it back into their respective repositories. Entirely within their rights, of course, but not so good for us weekend admins who want a painless install process.

Okay, so let’s get to work. I’m installing Webmin 1.510 via the remaining Debian packages.

Install the (easy) dependencies

Run this from a terminal. Expect some trouble from ‘libmd5-perl’.

$ sudo aptitude -y install perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl libmd5-perl apt-show-versions libapt-pkg-perl

You should find an error like this:

Couldn't find any package whose name or description matched "libmd5-perl"

The reason for this is that ‘libmd5-perl’ is persona non grata at both Debian and Ubuntu, as mentioned.

Install the deprecated dependencies

Download the libmd5-perl deb file and install it manually:

Open a browser and get the newest libmd5-perl package (from 2004 – lol)

The likely package is named: libmd5-perl_2.03-1_all.deb

so we download it and install it:$ wget
--2010-05-22 19:50:45--
Resolving, 2001:610:1908:a000::149:226
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5700 (5.6K) [application/x-debian-package]
Saving to: `libmd5-perl_2.03-1_all.deb'

100%[=======================================================================>] 5,700       30.3K/s   in 0.2s    

2010-05-22 19:50:46 (30.3 KB/s) - `libmd5-perl_2.03-1_all.deb' saved [5700/5700]$ sudo dpkg -i libmd5-perl_2.03-1_all.deb
Selecting previously deselected package libmd5-perl.
(Reading database ... 50494 files and directories currently installed.)
Unpacking libmd5-perl (from libmd5-perl_2.03-1_all.deb) ...
Setting up libmd5-perl (2.03-1) ...
Processing triggers for man-db ...$

Install Webmin

The dependencies should all be installed now. We can download the Webmin deb package from Sourceforge.

Use the most recent deb package. In my case it was ‘webmin_1.510-2_all.deb

Sourceforge will generate a link for you to use from their web site. My link was:$ wget
--2010-05-22 19:53:44--
Connecting to||:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: [following]
--2010-05-22 19:53:44--
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14504260 (14M) [application/octet-stream]
Saving to: `webmin_1.510-2_all.deb'

100%[===================================================================>] 14,504,260   512K/s   in 21s     

2010-05-22 19:54:06 (664 KB/s) - `webmin_1.510-2_all.deb' saved [14504260/14504260]$ sudo dpkg -i webmin_1.510-2_all.deb
Selecting previously deselected package webmin.
(Reading database ... 50500 files and directories currently installed.)
Unpacking webmin (from webmin_1.510-2_all.deb) ...
Setting up webmin (1.510-2) ...
Webmin install complete. You can now login to
as root with your root password, or as any user who can use sudo
to run commands as root.

Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot

You should now be able to visit your webmin login page on port 10000 (use your own IP number):

Ideally, the Webmin gurus will refactor the old MD5 code dependencies, but this seems to work fine for now.

Happy Harvey Milk day!

Tags: , , ,

Sending Redmine mail via Google Apps hosted GMail

So I don’t why I always have trouble setting up web apps to send mail through my hosted GMail accounts (like, um, Redmine?). Really, it seems like I end up pulling my hair out every time I have to do it.

So the problem we want to solve today should be straightforward. I have a Ruby-on-Rails app called Redmine which I want to have send notifications and whatnot via my Google Apps hosted GMail account. So here’s some basic instruction from Redmine. Great. And here is my email.yml file that actually works (passwords and other crap redacted, obviously):

  delivery_method: :smtp
    tls: true
    address: ""
    port: 587
    domain: ""
    authentication: :plain
    user_name: ""
    password: N1ceP@55wurd!

This set-up will allow Redmine to send emails from using hosted GMail

Tags: , , , , ,

Installing Fileinfo on CentOS with PHP 5.2.13

I ran into some trouble installing the Fileinfo extension for PHP on CentOS. I tried installing the extension using yum and all went well until I saw these errors:

PHP Warning:  PHP Startup: fileinfo: Unable to initialize module
Module compiled with module API=20050922, debug=0, thread-safety=0
PHP    compiled with module API=20060613, debug=0, thread-safety=0
These options need to match

Like I said, I installed the extension using yum, so I checked out what I had installed:

$ yum list installed | grep "php-pecl-Fileinfo"
php-pecl-Fileinfo.x86_64               1.0.4-3.el5.centos              installed

Oh crap, I always forget to pull these things from the Atomicorp repository because my php is from there.

$ yum list installed | grep "php.x86_64"
php.x86_64                                   installed

Well, the bad news is that the Fileinfo extension is not available on the Atomicorp repo. Hmm. Ok, so my next step is to remove the thing and install it with PECL.

$ sudo yum erase php-pecl-Fileinfo.x86_64
Running Transaction
  Erasing        : php-pecl-Fileinfo                                                                                         1/1 
warning: /etc/php.d/Fileinfo.ini saved as /etc/php.d/Fileinfo.ini.rpmsave

  php-pecl-Fileinfo.x86_64 0:1.0.4-3.el5.centos                                                                                  

$ sudo rm /etc/php.d/Fileinfo.ini.rpmsave

I didn’t need that config file, because I never got it to work!

Install Fileinfo with PECL

Before anything else, here is the system I’m using:

$ cat /etc/redhat-release 
CentOS release 5.4 (Final)
$ php -v
PHP 5.2.13 (cli) (built: Mar  2 2010 18:08:48) 
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies
$ pecl -V
PEAR Version: 1.9.0
PHP Version: 5.2.13
Zend Engine Version: 2.2.0
Running on: Linux system.local 2.6.18-028stab064.7 #1 SMP Wed Aug 26 13:11:07 MSD 2009 x86_64

If the PHP extension that I’m after is not in the yum repo then I usually just install it with PECL. It is fairly painless.

$ sudo pecl install fileinfo
audit_log_user_command(): Connection refused
WARNING: "pear/Fileinfo" is deprecated in favor of "channel://php-src/ext/fileinfo/in php sources"
downloading Fileinfo-1.0.4.tgz ...
Starting to download Fileinfo-1.0.4.tgz (5,835 bytes)
.....done: 5,835 bytes
3 source files, building
running: phpize

...lotsa crap...

build process completed successfully
Installing '/usr/lib64/php/modules/'
install ok: channel://

The PECL warning is trying to let you know that Fileinfo is bundled with PHP 5.3 and you don’t need to install if you are running PHP 5.3, but I’m running 5.2 so that warning is not for me. Okay, now we turn on the extension by putting “” somewhere in the /etc/php.ini file.

$ sudo vim /etc/php.ini

And make the change…


Now restart apache…

$ sudo /sbin/service httpd restart
audit_log_user_command(): Connection refused
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]

Okay, let’s find out if it is in the phpinfo…

$ php -r "phpinfo();" | grep "fileinfo"
fileinfo support => enabled


Tags: ,