Archive for May, 2008

Pricing: Google App Engine vs. Amazon EC2

Totally deprecated Like totally.
This post is ancient, out of date and I’m not going to update it. So, why does it get so much traffic???
More info at AppEngine and Amazon EC2

If you want to be the next Facebook at some point you will have to figure out how to scale to support millions of users. Amazon EC2 has been a popular choice for adding servers as they are needed and a number of busy Web 2.0 sites use it. Google App Engine is a new option. It allows application developers a chance to use the same building blocks used by Google, but it is different. GAE developers cannot change the underlying operating system or even write to the file system except though controlled methods. On one hand it’s a limitation but on the other hand it will force you to develop apps in a Googley way that can scale to millions of users. This is a very good thing if you’re developing the next MySpace. More on this later.

In the meantime, here’s some information on the announced Google App Engine pricing as mentioned on TechCrunch. I’m at Google I/O tomorrow and the day after.

Property Google App Engine Amazon EC2
Sources:

What is it??? The App Engine service is basically access to Google application service infrastructure including Datastore API/BigTable to store stuff. You have to use Python as your development language for now. You cannot open raw sockets to other machines, although you can pull stuff into your app using URLFetch on port 80 and 443. EC2 is a virtualized server. It will run whatever you put on the server space it gives you, but you have to set-up and configure everything yourself if you don’t use prebuilt AMIs. If you want one small server or a thousand big ones, you’ll pay for what you use.
Is it reliable? Preview service for now, but it should be as reliable as Google eventually. Some notable hiccups, but yeah its as reliable as your code and the platform you choose.
Try before you buy? Yes. Free quota to get started: 500MB storage and enough CPU and bandwidth for about 5 million pageviews per month No.
Scalable? This service is all about scalability. It is highly optimized for building apps that serve millions of users. Yes, if you use conventional methods for scaling web apps, you can add more machines quickly and easily. You do have to plan for it though.
Supported languages You can choose any language you want as long as it is Python. Ha. This will change! Anything you can load on your AMI.
Authentication built-in? You can easily authenticate your users with the Users API. Anybody with a Google Account can then add your app. Not applicable really, as it depends on how you set-up your server.
UNIX skilz required? Much of the sysadmin stuff is done for you by Google. You can use prebuilt AMIs, but if you are going to be making a busy site, you better know how to tune your OS and your development stack.
Memcached? Yes. Google supports memcached. Yes, apparently this can be done by installing memcached and controlling access with iptables. See the comments. Anyone got a link?
Computation charge per CPU core-hour $0.10 – $0.12 (billed on actual cycles used by scripts) $0.10 small, $0.40 med, $0.80 large (billed as long as your instance is active)
Storage of your stuff per GB/month $0.15 – $0.18 in BigTable Depends. $0.15 in Amazon S3 (plus data and query charges) or $1.50 in Amazon SimpleDB (plus SimpleDB machine costs and data transfer costs)
Bandwidth charges (outgoing GB) $0.11 – $0.13 $0.170 – $0.100 on a scale
Bandwidth charges (incoming GB) $0.09 – $0.11 $0.100

Tags:

Does my Mac really love me?

I have a cute little white MacBook that I use to write my code (and to listen to Pandora Radio). It is even more adorable now that it has 4GB of RAM and a 200GB hard-drive spinning at 7200RPM. Anyway, this post isn’t about my adorable computer.

I’m doing some simple image manipulation in Python 2.5. It is really basic stuff which can be done easily in PHP by using the GD2 image manipulation library. In Python, I have found two options: Py-GD (GD2 wrapped in Python) and the Python Imaging Library (PIL). The Python Imaging Library looks richer. I really haven’t tried much at this point other than resize some avatars.

In order to install the aforementioned libraries, you need to install a bunch of common image manipulation libraries like GD, JPEG, etc. Mac is missing these libraries, so you have to install them yourself. I’m lazy, so I basically have a choice of installing these using MacPorts or Fink. I’ve sworn off Fink (use it for a while and let me know if you agree) so that leaves MacPorts.

So I installed all the dependencies to get PIL installed. This is what I see:

sh-3.2# python setup.py build_ext -i
running build_ext
--- using frameworks at /System/Library/Frameworks
building '_imagingtk' extension
creating build
creating build/temp.macosx-10.3-i386-2.5
creating build/temp.macosx-10.3-i386-2.5/Tk
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Library/Frameworks/Python.framework/Versions/2.5/include -I/opt/local/include/freetype2/freetype -I/opt/local -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c _imagingtk.c -o build/temp.macosx-10.3-i386-2.5/_imagingtk.o -framework Tcl -framework Tk
In file included from /usr/include/math.h:26,
                 from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyport.h:231,
                 from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:57,
                 from _imagingtk.c:17:
/usr/include/architecture/ppc/math.h:675: warning: conflicting types for built-in function ‘scalb’
powerpc-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Library/Frameworks/Python.framework/Versions/2.5/include -I/opt/local/include/freetype2/freetype -I/opt/local -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c Tk/tkImaging.c -o build/temp.macosx-10.3-i386-2.5/Tk/tkImaging.o -framework Tcl -framework Tk
In file included from /usr/include/math.h:26,
                 from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/pyport.h:231,
                 from /Library/Frameworks/Python.framework/Versions/2.5/include/python2.5/Python.h:57,
                 from libImaging/ImPlatform.h:10,
                 from libImaging/Imaging.h:14,
                 from Tk/tkImaging.c:53:
/usr/include/architecture/ppc/math.h:675: warning: conflicting types for built-in function ‘scalb’
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
powerpc-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
gcc -arch i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-i386-2.5/_imagingtk.o build/temp.macosx-10.3-i386-2.5/Tk/tkImaging.o -L/opt/local/lib -L/Library/Frameworks/Python.framework/Versions/2.5/lib -L/opt/local -L/usr/lib -o PIL/_imagingtk.so -framework Tcl -framework Tk
ld: in /opt/local/lib/libJPEG.dylib, file is not of required architecture for architecture ppc
collect2: ld returned 1 exit status
lipo: can't open input file: /var/tmp//ccptvQXL.out (No such file or directory)
error: command 'gcc' failed with exit status 1
sh-3.2# 

She would only say such things to me if she really loved me. So the problem is that my Big Fat Cupertino Python was compiled as a universal binary with all this useless gunk for Power PC (what the hell is that, like a Beta VCR or something??). Of course my MacPort binaries are svelt little things compiled for my Intel chip, so naturally we got architecture problems. My modern clashes with your baroque.

My solution was stop her from trying to compile using the PPC flag. Where the hell is that set? The Makefile!

sh-3.2# cd /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/config/
sh-3.2# vi ./Makefile
sh-3.2# 

I changed these lines by removing the “arch ppc” flags from the BASECFLAGS and LDFLAGS (here is a patch):

sh-3.2# diff -urN ./Makefile.old ./Makefile
--- ./Makefile.old	2008-05-22 23:10:51.000000000 -0700
+++ ./Makefile	2008-05-22 23:11:51.000000000 -0700
@@ -57,13 +57,13 @@
 
 # Compiler options
 OPT=		-DNDEBUG -g -O3 
-BASECFLAGS=	-arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk  -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic
+BASECFLAGS=	-arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk  -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic
 CFLAGS=		$(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
 # Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py to
 # be able to build extension modules using the directories specified in the
 # environment variables
 CPPFLAGS=	-I. -IInclude -I$(srcdir)/Include 
-LDFLAGS=	-arch i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g 
+LDFLAGS=	-arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g 
 LDLAST=		
 SGI_ABI=	
 CCSHARED=
sh-3.2# 

So I asked her again, if she still loved me:

sh-3.2# python setup.py clean
running clean
removing 'build/temp.macosx-10.3-i386-2.5' (and everything under it)
removing 'build'
sh-3.2# python setup.py build_ext -i
running build_ext
--- using frameworks at /System/Library/Frameworks
building '_imagingtk' extension
creating build
creating build/temp.macosx-10.3-i386-2.5
creating build/temp.macosx-10.3-i386-2.5/Tk
gcc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Library/Frameworks/Python.framework/Versions/2.5/include -I/opt/local/include/freetype2/freetype -I/opt/local -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c _imagingtk.c -o build/temp.macosx-10.3-i386-2.5/_imagingtk.o -framework Tcl -framework Tk
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
gcc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Library/Frameworks/Python.framework/Versions/2.5/include -I/opt/local/include/freetype2/freetype -I/opt/local -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c Tk/tkImaging.c -o build/temp.macosx-10.3-i386-2.5/Tk/tkImaging.o -framework Tcl -framework Tk
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tcl: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: -framework: linker input file unused because linking not done
i686-apple-darwin9-gcc-4.0.1: Tk: linker input file unused because linking not done
gcc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-i386-2.5/_imagingtk.o build/temp.macosx-10.3-i386-2.5/Tk/tkImaging.o -L/opt/local/lib -L/Library/Frameworks/Python.framework/Versions/2.5/lib -L/opt/local -L/usr/lib -o PIL/_imagingtk.so -framework Tcl -framework Tk
building '_imagingmath' extension
gcc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Library/Frameworks/Python.framework/Versions/2.5/include -I/opt/local/include/freetype2/freetype -I/opt/local -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c _imagingmath.c -o build/temp.macosx-10.3-i386-2.5/_imagingmath.o
gcc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-i386-2.5/_imagingmath.o -L/opt/local/lib -L/Library/Frameworks/Python.framework/Versions/2.5/lib -L/opt/local -L/usr/lib -o PIL/_imagingmath.so
--------------------------------------------------------------------
PIL 1.1.6 BUILD SUMMARY
--------------------------------------------------------------------
version       1.1.6
platform      darwin 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
              [GCC 4.0.1 (Apple Computer, Inc. build 5363)]
--------------------------------------------------------------------
--- TKINTER support ok
--- JPEG support ok
--- ZLIB (PNG/ZIP) support ok
--- FREETYPE2 support ok
--------------------------------------------------------------------
To check the build, run the selftest.py script.
sh-3.2# python selftest.py 
57 tests passed.
sh-3.2#


She loves me!

These instructions work with PIL 1.1.7 and the older PIL 1.1.6.

To complete the installation, install the objects using:

$ sudo python setup.py install

Tags: ,