Google AppEngine – Java vs Python Performance Comparison
I am quite convinced that Google AppEngine is a very significant piece of “cloud computing” technology. More than other cloud components, it has been designed to scale from small, low request count websites to the big suckers. That is, until you come across your first Loading Request in the world of the AppEngine Java (and friends – Scala, Groovy, etc).
I found out about Loading Requests in AppEngine, when I pushed my first Gaelyk application to it’s home on AppEngine. I’ve been keeping an eye on the performance of my sites and applications using Pingdom monitoring, and unfortunately I saw something that made me feel a little sad when I saw the results.
| Platform | Avg Response | Slow Average | Fast Average |
| AppEngine Vosao CMS (JVM Java) | 1380 ms | 2063 ms | 938 ms |
| AppEngine Lift Framework (JVM Scala) | 506 ms | 600 ms | 438 ms |
| AppEngine Gaelyk Site (JVM Groovy) | 11,597 ms | 12,776 ms | 10,565 ms |
| AppEngine Site Proxy (Python) | 504 ms | 713 ms | 372 ms |
Comparing the performance of my Gaelyk skeleton site (WIP @ sparkz.distractable.net) with my own (pretty rough) Python AppEngine Proxy, the Vosao CMS running the Sidelab site, and the Lift examples on AppEngine you can see that Gaelyk really lags behind in terms of performance.
Update 2010-04-25: As comments around the place have suggested, note that the Scala Lift vs Groovy Gaelyk isn’t really an apples vs apples comparison. In my view this is primarily because I was running tests against the established appspot lift samples. I provided more detail on this in a comment below
Now, I had pretty much decided that I was going to use Groovy and Gaelyk in some future projects, but these loading requests really throw a cat amongst the pigeons. I believe some people are using cron jobs to keep the AppEngine JVM warm and prevent it from unloading, but this practice is discouraged by Google (understandably). Considering also that my pingdom check interval is 5 minutes if it’s unloading between those requests, then I don’t hold much hope until Google do something about this.
To that end, there are a number of things in the pipeline that will probably help things along. The ability to precompile was introduced in AppEngine 1.2.8, and while it does make a difference when you enable it looking at my detailed pingdom logs requests are still taking almost 10 seconds to complete. Additionally, there is an issue worth starring that if implemented will hopefully enable people to reserve a JVM instance. Doing this would bring response times of my very simple Gaelyk app down to around the 250ms time. Hurrah.
In the meantime though, I think I have to seriously consider either going back to Python for AppEngine development or investigate Scala and Lift. Any opinions? I definitely know more people who are passionate about Scala, but really, it’s still JVM based and definitely nowhere near as mature as Python. Excuse me now while I wipe my tears shed for Groovy and Gaelyk away.
While I’m doing that though, here are a couple of other useful resources on the topic:
- http://github.com/garyburd/app-eng-cold-start-test
- Google App Engine Loading Requests Happen Very Frequently @ StackOverflow
Update 2010-04-25: I have posted a follow up to this post in an effort to clear up a few points of confusion. Additionally, it looks at the whether making requests to your site every minute (instead of every 5 minutes) reduces the impact of a loading request.
14 Responses to “Google AppEngine – Java vs Python Performance Comparison”
Leave a Reply
-
About Me
I've got a bit of a reputation for finding new and interesting things to play with, and getting somewhat sucked into their "cool factor", it seemed only fitting that I have a blog called Distractable.
So take a look around, leave a comment and feel free to tweet me any blog-worthy distractions and I'll do what I can to do them justice here.
Thank you for doing some research and sharing your findings. I am currently using Gaelyk as my go-to stack because it is lighter than Grails but still ‘groovier’ than plain Java or python. Although, the downside of this is very apparent in your findings in this post. It would be interesting how this translates into actual dollars when an app has billing enabled and has enough traffic.
I hope there will be some more optimization that will help improve the performance of Gaelyk as I really enjoy using it and have invested quite a bit of time working out the kinks.
Cheers,
Royce
Hey Royce,
Thanks for your comment. Mate – I’m in exactly the same boat, and have a strong preference to use gaelyk in a production environment.
I’m going to investigate workarounds over the next couple of days, but if they don’t pan out then I think transitioning back to python might have to be considered.
For anyone who hasn’t already done so, make sure you star that issue on the AppEngine issue tracker so we can highlight to the AppEngine development team how important this is (especially if they want to make sure we don’t cron up requests to keep our applications warm). In my opinion this is a more serious issue than some others as it undermines what I consider to be one of the key features of AppEngine – it’s ability to scale and serve web sites/applications of any size.
Cheers,
Damon.
Go with Python! The Java Virtual Machine was my home for a long time. I developed with Java, Groovy and Scala. The Groovy based web framework Grails attracts me, since it looks as simple as Rails but is based on Java frameworks. But it looks only simply, while it is not simple. Under the hood are huge Java frameworks, Groovy is built on top of the JVM which was never designed for dynamic languages, what you will see on stacktraces. Scala is the best JVM language, but even Scala is very complex and the frameworks you will use with Scala are written in Java …
So I decided to leave the JVM and go with Python, since Python is very clear and leads to clear and simple software design. Python lets you focus on your real tasks rather than technical stuff.
You might be interested in trying JRuby instead of Gaelyk. As I understand it (I have not tried myself), JRuby + Sintra (a lightweight Ruby web framework) is only about 2x slower than Java, and works very well on GAE.
Can you compile with groovy++?
I agree with Christian. I spent way to many years in the JVM before finding the joys of Python,
Hey Guys,
First a big thanks to those who have commented and to those how have given the post a plug in reddit, dzone and on twitter – biggest couple of days I have had in traffic ever.
I’ve actually been away for a few days and wasn’t aware the post was attracting as much interest as it has been. From that perspective I guess I’ll just add a couple of pieces of information that are becoming apparent through the pingdom monitoring:
1 – Groovy and Gaelyk have gotten a bit of a bad wrap in this post as the comparison with the Lift framework examples on AppEngine isn’t really an “apples and apples” kind of affair (The Lift examples have been there for sometime and will probably have more established traffic patterns). After running up Lift myself and hosting it on AppEngine response times are more in line with what I’m seeing with Gaelyk.
2 – My installations of both Groovy Gaelyk and Scala Lift are showing slightly improved performance over time – not every pingdom request results in a high response time now. It’s more like 1 in every 5. The pingdom average response times still tell a fairly depressing tale though (~ 10s AVG, 15s MAX, 5s MIN for both).
I think it’s pretty clear that if you are a fan of any JVM based languages and Google AppEngine you should be petitioning Google via that tracker link I posted in the article to get the situation sorted. Regardless of whether or not response times improve as more and more people start to use your website or application, I don’t think Google would have been comfortable launching their search engine back in the day with a 14 second response time (maybe that’s why they picked Python back then, I don’t know). If they had, I dare say we still be using AltaVista or Lycos now :/
This was never a post designed to bag JVM based languages, as I personally enjoy working with the – especially Groovy.
If you do work with the AppEngine team in Google and can shed any light on what JVM AppEngine fans can do to improve the performance of their applications, or when and if the situation will improve without any special workarounds, I know that I (and probably others) would love to hear what you have to say.
Cheers,
Damon.
Can you change your Pingdom check interval to 1 minute? Give that a try and see what happens…
Hey Sean,
I’ve tweaked the 3 pingdom monitors to a one 1 minute check interval and I’ll report on the results once they’ve been running for a day or so. I was trying to steer clear from doing that as I didn’t want to go down the path that was discouraged by Google (i.e. cron jobs and the like to keep the JVM warm), but I’m interested in seeing the difference in times also.
Cheers,
Damon.
[...] Google AppEngine – Java vs Python Performance Comparison Interesting results [...]
[...] AppEngine Python vs Java comparison. [...]
[...] AppEngine Python vs Java comparison. [...]
I also abandoned Java web development for Python+Django and I’m so happy developer now
Also for me, I spent some long years with Java complexity before discovering Python simplicity!