Disclaimer: In a past life I was a PM on App Engine (and not surprisingly, we're using it for our new company), so the following comment has a fairly high bias.
Though it doesn't have the same HN buzz that Heroku does, App Engine has a fairly large developer community and a lot of applications using it. Like almost all PaaS product, there are restrictions that limit what you can do (library support, native code support, etc) and these limits frustrate developers that go in expecting the same experience as having a virtual machine. The limits mean that App Engine isn't the right solution for every application under the sun. But if your application doesn't have super unique requirements, the value that App Engine provides is hard to beat.
Ten second deployments, one click version management (and roll back), logs and monitoring, and autoscaling mean that we can be really agile pushing new versions of the application. App Engine's recent changes to the scheduler and Always On feature have really solved the slow instant start issue that still plagues a lot of Heroku developers. Last year the datastore had a ton of performance and reliability issues, but the new high replication configuration appears to have solved the problem. The biggest issue for us is the one that's mentioned here a couple of times and that's SSL.
For now, App Engine is nearly perfect for us. If we end up with unique requirements, we can definitely spin up a few EC2 instances and hook them up to App Engine's Task Queue. But I'm not excited about having to manage machines. Personally, if we hit a limit with App Engine, we'd be looking to another PaaS offering like Heroku or dotCloud, or Infrastructure as a Service like MongoHQ/RedisToGo before bothering to deal with machines again.
We use it at bynd.com for a bunch of projects; we've deployed at least 30 applications on it now. It has taken me quite a while to get used to the AE way, and I don't love it, but it is very good for some things. I can think of lots or pros and cons, but off the top of my head:
Pros:
- scaling is amazing; we've had sites go from 1 to over 1000 instances in a few minutes; from <1 to 1600 queries per second in the same time.
- reliability is pretty good now
- if you let go of the idea of doing everything synchronously life gets a lot easier, and deferred processing is really easy to do.
- the development process is great: download SDK, checkout code, point SDK at code and you're off. Then you develop locally and have it run exactly the same (with a handful of caveats) as it does appspot. It's a really low barrier to entry.
Cons:
- webapp (the default framework) sucks. Once you find something like tipfy, webapp2, or flask life gets a lot better
- changing your datastore structure can really suck with a live site. There's still no simple way of getting data on and off, or of doing backups. Messing with your live database if you need to fix some bug can be scary
- it takes some time to grok the right way to model data, and to use the datastore.
- there's a lot of "here's how to get started / build a simple blog" info, very little on best practices, how to model complex data, how to do maintenance...
What framework did you wind up using? Tipfy seemed great at first but the docs aren't quite up to date, so I wound up sticking with webapp for the prototype for my project.
App engine has the following awesome characteristics:
- the datastore is extremely scalable if you use it as intended.
- your app can automatically scale up and down (as if an invisible genie were moving the little "workers" slider that Heroku uses)
- always on instances make your app mega fast, even at low volume (but cost a tad bit more).
- there are a lot of useful apis (memcache, queue, email) and a nice development environment.
- it usually "just works" and "just scales".
a few bad things:
- they had some very annoying downtime while they sorted out the datastore architecture problems (where theory and practice had differed with horrible results)
- the OSX dev tools don't bundle the correct version of python and associated libs, which makes it a pain to get a working dev environment on OSX without breaking your system's python.
- no naked domains, SSL support for normal apps that aren't priced per user.
All in all I'd rather write an app for AppEngine than any other cloud system, though Heroku is catching up -- both could really benefit from prioritizing 24 hour support via irc though.
App Engine has a lot of really awesome features going for it. There are some pains for sure, but not the ones you find people typically griping about. The datastore and all of that aren't a big deal. What's annoying is the lack of library compatibility, and no real SSL suport. Having access to a relatively small subset of available libraries for Java/Python sucks, each time you find a library you'll have to find out for yourself it it works on AE or not.
As for SSL. I mean come on. It's 2011, it's absolutely unbelievable they don't support SSL for custom domains. Unbelievable.
FWIW, offering SSL is a giant pita considering that it requires a unique IP address for each endpoint. Scaling that is not easy. Add the diminishing pool of IP4 space to that and you've got something that you have to think very carefully about.
Completely agree that it's a much needed feature. We've worked around that by setting up an nginx server in AWS which has proxies to the https://*.appspot.com name. So it's still SSL all the way, but we just didn't want appspot.com as the URL we gave out.
The UK's Met Office - who provided Weather Forecasts use it for their new Weather Observations Website (WOW): http://wow.metoffice.gov.uk
It's a site that allows the public to submit weather observations from anywhere in the world. Since launching 31 days ago, we've got 1.7 million weather reports and probably spend about 0-30 seconds a day (on average) on maintenance.
App Engine has some big pains (SSL has already been mentioned) but if you don't require it (we don't), I think it's a fantastic tool.
The multiple versions, quick deployment (new build inc all testing&configuration can be completed by us in <5 min), automatic scaling and the fairly good dashboard are fantastic. Plus, our infrastructure costs in development (before heavy load testing) was $10 for 3 months of work - absolute bargain! Even after heavy load testing, it was a fraction of what a real server would have cost.
I'd say the biggest difficulty (or barrier to entry) is the datastore design. It's very easy to think you're in SQL land, and you're not. And because of that, it's very easy to burn yourself quickly. But, the design gives great scalability. We wouldn't be able to manage the size of the data we're expecting in 2 years+ in a typical environment.
So, I think it's great :)
(Disclaimer: I was part of the team at PA Consulting that built WOW)
I don't know what big sites use App Engine but I personally use it for everything. Every weekend project, every work prototype, everything! I just love the fact that it's free to start with, and the idea that it's on Google's server sort of brings a level of confidence for up-time and scalability.
Sure the fact that are some limitations (no IO and not many libraries) are a bit of a down-side, but the payoffs of using such a service instead of managing your own servers is really great. You're just able to crank a lot on the app code part without worrying at all about system administration work.
The simplicity of the datastore is also a great benefit, and it scales like nothing out there.
I use App Engine a ton. I won't try to list all the positives.
Of course "is it flexible enough to run big, serious projects" depends on the requirements of your big serious projects. Feel free to get more specific here, or (better yet) in the google app engine forums.
One thing not mentioned yet that is a legitimate concern applying generally to PaaS offerings including App Engine - that is vendor lock-in. There are some way to address that including choice of apis and frameworks and also projects like AppScale and TyphoonAE.
There is much to evaluate when considering App Engine for your projects. the flexibility of the platform from a technical perspective is one of things.
Also note, watch closely the changes that are coming in pricing and the way that Google is offering the service. You don't want to get caught evaluating the App Engine of 2010 for 2012.
We use it here at Optimizely and have been very happy. It hasn't been smooth sailing all the time, and we've gradually grown our stack to include EC2, S3, and several CDNs as well, but overall we're very happy with it and have no plans to leave.
The biggest benefits for us are the scalability (huge), ease of deployment, and the monitoring. The biggest cons have been lack of support for SSL on custom domains (HUGE hassle) and lack of official support.
I'm interested to hear that you think monitoring is easy - it was something I forgot to add to my "cons" list; all you get for free is the web dashboard that lets you look at logs, and ereporter to send you daily exception summaries. One of my biggest "want"s in fact is API access to logging.
Google itself is using App Engine on both internal and external projects as far as I know.
The websites below has tens of thousands of registered members, there are also similar websites for some other European countries (Italy, Spain, Germany etc)
So yeah, App Engine seems to be suitable for big projects if you play by their rules. Only problem seems to be backing up data, as their App Engine's bulkloader tool has rate limitation, so it takes quite a while to download/upload all data.
We've put Nginx in front of GAE and it works without any issue. I haven't tried any other methods such as Varnish though. We aren't really throwing too much traffic at the GAE app which is "behind" Nginx, but we fully expect it to handle a reasonable amount of traffic.
I'm sure there are many people who have done this. I'd be interested in hearing how other people have solved the lack of SSL...
Appengine is extremely good for auto scaling, if u have 100 users today and and u get 1 million users tomorrow, it will just work without u having to do anything, u dont need a scaling engineer.
I have used AppEngine for a few fun projects where I have tried to use FB Login/Connect for authentication. I like the fact that you can rapidly prototype and improve your skills and have it all published to the web for free.
Question: I was looking for a library for authentication mechanism other than FB or Google . Any suggestions ? (All I need is basic user/password)
I'm using Google App Engine in a new project ( http://www.homeyplanet.com - allowing travelers to share their favorite spots off the beaten track) and I have to say I am very happy about it.
This project requires geo spatial queries, so at first it wouldn't be a good choice given how inadequate the datastore is for it, but the trick is always to think how things work in GAE and see if you can optimize it. It only scales well if you do it their way...so it won't work for everything, but most problems can be solved quite well, you will just have to scratch the way you normally would solve them.
Note: I noticed that some have accessed the site from this link. The site is beta and is newly launched, and for now we are only covering Europe. If you want to check out how fluid are the geo spatial queries, you can try checking out Portugal, where half our spots are now.
I used appengine for my now defunct http://www.unedditreddit.com project, which sucked up all reddit comments for a few months. It was extremely easy to use and seemed to preform just fine, with the exception of those few weeks where the datastore had stability problems.
Windsoc uses App Engine for our Unified Social API, and while it has many benefits, we've always had our minds on how to move off of it someday. SSL is a big issue for us, and another is the support model (which should be changing soon from what I've seen).
So what we have now is a product that runs both on App Engine and on DotCloud. Since DotCloud is awesome, too, and we know that we can talk to a specific person when something goes wrong, it's currently a front-runner for us. We're also looking at Joyent, too, as opposed to EC2.
But even if we switch large parts of our product to somewhere else, I wouldn't be surprised if we still used App Engine for some things.
I was using this on a project for a while. But compared to running the application with PHP, it was always slower. Sure, I've developed applications with PHP since 2001, so I really know how to tweak it and optimize it. But, with App Engine it was really frustrating. Requests really took long to complete, and there was no real way to figure out what was going on behind the scenes. We created an 'always on' instance, but it was still quite sluggish compared to the exact app running on a standard linux, apache, PHP system.
I would still try it again down the road, but it's really not a good fit for what I'm currently doing.
Just out of curiosity, how would you describe what you are currently doing? Can you elaborate on why it might not be a good fit?
I'm honestly just trying to learn from your experience, not disputing anything.
Sure - no problem - the application is Generic Domain Finder - just search it on Google. The current app in the search results is still running GAE.
The things I loved:
- tight integration with Google accounts
- easy integration with other Google apis (we used a few)
The frustrating thing was dealing with the black box. We used an api to get domain availability. Every time it returns more slowly than the equivalent call on our linux based cloud server.
With GAE, you're basically stuck with what you get. With a generic Linux based cloud server - you have more room to optimize your environment, or even take your code and run it anywhere... AWS, Rackspace, Voxel...
I still believe in what GAE is trying to do - and I'll definitely try it out again for future projects.
We use it at www.bugsense.com and we love it. It has lot of great apis included but it requires a mind switch and you should study the limitations. Datastore is really scalable but slow.
It takes you a bit to get used to the catchas... I am using it in one of my side projects and it's working very well although we had some throuble to move from SQL to the BigTable and that slowed us down a bit in the architecture phase.
But I am betting on it because I think it will make my life easier in the future...
The memcache is ok to get.
The java subset is the part that bothers me as I had a lot of already made stuff I liked to use in my new site, and had to redo a loooooot of stuff to fit the dawn java subset.
I've used App Engine for web projects and backend server for Android apps. My web TV site http://www.topchan.tv, and the webpage snapshotting site http://www.previouslook.com are in App Engine.
I have been using GAE Python since it came out. Used it to build www.thesponty.com and www.gamedaytycoon.com (social game on FB). The latter has thousands of users and GAE scales super well. We deploy > 10 times a day. GAE is really awesome.
Optimizely. I can't remember other big ones but there more. App engine has proven to be serving millions of requests per day without any issue. There are some case studies around, also some web caches hosted on app engine.
I've only just started using it (this week) as an excuse to get my feet wet with Go. It seems easy enough to get started, things are well documented so far though I've admittedly only touched about half the packages.
What about the channel API, any experiences? I want to do a multiplayer quiz app as a side project and I'm wondering if I should go with the GAE or if I should dive into node (using socketstream, maybe).
We use GAE for Pulp Sync (acrylicapps.com/pulp). It's scales very well if you handle requests quickly and take advantage of the memcache and taskqueue APIs.
Though it doesn't have the same HN buzz that Heroku does, App Engine has a fairly large developer community and a lot of applications using it. Like almost all PaaS product, there are restrictions that limit what you can do (library support, native code support, etc) and these limits frustrate developers that go in expecting the same experience as having a virtual machine. The limits mean that App Engine isn't the right solution for every application under the sun. But if your application doesn't have super unique requirements, the value that App Engine provides is hard to beat.
Ten second deployments, one click version management (and roll back), logs and monitoring, and autoscaling mean that we can be really agile pushing new versions of the application. App Engine's recent changes to the scheduler and Always On feature have really solved the slow instant start issue that still plagues a lot of Heroku developers. Last year the datastore had a ton of performance and reliability issues, but the new high replication configuration appears to have solved the problem. The biggest issue for us is the one that's mentioned here a couple of times and that's SSL.
For now, App Engine is nearly perfect for us. If we end up with unique requirements, we can definitely spin up a few EC2 instances and hook them up to App Engine's Task Queue. But I'm not excited about having to manage machines. Personally, if we hit a limit with App Engine, we'd be looking to another PaaS offering like Heroku or dotCloud, or Infrastructure as a Service like MongoHQ/RedisToGo before bothering to deal with machines again.