Use of SQLLite too - does that scale and perform well against something like postgres?
I think what we're seeing here is an email/hype-grabbing attempt with an interesting vision, using the Heroku name for leverage.
Would like to see it materialise, but they really should have hardened something as simple as a signup form. I've emailed the admin telling him to lock it down.
To be fair, you wouldn't need much more than SQLite to gather emails. It's more than sufficient (I would say it's absolutely amazing for most use cases, but the one thing I am not sure about is how well it handles concurrency, if at all).
I tried SQLite for a few things, and it has always come out a champ. I would not hesitate to use it for most of the semi-static websites I make (web apps would be pushing it), if I were sure it can handle more than one connection (I'm not, sadly).
I am very, very impressed by it. Seriously solid database.
SQLite is a file and thus uses file-level locking and does not have row-level locking. Data can not be read the same time anything is being written, so it is usually not great for multi-user applications or web applications.
Development, or 1-3 user web app it should perform OK. It doesn't scale and you will get db locks and retries if its under too much load.
To speed this up (if you must use SQLite), use a solid state drive as I/O performance is the biggest bottleneck.
actually it could use row-level locking since it uses row-oriented storage and fcntl locks can be applied to a set of bytes within a file. it could also do file-level locking on tempfiles with names derived from the table name and primary key of the row.
anyone with actual knowledge of it care to chime in with what it does do?
I have actual knowledge of it. The product I work on uses it as the production database because we distribute the software as a desktop application (and MySQL has licensing issues).
It COULD do row level locking, but it doesn't. SQLitening is the closest thing to performant SQLite you will get. And it only runs on Windows.
I regularly use Fabric. After deploying a couple of projects with it, I have come up with a standard deployment layout and stack. Now its as simple as "fab deploy" and "fab revert". Thank you fab!
Same here, except I've set things up so that any changes I make to my project are automatically synced instantly to a live staging domain, and I don't have a revert command (although I do back up copies of previous deployments). I also keep apt and pypi dependencies in project-local config files and have a refresh command that updates those packages on both the remote and local machine along with some other stuff. I think I might have overengineered things a little.
If it helps, "li" is my current project's code name.
The vast majority of the code is concerned with configuring a fresh Linode, not pushing recent changes (for that, see the rf and stage functions, and the functions they call). I hate system administration. The only way I could make myself configure my server was by writing a script to configure it for me.
I'm wondering what the best practices are and since I have so many people around me who use Capistrano I'm thinking about just going with that. Any words of advice?
Never write form confirmations ("Thanks for your email") in red color. Looks like a warning. If a form is processed correctly, it should be black or green.
I hope that usability of djangy service will be better. Can't wait to try it :)
Btw, isn't Heroku itself planning anything for Django?
That's what I'd expect. They're only still in the early market penetration stage of the Ruby market; it doesn't make a whole lot of sense to complicate their product offering. (Particularly when App Engine is available for Python/Java.)
Hopefully it will work with any WSGI application because even Django applications might use WSGI components and thus have a custom WSGI application object.
App Engine is a pain in the ass, and I wouldn't recommend it to anyone. Heroku is so simple to use that I've many times felt that someone needs to build a Heroku for Python.
The only thing really wrong with App Engine is the datastore:
- It's very different than anything you've come across before
- The docs are not sufficient for it's complexity. You end up having to read lots of blogs and watch lots of videos to figure it out.
- It is ridiculously difficult to learn (one of the hardest things I ever had to do)
- Important parts of your stack won't work, like Admin
- You can't just add the fields you like to an object. Instead you need to meticulously plan _how_ your data is going to be accessed. This is exactly the wrong thing for a startup, since you can't get up and running fast.
- You can't iterate fast, because any schema change is very risky and very slow.
- If you don't get everything exactly right, your app will be very slow.
- There isn't much in the ecosystem.
By contrast, none of these are true for Heroku.
There are some things that are pretty good about App Engine:
- the tools are pretty good. Not as good as Heroku, but good.
- the billing is spot on. They give you about $1 per day of free stuff, which can maintain a site with hundreds of thousands of users. And it's pretty cheap from there on.
- They scale it automatically. On Heroku, it's really hard to tell how many "dynos" (whatever the hell that is) you need, except by trial and error. I would expect that nearly all Heroku apps are overprovisioned. App Engine just nails the scaling - it happens automatically, and you specify a maximum amount to pay per day, with lots of different dials to control it more precicely if you like.
While I do agree that the App Engine datastore does require for some planning above what you're used to from a traditional RDBMS, I think it really just reflects a learning curve that comes with any new technology: as you use it more often, you get better at it. I think calling App Engine a pain in the ass and that you wouldn't recommend it to anyone is a bit over the top.
No, I truly meant it. Everything related to the datastore is difficult and tedious. Goodbye to incremental development, goodbye to knocking out a feature in an hour. Every single change you make has to be planned.
Adding a field to an "object"? Where do you use that object? Is there somewhere that you don't use that field which becomes too slow?
How are you going to migrate that new field? There aren't tools to help you migrate. Generally, you have to change to an Explando, add the field, load every single object from the datastore, then save every object back to the datastore.
These are two examples off the top of my head. Every change you want to make in App Engine has similar questions raised. I really did mean it when I wouldn't recommend it to anyone.
I think it's a much different learning curve than with any other technology. I'm plenty adept at learning new things, but this was just unnecessary complex, with nothing to redeem it.
I'd have to agree that Django on appengine is a big pita for the reasons you mentioned. Django is a large framework, and was simply not designed with interchangeable backends in mind. Getting it to work either requires a lot of black magic monkey-patching or foregoing the majority of built in features.
On the other hand, I have found the datastore api (and most of the other apis) to be well designed. It works very well with lighter frameworks such as tornado.
It sounds like your problem with App Engine is that you're Django. Yeah, Django is not built for the App Engine data store, so why not use something that is? I've used the Tipfy and Flask frameworks (both are a thin layer on top of Werkzeug), and they experience has been great.
Not true, unfortunately. For example, creating objects and deleting them uses a different method (put instead of save). There are tiny syntax differences, also.
Not to mention the fact that designing an information schema for a SQL database is quite different from designing an info schema for a hash-table like datastore (which appengine has).
Maybe by not requiring any modifications of Django to get the basics running? Allowing standard databases? Or even the popular thing lately - by not being Google.
Or maybe they'll enable more stacks? Pylons shouldn't be that hard to plug in once you have Django deployments working. And since they reached 1.0 lately...
I would love to hear more about this. I'm currently in the middle of a Django app, and this would be a godsend. I like doing sysadmin stuff, but there's no way I will have time in the next year.
If you guys can pull this off, I will be a very happy customer.
Would be nice if there were something like this (like Heroku) for Java. I know that there are several "cloud" solutions for Java, but I haven't found something as simple to manage/extend/deploy like Heroku so far :(.
I used Stax during beta, and it's nice, but it's no near the Heroku ease of use (puls it's little helper Nezumi http://nezumiapp.com/ when on the way). With very small restrictions, one can install anything on Stax, so it's very flexible, but just not that easy to use as Heroku'.
I've been working on my Heroku for Django (or Python more generally) for a month or two now. It's good to see somebody else thinks this is a good idea.
The front end dispatches requests to a Varnish, which is in front of a router that queues up requests and sends them to available copies of the desired application. Persistence is Postgresql, and delayed and scheduled jobs are handled by celery over RabbitMQ.
My number one goal is to provide a deployment environment with as few additional restrictions on the developer as possible.
I wonder why Dave Paola (apparently admin at djangy according to DEBUG view at http://djangy.com/postreceive ) used room606 instead of his regular account endlessvoid94 for submitting this.
Because he doesn't want to be outed, for whatever reason. Outing someone by calling out their real name is a dick move. It used to be that HN recognized the occasional need for stealth pre-launch, instead of this braggery.
(Disclaimer: same thing happened to my company when we pre-launched.)
I forgot to turn debug off. Yep. And i'm using sqlite3 to accept email addresses. Yep.
I can assure you the quality of the ACTUAL project is far better :-)
I have a working prototype and am working my ass off to get something scalable up and running asap. keep checking back for details!