Python's greenlet libraries such as Gevent & eventlet makes websocket type of applications possible. It is not difficult to add in websocket support into WSGI frameworks such as Pyramid/Flask. One example http://blog.abourget.net/2011/3/17/new-and-hot-part-4-pyrami.... You can have both WSGI and websocket in the same stack.
Support for websocket is not yet fully baked into Mongrel2. The handshake etc needs to be done on the handler side. So gevent solution is a better bet at the moment. The alternative is to have a separate Tornado/Node.js to handle websocket alongside Django/Pyramid app. Mongrel2 does look promising, although it seems zedshaw is pretty resource tide at the moment.
Don't run on your whole app on gevent, but only the part which needs streaming/long poll. By decreasing the surface area, you can pin point the bugs arising due to monkey patching.
I'm really interested in this for building simple API endpoints. I tried a bunch of WSGI servers and lightweight frameworks to serve stuff on them, but realized that there is no way to support chunked request content encoding with WSGI (the WSGI spec is not compatible with it, so WSGI can't support HTTP/1.1). Hopefully Brubeck is good, please let us know if there's anything else worth checking out!
While we are at the topic of Python web frameworks, I seriously like Flask's design better than django's. True that django came out before SQLAlchemy or Jinja2 became popular/existed, but I would have loved it had they re-factored.
Django is great, with good re-usable components, but I like Jinja2 more than Django's templating. The argument for SQLAlchemy is more on the lines of I would prefer a standard ORM compared to everyone baking their own. SQLAlchemy used to be verbose, but the declarative extension now takes care of it.
Other things I like about flask are explicit app object, and context locals.
Totally agree with you on this. I think Flask is Django done right and I've been beating that drum for quite some time (see, for example, my comments on the settings file [1]).
The core devs claim that SQLAlchemy and Jinja weren't around at the time Django was developed, and that's fine. But in 2012, Jinja2 and SQLAlchemy are clearly better alternatives to Django's defaults. So why not incorporate them?
Django doesn't even have first class support for Jinja2 templates, let alone the python HAML variants.
What's ironic is that a few years ago Django was known for being loosely coupled when compared to Rails, which was considered monolithic ("my way or the highway"). Nowadays Rails is the flexible framework, allowing for different ORMs and templating systems, while Django is now the monolithic framework.
Don't get me wrong, I love stability. But when a great percentage of developers are using hacks to get around the default configuration system and to support excellent external libraries like Jinja2 and SQLALchemy, something is very wrong.
First, it's quite simple to replace Django's templating engine with Jinja. And second, Django's ORM isn't quite the equivalent of SQLAlchemy; the latter is supposed to be a true universal ORM, while the former is more limited, but focusing on the most important parts for a Web app.
> First, it's quite simple to replace Django's templating engine with Jinja.
Yes, it is. But most of the applications are tightly coupled with Django's ORM(auth, for eg, which I use very frequently), and in the end, I end up maintaining two types of templates which I don't like.
> And second, Django's ORM isn't quite the equivalent of SQLAlchemy;
No, it isn't. But SQLAlchemy does what django's orm does, and then some.
I would prefer a compartmentalized model, where a web framework doesn't implement templating or ORM, provided a robust solution exists. The templating engine and ORM should be independent libs, and the web framework should glue them together, possibly adding a declarative layer above them(render_to_response, declarative ORM etc).
We've been using Flask recently, and I do like the simple, no-nonsense approach. Our project is modest in its library needs, and it's nice to have a useful toolkit without imposing a heavyweight structure on our code.
There are significant limitations with some of these frameworks today, though. For example, lack of byte range/partial content support is bad news if you're serving large files (think multimedia).
We're still investigating options here, but WSGI doesn't seem ideally suited to working that way, and as far as we can tell quite a few of the "microframeworks" lack any kind of support for returning partial content automatically. The concern seems to be that WSGI would need the application to generate a complete response even if it was going to serve only part of it as a 206, because there's no standardised part of the routing set-up that says "I need these byte ranges from whatever full response you would have given me".
A common philosophy seems to be that such files should be served statically by your front-end web server anyway, but that is not sufficient in all cases. For example, you might need to process the request through your framework to generate the response on demand, implement access controls, or integrate with a custom logging/analytics framework.
A common philosophy seems to be that such files should be served statically by your front-end web server anyway, but that is not sufficient in all cases. For example, you might need to process the request through your framework to generate the response on demand, implement access controls, or integrate with a custom logging/analytics framework.
Depending on your use case, you can either use X-sendfile header - validate request, then return a X-sendfile response for your front-end server; or use gevent to stream the response.
In case it helps anyone else: It's something about the way that Flask handles a direct send_file that seems to be causing problems in our tests. However, we've found reports of sendfile not working properly on certain platforms and we're still investigating, so I'm not convinced at this stage that the problem is with Flask itself.
> However, we've found reports of sendfile not working properly on certain platforms and we're still investigating, so I'm not convinced at this stage that the problem is with Flask itself.
I am confused by your sendfile not working properly on certain platforms. If you set the X-Sendfile header, and the web server supports it, it should work fine. Flask send_file doesn't do much other than setting up the headers; and then either setting the X-Senfile header so that web server handles it, or sends it using WSGI file wrapper supports.
I was referring to the sendfile system call there. We've seen reports that on some versions of Linux kernel, it doesn't work, for some value of "doesn't work" that we haven't yet properly determined. We've also seen web pages that suggest Flask's send_file and send_from_directory would ultimately rely on that system call in some circumstances, though again I haven't checked through the code in detail yet so I don't know how accurate those reports are.
In any case, we haven't tried using the X-Sendfile functionality yet, and I'm making no comment about how well that works.
We do know, without any doubt at this point, that serving our video files using Flask's send_from_directory (using the default file wrapping behaviour, not X-Sendfile) is not working reliably for us with many different clients, where serving the same file statically straight from a web server like Apache works fine. We're still trying to identify exactly what it is that doesn't work, but in light of our discussion here today and what I've learned since then, I'm now hoping that we can just punt the whole set-up over to the known-working Apache implementation by using X-Sendfile and avoid the problem in the first place.
(Edit: Thanks again for the suggestions, BTW. I had a fairly long list of ideas to follow up in relation to this problem, including investigating X-Sendfile, but your mention of that here prompted me to look into it first and will probably have saved me quite a bit of time if it does work for us.)
That's environment specific; the wsgi server should define a environ['wsgi.file_wrapper'] if it needs to provide specific file handling functionality. I don't think you have an issue with Linux kernel - 2.2 and beyond will always have sendfile. It's more likely that your wsgi server isn't defining environ['wsgi.file_wrapper'] to something that uses sendfile.
If you see the flask send_file code, it's using werkzeug.wsgi.wrap_file, which in turn either uses environ['wsgi.file_wrapper'], or uses simple Python code to read the file and send the contents.
Reading file and sending the contents in your python code is a bad idea. You can either set debugging points and see if wsgi server is defining environ['wsgi.file_wrapper'] properly. Or more conveniently, you can just set 'USE_X_SENDFILE = True' in your flask settings, and then directly use send_file.
We're working with flask at the moment, and I have found it excellent. I really don't want to be bothered by anything to do with the web framework because I am already busy with the other stuff, and I've been able to ignore it all with flask because it is simple and lightweight it works.
We used pylons for the last project, and there are areas I still don't make improvements on because it is too much effort to get into it.
"There should one - and preferably only one - obvious way to do it" is applicable only at language level, not at application level. These are just another applications and one cannot enforce a rule that "there should be only one application/solution for a problem at hand", is it ?
There is no Pope of Python to tell you "you can't write new code to do xyz, because someone else is already working on that."
How would that be enforced? Would someone's Python privileges be taken away?
What if the person working on xyz was inactive for a while? What if their xyz had a problem and they didn't want to fix it?
The "obvious way to do it" does not and was never, ever intended to bind anyone's hands from writing a library or app similar to something someone else wrote. It is a principle of language design, that it should give SOME consideration to readability and learnability rather than giving 100% of everything to nifty obscure features that help you write awesomely clever executable line noise.
If you read to the end of the article it tells you (Implies) that making Yet Another X is a bad idea, and that you should really go work on an already established project.
EDIT: Or rather, skimmed to the end of the article. (Like I did because I have no interest in building a python web framework and thought the title was sarcastic.)
Rumor (and by rumor I mean a couple of opinionated coworkers) says: the Python community has a lot of cases of "This project didn't have the feature I needed and has a couple of problems, so I went and wrote a whole new one with a different set of problems." By contrast, the Ruby community settles down on one, maybe two things as the primary way to do it.
It works the way that most open source third party libraries do: There'll initially be some hacked together stuff, then a couple of contenders will emerge, copying each other's features. After the dust has settled there'll be a few decent libraries.
Examples that I can think of off the top of my head are freshen/lettuce/behave, there were also a bunch of XML parsers leading into elementree (which is now in the standard library). In Ruby you have Rails, Sinatra, and a couple of other frameworks, as well as Mongrel2 and friends to run it all.
Yep, except nobody cares about "the rest" and Rails becomes almost de-facto choice for web-app with Sinatra trailing for the so-called "API" project (whether that choices make sense or not is not of my concern, in Java, API == XYZService.java with a facade skeleton implementation utilizing JAX-WS [SOAP, WS-*], JAX-RS [RESTful], or Servlet (Web) all deployable easily in 1 JAR).
Well, ditto with Python and Django these days. There are other frameworks, each with their own strengths (werkzeug, web.py, bottle, etc.), but Django is the default choice.
Back when this article was written there were at least five or six Python frameworks, all competing. If Django ever starts sucking, I'm sure one of them, or a completely new one, will step up to take it's place. Ditto for Rails.
While some people may choose Django per default, there are also quite a lot of people that will choose Flask/Werkzeug, Pyramid or one of the less popular frameworks.
The Python web development community is very much divided into different, although largely cooperating, groups.
Django is definitely not the default choice when it comes to web development in Python and I doubt anyone seriously involved in the Django project would ever claim that.
Django has such numeric superiority that there is not a lot of point splitting hairs on whether it is "default." While it isn't in the stdlib and there are other good choices, Django might as well be the default, just as Rails might as well be the default in Ruby.
You sure about that? It is my understanding that Zope/Plone has a fairly sizable community. They're just not as loud on the web as other framework fans. No one will know for sure until they are all counted though.
I'm heuristically pretty sure. From my personal experience talking to people, and from some surveys or other I have casually looked at over the years, what is mentioned most in job ads and mailing lists, heuristic stuff like that. The same kind of stuff which leads people to say that Rails is much more popular than other Ruby frameworks.
I would love to see a scientifically exact census but I don't think it can be done. Maybe PyPI could roughly tell the story? But you won't get a real unique-users count unless your logging identifies unique users, who wants that?
Please understand: I am not a big Django promoter, I disagree with many of its design principles, and I don't think that other things are "dead". I know that there are reasonable numbers of people out there using Zope/Plone, Flask, Pyramid, and other things. Just because Django is huge doesn't mean they are nothing or not worth looking at.
I believe Rails is significantly bigger than Django, but that doesn't mean I'm switching to Rails.
I would disagree with that fairly strongly. While there are other choices for specific niches, for most web development Django is the first thing that most Python developers will look at, mainly due to it's user base and libraries.
i would say django is the default choice for "yet another website" with some application logic sprinkled in. I wouldn't use it for anything other than a website serving up dynamic pages. I also feel I am fortunate to not have to build those websites anymore.
Seems like it too. I like Python language because whenever I read them I can guess/kind of know what it's trying to do (unlike Ruby) despite my short learning session of Byte of Python.
But when it comes to web-framework, boy... this one is tough. I lost count on how many Python web-framework or web-library out there (feel free to debate the semantic of Flask vs Bottle vs Django vs Pylons vs Pyramid vs Plone vs Zope).
It may be easy to link together well developed Python libraries for web development, I think it's one of the benefits of working in Python, but full stack frameworks like Django that provide a somewhat unified API exist at least in my mind on a different level of framework. I think there is a spectrum of complexity represented in the list of frameworks linked by the author, and the word "framework" is used to describe a lot of different kinds of situations.
A lot of people will make snide remarks when someone says they are creating their own framework, but obviously if someone has evaluated the existing frameworks and still decides to do it, there is an unscratched itch. A lot of people who aren't happy with the larger monolithic frameworks have adopted flask (myself included) as a simple base to build more interesting things on.
There are a number of things no framework I have yet run into does well:
1. Unification of client side and server side events. With options like Backbone or (ugh) ExtJS on the client side, there is a lot to be gained from a coupled event subsystem over websockets, or via server sent events.
2. A true viewmodel based page composition framework, where components are abstract view elements with data bindings on the server side, which then have associated renderers for the output to the client.
I've played with both of these, and found them to have a lot of advantages over standard template + ajax + callbacks style of design, though creating a library for others is a lot more work than hacking something for yourself :)
People writing tools to learn, or because the existing ones didn't suit their needs or tastes, occurs in every language and is very different from Perl-style "more than one way to do it".
It's ridiculous to act like the mere existence of multiple independent projects for one task is some kind of searing indictment of the "one obvious way" principle in language and API design.
I've never actually noticed "one obvious way". There are a couple ways to do almost everything in Python; loops or map, Twisted or threads, unittest2 or nose, and so on. I like Python, but human nature pushes everything towards "there are many ways to do it", and humans have influenced the direction of Python.
I prefer Perl's approach of embracing more than one way to do things. I don't really want there to be 8 ways to do one thing, but it's just a more realistic outlook. You go into Perl knowing that you are going to have to try a bunch of different things to see what fits your mental model, instead of being told what to do. (I used Python at a Bank where they wrote their own style guide, completely different from the standard Python style guide. WTF? Humans have a way of ruining everything.)
Ironically, many things in Perl have converged into "One Obvious Way"; PSGI/Plack for web frameworks, Test::Builder for unit tests, and so on.
> There are a couple ways to do almost everything in Python; loops or map, Twisted or threads, unittest2 or nose, and so on
There are always more than 1 ways to do something, but Python tries to be directive towards the recommended way for the core.
For the given snippet, using reduce for loops is frowned upon; nothing is stopping you from doing it, but the general opinion is you shouldn't do it. I think that counts as there being an obvious way.
def pipe(val, fns):
return reduce(lambda val, fn: fn(val), fns, val)
def pipe2(val, fns):
for fn in fns:
val = fn(val)
return val
fns = [lambda x: x + 1, lambda x: x * x]
print pipe(5, fns)
print pipe2(5, fns)
As far as maps and list comprehension go, that isn't clear-cut but people incline towards comprehensions.
<nitpick>
Twisted(reactors) and threads implement two different things which serve different purposes, and unittest2 is a lib while nose is a test runner.
</nitpick>
That said, there is always going to be many ways to do something, but it helps if the core tries to stick with a uniform way to do things.
Because writing your own web framework seems to be a "rite of passage" for Python programmers, or did around the time that article was first written. The same thing is happening with node.js right now.
If the paradox of choice is making the options an uncomfortable situation for you, just write down the top 6 and roll a die. Problem fucking solved. Or act like a grown up and evaluate your options.
A ZeroMQ-based framework might be interesting -- Brubeck (http://brubeck.io/) is an example of this.
Nginx (https://github.com/FRiCKLE/ngx_zeromq) is starting to get support for ZeroMQ, and Mongrel2 (http://mongrel2.org/) was designed around it.