* the site module, which is imported by default and is what is responsible for setting up the default sys.path. You can skip 'import site' by running python with the -S switch. the site module is written in python, so you can scan through it and understand how python starts up and inits.
* PYTHONSTARTUP env variable, which points to a python file that is run (like a bashrc, or AUTOEXEC.BAT, if you prefer) on interactive prompt startup. I use this to import custom paths and modules that I want to access from the REPL, such as Google App Engine
* I use pip with local repositories. clone the repos of the libs you need, and then pip install in the virtualenv from that local clone:
this can keep your versions in sync across all projects and virtualenvs and it means no re-downloading and you can setup and update projects while offline.
* don't store the actual project inside the virtualenv. the virtualenv provides the execution context (setup and torn down using the virtualenvwrapper helper scripts). a common practice is to place all your virtualenvs into a directory like ~/.virtualenvs. you should never have to cd into this dir, access it using the wrappers and pip. (edit: also agree with comment below that you shouldn't be sudo'ing).
* just a quick add, I think it is definitely worth learning how to install python from source.
It might be matter of taste but recommendations given starting from "Understanding the packages" and to "Install packages that need compiling" are almost harmful.
My preference:
* you should not care what is your `sys.path` looks like. You need it for debugging if something goes horribly wrong. A tutorial might mention it but things like `sys.path.insert(0,..)` should be avoided or accompanied with a big disclaimer (don't use nuclear weapons if you care about the future)
* the same goes for `PYTHONPATH`. It is a hack that rarely
needed
* don't use `sudo pip`. System packages should be managed by a system packager. Use `pip --user` or create a `virtualenv`
* `pip` can handle tarballs there is no need for `python setup.py install` in this case.
My indented audience was not the pure beginners. I was targeting for developers coming to python from other platforms. Over past 2 years, if I have learned one thing while training interns, trainees and experience devs in Python/Django, it is that packaging confuses people a lot. apt-get vs easy_install vs pip. That is why I choose to spend most time on package management and virtualenv etc. I think, I should have included stronger indications of this.
In general, your feedback is very good and important. Idiomatic Python specifically is an important resource and should have been included.
I will incorporate your feedback as soon as I can squeeze out some time.
This happens to me all the time. My most favorite is when I have to type the word "important", my mind spells "I M P O R T A N T" but my fingers instinctively stop at "import".
Yes, system packages should be managed by a system packager. Unfortunately, for Debian and/or Ubuntu, those packages are often a full version number behind the PyPi version (I'm guessing it's the same for RHL/Fedora). Be very careful using a system packager like apt-get or yum to install a package that is under active development. In that case, it's often better to install from source or use pip/easy_install. But yes, if it's not a system package, don't use sudo for security reasons (although PyPi is pretty safe).
Agree with your points. Would add emphasis on ipython and heavy use of interpreter, pylint instead of pyflakes, "learn and prefer builtin modules" rule, and Python in a Nutshell on the desk.
> System packages should be managed by a system packager.
You shouldn't be using the system Python. Should compile from source and put in /opt/<name of company>/. And that shouldn't be owned by some random user. So, you should be using sudo.
Building your own Python and using system package manager do not exclude each other. On the contrary, I'd even recommend installing your own built software with your systems package manager
I'd love to have one of these for Ruby. Every time I want to try out something written in Ruby I run head-first in to the packaging problem - Debian and Ubuntu don't appear to like shipping a working gem (presumably because it conflicts with how apt likes to do things) and the documentation on how to resolve the resulting inscrutable error messages isn't particularly easy to find. The Mac is a bit better, but I still run in to problems far too often.
I'm pretty sure a "Ruby Ecosystem, An Introduction" guide is exactly what I need.
It seemed to cover everything that I wanted, coming from a familiar Python background of pip/virtualenv/etc. For reference here's the associated HN commentary:
Up-vote, because it's a great idea for any language/tool out there. I happen to know ruby, but taking the first steps in anything is difficult, and usually involves looking up a lot of the things you already know from a more familiar place (setting up, using libs, etc). I'd really love to see a community resource like this - perhaps I'll start working on one :)
The argument (for installing 2 as a default) is that a novice to the ecosystem will be disproportionately harmed by the inevitable Python3 experience:
Novice: "I want to do X"
Internet advice: "Use package Y"
Novice: "Okay <install, install>, wtf nothing is working"
<long frustrating debugging session>
Novice: "Oh, wow, this doesn't support Python3 yet. Now I have to ignore all the internet advice and forge my own path, OR port all my code back to 2.7! This language sucks!"
The alternative seems preferable:
Stuff works, but occasionally you don't get a whiz-bang feature (you "only" get 2.7's feature set, poor you). Five years pass, you learn the language. Now you've gotta learn a bunch of new habits, which sucks, but it's not as bad because now you're pretty good at Python, so the easy parts are easy.
I think they should be asking "How do I do X" before choosing a version to install. The answer might lead them towards IronPython or Jython, not a version of CPython from python.org. Some questions might even lead people straight to PyPy.
More and more questions have problems that can be solved by Python 3, but it's probably a pain in the ass if you start with 3 then work out how to solve your problem.
tl;dr "If 3 does everything you need, great. However, there's a good bit of things that still don't work with 3, in which case 2 is the safer choice. Here's a pretty long list of reasons why 2 is probably better for you. ..."
I agree that as of late November 2011, Python 2.7 is the safer choice for a novice.
But I bet within a year, that's no longer true. (This is dependent on package migration, but there's been a lot of progress lately, and the chances of a novice needing a sophisticated package day one is slim anyway.)
There are a few items that Python 3 fixes that will make this a no-brainer when the vast majority of major packages are ported to 3. Specifically, floating point results of integer division, and print as a function are two that come to mind.
I can't tell you how many times I cursed at the same bug as a beginner (back before division was importable from __future__). 1/2 + 1/2 = 0. Uggh!
And the beginner may as well get in the habit day 1 of using parentheses in statements like print("Hello World").
I still thing that Python 2.7.2 is the only way to go for development unless you already know Python and its ecosystem well enough to know that 3.x will work for you.
That said, I try to use any backported 3.0 features such as .format for string formatting. And for beginners, I heartily recommend using the 'six' module. That way when you need to move to 3.0, porting your code or your skills, will be easy-peasy.
I think that
$ sudo apt-get install python-pip
is a bad idea.
You should not mix multiple packaging system on your operating system.
And more you can dammage it pip provide more recent package than your distro. And if you upgrade a lib that have an incompatibility with a part of the system, you can corrupt it. I have no example to give but I am sure you can find it... Ubuntu now have many tools written in python.
You should use pip inside a virtualenv only.
And, fortunatelly when you create a virtualenv, pip is installed in it, and you don't need to use the --distribute to have it.
What I tell people is that Python (as installed by apt-get, yum) is not on your system for you to develop with. It is there because some of the system tools are written in Python, including at least part of the apt packaging system. It's also there for System admins to use for writing Python scripts instead of bash scripts.
But for app development, get your own Python, manage it yourself and install 'distribute' so that you have both easy_install and pip to work with. I've taken that to extreme by making a portable Python distro that comes in a tarball and runs on any Linux distro, but even if you only untar the source and run ./configure --prefix=/home/python;make
That will build a default Python with support for any shared libraries for which you have a development version installed ( -dev version on debian/ubuntu, -devel version on redhat/suse)
sudo make install will install it, assuming that you have write permissions on the target prefix that you specified. You can even hide it in your home directory with --prefix=~/tools/python272
Definitely; IPython is great at making Python more self-documenting, with completion, the ? help function, and the ability to embed an interpreter anywhere in your program.
ipython has known issues running certain code. It is not guaranteed to function like a normal interpreter. If you insist on a fancy interpreter, use bpython.
References last far longer than they should, because the interpreter doesn't always let go of references to returned results.
There used to be a problem with certain statements being executed as expressions and printed, but I'm told that's been fixed.
The encoding is always Latin-1. Always. Hope you don't use Unicode literals.
It's not Python. It's Python plus other things. That's always a Bad Thing because it precludes taking results from the interpreter and using them in plain Python contexts. web2py also has this problem.
Though the article claims to be targeted at users running on linux, most of the the info is still quite useful regardless of the platform - just figure out how to install python and and pip and the rest is pretty platform agnostic.
I mostly dabble with python. I learned a little reading this, and it raised some questions for me. Is there a reason --distribute is not the default behavior of virtualenv? Is there a plan to incorporate the stuff virtualenvwrapper does into virtualenv (virtualenvwrapper is a pretty cumbersome name, if for no other reason)?
As per the Pragmatic Programmer, I thought I would learn Python this year. It's been a tremendously frustrating experience getting a workable stack installed.
I wish the famous "One -- and preferably only one -- obvious way to do it" Python design philosophy extended to actually installing everything :(
Just curious - which OS/distribution are you on? Ubuntu and Gentoo (and virtualenv+pip if necessary) both make the Python stack a total no-brainer for me.
If you're on Ubuntu LTS you should install PIP from PyPI (easy_install pip), since the system package management version is outdated and it doesn't have the (very useful, since PyPI likes to go down) --use-mirrors install option. That would be my only recommendation.
This is very good. I wish a resource like this was around when I was first learning Python. The difficulty of getting things to work around the language has always been a pretty stark contrast to the ease of the language itself.
Looks great. I've started turning my attention to python recently so skimming through this I can already see lots of stuff that'll be very useful. So much that perhaps a linked TOC at the top of the page could be an idea?
Thank you, thank you, thank you. I've been looking for something like this for ages. While Python the language is fantastic, I have found getting into the whole environment quite tricky so your guide is fantastic.
Great article. Is it just me or the prepend/append examples are swapped? ie. If you want to append TO your PYTHONPATH you should do PYTHONPATH=$PYTHONPATH:/some/new/path
It depends on what do you mean by "append", I guess. Append the search path (e.g. "try this one first"), then /path:$PYTHONPATH is correct. If you mean append the variable then your example is correct.
NOTE: there's brokenness lurking in urllib and email modules. The python library modules are more often than not modeled on perfect reality rather than a pragmatic one (e.g., complete violations of RFCs). BTW, if one reads re.py in the dist, you'll notice that it hasn't been touched by Fredrik Lundh since 2001!
I should say I still love python. Its the most fun I've had programming next to Scheme, and well, NodeJS is kind of fun too (in an algol way).
Updating the source package is easier for one. Ubuntu's packages are often out of date, especially if you are running an LTS. It also integrates nicely with virtualenv which makes managing dependencies easier, via pip freeze.
System package's availability vary a lot with the distribution you're using. For instance I use Debian and my web hosting provider is based on CentOS and share my REQUIREMENTS.txt file.I can't really depend on the packages being available on every distro's repository, let alone the same version.
Also, I wouldn't like to install some packages system-wide (a friend's personal project from a github repository) so I use pip and virtualenv in a similar manner as the article recommends to maintain a per project dependency library.
System packages are going to miss some package that exists in pypi. Using pip is also good if your company maintains their own internal toolsets, and want to be able to easily reuse those across multiple applications.
if you have multiple projects on a system you may have different dependency requirements. module with version 1 for one project and the same module with version 2 for the other project.
In the section of "The Development Environment" you should mention the buildout (http://www.buildout.org/). It is a very simple way to reproduce an enviroment.
* the site module, which is imported by default and is what is responsible for setting up the default sys.path. You can skip 'import site' by running python with the -S switch. the site module is written in python, so you can scan through it and understand how python starts up and inits.
* PYTHONSTARTUP env variable, which points to a python file that is run (like a bashrc, or AUTOEXEC.BAT, if you prefer) on interactive prompt startup. I use this to import custom paths and modules that I want to access from the REPL, such as Google App Engine
* I use pip with local repositories. clone the repos of the libs you need, and then pip install in the virtualenv from that local clone:
(note the triple slash). Or straight from GH: this can keep your versions in sync across all projects and virtualenvs and it means no re-downloading and you can setup and update projects while offline.* don't store the actual project inside the virtualenv. the virtualenv provides the execution context (setup and torn down using the virtualenvwrapper helper scripts). a common practice is to place all your virtualenvs into a directory like ~/.virtualenvs. you should never have to cd into this dir, access it using the wrappers and pip. (edit: also agree with comment below that you shouldn't be sudo'ing).
* just a quick add, I think it is definitely worth learning how to install python from source.