> Even something as simple as installing a library is a conceptual leap for these people (why wouldn't the software just come with everything needed to work?).
> Have you ever tried explaining the various python package and environment management options to someone with a background in Excel/SQL?
I don't understand the difficulty I've often seen voiced against this. Why would a newbie or someone who just wants to get analytical work done need anything beyond installing Python and doing `pip install library`? It's certainly orders of magnitude easier and faster than, say, using a C library. The only trouble I can see a newbie running into is if they want to install a library which doesn't have precompiled wheels and they need some dependencies to build it, but that's rarely an issue for popular packages.
Pip install needs root on my ubuntu install, my lab's and university's old redhat servers and my windows for linux install. I've had to install anaconda python to get any real work done on all three systems. Anaconda works fine for me but I've not even had to think about anything to install packages in R.
Ubuntu doesn't ship with pip or virtualenv. In fact it ships with a version of Python where the built-in equivalent to virtualenv, pyvenv, is explicitly disabled.
So you have to install extra Python packages, as root. You have to have that Python experience that guides you to install as few of them as you can, just enough so you can get started with a virtualenv, so you don't end up relying on your system Python environment.
And this is really hard to explain to people who aren't deeply familiar with Python. "Never use sudo to install Python packages! Oh, you got errors. We obviously meant use sudo for two particular packages and never again after that."
In the terrible case where you don't have root, you have to ignore Ubuntu's version of Python and compile it yourself from scratch. Hope the right development libraries are installed!
Maybe I'm wrong and there's a method I've overlooked. If there is: please show me how to install a Python package on a fresh installation of Ubuntu 16.04, without ever using sudo, and I will happily spread the good news.
That sounds like a major problem with Ubuntu, rather than with Python or pip.
On Windows, meanwhile, the standard Python installer gets all this set up properly in like three clicks. Better yet, because it installs per-user by default, "pip install" just works. And if you still choose to install it globally, it will fail, but it will tell you exactly what you need to do to make it work:
Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: ...
Consider using the `--user` option or check the permissions.
One can't help but wonder how we ended up in a situation where the most popular Linux distro somehow does Python worse than Windows.
Don't despair, in the Anaconda installed with visual studio (now a default) you can't update or install packages without being admin! And if you install Anaconda again it merges the start menu entries and you can't tell which is which...
Eh, that has always been the case for windows vs linux, that you don't have to compile anything yourself because there is always an installer that will deploy precompiled binaries for whatever you want to install (except for when there isn't, because nobody has compiled it for windows, at which point you're in deeper shit) (or except when something installs itself but doesn't update your envars, so you have to do it yourself, which kind of defeats the purpose of the whole "installer" thing).
Iiish. For small projects or when you want to get development versions etc that are not in a distro's repos it's pretty common to have to do a make-configure.
Then again, with Python in particular, I have often had errors either with pip-install, or after "successful" installation, for various reasons.
In this case, we were talking about Python itself. I don't see any particular reason why most people should need to build it themselves, whether on Windows or on Linux. Packages are another matter, but here the issue is the way Python itself is packaged on Ubuntu.
Not on a personal computer, no, but the vast majority of managed systems won't let you install anything outside of your home directory. Of course you could install using `pip install --user` but you will inevitably run into problems when something you install locally needs an updated version of something installed on the system.
Makes it fun when running on a VM in the cloud which only has a root user. Docker becomes almost essential to preventing errant Python scripts fudging up the system.
While you're right that it's bad advice, it also highlights the problem with pip that these less experienced people have. The ideal way to deal with Python packages is virtualenvs, but setting up a virtualenv, and then activating it every time you want to use it (or setting up tools to do it for you) is an incredibly huge headache for less experienced people to deal with. R doesn't require that whatsoever.
Neither language requires an isolated dev environment, but it can help with avoiding headaches. As python has things like virtualenv and buildout, fortunately R has 'packrat' available, which provides a similar isolated/reproducible dev environment solution.
You can certainly update multiple packages at once using pip. Just use a requirements.txt file, which you should be doing anyway if you're using multiple packages (or just want to be able to reproduce your environment).
>> Why would a newbie or someone who just wants to get analytical work done need anything beyond installing Python and doing `pip install library`? It's certainly orders of magnitude easier and faster than, say, using a C library.
Except when it isn't. For instance, because some wheel fails to build because you're lacking the VC++ redistributable (or it's not where pip thinks it should be):
C:\Users\YeGoblynQueenne\Documents\Python> pip install -U spacy
Collecting spacy
Downloading spacy-1.2.0.tar.gz (2.5MB)
100% |################################| 2.5MB 316kB/s
Collecting numpy>=1.7 (from spacy)
Downloading numpy-1.11.2-cp27-none-win_amd64.whl (7.4MB)
100% |################################| 7.4MB 143kB/s
Collecting murmurhash<0.27,>=0.26 (from spacy)
Downloading murmurhash-0.26.4-cp27-none-win_amd64.whl
Collecting cymem<1.32,>=1.30 (from spacy)
Downloading cymem-1.31.2-cp27-none-win_amd64.whl
Collecting preshed<0.47.0,>=0.46.0 (from spacy)
Downloading preshed-0.46.4-cp27-none-win_amd64.whl (55kB)
100% |################################| 61kB 777kB/s
Collecting thinc<5.1.0,>=5.0.0 (from spacy)
Downloading thinc-5.0.8-cp27-none-win_amd64.whl (361kB)
100% |################################| 368kB 747kB/s
Collecting plac (from spacy)
Downloading plac-0.9.6-py2.py3-none-any.whl
Requirement already up-to-date: six in c:\program files\anaconda2\lib\site-packages (from spacy)
Requirement already up-to-date: cloudpickle in c:\program files\anaconda2\lib\site-packages (from spacy)
Collecting pathlib (from spacy)
Downloading pathlib-1.0.1.tar.gz (49kB)
100% |################################| 51kB 800kB/s
Collecting sputnik<0.10.0,>=0.9.2 (from spacy)
Downloading sputnik-0.9.3-py2.py3-none-any.whl
Collecting ujson>=1.35 (from spacy)
Downloading ujson-1.35.tar.gz (192kB)
100% |################################| 194kB 639kB/s
Collecting semver (from sputnik<0.10.0,>=0.9.2->spacy)
Downloading semver-2.7.2.tar.gz
Building wheels for collected packages: spacy, pathlib, ujson, semver
Running setup.py bdist_wheel for spacy ... error
Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\spacy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'ex
ec'))" bdist_wheel -d c:\users\yegobl~1\appdata\local\temp\tmpypkonqpip-wheel- --python-tag cp27:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-2.7
creating build\lib.win-amd64-2.7\spacy
copying spacy\about.py -> build\lib.win-amd64-2.7\spacy
[217 lines truncated for brevity]
copying spacy\tests\sun.tokens -> build\lib.win-amd64-2.7\spacy\tests
running build_ext
building 'spacy.parts_of_speech' extension
error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
----------------------------------------
Failed building wheel for spacy
Running setup.py clean for spacy
Running setup.py bdist_wheel for pathlib ... done
Stored in directory: C:\Users\YeGoblynQueenne\AppData\Local\pip\Cache\wheels\2a\23\a5\d8803db5d631e9f391fe6defe982a238bf5483062eeb34e841
Running setup.py bdist_wheel for ujson ... error
Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'ex
ec'))" bdist_wheel -d c:\users\yegobl~1\appdata\local\temp\tmp8wtgikpip-wheel- --python-tag cp27:
running bdist_wheel
running build
running build_ext
building 'ujson' extension
error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
----------------------------------------
Failed building wheel for ujson
Running setup.py clean for ujson
Running setup.py bdist_wheel for semver ... done
Stored in directory: C:\Users\YeGoblynQueenne\AppData\Local\pip\Cache\wheels\d6\df\b6\0b318a7402342c6edca8a05ffbe8342fbe05e7d730a64db6e6
Successfully built pathlib semver
Failed to build spacy ujson
Installing collected packages: numpy, murmurhash, cymem, preshed, thinc, plac, pathlib, semver, sputnik, ujson, spacy
Found existing installation: numpy 1.11.0
Uninstalling numpy-1.11.0:
Successfully uninstalled numpy-1.11.0
Running setup.py install for ujson ... error
Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, '
exec'))" install --record c:\users\yegobl~1\appdata\local\temp\pip-ibtvwu-record\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'ujson' extension
error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
----------------------------------------
Command ""c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --recor
d c:\users\yegobl~1\appdata\local\temp\pip-ibtvwu-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\yegobl~1\appdata\local\temp\pip-build-7o0roa\ujson\
Now that's newbie scary.
Note that this is just one case where I was trying to install one particular package. I got a couple more examples like this in my installation diary, notably one when I tried to install matplotlib, this time on Windows Subystem for Linux, a.k.a. Ubuntu, and hit a conda bug that meant I had to use an older version of QT until upstream fixed it and other fun times like that.
I don't understand the difficulty I've often seen voiced against this. Why would a newbie or someone who just wants to get analytical work done need anything beyond installing Python and doing `pip install library`? It's certainly orders of magnitude easier and faster than, say, using a C library. The only trouble I can see a newbie running into is if they want to install a library which doesn't have precompiled wheels and they need some dependencies to build it, but that's rarely an issue for popular packages.