You missed my point. Ubuntu packages pip via apt, not via pypi. You can swap from an installation provided by apt to one provided by pypi using your method, but it means you're off the supported version and any guarantees of support are lost. The quick fix here is to not update pip to the latest, but the latest supported by python3.5 and carry on with your migration as scheduled.
I realise that this is a common practice, and I've been caught out myself in the past with CentOS7+Python3.4, but be aware that it comes with its own set of tradeoffs. Sadly, the packaged version of pip is usually quite old and pip is loud about outdated versions, so the solution is either to upgrade the OS or leverage something that is not as tied to the OS.
If you want to explore the latter, I'd suggest python:3.5 containers if you're building pure python code. Alternatively, I'd suggest manylinux containers if you have C extensions and run on a glibc distro like Debian/RHEL. Both will have the last supported version of pip for your python version installed and neither will randomly break on you.
The final lesson I've learned is to be aware that pip can install the latest version supported by your python version as long as it can access the metadata for the package in the index. That's available on pypi, but not all self-hosted indexes have it, notably nexus.
So I wondered if there was an issue here where pip 8.1.1 is just so old that it doesn't know how to respect that constraint? As an experiment, I started with a new venv and upgraded just to the latest pip 20:
$ test/bin/pip install pip==20.3.4
Collecting pip==20.3.4
Downloading https://files.pythonhosted.org/packages/27/79/8a850fe3496446ff0d584327ae44e7500daf6764ca1a382d2d02789accf7/pip-20.3.4-py2.py3-none-any.whl (1.5MB)
100% |################################| 1.5MB 695kB/s
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Successfully uninstalled pip-8.1.1
Successfully installed pip-20.3.4
You are using pip version 20.3.4, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Now interestingly I don't get any prompts after that, and it doesn't try to upgrade past 20.3, so I think that might have been the last gasp of pip 8.1.1:
$ test/bin/pip install -U pip
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Requirement already satisfied: pip in ./test/lib/python3.5/site-packages (20.3.4)
Collecting pip
Using cached pip-20.3.4-py2.py3-none-any.whl (1.5 MB)
Using cached pip-20.3.3-py2.py3-none-any.whl (1.5 MB)
So yeah, basically the fix is just to hard-code the version you know works. But I think this demonstrates an issue with the PyPA being out of touch with real-world use-cases. No one is upgrading pip version by version— 99% of pip installations are either going to be get-pip.py or upgrading a venv directly from a distro-shipped version. IMO those use-cases should be carefully tested on each release and checked for the versions in popular distros, even at the tail end of the support windows.
The other side of this is Debian/Ubuntu, of course, and with sufficient coordination (knowing the a pip was coming which their "supported" pip would auto-upgrade to and break itself) they could have patched their `python3-pip` package to only upgrade as far as 20.3.4.
I realise that this is a common practice, and I've been caught out myself in the past with CentOS7+Python3.4, but be aware that it comes with its own set of tradeoffs. Sadly, the packaged version of pip is usually quite old and pip is loud about outdated versions, so the solution is either to upgrade the OS or leverage something that is not as tied to the OS.
If you want to explore the latter, I'd suggest python:3.5 containers if you're building pure python code. Alternatively, I'd suggest manylinux containers if you have C extensions and run on a glibc distro like Debian/RHEL. Both will have the last supported version of pip for your python version installed and neither will randomly break on you.
The final lesson I've learned is to be aware that pip can install the latest version supported by your python version as long as it can access the metadata for the package in the index. That's available on pypi, but not all self-hosted indexes have it, notably nexus.