Factsheet
» pip install pip
Videos
» pip install pip-manager
The biggest disadvantage I see with using pip to install Python modules on your system, either as system modules or as user modules, is that your distribution’s package management system won’t know about them. This means that they won’t be used for any other package which needs them, and which you may want to install in the future (or which might start using one of those modules following an upgrade); you’ll then end up with both pip- and distribution-managed versions of the modules, which can cause issues (I ran into yet another instance of this recently). So your question ends up being an all-or-nothing proposition: if you only use pip for Python modules, you can no longer use your distribution’s package manager for anything which wants to use a Python module...
The general advice given in the page you linked to is very good: try to use your distribution’s packages as far as possible, only use pip for modules which aren’t packaged, and when you do, do so in your user setup and not system-wide. Use virtual environments as far as possible, in particular for module development. Especially on Arch, you shouldn’t run into issues caused by older modules; even on distributions where that can be a problem, virtual environments deal with it quite readily.
It’s always worth considering that a distribution’s library and module packages are packaged primarily for the use of other packages in the distribution; having them around is a nice side-effect for development using those libraries and modules, but that’s not the primary use-case.
TL;DR
- use
pip(+ virtualenv) for stuff (libs, frameworks, maybe dev tools) your projects (that you develop) use - use the package manager for applications you use (as an end-user)
Development dependencies
If you're developing software in Python, you'll want to use pip for all of the project's dependencies, be they runtime dependencies, build-time dependencies or stuff needed for automated testing and automated compliance checks (linter, style checker, static type checker ...)
There are several reasons for this:
- This allows you to use virtualenv (either directly or through virtualenvwrapper or pipenv or other means) to separate dependencies of different projects from each other and to isolate the python applications you use "in production" (as a user) from any exotic shenanigans (or even just incompatibilities) that may go on in development.
- This allows you to track all of a project's dependencies in a
requirements.txt(if your project is an application) orsetup.py(if your project is a library or framework) file. This can be checked into revision control (e.g. Git) together with the source code, so that you always know which version of your code relied on what versions of your dependencies. - The above enables other developers to collaborate on your project even if they don't use the same Linux distribution or not even the same operating system (if the used dependencies are also available on Mac and Windows or whatever they happen to use, that is)
- You don't want automatic updates of your operating system's package manager to break your code. You should update your dependencies, but you should do so consciously and at times you choose, so that you can be ready to fix your code or roll back the update. (Which is easy if you track the complete dependency declaration in your revision control system, together with your code.)
If you feel you need to separate direct and indirect dependencies (or distinguish between acceptable version range for a dependency and actual version used, cf. "version pinning") look into pip-tools and/or pipenv. This will also allow you to distinguish between build and test dependencies. (The distinction between build and runtime dependencies can probably be encoded in setup.py)
Applications you use
For stuff that you use as normal application and that just happens to be written in Python, prefer your operating system's package manager. It'll make sure it stays reasonably up-to-date and compatible to other stuff installed by the package manager. Most Linux distributions will also assert that they don't distribute any malware.
If something you need isn't available in your distribution's default package repo, you can check out additional package repos (e.g. launchpad of deb-based distros) or use pip anyway. If the latter, use --user to install into your user's home instead of system-wide, so that you're less likely to break your Python installation. (For stuff you only need temporarily or seldom, you may even use a virtualenv.)
I’ve been seeing pip/pip install and pypi everywhere but still unsure of what exactly it is. Anyone willing to explain in simple terms?
Types of Packages
Egg vs Wheel vs Neither. What's meant by neither is that a python package can be installed from its "source" without being packaged as an egg or wheel.
Packaging Utilities
There are several libraries which provide utilities for packaging python applications, including distutils and setuptools. There is already an excellent post on this.
easy_install
Part of setuptools, allows building and installing python packages. Often discouraged in favor of Pip. Designed to make installation of packages easy, doing the chore of downloading and moving them to the correct place for you (hence the name).
Pip
A package manager for python packages, and a replacement for easy_install! See here for some reasons why people prefer it over easy_install. Can do neat things like install a package directly from a git repository or compile C extensions on the target machine. The latter is debatable as to whether or not it's desirable, but nonetheless it's a nice feature to have if you want it.
PyPI
The python package index, where easy_install and Pip search for available packages, by default. Basically a giant online repository of modules that are accepted by the community.
virtualenv
A way of hacking your environment variables to "isolate" an installation of python and it's related modules. Prefers Pip, because Ian Bicking wrote them both. Basically, you use pip to install virtualenv system wide, which then allows you to create python virtual environments, each with their own copy of python, pip, and assorted modules. This lets you have multiple versions of python or install a module just for testing, without mucking up your system-wide python install.
virtualenvwrapper
A really handy shell script that makes creating and tearing down virtual environments easier.
site-packages
One of the supported locations for installing python modules into. Lives someplace like /usr/lib/pythonX.X/site-packages. There are other supported locations, like dist-packages or user specific locations.
What does all this mean for you?
I'd recommend you don't pay any attention to easy_install and just use pip. Please also always use virtualenv. Usually, the only python modules you should install system-wide on your workstation are pip and virtualenv. I've completely ignored eggs and wheels, but if you plan to distribute packages professionally or host them on PyPI, you probably want to investigate those. Also, if you are creating python packages, you will need to learn to write a setup script, with setuptools. My recommendation is to never use distutils.
Some more Reading
A page on python.org about packaging which covers a lot of these topics
Python packaging is a nightmare
A great post that goes against the most common recommendations, including mine!
There's some mixing in the options you are listing:
- virtualenv - is used to create isolated environments
site-packages - standard location where python packages / libs reside
pypi - is a repository
easy_install - is found on the setuptools package
- pip - was written to improve easy_install.
about python eggs