2017-12-24 EDIT: These days, just use pipenv. It’s easier and does all the things below better.

So, you’ve written an amazing Python program, it does everything you want and then some. You stop and stare at it in amazement, and realize that it’s just so great, that it would be selfish not to share it with the entire world.

However, there’s one small problem. Your program uses various libraries, which your users will need to install, and you want to make that process as painless as possible, or, if it’s a web app, you want to put it up on a server for everyone to enjoy, but you need to figure out what libraries it uses and install those on the server as well, which is a hassle. What’s a good way to ease this pain?

Suppose that your first program has made you a millionaire, and now you’re running low on cash, so you decide to spend a few hours writing another one. However, your system still has all the old versions of the libraries you used to build the first program, and when you try to upgrade them, it breaks because it hasn’t been maintained in ages. How do you solve that problem?

Luckily, there’s an easy way to solve both problems. We can use virtualenv to create separate, self-contained virtual environments where each environment is contained in a its own directory. Each environment can be activated when you want to work on the program that uses it.

The best way to show how virtualenv works is by example, so let’s go through the process of installing and using it. Assuming you have the average Python installation (probably 2.7, on any of the three main OSes), the following steps should work as they are. Windows might do some things slightly differently, but mostly it should all work the same. In the examples, the lines starting with an “$” is what you will need to type in ($ is my prompt), so ignore the $ and type in the rest of the command.

To install virtualenv (and pip, while we’re at it), you can use setuptools:

$ easy_install pip virtualenv

After that, both packages will be installed globally in your system, so you might want to use sudo if you are on Linux or OS X. We can now continue to creating an environment. I prefer to put the environment in the directory that contains the project I’m working on, so switch to the directory that contains your scripts and run (you shouldn’t need to use sudo):

$ virtualenv env

That will create a directory called env in your current directory and put the environment there. If you look inside, you should see some subdirectories, containing various files. The bin directory, in particular, should contain executables called python, pip, easy_install, etc. These executables differ from their globally-installed namesakes in that the former will be run in the virtual environment you’ve just created, while the latter will run system-wide.

To illustrate this point, we need to install some packages. To do this, we will use pip, because it’s great. We already installed it earlier, so we can use it simply by doing:

$ env/bin/pip install ipython shortuuid

ipython is a very nice Python shell, which you might already be familiar with, and shortuuid is a small library for creating short unique IDs, which we’ll use to demonstrate how virtualenv works.

You might have noticed that we didn’t just run pip, we ran env/bin/pip, which means that the two packages are now installed inside the virtualenv. Sure enough, if you run:

$ env/bin/ipython

you will see that the ipython shell opens up. If you try to import shortuuid, you will see the following:

IPython 0.12.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
In [1]: import shortuuid
In [2]: shortuuid.uuid()
Out[2]: 'ogxPuKH7qvtoXSzNmVECW'

You will get a different ID, but the gist is the same. shortuuid has been installed and is working properly. If you now type exit to exit ipython, run the system-wide python installation (by typing python), and try to import shortuuid, what do you think will happen? Let’s try it out:

Python 2.7.2+ (default, Oct  4 2011, 20:03:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import shortuuid
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: No module named shortuuid
>>>

This is because, as I mentioned before, these packages have only been installed in the virtualenv directory, and we didn’t run the Python interpreter that’s inside the virtualenv, we ran the system-wide one. These packages don’t exist globally in the system, so we can be sure that whatever package we install will be in its own, tidy little directory.

What happens if there’s already a package installed system-wide and we try to install it in the virtualenv, or even if we don’t install it in the virtualenv? Can we still access it? The answer is “no, we cannot”, because virtualenv, by default, restricts all access to system packages. This can be changed when creating the virtualenv, but it’s beyond our scope now. As far as the virtualenv is concerned, the system has no packages other than the ones actually installed by us in the virtualenv. If a library is at version X globally, but we install version Y in the virtualenv, programs in that virtualenv will only be able to see version Y.

So, we have installed our packages and are able to write our program knowing that we won’t be messing our computer up with the various packages, but using env/bin/python all the time is a bit of a hassle. To activate one virtualenv to work with for one terminal session, we can run:

$ source env/bin/activate

This should put the environment’s name somewhere on our prompt, and, from now on, any program we run that’s installed in the virtualenv, will be run from it. For example, if we run python, or pip, or ipython, we will actually be running the versions inside the environment, so they will be able to see (and install or remove, in the case of pip) only the packages inside the environment.

You might be thinking “This is all very nice so far, but you said I could easily distribute my program, and I see none of that!”, and you would be right. Let’s see how we can obtain a list of all the packages that are installed in the current virtualenv. Fortunately, pip makes this trivial:

$ pip freeze
ipython==0.12.1
shortuuid==0.2

You can see that pip gave us a list of all the packages we have installed, isn’t that fantastic? Yes, of course it is. We can get that list and stick it in a text file (I like to call it requirements.txt). We can then send that file to a friend, along with the source code, and all they have to do to ensure that our program will run is create a virtualenv and install the packages we’ve provided, with three simple commands:

$ virtualenv env
$ source env/bin/activate
$ pip install -r requirements.txt

This will create the virtualenv and install all the packages we have specified in the requirements file, at the specific versions we need, no less. You’re happy, your friend is happy, I’m happy, everyone’s happy.

That’s not all pip can do, though. It can remove packages, install them directly from source directories, from remote repositories, just download (but not install) them, and much more, all for one low, low price. To get a better view of what pip (or virtualenv) can do, just look at their respective help sections:

$ pip --help
$ virtualenv --help

They contain much more useful functionality, so feel free to explore around. You can get rid of virtualenvs just by deleting the directory, absolutely nothing else in your system is touched in any way.

As always, if you think I’ve missed something, or have an error, or if you have any sort of feedback, please leave a comment below, and remember, if you want to stay up to date on these tutorials and have them available in DRM-free form in one place, you can get the ebook version.