How Shrinkwrap Works
====================
Shrinkwrap was designed to scratch a particular itch: automating the
installation of software in a self-contained environment.
Nearly all package managers assume they are managing software installation for
the entire system, and don't allow isolated environments. `ZeroInstall `_ is a notable exception which installs
software into a user's home directory and can manage multiple versions of a
single package. However, ZeroInstall is more aimed at providing specific
applications to an end user, rather than providing a shell environment with a
set of packages installed and ready for use.
Python's virtualenv and easy_install/pip tools are a great example of a
userspace packaging solution that creates self-contained, easy-to-use
environments. Since we work in mixed environments where C, C++ and Python
software needs to coexist, shrinkwrap is our attempt to bring all that
software under one roof.
Shrinkwrap achieves this goal by modifying the virtualenv in three ways:
1. It provides an alternate implementation of the ``python setup.py install``
command that makes it easy to insert an arbitrary installer function. This
function typically downloads source tarballs, unpacks them, compiles and
installs to the virtualenv base directory.
2. It adds new optional keyword arguments to ``setup()`` to specify the
installer function, and also to list packages that are build dependencies of
this package. See :ref:`package_dependencies` for details.
3. It creates a new ``$VIRTUAL_ENV/env.d`` directory and patches the
``$VIRTUAL_ENV/bin/activate`` script to source any files found in that
directory. This allows shrinkwrap packages to install scripts that make
changes to the shell environment. Many of the software packages we work with
require environment variables to function properly.
With these changes, it becomes very easy to write small Python packages that
download, compile, and install software into the virtualenv environment.
Installing shrinkwrap packages
``````````````````````````````
Installing a shrinkwrapped package is identical to installing any package with
pip. You can install the package file directly from the filesystem::
pip install curl-7.27.0.tar.gz
or from a URL::
pip install http://mtrr.org/packages/curl/curl-7.27.0.tar.gz
or set the URL of extra package repository which contains shrinkwrapped
packages::
export PIP_EXTRA_INDEX_URL=http://mtrr.org/packages/
pip install curl
If the shrinkwrapped package contains the standard test for the shrinkwrap
module at the top (see :ref:`built_in_installers`), then shrinkwrap will be
automatically installed to your virtualenv.
Note that environment files in the ``$VIRTUAL_ENV/env.d/`` directory are only
sourced when the ``$VIRTUAL_ENV/bin/activate`` script is sourced. You will
need to source that script again to refresh your environment if the
shrinkwrapped package added new environment files.
As true Python packages, shrinkwrapped packages can be used in a
requirements.txt files passed to pip. Just place the extra index URL
argument at the top::
--extra-index-url http://mtrr.org/packages/
curl
fftw
nose
Disabling system software autodetection
'''''''''''''''''''''''''''''''''''''''
Some shrinkwrap packages will auto-detect whether the software is already
installed system-wide. If it is, installation is skipped and the package is
marked as installed since the dependency is satisfied.
To force installation into the virtual environment, this auto-detection can be
disabled with an environment variable:
.. envvar:: SHRINKWRAP_NEVER_SKIP
Set this variable to '1' to forego system package detection and always
install the shrinkwrap package. Unset or change to '0' to restore the
default behavior.
.. _limitations:
Limitations
```````````
We are the first to admit that shrinkwrap is straining the intended purpose of
the Python packaging tools. As a result, shrinkwrap+pip has some
shortcomings compared to more sophisticated package managers:
* Shrinkwrap can only be used with a virtualenv environment. Shrinkwrap tests
for the presence of ``$VIRTUAL_ENV`` and will throw an exception if it is not
found.
* Shrinkwrapped packages have only been tested to work with ``pip`` as the
installer. We do not support using ``easy_install``, although it might work.
* ``pip uninstall`` does not remove files installed by the package because we
do not yet have a way to track changes made by arbitrary installer functions
to the filesystem.
* We do not provide any mechanism to have build options selected at
installation time (such as
`variants in MacPorts `_ or
`USE variables in Gentoo emerge `_).
Repositories of shrinkwrapped packages are easy to make with the
``shrinkwrap`` command line tool, so we encourage you to have different
repositories for different kinds of deployment environments. Then you can
tailor the build options of your source packages for each one.
* Shrinkwrap is currently focused on deploying source packages rather than
precompiled packages. There is no reason you can't unpack a tarball of
compiled code in a shrinkwrap ``installer()`` function, but we do not
provide any mechanism for having packages compiled for different
architectures in the same repository. Again, repositories are easy to
make, so we would suggest one per architecture if this is your use case.
(Selecting platform-specific build options in your ``installer()`` function,
however, is no problem.)