Using packages

Guido.van.Rossum@cwi.nl
Thu, 14 Apr 1994 17:13:15 +0200

I propose to place the following module, called "addpack.py", in the
1.0.2 distribution. It addesses the problem of using "packages", or
groups of related modules (usually all modules of a package are
provided by a single author). It doesn't solve all problems, but it
does make it easier to distribute and use packages.

In particular it doesn't address the problem of multiple versions.
This is left for future discussion (but see below).

It also doesn't prevent name clashes between modules in packages that
are developed indepently. Package authors are encouraged to use some
convention to make their module names unique, e.g. prefix the modules
with a prefix that is an abbreviation of the package name.

A package's author gives the package its name and distributes it as a
directory full of modules.

The installer of a package installs it by its package name in a
subdirectory of the standard python library -- or, alternatively, in a
system-wide directory (e.g. /usr/local/lib/python_packages) which is
added at build time to the default module search path of the python
interpreter.

The user of a package places the sequence

from addpack import addpack
addpack(<packagename>)

before importing any modules from the package. This adds components
to sys.path; entries already in sys.path are not added again (but the
module can be fooled by different names for the same directory). Only
names of directories that exist at the time of calling are added.

A suggestion for limited version management is to install different
versions of a package as <packagename>_<version> and to make a
symbolic link <packagename> pointing to the preferred version. Users
can either say addpack(<packagename>_<version>) to use a specific
version, or addpack(<packagename>) to use the preferred version.

A feature of the addpack module is that a subdirectory <packagename>
is searched in all components of sys.path; thus a user may have a
private directory <packagename> containing a few hacked modules in
his/her private module directory, and add the private module directory
to PYTHONPATH (which gets prefixed to the built-in default module
search path). Alas, there is no easy way to create a module that is
an extension of a module of the *same* name further down in the module
search path (except for Steve Majewski :).

You may find that this module overloads the "module search path" with
the "package search path" and that these should ideally be different
beasts. While I agree in theory, I believe that in practive there
will be few problems or conflicts, as packages will be directories and
modules will be plain files whose name ends in ".py". Just don't give
a package a name ending in ".py"... In any case an additional
"package search path can be added to individual addpack calls.

============================== addpack.py ==============================
# This module provides standard support for "packages".
#
# The idea is that large groups of related modules can be placed in
# their own subdirectory, which can be added to the Python search path
# in a relatively easy way.
#
# The current version takes a package name and searches the Python
# search path for a directory by that name, and if found adds it to
# the module search path (sys.path). It maintains a list of packages
# that have already been added so adding the same package many times
# is OK.
#
# It is intended to be used in a fairly stylized manner: each module
# that wants to use a particular package, say 'Foo', is supposed to
# contain the following code:
#
# from addpack import addpack
# addpack('Foo')
# <import modules from package Foo>
#
# Additional arguments, when present, provide additional places where
# to look for the package before trying sys.path (these may be either
# strings or lists/tuples of strings). Also, if the package name is a
# full pathname, first the last component is tried in the usual way,
# then the full pathname is tried last. If the package name is a
# *relative* pathname (UNIX: contains a slash but doesn't start with
# one), then nothing special is done. The packages "/foo/bar/bletch"
# and "bletch" are considered the same, but unrelated to "bar/bletch".
#
# If the algorithm finds more than one suitable subdirectory, all are
# added to the search path -- this makes it possible to override part
# of a package. The same path will not be added more than once.
#
# If no directory is found, ImportError is raised.

_packs = {} # {pack: [pathname, ...], ...}

def addpack(pack, *locations):
import os
if os.path.isabs(pack):
base = os.path.basename(pack)
else:
base = pack
if _packs.has_key(base):
return
import sys
path = []
for loc in _flatten(locations) + sys.path:
fn = os.path.join(loc, base)
if fn not in path and os.path.isdir(fn):
path.append(fn)
if pack != base and pack not in path and os.path.isdir(pack):
path.append(pack)
if not path: raise ImportError, 'package ' + pack + ' not found'
_packs[base] = path
for fn in path:
if fn not in sys.path:
sys.path.append(fn)

def _flatten(locations):
locs = []
for loc in locations:
if type(loc) == type(''):
locs.append(loc)
else:
locs = locs + _flatten(loc)
return locs
========================================================================

--Guido van Rossum, CWI, Amsterdam <Guido.van.Rossum@cwi.nl>
URL: <http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>