skip to navigation
skip to content

Python Wiki

Python Insider Blog

Python 2 or 3?

Help Fund Python

[Python resources in languages other than English]

Non-English Resources

Add an event to this calendar.

Times are shown in UTC/GMT.

Add an event to this calendar.

PEP:440
Title:Version Identification and Dependency Specification
Version:a71810168398
Last-Modified:2014-01-28 22:37:42 +1000 (Tue, 28 Jan 2014)
Author:Nick Coghlan <ncoghlan at gmail.com>
BDFL-Delegate:Nick Coghlan <ncoghlan@gmail.com>
Discussions-To:Distutils SIG <distutils-sig at python.org>
Status:Draft
Type:Standards Track
Content-Type:text/x-rst
Created:18 Mar 2013
Post-History:30 Mar 2013, 27 May 2013, 20 Jun 2013, 21 Dec 2013, 28 Jan 2014
Replaces:386

Abstract

This PEP describes a scheme for identifying versions of Python software distributions, and declaring dependencies on particular versions.

This document addresses several limitations of the previous attempt at a standardised approach to versioning, as described in PEP 345 and PEP 386.

Note

This PEP was broken out of the metadata 2.0 specification in PEP 426.

Unlike PEP 426, the notes that remain in this document are intended as part of the final specification (except for this one).

Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The following terms are to be interpreted as described in PEP 426:

  • "Distributions"
  • "Releases"
  • "Build tools"
  • "Index servers"
  • "Publication tools"
  • "Installation tools"
  • "Automated tools"
  • "Projects"

Version scheme

Distributions are identified by a public version identifier which supports all defined version comparison operations

Distributions may also define a source label, which is not used by automated tools. Source labels are useful when a project internal versioning scheme requires translation to create a compliant public version identifier.

The version scheme is used both to describe the distribution version provided by a particular distribution archive, as well as to place constraints on the version of dependencies needed in order to build or run the software.

Public version identifiers

Public version identifiers MUST comply with the following scheme:

[N:]N(.N)*[{a|b|c|rc}N][.postN][.devN]

Public version identifiers MUST NOT include leading or trailing whitespace.

Public version identifiers MUST be unique within a given distribution.

Installation tools SHOULD ignore any public versions which do not comply with this scheme. Installation tools MAY warn the user when non-compliant or ambiguous versions are detected.

Public version identifiers are separated into up to five segments:

  • Epoch segment: N:
  • Release segment: N(.N)*
  • Pre-release segment: {a|b|c|rc}N
  • Post-release segment: .postN
  • Development release segment: .devN

Any given release will be a "final release", "pre-release", "post-release" or "developmental release" as defined in the following sections.

All numeric components MUST be non-negative integers.

All numeric components MUST be interpreted and ordered according to their numeric value, not as text strings.

All numeric components MAY be zero. Except as described below for the release segment, a numeric component of zero has no special significance aside from always being the lowest possible value in the version ordering.

Note

Some hard to read version identifiers are permitted by this scheme in order to better accommodate the wide range of versioning practices across existing public and private Python projects.

Accordingly, some of the versioning practices which are technically permitted by the PEP are strongly discouraged for new projects. Where this is the case, the relevant details are noted in the following sections.

Local version identifiers

Local version identifiers MUST comply with the following scheme:

<public version identifier>[-N[.N]+]

Local version identifiers are used to denote fully API compatible patched versions of upstream projects. These are created by application developers and system integrators when upgrading to a new upstream release would be too disruptive to the application or other integrated system (such as a Linux distribution).

Local version identifiers may be used anywhere a public version identifier is expected.

Local version identifiers MUST NOT include leading or trailing whitespace.

Numeric components in the integrator suffix are interpreted in the same way as the numeric components of the release segment.

The additional segment after the hyphen is referred to as the "integrator suffix", and makes it possible to differentiate upstream releases from potentially altered rebuilds by downstream integrators. The inclusion of an integrator suffix does not affect the kind of a release, but indicates that it may not contain the exact same code as the corresponding upstream release.

Public index servers SHOULD NOT allow the use of local version identifiers in uploaded distributions. Local version identifiers are intended as a tool for software integrators rather than publishers.

Distributions using a local version identifier SHOULD provide the python.integrator extension metadata (as defined in PEP 459).

Source labels

Source labels are text strings with minimal defined semantics.

To ensure source labels can be readily incorporated as part of file names and URLs, and to avoid formatting inconsistences in hexadecimal hash representations they MUST be limited to the following set of permitted characters:

  • Lowercase ASCII letters ([a-z])
  • ASCII digits ([0-9])
  • underscores (_)
  • hyphens (-)
  • periods (.)
  • plus signs (+)

Source labels MUST start and end with an ASCII letter or digit.

Source labels MUST be unique within each project and MUST NOT match any defined version for the project.

Final releases

A version identifier that consists solely of a release segment is termed a "final release".

The release segment consists of one or more non-negative integer values, separated by dots:

N[.N]+

Final releases within a project MUST be numbered in a consistently increasing fashion, otherwise automated tools will not be able to upgrade them correctly.

Comparison and ordering of release segments considers the numeric value of each component of the release segment in turn. When comparing release segments with different numbers of components, the shorter segment is padded out with additional zeroes as necessary.

While any number of additional components after the first are permitted under this scheme, the most common variants are to use two components ("major.minor") or three components ("major.minor.micro").

For example:

0.9
0.9.1
0.9.2
...
0.9.10
0.9.11
1.0
1.0.1
1.1
2.0
2.0.1
...

A release series is any set of final release numbers that start with a common prefix. For example, 3.3.1, 3.3.5 and 3.3.9.45 are all part of the 3.3 release series.

Note

X.Y and X.Y.0 are not considered distinct release numbers, as the release segment comparison rules implicit expand the two component form to X.Y.0 when comparing it to any release segment that includes three components.

Date based release segments are also permitted, and are treated differently in some cases when used in version specifiers. Any version identifier where the leading component in the release segment is greater than or equal to 1980 is considered to be a date based release.

An example of a date based release scheme using the year and month of the release:

2012.04
2012.07
2012.10
2013.01
2013.06
...

Pre-releases

Some projects use an "alpha, beta, release candidate" pre-release cycle to support testing by their users prior to a final release.

If used as part of a project's development cycle, these pre-releases are indicated by including a pre-release segment in the version identifier:

X.YaN  # Alpha release
X.YbN  # Beta release
X.YcN  # Candidate release (alternative notation: X.YrcN)
X.Y    # Final release

A version identifier that consists solely of a release segment and a pre-release segment is termed a "pre-release".

The pre-release segment consists of an alphabetical identifier for the pre-release phase, along with a non-negative integer value. Pre-releases for a given release are ordered first by phase (alpha, beta, release candidate) and then by the numerical component within that phase.

Installation tools MAY accept both c and rc releases for a common release segment in order to handle some existing legacy distributions.

Installation tools SHOULD interpret all rc versions as coming after all c versions (that is, rc1 indicates a later version than c2). Installation tools MAY warn the user when such ambiguous versions are detected, or even reject them entirely.

Build tools, publication tools and index servers SHOULD disallow the creation of both c and rc releases for a common release segment.

Post-releases

Some projects use post-releases to address minor errors in a final release that do not affect the distributed software (for example, correcting an error in the release notes).

If used as part of a project's development cycle, these post-releases are indicated by including a post-release segment in the version identifier:

X.Y.postN    # Post-release

A version identifier that includes a post-release segment without a developmental release segment is termed a "post-release".

The post-release segment consists of the string .post, followed by a non-negative integer value. Post-releases are ordered by their numerical component, immediately following the corresponding release, and ahead of any subsequent release.

Note

The use of post-releases to publish maintenance releases containing actual bug fixes is strongly discouraged. In general, it is better to use a longer release number and increment the final component for each maintenance release.

Post-releases are also permitted for pre-releases:

X.YaN.postM  # Post-release of an alpha release
X.YbN.postM  # Post-release of a beta release
X.YcN.postM  # Post-release of a release candidate

Note

Creating post-releases of pre-releases is strongly discouraged, as it makes the version identifier difficult to parse for human readers. In general, it is substantially clearer to simply create a new pre-release by incrementing the numeric component.

Developmental releases

Some projects make regular developmental releases, and system packagers (especially for Linux distributions) may wish to create early releases directly from source control which do not conflict with later project releases.

If used as part of a project's development cycle, these developmental releases are indicated by including a developmental release segment in the version identifier:

X.Y.devN    # Developmental release

A version identifier that includes a developmental release segment is termed a "developmental release".

The developmental release segment consists of the string .dev, followed by a non-negative integer value. Developmental releases are ordered by their numerical component, immediately before the corresponding release (and before any pre-releases with the same release segment), and following any previous release (including any post-releases).

Developmental releases are also permitted for pre-releases and post-releases:

X.YaN.devM      # Developmental release of an alpha release
X.YbN.devM      # Developmental release of a beta release
X.YcN.devM      # Developmental release of a release candidate
X.Y.postN.devM  # Developmental release of a post-release

Note

Creating developmental releases of pre-releases is strongly discouraged, as it makes the version identifier difficult to parse for human readers. In general, it is substantially clearer to simply create additional pre-releases by incrementing the numeric component.

Developmental releases of post-releases are also strongly discouraged, but they may be appropriate for projects which use the post-release notation for full maintenance releases which may include code changes.

Version epochs

If included in a version identifier, the epoch appears before all other components, separated from the release segment by a colon:

E:X.Y  # Version identifier with epoch

If no explicit epoch is given, the implicit epoch is 0.

Most version identifiers will not include an epoch, as an explicit epoch is only needed if a project changes the way it handles version numbering in a way that means the normal version ordering rules will give the wrong answer. For example, if a project is using date based versions like 2014.04 and would like to switch to semantic versions like 1.0, then the new releases would be identified as older than the date based releases when using the normal sorting scheme:

1.0
1.1
2.0
2013.10
2014.04

However, by specifying an explicit epoch, the sort order can be changed appropriately, as all versions from a later epoch are sorted after versions from an earlier epoch:

2013.10
2014.04
1:1.0
1:1.1
1:2.0

Examples of compliant version schemes

The standard version scheme is designed to encompass a wide range of identification practices across public and private Python projects. In practice, a single project attempting to use the full flexibility offered by the scheme would create a situation where human users had difficulty figuring out the relative order of versions, even though the rules above ensure all compliant tools will order them consistently.

The following examples illustrate a small selection of the different approaches projects may choose to identify their releases, while still ensuring that the "latest release" and the "latest stable release" can be easily determined, both by human users and automated tools.

Simple "major.minor" versioning:

0.1
0.2
0.3
1.0
1.1
...

Simple "major.minor.micro" versioning:

1.1.0
1.1.1
1.1.2
1.2.0
...

"major.minor" versioning with alpha, beta and candidate pre-releases:

0.9
1.0a1
1.0a2
1.0b1
1.0c1
1.0
1.1a1
...

"major.minor" versioning with developmental releases, release candidates and post-releases for minor corrections:

0.9
1.0.dev1
1.0.dev2
1.0.dev3
1.0.dev4
1.0rc1
1.0rc2
1.0
1.0.post1
1.1.dev1
...

Date based releases, using an incrementing serial within each year, skipping zero:

2012.1
2012.2
2012.3
...
2012.15
2013.1
2013.2
...

Summary of permitted suffixes and relative ordering

Note

This section is intended primarily for authors of tools that automatically process distribution metadata, rather than developers of Python distributions deciding on a versioning scheme.

The epoch segment of version identifiers MUST be sorted according to the numeric value of the given epoch. If no epoch segment is present, the implicit numeric value is 0.

The release segment of version identifiers MUST be sorted in the same order as Python's tuple sorting when the release segment is parsed as follows:

tuple(map(int, release_segment.split(".")))

All release segments involved in the comparison MUST be converted to a consistent length by padding shorter segments with zeroes as needed.

Within a numeric release (1.0, 2.7.3), the following suffixes are permitted and MUST be ordered as shown:

.devN, aN, bN, cN, rcN, <no suffix>, .postN

Note that rc will always sort after c (regardless of the numeric component) although they are semantically equivalent. Tools MAY reject this case as ambiguous and remain in compliance with the PEP.

Within an alpha (1.0a1), beta (1.0b1), or release candidate (1.0c1, 1.0rc1), the following suffixes are permitted and MUST be ordered as shown:

.devN, <no suffix>, .postN

Within a post-release (1.0.post1), the following suffixes are permitted and MUST be ordered as shown:

.devN, <no suffix>

Note that devN and postN MUST always be preceded by a dot, even when used immediately following a numeric version (e.g. 1.0.dev456, 1.0.post1).

Within a pre-release, post-release or development release segment with a shared prefix, ordering MUST be by the value of the numeric component.

The following example covers many of the possible combinations:

1.0.dev456
1.0a1
1.0a2.dev456
1.0a12.dev456
1.0a12
1.0b1.dev456
1.0b2
1.0b2.post345.dev456
1.0b2.post345
1.0c1.dev456
1.0c1
1.0
1.0.post456.dev34
1.0.post456
1.1.dev1

The integrator suffix of local version identifiers that share a common public version identifier prefix MUST be sorted in the same order as Python's tuple sorting when the integrator suffix is parsed as follows (this is the same definition as is used for the release segment):

tuple(map(int, integrator_suffix.split(".")))

All integrator suffixes involved in the comparison MUST be converted to a consistent length by padding shorter segments with zeroes as needed.

All local version identifiers (even the -0 suffix) are sorted after the corresponding unqualified public version identifier.

Version ordering across different metadata versions

Metadata v1.0 (PEP 241) and metadata v1.1 (PEP 314) do not specify a standard version identification or ordering scheme. This PEP does not mandate any particular approach to handling such versions, but acknowledges that the de facto standard for ordering them is the scheme used by the pkg_resources component of setuptools.

Software that automatically processes distribution metadata SHOULD attempt to normalize non-compliant version identifiers to the standard scheme, and ignore them if normalization fails. As any normalization scheme will be implementation specific, this means that projects using non-compliant version identifiers may not be handled consistently across different tools, even when correctly publishing the earlier metadata versions.

For distributions currently using non-compliant version identifiers, these filtering guidelines mean that it should be enough for the project to simply switch to the use of compliant version identifiers to ensure consistent handling by automated tools.

Distribution users may wish to explicitly remove non-compliant versions from any private package indexes they control.

For metadata v1.2 (PEP 345), the version ordering described in this PEP SHOULD be used in preference to the one defined in PEP 386.

Compatibility with other version schemes

Some projects may choose to use a version scheme which requires translation in order to comply with the public version scheme defined in this PEP. In such cases, the source label can be used to record the project specific version as an arbitrary label, while the translated public version is published in the version field.

This allows automated distribution tools to provide consistently correct ordering of published releases, while still allowing developers to use the internal versioning scheme they prefer for their projects.

Semantic versioning

Semantic versioning [3] is a popular version identification scheme that is more prescriptive than this PEP regarding the significance of different elements of a release number. Even if a project chooses not to abide by the details of semantic versioning, the scheme is worth understanding as it covers many of the issues that can arise when depending on other distributions, and when publishing a distribution that others rely on.

The "Major.Minor.Patch" (described in this PEP as "major.minor.micro") aspects of semantic versioning (clauses 1-9 in the 2.0.0-rc-1 specification) are fully compatible with the version scheme defined in this PEP, and abiding by these aspects is encouraged.

Semantic versions containing a hyphen (pre-releases - clause 10) or a plus sign (builds - clause 11) are not compatible with this PEP and are not permitted in the public version field.

One possible mechanism to translate such semantic versioning based source labels to compatible public versions is to use the .devN suffix to specify the appropriate version order.

DVCS based version labels

Many build tools integrate with distributed version control systems like Git and Mercurial in order to add an identifying hash to the version identifier. As hashes cannot be ordered reliably such versions are not permitted in the public version field.

As with semantic versioning, the public .devN suffix may be used to uniquely identify such releases for publication, while the source label is used to record the original DVCS based version label.

Olson database versioning

The pytz project inherits its versioning scheme from the corresponding Olson timezone database versioning scheme: the year followed by a lowercase character indicating the version of the database within that year.

This can be translated to a compliant public version identifier as <year>.<serial>, where the serial starts at zero or one (for the '<year>a' release) and is incremented with each subsequent database update within the year.

As with other translated version identifiers, the corresponding Olson database version could be recorded in the source label field.

Version specifiers

A version specifier consists of a series of version clauses, separated by commas. For example:

0.9, ~= 0.9, >= 1.0, != 1.3.4.*, < 2.0

The comparison operator (or lack thereof) determines the kind of version clause:

The comma (",") is equivalent to a logical and operator: a candidate version must match all given version clauses in order to match the specifier as a whole.

Whitespace between a conditional operator and the following version identifier is optional, as is the whitespace around the commas.

When multiple candidate versions match a version specifier, the preferred version SHOULD be the latest version as determined by the consistent ordering defined by the standard Version scheme. Whether or not pre-releases are considered as candidate versions SHOULD be handled as described in Handling of pre-releases.

Compatible release

A compatible release clause consists of either a version identifier without any comparison operator or else the compatible release operator ~= and a version identifier. It matches any candidate version that is expected to be compatible with the specified version.

The specified version identifier must be in the standard format described in Version scheme.

Automated tools SHOULD report an error when this operator is used in conjunction with a date based version identifier, as it assumes the use of semantic API versioning.

For a given release identifier V.N, the compatible release clause is approximately equivalent to the pair of comparison clauses:

>= V.N, == V.*

For example, the following groups of version clauses are equivalent:

2.2
~= 2.2
>= 2.2, == 2.*

1.4.5
~= 1.4.5
>= 1.4.5, == 1.4.*

If a pre-release, post-release or developmental release is named in a compatible release clause as V.N.suffix, then the suffix is ignored when determining the required prefix match:

2.2.post3
~= 2.2.post3
>= 2.2.post3, == 2.*

1.4.5a4
~= 1.4.5a4
>= 1.4.5a4, == 1.4.*

The padding rules for release segment comparisons means that the assumed degree of forward compatibility in a compatible release clause can be controlled by appending additional zeroes to the version specifier:

2.2.0
~= 2.2.0
>= 2.2.0, == 2.2.*

1.4.5.0
~= 1.4.5.0
>= 1.4.5.0, == 1.4.5.*

Version matching

A version matching clause includes the version matching operator == and a version identifier.

The specified version identifier must be in the standard format described in Version scheme, but a trailing .* is permitted as described below.

If the specified version identifier is a public version identifier (no integrator suffix), then the integrator suffix of any candidate versions MUST be ignored when matching versions.

By default, the version matching operator is based on a strict equality comparison: the specified version must be exactly the same as the requested version. The only substitution performed is the zero padding of the release segment to ensure the release segments are compared with the same length (and similarly for the integrator suffix, if matching against a specified local version identifier).

Whether or not strict version matching is appropriate depends on the specific use case for the version specifier. Automated tools SHOULD at least issue warnings and MAY reject them entirely when strict version matches are used inappropriately.

Prefix matching may be requested instead of strict comparison, by appending a trailing .* to the version identifier in the version matching clause. This means that additional trailing segments will be ignored when determining whether or not a version identifier matches the clause. If the version includes only a release segment, than trailing components in the release segment are also ignored.

For example, given the version 1.1.post1, the following clauses would match or not as shown:

== 1.1        # Not equal, so 1.1.post1 does not match clause
== 1.1.post1  # Equal, so 1.1.post1 matches clause
== 1.1.*      # Same prefix, so 1.1.post1 matches clause

The use of == (without at least the wildcard suffix) when defining dependencies for published distributions is strongly discouraged as it greatly complicates the deployment of security fixes. The strict version comparison operator is intended primarily for use when defining dependencies for repeatable deployments of applications while using a shared distribution index.

Version exclusion

A version exclusion clause includes the version exclusion operator != and a version identifier.

The allowed version identifiers and comparison semantics are the same as those of the Version matching operator, except that the sense of any match is inverted.

If the specified version identifier is a public version identifier (no integrator suffix), then the integrator suffix of any candidate versions MUST be ignored when excluding versions.

For example, given the version 1.1.post1, the following clauses would match or not as shown:

!= 1.1        # Not equal, so 1.1.post1 matches clause
!= 1.1.post1  # Equal, so 1.1.post1 does not match clause
!= 1.1.*      # Same prefix, so 1.1.post1 does not match clause

Inclusive ordered comparison

An inclusive ordered comparison clause includes a comparison operator and a version identifier, and will match any version where the comparison is correct based on the relative position of the candidate version and the specified version given the consistent ordering defined by the standard Version scheme.

The inclusive ordered comparison operators are <= and >=.

As with version matching, the release segment is zero padded as necessary to ensure the release segments are compared with the same length.

Local version identifiers are handled according to the combination of their handling by the version matching operator and the consistent ordering defined by the standard version scheme.

Exclusive ordered comparison

Exclusive ordered comparisons are similar to inclusive ordered comparisons, except that the comparison operators are < and > and the clause MUST be effectively interpreted as implying the prefix based version exclusion clause != V.*.

The exclusive ordered comparison > V MUST NOT match a post-release or maintenance release of the given version. Maintenance releases can be permitted by using the clause > V.0, while both post releases and maintenance releases can be permitted by using the inclusive ordered comparison >= V.post1.

The exclusive ordered comparison < V MUST NOT match a pre-release of the given version, even if acceptance of pre-releases is enabled as described in the section below.

Local version identifiers are handled according to the combination of their handling by the version exclusion operator and the consistent ordering defined by the standard version scheme.

Handling of pre-releases

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

By default, dependency resolution tools SHOULD:

  • accept already installed pre-releases for all version specifiers
  • accept remotely available pre-releases for version specifiers where there is no final or post release that satisfies the version specifier
  • exclude all other pre-releases from consideration

Dependency resolution tools MAY issue a warning if a pre-release is needed to satisfy a version specifier.

Dependency resolution tools SHOULD also allow users to request the following alternative behaviours:

  • accepting pre-releases for all version specifiers
  • excluding pre-releases for all version specifiers (reporting an error or warning if a pre-release is already installed locally, or if a pre-release is the only way to satisfy a particular specifier)

Dependency resolution tools MAY also allow the above behaviour to be controlled on a per-distribution basis.

Post-releases and final releases receive no special treatment in version specifiers - they are always included unless explicitly excluded.

Examples

  • 3.1: version 3.1 or later, but not version 4.0 or later.
  • 3.1.2: version 3.1.2 or later, but not version 3.2.0 or later.
  • 3.1a1: version 3.1a1 or later, but not version 4.0 or later.
  • == 3.1: specifically version 3.1 (or 3.1.0), excludes all pre-releases, post releases, developmental releases and any 3.1.x maintenance releases.
  • == 3.1.*: any version that starts with 3.1. Equivalent to the 3.1.0 compatible release clause.
  • 3.1.0, != 3.1.3: version 3.1.0 or later, but not version 3.1.3 and not version 3.2.0 or later.

Direct references

Some automated tools may permit the use of a direct reference as an alternative to a normal version specifier. A direct reference consists of the word from and an explicit URL.

Whether or not direct references are appropriate depends on the specific use case for the version specifier. Automated tools SHOULD at least issue warnings and MAY reject them entirely when direct references are used inappropriately.

Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers.

Depending on the use case, some appropriate targets for a direct URL reference may be a valid source_url entry (see PEP 426), an sdist, or a wheel binary archive. The exact URLs and targets supported will be tool dependent.

For example, a local source archive may be referenced directly:

pip (from file:///localbuilds/pip-1.3.1.zip)

Alternatively, a prebuilt archive may also be referenced:

pip (from file:///localbuilds/pip-1.3.1-py33-none-any.whl)

All direct references that do not refer to a local file URL SHOULD specify a secure transport mechanism (such as https) AND include an expected hash value in the URL for verification purposes. If a direct reference is specified without any hash information, with hash information that the tool doesn't understand, or with a selected hash algorithm that the tool considers too weak to trust, automated tools SHOULD at least emit a warning and MAY refuse to rely on the URL. If such a direct reference also uses an insecure transport, automated tools SHOULD NOT rely on the URL.

It is RECOMMENDED that only hashes which are unconditionally provided by the latest version of the standard library's hashlib module be used for source archive hashes. At time of writing, that list consists of 'md5', 'sha1', 'sha224', 'sha256', 'sha384', and 'sha512'.

For source archive and wheel references, an expected hash value may be specified by including a <hash-algorithm>=<expected-hash> entry as part of the URL fragment.

For version control references, the VCS+protocol scheme SHOULD be used to identify both the version control system and the secure transport, and a version control system with hash based commit identifiers SHOULD be used. Automated tools MAY omit warnings about missing hashes for version control systems that do not provide hash based commit identifiers.

To handle version control systems that do not support including commit or tag references directly in the URL, that information may be appended to the end of the URL using the @<commit-hash> or the @<tag>#<commit-hash> notation.

Note

This isn't quite the same as the existing VCS reference notation supported by pip. Firstly, the distribution name is moved in front rather than embedded as part of the URL. Secondly, the commit hash is included even when retrieving based on a tag, in order to meet the requirement above that every link should include a hash to make things harder to forge (creating a malicious repo with a particular tag is easy, creating one with a specific hash, less so).

Remote URL examples:

pip (from https://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686)
pip (from git+https://github.com/pypa/pip.git@7921be1537eac1e97bc40179a57f0349c2aee67d)
pip (from git+https://github.com/pypa/pip.git@1.3.1#7921be1537eac1e97bc40179a57f0349c2aee67d)

Updating the versioning specification

The versioning specification may be updated with clarifications without requiring a new PEP or a change to the metadata version.

Actually changing the version comparison semantics still requires a new versioning scheme and metadata version defined in new PEPs.

Summary of differences from PEP 386

  • Moved the description of version specifiers into the versioning PEP
  • Added the "source label" concept to better handle projects that wish to use a non-compliant versioning scheme internally, especially those based on DVCS hashes
  • Added the "direct reference" concept as a standard notation for direct references to resources (rather than each tool needing to invent its own)
  • Added the "local version identifier" and "integrator suffix" concepts to allow system integrators to indicate patched builds in a way that is supported by the upstream tools
  • Added the "compatible release" clause
  • Added the trailing wildcard syntax for prefix based version matching and exclusion
  • Changed the top level sort position of the .devN suffix
  • Allowed single value version numbers
  • Explicit exclusion of leading or trailing whitespace
  • Explicit support for date based versions
  • Implicitly exclude pre-releases unless they're already present or needed to satisfy a dependency
  • Treat post releases the same way as unqualified releases
  • Discuss ordering and dependencies across metadata versions

The rationale for major changes is given in the following sections.

Adding source labels

The new source label support is intended to make it clearer that the constraints on public version identifiers are there primarily to aid in the creation of reliable automated dependency analysis tools. Projects are free to use whatever versioning scheme they like internally, so long as they are able to translate it to something the dependency analysis tools will understand.

Changing the version scheme

The key change in the version scheme in this PEP relative to that in PEP 386 is to sort top level developmental releases like X.Y.devN ahead of alpha releases like X.Ya1. This is a far more logical sort order, as projects already using both development releases and alphas/betas/release candidates do not want their developmental releases sorted in between their release candidates and their final releases. There is no rationale for using dev releases in that position rather than merely creating additional release candidates.

The updated sort order also means the sorting of dev versions is now consistent between the metadata standard and the pre-existing behaviour of pkg_resources (and hence the behaviour of current installation tools).

Making this change should make it easier for affected existing projects to migrate to the latest version of the metadata standard.

Another change to the version scheme is to allow single number versions, similar to those used by non-Python projects like Mozilla Firefox, Google Chrome and the Fedora Linux distribution. This is actually expected to be more useful for version specifiers, but it is easier to allow it for both version specifiers and release numbers, rather than splitting the two definitions.

The exclusion of leading and trailing whitespace was made explicit after a couple of projects with version identifiers differing only in a trailing \n character were found on PyPI.

The exclusion of major release numbers that look like dates was implied by the overall text of PEP 386, but not clear in the definition of the version scheme. This exclusion has been made clear in the definition of the release component.

Appendix A shows detailed results of an analysis of PyPI distribution version information, as collected on 19th February, 2013. This analysis compares the behaviour of the explicitly ordered version schemes defined in this PEP and PEP 386 with the de facto standard defined by the behaviour of setuptools. These metrics are useful, as the intent of both PEPs is to follow existing setuptools behaviour as closely as is feasible, while still throwing exceptions for unorderable versions (rather than trying to guess an appropriate order as setuptools does).

Overall, the percentage of compatible distributions improves from 97.7% with PEP 386 to 98.7% with this PEP. While the number of projects affected in practice was small, some of the affected projects are in widespread use (such as Pinax and selenium). The surprising ordering discrepancy also concerned developers and acted as an unnecessary barrier to adoption of the new metadata standard, even for projects that weren't directly affected.

The data also shows that the pre-release sorting discrepancies are seen only when analysing all versions from PyPI, rather than when analysing public versions. This is largely due to the fact that PyPI normally reports only the most recent version for each project (unless maintainers explicitly configure their project to display additional versions). However, installers that need to satisfy detailed version constraints often need to look at all available versions, as they may need to retrieve an older release.

Even this PEP doesn't completely eliminate the sorting differences relative to setuptools:

  • Sorts differently (after translations): 38 / 28194 (0.13 %)
  • Sorts differently (no translations): 2 / 28194 (0.01 %)

The two remaining sort order discrepancies picked up by the analysis are due to a pair of projects which have PyPI releases ending with a carriage return, alongside releases with the same version number, only without the trailing carriage return.

The sorting discrepancies after translation relate mainly to differences in the handling of pre-releases where the standard mechanism is considered to be an improvement. For example, the existing pkg_resources scheme will sort "1.1beta1" after "1.1b2", whereas the suggested standard translation for "1.1beta1" is "1.1b1", which sorts before "1.1b2". Similarly, the pkg_resources scheme will sort "-dev-N" pre-releases differently from "devN" pre-releases when they occur within the same release, while the scheme in this PEP requires normalizing both representations to ".devN" and sorting them by the numeric component.

A more opinionated description of the versioning scheme

As in PEP 386, the primary focus is on codifying existing practices to make them more amenable to automation, rather than demanding that existing projects make non-trivial changes to their workflow. However, the standard scheme allows significantly more flexibility than is needed for the vast majority of simple Python packages (which often don't even need maintenance releases - many users are happy with needing to upgrade to a new feature release to get bug fixes).

For the benefit of novice developers, and for experienced developers wishing to better understand the various use cases, the specification now goes into much greater detail on the components of the defined version scheme, including examples of how each component may be used in practice.

The PEP also explicitly guides developers in the direction of semantic versioning (without requiring it), and discourages the use of several aspects of the full versioning scheme that have largely been included in order to cover esoteric corner cases in the practices of existing projects and in repackaging software for Linux distributions.

Describing version specifiers alongside the versioning scheme

The main reason to even have a standardised version scheme in the first place is to make it easier to do reliable automated dependency analysis. It makes more sense to describe the primary use case for version identifiers alongside their definition.

Changing the interpretation of version specifiers

The previous interpretation of version specifiers made it very easy to accidentally download a pre-release version of a dependency. This in turn made it difficult for developers to publish pre-release versions of software to the Python Package Index, as even marking the package as hidden wasn't enough to keep automated tools from downloading it, and also made it harder for users to obtain the test release manually through the main PyPI web interface.

The previous interpretation also excluded post-releases from some version specifiers for no adequately justified reason.

The updated interpretation is intended to make it difficult to accidentally accept a pre-release version as satisfying a dependency, while still allowing pre-release versions to be retrieved automatically when that's the only way to satisfy a dependency.

The "some forward compatibility assumed" default version constraint is derived from the Ruby community's "pessimistic version constraint" operator [2] to allow projects to take a cautious approach to forward compatibility promises, while still easily setting a minimum required version for their dependencies. It is made the default behaviour rather than needing a separate operator in order to explicitly discourage overspecification of dependencies by library developers. The explicit comparison operators remain available to cope with dependencies with unreliable or non-existent backwards compatibility policies, as well as for legitimate use cases related to deployment of integrated applications.

The optional explicit spelling of the compatible release clause (~=) is inspired by the Ruby (~>) and PHP (~) equivalents. It is defined in order to allow easier conversion to the legacy pkg_resources version specifier format (which omits the parentheses, but requires a comparison operator).

Further improvements are also planned to the handling of parallel installation of multiple versions of the same library, but these will depend on updates to the installation database definition along with improved tools for dynamic path manipulation.

The trailing wildcard syntax to request prefix based version matching was added to make it possible to sensibly define both compatible release clauses and the desired pre- and post-release handling semantics for < and > ordered comparison clauses.

Support for date based version identifiers

Excluding date based versions caused significant problems in migrating pytz to the new metadata standards. It also caused concerns for the OpenStack developers, as they use a date based versioning scheme and would like to be able to migrate to the new metadata standards without changing it.

The approach now adopted in the PEP is to:

  • consider a leading release segment component greater than or equal to 1980 to denote a "date based release"
  • using >= rather than ~= as the default comparison operator for version specifier clauses based on a date based release
  • recommend reporting an error if ~= is used with a date based release

This approach means that date based version identifiers should "just work" for pytz and any other projects with stable APIs, and at least be usable (through the use of appropriate version specifiers on the consumer side) for projects with less stable APIs.

Adding version epochs

Version epochs are added for the same reason they are part of other versioning schemes, such as those of the Fedora and Debian Linux distributions: to allow projects to gracefully change their approach to numbering releases, without having a new release appear to have a lower version number than previous releases and without having to change the name of the project.

In particular, supporting version epochs allows a project that was previously using date based versioning to switch to semantic versioning by specifying a new version epoch.

Adding direct references

Direct references are added as an "escape clause" to handle messy real world situations that don't map neatly to the standard distribution model. This includes dependencies on unpublished software for internal use, as well as handling the more complex compatibility issues that may arise when wrapping third party libraries as C extensions (this is of especial concern to the scientific community).

Index servers are deliberately given a lot of freedom to disallow direct references, since they're intended primarily as a tool for integrators rather than publishers. PyPI in particular is currently going through the process of eliminating dependencies on external references, as unreliable external services have the effect of slowing down installation operations, as well as reducing PyPI's own apparent reliability.

Adding local version identifiers

It's a fact of life that downstream integrators often need to backport upstream bug fixes to older versions. It's one of the services that gets Linux distro vendors paid, and application developers may also apply patches they need to bundled dependencies.

Historically, this practice has been invisible to cross-platform language specific distribution tools - the reported "version" in the upstream metadata is the same as for the unmodified code. This inaccuracy then can then cause problems when attempting to work with a mixture of integrator provided code and unmodified upstream code, or even just attempting to identify exactly which version of the software is installed.

The introduction of local version identifiers and the "integrator suffix" into the versioning scheme, with the corresponding python.integrator metadata extension allows this kind of activity to be represented accurately, which should improve interoperability between the upstream tools and various integrated platforms.

The exact scheme chosen is largely modelled on the existing behaviour of pkg_resources.parse_version and pkg_resources.parse_requirements, with the main distinction being that where pkg_resources currently always takes the suffix into account when comparing versions for exact matches, the PEP requires that the integrator suffix of the candidate version be ignored when no integrator suffix is present in the version specifier clause.

The hyphen is chosen primarily for readability of local version identifiers. While the wheel format also uses hyphens as separators between components, the escaping rules defined in PEP 427 will convert the hyphen in a local version identifier to an underscore before using it in a wheel filename.

This change is designed to ensure that an integrator provided version like pip 1.5-1 will still satisfy a version specifier like pip (== 1.1).

References

The initial attempt at a standardised version scheme, along with the justifications for needing such a standard can be found in PEP 386.

[1]Version compatibility analysis script: http://hg.python.org/peps/file/default/pep-0426/pepsort.py
[2]Pessimistic version constraint http://docs.rubygems.org/read/chapter/16
[3]http://semver.org/

Appendix A

Metadata v2.0 guidelines versus setuptools (note that this analysis was run when this PEP was still embedded as part of PEP 426):

$ ./pepsort.py
Comparing PEP 426 version sort to setuptools.

Analysing release versions
  Compatible: 24477 / 28194 (86.82 %)
  Compatible with translation: 247 / 28194 (0.88 %)
  Compatible with filtering: 84 / 28194 (0.30 %)
  No compatible versions: 420 / 28194 (1.49 %)
  Sorts differently (after translations): 0 / 28194 (0.00 %)
  Sorts differently (no translations): 0 / 28194 (0.00 %)
  No applicable versions: 2966 / 28194 (10.52 %)

Analysing public versions
  Compatible: 25600 / 28194 (90.80 %)
  Compatible with translation: 1505 / 28194 (5.34 %)
  Compatible with filtering: 13 / 28194 (0.05 %)
  No compatible versions: 420 / 28194 (1.49 %)
  Sorts differently (after translations): 0 / 28194 (0.00 %)
  Sorts differently (no translations): 0 / 28194 (0.00 %)
  No applicable versions: 656 / 28194 (2.33 %)

Analysing all versions
  Compatible: 24239 / 28194 (85.97 %)
  Compatible with translation: 2833 / 28194 (10.05 %)
  Compatible with filtering: 513 / 28194 (1.82 %)
  No compatible versions: 320 / 28194 (1.13 %)
  Sorts differently (after translations): 38 / 28194 (0.13 %)
  Sorts differently (no translations): 2 / 28194 (0.01 %)
  No applicable versions: 249 / 28194 (0.88 %)

Metadata v1.2 guidelines versus setuptools:

$ ./pepsort.py 386
Comparing PEP 386 version sort to setuptools.

Analysing release versions
  Compatible: 24244 / 28194 (85.99 %)
  Compatible with translation: 247 / 28194 (0.88 %)
  Compatible with filtering: 84 / 28194 (0.30 %)
  No compatible versions: 648 / 28194 (2.30 %)
  Sorts differently (after translations): 0 / 28194 (0.00 %)
  Sorts differently (no translations): 0 / 28194 (0.00 %)
  No applicable versions: 2971 / 28194 (10.54 %)

Analysing public versions
  Compatible: 25371 / 28194 (89.99 %)
  Compatible with translation: 1507 / 28194 (5.35 %)
  Compatible with filtering: 12 / 28194 (0.04 %)
  No compatible versions: 648 / 28194 (2.30 %)
  Sorts differently (after translations): 0 / 28194 (0.00 %)
  Sorts differently (no translations): 0 / 28194 (0.00 %)
  No applicable versions: 656 / 28194 (2.33 %)

Analysing all versions
  Compatible: 23969 / 28194 (85.01 %)
  Compatible with translation: 2789 / 28194 (9.89 %)
  Compatible with filtering: 530 / 28194 (1.88 %)
  No compatible versions: 547 / 28194 (1.94 %)
  Sorts differently (after translations): 96 / 28194 (0.34 %)
  Sorts differently (no translations): 14 / 28194 (0.05 %)
  No applicable versions: 249 / 28194 (0.88 %)