Abstract

This PEP provides a convention to ensure that Python scripts can continue to be portable across *nix systems, regardless of the default version of the Python interpreter (i.e. the version invoked by the python command).

• python2 will refer to some version of Python 2.x.
• python3 will refer to some version of Python 3.x.
• for the time being, all distributions should ensure that python refers to the same target as python2.
• however, end users should be aware that python refers to python3 on at least Arch Linux (that change is what prompted the creation of this PEP), so python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3.
• in preparation for an eventual change in the default version of Python, Python 2 only scripts should either be updated to be source compatible with Python 3 or else to use python2 in the shebang line.

Recommendation

• Unix-like software distributions (including systems like Mac OS X and Cygwin) should install the python2 command into the default path whenever a version of the Python 2 interpreter is installed, and the same for python3 and the Python 3 interpreter.
• When invoked, python2 should run some version of the Python 2 interpreter, and python3 should run some version of the Python 3 interpreter.
• The more general python command should be installed whenever any version of Python 2 is installed and should invoke the same version of Python as the python2 command (however, note that some distributions have already chosen to have python implement the python3 command; see the Rationale and Migration Notes below).
• The Python 2.x idle, pydoc, and python-config commands should likewise be available as idle2, pydoc2, and python2-config, with the original commands invoking these versions by default, but possibly invoking the Python 3.x versions instead if configured to do so by the system administrator.
• In order to tolerate differences across platforms, all new code that needs to invoke the Python interpreter should not specify python, but rather should specify either python2 or python3 (or the more specific python2.x and python3.x versions; see the Migration Notes). This distinction should be made in shebangs, when invoking from a shell script, when invoking via the system() call, or when invoking in any other context.
• One exception to this is scripts that are deliberately written to be source compatible with both Python 2.x and 3.x. Such scripts may continue to use python on their shebang line without affecting their portability.
• When reinvoking the interpreter from a Python script, querying sys.executable to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach.

These recommendations are the outcome of the relevant python-dev discussions in March and July 2011 ([1], [2]), February 2012 ([4]) and September 2014 ([6]).

Rationale

This recommendation is needed as, even though the majority of distributions still alias the python command to Python 2, some now alias it to Python 3 ([5]). As some of the former distributions did not provide a python2 command by default, there was previously no way for Python 2 code (or any code that invokes the Python 2 interpreter directly rather than via sys.executable) to reliably run on all Unix-like systems without modification, as the python command would invoke the wrong interpreter version on some systems, and the python2 command would fail completely on others. The recommendations in this PEP provide a very simple mechanism to restore cross-platform support, with minimal additional work required on the part of distribution maintainers.

Future Changes to this Recommendation

It is anticipated that there will eventually come a time where the third party ecosystem surrounding Python 3 is sufficiently mature for this recommendation to be updated to suggest that the python symlink refer to python3 rather than python2.

This recommendation will be periodically reviewed over the next few years, and updated when the core development team judges it appropriate. As a point of reference, regular maintenance releases for the Python 2.7 series will continue until at least 2020.

Migration Notes

This section does not contain any official recommendations from the core CPython developers. It's merely a collection of notes regarding various aspects of migrating to Python 3 as the default version of Python for a system. They will hopefully be helpful to any distributions considering making such a change.

• The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3. For example, while the change of print from a statement to a builtin function is relatively simple for automated converters to handle, the SyntaxError from attempting to use the Python 2 notation in versions of Python 3 prior to 3.4.2 is thoroughly confusing if you aren't already aware of the change:

  $ python3 -c 'print "Hello, world!"'
    File "<string>", line 1
      print "Hello, world!"
                          ^
  SyntaxError: invalid syntax
  

  (In Python 3.4.2+, that generic error message has been replaced with the more explicit "SyntaxError: Missing parentheses in call to 'print'")

• Avoiding breakage of such third party scripts is the key reason this PEP recommends that python continue to refer to python2 for the time being. Until the conventions described in this PEP are more widely adopted, having python invoke python2 will remain the recommended option.

• The pythonX.X (e.g. python2.6) commands exist on some systems, on which they invoke specific minor versions of the Python interpreter. It can be useful for distribution-specific packages to take advantage of these utilities if they exist, since it will prevent code breakage if the default minor version of a given major version is changed. However, scripts intending to be cross-platform should not rely on the presence of these utilities, but rather should be tested on several recent minor versions of the target major version, compensating, if necessary, for the small differences that exist between minor versions. This prevents the need for sysadmins to install many very similar versions of the interpreter.

• When the pythonX.X binaries are provided by a distribution, the python2 and python3 commands should refer to one of those files rather than being provided as a separate binary file.

• It is suggested that even distribution-specific packages follow the python2/python3 convention, even in code that is not intended to operate on other distributions. This will reduce problems if the distribution later decides to change the version of the Python interpreter that the python command invokes, or if a sysadmin installs a custom python command with a different major version than the distribution default. Distributions can test whether they are fully following this convention by changing the python interpreter on a test box and checking to see if anything breaks.

• If the above point is adhered to and sysadmins are permitted to change the python command, then the python command should always be implemented as a link to the interpreter binary (or a link to a link) and not vice versa. That way, if a sysadmin does decide to replace the installed python file, they can do so without inadvertently deleting the previously installed binary.

• If the Python 2 interpreter becomes uncommon, scripts should nevertheless continue to use the python3 convention rather that just python. This will ease transition in the event that yet another major version of Python is released.

• If these conventions are adhered to, it will become the case that the python command is only executed in an interactive manner as a user convenience, or to run scripts that are source compatible with both Python 2 and Python 3.

Backwards Compatibility

A potential problem can arise if a script adhering to the python2/python3 convention is executed on a system not supporting these commands. This is mostly a non-issue, since the sysadmin can simply create these symbolic links and avoid further problems. It is a significantly more obvious breakage than the sometimes cryptic errors that can arise when attempting to execute a script containing Python 2 specific syntax with a Python 3 interpreter.

Application to the CPython Reference Interpreter

While technically a new feature, the make install and make bininstall command in the 2.7 version of CPython were adjusted to create the following chains of symbolic links in the relevant bin directory (the final item listed in the chain is the actual installed binary, preceding items are relative symbolic links):

python -> python2 -> python2.7
python-config -> python2-config -> python2.7-config

Similar adjustments were made to the Mac OS X binary installer.

This feature first appeared in the default installation process in CPython 2.7.3.

The installation commands in the CPython 3.x series already create the appropriate symlinks. For example, CPython 3.2 creates:

python3 -> python3.2
idle3 -> idle3.2
pydoc3 -> pydoc3.2
python3-config -> python3.2-config

And CPython 3.3 creates:

python3 -> python3.3
idle3 -> idle3.3
pydoc3 -> pydoc3.3
python3-config -> python3.3-config
pysetup3 -> pysetup3.3

The implementation progress of these features in the default installers was managed on the tracker as issue #12627 ([3]).

Impact on PYTHON* Environment Variables

The choice of target for the python command implicitly affects a distribution's expected interpretation of the various Python related environment variables. The use of *.pth files in the relevant site-packages folder, the "per-user site packages" feature (see python -m site) or more flexible tools such as virtualenv are all more tolerant of the presence of multiple versions of Python on a system than the direct use of PYTHONPATH.

Exclusion of MS Windows

This PEP deliberately excludes any proposals relating to Microsoft Windows, as devising an equivalent solution for Windows was deemed too complex to handle here. PEP 397 and the related discussion on the python-dev mailing list address this issue (like this PEP, the PEP 397 launcher invokes Python 2 by default if versions of both Python 2 and 3 are installed on the system).

References

Copyright

This document has been placed in the public domain.