Abstract

    This PEP describes a Python launcher for the Windows platform.  A 
    Python launcher is a single executable which uses a number of 
    heuristics to locate a Python executable and launch it with a
    specified command line.

Rationale

    Windows provides "file associations" so an executable can be associated
    with an extension, allowing for scripts to be executed directly in some
    contexts (eg., double-clicking the file in Windows Explorer.)  Until now,
    a strategy of "last installed Python wins" has been used and while not
    ideal, has generally been workable due to the conservative changes in
    Python 2.x releases.  As Python 3.x scripts are often syntactically
    incompatible with Python 2.x scripts, a different strategy must be used
    to allow files with a '.py' extension to use a different executable based
    on the Python version the script targets.  This will be done by borrowing
    the existing practices of another operating system - scripts will be able
    to nominate the version of Python they need by way of a "shebang" line, as
    described below.

    Unix-like operating systems (referred to simply as "Unix" in this
    PEP) allow scripts to be executed as if they were executable images
    by examining the script for a "shebang" line which specifies the
    actual executable to be used to run the script.  This is described in
    detail in the evecve(2) man page [1] and while user documentation will
    be created for this feature, for the purposes of this PEP that man
    page describes a valid shebang line.

    Additionally, these operating systems provide symbolic-links to
    Python executables in well-known directories.  For example, many
    systems will have a link /usr/bin/python which references a
    particular version of Python installed under the operating-system.
    These symbolic links allow Python to be executed without regard for
    where Python it actually installed on the machine (eg., without
    requiring the path where Python is actually installed to be
    referenced in the shebang line or in the PATH.)  PEP 394 'The "python"
    command on Unix-Like Systems' [2] describes additional conventions
    for more fine-grained specification of a particular Python version.

    These 2 facilities combined allow for a portable and somewhat 
    predictable way of both starting Python interactively and for allowing
    Python scripts to execute.  This PEP describes an implementation of a 
    launcher which can offer the same benefits for Python on the Windows 
    platform and therefore allows the launcher to be the executable
    associated with '.py' files to support multiple Python versions
    concurrently.

    While this PEP offers the ability to use a shebang line which should
    work on both Windows and Unix, this is not the primary motivation for
    this PEP - the primary motivation is to allow a specific version to be
    specified without inventing new syntax or conventions to describe
    it.

Specification

    This PEP specifies features of the launcher; a prototype
    implementation is provided in [3] which will be distributed
    together with the Windows installer of Python, but will also be
    available separately (but released along with the Python
    installer). New features may be added to the launcher as
    long as the features prescribed here continue to work.

Installation

    The launcher comes in 2 versions - one which is a console program and
    one which is a "windows" (ie., GUI) program.  These 2 launchers correspond
    to the 'python.exe' and 'pythonw.exe' executables which currently ship
    with Python.  The console launcher will be named 'py.exe' and the Windows
    one named 'pyw.exe'.  The "windows" (ie., GUI) version of the launcher
    will attempt to locate and launch pythonw.exe even if a virtual shebang
    line nominates simply "python" - infact, the trailing 'w' notation is
    not supported in the virtual shebang line at all.

    The launcher is installed into the Windows directory (see
    discussion below) if installed by a privileged user. The
    stand-alone installer asks for an alternative location of the
    installer, and adds that location to the user's PATH. 

    The installation in the Windows directory is a 32-bit executable
    (see discussion); the standalone installer may also offer to install
    64-bit versions of the launcher.

    The launcher installation is registered in
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CurrentVersion\SharedDLLs
    with a reference counter.
    It contains a version resource matching the version number of the
    pythonXY.dll with which it is distributed. Independent
    installations will overwrite older version
    of the launcher with newer versions. Stand-alone releases use
    a release level of 0x10 in FIELD3 of the CPython release on which
    they are based.

    Once installed, the "console" version of the launcher is
    associated with .py files and the "windows" version associated with .pyw
    files.

    The launcher is not tied to a specific version of Python - eg., a
    launcher distributed with Python 3.3 should be capable of locating and
    executing any Python 2.x and Python 3.x version. However, the
    launcher binaries have a version resource that is the same as the
    version resource in the Python binaries that they are released with.

Python Script Launching

    The launcher is restricted to launching Python scripts.
    It is not intended as a general-purpose script launcher or
    shebang processor.

    The launcher supports the syntax of shebang lines as described
    in [1], including all restrictions listed.

    The launcher supports shebang lines referring to Python
    executables with any of the (regex) prefixes "/usr/bin/", "/usr/local/bin"
    and "/usr/bin/env *", as well as binaries specified without 

    For example, a shebang line of '#! /usr/bin/python' should work even 
    though there is unlikely to be an executable in the relative Windows 
    directory "\usr\bin".  This means that many scripts can use a single
    shebang line and be likely to work on both Unix and Windows without
    modification.

    The launcher will support fully-qualified paths to executables.
    While this will make the script inherently non-portable, it is a
    feature offered by Unix and would be useful for Windows users in
    some cases.

    The launcher will be capable of supporting implementations other than
    CPython, such as jython and IronPython, but given both the absence of
    common links on Unix (such as "/usr/bin/jython") and the inability for the
    launcher to automatically locate the installation location of these
    implementations on Windows, the launcher will support this via
    customization options.  Scripts taking advantage of this will not be
    portable (as these customization options must be set to reflect the
    configuration of the machine on which the launcher is running) but this
    ability is nonetheless considered worthwhile.

    On Unix, the user can control which specific version of Python is used
    by adjusting the links in /usr/bin to point to the desired version.  As
    the launcher on Windows will not use Windows links, cutomization options
    (exposed via both environment variables and INI files) will be used to
    override the semantics for determining what version of Python will be
    used.  For example, while a shebang line of "/usr/bin/python2" will
    automatically locate a Python 2.x implementation, an environment variable
    can override exactly which Python 2.x implementation will be chosen.
    Similarly for "/usr/bin/python" and "/usr/bin/python3".  This is
    specified in detail later in this PEP.

Shebang line parsing

    If the first command-line argument does not start with a dash ('-')
    character, an attempt will be made to open that argument as a file
    and parsed for a shebang line according to the rules in [1]::

        #! interpreter [optional-arg]

    Once parsed, the command will be categorized according to the following rules:

    * If the command starts with the definition of a customized command
      followed by a whitespace character (including a newline), the customized
      command will be used.  See below for a description of customized
      commands.

    * The launcher will define a set of prefixes which are considered Unix
      compatible commands to launch Python, namely "/usr/bin/python",
      "/usr/local/bin/python", "/usr/bin/env python", and "python".
      If a command starts with one of these strings will be treated as a
      'virtual command' and the rules described in Python Version Qualifiers
      (below) will be used to locate the executable to use.

    * Otherwise the command is assumed to be directly ready to execute - ie.
      a fully-qualified path (or a reference to an executable on the PATH)
      optionally followed by arguments.  The contents of the string will not
      be parsed - it will be passed directly to the Windows CreateProcess
      function after appending the name of the script and the launcher
      command-line arguments.  This means that the rules used by
      CreateProcess will be used, including how relative path names and
      executable references without extensions are treated.  Notably, the
      Windows command processor will not be used, so special rules used by the
      command processor (such as automatic appending of extensions other than
      '.exe', support for batch files, etc) will not be used.

    The use of 'virtual' shebang lines is encouraged as this should
    allow for portable shebang lines to be specified which work on
    multiple operating systems and different installations of the same
    operating system.

    If the first argument can not be opened as a file or if no valid
    shebang line can be found, the launcher will act as if a shebang line of
    '#!python' was found - ie., a default Python interpreter will be
    located and the arguments passed to that.  However, if a valid
    shebang line is found but the process specified by that line can not
    be started, the default interpreter will not be started - the error
    to create the specified child process will cause the launcher to display
    an appropriate message and terminate with a specific exit code.

Configuration file

    Two .ini files will be searched by the launcher - ``py.ini`` in the
    current user's "application data" directory (i.e. the directory returned
    by calling the Windows function SHGetFolderPath with CSIDL_LOCAL_APPDATA,
    %USERPROFILE%\AppData\Local on Vista+,
    %USERPROFILE%\Local Settings\Application Data on XP)
    and ``py.ini`` in the same directory as the launcher.  The same .ini
    files are used for both the 'console' version of the launcher (i.e.
    py.exe) and for the 'windows' version (i.e. pyw.exe)


    Customization specified in the "application directory" will have
    precedence over the one next to the executable, so a user, who may not
    have write access to the .ini file next to the launcher, can override
    commands in that global .ini file)
 

Virtual commands in shebang lines

    Virtual Commands are shebang lines which start with strings which would
    be expected to work on Unix platforms - examples include
    '/usr/bin/python', '/usr/bin/env python' and 'python'.  Optionally, the 
    virtual command may be suffixed with a version qualifier (see below),
    such as '/usr/bin/python2' or '/usr/bin/python3.2'.  The command executed
    is based on the rules described in Python Version Qualifiers
    below.

Customized Commands

    The launcher will support the ability to define "Customized Commands" in a
    Windows .ini file (ie, a file which can be parsed by the Windows function
    GetPrivateProfileString).  A section called '[commands]' can be created 
    with key names defining the virtual command and the value specifying the
    actual command-line to be used for this virtual command.

    For example, if an INI file has the contents:

    [commands]
    vpython=c:\bin\vpython.exe -foo

    Then a shebang line of '#! vpython' in a script named 'doit.py' will 
    result in the launcher using the command-line 'c:\bin\vpython.exe -foo 
    doit.py'

    The precise details about the names, locations and search order of the
    .ini files is in the launcher documentation [4]

Python Version Qualifiers

    Some of the features described allow an optional Python version qualifier 
    to be used.

    A version qualifier starts with a major version number and can optionally
    be followed by a period ('.') and a minor version specifier.  If the minor
    qualifier is specified, it may optionally be followed by "-32" to indicate
    the 32bit implementation of that version be used.  Note that no "-64"
    qualifier is necessary as this is the default implementation (see below).

    On 64bit Windows with both 32bit and 64bit implementations of the
    same (major.minor) Python version installed, the 64bit version will
    always be preferred.  This will be true for both 32bit and 64bit
    implementations of the launcher - a 32bit launcher will prefer to
    execute a 64bit Python installation of the specified version if
    available.  This is so the behavior of the launcher can be predicted
    knowing only what versions are installed on the PC and without
    regard to the order in which they were installed (ie, without knowing
    whether a 32 or 64bit version of Python and corresponding launcher was
    installed last).  As noted above, an optional "-32" suffix can be used
    on a version specifier to change this behaviour.

    If no version qualifiers are found in a command, the environment variable
    ``PY_PYTHON`` can be set to specify the default version qualifier - the default
    value is "2". Note this value could specify just a major version (e.g. "2") or
    a major.minor qualifier (e.g. "2.6"), or even major.minor-32.

    If no minor version qualifiers are found, the environment variable
    ``PY_PYTHON{major}`` (where ``{major}`` is the current major version qualifier
    as determined above) can be set to specify the full version. If no such option
    is found, the launcher will enumerate the installed Python versions and use
    the latest minor release found for the major version, which is likely,
    although not guaranteed, to be the most recently installed version in that
    family.

    In addition to environment variables, the same settings can be configured
    in the .INI file used by the launcher.  The section in the INI file is
    called ``[defaults]`` and the key name will be the same as the
    environment variables without the leading ``PY_`` prefix (and note that
    the key names in the INI file are case insensitive.)  The contents of
    an environment variable will override things specified in the INI file.

Command-line handling

    Only the first command-line argument will be checked for a shebang line
    and only if that argument does not start with a '-'.

    If the only command-line argument is "-h" or "--help", the launcher will
    print a small banner and command-line usage, then pass the argument to
    the default Python.  This will cause help for the launcher being printed
    followed by help for Python itself.  The output from the launcher will
    clearly indicate the extended help information is coming from the
    launcher and not Python.

    As a concession to interactively launching Python, the launcher will
    support the first command-line argument optionally being a dash ("-")
    followed by a version qualifier, as described above, to nominate a
    specific version be used.  For example, while "py.exe" may locate and
    launch the latest Python 2.x implementation installed, a command-line such
    as "py.exe -3" could specify the latest Python 3.x implementation be
    launched, while "py.exe -2.6-32" could specify a 32bit implementation
    Python 2.6 be located and launched.  If a Python 2.x implementation is
    desired to be launched with the -3 flag, the command-line would need to be
    similar to "py.exe -2 -3" (or the specific version of Python could
    obviously be launched manually without use of this launcher.)  Note that
    this feature can not be used with shebang processing as the file scanned
    for a shebang line and this argument must both be the first argument and
    therefore are mutually exclusive. 

    All other arguments will be passed untouched to the child Python process.

Process Launching

    The launcher offers some conveniences for Python developers working
    interactively - for example, starting the launcher with no command-line
    arguments will launch the default Python with no command-line arguments.
    Further, command-line arguments will be supported to allow a specific
    Python version to be launched interactively - however, these conveniences
    must not detract from the primary purpose of launching scripts and must
    be easy to avoid if desired.

    The launcher creates a subprocess to start the actual
    interpreter. See `Discussion´ below for the rationale.

Discussion

    It may be surprising that the launcher is installed into the
    Windows directory, and not the System32 directory. The reason is
    that the System32 directory is not on the Path of a 32-bit process
    running on a 64-bit system. However, the Windows directory is
    always on the path.

    The launcher that is installed into the Windows directory is a 32-bit
    executable so that the 32-bit CPython installer can provide the same
    binary for both 32-bit and 64-bit Windows installations.

    Ideally, the launcher process would execute Python directly inside
    the same process, primarily so the parent of the launcher process could
    terminate the launcher and have the Python interpreter terminate.  If the
    launcher executes Python as a sub-process and the parent of the launcher
    terminates the launcher, the Python process will be unaffected.

    However, there are a number of practical problems associated with this
    approach.  Windows does not support the execv* family of Unix functions,
    so this could only be done by the launcher dynamically loading the Python
    DLL, but this would have a number of side-effects.  The most serious
    side effect of this is that the value of sys.executable would refer to the
    launcher instead of the Python implementation.  Many Python scripts use the
    value of sys.executable to launch child processes, and these scripts may
    fail to work as expected if the launcher is used.  Consider a "parent"
    script with a shebang line of '#! /usr/bin/python3' which attempts to
    launch a child script (with no shebang) via sys.executable - currently the
    child is launched using the exact same version running the parent script.
    If sys.executable referred to the launcher the child would be likely
    executed using a Python 2.x version and would be likely to fail with a
    SyntaxError.

    Another hurdle is the support for alternative Python implementations
    using the "customized commands" feature described above, where loading
    the command dynamically into a running executable is not possible.

    The final hurdle is the rules above regarding 64bit and 32bit programs -
    a 32bit launcher would be unable to load the 64bit version of Python and
    vice-versa.

    Given these considerations, the launcher will execute its command in a
    child process, remaining alive while the child process is executing, then
    terminate with the same exit code as returned by the child.  To address
    concerns regarding the termination of the launcher not killing the child,
    the Win32 Job API will be used to arrange so that the child process is
    automatically killed when the parent is terminated (although children of
    that child process will continue as is the case now.)  As this Windows API
    is available in Windows XP and later, this launcher will not work on 
    Windows 2000 or earlier.

References

    [1] http://linux.die.net/man/2/execve

    [2] http://www.python.org/dev/peps/pep-0394/

    [3] https://bitbucket.org/vinay.sajip/pylauncher

    [4] https://bitbucket.org/vinay.sajip/pylauncher/src/tip/Doc/launcher.rst

Copyright

    This document has been placed in the public domain.