skip to navigation
skip to content

Python Wiki

Python Insider Blog

Python 2 or 3?

Help Fund Python

[Python resources in languages other than English]

Non-English Resources

Add an event to this calendar.

Times are shown in UTC/GMT.

Add an event to this calendar.

PEP:448
Title:Additional Unpacking Generalizations
Version:6e7708f29c04
Last-Modified:2013-08-13 17:04:22 -0400 (Tue, 13 Aug 2013)
Author:Joshua Landau <joshua at landau.ws>
Discussions-To:python-ideas at python.org
Status:Draft
Type:Standards Track
Content-Type:text/x-rst
Created:29-Jun-2013
Python-Version:3.4
Post-History:

Abstract

This PEP proposes extended usages of the * iterable unpacking operator to allow unpacking in more positions, an arbitrary number of times, and in several additional circumstances.

Specifically:

Arbitrarily positioned unpacking operators:

>>> print(*[1], *[2], 3)
1 2 3
>>> dict(**{'x': 1}, y=2, **{'z': 3})
{'x': 1, 'y': 2, 'z': 3}

Function calls currently have the restriction that keyword arguments must follow positional arguments and ** unpackings must additionally follow * unpackings. Because of the new levity for * and ** unpackings, it may be advisable to lift some or all of these restrictions.

As currently, if an argument is given multiple times - such as a positional argument given both positionally and by keyword - a TypeError is raised.

Unpacking is proposed to be allowed inside tuples, lists, sets, dictionaries and comprehensions:

>>> *range(4), 4
(0, 1, 2, 3, 4)
>>> [*range(4), 4]
[0, 1, 2, 3, 4]
>>> {*range(4), 4}
{0, 1, 2, 3, 4}
>>> {'x': 1, **{'y': 2}}
{'x': 1, 'y': 2}

>>> ranges = [range(i) for i in range(5)]
>>> [*item for item in ranges]
[0, 0, 1, 0, 1, 2, 0, 1, 2, 3]

Rationale

Current usage of the * iterable unpacking operator features unnecessary restrictions that can harm readability.

Unpacking multiple times has an obvious rationale. When you want to unpack several iterables into a function definition or follow an unpack with more positional arguments, the most natural way would be to write:

function(**kw_arguments, **more_arguments)

function(*arguments, argument)

Simple examples where this is useful are print and str.format. Instead, you could be forced to write:

kwargs = dict(kw_arguments)
kwargs.update(more_arguments)
function(**kwargs)

args = list(arguments)
args.append(arg)
function(*args)

or, if you know to do so:

from collections import ChainMap
function(**ChainMap(more_arguments, arguments))

from itertools import chain
function(*chain(args, [arg]))

which add unnecessary line-noise and, with the first methods, causes duplication of work.

There are two primary rationales for unpacking inside of containers. Firstly there is a symmetry of assignment, where fst, *other, lst = elems and elems = fst, *other, lst are approximate inverses, ignoring the specifics of types. This, in effect, simplifies the language by removing special cases.

Secondly, it vastly simplifies types of "addition" such as combining dictionaries, and does so in an unambiguous and well-defined way:

combination = {**first_dictionary, "x": 1, "y": 2}

instead of:

combination = first_dictionary.copy()
combination.update({"x": 1, "y": 2})

which is especially important in contexts where expressions are preferred. This is also useful as a more readable way of summing iterables into a list, such as my_list + list(my_tuple) + list(my_range) which is now equivalent to just [*my_list, *my_tuple, *my_range].

The addition of unpacking to comprehensions is a logical extension. It's usage will primarily be a neat replacement for [i for j in 2D_list for i in j], as the more readable [*l for l in 2D_list]. Other uses are possible, but expected to occur rarely.

Specification

Function calls may accept an unbound number of * and ** unpackings. There will be no restriction of the order of positional arguments with relation to * unpackings nor any restriction of the order of keyword arguments with relation to ** unpackings.

Function calls currently have the restriction that keyword arguments must follow positional arguments and ** unpackings must additionally follow * unpackings. Because of the new levity for * and ** unpackings, it may be advisable to list some or all of these restrictions.

As currently, if an argument is given multiple times - such as a positional argument given both positionally and by keyword - a TypeError is raised.

If the restrictions are kept, a function call will look like this:

function(
    argument or *args, argument or *args, ...,
    kwargument or *args, kwargument or *args, ...,
    kwargument or **kwargs, kwargument or **kwargs, ...
)

If they are removed completely, a function call will look like this:

function(
    argument or keyword_argument or *args or **kwargs,
    argument or keyword_argument or *args or **kwargs,
    ...
)

Tuples, lists, sets and dictionaries will allow unpacking. This will act as if the elements from unpacked items were inserted in order at the site of unpacking, much as happens in unpacking in a function-call. Dictionaries require ** unpacking; all the others require * unpacking. A dictionary's key remain in a right-to-left priority order, so {**{'a': 1}, 'a': 2, **{'a': 3}} evaluates to {'a': 3}. There is no restriction on the number or position of unpackings.

Comprehensions, by simple extension, will support unpacking. As before, dictionaries require ** unpacking, all the others require * unpacking and key priorities are unchanged.

Examples include:

{*[1, 2, 3], 4, 5, *{6, 7, 8}}

(*e for e in [[1], [3, 4, 5], [2]])

{**dictionary for dictionary in (globals(), locals())}

{**locals(), "override": None}

Disadvantages

If the current restrictions for function call arguments (keyword arguments must follow positional arguments and ** unpackings must additionally follow * unpackings) are kept, the allowable orders for arguments in a function call is more complicated than before. The simplest explanation for the rules may be "positional arguments come first and keyword arguments follow, but * unpackings are allowed after keyword arguments".

If the current restrictions are lifted, there are no obvious gains to code as the only new orders that are allowed look silly: f(a, e=e, d=d, b, c) being a simpler example.

Whilst *elements, = iterable causes elements to be a list, elements = *iterable, causes elements to be a tuple. The reason for this may not be obvious at first glance and may confuse people unfamiliar with the construct.

Implementation

An implementation for an old version of Python 3 is found at Issue 2292 on bug tracker [1], although several changes should be made:

  • It has yet to be updated to the most recent Python version
  • It features a now redundant replacement for "yield from" which should be removed
  • It also loses support for calling function with keyword arguments before positional arguments, which is an unnecessary backwards-incompatible change
  • If the restrictions on the order of arguments in a function call are partially or fully lifted, they would need to be included

References

[1]Issue 2292, "Missing *-unpacking generalizations", Thomas Wouters (http://bugs.python.org/issue2292)
[2]Discussion on Python-ideas list, "list / array comprehensions extension", Alexander Heger (http://mail.python.org/pipermail/python-ideas/2011-December/013097.html)