Numbering Note

This PEP has been renumbered to PEP 3135. The text below is the last version submitted under the old number.

Abstract

This PEP proposes syntactic sugar for use of the super type to automatically construct instances of the super type binding to the class that a method was defined in, and the instance (or class object for classmethods) that the method is currently acting upon.

The premise of the new super usage suggested is as follows:

super.foo(1, 2)

to replace the old:

super(Foo, self).foo(1, 2)

and the current __builtin__.super be aliased to __builtin__.__super__ (with __builtin__.super to be removed in Python 3.0).

It is further proposed that assignment to super become a SyntaxError, similar to the behaviour of None.

Rationale

The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don't Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many.

Specification

Within the specification section, some special terminology will be used to distinguish similar and closely related concepts. "super type" will refer to the actual builtin type named "super". A "super instance" is simply an instance of the super type, which is associated with a class and possibly with an instance of that class.

Because the new super semantics are not backwards compatible with Python 2.5, the new semantics will require a __future__ import:

from __future__ import new_super

The current __builtin__.super will be aliased to __builtin__.__super__. This will occur regardless of whether the new super semantics are active. It is not possible to simply rename __builtin__.super, as that would affect modules that do not use the new super semantics. In Python 3.0 it is proposed that the name __builtin__.super will be removed.

Replacing the old usage of super, calls to the next class in the MRO (method resolution order) can be made without explicitly creating a super instance (although doing so will still be supported via __super__). Every function will have an implicit local named super. This name behaves identically to a normal local, including use by inner functions via a cell, with the following exceptions:

1. Assigning to the name super will raise a SyntaxError at compile time;
2. Calling a static method or normal function that accesses the name super will raise a TypeError at runtime.

Every function that uses the name super, or has an inner function that uses the name super, will include a preamble that performs the equivalent of:

super = __builtin__.__super__(<class>, <instance>)

where <class> is the class that the method was defined in, and <instance> is the first parameter of the method (normally self for instance methods, and cls for class methods). For static methods and normal functions, <class> will be None, resulting in a TypeError being raised during the preamble.

Note: The relationship between super and __super__ is similar to that between import and __import__.

Much of this was discussed in the thread of the python-dev list, "Fixing super anyone?" [1].

class A(object):
    def __call__(self):
        return '__call__'
    def __getattribute__(self, attr):
        if attr == '__call__':
            return lambda: '__getattribute__'
a = A()
assert a() == '__call__'
assert a.__call__() == '__getattribute__'

Reference Implementation

It is impossible to implement the above specification entirely in Python. This reference implementation has the following differences to the specification:

1. New super semantics are implemented using bytecode hacking.
2. Assignment to super is not a SyntaxError. Also see point #4.
3. Classes must either use the metaclass autosuper_meta or inherit from the base class autosuper to acquire the new super semantics.
4. super is not an implicit local variable. In particular, for inner functions to be able to use the super instance, there must be an assignment of the form super = super in the method.

The reference implementation assumes that it is being run on Python 2.5+.

#!/usr/bin/env python
#
# autosuper.py

from array import array
import dis
import new
import types
import __builtin__
__builtin__.__super__ = __builtin__.super
del __builtin__.super

# We need these for modifying bytecode
from opcode import opmap, HAVE_ARGUMENT, EXTENDED_ARG

LOAD_GLOBAL = opmap['LOAD_GLOBAL']
LOAD_NAME = opmap['LOAD_NAME']
LOAD_CONST = opmap['LOAD_CONST']
LOAD_FAST = opmap['LOAD_FAST']
LOAD_ATTR = opmap['LOAD_ATTR']
STORE_FAST = opmap['STORE_FAST']
LOAD_DEREF = opmap['LOAD_DEREF']
STORE_DEREF = opmap['STORE_DEREF']
CALL_FUNCTION = opmap['CALL_FUNCTION']
STORE_GLOBAL = opmap['STORE_GLOBAL']
DUP_TOP = opmap['DUP_TOP']
POP_TOP = opmap['POP_TOP']
NOP = opmap['NOP']
JUMP_FORWARD = opmap['JUMP_FORWARD']
ABSOLUTE_TARGET = dis.hasjabs

def _oparg(code, opcode_pos):
    return code[opcode_pos+1] + (code[opcode_pos+2] << 8)

def _bind_autosuper(func, cls):
    co = func.func_code
    name = func.func_name
    newcode = array('B', co.co_code)
    codelen = len(newcode)
    newconsts = list(co.co_consts)
    newvarnames = list(co.co_varnames)

    # Check if the global 'super' keyword is already present
    try:
        sn_pos = list(co.co_names).index('super')
    except ValueError:
        sn_pos = None

    # Check if the varname 'super' keyword is already present
    try:
        sv_pos = newvarnames.index('super')
    except ValueError:
        sv_pos = None

    # Check if the callvar 'super' keyword is already present
    try:
        sc_pos = list(co.co_cellvars).index('super')
    except ValueError:
        sc_pos = None

    # If 'super' isn't used anywhere in the function, we don't have anything to do
    if sn_pos is None and sv_pos is None and sc_pos is None:
        return func

    c_pos = None
    s_pos = None
    n_pos = None

    # Check if the 'cls_name' and 'super' objects are already in the constants
    for pos, o in enumerate(newconsts):
        if o is cls:
            c_pos = pos

        if o is __super__:
            s_pos = pos

        if o == name:
            n_pos = pos

    # Add in any missing objects to constants and varnames
    if c_pos is None:
        c_pos = len(newconsts)
        newconsts.append(cls)

    if n_pos is None:
        n_pos = len(newconsts)
        newconsts.append(name)

    if s_pos is None:
        s_pos = len(newconsts)
        newconsts.append(__super__)

    if sv_pos is None:
        sv_pos = len(newvarnames)
        newvarnames.append('super')

    # This goes at the start of the function. It is:
    #
    #   super = __super__(cls, self)
    #
    # If 'super' is a cell variable, we store to both the
    # local and cell variables (i.e. STORE_FAST and STORE_DEREF).
    #
    preamble = [
        LOAD_CONST, s_pos & 0xFF, s_pos >> 8,
        LOAD_CONST, c_pos & 0xFF, c_pos >> 8,
        LOAD_FAST, 0, 0,
        CALL_FUNCTION, 2, 0,
    ]

    if sc_pos is None:
        # 'super' is not a cell variable - we can just use the local variable
        preamble += [
            STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
        ]
    else:
        # If 'super' is a cell variable, we need to handle LOAD_DEREF.
        preamble += [
            DUP_TOP,
            STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
            STORE_DEREF, sc_pos & 0xFF, sc_pos >> 8,
        ]

    preamble = array('B', preamble)

    # Bytecode for loading the local 'super' variable.
    load_super = array('B', [
        LOAD_FAST, sv_pos & 0xFF, sv_pos >> 8,
    ])

    preamble_len = len(preamble)
    need_preamble = False
    i = 0

    while i < codelen:
        opcode = newcode[i]
        need_load = False
        remove_store = False

        if opcode == EXTENDED_ARG:
            raise TypeError("Cannot use 'super' in function with EXTENDED_ARG opcode")

        # If the opcode is an absolute target it needs to be adjusted
        # to take into account the preamble.
        elif opcode in ABSOLUTE_TARGET:
            oparg = _oparg(newcode, i) + preamble_len
            newcode[i+1] = oparg & 0xFF
            newcode[i+2] = oparg >> 8

        # If LOAD_GLOBAL(super) or LOAD_NAME(super) then we want to change it into
        # LOAD_FAST(super)
        elif (opcode == LOAD_GLOBAL or opcode == LOAD_NAME) and _oparg(newcode, i) == sn_pos:
            need_preamble = need_load = True

        # If LOAD_FAST(super) then we just need to add the preamble
        elif opcode == LOAD_FAST and _oparg(newcode, i) == sv_pos:
            need_preamble = need_load = True

        # If LOAD_DEREF(super) then we change it into LOAD_FAST(super) because
        # it's slightly faster.
        elif opcode == LOAD_DEREF and _oparg(newcode, i) == sc_pos:
            need_preamble = need_load = True

        if need_load:
            newcode[i:i+3] = load_super

        i += 1

        if opcode >= HAVE_ARGUMENT:
            i += 2

    # No changes needed - get out.
    if not need_preamble:
        return func

    # Our preamble will have 3 things on the stack
    co_stacksize = max(3, co.co_stacksize)

    # Conceptually, our preamble is on the `def` line.
    co_lnotab = array('B', co.co_lnotab)

    if co_lnotab:
        co_lnotab[0] += preamble_len

    co_lnotab = co_lnotab.tostring()

    # Our code consists of the preamble and the modified code.
    codestr = (preamble + newcode).tostring()

    codeobj = new.code(co.co_argcount, len(newvarnames), co_stacksize,
                       co.co_flags, codestr, tuple(newconsts), co.co_names,
                       tuple(newvarnames), co.co_filename, co.co_name,
                       co.co_firstlineno, co_lnotab, co.co_freevars,
                       co.co_cellvars)

    func.func_code = codeobj
    func.func_class = cls
    return func

class autosuper_meta(type):
    def __init__(cls, name, bases, clsdict):
        UnboundMethodType = types.UnboundMethodType

        for v in vars(cls):
            o = getattr(cls, v)
            if isinstance(o, UnboundMethodType):
                _bind_autosuper(o.im_func, cls)

class autosuper(object):
    __metaclass__ = autosuper_meta

if __name__ == '__main__':
    class A(autosuper):
        def f(self):
            return 'A'

    class B(A):
        def f(self):
            return 'B' + super.f()

    class C(A):
        def f(self):
            def inner():
                return 'C' + super.f()

            # Needed to put 'super' into a cell
            super = super
            return inner()

    class D(B, C):
        def f(self, arg=None):
            var = None
            return 'D' + super.f()

    assert D().f() == 'DBCA'

Disassembly of B.f and C.f reveals the different preambles used when super is simply a local variable compared to when it is used by an inner function.

>>> dis.dis(B.f)

214           0 LOAD_CONST               4 (<type 'super'>)
              3 LOAD_CONST               2 (<class '__main__.B'>)
              6 LOAD_FAST                0 (self)
              9 CALL_FUNCTION            2
             12 STORE_FAST               1 (super)

215          15 LOAD_CONST               1 ('B')
             18 LOAD_FAST                1 (super)
             21 LOAD_ATTR                1 (f)
             24 CALL_FUNCTION            0
             27 BINARY_ADD
             28 RETURN_VALUE

>>> dis.dis(C.f)

218           0 LOAD_CONST               4 (<type 'super'>)
              3 LOAD_CONST               2 (<class '__main__.C'>)
              6 LOAD_FAST                0 (self)
              9 CALL_FUNCTION            2
             12 DUP_TOP
             13 STORE_FAST               1 (super)
             16 STORE_DEREF              0 (super)

219          19 LOAD_CLOSURE             0 (super)
             22 LOAD_CONST               1 (<code object inner at 00C160A0, file "autosuper.py", line 219>)
             25 MAKE_CLOSURE             0
             28 STORE_FAST               2 (inner)

223          31 LOAD_FAST                1 (super)
             34 STORE_DEREF              0 (super)

224          37 LOAD_FAST                2 (inner)
             40 CALL_FUNCTION            0
             43 RETURN_VALUE

Note that in the final implementation, the preamble would not be part of the bytecode of the method, but would occur immediately following unpacking of parameters.

Alternative Proposals

def f(self, *p, **kw):
    super.f(*p, **kw)

def f(self, *p, **kw):
    super(*p, **kw)

History

29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super"

• Updated much of the language and added a terminology section for clarification in confusing places.
• Added reference implementation and history sections.

06-May-2007 - Updated by Tim Delaney to reflect discussions on the python-3000

and python-dev mailing lists.

References

Copyright

This document has been placed in the public domain.