skip to navigation
skip to content

Python Wiki

Python Insider Blog

Python 2 or 3?

Help Fund Python

[Python resources in languages other than English]

Non-English Resources

Add an event to this calendar.

Times are shown in UTC/GMT.

Add an event to this calendar.

PEP: 306
Title: How to Change Python's Grammar
Version: dca0d9a4476c
Last-Modified:  2011-01-18 00:31:57 +0000 (Tue, 18 Jan 2011)
Author: Michael Hudson <mwh at>, Jack Diederich <jackdied at>, Nick Coghlan <ncoghlan at>, Benjamin Peterson <benjamin at>
Status: Withdrawn
Type: Informational
Content-Type: text/plain
Created: 29-Jan-2003
Post-History: 30-Jan-2003


    This PEP has been moved to the Python dev guide.


    There's more to changing Python's grammar than editing
    Grammar/Grammar and Python/compile.c.  This PEP aims to be a
    checklist of places that must also be fixed.

    It is probably incomplete.  If you see omissions, just add them if
    you can -- you are not going to offend the author's sense of
    ownership.  Otherwise submit a bug or patch and assign it to mwh.

    This PEP is not intended to be an instruction manual on Python
    grammar hacking, for several reasons.


    People are getting this wrong all the time; it took well over a
    year before someone noticed[1] that adding the floor division
    operator (//) broke the parser module.


    __ Grammar/Grammar: OK, you'd probably worked this one out :)

    __ Parser/Python.asdl may need changes to match the Grammar.  Run
       make to regenerate Include/Python-ast.h and

    __ Python/ast.c will need changes to create the AST objects
       involved with the Grammar change.  Lib/compiler/ will
       need matching changes to the pure-python AST objects.

    __ Parser/pgen needs to be rerun to regenerate Include/graminit.h
       and Python/graminit.c. (make should handle this for you.)

    __ Python/symbtable.c: This handles the symbol collection pass
       that happens immediately before the compilation pass.

    __ Python/compile.c: You will need to create or modify the
       compiler_* functions to generate opcodes for your productions.

    __ You may need to regenerate Lib/ and/or Lib/
       and/or Lib/

    __ The parser module.  Add some of your new syntax to test_parser,
       bang on Modules/parsermodule.c until it passes.

    __ Add some usage of your new syntax to

    __ The compiler package.  A good test is to compile the standard
       library and test suite with the compiler package and then check
       it runs.  Note that this only needs to be done in Python 2.x.

    __ If you've gone so far as to change the token structure of
       Python, then the Lib/ library module will need to
       be changed.

    __ Certain changes may require tweaks to the library module

    __ Documentation must be written!

    __ After everything's been checked in, you're likely to see a new
       change to Python/Python-ast.c.  This is because this
       (generated) file contains the SVN version of the source from
       which it was generated.  There's no way to avoid this; you just
       have to submit this file separately.


    [1] SF Bug #676521, parser module validation failure


    This document has been placed in the public domain.