This draft does not include a set of notes as this drafts resolves the
nores as follows:
> 1
> Some special routines currently return 0 on success and -1 on
> failure. In the proposal above, I return 1 on success and 0 on
> failure. This makes error checking more consistent. For example, I
> find the following macro very useful:
>
> #define TRY(E) if(! (E)) return NULL;
>
> This allows things like:
>
> TRY(spam=PySequence_getitem(o,i));
>
> which I find more readable (especially if there are a lot of these)
> than:
>
> if(! (spam=PySequence_getitem(o,i))
> return NULL;
>
> or:
>
> spam=PySequence_getitem(o,i);
> if(! spam)
> return NULL;
>
> The proposed change in return semantics could be implemented by an
> abstraction layer without changing existing code.
> 2
> Some special routines return values that may be 0 for a normal
> return. For these routines, I have made the return value an output
> argument so as to retain the ability to check for errors by checking
> whether the return value is zero.
I decided to abandon this idea and stick with the more standard (for
Python C routines) behavior as follows: all non-boolean integer
functions return -1 on failure.
> 3
> What, if any, coercion takes place? Is the same model supported for
> both classes and built-in types? What needs to be said here?
> I don't see any hooks at the C level for providing "right" versions
> of these.
This was resolved by what became a guiding principle of the interface
design, which is that, where possible, all routines should provide the
same samantics as equivalent facilities in the interpretive
environment. As a result of this, many of the proposed routines make
calls to the same routines that the interpreter calls.
> 4
> Some of the proposed routines already exists and provide no way to
> separately return an error result and a normal result. In this
> case, I have not included the normal result in the argument list.
This is made moot by my response to notes 1 and 2. The only exception
is the routine PyObject_Cmp, which returns it's value through an
output argument, because -1 is a valid return value.
> 5
> (snip)
This note was not used in the original proposal.
> 7
> I wonder if it might not be a good idea, in Python 1.2 to fix the
> inconsistent memory management behavior of built-in types. In 1.2,
> the new Python naming scheme is being introduced. In most cases,
> new names are introduced (or equivalently old names are retained)
> through preprocessor macros. Perhaps where we desire to change the
> memory management semantics, the new names should have different
> semantics that the old names. For example, setlistitem could retain
> the current semantics and PyList_SetItem could provide the semantics
> described here.
The decision was to leave the existing routines alone, as too much
code would be affected. It turns out that the built-in types have
the propsed memory management semantics, however, the type-specific
functions used to access them do not have the proposed semantics.
When accessed through this proposed interface, the proposed semantics
are provided.
I added three functions: PyObject_CallFunction, PyObject_CallMethod,
and PyArg_ParseReturn.
I removed three functions: PyObject_Spam, and PyMapping_Clear.
So here is what I hope will be a more or less final proposal/spec for
the generic python object interface.
PROPOSAL: A Generic Python Object Interface for Python C Modules
DRAFT 2
Problem
Python modules written in C that must access Python objects must do
so through routines whose interfaces are described by a set of
include files. Unfortunately, these routines vary according to the
object accessed. To use these routines, the C programmer must check
the type of the object being used and must call a routine based on
the object type. For example, to access an element of a sequence,
the programmer must determine whether the sequence is a list or a
tuple:
if(is_tupleobject(o))
e=gettupleitem(o,i)
else if(is_listitem(o))
e=getlistitem(o,i)
If the programmer wants to get an item from another type of object
that provides sequence behavior, there is no clear way to do it
correctly.
The persistent programmer may peruse object.h and find that the
_typeobject structure provides a means of invoking up to (currently
about) 41 special operators. So, for example, a routine can get an
item from any object that provides sequence behavior. However, to
use this mechanism, the programmer must make their code dependent on
the current Python implementation.
Also, certain semantics, especially memory management semantics, may
differ by the type of object being used. Unfortunately, these
semantics are not clearly described in the current include files.
An abstract interface providing more consistent semantics is needed.
Proposal
I propose the creation of a standard interface (with an associated
library of routines and/or macros) for generically obtaining the
services of Python objects. This proposal can be viewed as one
components of a Python C interface consisting of several components.
From the viewpoint of of C access to Python services, we have (as
suggested by Guido in off-line discussions):
- "Very high level layer": two or three functions that let you exec or
eval arbitrary Python code given as a string in a module whose name is
given, passing C values in and getting C values out using
mkvalue/getargs style format strings. This does not require the user
to declare any variables of type "PyObject *". This should be enough
to write a simple application that gets Python code from the user,
execs it, and returns the output or errors. (Error handling must also
be part of this API.)
- "Abstract objects layer": which is the subject of this proposal.
It has many functions operating on objects, and lest you do many
things from C that you can also write in Python, without going
through the Python parser.
- "Concrete objects layer": This is the public type-dependent
interface provided by the standard built-in types, such as floats,
strings, and lists. This interface exists and is currently
documented by the collection of include files provides with the
Python distributions.
From the point of view of Python accessing services provided by C
modules:
- "Python module interface": this interface consist of the basic
routines used to define modules and their members. Most of the
current extensions-writing guide deals with this interface.
- "Built-in object interface": this is the interface that a new
built-in type must provide and the mechanisms and rules that a
developer of a new built-in type must use and follow.
This proposal is a "first-cut" that is intended to spur
discussion. See especially the lists of notes.
The Python C object interface will provide four protocols: object,
numeric, sequence, and mapping. Each protocol consists of a
collection of related operations. If an operation that is not
provided by a particular type is invoked, then a standard exception,
NotImplementedError is raised with a operation name as an argument.
In addition, for convenience this interface defines a set of
constructors for building objects of built-in types. This is needed
so new objects can be returned from C functions that otherwise treat
objects generically.
Memory Management
For all of the functions described in this proposal, if a function
retains a reference to a Python object passed as an argument, then the
function will increase the reference count of the object. It is
unnecessary for the caller to increase the reference count of an
argument in anticipation of the object's retention.
All Python objects returned from functions should be treated as new
objects. Functions that return objects assume that the caller will
retain a reference and the reference count of the object has already
been incremented to account for this fact. A caller that does not
retain a reference to an object that is returned from a function
must decrement the reference count of the object (using
Py_DECREF(object)) to prevent memory leaks.
Note that the behavior mentioned here is different from the current
behavior for some objects (e.g. lists and tuples) when certain
type-specific routines are called directly (e.g. setlistitem). The
proposed abstraction layer will provide a consistent memory
management interface, correcting for inconsistent behavior for some
built-in types.
Protocols
Object Protocol:
int PyObject_Print(PyObject *o, FILE *fp, int flags)
Print an object, o, on file, fp. Returns -1 on
error. The flags argument is used to enable certain printing
options. The only option currently supported is Py_Print_RAW.
(What should be said about Py_Print_RAW?)
int PyObject_HasAttrString(PyObject *o, char *attr_name)
Returns 1 if o has the attribute attr_name, and 0 otherwise.
This is equivalent to the Python expression:
hasattr(o,attr_name).
This function always succeeds.
PyObject* PyObject_GetAttrString(PyObject *o, char *attr_name)
Retrieve an attributed named attr_name form object o.
Returns the attribute value on success, or NULL on failure.
This is the equivalent of the Python expression: o.attr_name.
int PyObject_HasAttr(PyObject *o, PyObject *attr_name)
Returns 1 if o has the attribute attr_name, and 0 otherwise.
This is equivalent to the Python expression:
hasattr(o,attr_name).
This function always succeeds.
PyObject* PyObject_GetAttr(PyObject *o, PyObject *attr_name)
Retrieve an attributed named attr_name form object o.
Returns the attribute value on success, or NULL on failure.
This is the equivalent of the Python expression: o.attr_name.
int PyObject_SetAttrString(PyObject *o, char *attr_name, PyObject *v)
Set the value of the attribute named attr_name, for object o,
to the value, v. Returns -1 on failure. This is
the equivalent of the Python statement: o.attr_name=v.
int PyObject_SetAttr(PyObject *o, PyObject *attr_name, PyObject *v)
Set the value of the attribute named attr_name, for object o,
to the value, v. Returns -1 on failure. This is
the equivalent of the Python statement: o.attr_name=v.
int PyObject_DelAttrString(PyObject *o, char *attr_name)
Delete attribute named attr_name, for object o. Returns
-1 on failure. This is the equivalent of the Python
statement: del o.attr_name.
int PyObject_DelAttr(PyObject *o, PyObject *attr_name)
Delete attribute named attr_name, for object o. Returns
-1 on failure. This is the equivalent of the Python
statement: del o.attr_name.
int PyObject_Cmp(PyObject *o1, PyObject *o2, int *result)
Compare the values of o1 and o2 using a routine provided by
o1, if one exists, otherwise with a routine provided by o2.
The result of the comparison is returned in result. Returns
-1 on failure. This is the equivalent of the Python
statement: result=cmp(o1,o2).
int PyObject_Compare(PyObject *o1, PyObject *o2)
Compare the values of o1 and o2 using a routine provided by
o1, if one exists, otherwise with a routine provided by o2.
Returns the result of the comparison on success. On error,
the value returned is undefined. This is equivalent to the
Python expression: cmp(o1,o2).
PyObject *PyObject_Repr(PyObject *o)
Compute the string representation of object, o. Returns the
string representation on success, NULL on failure. This is
the equivalent of the Python expression: repr(o).
Called by the repr() built-in function and by reverse quotes.
PyObject *PyObject_Str(PyObject *o)
Compute the string representation of object, o. Returns the
string representation on success, NULL on failure. This is
the equivalent of the Python expression: str(o).)
Called by the str() built-in function and by the print
statement.
int PyCallable_Check(PyObject *o);
Determine if the object, o, is callable. Return 1 if the
object is callable and 0 otherwise.
This function always succeeds.
PyObject *PyObject_CallObject(PyObject *callable_object, PyObject *args)
Call a callable Python object, callable_object, with
arguments given by the tuple, args. If no arguments are
needed, then args may be NULL. Returns the result of the
call on success, or NULL on failure. This is the equivalent
of the Python expression: apply(o,args).
PyObject *PyObject_CallFunction(PyObject *callable_object, char *format, ...)
Call a callable Python object, callable_object, with a
variable number of C arguments. The C arguments are described
using a mkvalue-style format string. The format may be NULL,
indicating that no arguments are provided. Returns the
result of the call on success, or NULL on failure. This is
the equivalent of the Python expression: apply(o,args).
PyObject *PyObject_CallMethod(PyObject *o, char *m, char *format, ...)
Call the method named m of object o with a variable number of
C arguments. The C arguments are described by a mkvalue
format string. The format may be NULL, indicating that no
arguments are provided. Returns the result of the call on
success, or NULL on failure. This is the equivalent of the
Python expression: o.method(args).
Note that Special method names, such as "__add__",
"__getitem__", and so on are not supported. The specific
abstract-object routines for these must be used.
int PyArg_ParseReturn(PyObject *o, char *format, ...);
Parse the object, o, according to the newgetargs style
format, format. Place the result in the C objects pointed to
by the additional arguments. The reference count of the
object, o, is reduced by one. This functions is equivalent
to PyArg_ParseTuple (newgetargs), except that the object
being parsed is "destroyed" after use. The object being
parsed is also wrapped in a tuple, as expected by
PyArg_ParseTuple. If o is NULL and an error has already been
raised, then the function simply returns 0, without raising
an error. This function is intended to be used with other
functions that return PyObject pointers to convert the
results of these functions directly to C objects. For
example:
PyArg_ParseReturn(
PyObject_CallMethodStringC(foo,"bar","fi",2.5,9),
"i",&j)
The function returns non-zero on success, and zero on
failure.
long PyObject_Hash(PyObject *o)
Compute and return the hash, hash_value, of an object, o. On
failure, return -1. This is the equivalent of the Python
expression: hash(o).
int PyObject_IsTrue(PyObject *o)
Returns 1 if the object, o, is considered to be true, and
0 otherwise. This is equivalent to the Python expression:
not not o
This function always succeeds.
PyObject *PyObject_Type(PyObject *o)
On success, returns a type object corresponding to the object
type of object o. On failure, returns NULL. This is
equivalent to the Python expression: type(o).
int PyObject_Length(PyObject *o)
Return the length of object o. If the object, o, provides
both sequence and mapping protocols, the sequence length is
returned. On error, -1 is returned. This is the equivalent
to the Python expression: len(o).
PyObject *PyObject_GetItem(PyObject *o, PyObject *key)
Return element of o corresponding to the object, key, or NULL
on failure. This is the equivalent of the Python expression:
o[key].
int PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v)
Map the object, key, to the value, v. Returns -1 on failure.
This is the equivalent of the Python statement: o[key]=v.
Number Protocol:
int PyNumber_Check(PyObject *o)
Returns 1 if the object, o, provides numeric protocols, and
false otherwise.
This function always succeeds.
PyObject *PyNumber_Add(PyObject *o1, PyObject *o2)
Returns the result of adding o1 and o2, or null on failure.
This is the equivalent of the Python expression: o1+o2.
PyObject *PyNumber_Subtract(PyObject *o1, PyObject *o2)
Returns the result of subtracting o2 from o1, or null on
failure. This is the equivalent of the Python expression:
o1-o2.
PyObject *PyNumber_Multiply(PyObject *o1, PyObject *o2)
Returns the result of multiplying o1 and o2, or null on
failure. This is the equivalent of the Python expression:
o1*o2.
PyObject *PyNumber_Divide(PyObject *o1, PyObject *o2)
Returns the result of dividing o1 by o2, or null on failure.
This is the equivalent of the Python expression: o1/o2.
PyObject *PyNumber_Remainder(PyObject *o1, PyObject *o2)
Returns the remainder of dividing o1 by o2, or null on
failure. This is the equivalent of the Python expression:
o1%o2.
PyObject *PyNumber_Divmod(PyObject *o1, PyObject *o2)
See the built-in function divmod. Returns NULL on failure.
This is the equivalent of the Python expression:
divmod(o1,o2).
PyObject *PyNumber_Power(PyObject *o1, PyObject *o2, PyObject *o3)
See the built-in function pow. Returns NULL on failure.
This is the equivalent of the Python expression:
pow(o1,o2,o3), where o3 is optional.
PyObject *PyNumber_Negative(PyObject *o)
Returns the negation of o on success, or null on failure.
This is the equivalent of the Python expression: -o.
PyObject *PyNumber_Positive(PyObject *o)
Returns the (what?) of o on success, or NULL on failure.
This is the equivalent of the Python expression: +o.
PyObject *PyNumber_Absolute(PyObject *o)
Returns the absolute value of o, or null on failure. This is
the equivalent of the Python expression: abs(o).
PyObject *PyNumber_Invert(PyObject *o)
Returns the bitwise negation of o on success, or NULL on
failure. This is the equivalent of the Python expression:
~o.
PyObject *PyNumber_Lshift(PyObject *o1, PyObject *o2)
Returns the result of left shifting o1 by o2 on success, or
NULL on failure. This is the equivalent of the Python
expression: o1 << o2.
PyObject *PyNumber_Rshift(PyObject *o1, PyObject *o2)
Returns the result of right shifting o1 by o2 on success, or
NULL on failure. This is the equivalent of the Python
expression: o1 >> o2.
PyObject *PyNumber_And(PyObject *o1, PyObject *o2)
Returns the result of "anding" o2 and o2 on success and NULL
on failure. This is the equivalent of the Python
expression: o1 and o2.
PyObject *PyNumber_Xor(PyObject *o1, PyObject *o2)
Returns the bitwise exclusive or of o1 by o2 on success, or
NULL on failure. This is the equivalent of the Python
expression: o1^o2.
PyObject *PyNumber_Or(PyObject *o1, PyObject *o2)
Returns the result or o1 and o2 on success, or NULL on
failure. This is the equivalent of the Python expression:
o1 or o2.
int PyNumber_Coerce(PyObject *o1, PyObject *o2)
On success, returns a tuple containing o1 and o2 converted to
a common numeric type, or None if no conversion is possible.
Returns NULL on failure. This is equivalent to the Python
expression: coerce(o1,o2).
PyObject *PyNumber_Int(PyObject *o)
Returns the o converted to an integer object on success, or
NULL on failure. This is the equivalent of the Python
expression: int(o).
PyObject *PyNumber_Long(PyObject *o)
Returns the o converted to a long integer object on success,
or NULL on failure. This is the equivalent of the Python
expression: long(o).
PyObject *PyNumber_Float(PyObject *o)
Returns the o converted to a float object on success, or NULL
on failure. This is the equivalent of the Python expression:
float(o).
Sequence protocol:
int PySequence_Check(PyObject *o)
Return 1 if the object provides sequence protocol, and zero
otherwise.
This function always succeeds.
PyObject *PySequence_Concat(PyObject *o1, PyObject *o2)
Return the concatination of o1 and o2 on success, and NULL on
failure. This is the equivalent of the Python
expression: o1+o2.
PyObject *PySequence_Repeat(PyObject *o, int count)
Return the result of repeating sequence object o count times,
or NULL on failure. This is the equivalent of the Python
expression: o1*count.
PyObject *PySequence_GetItem(PyObject *o, int i)
Return the ith element of o, or NULL on failure. This is the
equivalent of the Python expression: o[i].
PyObject *PySequence_GetSlice(PyObject *o, int i1, int i2)
Return the slice of sequence object o between i1 and i2, or
NULL on failure. This is the equivalent of the Python
expression: o[i1:i2].
int PySequence_SetItem(PyObject *o, int i, PyObject *v)
Assign object v to the ith element of o. Returns
-1 on failure. This is the equivalent of the Python
statement: o[i]=v.
int PySequence_SetSlice(PyObject *o, int i1, int i2, PyObject *v)
Assign the sequence object, v, to the slice in sequence
object, o, from i1 to i2. This is the equivalent of the Python
statement: o[i1:i2]=v. Returns -1 on failure.
PyObject *PySequence_Tuple(PyObject *o)
Returns the o as a tuple on success, and NULL on failure.
This is equivalent to the Python expression: tuple(o)
int PySequence_Count(PyObject *o, PyObject *value)
Return the number of occurrences on value on o, that is,
return the number of keys for which o[key]==value. On
failure, return -1. This is equivalent to the Python
expression: o.count(value).
int PySequence_In(PyObject *o, PyObject *value)
Determine if o contains value. If an item in o is equal to
X, return 1, otherwise return 0. On error, return -1. This
is equivalent to the Python expression: value in o.
int PySequence_Index(PyObject *o, PyObject *value)
Return the first index for which o[i]=value. On error,
return -1. This is equivalent to the Python
expression: o.index(value).
Mapping protocol:
int PyMapping_Check(PyObject *o)
Return 1 if the object provides mapping protocol, and zero
otherwise.
This function always succeeds.
int PyMapping_Length(PyObject *o)
Returns the number of keys in object o on success, and -1 on
failure. For objects that do not provide sequence protocol,
this is equivalent to the Python expression: len(o).
int PyMapping_DelItemString(PyObject *o, char *key)
Remove the mapping for object, key, from the object *o.
Return -1 on failure. This is equivalent to the Python
statement: del o[key].
int PyMapping_DelItem(PyObject *o, PyObject *key)
Remove the mapping for object, key, from the object *o.
Return -1 on failure. This is equivalent to the Python
statement: del o[key].
int PyMapping_HasKeyString(PyObject *o, char *key)
Return 1 if the mapping object has the key, key, and 0
otherwise. This is equivalent to the Python expression:
o.has_key(key).
This function always succeeds.
int PyMapping_HasKey(PyObject *o, PyObject *key)
Return 1 if the mapping object has the key, key, and 0
otherwise. This is equivalent to the Python expression:
o.has_key(key).
This function always succeeds.
PyObject *PyMapping_Keys(PyObject *o)
On success, return a list of the keys in object o. On
failure, return NULL. This is equivalent to the Python
expression: o.keys().
PyObject *PyMapping_Values(PyObject *o)
On success, return a list of the values in object o. On
failure, return NULL. This is equivalent to the Python
expression: o.values().
PyObject *PyMapping_Items(PyObject *o)
On success, return a list of the items in object o, where
each item is a tuple containing a key-value pair. On
failure, return NULL. This is equivalent to the Python
expression: o.items().
PyObject *PyMapping_GetItemString(PyObject *o, char *key)
Return element of o corresponding to the object, key, or NULL
on failure. This is the equivalent of the Python expression:
o[key].
PyObject *PyMapping_SetItemString(PyObject *o, char *key, PyObject *value)
Map the object, key, to the value, v. Returns -1 on failure.
This is the equivalent of the Python statement: o[key]=v.
Constructors:
PyObject *PyFile_FromString(char *file_name, char *mode)
On success, returns a new file object that is opened on the
file given by file_name, with a file mode given by mode,
where mode has the same semantics as the standard C routine,
fopen. On failure, return NULL.
PyObject *PyFile_FromFile(FILE *fp,
char *file_name, char *mode,
int close_on_del)
Return a new file object for an already opened standard C
file pointer, fp. A file name, file_name, and open mode,
mode, must be provided as well as a flag, close_on_del, that
indicates whether the file is to be closed when the file
object is destroyed. On failure, return NULL.
PyObject *PyFloat_FromDouble(double v)
Returns a new float object with the value, v, on success, and
NULL on failure.
PyObject *PyInt_FromLong(long v)
Returns a new int object with the value, v, on success, and
NULL on failure.
PyObject *PyList_New(int l)
Returns a new list of length, l, on success, and NULL on
failure.
PyObject *PyLong_FromLong(long v)
Returns a new long object with the value, v, on success, and
NULL on failure.
PyObject *PyLong_FromDouble(double v)
Returns a new long object with the value, v, on success, and
NULL on failure.
PyObject *PyDict_New()
Returns a new empty dictionary on success, and NULL on
failure.
PyObject *PyString_FromString(char *v)
Returns a new string object with the value, v, on success, and
NULL on failure.
PyObject *PyString_FromStringAndSize(char *v, int l)
Returns a new string object with the value, v, and length, l,
on success, and NULL on failure.
PyObject *PyTuple_New(int l)
Returns a new tuple of length, l, on success, and NULL on
failure.
$Id: Python-C-Interface,v 1.4 1995/03/27 15:02:50 jfulton Exp $
--
-- Jim Fulton jfulton@mailqvarsa.er.usgs.gov (703) 648-5622
U.S. Geological Survey, Reston VA 22092
This message is being posted to obtain or provide technical information
relating to my duties at the U.S. Geological Survey.