Re: [Q] Extending Python for computer vision [LONG]

Guido.van.Rossum@cwi.nl
Fri, 24 Feb 1995 14:00:41 +0100

In a reply to a message that I seem to have never received, Dan
Connolly gives lots of useful advice on designing a Python extension
for Khoros. Some additional remarks:

> import pykhoros
>
> A = pykhoros.get("mandrill")
> B = pykhoros.get("lena")
>
> C = pykhoros.add(A,B)
>
> You could do it smalltalk-style where op(x,y,z) is implemented
> as a method on x, i.e. x.op(y,z). This would facilitate A + B
> style overloading as you suggest...
>
> We could then make some intelligent mappings when we treat a kimage
> as_a_number so:
>
> C = A + B

Yes, it should be possible to hijack a few numerical operations for
this. You will just have to provide dummies for for those operations
that you don't implement (e.g. pow(A, B)?) that raise an exception
like "OperationNotImplemented". You may have to provide a dummy
"coerce" method for your type, too, which should always succeed, so
you can do the necessary type checking in the individual functions.

> I'm afraid you would have to do your own default handling, if
> you want to use keyword parameter passing. If only python supported
> Modula-3 style argument lists! See:
>
> http://www.research.digital.com/SRC/m3defn/html/calls.html

Well, the new getargs at least supports positional default arguments.
Using the new routine names, you can write

double x, y;
double z = 1.0;
int flags = 0;
if (!PyArg_ParseTuple(args, "dd|di", &x, &y, &z, &i))
return NULL;

As for true keyword parameters, I am considering adding this to the
language at some point (very much copying the M3 style). If you have
a pressing need for it we may work together on getting this in the
language earlier...

> I'm afraid that Modula-3 has the advantage of resolving these
> bindings statically at compile-time, whereas python has to
> do it at runtime for each function application, so I think
> the performance costs would be prohibitive.

Well it depends -- since we're talking about image processing
operations here, it sounds like the overhead is negligeable compared
to the cost of the actual operation, and my idea for an implementation
only incurs costs when the feature is actually being used (same as for
positional defaults).

> [2] The last issue. When confronted with the possibility of extending
> Python with a very large library one must seriously how much
> programming effort it takes to add each method to Python. I don't
> quite know at what point in the design to delegate task to a python
> layer module and what to perform in the type extension. Essentially
> this project would entail writing a large number of routines that are
> probably each just different enough that it could not be easily
> automated. Any general opinions and experience with this would be
> welcome.

I am working on a set of classes that will automate a large fraction
of this process. You will still have to write some code of your own,
but the amount of code you need to write is proportional to the number
of different types that the library talks about, rather than to the
number of functions in the library. If the library has a decent set
of headers (with function prototypes), it will be easy to tailor a
scanner (also part of my package) to extract the function definitions
from them. My package allows for easy adding of heuristics needed to
guess parameter properties that function prototypes don't show,
e.g. parameter transfer mode (in, out or in-out), and association of
parameters to specify buffer sizes etc. I have currently tested my
package only on some Macintosh toolboxes, and it's not yet foolproof
or complete, but it works pretty well once you have figured out the
conventions that the library uses.

As Tom Culliton mentions, using Jack Jansen's Modulator doesn't
automate as much for you, but is also very helpful in generating the
boilerplate... <URL:ftp://ftp.cwi.nl/pub/jack/modulator.tar.gz>

And Dan correctly mentions that getargs() and mkvalue() (or rather:
PyArgs_ParseTuple(0 and Py_BuildValue()) are your friends -- if you
can manage to use just these, there's very little worry about
reference counts...

--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>