Re: More embedding questions

Guido.van.Rossum@cwi.nl
Mon, 13 Feb 1995 13:49:44 +0100

> First of all, is there any source on embedding Python other than the
> "Extending Python" document? In particular, I am looking for some kind of
> reference on all the available C functions; they are briefly mentioned in
> the manual, but I can't find any kind of a complete list. I have had a
> brief fling with the source code, but if there is any other place... :)

I'm afraid for a complete list, the source is currently all there is.
I have plans for writing much more complete documentation for the API,
but it's a tedious job...

> Second, the application needs to have multiple Python scripts running in
> their own context. Not multi-threaded, but it may have multiple windows
> open, each of which has a completely different Python script hooked into
> its callbacks. It appears that I can supply explicate global and local
> symbol tables when executing the scripts [assuming I can figure out which
> functions to use and how to use them ;)], which I think would solve most of
> the problems here. But I'm not sure about the builtin modules -- is it
> okay to have different unrelated scripts using them? At first glance, it
> appears that the only potential problem would be the 'sys' module; most of
> the attributes there are okay since all the scripts will share stdin etc.,
> and I could probably hack around to get things to work for the others...
> but if there is any way to have something like an instantiation of sys for
> each script...

This is currently impossible. As you mention, for most built-in
modules it won't be a problem if they are shared. For sys, sharing
may be a problem, but it isn't easy to solve -- the interpreter looks
in sys at various times, so you can change sys.stdout etc., but it has
a static pointer that tells it where sys is. I may change this in a
future release (and 1.2 will already contain a few things that make it
easier), but at the moment, I think the best you can do is probably
decide which elements of sys you want to keep per context, and then
implement the context swapping yourself. Since you don't need
threads, you have total control over which context you activate, so it
isn't that much work to have a dictionary for each context that
contains the values of sys.path, sys.std{in,out,err} and whatever else
you want to preserve. (This is beginning to look like the Macintosh
implementation of application switching, where when an app is
activated its low memory globals are switched... :-)

BTW I have done some work on setting up separate contexts in 1.2, for
restricted execution but perhaps usable for other purposes as well.
If you'd like a peek you can register (with me) as a beta tester.

> And could there be any problems with this in any other modules? Do any
> modules keep internal state variables that could cause independent scripts
> to step on each other?

I'd have to inspect each module in turn to answer that one. Off the
top of my head, I'd say that most built-in modules are pretty sane and
only provide functions (__builtin__, math, posix are definitely OK).
There's a global in regex which has the current regular expression
syntax (a bad idea and in the future I will make the syntax an
optional parameter to the compile() function). The signal module
definitely has lots of per-process state. The stdwin and X modules
probably have some tables that are used in translating "raw" window
pointers back to Python objects, but with some care these would
probably be alright.

Note that there is also context maintained by the UNIX process --
e.g. although the posix module is safe, one context doing a chdir()
will affect all other contexts!

> Third, is there any simple way to set things up so that the interpreter
> will allow my program to keep a check on the scripts while they are
> running? What I am thinking of is some kind of callback into my program it
> can make when it checks for signals as per the sys.check_interval, so that
> my program can check for some kind of user input to abort run-away scripts.

You could use the mechanisms intended for the debugger, which give you
control at each executed line, but this slows things down quite a bit.
You can also use this to get control only once per function call, but
that means a simple infinite loop can't be caught. If you hide the
signal module from your users, you could use it yourself to set an
alarm timer which executes every second or so. If you have threads,
you could have another thread keep an eye open as per
sys.check_interval.

> Thanks for any help with all this! :)

You're welcome!

--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>