Have a look at t_bootstrap() in Modules/threadmodule.c. This little C
function is run when a new thread is fired by Python:
static void
t_bootstrap(args_raw)
void *args_raw;
{
object *args = (object *) args_raw;
object *func, *arg, *res;
threads_started++;
restore_thread((void *)NULL);
func = gettupleitem(args, 0);
arg = gettupleitem(args, 1);
res = call_object(func, arg);
DECREF(args); /* Matches the INCREF(args) in thread_start_new_thread */
if (res == NULL) {
fprintf(stderr, "Unhandled exception in thread:\n");
print_error(); /* From pythonmain.c */
}
else
DECREF(res);
(void) save_thread(); /* Should always be NULL */
exit_thread();
}
(Ignore the manipulations with args_raw and args for the moment --
this is only needed to pass arguments from the originating thread to
the new thread.)
Since the new thread is started at an unknown point by the operating
system's thread facility, it is in exactly the same predicament as you
are: it wants to run the Python interpreter but it doesn't have the
"interpreter lock". The way to get the interpreter lock is to call
restore_thread() -- the argument, which is NULL in the case of
t_bootstrap() is the new value for the "current frame pointer". A
NULL frame pointer means that there is no Python stack frame in the
current thread. When restore_thread() returns, the current thread has
the interpreter lock and can make arbitrary use of the Python
interpreter. When it wants to relinquish the interpreter lock, it
should call save_thread() -- this returns the value passed to
restore_thread(), in our example NULL. After the save_thread() call
it is no longer safe to call the interpreter.
> [The reason I am asking is that I am looking for a decent method of
> "breaking" into the debugger when a Python statement is deemed "insane".
> For example, I would like to see a "worker" thread that could monitor the
> primary Python thread, and when it deems necessary, it could call a
> "set_trace" to allow the debugger to be used. But not withstanding this
> requirement, the general answer on threads still interests me]
Clever idea... It is indeed possible to enable or disable tracing for
an already executing procedure by manipulating the f_trace member of
its stack frame. How to get access to its stack frame is another
story, but from C it's accessible as current_frame.
> Second Question:
> I am primarily running Python in a GUI environment. What I would like to do
> is have a number of "servers" running in the same process. A "traditional"
> solution would be to use seperate processes, but I want a single process so
> all output is directed to different windows in the same process.
> I would like to implement each "server" as a Python thread. However, each
> server calls "print". AFAIK, there can only ever be one "sys.stdout". This
> means that messages from threads are all sent to the same "device", meaning
> all output is mixed and jumbled.
>
> My only solution to this problem is:
> * define a "write" method in a "thread_print_control" object.
> * assign sys.stdout to this object.
> * When "write" is called, get the current thread ID, and redirect the output
> to an object specific to that thread.
>
> This implies that:
> * as each thread starts, it must tell the "thread_print_control" object what
> object to use for this thread.
> * That a call the "thread_print_control.write" may need to be protected from
> context switches - ie, we can not have another thread take execute while we
> are in our "write" code. The standard thread mechanisms can probably help
> here.
>
> Does anyone have any comments/better ideas for this problem?
Not really. You have pointed out a weakness in the current thread
model -- perhaps sys.std{in,out,err} should be per-thread instead of
per-process. I personally haven't used threads for such uses to have
a good feel for this -- my onw use of threads has been mostly
restricted to less interactive applications. I can imagine that the
global nature of other sys.* variables (e.g. ps1 and ps2, and the
effect of setprofile and settrace) should also be reconsidered. On
the other hand, sys.path and sys.modules should remain global (since
the assumption of using multiple threads is that they share global
data such as modules) and others are read-only or have a global effect
by definition (maxint, version, argv, exit etc.).
BTW, I don't think you need to protect your thread_print_control.write
with a mutex -- only registration of a new thread modifies shared data
and the dictionary insert operation is internally protected. Here's
my minimal (untested) proposal:
import thread, sys
class ThreadWriter:
"Assign an instance to sys.stdout for per-thread printing objects"
def __init__(self):
"Constructor -- initialize the table of writers"
self.writers = {}
def register(self, writer):
"Register the writer for the current thread"
self.writers[thread.get_ident()] = writer
def unregister(self):
"Remove the writer for the current thread, if any"
try:
del self.writers[thread.get_ident()]
except KeyError:
pass
def getwriter(self):
"Return the current thread's writer, default sys.stdout"
try:
return self.writers[thread.get_ident()]
except KeyError:
return sys.stdout
def write(self, str):
"Write to the current thread's writer, default sys.stdout"
self.getwriter().write(str)
The only problem that I can see with this class is that if a thread
exits without unregistering, its writer will never be removed. (Also
there is no method to unregister another thread's writer, but that's
easily added.)
--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>