Concluding, it was essential because of the particular example you
chose. I got confused because you actually made a point of claiming
that this was essential -- I now understand that it was essential for
the example, not for restartable functions. Maybe a simpler example
would have been in order :-)
> > I would prefer to tackle such a feature separately, and concentrate on
> > generators right now.
Yes.
> If, before that, you add
> self.func = func
> self.args = args
Actually, I first coded it like that! I should trust my intuition
more before starting to think about programs :-)
> then you can also have
>
> def clone(self): # return fresh version of same generator
> return Generator(self.func, self.args)
You don't give up easily don't you :-)
> apply(func, args + (self,))
>
> Believe that line would be better as
>
> apply(func, (self,) + args)
>
> I.e., the called function surely knows it needs a Generator argument, but
> other arguments may be optional.
On second thought I agree here -- I started writing before letting my
intuition catch up :-)
> The worst aspect is that if the caller doesn't run the generator to
> completion (and it can't if the generator is modeling an infinite
> sequence, and it merely won't if the caller is doing a search for a
> sequence element that meets some condition -- both are common uses), the
> spawned thread will hang forever at a putlock acquire; unless I'm
> mistaken, the thread won't go away even if the generator is garbage-
> collected. OS's are improving, but a few thousand stray threads still
> give most of 'em indigestion <wink>.
Agreed. See my new implementation at the bottom. Note that you can't
rely on __del__() being called since the producer still has a
reference to the object, so I added a kill() method.
> > 10) I have also thought of an implementation in ceval.c. This would
> > require an extension to eval_code() to execute "continuations".
>
> I saw Steve's eyes light up all the way from Boston <wink>. I'll study
> this one later -- outta time.
Haven't heard from him yet -- he must be VERY busy :-)
> > 11) It should be possible that g.put() be called not just by the
> > generating function (pi() in our example) but also by any function
> > that it calls. ... [various complications and irregularities]
>
> I don't have a clear use in mind for this capability -- do you? Not to
> say I wouldn't stumble into one if it was there.
If you think of get() as read(), you can surely think of put() as
write(), and then surely you can think of tons of applications for
this -- it's so natural to structure a big generator as a set of
subroutines. (Haven't any real examples yet -- sorry.)
> BTW, I looked up Scheme's continuation gimmicks, and confess my head's
> still spinning too fast to think straight.
Glad we agree on this one...
> OTOH, how to deal with "get()" is more troublesome. If a "consumer"
> function is invoked via
>
> g = Generator(producer, args)
> consumer(g)
>
> then it's likely to be littered with g.get(), and this stops you from
> invoking it with (say)
>
> consumer(stdin.readline)
>
> So I'd favor calling via consumer(g.get). The similarity between
> generators and file input is well worth exploiting! For that reason I
> liked overloading EOFError with "generator exhausted" too.
Then why doesn't this apply to producers as well? Anyway for
consumers it's a user decision -- the interface doesn't specify at all
how your consumer is called. It does specify how your producer is
called though. Anyway it's trivial to derive a class from Generator
which defines read() and write() as aliases for get() and put()...
============================ Generator.py ============================
# Generator implementation using threads
import thread
Killed = 'Generator.Killed'
class Generator:
# Constructor
def __init__(self, func, args):
self.getlock = thread.allocate_lock()
self.putlock = thread.allocate_lock()
self.getlock.acquire()
self.putlock.acquire()
self.func = func
self.args = args
self.done = 0
self.killed = 0
thread.start_new_thread(self._start, ())
# Internal routine
def _start(self):
try:
self.putlock.acquire()
if not self.killed:
try:
apply(self.func, (self,) + self.args)
except Killed:
pass
finally:
if not self.killed:
self.done = 1
self.getlock.release()
# Called by producer for each value; raise Killed if no more needed
def put(self, value):
if self.killed:
raise TypeError, 'put() called on killed generator'
self.value = value
self.getlock.release() # Resume consumer thread
self.putlock.acquire() # Wait for next get() call
if self.killed:
raise Killed
# Called by producer to get next value; raise EOFError if no more
def get(self):
if self.killed:
raise TypeError, 'get() called on killed generator'
self.putlock.release() # Resume producer thread
self.getlock.acquire() # Wait for value to appear
if self.done:
raise EOFError # Say there are no more values
return self.value
# Called by consumer if no more values wanted
def kill(self):
if self.killed:
raise TypeError, 'kill() called on killed generator'
self.killed = 1
self.putlock.release()
# Clone constructor
def clone(self):
return Generator(self.func, self.args)
def pi(g):
k, a, b, a1, b1 = 2L, 4L, 1L, 12L, 4L
while 1:
# Next approximation
p, q, k = k*k, 2L*k+1L, k+1L
a, b, a1, b1 = a1, b1, p*a+q*a1, p*b+q*b1
# Print common digits
d, d1 = a/b, a1/b1
while d == d1:
g.put(int(d))
a, a1 = 10L*(a%b), 10L*(a1%b1)
d, d1 = a/b, a1/b1
def test():
g = Generator(pi, ())
g.kill()
g = Generator(pi, ())
for i in range(10): print g.get(),
print
h = g.clone()
g.kill()
while 1:
print h.get(),
test()
========================================================================
--Guido van Rossum, CWI, Amsterdam <Guido.van.Rossum@cwi.nl>
URL: <http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>