class _Wrapper: # same as last time
def __init__(self, func, initstate):
self.worker = func._work
self.state = initstate
def invoke(self):
return self.worker(self.state)
class intsfrom: # same as last time
def __init__(self, n): # output = n, n+1, n+2, ...
self.n = n
def _wrap(self):
return _Wrapper(self, [self.n]).invoke
def _work(self, state):
[i] = state
state[0] = i+1
return i
class PipeStage: # for resfuncs in the pipe beyond the first, w/ no state
def _wrap(self, sourcewrap):
return _Wrapper(self, sourcewrap).invoke
class linear_combination(PipeStage):
def __init__(self, a, b): # output = input * a + b
self.a, self.b = a, b
def _work(self, source):
return source()*self.a + self.b
class filter_divby(PipeStage):
def __init__(self, n): # output = elts of input divisible by n
self.n = n
def _work(self, source):
while 1:
next = source()
if next % self.n == 0:
return next
def makepipe(*stages): # make a left-to-right pipeline from resfuncs
pipe = stages[0]._wrap()
for stage in stages[1:]:
pipe = stage._wrap(pipe)
return pipe
if3 = intsfrom(3)
lc31 = linear_combination(3,1)
fdb5 = filter_divby(5)
pipe1 = makepipe(if3, lc31, fdb5)
pipe2 = makepipe(if3, lc31, filter_divby(7))
for i in range(10):
print pipe1(), pipe2()
That prints
10 28
25 49
40 70
55 91
70 112
85 133
100 154
115 175
130 196
145 217
I.e., a pipe stage is just a resfunc with a resfunc argument
(corresponding to "stdin").
Notes:
+ The main pain is the same: faking the preservation of the "program
counter" can be excruciating (not really shown above; see preceding
msg).
+ This scheme produces an "output driven" pipe: no stage produces a
result until a later stage asks for one. So, as the above shows, it's
pleasant for handling unbounded streams.
+ It's even _possible_ to overload "|" so that
pipe1 = if3 | lc31 | fdb5
pipe2 = if3 | lc31 | filter_divby(7)
works. However, it's painful to do that cuz Python is picky about what
it allows as arguments to "|". Specifically, the _wrap methods have to
be changed to return class instances instead of instance method
objects, and that change propagates in icky ways throughout the scheme.
> ...
> ( So we're assigning you chapter 14 of "Python - The BOOK!" :-)
Cool, but I'd rather make that chapter unnecessary <wink>.
> ...
> The possibility of deeper change enters if one considers generalizing
> continuations to provide "restartable" exceptions
Given that the longer I use exceptions, the less places I don't regret
having used them, I've got no interest in making exceptions even more
bizarre <wink>. One advantage of resfuncs over general coroutines is
that resfunc resumptions and suspensions occur in ordinary stack-like
order, so that resfuncs don't necessarily require significant reworking
of the existing exception mechanism; could be that all it would take for
_nice_ semantics is to mark a resfunc as unresumable if an uncaught
exception passed through it while it was active.
> - but the way around this is to NOT change the semantics of exceptions,
> but merely to allow a 'raise' to pass a continuation object as its
> optional arg.
Agreed that if it must be done <wink>, that's the way to do it.
> ...
> One way is to make continuation objects callable, just like functions,
> methods, and class creators:
>
> result, cont = cofunc( args ) # returns a value, continuation tuple
> use( result )
> result, cont = cont() # re-enter
You don't want the 1st call to look different than all the others; any
concrete, specific example would convince you of that instantly <smile>.
I'd suggest this instead, where "resfunc" is a new built-in (whatever)
with the same signature as the built-in "apply":
resfuncobject = resfunc( callable_object, argument_tuple )
In Python pseudo-code, the implementation of resfunc would look something
like:
ResumeError = 'ResumeError'
class _Resfunc:
def __init__(self, args):
if len(args) != 2:
raise TypeError, 'need 2 args'
func, a = args
if not is_callable_object(func):
raise TypeError, '1st arg must be callable object'
if type(a) is not type(()):
raise TypeError, '2nd arg must be tuple'
self.func, self.args = func, a
self.frameobject = build a frame object, wave hands
self.called = 0 # function not yet called
self.resumable = 1
def invoke(self):
if not self.resumable:
raise ResumeError
if self.called:
resume self.frameobject and return its result
else:
self.called = 1
return apply(self.func, self.args) using self.frameobject
# if 'suspend' is introduced, would be nice to have
# return via 'return' set self.resumable = 0
def resfunc(*args):
return _Resfunc(args).invoke
def resfunc_fresh_copy(rf):
me = rf.im_self
if me is not a _Resfunc instance: raise TypeError, etc
return resfunc(me.func, me.args)
This gives an interface that _just_ returns a result, each time
resfuncobject is invoked. That's all I'm after. Of course there's
nothing to stop callable_object from returning a (result, continuation)
tuple if you prefer that instead. 99+% of the time the "continuation"
I'm after will simply be the ability to resume the resfunc exactly where
it left off, and the implementation sketch caters to that case by making
it a no-work-on-the-user's-part case.
Subtlety: Since resfunc resumptions and suspensions _do_ occur in a
stack-like fashion, what to do with the back-pointer in a frameobject
during a resumption is clear (you just set it to the caller's frame-- in
most respects resumption is an ordinary call! --and upon suspension the
back-pointer can be nulled-out (no need to keep a reference to the resumer
active).
Believe that life is more complicated with general co-routines:
A calls B
B calls C
C co-routine transfers to D
D contains a return: where does it go?
"C"'s a wrong answer, because D wasn't _called_ by C. "It returns to B"
is about the best answer I've seen. But then what if B co-routine
transfers to C? C never "returned", so you might think it should still
be active. OTOH, how do you implement that? Throw in some recursive
calls for some real fun <wink>. I believe resfuncs avoid all those
mysteries. E.g.,
A calls B
B calls C
C resumes a resfunc object D
D contains a return: where does it go? Back to C.
I.e. for a resumed resfunc, return/suspend always returns to the point of
resumption.
> [ I'm not yet sure how or whether to fit args to the continuation into
> this scheme.
Think I'd rarely (never?) want that. If necessary, the signature of
_Resfunc.invoke could be changed to invoke(self,*args), and rules made up
to say how invoke overwrites pieces of self.frameobject's locals based on
"args" before resuming self.func.
> Or how exactly to express "capture the current continuation" in a
> return statement.
My hope is that this would be close to a nop. I.e., so long as the
resfunc is alive, a reference to the frameobject is active in the
_Resfunc instance, so the frameobject should persist by magic. In other
words, "return" isn't enough to make the frameobject vanish. Everything
needed should survive by magic, assuming-- as you said earlier --that
frameobjects are extended to preserve enough interpreter state to
_enable_ resumption. But then I'm starting to suspect I'm after
something simpler than you're after ...
> ... vague mumbling about whether the thing that creates a
> continuation ought to be a (pseudo)function or an expression in
> Python goes here ... :-) ]
The sketch above spells "resfunc" as an ordinary function. An
implication is that it can _only_ create a resumable callable_object
object; in contrast, Icon's "create <expr>" accepts an arbitrary
expression, and a magical keyword is essential because <expr> must not be
evaluated at "create" time (else, e.g., "create func(3)" would go ahead
and _call_ func). resfunc worms around that by separating the
callable_object specification from the specification of its argument
list; both of those can be evaluated separately without harm (in fact, I
think it's clearer to evaluate them at the point of resfunc object
creation).
> The type of "wrapper" I was envisioning in the pipe example would be a
> class that takes two functions as args, and rebinds stdin/stdout to it
> self. Its read and write methods encapsulate the coroutine control:
> the called functions do no explicit save/restore of state - they
> merely call obj.read or obj.write. ( Steve waves his hand magically,
> here: ...
There seems to be an assumption here that func #1 never reads & that func
#2 never writes. Is that so? If not, you've got a hairier problem
figuring out whether obj.write is being invoked from a print in func #1
or func #2, and similarly for obj.read. E.g., your disco function both
reads and writes, so after you rebind stdout it may be a mess to separate
disco's output from the library disassembler's output.
One of the "hidden" reasons for executing func #1 to completion before
starting on func #2 is so that these complications don't come up in the
first place <wink>.
Still, assuming func #1 only writes and func #2 only reads,
+ In a language with resfuncs, the natural way to code func #1 is as a
resfunc that produces a sequence of results (via "suspend" or a moral
equivalent), and func #2 as a (ordinary) function that consumes a
sequence of inputs (obtained via repeatedly resuming an "input"
resfunc). Then hooking them together is easy & natural; it's also no
problem if either or both want to "read" and "write". Getting func #2
to read from stdin just requires passing stdin.readline as the
"resfunc" object (func #2 won't care that it's not "really" a resfunc
object, cuz stdin.readline _acts_ like one). Getting func #1 to write
to stdout isn't _that_ simple, but easy enough (it's just hooked to the
obvious "write a result sequence to stdout" function).
+ I do believe things are tougher if you want to spoof read/write: a
resfunc decides when _it_ wants to yield control, but in spoofing
read/write you want something else (namely the read/write proxies) to
decide that its _invoking_ function wants to yield control. So that
notion of resumability spills across frames.
Should have taken you at your word to begin with <wink>: you've convinced
me we're really after different things here. I'm trying to find an
easier way to write some kinds of programs to begin with, and you're
trying to find a way to trick existing programs into doing things their
authors didn't intend. How's that for a fair summary <grin>?
> ...
> Well - I started out trying to solve the "pipe" problem, but then
> discovered (suprisingly!) that the simplest solution is also the
> most general one.
For the kinds of problems I envision solving via the resfunc approach,
messing with tuple returns and explicit continuations would be painful --
although less painful than what I'm doing now. So I think of explicit
continuations as being "simplest and most general" in much the way I
think "go to" is the simplest and most general control structure <0.4
grin>.
> However - what I DON'T want is scheme-like continuation syntax.
Have a brief sketch of that handy?
> ...
> Scheme continuation are extremely powerful, but I think they are rather
> confusing to use. Icon co-routines and co-expressions, on the other
> hand, give a subset of the capabilities of generalized continuations,
> but that subset is quite easy to use and understand, and captures a
> significant portion of those capabilities.
Typical Icon co-expressions are well-behaved, and I've tried to capture
their most useful qualities in the resfunc notion. OTOH, I've found
Icon's _general_ co-routine facilities to be confusing, error-prone in
practice, and-- so far --never actually less work than clearer
alternatives.
So whatever "generalized continuations" means exactly, if it subsumes
general co-routines I bet I could learn to hate it <0.9 grin>.
> Exception handling capture the other sizable subset. I prefer the
> functionality of exceptions and coroutines, without binding them
> to any particular implementation. My use of the word continuation
> in this discussion may lead you to think otherwise, but that's
> just a side effect of the fact that I don't know any better way to
> talk about it.
Well, that's what examples are for (else we're using words without shared
referents). Writing up my examples in excruciating detail has really
helped me understand what I would want here, and I'm surprised to find
that "what I want" (resfuncs) really wouldn't give you a pleasant
solution to your examples. But I'm not sure that what you want would
give you a pleasant solution either! How about fleshing out your general
redirection problem via coding a full solution "as if" Python already had
what you want? I find a lot of things I _don't_ want that way <smile>.
grimy-handed-ly y'rs - tim
Tim Peters tim@ksr.com
not speaking for Kendall Square Research Corp