Re: coroutines and continuations ( in Python? - long discussion )

Steven D. Majewski (sdm7g@elvis.med.virginia.edu)
Sat, 30 Apr 1994 15:19:31 -0400

Thanks for that long and thoughtful reply, Tim.

I've used the same sort of techniques, but you've clearly thought
it out into a more systematic method.
( So we're assigning you chapter 14 of "Python - The BOOK!" :-)

Skipping over the current usage example to proposed mods discussion,
On Apr 30, 1:29, Tim Peters wrote:
>
> So what would I like? I'm not sure. Two things I don't want to see:
>
> 1) Changing Python at a deep level to support coroutines/closures.
>

[ With due caveat's that I haven't worked out the implementation
and there still may be a fatal flaw in my proposed scheme... ]

Well - one of the things that excited me was that it looked like
a deep change is not necessary. Minimal change to the language,
addition of some fields to frame-objects or creation of a new
continuation object with ref to a frame-object, one more byte
code to create one of these objects, and a way to enter/resume one.

I see minimal or no impact to the rest of the language or to
programs that don't use these features. ( the pyc file magic
will have to be changed, but since it is a downwardly compatible
change, there is no reason to reject the previous compiled code.
Old compliers can and should reject newer .pyc files that may
use this feature. )

The possibility of deeper change enters if one considers
generalizing continuations to provide "restartable"
exceptions - but the way around this is to NOT change
the semantics of exceptions, but merely to allow a 'raise'
to pass a continuation object as its optional arg.

> 2) Any scheme that shares with Icon that the _resumption_ of such beasts
> requires special syntax. The technique above makes a special case out
> of _creating_ a "resumable function" object, but the resumptions
> themselves look and act like ordinary function calls. That has
> practical value, in limiting the visibility of implementation
> decisions.

One way is to make continuation objects callable, just like functions,
methods, and class creators:

result, cont = cofunc( args ) # returns a value, continuation tuple
use( result )
result, cont = cont() # re-enter

[ I'm not yet sure how or whether to fit args to the continuation into
this scheme. Or how exactly to express "capture the current
continuation" in a return statement. ... vague mumbling about
whether the thing that creates a continuation ought to be a
(pseudo)function or an expression in Python goes here ... :-) ]

> OTOH, I'd very much _like_ some language help for this style of
> programming, provided it can be done with little effect on the language
> as a whole. E.g., I have no problem at all with having to explicitly
> "wrap" a function that I want to use in coroutine fashion, and that makes
> me suspect almost all of it could be achieved via a new "wrapped
> function" object that doesn't affect the visible semantics of existing
> objects, or the implementation of most existing objects.

The type of "wrapper" I was envisioning in the pipe example would be a
class that takes two functions as args, and rebinds stdin/stdout to it
self. Its read and write methods encapsulate the coroutine control:
the called functions do no explicit save/restore of state - they
merely call obj.read or obj.write. ( Steve waves his hand magically,
here: I worked this all out BEFORE I looked at a possible implementation -
I'll go back and see if I can translate my squiggly flow-of-control drawing
into terms of my proposed solution, and see if it *STILL* makes any sense!
Once I convinced myself that coroutines were the sufficient and
necessary missing element for a solution, I stopped thinking about the
problem and started thinking about coroutines. ;-)

> BTW, the primary difference between "suspend" and "return" in Icon is
> that while both return a result, "suspend" explicitly says that the
> function can-- and "return" that it cannot --be resumed again. This is
> nice for self-documentation and error-catching purposes, but isn't
> essential.

So, what I see is no special syntax to resume a continuation, and no
built-in wrapper is required; the "special" syntax is in the return.
Maybe "suspend value" returning a value,continuation tuple is better
than "return value, current_continuation()" , in avoiding my problem
of expressing that current_continuation() really ought to refer to
the next-line after return, not the instruction immediately AFTER
the capture-continuation code ( which is going to be 'build-tuple' &
'return' )

> Warning: What I called "resumable functions" above can be implemented via
> coroutines, but aren't as general as the latter, as I understand them.
> And if _any_ discussion needs complete examples to illustrate the
> intended concepts, it's this one, because the same phrases are used by
> many people with different meanings in mind.
>
> One difference: a "resumable function" is all about preserving local
> execution state across resumptins, and nothing about arbitrary (goto-
> like) transfer of control. General coroutines are, I believe, about
> both.

Well - I started out trying to solve the "pipe" problem, but then
discovered (suprisingly!) that the simplest solution is also the
most general one.

However - what I DON'T want is scheme-like continuation syntax.
Python is ( and should continue to be ) and easy to use language.
Scheme continuation are extremely powerful, but I think they are
rather confusing to use. Icon co-routines and co-expressions, on
the other hand, give a subset of the capabilities of generalized
continuations, but that subset is quite easy to use and understand,
and captures a significant portion of those capabilities.
Exception handling capture the other sizable subset. I prefer the
functionality of exceptions and coroutines, without binding them
to any particular implementation. My use of the word continuation
in this discussion may lead you to think otherwise, but that's
just a side effect of the fact that I don't know any better way to
talk about it.

Despite scurrilous rumors to the contrary, I DO NOT want to change
Python into Scheme! :-)

[ And why I'm trying to steer clear of fork-like magic functions
that seem to return different things at different instances. ]

- Steve Majewski (804-982-0831) <sdm7g@Virginia.EDU>
- UVA Department of Molecular Physiology and Biological Physics