Full duplex pipes ( and imaplib.py )

Steven D. Majewski (sdm7g@elvis.med.virginia.edu)
Tue, 28 Jun 1994 12:19:32 -0400

The imapd server which I'm trying to talk to with my python imap
module can be run as a server, listening on sockets for requests,
(in which case on has to login or authenticate the session in some way)
or it can be run from a logged in shell ( as a pre-authenticated
server ) where it reads and writes to stdin/stdout.

The latter seems to be the obvious way to run it when I'm trying to
create a Read Only gateway to a public archive ( this gateway will not
offer raw access to the imap protocol, so it will be impossible to
delete a message, and it will be run from a WWW-cgi script, which will
have already established a context for the server. )

Which brings up the question of full-duplex pipes again ( Which has
been discussed previously, but perhaps, inconclusively. ) and how to
fork() another python process to read and write to/from.

[ One point of view of the discussion was to just stay away from
full duplex pipes because there were too may possible deadlock
problems if one did it the simple way, and doing it the "right"
way might not fit into a general purpose interface. However, in
this instance, the protocol provides a pretty good handshaking
so that select() or timeouts are not necessary. I can tell
unambiguously when I have gotten the last line from a request,
and as long as I don't try to read unless there is a request
pending, I don't think I can get into trouble. I thought that
tim or someone had once posted an explicit example of "how-to-do-
it-if-you-want-to-ignore-my-advice-and-go-ahead-anyway", but I
couldn't dredge it up from my archive! ]

The first time or two I tried to do something like this, it didn't work
too well - but I was just playing around at the time. Now that I
really *needed* that capability, I actually stopped to think about what
I was doing first, and came up with something that works.
[ Properly redirecting stdin and stdout was the problem! ]

I won't post the whole sub-class definition at present. I just want to
show you how I did this, and see if anyone has any better methods or
suggestions:

class LocalSession( ImapSession ):
def __init__( self ):
r, self.W = posix.pipe() # parent-write / child-read pipe
self.R, w = posix.pipe() # parent-read / child-write pipe
pid = posix.fork()
if not pid: # This is the child:
posix.dup2( w, sys.stdout.fileno() ) # redirect stdout and
posix.dup2( r, sys.stdin.fileno() ) # stdin to/from pipes
posix.execv( IMAPD, () ) # and exec program
self.file = open( '/dev/null', 'r' ) # this is a dummy file
posix.dup2( self.R, self.file.fileno() ) # to coerce the ...
print self.file.readline() # posix-file into a python-file obj
...

The child reads and writes on sys.stdin and sys.stdout.
The parent process instance reads on self.R and writes on self.W.

I hadn't realized that duping the posix file descriptor out from
under the python file object was the cheap way to turn posix files
into python files. This is necessary to redirect io in the child
process. It is also done in the parent process for the read-pipe,
so that readline() and other python file methods can be used, which
keeps to a minimum the methods that have to be changed for this
sub-class. I didn't bother to do the same for the write pipe: if
(python-)write is used in place of posix.write, then an explicit
file.flush() call is required to avoid deadlock, so I just used
posix.write in place of socket.send in the write/send methods.

- Steve Majewski (804-982-0831) <sdm7g@Virginia.EDU>
- UVA Department of Molecular Physiology and Biological Physics