Pipes or Socket Help

Siebren van der Zee (siebren@vdzee.xs4all.nl)
Sun, 19 Mar 1995 17:52:03 +0100 (MET)

rwb3y@virginia.edu (Robert Bingler) types:
> Hey,
> Can someone lead me to a good example or explain some things
> about using either sockets or pipes to communicate with forked processes?
> I want to be able to read data from several forked processes when they
> send info, but not wait for each individual process. I know select() is
> what I'm looking for, but I couldn't figure out how to get it to work with
> sockets.

See below.

> On the other hand, I couldn't figure out how to create a pipe
> with a forked process either. The pipe() function is to is supposed to
> return a pair of file descriptors, but it only returns a pair of integers.
> How should this work?

Those integers are filehandles, in Unix terminology, they are not file
objects in python terminology. The first element, os.pipe()[0], is the
reading end of the pipe, the second one, os.pipe()[1], is the one you
write to. The 'os' module knows how to deal with filehandles. You can
turn them into a regular python file object using os.fdopen(handle).

Your question is probably basically a comp.os.unix question, but since
you're asking on comp.lang.python, and since you should have been asleep
when you made the original posting, here is the answer:

#!/usr/local/bin/python
#
# Script to demonstrate use of pipes and select to do asynchronous stuff

import os, select, sys, time
HUGE = 10000 # That's a lot, isn't it? 5 Kb will do on most Unixes

def listexcl(l, i):
'Return a copy of list "l", excluding all items "i"'

new = []
for x in l:
if x <> i:
new.append(x)
return new

# Keep a list of forked children:
Kids = []

def forkit(n):
'''
Fork a process that prints a line each n seconds,
return a filehandle for its output to the parent process
'''
global Kids

cmd = 'for i in range(' + str(20 / n) + '):\n\t' + \
'time.sleep(' + str(n) + ')\n\t' + \
'print "I always sleep for ' + str(n) + ' seconds"\n\t' + \
'sys.stdout.flush()\n' # Pipes are normally buffered by stdio

rd, wr = os.pipe() # Create the pipe
pid = os.fork() # Create a new process
if pid: # Parent code
os.close(wr) # The parent won't write the pipe
Kids.append(pid) # Remember the process-id
print 'Starting', rd, ':', cmd
return rd
else: # Child code
os.close(rd) # The child won't read the pipe
# Make sure the child won't read input intended for the parent:
null = open('/dev/null')
os.dup2(null.fileno(), sys.stdin.fileno())
null.close()
# Redirect standard output to the pipe:
os.dup2(wr, sys.stdout.fileno())
os.close(wr) # Close original write-end of the pipe
# You could also use os.system(cmd) or os.exec*() here:
exec(cmd)
sys.exit(0) # Terminate the child process

def loop(fdlist):
'''
Print any output coming from the filedescriptors in the fdlist,
return when no more filedescriptors are valid
'''

while fdlist <> []:
(readable, writeable, oob) = select.select(fdlist, [], [])
# writeable and oob will always be '[]' here, since we're
# not waiting for them

# This may not happen, unless you uncomment the sleep at
# the end of the while loop:
if len(readable) > 1: print 'readable:', readable

# Handle all filedescriptors on which input is awaiting:
for f in readable:
msg = os.read(f, HUGE) # On sockets, you'd "msg = f.recv()"
if msg <> '':
if msg[-1:] == '\n': # This is always the case
msg = msg[:-1] # Strip the newline
print '%d: %s' % (f, msg)
else: # Read zero bytes => child terminated
print '%d: terminated' % (f)
os.close(f)
fdlist = listexcl(fdlist, f)
#time.sleep(1.5) # Uncomment this to simulate doing real work

print 'No more filedescriptors to wait for'

loop([forkit(1), forkit(2), forkit(3), forkit(4)])

# Never forget to pay attention to your kids,
# or The Unix Kernel won't let you create new ones.
# You wouldn't want to let that happen to you ;-)
for kid in Kids:
x = os.waitpid(kid, 0)[1]
if x <> 0: print str(kid) + ': exit', x

> Thanks,
> Rob

You're welcome

> --
> ------------------------------------------------------------------------------
> Robert Bingler rwb3y@virginia.edu http://uvacs.cs.virginia.edu/~rwb3y/
> LINUX - That way cool program by that finnish dude
> ------------------------------------------------------------------------------

Siebren van der Zee, siebren@xirion.nl, siebren@xs4all.nl