newest redirect.py - module to redirect input & output of python functions

Steven D. Majewski (sdm7g@elvis.med.Virginia.EDU)
Tue, 29 Mar 1994 16:24:59 GMT

In an attempt to raise the comp.lang.python signal above the
alt.fan.monty-python noise, I'll post the latest version of
my redirect.py module, with two additional classes - Tee & Echo.
[ Also, because the example shows off some of Python's more
advanced tricks. ]

For those new to (comp.lang.)Python, it is an object-oriented
language, somewhat inspried my Modula-3, and like that language
in that it is object-based, but not Class based, with respect to
low level built-in objects. Thus character-strings, lists, tuples,
dictionaries, and files - all built-in objects to the interpreter,
cannot be the base class for a user written class. However, you
can define an object that shadows the methods of a built-in object.
( Some would call this "Inheriting the interface" but there is
no syntax in Python to do this automatically, and inheritance of
interface is a problematical concept, anyway. [*flame-bait*] )
This create a object that is the same Abstract Data Type as a
built-in object, and that user-defined class can then be inherited
from by other user defined object classes.

The following module defines three functions that redirect the
output from a python function. 'tofile( file, func, *args )'
redirects the output of function( *args ) to file. 'tostring'
and 'tolines' return the output as a string ( each line separated
by newline chars. ) or a list of line-strings ( saving you the
work of separating them. )

This module was written so that I could capure the printed output
of another python function, which printed it's output rather that
returning it to the caller. I wanted to further process the
output before printing it. This gives Python the equivalent of
the shell's pipe or backquote - any print statement or write to
sys.stdout can be captured and processed.

The tostring and tolines functions are really calls to tofile,
using a StringFile object. StringFile is a class that mimics the
interface of a File object, but whatever is written to it is
appended to a string. ( And readling this pseudo-file object
gets it back. )

The two additional classed are Tee and Echo.
Tee( file1, file2 [, ... filen ] ) returns a file object that
writes it's output to all of the files.
Echo( filein, fileout ) returns an object that echo's whatever
is read from filein to fileout.
These objects can be assigned to sys.stdin, sys.stdout, & sys.stderr,
and they can be combined:
sys.stdin = Echo( sys.stdin, Tee( sys.stdout, filename ) )
I had hoped I could create a 'dribble()' function that log's all
of the interpreter's input and output to a file, however the
interpreter's input comes via a different mechanism. Tee-ing
sys.stdout and sys.stderr does, however, work as expected.

A kludge that I'm not entirely happy with is that the returned
value of the function is also printed/appended to the output.
tofile() returns the VALUE of it's function, while redirecting
the printed output. Since tostring/tolines returns the output -
appending the value seemed the least awkward way to make sure
you could easily retrieve the value. You can get the value
by eval-ing the last line:
lines = tolines( f, x,y,z ); retval = eval( lines[-1] )
For tostring/tolines, the returned value is also printed on
stdout, unless it is 'None' ( but even None is appended, so that
you can always eval the last line. )

The test function at the end does not test the new additions
of Tee and Echo. Try: 'sys.stdout = Tee( sys.stdout, "out.tmp" )'

- Steve Majewski (804-982-0831) <sdm7g@Virginia.EDU>
- UVA Department of Molecular Physiology and Biological Physics

#!/usr/local/bin/python
#
# <module 'redirect'>
#
# - Steven D. Majewski <sdm7g@Virginia.EDU>
#
# Functions:
# tofile( file, func, *args )
# tostring( func, *args ) ==> string
# tolines( func, *args ) ==> [ line0, line1, ... lineN ]
#
# Functions apply a function to args, either redirecting the output
# or returning it as a string or a readlines() like list of lines.
#
# tofile will print (to the file) and return the value returned by
# apply( func, *args ). The value is also in the last string in
# tolines ( or the last line in tostring ). tolines and tostring,
# will print the value on the original sys.stdout as well (unless
# it's == None ).
#
#
# Class StringFile()
# Methods: [ all of the typical file object methods ]
# read(),write(),readline(),readlines(),writelines(),
# seek(),tell(),flush(),close(),isatty() [NO fileno()]
#
# Creates a file-like interface to a character array.
# Write's append to the array; Read's return the characters in the array.
#
# Class Tee( file1, file2 [, ... filen ] )
# create a fileobject that writes it's output to all of the files.
# Class Echo( filein, fileout )
# create a fileobject that automatically echo's whatever is read
# from filein to fileout.
#
# An instance of a Tee object can be assigned to sys.stdout and sys.stderr,
# and all Python output will be 'tee'-ed to that file.
# Unfortunately, 'Echo'-ing stdin does not reproduce YOUR typed input to
# the interpreter, whose input comes via a different mechanism.
# Implementing a 'dribble()' function, that logs all input and output to
# a file will require another trick.
#
#
#
# 'tofile()' temporarily reassigns sys.stdout while doing func.
# 'tostring()' and 'tolines()' both call 'tofile()' with an instance
# of StringFile().
#
#
# tofile( '|lpr', func, output )
#
import sys
import os

def filew( file ):
# file is a filename, a pipe-command, a fileno(), or a file object
# returns file.
if not hasattr( file, 'write' ) :
if file[0] == '|' : file = os.popen( file[1:], 'w' )
else: file = open( file, 'w' )
return file

def filer( file ):
# file is a filename, a pipe-command, or a file object
# returns file.
if not hasattr( file, 'read' ) :
if file[-1] == '|' : file = os.popen( file[1:], 'r' )
else: file = open( file, 'r' )
return file

def tofile( file, func, *args ):
# apply func( args ), temporarily redirecting stdout to file.
# file can be a file or any writable object, or a filename string.
# a "|cmd" string will pipe output to cmd.
# Returns value of apply( func, *args )
ret = None
file = filew( file )
sys.stdout, file = file, sys.stdout
try:
ret = apply( func, args )
finally:
print ret
sys.stdout, file = file, sys.stdout
return ret

def tostring( func, *args ):
# apply func( *args ) with stdout redirected to return string.
string = StringFile()
apply( tofile, ( string, func ) + args )
return string.read()

def tolines( func, *args ):
# apply func( *args ), returning a list of redirected stdout lines.
string = StringFile()
apply( tofile, ( string, func ) + args )
return string.readlines()

from array import array

# A class that mimics a r/w file.
# strings written to the file are stored in a character array.
# a read reads back what has been written.
# Note that the buffer pointer for read is independent of write,
# which ALWAYS appends to the end of buffer.
# Not exactly the same as file semantics, but it happens to be
# what we want!
# Some methods are no-ops, or otherwise not bery useful, but
# are included anyway: close(), fileno(), flush(),

class StringFile:
def __init__( self ):
self._buf = array( 'c' )
self._bp = 0
def close(self):
return self
# On second thought, I think it better to leave this out
# to cause an exception, rather than letting someone try
# posix.write( None, string )
# def fileno(self):
# return None
def flush(self):
pass
def isatty(self):
return 0
def read(self, *howmuch ):
buf = self._buf.tostring()[self._bp:]
if howmuch:
howmuch = howmuch[0]
else:
howmuch = len( buf )
ret = buf[:howmuch]
self._bp = self._bp + len(ret)
return ret
def readline(self):
line = ''
for c in self._buf.tostring()[self._bp:] :
line = line + c
self._bp = self._bp + 1
if c == '\n' : return line
def readlines(self):
lines = []
while 'True' :
lines.append( self.readline() )
if not lines[-1] : return lines[:-1]
def seek(self, where, how ):
if how == 0 :
self._bp = where
elif how == 1 :
self._bp = self._bp + where
elif how == 2 :
self._bp = len(self._buf.tostring()) + where
def tell(self):
return self._bp
def write(self, what ):
self._buf.fromstring( what )
def writelines( self, lines ):
for eachl in lines:
self._buf.fromstring( eachl )

class Tee:
# Tee( file1, file2 [, filen ] )
# creates a writable fileobject where the output is tee-ed to all of
# the individual files.
def __init__( self, *optargs ):
self._files = []
for arg in optargs:
self.addfile( arg )
def addfile( self, file ):
self._files.append( filew( file ) )
def remfile( self, file ):
file.flush()
self._files.remove( file )
def files( self ):
return self._files
def write( self, what ):
for eachfile in self._files:
eachfile.write( what )
def writelines( self, lines ):
for eachline in lines: self.write( eachline )
def flush( self ):
for eachfile in self._files:
eachfile.flush()
def close( self ):
for eachfile in self._files:
self.remfile( eachfile ) # Don't CLOSE the real files.
def CLOSE( self ):
for eachfile in self._files:
self.remfile( eachfile )
self.eachfile.close()
def isatty( self ):
return 0

class Echo:
def __init__( self, input, *output ):
self._infile = filer( input )
if output : self._output = filew(output[0])
else: self._output = None
def read( self, *howmuch ):
stuff = apply( self._infile.read, howmuch )
if output: self._output.write( stuff )
return stuff
def readline( self ):
line = self._infile.readline()
self._output.write( line )
return line
def readlines( self ):
out = []
while 1:
out.append( self.readline() )
if not out[-1]: return out[:-1]
def flush( self ):
self._output.flush()
def seek( self, where, how ):
self._infile.seek( where, how )
def tell( self ): return self._infile.tell()
def isatty( self ) : return self._infile.isatty()
def close( self ) :
self._infile.close()
self._output.close()


if __name__ == '__main__':
def testf( n ):
for i in range( n ):
print '._.'*10 + '[', '%03d' % i, ']' + 10*'._.'
if hasattr( os, 'popen' ):
tofile( '|more', testf, 300 )
print '\n# Last 10 lines "printed" by testf(): '
print '# (the Python equivalent of \'tail\'.)'
for line in tolines( testf, 300 )[-10:] :
print line[:-1]