> """prob.py - succinct example demonstrating pipe IO select problem.
> ken.manheimer@nist.gov, 10-Jan-1995."""
>
> import sys, os, select
>
> rf, wf = os.pipe()
> rfp = os.fdopen(rf, 'r'); wfp = os.fdopen(wf, 'w')
>
> wfp.write('the\n'); wfp.write('spanish\n'); wfp.write('inquisition\n')
> wfp.flush()
> print 'Pipe primed with three lines written.'
>
> for i in range(3):
> print 'Select on read handle:', select.select([rfp], [],[], 0)[0]
> print '... and read yields:', rfp.readline()[:-1]
>
> What i believe *ought* to be happening in the loop is that the select
> should be returning the read file-descriptor every time through, but
> it only does so before the first read. All the reads, even those
> following empty select returns, each yield one of the three lines with
> which the pipe was primed.
Sjoerd Mullender:
> This is not what *ought* to happen. Select works on file descriptors
> and returns any descriptors that have unread data. The readline
> method uses stdio for its implementation, and stdio buffers. So after
> the first readline all data has been read from the pipe and is stored
> in an internal buffer from which the second and third line are
> returned in subsequent calls. So there is nothing left in the pipe
> after the first readline, and thus select does not return the an
> indication that there is something.
Indeed. The problem is that stdio has no (portable, documented) way
of checking whether there's anything buffered, and the select() system
call (which operates on file descriptors) doesn't know there's data in
the stdio buffer.
The solution is to declare the file unbuffered. Unfortunately that
can't be done for fdopen() and popen() in Python 1.1.1. I will add
it to 1.2, using the same convention as for the built-in open()
function (an optional buffer size argument after the mode argument).
In 1.1.1, a work-around is not to use fdopen but to read a byte at a
time directly from the file descriptor using os.read(1).
--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>