Re: Binary I/O

Guido.van.Rossum@cwi.nl
Mon, 15 Aug 1994 10:40:49 +0200

Rich Neitzel wrote:
> > Is there any way to read/write integers, doubles, etc. except as strings?
> > I want to use python to control a realtime system via sockets. The
> > overhead of converting to/from strings is more then I want to inflict on
> > the realtime system.

and I answered:
> Yes, the array module can read most numeric formats from strings
> (where they are expressed in machine format) or files. It can also
> write them back. An alternative would be to use the struct module,
> which can also convert between strings containing machine format
> numbers and Python values.

I forgot that there's a third way of passing Python values efficiently
via files, strings or sockets: the 'marshal' module. It does actually
pass floating point numbers as decimal ASCII strings, but does so
without the Python overhead of calling repr() and atof() so should be
blindingly fast compared to a Python version. The interface is also
somewhat simpler, since it is intended to read/write Python values
to/from files (and strings).

A simple example which writes a list of numbers (l) onto a socket (s):

buf = marshal.dumps(l)
s.send(buf)

or, if you've bound a stdio file to the socket using
"f=s.makefile('w')":

marshal.dump(l, f)

To read it back, use

buf = s.recv(<buffercount>)
l = marshal.loads(buf)

or

l = marshal.load(f)

(Note that in the first case you have to know how many bytes the list
occupies in the buffer, unless it's the last thing in the buffer.)

--Guido van Rossum, CWI, Amsterdam <Guido.van.Rossum@cwi.nl>
<URL:http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>