Lazy lists for use with large dictionaries

Anton Hartl (hartl@Informatik.TU-Muenchen.DE)
27 Sep 1994 18:21:59 GMT

I ran into the following problem: I have a large dictionary (some 1000 or
even 10000 entries) which is created by reading a file. I've defined a
class wrapper around that dictionary that supports the following operations

open(file) open the file and setup everything to support the
following three operations
next() advance cursor to the next item
item() return the current item
rewind() restart from the beginning

When open(...) has been called you can enumerate all items of the object
one by one in some order which is of no interest to me. If next()
returns 0 it indicates that there are no more items, now you can call
rewind() and start all over again.

But once the file has been read (i.e. after the very first enumeration
phase) there is already a dictionary with all items. How shall further
scans of the dictionary be implemented?

The obvious way (and the only one I'm currently aware of) is to have
an integer variable ranging from 0 to len(dict.keys()), set by rewind() to
0 and incremented by next() but that has the effect that the
whole list of keys is actually computed. I haven't looked into the code
but I doubt that dict.keys() returns such a thing like a lazy list, i.e.
the equivalent to the sequence object returned by xrange.

Perhaps all this isn't a problem because the list code is fast enough and
memory isn't an issue anymore ;), or is it?

-Toni