optimizations ( was: invertible id() )

Steven D. Majewski (sdm7g@elvis.med.virginia.edu)
Wed, 6 Apr 1994 14:12:53 -0400

Looking at output from the byte-code disassembler, one lesson is that
( if you want optimized code ) you have to forget habits acquired from
smart optimizing C compilers, and code intermediate references into
the source. Every "dot" in you inner loops is another lookup.
So, for a big list, it pays to do:
a = [];
la = a.append
for item in BIGLIST: la( item )
The problem with making the Python compiler optimization smarter, is
that Python is so dynamic, there is no guarantee that a.append is
the same thing each time thru the loop. That case may be odd, but it's
not disallowed/impossible or even difficult ( although I'm hard pressed
to think up a useful class that changes it's methods on the fly. )

That's one reason I was considering a POST-optimizer. Just as source
coding the optimizations yourself is adding the semantic information
that you PROMISE that something that tricky is NOT going to happen,
a post-compiler code optimizer can be optionally applied - which is
again, to be taken as a promise that nothing TOO dynamic is going on.
( I'm still not certain that even than is a good idea, but it's
worth experimenting. )

One thing that CAN'T be source optimized is :
thing[index] = thing[index] + somthing
or thing.attrib = thing.attrib + something
because you want a reference on the LHS and a value on the RHS,
and there is not way to express that reference in Python.

I took this as an argument in favor of adding C-like assignment
operators ( +=, *=, etc. ) to Python.

| dict[k] = dict[k] + value # 18

49 SET_LINENO 18
52 LOAD_FAST 0
55 LOAD_FAST 2
58 BINARY_SUBSCR
59 LOAD_FAST 1
62 BINARY_ADD
63 LOAD_FAST 0
66 LOAD_FAST 2
69 STORE_SUBSCR

If the byte code ( & compiler & ceval ) were changed so that BINARY_SUBSCR
left a reference on the stack for a STORE|FETCH command to use, it could be
compiled into something like:
LOAD_FAST 0
LOAD_FAST 2
BINARY_SUBSRC
DUP
FETCH
LOAD_FAST 1
BINARY_ADD
STORE

Except that some additional semantic input is needed to assert that
that both references are, in fact, identical. ( Thus the need for
"+=", et.al. )

Anyway: THAT's the argument I *would* have made a while ago, if I
hadn't come around to NOW thinking that the big win just might be
in *improving* lookup, functions and classes, rather that avoiding
them.

My thoughts are still in flux on this matter, but I bring it all
up (perhaps prematurely - certainly before doing any measuring.)
because Jim's remarks:

On Apr 4, 23:41, Jim Roskind wrote:
>
> Note that the standard "id()" function can't really be inverted,
> because it does *not* bump the refcount on the object it processes.
> As a result, the object can "go away," while the id number persists
> (the ref manual hints that id() is really the address of the object.)
> I could write an "unsafe" id inverter in C, but I'd be guilt ridden
> and unable to sleep for hours ;-). I think IF I had an invertible id
> function, it would really produce a "integer like" object that
> effectively included a reference to the "real" class, with the
> ref-count bumped accordingly. The critical functionality is that it
> would compare properly under equality testing, and hence could be used
> as an index into an associative array :-).
>
> Suggestions??
>
>

could suggest that the Other lesson that could be drawn from the same
facts is that Python ought to have real pointer/references. At first
this seems a totally low-level and non-Pythonish concept, but folks
have been doing work with "smart-pointers" in C++ -- which would
be the sort or pointers that Python would require - reference counting
pointers that keep the object alive while there are existing
references.

I'm not exactly sure how I feel about either of these ideas,
( I'm still feeling chastened by Tim from the last Change-To-Python
I ventured, so I would have kept quite if this topic hadn't *already*
erupted. )

- Steve Majewski (804-982-0831) <sdm7g@Virginia.EDU>
- UVA Department of Molecular Physiology and Biological Physics