Timing of Python constructs

Jim Roskind (jar@infoseek.com)
Sat, 2 Apr 1994 15:16:01 +0800

> Cc: python-list@cwi.nl
> Date: Sat, 02 Apr 94 15:07:59 EST
> From: Tim Peters <tim@ksr.com>
> Subject: Re: Internal Objects: method vs function/procedure vs both
>
> > [lots of seemingly general agreement]
>
> > [a nice timing skeleton, plus tricks of the timing trade, deleted]
>
> If people want to try this (go ahead -- it's fun <smile>!), add
>
> import os
> ostimes = os.times

Ooops... I'm ashamed that I left that out. It was part of the general
file that I use to do benchmark timings. It is significant that if
you use this assignment, then you should either have ostimes defined
as a local variable, or else you should use the global statement.
To be specific, do either:

import os
ostimes = os.times

def typical_test(n):
global ostimes
...
t = ostimes()
...

or you should have it locally imported:

def typical_test(n):
import os
ostimes = os.times
...
t = ostimes()
...

This is one of many tricks that I noted as I developed my python
profiler. Note that without this, you add unnecessary noise to the
measurements you are making (though you can improve the S/N ration by
upping the loop count sufficiently). This comment also identifies an
interesting optimization, when you try to understand why it runs
faster (at least I thought it was interesting). To make sure my point
is clear, the following two approach are progressively worse (when
doing timings):

import os
ostimes = os.times

def typical_test(n):
...
t = ostimes()
...

Note that the search for the identifier "ostimes" must be made in two
dynamic scopes before the function can be called. Worse yet is:

import os

def typical_test(n):
...
t = os.times()
...

Here not only does "os" have to be looked up in two dicts, its member
"times" must be found in a dict before the function call can be made.

This sort of optimizations are not suggested for general code, but
when time is of the essence (such as when benchmarking times, or in a
critical inner function as identified by a working profiler) the above
two factoids can be very helpful.

> [discussion of timing examples]
>
> OTOH hand, if you want to "prove" that local assignments are horribly
> expensive <wink>, throw the calls out of the timing loops entirely,
> replacing the loops with
>
> while n > 0:
> temp = n - 1
> n = temp
>
> and
>
> while n > 0:
> n = n - 1

Actually, when running tests, you should be very careful to isolate
the essence of what you are looking for. I was careful to use
identical lengths in my function names, and short local variable names
(to prevent such issues from having an impact on the results). In
addition, your test has unwittingly contaminated the local dictionary
that is used in the second while loop with the addition of another
local variable to its context (the "temp" from the first loop).
Designing tests is not easy, and shortcuts often invalidate the focus
of the tests. The tests I presented in my example are much more
focused.

IF I wanted to *really* worry about such issues, I'd even have several
functions with different names to call, just to make sure that a
hashing collision in the function name lookup was not biasing the
results.

Jim

Jim Roskind
408-982-4469
jar@infoseek.com