Impressive! Mind if I stuff your archive in the Demo directory of the next
distribution?
> RemoteCall has a method called Exec, which allows arbitrary Python
> code to be sent to the server. Probably not a very good idea if you
> are the slightest nervous about security issues ;^).
Actually, since you liked 1.2-beta's pickle.py, you should also have a
look at rexec.py in that distribution. This isn't quite as finished
as pickle.py (and won't be officially documented) but it shows how 1.2
has the beginnings of support for secure / restricted execution.
> When sending a tuple over the wire, there is quite a lot of copying
> behind the scenes on the server side. When data is comming in, it is
> scanned for agents to be substituted. Since tuples are immutable,
> they are always copied in the process regardless of whether they
> contain agents or not. The same is true when returning results. I
> see no easy way out here.
I presume you are referring to the following code fragment in
MakeAgents, and similar code in ReplaceAgents:
> elif resultType == TupleType:
> new_tuple = ()
> for ix in range(len(result)):
> new_tuple = new_tuple + (self.MakeAgents(result[ix]),)
> result = new_tuple
I see a couple of inefficiencies here, and I'll treat them one at a
time. First, you are building a tuple by appending an element at a
time using the '+' operator. This is a quadratic algorithm, as you
have noticed. You can start by building a list and turning it into a
tuple using the tuple() built-in function:
new_list = []
for ix in range(len(result)):
new_list.append(self.MakeAgents(result[ix]))
result = tuple(new_list)
Next, you're iterating over the "index set" of the tuple, while in
fact you could just as well iterate over its "element set":
new_list = []
for x in result:
new_list.append(self.MakeAgents(x))
result = tuple(new_list)
This saves building a list with range(len(result)), which, although
fast, isn't free.
Then, for this particular idiom, you can use the built-in function
map() to avoid writing the loop out at all:
new_list = map(self.MakeAgents, result)
result = tuple(new_list)
or, of course,
result = tuple(map(self.MakeAgents, result))
Finally, but you may have to experiment to figure out whether this
actually saves something or not, you may prefer to return the original
tuple if nothing changed:
new_tuple = tuple(map(self.MakeAgents, result))
if new_tuple != result:
result = new_tuple
This saves storage if no agent replacements were actually made (by
throwing away identical copies), but it incurs some overhead because
it has to compare the tuples -- this is very fast, however, because
the items will either be the same object, in which case the comparison
is a single pointer comparison, or they are an object and the agent
that represents it, in which case the comparison finds that the types
don't match and quickly decides the items are unequal.
BTW I noticed that your code patches lists and dictionaries in-place.
While this is usually fine, it means that if you pass a reference to
an object that is part of your own data structures, your own version
is patched! I'm not sure whether this is what you want (or whether it
isn't a problem because of other reasons).
--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>