Re: How to do a controlled exec compiled and dynamic-load files?

Guido.van.Rossum@cwi.nl
Wed, 25 Jan 1995 23:38:09 +0100

> I need to be able to control loads of compiled and dynamic-load (as
> well as normal source) files, in order to implement an extension to
> the python import statement. Is there a way to do so that i'm
> missing?
>
> In particular, i need to load the files with respect to specific
> environment bindings, which i do not seem to be able to control using
> the new 'imp' module's '.load_compiled()' and '.load_dynamic()'. Is
> there any way from within python to read in a compiled or dynamic-load
> file and execute it, wrt a particular binding environment?

OK, I'll byte. What do you mean by "environment bindings" and
"binding environment"? This is not a technical term I recall using in
Python.

Reading your reply to Steve's reply, I see some profound confusion as
well about what kind of data the various types of files mean, and how
they should be treated, and what "exec" or "execfile" do.

A .py file contains Python source code. It contains lines separated
using the local newline convention. It could be read by execfile, and
if you read its contents into a string with f.read(), exec will also
like it (as long as it's valid Python). (On non-UNIX systems, the
local newline convention is automatically translated into '\n' line
separators, as long as the file is opened in text mode -- the mode
should be 'r', not 'rb'.)

A .pyc file contains binary data. (On non-UNIX systems, these files
should always be read using open mode 'rb'.) The format is simple: 4
bytes magic number (changes whenever the interpreter's instruction set
or the marshalling of code objects changes), 4 byte timestamp of the
corresponding .py file, and a "code object" which should be read using
marshal.load(f). The byte order of the time stamp (as well as the
data written by the marshal module) is little-endian. This makes .pyc
files machine independent. To execute the code in the .pyc file,
you skip (better: check!) the first 8 bytes, then use marshal.load(f)
to get a code object, and pass the code object to exec. The exec
command magically executes code object.

.pyc files are created automatically by importing the corresponding
module, if the file system allows the importing process to write the
.pyc file.

AHA! I now understand what you mean by binding environments: the
local/global dictionaries that you can pass to exec/eval/execfile.
And I guess I've answered your question, at least the first bit.
(BTW in 1.2, execfile is documented -- search for it in
Doc/libfuncs.tex.)

For built-in modules and modules loaded dynamically from shared
library files, the question about binding environments is moot --
these modules always create their own dictionary and never execute any
Python code. (Actually, passing the module dictionary into the init
function instead of letting the init function create it makes some
sense. There are some other changes i'd like to make too -- e.g. a
failing module initialization shouldn't be a fatal error. Maybe I
can afford changing the protocol for the init module when the naming
convention for modules changes...)

--Guido

PS The "Bob(tm)" piece was funny. Especially the bit about the
tattooed VP...