Re: JavaVM compiler for Python?

Chak Tan (tan@ohm.ee.rochester.edu)
19 Apr 1995 16:09:54 -0400

Now, that Python 1.2 is out the door, the time seems ripe to chew on
some ideas for the next release. I think that a compiler would be
very good for the proliferation of Python, and a JavaVM based one
would be very nice indeed.

---- excerpt from Farshad Nayeri ----
Getting back to the point, all of this may be important *if* Java is
widely deployed. Dan seems to think so. Dan, can you explain why you
think this is the case?
-------------------------------------

I'm not Dan, but I have the impression that Java will really take off
because the WWW is such a huge success. From what I gather, Netscape,
who dominates the WWW scene, is working with Sun on future WWW based
products. I would guess that Java is the language they will use as
the base language. Right now, about 60% of the browsers out there are
NetScape and the number is climbing. So, in the future, if the number
of browsers out there that uses Java as the base agent language is 70%
(a guess based on the combined strength of Netscape and Sun), then not
many people are going to have access to all those cool Python
agents/programs unless Python compiles to JavaVM (assuming that the
browsers will accept programs/applets/agents or
whatever-they-will-be-call in compiled form). As you can tell, my
views of a successful language in the near future is heavily biased
towards network based activities.

Some reasons I chose Python over all the other currently available
dynamic languages are: the licensing agreement, the ILU interface,
and the elegant nature of the language which greatly simplifies
complex operations (in my opinion that is).

Java looks to be a great language and I have no reservations about
including it in my arsenal. At this time however, I still preferr
Python because I perceive it to be more elegant (although Java's
underlying design is great, Java syntax is too C like to
be consider really elegant to me). I would very much like to keep
programming in Python with the knowledge that the language will not
die out in a couple of years because it did not reach critical mass.

I think many of the ideas in S. Majewski's WWW page should be
considered now for future versions of Python. First, reference
counting should be eliminated in favor of garbage collection which
sets the anchor to other enhancements. Java's has its own memory
management and threads support, if the Python is complied to JavaVM
then we might as well use Java's underlying MM for Python too.
Otherwise the Xerox based system sounds very good too. Second,
speeding up the interpreter would be nice. I looked at using jump
tables instead of one big switch statement in the ceval.c but decided
against it because it would only compile under gcc. A threaded
interpreter, as mentioned by SM sounds ideal if Python is amendable to
such a change. The ability to build user-defined constructs as in
Forth would be a big bonus (BTW are there things you can do with
macros that are not do-able with user-defined constructs a la Forth?)
Third, is the issue of compiling to native code. Sun is providing a
native code compiler (or is that translator) for their JavaVM.
Compiling to RTL and using GCC was also mentioned in SM's WWW page.

Right now, I do not have the time or expertise to really bring off
these ambitious projects for Python, but I intend to contribute as
much as possibile in about seven months or so (after I finish my PhD).
I just thought I fuel some discussion by tossing around my ignorance
and opinions.

Well, since I have your attention, I might as well ask for some Alpha
testers for a module I hacked up. In short, the purpose of my module
is to expose C objects for interacive exploration in Python. I'm sure
it'll be a subset of what Donald Beaudry's CTypes system will do, but
he hasn't responded to my latest email so I can't exactly tell (are
you on vacation or just really busy Mr. Beaudry?) Here is an example
of it in action.

---- C code to expose C variables to Python ----
#include "allobjects.h"
#include "modsupport.h" /* For getargs() etc. */
#include "addermodule.h"

static PyObject *ErrorObject;

/* ----------------------------------------------------- */

/* List of methods defined in the module */

static struct PyMethodDef test_methods[] = {
{NULL, NULL} /* sentinel */
};

int test_int = 4;
char test_string[20] = "Hi, there!\n";
char *test_ptr = test_string;
double test_a_double[9] = {1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0};
struct test_struct {
double one;
int two;
char *three;
double *four;
} test_struct = {
90.5, 69, test_string, test_a_double
};

#define OFF(x) offsetof(struct test_struct, x)
Fgn_m_info test_members[] = {
{"one", OFF(one), FFLAG_RO, &C_double},
{"two", OFF(two), FFLAG_RO, &C_int},
{"three", OFF(three), FFLAG_RO, NULL},
{"four", OFF(four), FFLAG_RO, NULL},
{NULL}
};

Fgn_a_info test_3x3[] = {{3,-1,1},{1,-1,1},{0,0,0}};

int test_sq(int in) {
return in*in;
}

Fgn_f_info C_sq_info = {
FFI_int_C_int,
&Py_obj,
"i",
&test_sq,
NULL
};

void
inittest()
{
PyObject *m, *d, *key, *value;
PyObject *f1, *f2, *f3, *f4, *f5, *f6, *flag_enum;
foreignobject *temp;

Fgn_desc *C_ptr_C_string = clone_Fgn_desc(&C_ptr);
Fgn_desc *C_sq = clone_Fgn_desc(&C_func);
Fgn_desc *C_a_double = clone_Fgn_desc(&C_array);
Fgn_desc *C_ptr_double = clone_Fgn_desc(&C_ptr);
Fgn_desc *C_st = clone_Fgn_desc(&C_struct);
Fgn_desc *T_enum = clone_Fgn_desc(&C_enum);

T_enum->next = &C_int;
flag_enum = new_C_enum(T_enum, TCL_ONE_WORD_KEYS);

C_ptr_C_string->next = &C_string;
C_ptr_C_string->magic = (char *)&C_string; /* want to access content */
C_sq->magic = (char *)&C_sq_info;
C_a_double->size = sizeof(test_a_double);
C_a_double->next = &C_double;
C_a_double->magic = (char *)&test_3x3;
C_ptr_double->next = &C_double;
C_st->size = sizeof(test_struct);
C_st->magic = (char *)&test_members;

test_members[2].m_Fdesc = C_ptr_C_string;
test_members[3].m_Fdesc = C_ptr_double;

/* Create the module and add the functions */
m = Py_InitModule("test", test_methods);

f1 = newforeignobject((char*) &test_int, &C_int, 0);
f2 = newforeignobject((char*) &test_string, &C_string, FFLAG_RO);
f3 = newforeignobject((char*) &test_ptr, C_ptr_C_string, 0);
f4 = newforeignobject((char*) &test_a_double, C_a_double, FFLAG_RO);
f5 = newforeignobject(NULL, C_sq, 0);
f6 = newforeignobject((char*) &test_struct, C_st, FFLAG_RO);


/* Add some symbolic constants to the module */
d = PyModule_GetDict(m);
ErrorObject = PyString_FromString("test.error");
PyDict_SetItemString(d, "error", ErrorObject);
PyDict_SetItemString(d, "int" , f1);
PyDict_SetItemString(d, "string", f2);
PyDict_SetItemString(d, "ptr", f3);
PyDict_SetItemString(d, "array", f4);
PyDict_SetItemString(d, "func", f5);
PyDict_SetItemString(d, "struct", f6);
PyDict_SetItemString(d, "enum", flag_enum);

/* XXXX Add constants here */

/* Check for errors */
if (PyErr_Occurred())
Py_FatalError("can't initialize module test");
}

---- Python code to test enum -----
import test
bits = test.enum.new(1)
t = test.int.new()
for i in range(1,33):
t.py = 1 << (i-1)
bits[t] = 'bit'+ repr(i)
flags = test.enum.new(1)
t.py = 1
flags[t] = 'ReadOnly'
t.py = 2
flags[t] = 'Dynamic'
t.py = 4
flags[t] = 'Static'
t.py = 8
flags[t] = 'Extern'
t.py = 16
flags[t] = 'Boss'
t.py = 32
flags[t] = 'Temporary'

---- Example of Python session ----Python 1.2-beta-3 (Mar 31 1995) [GCC 2.6.3]

Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from enum import *
>>> test.int
4
>>> a = test.int
>>> a.desc
'C_int'
>>> a.__methods__
['cast', 'new']
>>> type(a)
<type 'foreign'>
>>> type(a.py)
<type 'int'>
***** you have to explicitly declare it as a Python object here
in order to change its content *****
>>> a.py = a.py + 10
>>> a
14
>>> a.py
14
>>> a = test.ptr
***** pointers are displayed in a special format *****
>>> a
&ef76d078
>>> a.deref()
'Hi, there!\012'
>>> a = test.string
>>> a
'Hi, there!\012'
>>> a = test.array
>>> a
<fgnDesc: &7b008, flags: 1 = RO+, type: C_array C_double>
>>> a.special
['bounds', 'dim', '...[[n,...]]']
>>> a.bounds
[(-1, 1), (-1, 1)]
>>> a.dim
2
>>> a[[1,2]]
Traceback (innermost last):
File "<stdin>", line 1, in ?
ConflictError: index 2 is out of bound
**** arrays are accessed using [[...]] syntax ****
>>> a[[1,1]]
9.0
>>> a[[-1,-1]]
1.0
>>> a = test.struct
>>> a.one
90.5
>>> a.two
69
>>> a.three
&ef76d078
**** pointes are access using [...] syntax ****
>>> a.three[0]
'Hi, there!\012'
>>> a.four
&ef76d090
***** four is a member of a structure that
is a pointer to an array of double *****
>>> a.four[0]
1.0
>>> a.four[8]
9.0
**** test.func is a function that squares its argument ****
>>> a = test.func
>>> a(10)
100
>>> a(20)
400
>>> a.special
['args', 'funcType', 'returns']
>>> a.args
'i'
>>> a.returns
'PyObject'
**** enums have a table of allowed keys and values ****
>>> bits.keys()
[-2147483648, 1073741824, 1, 2, 4, 256, 8192, 2097152, 65536, 8, 512,
268435456, 16384, 33554432, 4194304, 1024, 131072, 16, 536870912, 128,
1048576, 32768, 4096, 134217728, 16777216, 524288, 64, 2048, 67108864,
8388608, 262144, 32]
>>> bits.values()
['bit32', 'bit31', 'bit1', 'bit2', 'bit3', 'bit9', 'bit14', 'bit22',
'bit17', 'bit4', 'bit10', 'bit29', 'bit15', 'bit26', 'bit23', 'bit11',
'bit18', 'bit5', 'bit30', 'bit8', 'bit21', 'bit16', 'bit13', 'bit28',
'bit25', 'bit20', 'bit7', 'bit12', 'bit27', 'bit24', 'bit19', 'bit6']
>>> bits.dict()
{1: 'bit1', 2: 'bit2', 64: 'bit7', 4: 'bit3', 4194304: 'bit23', 128:
'bit8', 8: 'bit4', 4096: 'bit13', 8388608: 'bit24', 32768: 'bit16',
256: 'bit9', 268435456: 'bit29', 16: 'bit5', 8192: 'bit14', 67108864:
'bit27', 16777216: 'bit25', 65536: 'bit17', 512: 'bit10', 262144:
'bit19', 536870912: 'bit30', 32: 'bit6', 2097152: 'bit22', 2048:
'bit12', 16384: 'bit15', 134217728: 'bit28', 33554432: 'bit26',
131072: 'bit18', 1048576: 'bit21', 1024: 'bit11', 524288: 'bit20',
-2147483648: 'bit32', 1073741824: 'bit31'}
**** keys is suppose to be a foreignobject in this case ****
>>> bits[1]
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: key is not a foreignobject
>>> test.int.py = 16
>>> bits[test.int]
'bit5'
***** mask accepts PyInt as argument too *****
>>> bits.mask(1111)
['bit1', 'bit2', 'bit3', 'bit5', 'bit7', 'bit11']
>>> bits.value
Traceback (innermost last):
File "<stdin>", line 1, in ?
AccessError: not initialized
>>> bits.value = 'bit7'
>>> bits.key
64
>>> bits.key = test.int
>>> bits.value
'bit5'
**** enum variables can only take on values and keys that
are in its table of values and keys ****
>>> bits.value = 'hi'
Traceback (innermost last):
File "<stdin>", line 1, in ?
AccessError: item's value not in table
>>> flags.dict()
{1: 'ReadOnly', 2: 'Dynamic', 16: 'Boss', 4: 'Static', 32:
'Temporary', 8: 'Extern'}
>>> bits.flags
18
>>> flags.mask(bits.flags)
['Dynamic', 'Boss']
>>> flags.__methods__
['cast', 'del_key', 'del_value', 'dict', 'get_key', 'keys', 'mask',
'new', 'values']
>>>

If you are interested in looking this over, email me at: tan@ee.rochester.edu

Chak Tan