problems with tokenizer and line numbers

lance@fox.com
Fri, 5 Aug 94 15:53:33 PDT

I was trying to get some old code up and running and noticed
that the "SyntaxError" return from compile() is broken.
I know this happens if you load a program into a string object
and then call compile(stringobject,'whatever','exec').
What happens is that the line number is ALWAYS 0 and the offset is
how many bytes INTO the stringobject the error was and the returned
error text is the WHOLE stringobject.

Example of how it works now and then how I think should work:

>>> a = 'def test():\n\tprint "test2"\n\t\tprint "test3"\n\tprint "test4"\n'
>>> try:
... t = compile(a,'from a','exec')
... except SyntaxError, msg:
... print msg
...
('invalid syntax', (None, 0, 29, 'def test():\012\011print "test2"\012\011\011print "test3"\012\011print "test4"\012'))
>>>

>>> a = 'def test():\n\tprint "test2"\n\t\tprint "test3"\n\tprint "test4"\n'
>>> try:
... t = compile(a,'from a','exec')
... except SyntaxError, msg:
... print msg
...
('invalid syntax', (None, 2, 2, '\011\011print "test3"\012'))
>>>

Can this be done easily? I have not looked at the code yet, but I
imagine it should not be too hard to do... 

--
Lance Ellinghouse                lance@fox.com