Re: Actual USEFULNESS of old Syntax (was Re: New Syntax ...)

Tim Peters (tim@ksr.com)
Thu, 02 Jun 94 15:22:21 -0400

> [jt asks a question dear to my heart]
> ... how much more complex will Python-mode need to be to check for all
> possible syntaxes?

Since Guido has banned me from the Python portion of this debate (for
which I am indecently grateful <wink>), here's the scoop on py-mode:

Anything that doesn't look exactly like Lisp or C is a puzzle for Emacs
modes. That's because they're in the business of parsing small pieces of
your code over & over & over again, mostly parsing _backwards_ & from
arbitrary starting points, are written in (slow) elisp, and just to make
it easy <snort> need to give the appearance of instantaneous response.

The saving grace for things that "look like" Lisp and C is that Emacs
comes with built-in parsing functions (written in (fast) C) that
understand important parts of Lisp and C syntax.

The saving grace for Python used to be that, although Emacs didn't have
much applicable built-in support, the syntax was so simple that cheap
tricks sufficed to make (slow) elisp "parsing" effective & bulletproof.
E.g., "a line opens a new block if and only if its last non-whitespace
character is a colon that's not in a comment" used to work, and is cheap
both to test and search backwards for even in (slow) elisp. There are a
lot of cheap tricks like that in py-mode.

I believe triple-quoted strings broke all of them, though, so it's not
like it could get much worse in theory. But in practice, py-mode is
still reliable so long as you're not near a triple-quoted string.

Other changes that make it hard to "parse backwards fast" will have
similar consequences. The _usual_ thing is to adopt cheap heuristics
that sometimes fail. E.g., the person who wrote Perl mode is a brilliant
guy, but if you've used it in the presence of Perl's non-Lisp/C-ish
syntax features you know it's hard _not_ to confuse it. That's not
because the author didn't try & is certainly not because he's incompetent
-- getting it 100% right & fast in elisp just doesn't seem possible.

Far as I know, py-mode's tricks used to be 100% reliable. That's not
true anymore, and I doubt it will ever be true again. So the only
question now is how badly broken it gets in practice. But: (a) It's not
_necessarily_ true that an alternative blocking scheme will hurt it more
in the end; and, (b) py-mode shouldn't drive anything.

one-of-triple-quoted-strings'-biggest-fans-ly y'rs - tim

Tim Peters tim@ksr.com
not speaking for Kendall Square Research Corp