Python vs. Lisp

(Source and copyright: http://mail.python.org/pipermail/python-list/2006-February/326750.html)

> Hi, I've been thinking about Python vs. Lisp.  I've been learning
> Python the past few months and like it very much.  A few years ago I
> had an AI class where we had to use Lisp, and I absolutely hated it,
> having learned C++ a few years prior.  They didn't teach Lisp at all
> and instead expected us to learn on our own.  I wasn't aware I had to
> uproot my thought process to "get" it and wound up feeling like a
> moron.
>
> In learning Python I've read more about Lisp than when I was actually
> trying to learn it, and it seems that the two languages have lots of
> similarities:
>
> http://www.norvig.com/python-lisp.html
>
> I'm wondering if someone can explain to me please what it is about
> Python that is so different from Lisp that it can't be compiled into
> something as fast as compiled Lisp?

Nothing. Given a sufficiently smart implementation any language can be as fast as any other -- it might just be a billion times harder to write that implementation for language A than for language B.

> From this above website and others, I've learned that compiled Lisp can be
> nearly as fast as C/C++, so I don't understand why Python can't also
> eventually be as efficient? Is there some specific basic reason it's
> tough? Or is it that this type of problem in general is tough, and Lisp has
> 40+ years vs Python's ~15 years?

I think if you're looking for one single reason, it is presumably that (IIRC) python was designed on the erronous assumption that dynamically typed languages have to be slow (and are unsuitable for real applications anyway) wheras common lisp wasn't. Furthermore the people involved in common lisp were much more knowledgeable and experienced in things like compiler design and had a long history of similar languages and various implementations to build upon.

As common lisp and scheme demonstrate you can have high level of dynamism (and in a number of things both are more dynamic than python) and still get very good performance (in some cases close to or better than C). But both these languages have been designed with compiler writers and the generation of fast code in mind, so they made design decisions to ease writing fast lisp compilers and programs.

For example:

  • python classes (and to some extent modules) are essentially dictionaries that you can modify and customize more or less at will at run-time and that behave interchangeably in many respects. I'm sure that presents several optimization headaches.

    By contrast if the common lisp compiler sees the symbol CL:LIST (usually written just LIST, because the CL package is imported by default) it can safely assume that it refers to the builtin LIST function, because you're not allowed to rebind the function value of functions in the CL package. Python can assume no such thing if it comes across list -- for all it knows it might as well be the number 42. Also the package and class system are completely separate and although common lisp's OO system is rather more powerful than python's it has been designed to be implementable efficiently.

  • in python almost everything has to happen at run-time, whereas in common lisp you can do things at compile time, load time or run-time e.g:

    • common lisp has a mechanism for making compiler declarations (so you can tell the compiler to inline a function, or the type of a variable, or to optimize something for speed and not for space etc.)
    • common lisp has macros (proper ones, not C style) which allow you to build efficient abstractions
    • common lisp has compiler macros. This sort of means that you can write your own compiler optimizations for your functions (i.e. if you know that your expensive FOO function is indempotent you could arrange for all calls of the form (FOO (FOO A)) to be replaced with simply A, in a similar way as an optimizing compiler might replace (a+b+c+d)*0 with 0).

What's far more interesting to me, however, is that I think there a good reasons to suspect python's slowness is more of a feature than a flaw: I'd not be suprised if on the whole it greatly increases programmer productivity and results in clearer and more uniform code.

If you know the language to be dog slow any way, you're much less likely to waste your time (and that of future maintainers) on the pointless microoptimizations that geeks so love. Also, since only builtins have reasonable performance there's added motiviation to become very familiar with the available builtins (and standard libarary) and far less temptation to roll one's own version of say dict.setdefault (even if it it sucks). The fact that non-standard library code is inherently somewhat inferior (because it will either be written in python and slow or written in C and a pain to install) adds further incentive to attempt community wide standardization.

I think it's not unreasonable to speculate that all this decreases production, maintenance and reuse costs of python code considerably, so much in fact that python's very slowness represents part of its competetive edge over languages that are in some ways better engineered and more capable.

So ironically, some share of python's success might actually be due to ignorance on Guido's part (of course python in most respects is a marvellously well designed language that to my mind places Guido at least on par with the designers of any contemporary language; I'm just intrigued by the possiblity that had he known as much about performance issues in very dynamic languages as say some of the lisp and self people, python might have turned out to be a faster albeit less popular and productive language).

'as

Wow, you really are a lisp

Wow, you really are a lisp zealot. Did you even read what you wrote? Python 'slowness' a feature. bleh. Implement that syntax to force types... no thanks.

Slowness is a feature...

Slowness is a feature... That is an extremely interesting (and amusing) idea. Not sure if I buy that, but I give you points for creativity.

"python might have

"python might have turned out to be a faster albeit less popular and productive language"

I parse that to be: faster albeit less (popular and productive language)