On Tuesday, April 11, 2017 at 7:28:46 PM UTC-5, Nathan Ernst wrote: > > [...] > > Writing performant Python code is possible, but like > writing performant code in any other language, you need to > be aware of what's happening. This means paying attention > to things that may cause memory allocations (which are > largely hidden from you in Python).
But what you're asking people to do is not unlike reinventing wheels. Some are not aware of every subtle case that will cause a bottleneck. For this reason, it would be wise to create a database where these bottlenecks can be logged, with the intention of creating a python optimizing tool at some time in the future. Like you said, since a large subset of python coders are not CS majors, we cannot expect them to understand Python at such a fundamental level as someone such as GvR. That is the price of creating a language that abstracts away all the dirty details so that laymen can wield it. But just as Python should _never_ seg- fault (but it does!), high level languages like Python should make it difficult, if not impossible, to write sub- optimal code (at least in the blatantly obvious cases). The whole fundamental nature of Python is to relieve the programmer of all the onerous tedium and drudgery (memory management, cryptic syntaxes, etc...), so that the programmer's mental energy can be devoted solely to the creative demands of logic and interfaces. > I worked on http://www.marketswiki.com/wiki/CMDX - in > particular I wrote most of the Migration Utility mentioned > to migrate paper CDS trades to standardized CDS contracts > against CME. Most of the migration util was written in > native Python 2.5 (it was written in 2008) using a single > thread. Performance wasn't super critical, but desired. At > the end of the project, I was processing ~100K positions > per second. Yeah, but what kind of hardware were you running it on? It seems in this case, that you wrote up a utility and ran it on a machine of your choosing, but not all developers have the advantage of choosing which machine their code executes on. Most code is written and released for mass consumption on client machines which come in all shapes and sizes. Sure, you can write sloppy code on a top-of-the-line hardware and not notice, but try the same code on a windoze 95 box and see what happens... > Threading in Python is bad - don't bother (until we can get > rid of the GIL, I doubt the situation will improve). If you search the archives (and internet) for "remove GIL", you'll find more hits than you could parse in a lifetime. Basically, the community has been begging for the GIL to be removed since the early 90s, but the Python royalty have no interest to do so. And the reasons for this could be many. I have my personal suspicions... > If you have a performance problem with Python, before you > blame Python, take a step back and look at your own code > (or libraries you're using) and ask yourself: "Is my code > optimal?" > > Yes, Python is not the faster language/runtime in > existence. But for probably 99% I don't think it's fair to make broad-brushed assertions as to the technical prowess of everyday Python programmers. Perhaps the incidents of non-optimal code are high, but 99% is difficult to swallow and sounds an awful lot like hyperbole. And although i'm sure some folks here will lap-up such gross exaggerations with prideful glee, sorry, but i'm not one of them. -- https://mail.python.org/mailman/listinfo/python-list
