On Tue, Oct 26, 2010 at 11:47 AM, Anand Balachandran Pillai < abpil...@gmail.com> wrote:
> On Mon, Oct 25, 2010 at 7:33 AM, Dhananjay Nene <dhananjay.n...@gmail.com > >wrote: > > > On Wed, Oct 20, 2010 at 4:57 PM, Noufal Ibrahim <nou...@gmail.com> > wrote: > > > > > > > > > > > Assuming that most of the people here are mostly python enthusiasts or > > > learners, I'm wondering when you would *not* use Python. Let's not > > > conflate this with Open Source/Closed Source etc. > > > > > > > > Python is the language which has (what in business terms is coined as) > the > > right of first refusal. > > > > ie. evaluate something whether it really makes sense in python, and if > not > > in python then start exploring what else. > > > > The primary reasons which may encourage me to use a language other than > > python are : > > > > a. Need for raw CPU power. Yes there are a few of such use cases. When > > purely CPU driven and when you really really need the CPU badly (which is > > probably less than 5% cases) python performance is really poor. > > b. Need for high amounts of multithreading even while being largely CPU > > bound. > > > > Yes - both situations can be handled by escaping to C from python, but I > > would much rather handle these in Java (or languages based on JVM) rather > > than C unless I am in a very constrained hardware situation. > > > > On a separate note I do believe as we start heading towards the 32 core > > situation (say in 5 yrs time) - there better exist a better solution than > > the GIL :) > > > > > I think we should move from the thinking that concurrency <= threading > and move over to better solutions in languages like Python which have > better ways of solving the concurrency problem than actually having to > create OS level threads. > Threading is not the only solution to address concurrency but it is a valid one. The only issue I have with the statement above is that it seems to suggest that the move will address all use cases. It simply will not. At the same time I do agree that imagining threading to be the one and only solution to address concurrency is equally inadvisable. > > If you look at it differently, concurrency is the real problem and > threading > is just one approach to it which is kind of resource intensive and doesn't > work great in Python due to the GIL. But instead of GIL bashing, one > should > look at other approaches to concurrency such as generator based > co-operative > "threading" and also look at numerous other concurrency libraries in > Python > before deciding that going the way of actual threads is his solution. > Here's an example of what I had done once for an extremely extremely high performance application (million+ financial transactions per hour). There were layers of caches (with some cache coordination to ensure consistency). It was the high speed caches and cache access and which really helped achieve that performance. And that was possible because I had a very large number of threads sharing some of the caches (some caches were thread level). I cannot imagine making such an application work in a multi process environment. While I did not evaluate cooperative threading, I suspect that would also introduce its own overheads. When one really really needs threads, GIL sucks. What I am saying is that GIL bashing is not unreasonable given the context of specific use cases (we were discussing situations where one would prefer to not use python). > > Python should go the direction shown by languages such as Erlang which has > true user-space multitasking with light-weight "processes". It already has > the infrastructure for it with generators etc but support for true > multitasking > at user level should be part of the language rather than added on by > third party libraries. Right now if one decides to explore Python > concurrency, > the scene is truly confusing and a little scary, since there are too many > approaches out there which also share a lot of similarities. > > I agree (go down direction of light weight processes). > Hopefully the "futures" have something to offer here :) > > Once again agreed. > --Anand > > > > > > > > > > > > -- > > > ~noufal > > > http://nibrahim.net.in > > > > > > _______________________________________________ > > > BangPypers mailing list > > > BangPypers@python.org > > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > > > > > > -- > > -------------------------------------------------------- > > blog: http://blog.dhananjaynene.com > > twitter: http://twitter.com/dnene > > _______________________________________________ > > BangPypers mailing list > > BangPypers@python.org > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > -- > --Anand > _______________________________________________ > BangPypers mailing list > BangPypers@python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- -------------------------------------------------------- blog: http://blog.dhananjaynene.com twitter: http://twitter.com/dnene _______________________________________________ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
