+1
To give an example as to why eventlet implicit monkey patch the world isn't
especially great (although it's what we are currently using throughout
openstack).
The way I think about how it works is to think about what libraries that a
single piece of code calls and how it is very hard to predict whether that code
will trigger a implicit switch (conceptually similar to a context switch).
Let's take a simple naive piece of code.
>>> import logging
>>> LOG = logging.getLogger(__name__)
>>> LOG.info("hi")
This seems rather straightforward, write 'hi' to some log location. With
eventlets implicitness (via ye-old monkey patch everything) it is entirely
possible that somewhere inside the logging code there will be a write to a
socket (say perhaps this person enabled syslog/socket logger or something like
that) and that will block. This causes an implicit switch to another
greenthread (and so-on for the applications life-cycle). Now just magnify the
amount of understanding required to reason about how the logging library (which
is pretty well understood) works with eventlet by the number of libraries in
https://github.com/openstack/requirements/blob/master/global-requirements.txt.
To understand how all these libraries interact with I/O, threading or other
locations where things can implicitly switch is pretty much near impossible. It
becomes even more 'hairy' when those libraries themselves acquire some type of
locks (did you as an eventlet user remember to monkey patch the threading
module?)...
IMHO eventlet has 'seduced' many developers into thinking that it magically
makes an application C10K ready even though it easily makes it possible to
'crash and burn' without to much trouble. Is the benefit worth it? Maybe, maybe
not...
I'm not saying we should abandon eventlet (likely we can't easily pull this off
even if we wanted to), but I do agree that the randomness it provides is not
easy to follow, debug, analyze... It gets even more complicated when you start
to mix threads (which do exist in python, but are GIL handicapped, although
this has been getting better in 3.2+ with GIL improvements) with greenthreads
(try figuring out which one is causing race conditions in a gdb session for
example).
Anyways, the future of this whole situation looks bright, it will be an
interesting balance between making it easy to read/understand (eventlet tries
to make it look so easy and no-changes needed, see above seduction) vs.
requiring a "big" mind-set change in how libraries and applications are
currently written.
Which is the right path to get to the final destination, only time will tell :-)
-Josh
Sent from my really tiny device...
On Feb 6, 2014, at 6:55 PM, "Zane Bitter"
<zbit...@redhat.com<mailto:zbit...@redhat.com>> wrote:
On 04/02/14 13:53, Kevin Conway wrote:
On 2/4/14 12:07 PM, "victor
stinner"<victor.stin...@enovance.com<mailto:victor.stin...@enovance.com>>
wrote:
>The purpose of replacing eventlet with asyncio is to get a well defined
>control flow, no more surprising task switching at random points.
I disagree with this. Eventlet and gevent yield the execution context
anytime an IO call is made or the 'sleep()' function is called explicitly.
The order in which greenthreads grain execution context is deterministic
even if not entirely obvious. There is no context switching at random.
This is technically correct of course, but in reality there's no way to know
whether a particular piece of code is safe from context switches unless you
have the entire codebase of the program and all of its libraries in your head
at the same time. So no, it's not *random*, but it might as well be. And it's
certainly not explicit in the way that asyncio is explicit; it's very much
implicit in other operations.
What's more is it shouldn't matter when the context switch happens. When
writing green threaded code you just pretend you have real threads and
understand that things happen in an order other than A => B => C.
If you like pretending you have real threads, you could just use Python
threads. Greenthreads exist because people don't want to deal with actual
pretend threads.
One of the great benefits of using a green thread abstraction, like
eventlet or gevent, is that it lets you write normal Python code and slap
your concurrency management over the top.
Right, it lets you do that and neglects to mention that it doesn't actually
work.
The whole premise of eventlet is that it allows you to write your code without
thinking about thread safety, except that you *do* still have to think about
thread safety. So the whole reason for its existence is to write cheques that
it can't cash. It's conceptually unsound at the most fundamental level.
I'm not suggesting for a second that it would be an easy change - I'm not even
sure it would be a good change - but let's not kid ourselves that everything is
fine here in happy-land and there's nothing to discuss.
cheers,
Zane.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
