On Mon Feb 02 2015 at 11:49:31 AM Julien Danjou wrote:
> On Fri, Jan 30 2015, Yuriy Taraday wrote:
>
> > That's a great research! Under its impression I've spent most of last
> > evening reading PyMySQL sources. It looks like it not as much need C
> > speedups currently as plain old Python optimi
On Fri, Jan 30 2015, Yuriy Taraday wrote:
> That's a great research! Under its impression I've spent most of last
> evening reading PyMySQL sources. It looks like it not as much need C
> speedups currently as plain old Python optimizations. Protocol parsing code
> seems very inefficient (chained s
On 01/28/2015 06:33 PM, Johannes Erdfelt wrote:
> On Wed, Jan 28, 2015, Mike Bayer wrote:
>> I can envision turning this driver into a total monster, adding
>> C-speedups where needed but without getting in the way of async
>> patching, adding new APIs for explicit async, and everything else.
>> H
On Fri, 30 Jan 2015, Doug Hellmann wrote:
My impression of this thread is that we're reaching consensus that we
should move to PyMySQL, but that we want to make that move carefully
because we expect issues due to the changes in performance and context
switching behaviors. We've seen in the past
On Thu, Jan 29, 2015, at 07:03 PM, Angus Lees wrote:
> The other factor I haven't seen mentioned here is that our usual
> eventlet+mysqldb setup can deadlock rather easily(*), resulting in the
> entire python process sitting idle for 30s until the mysqldb deadlock
> timer
> goes off and raises an
On Thu Jan 29 2015 at 12:59:34 AM Mike Bayer wrote:
> Hey list -
>
Hey, Mike.
While PyMySQL is lacking test coverage in some areas, has no external
> documentation, and has at least some areas where Python performance can be
> improved, the basic structure of the driver is perfectly fine and
>
Angus Lees wrote:
>
> If we absolutely can't switch to another mysql driver, another option that
> was suggested recently (and passes the above test) is using
> eventlet.monkey_patch(MySQLdb=True). I haven't done the investigation to
> find out why that isn't the default, or what the downs
The other factor I haven't seen mentioned here is that our usual
eventlet+mysqldb setup can deadlock rather easily(*), resulting in the
entire python process sitting idle for 30s until the mysqldb deadlock timer
goes off and raises an exception. At this point (in nova at least), the
request is usu
On Thu, Jan 29, 2015 at 11:05 AM, Roman Podoliaka
wrote:
> Mike,
>
> I can't agree more: as far as we are concerned, every service is yet
> another WSGI app. And it should be left up to operator, how to deploy
> it.
>
> So 'green thread awareness' (i.e. patching of the world) should go to
> separ
On 2015-01-29 10:21:58 -0800 (-0800), Vishvananda Ishaya wrote:
[...]
> I don’t think switching is going to give us much benefit right
> away. We will need a few optimizations and bugfixes in other areas
> (particularly in our sqlalchemy usage) before we will derive any
> benefit from the switch.
Excerpts from Vishvananda Ishaya's message of 2015-01-29 10:21:58 -0800:
>
> On Jan 29, 2015, at 8:57 AM, Roman Podoliaka wrote:
>
> > Jeremy,
> >
> > I don't have exact numbers, so yeah, it's just an assumption based on
> > looking at the nova-api/scheduler logs with connection_debug set to
>
On Jan 29, 2015, at 8:57 AM, Roman Podoliaka wrote:
> Jeremy,
>
> I don't have exact numbers, so yeah, it's just an assumption based on
> looking at the nova-api/scheduler logs with connection_debug set to
> 100.
>
> But that's a good point you are making here: it will be interesting to
> see
On 01/29/2015 05:57 PM, Roman Podoliaka wrote:
Jeremy,
I don't have exact numbers, so yeah, it's just an assumption based on
looking at the nova-api/scheduler logs with connection_debug set to
100.
But that's a good point you are making here: it will be interesting to
see what difference enabli
Mike,
I can't agree more: as far as we are concerned, every service is yet
another WSGI app. And it should be left up to operator, how to deploy
it.
So 'green thread awareness' (i.e. patching of the world) should go to
separate keystone|*-eventlet binary, while everyone else will still be
able to
On Thursday, January 29, 2015, Roman Podoliaka
wrote:
> Hi all,
>
> Mike, thanks for summarizing this in
> https://wiki.openstack.org/wiki/PyMySQL_evaluation !
>
> On PyMySQL: this is something we need to enable testing of oslo.db on
> Python 3.x and PyPy. Though, I doubt we want to make PyMySQL
Jeremy,
I don't have exact numbers, so yeah, it's just an assumption based on
looking at the nova-api/scheduler logs with connection_debug set to
100.
But that's a good point you are making here: it will be interesting to
see what difference enabling of PyMySQL will make for tempest/rally
workloa
Roman Podoliaka wrote:
>
> On native threads vs green threads: I very much like the Keystone
> approach, which allows to run the service using either eventlet or
> Apache. It would be great, if we could do that for other services as
> well.
but why do we need two approaches to be at all expli
Hi all,
Mike, thanks for summarizing this in
https://wiki.openstack.org/wiki/PyMySQL_evaluation !
On PyMySQL: this is something we need to enable testing of oslo.db on
Python 3.x and PyPy. Though, I doubt we want to make PyMySQL the
default DB API driver for OpenStack services for Python 2.x. At
On 2015-01-29 18:35:20 +0200 (+0200), Roman Podoliaka wrote:
[...]
> Otherwise, PyMySQL would be much slower than MySQL-Python for the
> typical SQL queries we do (e.g. ask for *a lot* of data from the DB).
[...]
Is this assertion based on representative empirical testing (for
example profiling de
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 01/28/2015 06:57 PM, Johannes Erdfelt wrote:
> Not sure if it helps more than this explanation, but there was a
> blueprint and accompanying wiki page that explains the move from twisted
> to eventlet:
Yeah, it was never threads vs. greenthreads.
On Wed, Jan 28, 2015, at 07:57 PM, Johannes Erdfelt wrote:
> On Wed, Jan 28, 2015, Vishvananda Ishaya wrote:
> > On Jan 28, 2015, at 4:03 PM, Doug Hellmann wrote:
> > > I hope someone who was around at the time will chime in with more detail
> > > about why green threads were deemed better than
On Wed, Jan 28, 2015, Vishvananda Ishaya wrote:
> On Jan 28, 2015, at 4:03 PM, Doug Hellmann wrote:
> > I hope someone who was around at the time will chime in with more detail
> > about why green threads were deemed better than regular threads, and I
> > look forward to seeing your analysis of a
Johannes Erdfelt wrote:
> On Wed, Jan 28, 2015, Mike Bayer wrote:
>> I can envision turning this driver into a total monster, adding
>> C-speedups where needed but without getting in the way of async
>> patching, adding new APIs for explicit async, and everything else.
>> However, I’ve no idea
On Jan 28, 2015, at 4:03 PM, Doug Hellmann wrote:
>
>
> On Wed, Jan 28, 2015, at 06:50 PM, Johannes Erdfelt wrote:
>> On Wed, Jan 28, 2015, Clint Byrum wrote:
>>> As is often the case with threading, a reason to avoid using it is
>>> that libraries often aren't able or willing to assert threa
+1
Will be an interesting analysis; I've also never understood this desire
to use eventlet (perhaps it's written somewhere?). It seems like most of
the applications in openstack (maybe leaving out the WSGI entrypoints)
can be scaled horizontally and mostly just do a large amount of work
that
On Wed, Jan 28, 2015, at 06:50 PM, Johannes Erdfelt wrote:
> On Wed, Jan 28, 2015, Clint Byrum wrote:
> > As is often the case with threading, a reason to avoid using it is
> > that libraries often aren't able or willing to assert thread safety.
> >
> > That said, one way to fix that, is to fix
On Wed, Jan 28, 2015, Clint Byrum wrote:
> As is often the case with threading, a reason to avoid using it is
> that libraries often aren't able or willing to assert thread safety.
>
> That said, one way to fix that, is to fix those libraries that we do
> want to use, to be thread safe. :)
I flo
Excerpts from Johannes Erdfelt's message of 2015-01-28 15:33:25 -0800:
> On Wed, Jan 28, 2015, Mike Bayer wrote:
> > I can envision turning this driver into a total monster, adding
> > C-speedups where needed but without getting in the way of async
> > patching, adding new APIs for explicit async,
On Wed, Jan 28, 2015, Mike Bayer wrote:
> I can envision turning this driver into a total monster, adding
> C-speedups where needed but without getting in the way of async
> patching, adding new APIs for explicit async, and everything else.
> However, I’ve no idea what the developers have an appet
Mike Bayer wrote:
> Hey list -
>
> As many are aware, we’ve been looking for the one MySQL driver to rule them
> all. As has been the case for some weeks now, that driver is PyMySQL,
> meeting all the critical requirements we have of: 1. pure python, so
> eventlet patchable, 2. Python 3 s
Hey list -
As many are aware, we’ve been looking for the one MySQL driver to rule them
all. As has been the case for some weeks now, that driver is PyMySQL, meeting
all the critical requirements we have of: 1. pure python, so eventlet
patchable, 2. Python 3 support 3. Is released on Pypi (whi
31 matches
Mail list logo