Re: [openstack-dev] [oslo.db]A proposal for DB read/write separation

at we have. I'm not clear what exactly > you are proposing, hopefully you can answer some of my questions inline. The > thing I had thought of immediately was detection of whether an operation is > read or write and integrating that into oslo.db or sqlalchemy. Mike Bayer has >

Re: [openstack-dev] [oslo.db]A proposal for DB read/write separation

On Aug 10, 2014, at 11:17 AM, Li Ma wrote: > > How about Galera multi-master cluster? As Mike Bayer said, it is virtually > synchronous by default. It is still possible that outdated rows are queried > that make results not stable. not sure if I said that :). I know extremely

Re: [openstack-dev] [nova][core] Expectations of core reviewers

On Aug 13, 2014, at 1:44 PM, Russell Bryant wrote: > On 08/13/2014 01:09 PM, Dan Smith wrote: >> Expecting cores to be at these sorts of things seems pretty reasonable >> to me, given the usefulness (and gravity) of the discussions we've been >> having so far. Companies with more cores will have

[openstack-dev] [all] Acceptable methods for establishing per-test-suite behaviors

Hi all - I’ve spent many weeks on a series of patches for which the primary goal is to provide very efficient patterns for tests that use databases and schemas within those databases, including compatibility with parallel tests, transactional testing, and scenario-driven testing (e.g. a test th

Re: [openstack-dev] [Openstack-stable-maint] [Neutron][stable] How to backport database schema fixes

On Aug 29, 2014, at 7:23 AM, Alan Pevec wrote: >> It seems that currently it's hard to backport any database schema fix to >> Neutron [1] which uses alembic to manage db schema version. Nova has the >> same issue before >> and a workaround is to put some placeholder files before each release. >>

Re: [openstack-dev] Kilo Cycle Goals Exercise

On Sep 7, 2014, at 9:27 PM, Anita Kuno wrote: > On 09/07/2014 09:12 PM, Angus Salkeld wrote: >> Lets prevent blogs like this: http://jimhconsulting.com/?p=673 by making >> users happy. > I don't understand why you would encourage writers of blog posts you > disagree with by sending them traffic.

Re: [openstack-dev] Kilo Cycle Goals Exercise

On Sep 7, 2014, at 8:14 PM, Monty Taylor wrote: > > > 2. Less features, more win > > In a perfect world, I'd argue that we should merge exactly zero new features > in all of kilo, and instead focus on making the ones we have work well. Some > of making the ones we have work well may wind up

Re: [openstack-dev] Kilo Cycle Goals Exercise

On Sep 8, 2014, at 11:30 AM, Anita Kuno wrote: > Wow, we are really taking liberties with my question today. > > What part of any of my actions current or previous have led you to > believe that I want to now or ever have silenced anyone? I am curious > what led you to believe that silencing us

Re: [openstack-dev] memory usage in devstack-gate (the oom-killer strikes again)

Hi All - Joe had me do some quick memory profiling on nova, just an FYI if anyone wants to play with this technique, I place a little bit of memory profiling code using Guppy into nova/api/__init__.py, or anywhere in your favorite app that will definitely get imported when the thing first runs

Re: [openstack-dev] memory usage in devstack-gate (the oom-killer strikes again)

of them, and then add it to our standard toolset. On Sep 9, 2014, at 10:35 AM, Doug Hellmann wrote: > > On Sep 8, 2014, at 8:12 PM, Mike Bayer wrote: > >> Hi All - >> >> Joe had me do some quick memory profiling on nova, just an FYI if anyone >> wants to

Re: [openstack-dev] [all] i need some help on this bug Bug #1365892

On Sep 10, 2014, at 4:11 AM, Li Tianqing wrote: > After some research, i find the reason for the cycle reference. In closure, > the _fix_paswords.func_closre reference the _fix_passwords. So the > cycle reference happened. > And in https://thp.io/2012/python-gc/python_gc_final_2012-01-22.pdf

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

On Sep 12, 2014, at 7:20 AM, Sean Dague wrote: > > Because we are in Feature Freeze. Now is the time for critical bug fixes > only, as we start to stabalize the tree. Releasing dependent libraries > that can cause breaks, for whatever reason, should be soundly avoided. > > If this was August,

[openstack-dev] battling stale .pyc files

I’ve just found https://bugs.launchpad.net/nova/+bug/1368661, "Unit tests sometimes fail because of stale pyc files”. The issue as stated in the report refers to the phenomenon of .pyc files that remain inappropriately, when switching branches or deleting files. Specifically, the kind of scenar

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 7:39 AM, Sean Dague wrote: > I assume you, gentle OpenStack developers, often find yourself in a hair > tearing out moment of frustration about why local unit tests are doing > completely insane things. The code that it is stack tracing on is no > where to be found, and yet i

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

On Sep 12, 2014, at 10:40 AM, Ihar Hrachyshka wrote: > Signed PGP part > On 12/09/14 16:33, Mike Bayer wrote: >> I agree with this, changing the MySQL driver now is not an option. > > That was not the proposal. The proposal was to introduce support to > run against som

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 11:24 AM, Sean Dague wrote: > On 09/12/2014 11:21 AM, Mike Bayer wrote: >> >> On Sep 12, 2014, at 7:39 AM, Sean Dague wrote: >> >>> I assume you, gentle OpenStack developers, often find yourself in a hair >>> tearing out moment of

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 11:33 AM, Mike Bayer wrote: > > On Sep 12, 2014, at 11:24 AM, Sean Dague wrote: > >> On 09/12/2014 11:21 AM, Mike Bayer wrote: >>> >>> On Sep 12, 2014, at 7:39 AM, Sean Dague wrote: >>> >>>> I assume you, gent

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

On Sep 12, 2014, at 11:56 AM, Johannes Erdfelt wrote: > On Fri, Sep 12, 2014, Doug Hellmann wrote: >> I don’t think we will want to retroactively change the migration scripts >> (that’s not something we generally like to do), > > We don't allow semantic changes to migration scripts since peopl

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 11:55 AM, Julien Danjou wrote: > On Fri, Sep 12 2014, Sean Dague wrote: > >> Which sets PYTHONDONTWRITEBYTECODE=true in the unit tests. >> >> This prevents pyc files from being writen in your git tree (win!). It >> doesn't seem to impact what pip installs... and if anyone k

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 12:03 PM, Sean Dague wrote: > On 09/12/2014 11:33 AM, Mike Bayer wrote: >> >> On Sep 12, 2014, at 11:24 AM, Sean Dague wrote: >> >>> On 09/12/2014 11:21 AM, Mike Bayer wrote: >>>> >>>> On Sep 12, 2014, at 7:39 A

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 12:13 PM, Jeremy Stanley wrote: > On 2014-09-12 11:36:20 -0400 (-0400), Mike Bayer wrote: > [...] >> not to mention PYTHONHASHSEED only works on Python 3. What is the >> issue in tox you’re referring to ? > > Huh? The overrides we added to numerous

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

On Sep 12, 2014, at 12:29 PM, Daniel P. Berrange wrote: > On Fri, Sep 12, 2014 at 04:23:09PM +, Jeremy Stanley wrote: >> On 2014-09-12 17:16:11 +0100 (+0100), Daniel P. Berrange wrote: >> [...] >>> Agreed, the problem with stale .pyc files is that it never occurs to >>> developers that .pyc

Re: [openstack-dev] [Octavia] Which threading library?

On Sep 12, 2014, at 5:53 PM, Eichberger, German wrote: > Hi, > > I think the “standard” threading library for OpenStack is eventlet – however, > it seems that Oslo is spearheading efforts to get to a more compatible one > (see http://techs.enovance.com/6562/asyncio-openstack-python3) I am n

Re: [openstack-dev] battling stale .pyc files

On Sep 15, 2014, at 7:34 AM, Lucas Alvares Gomes wrote: > > So this ordering thing, I don't think that it's caused by the > PYTHONDONTWRITEBYTECODE, I googled that but couldn't find anything > relating this option to the way python hash things (please point me to > a document/code if I'm wrong)

Re: [openstack-dev] Please do *NOT* use "vendorized" versions of anything (here: glanceclient using requests.packages.urllib3)

On Sep 17, 2014, at 2:46 PM, Clint Byrum wrote: > Excerpts from Davanum Srinivas's message of 2014-09-17 10:15:29 -0700: >> I was trying request-ifying oslo.vmware and ran into this as well: >> https://review.openstack.org/#/c/121956/ >> >> And we don't seem to have urllib3 in global-requiremen

Re: [openstack-dev] Please do *NOT* use "vendorized" versions of anything (here: glanceclient using requests.packages.urllib3)

On Sep 17, 2014, at 3:42 PM, Ian Cordasco wrote: > > Circling back to the issue of vendoring though: it’s a conscious decision > to do this, and in the last two years there have been 2 CVEs reported for > requests. There have been none for urllib3 and none for chardet. (Frankly > I don’t think

Re: [openstack-dev] Please do *NOT* use "vendorized" versions of anything (here: glanceclient using requests.packages.urllib3)

On Sep 17, 2014, at 4:31 PM, Ian Cordasco wrote: > Project X pins a version of requests. Alice doesn’t know anything about > requests and does pip install X. Until Alice takes a more active role in > the development of Project X and looks into requests, she will never know > she’s installed soft

Re: [openstack-dev] [neutron][db] Ensuring db isolation between tests

I’ve done a lot of work on this issue and from my perspective, the code is mostly ready to go, however we’re in an extended phase of getting folks to sign off as well as that I’m waiting for some last-minute fixup from Robert Collins. Patch: [1] Blueprint, which is to be moved to Kilo: [2] A

Re: [openstack-dev] oslo.db 1.1.0 released

> On Nov 18, 2014, at 11:47 AM, Sean Dague wrote: > > > Also can I request that when deprecating methods in oslo libraries we > use a standard deprecation mechanism so that warnings are emitted when > this method is used. +1 for DeprecationWarnings, I noticed oslo.db doesn’t seem to use these

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

> On Nov 18, 2014, at 1:38 PM, Eugene Nikanorov wrote: > > Hi neutron folks, > > There is an ongoing effort to refactor some neutron DB logic to be compatible > with galera/mysql which doesn't support locking (with_lockmode('update')). > > Some code paths that used locking in the past were re

Re: [openstack-dev] [oslo.db][nova] NovaObject.save() needs its own DB transaction

> On Nov 19, 2014, at 11:46 AM, Matthew Booth wrote: > > We currently have a pattern in Nova where all database code lives in > db/sqla/api.py[1]. Database transactions are only ever created or used > in this module. This was an explicit design decision: > https://blueprints.launchpad.net/nova/+

Re: [openstack-dev] [oslo.db][nova] NovaObject.save() needs its own DB transaction

> On Nov 19, 2014, at 12:59 PM, Boris Pavlovic wrote: > > Matthew, > > > LOL ORM on top of another ORM > > https://img.neoseeker.com/screenshots/TW92aWVzL0RyYW1h/inception_image33.png > I know where you sta

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

in a way that is atomic come out of that effort “for free” ? A home-rolled “multi master” scenario would have to start with a system that has multiple create_engine() calls, since we need to communicate directly to multiple database servers. From there it gets really crazy. Where’s all tha

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

> On Nov 19, 2014, at 2:58 PM, Jay Pipes wrote: > > >> In other words, the retry logic like the following will not work: >> >> def allocate_obj(): >> with session.begin(subtrans=True): >> for i in xrange(n_retries): >> obj = session.query(Model).filter_by(filters) >>

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

> On Nov 19, 2014, at 3:47 PM, Ryan Moats wrote: > > > > BTW, I view your examples from oslo as helping make my argument for > me (and I don't think that was your intent :) ) > I disagree with that as IMHO the differences in producing MM in the app layer against arbitrary backends (Postgresq

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

> On Nov 19, 2014, at 4:14 PM, Clint Byrum wrote: > > > One simply cannot rely on multi-statement transactions to always succeed. agree, but the thing you want is that the transaction either succeeds or explicitly fails, the latter hopefully in such a way that a retry can be added which has

Re: [openstack-dev] [Neutron] DB: transaction isolation and related questions

ng short >>separate transactions instead of long-lived, multi-statement >>transactions and relying on the behaviour of the DB's isolation >>level (default or otherwise) to "solve" the problem of reading >>changes to a record that you intend to upd

[openstack-dev] Alembic 0.7.0 - hitting Pypi potentially Sunday night

Hi all - I’d like to announce / give a prominent heads up that Alembic 0.7.0 is ready for release. Over the past several weeks I’ve completed the initial implementations for two critical features, SQLite migrations and multiple branch/ merge support, as well as merged over a dozen bug fixes an

Re: [openstack-dev] Alembic 0.7.0 - hitting Pypi potentially Sunday night

> On Nov 21, 2014, at 7:35 PM, Kevin Benton wrote: > > This is great! I'm not sure if you have been following some of the discussion > about the separation of vendor drivers in Neutron, but one of the things we > decided was to leave the vendor data models in the main repo so we have a > nice

Re: [openstack-dev] Alembic 0.7.0 - hitting Pypi potentially Sunday night

> On Nov 21, 2014, at 8:07 PM, Mike Bayer wrote: > > >> On Nov 21, 2014, at 7:35 PM, Kevin Benton > <mailto:blak...@gmail.com>> wrote: >> >> This is great! I'm not sure if you have been following some of the >> discussion about the separ

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 23, 2014, at 6:13 PM, Robert Collins wrote: > > > So - the technical bits of the plan sound fine. > > On WSGI - if we're in an asyncio world, *looks around*, we are? when did that happen?Assuming we’re talking explicit async. Rewriting all our code as verbose, “inside out

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 23, 2014, at 6:35 PM, Donald Stufft wrote: > > > For whatever it’s worth, I find explicit async io to be _way_ easier to > understand for the same reason I find threaded code to be a rats nest. web applications aren’t explicitly “threaded”. You get a request, load some data, manipu

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 23, 2014, at 7:30 PM, Donald Stufft wrote: > > >> On Nov 23, 2014, at 7:21 PM, Mike Bayer wrote: >> >> Given that, I’ve yet to understand why a system that implicitly defers CPU >> use when a routine encounters IO, deferring to other routines,

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 23, 2014, at 8:23 PM, Donald Stufft wrote: > > I don’t really take performance issues that seriously for CPython. If you > care about performance you should be using PyPy. I like that argument though > because the same argument is used against the GCs which you like to use as an > ex

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 23, 2014, at 9:24 PM, Donald Stufft wrote: > > > There’s a long history of implicit context switches causing buggy software > that breaks. As far as I can tell the only downsides to explicit context > switches that don’t stem from an inferior interpreter seem to be “some > particula

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 24, 2014, at 9:23 AM, Adam Young wrote: > > > > For pieces such as the Nova compute that talk almost exclusively on the > Queue, we should work to remove Monkey patching and use a clear programming > model. If we can do that within the context of Eventlet, great. If we need > to

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

> On Nov 24, 2014, at 12:40 PM, Doug Hellmann wrote: > > > This is a good point. I’m not sure we can say “we’ll only use > explicit/implicit async in certain cases" because most of our apps actually > mix the cases. We have WSGI apps that send RPC messages and we have other > apps that recei

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

> On Nov 24, 2014, at 5:20 PM, Michael Still wrote: > > Heya, > > Review https://review.openstack.org/#/c/135644/4 proposes the addition > of a new database for our improved implementation of cells in Nova. > However, there's an outstanding question about how to handle soft > delete of rows --

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

> On Nov 24, 2014, at 7:32 PM, Michael Still wrote: > > Interesting. I hadn't seen consistency between the two databases as > trumping doing this less horribly, but it sounds like its more of a > thing that I thought. it really depends on what you need to do. if you need to get a result set of

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

> On Nov 25, 2014, at 8:15 PM, Ahmed RAHAL wrote: > > Hi, > > Le 2014-11-24 17:20, Michael Still a écrit : >> Heya, >> >> This is a new database, so its our big chance to get this right. So, >> ideas welcome... >> >> Some initial proposals: >> >> - we do what we do in the current nova datab

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

> > Precisely. Why is the RDBMS the thing that is used for archival/audit > logging? Why not a NoSQL store or a centralized log facility? All that would > be needed would be for us to standardize on the format of the archival > record, standardize on the things to provide with the archival rec

Re: [openstack-dev] sqlalchemy-migrate call for reviews

I’ve +2’ed it, it was caused by https://review.openstack.org/#/c/81955/. > On Nov 29, 2014, at 9:54 PM, Davanum Srinivas wrote: > > Looks like there is a review in the queue - > https://review.openstack.org/#/c/111485/ > > -- dims > > On Sat, Nov 29, 2014 at 6:28 PM, Jeremy Stanley wrote: >>

Re: [openstack-dev] sqlalchemy-migrate call for reviews

I can +2 whichever patches are needed by Openstack projects, or that are critically needed in general, that you can point me towards directly. Overall I’m not the “maintainer” of sqlalchemy-migrate, I’ve only volunteered to have a +2 role for critically needed issues, so in the absence of so

[openstack-dev] [neutron] alembic 0.7.1 will break neutron's "heal" feature which assumes a fixed set of potential autogenerate types

hey neutron - Just an FYI, I’ve added https://review.openstack.org/#/c/137989/ / https://launchpad.net/bugs/1397796 to refer to an issue in neutron’s “heal” script that is going to start failing when I put out Alembic 0.7.1, which is potentially later today / this week. The issue is pretty str

Re: [openstack-dev] [neutron] alembic 0.7.1 will break neutron's "heal" feature which assumes a fixed set of potential autogenerate types

e [1]? > > Salvatore > > [1] > http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/alembic_migrations/heal_script.py#n205 > > <http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/alembic_migrations/heal_script.py#n205> > > >

Re: [openstack-dev] [Nova] sqlalchemy-migrate vs alembic for new database

in practice. >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > I don't have experience with Alembic but

[openstack-dev] [all] [ha] potential issue with implicit async-compatible mysql drivers

Hey list - I’m posting this here just to get some ideas on what might be happening here, as it may or may not have some impact on Openstack if and when we move to MySQL drivers that are async-patchable, like MySQL-connector or PyMySQL. I had a user post this issue a few days ago which I’ve sin

[openstack-dev] Mike Bayer 20141205

1. Alembic release - I worked through some regressions introduced by Alembic 0.7.0 and the subsequent 0.7.1 with the Neutron folks. This started on Monday with https://review.openstack.org/#/c/137989/, and by Wednesday I had identified enough small regressions in 0.7.0 that I had to put 0.7.1 o

Re: [openstack-dev] Mike Bayer 20141205

this was sent to the wrong list! please ignore. (or if you find it interesting, then great!) > On Dec 5, 2014, at 6:13 PM, Mike Bayer wrote: > > 1. Alembic release - I worked through some regressions introduced by Alembic > 0.7.0 and the subsequent 0.7.1 with the Neutron

[openstack-dev] [oslo.db] engine facade status, should reader transactions COMMIT or ROLLBACK?

Hi folks - Just a reminder that the majority of the enginefacade implementation is up for review, see that at: https://review.openstack.org/#/c/138215/.Needs a lot more people looking at it. Matthew Booth raised a good point which I also came across, which is of transactions that are “read

Re: [openstack-dev] [all] [ha] potential issue with implicit async-compatible mysql drivers

s fixed in MySQL-connector, which seems to have a similar issue. It’s just so difficult to get responses from MySQL-connector that the PyMySQL thread is at least informative. > > /Ihar > > On 05/12/14 23:43, Mike Bayer wrote: >> Hey list - >> >> I’m posting t

Re: [openstack-dev] [all] [oslo] [ha] potential issue with implicit async-compatible mysql drivers

> On Dec 12, 2014, at 1:16 PM, Mike Bayer wrote: > > >> On Dec 12, 2014, at 9:27 AM, Ihar Hrachyshka wrote: >> >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA512 >> >> Reading the latest comments at >> https://github.com/PyMySQL/PyMySQL/

[openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'

Hello - Victor Sergeyev and I are both observing the following test failure which occurs with all the tests underneath nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.This is against master with a brand new tox environment and everything at the default. It does not seem to be oc

Re: [openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'

cted (0.12 sec) MariaDB [test]> alter table foo change column blah blah int not null; ERROR 1833 (HY000): Cannot change column 'blah': used in a foreign key constraint 'bar_ibfk_1' of table 'test.bar' MariaDB [test]> Matt Riedemann wrote: > > > On 1

Re: [openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'

mystery solved! (at least the part that I didn’t feel like getting into….which I did anyway). Mike Bayer wrote: > working with sdague on IRC, the first thing I’m seeing is that my MariaDB > server is disallowing a change in column that is UNIQUE and has an FK > pointing to it, an

[openstack-dev] [nova] [oslo] compare and swap progress

For those who haven’t seen it, I’d like to first share Jay Pipes’ unbelievably thorough blog post on Nova update concurrency, specifically as it relates to the issue of emitting an UPDATE on a "locked” row without using SELECT..FOR UPDATE (as well as why we *can’t* keep using SELECT..FOR UPDATE)

[openstack-dev] [oslo] [nova] [all] potential enginefacade adjustment - can everyone live with this?

Hey all - Concerning the enginefacade system, approved blueprint: https://review.openstack.org/#/c/125181/ which will replace the use of oslo_db.sqlalchemy.EngineFacade ultimately across all projects that use it (which is, all of them that use a database). We are struggling to find a solution

Re: [openstack-dev] [oslo] [nova] [all] potential enginefacade adjustment - can everyone live with this?

Doug Hellmann wrote: > We put the new base class for RequestContext in its own library because > both the logging and messaging code wanted to influence it's API. Would > it make sense to do this database setup there, too? whoa, where’s that? is this an oslo-wide RequestContext class ? that wo

Re: [openstack-dev] [oslo] [nova] [all] potential enginefacade adjustment - can everyone live with this?

Ihar Hrachyshka wrote: > On 01/23/2015 05:38 PM, Mike Bayer wrote: >> Doug Hellmann wrote: >> >>> We put the new base class for RequestContext in its own library because >>> both the logging and messaging code wanted to influence it's API. Would >&

Re: [openstack-dev] [oslo] [nova] [all] potential enginefacade adjustment - can everyone live with this?

Mike Bayer wrote: > > > Ihar Hrachyshka wrote: > >> On 01/23/2015 05:38 PM, Mike Bayer wrote: >>> Doug Hellmann wrote: >>> >>>> We put the new base class for RequestContext in its own library because >>>> both the logging an

Re: [openstack-dev] [oslo] [nova] [all] potential enginefacade adjustment - can everyone live with this?

Doug Hellmann wrote: > > > On Fri, Jan 23, 2015, at 12:49 PM, Mike Bayer wrote: >> Mike Bayer wrote: >> >>> Ihar Hrachyshka wrote: >>> >>>> On 01/23/2015 05:38 PM, Mike Bayer wrote: >>>>> Doug Hellmann wrote: >>>

Re: [openstack-dev] [nova] proposal for unwinding database usage from tests

Sean Dague wrote: > I've been looking at the following patch series - > https://review.openstack.org/#/c/131691/13 for removing database > requirements from some tests. > > I whole heartedly support getting DB usage out of tests, but I'd like to > make sure that we don't create new challenges

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

; NoSuchTableError: `chassis` > ERROR 1007 (HY000) at line 1: Can't create database 'test_migrations'; > database exists > ProgrammingError: (ProgrammingError) (1146, "Table > 'test_migrations.alembic_version' doesn't exist") > > As far as I can tell, this i

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

On Jun 8, 2014, at 11:46 AM, Roman Podoliaka wrote: > > Overall, the approach with executing a test within a transaction and > then emitting ROLLBACK worked quite well. The only problem I ran into > were tests doing ROLLBACK on purpose. But you've updated the recipe > since then and this can pr

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

On Jun 7, 2014, at 4:38 PM, Eugene Nikanorov wrote: > Hi folks, > > There was a small discussion about the better way of doing sql operations for > vni synchronization with the config. > Initial proposal was to handle those in chunks. Carl also suggested to issue > a single sql query. > I've

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

DDL statement commits the current >> transaction implicitly on MySQL and SQLite AFAIK). >> >> Thanks, >> Roman >> >> [1] https://review.openstack.org/#/c/33236/ >> [2] https://blueprints.launchpad.net/nova/+spec/db-api-tests-on-all-backends >> >

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

On Jun 9, 2014, at 1:08 PM, Mike Bayer wrote: > > On Jun 9, 2014, at 12:50 PM, Devananda van der Veen > wrote: > >> There may be some problems with MySQL when testing parallel writes in >> different non-committing transactions, even in READ COMMITTED mode, >>

Re: [openstack-dev] Gate proposal - drop Postgresql configurations in the gate

On 6/12/14, 8:26 AM, Julien Danjou wrote: > On Thu, Jun 12 2014, Sean Dague wrote: > >> That's not cacthable in unit or functional tests? > Not in an accurate manner, no. > >> Keeping jobs alive based on the theory that they might one day be useful >> is something we just don't have the liberty to

Re: [openstack-dev] mysql/mysql-python license "contamination" into openstack?

On Thu Jun 12 14:13:05 2014, Chris Friesen wrote: > Hi, > > I'm looking for the community viewpoint on whether there is any chance > of license contamination between mysql and nova. I realize that > lawyers would need to be involved for a proper ruling, but I'm curious > about the view of the dev

Re: [openstack-dev] [all] [glance] python namespaces considered harmful to development, lets not introduce more of them

On Sep 23, 2014, at 7:03 PM, Robert Collins wrote: > On 29 August 2014 04:42, Sean Dague wrote: >> On 08/28/2014 12:22 PM, Doug Hellmann wrote: > ... >>> The problem is that the setuptools implementation of namespace packages >>> breaks in a way that is repeatable but difficult to debug when a

Re: [openstack-dev] [Neutron][LBaaS] Migrations in feature branch

If Neutron is ready for more Alembic features I could in theory begin work on https://bitbucket.org/zzzeek/alembic/issue/167/multiple-heads-branch-resolution-support .Folks should ping me on IRC regarding this. On Sep 24, 2014, at 5:30 AM, Salvatore Orlando wrote: > Relying again on auto

Re: [openstack-dev] [nova][cinder] (OperationalError) (1040, 'Too many connections') None None

On Sep 28, 2014, at 5:56 PM, Nader Lahouti wrote: > Hi All, > > I am seeing 'Too many connections' error in nova api and cinder when when > installing openstack using the latest.. > The error happens when launching couple of VMs (in this test around 5 VMs). > > Here are the logs when error ha

Re: [openstack-dev] [nova][cinder] (OperationalError) (1040, 'Too many connections') None None

On Sep 29, 2014, at 12:31 PM, Nader Lahouti wrote: > Hi Jay, > > Thanks for your reply. > > I'm not able to use mysql command line. > $ mysql > ERROR 1040 (HY000): Too many connections > $ > Is there any other way to collect the information? you can try stopping everything, getting on th

Re: [openstack-dev] [nova][cinder] (OperationalError) (1040, 'Too many connections') None None

what’s spooking me is the original paste at http://paste.openstack.org/show/116425/ which showed: icehouse: Fri Sep 26 17:00:50 PDT 2014 Number of open TCP:3306 - 58 Number of open TCP:3306 nova-api - 5 Number of open TCP:3306 mysqld - 29 Number of open TCP:8774 - 10 Number of nova-api - 14 fr

Re: [openstack-dev] [Group-based Policy] Database migration chain

On Oct 4, 2014, at 1:10 AM, Kevin Benton wrote: > Does sqlalchemy have good support for cross-database foreign keys? I was > under the impression that they cannot be implemented with the normal syntax > and semantics of an intra-database foreign-key constraint. cross “database” is not typic

Re: [openstack-dev] [Group-based Policy] Database migration chain

On Oct 4, 2014, at 11:24 AM, Clint Byrum wrote: > > Excerpts from Mike Bayer's message of 2014-10-04 08:10:38 -0700: >> >> On Oct 4, 2014, at 1:10 AM, Kevin Benton wrote: >> >>> Does sqlalchemy have good support for cross-database foreign keys? I was >>> under the impression that they canno

Re: [openstack-dev] [all][oslo][db][docs] RFC: drop support for libpq < 9.1

On Oct 6, 2014, at 9:56 AM, Ihar Hrachyshka wrote: > > > > But we can do better. We should also enforce utf8 on client side, > > so that there is no way to run with a different encoding, and so > > that we may get rid of additional options in sql connection > > strings. I've sent a patch for osl

Re: [openstack-dev] [all][oslo][db][docs] RFC: drop support for libpq < 9.1

On Oct 7, 2014, at 8:29 AM, Ihar Hrachyshka wrote: > > That said, I wonder how we're going to manage cases when those > *global* settings for the whole server should be really limited to > specific databases. Isn't it better to enforce utf8 on service side, > since we already know that we alway

Re: [openstack-dev] [oslo] Meeting time

On Oct 7, 2014, at 3:15 PM, Doug Hellmann wrote: > > I remember talking about that at some point, but I don’t remember why we > decided against it. Since we have several options for earlier in the week > using the existing meeting rooms I think it’s safe to pick one of those. > > I have a sl

Re: [openstack-dev] [all] [oslo] Acceptable methods for establishing per-test-suite behaviors

we put it, but in reality I think it’s fine that it would be placed in more database-test-specific locations. I’d really like to get these patches in so please feel free to review the spec as well as the patches. - mike On Aug 22, 2014, at 3:35 PM, Mike Bayer wrote: > Hi all - >

[openstack-dev] [all] [oslo] Proposed database connectivity patterns

Hi all - I’ve drafted up my next brilliant idea for how to get Openstack projects to use SQLAlchemy more effectively. The proposal here establishes significant detail on what’s wrong with the current state of things, e.g. the way I see EngineFacade, get_session() and get_engine() being used,

Re: [openstack-dev] [all] [oslo] Proposed database connectivity patterns

different context identities in a single call stack, it should raise an exception - not that we can’t support that, but whether it means we should push new state onto the “stack” or not is ambiguous at the moment so we should refuse to guess. On Oct 8, 2014, at 5:07 PM, Mike Bayer wrote

Re: [openstack-dev] [all] [oslo] Proposed database connectivity patterns

On Oct 10, 2014, at 6:13 AM, Ihar Hrachyshka wrote: > Signed PGP part > On 09/10/14 21:29, Mike Bayer wrote: > > So so far, everyone seems really positive and psyched about the > > proposal. > > > > It looks like providing some options for how to use woul

Re: [openstack-dev] [all] [oslo] Proposed database connectivity patterns

On Oct 10, 2014, at 11:41 AM, Mike Bayer wrote: > I’ve been asking a lot about “hey are people cool with thread locals?” and > have been waiting for what the concerns are. > > Since I wrote that email I’ve shifted, and I’ve been considering only: > > @sql.reader &

[openstack-dev] [neutron] [oslo.db] model_query() future and neutron specifics

As I’ve established oslo.db blueprints which will roll out new SQLAlchemy connectivity patterns for consuming applications within both API [1] and tests [2], one of the next big areas I’m to focus on is that of querying. If one looks at how SQLAlchemy ORM queries are composed across Openstack,

[openstack-dev] [oslo.db] Add long-lived-transactionalized-db-fixtures - can we comment ?

Hi all - Thanks for all the responses I got on my "Make EngineFacade a Facade” spec - plenty of people have commented, pretty much all positively so I’m pretty confident we can start building the basic idea of that out into a new review. I want to point out that there is another, closely relat

Re: [openstack-dev] [neutron] [oslo.db] model_query() future and neutron specifics

> On Oct 23, 2014, at 11:27 AM, Kyle Mestery wrote: > > Mike, first, thanks for sending out this detailed analysis. I'm hoping > that some of the DB experts from the Neutron side have read this. > Would it make sense to add this to our weekly meeting [1] for next > week and discuss it during the

[openstack-dev] [oslo][all] Alembic migrations for SQLite

Bonjour openstackers - While you were all sipping champagne on the Champs-Élysées, I took some time to tackle one of the two most critically wanted features in Alembic, which is that of being able to migrate tables on a SQLite database with some degree of sanity. My immediate focus on Alembic

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

> On Nov 12, 2014, at 12:45 PM, Dan Smith wrote: > >> I personally favour having consistent behaviour across the board. How >> about updating them all to auto-refresh by default for consistency, >> but adding an additional option to save() to disable it for particular >> calls? > > I think thes

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

> On Nov 12, 2014, at 10:56 AM, Matthew Booth wrote: > > For brevity, I have conflated what happens in object.save() with what > happens in db.api. Where the code lives isn't relevant here: I'm only > looking at what happens. > > Specifically, the following objects refresh themselves on save: >

  1   2   3   4   >