> I’m not sure if I’m seeing the second SELECT here either but I’m less
> familiar with what I’m looking at. compute_node_update() does the
> one SELECT as we said, then it doesn’t look like
> self._from_db_object() would emit any further SQL specific to that
> row.
I don't think you're missin
> On 12/11/14 19:39, Mike Bayer wrote:
>> lets keep in mind my everyone-likes-it-so-far proposal for reader()
>> and writer(): https://review.openstack.org/#/c/125181/ (this is
>> where it’s going to go as nobody has -1’ed it, so in absence of any
>> “no way!” votes I have to assume this is what
> Can we guarantee that the lifetime of a context object in conductor is
> a single rpc call, and that the object cannot be referenced from any
> other thread?
Yes, without a doubt.
--Dan
signature.asc
Description: OpenPGP digital signature
___
OpenS
> Yep, it is possible to run the tests inside VMs - the key is that when
> you create the VMs you need to be able to give them NUMA topology. This
> is possible if you're creating your VMs using virt-install, but not if
> you're creating your VMs in a cloud.
I think we should explore this a bit mo
> That sounds like something worth exploring at least, I didn't know
> about that kernel build option until now :-) It sounds like it ought
> to be enough to let us test the NUMA topology handling, CPU pinning
> and probably huge pages too.
Okay. I've been vaguely referring to this as a potential
>> Unfortunately this model doesn't apply to Nova objects, which are
>> persisted remotely. Unless I've missed something, SQLA doesn't run
>> on Nova Compute at all. Instead, when Nova Compute calls
>> object.save() this results in an RPC call to Nova Conductor, which
>> persists the object in the
> I don't think it's a problem. It puts a practical limit on the scope
> of an 'api call' which can be covered by a single database transaction
> though, because it would be difficult to arrange for 2 RPC calls to
> both use the same DB transaction on the remote end. I think we agree
> on this.
Su
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
> What if 3rd Party CI didn't vote in Gerrit? What if it instead
> published to some 3rd party test reporting site (a thing that
> doesn't yet exist). Gerrit has the facility so that we could inject
> the dashboard content for this in Gerrit in a littl
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
> There is a similar old bug for that, with a good suggestion for how
> it could possibly be done:
>
> https://bugs.launchpad.net/openstack-ci/+bug/1251758
This isn't what I'm talking about. What we need is, for each new
patchset on a given change, a
> I'm contemplating how to fix
> https://bugs.launchpad.net/nova/+bug/1339823 and it seems that a part of
> the fix would be to track the state of live migrations in the database,
> more or less the same way that cold migrations are tracked. The
> thinking is that the logic could retrieve informat
>> Based on these experiences, libvirt version differences seem to be as
>> substantial as major hypervisor differences.
>
> I think that is a pretty dubious conclusion to draw from just a
> couple of bugs. The reason they really caused pain is that because
> the CI test system was based on old ve
> In addition to seeking a spec-freeze-exception for 95025, I would also
> like some clarification of the requirement to test this upgrade
> path. Some nova-core folks have pointed out that they do not want to
> accept the nova.virt.ironic driver until the upgrade path from
> nova.virt.baremetal *h
> We've already approved many other blueprints for Juno that involve features
> from new libvirt, so I don't think it is credible to reject this or any
> other feature that requires new libvirt in Juno.
>
> Furthermore this proposal for Nova is a targetted feature which is not
> enabled by default
>> I'm just do not suppor the idea that Nova needs to change its
>> fundamental design in order to support the *design* of other host
>> management platforms.
>
> The current implementation doesn't make nova change its design, the
> scheduling decisions are still done by nova.
Nova's design is
> Hmm, that is not right. There's no intention to remove the vif_driver
> parameter itself. We were supposed to merely deprecate the various
> legacy VIF driver implementations in Nova, not remove the ability
> to use 3rd party ones.
I'm pretty sure it was deprecated specifically for that reason.
> I don't see an issue with allowing people to configure 3rd party impl
> for the VIF driver, provided we don't claim that the VIF driver API
> contract is stable, same way we don't claim virt driver API is stable.
> It lets users have a solution to enable custom NIC functionality while
> waiting f
> If we're going to do that, then we should be consistent. eg there is
> a volume_drivers parameter that serves the same purpose as
> vif_driver
There are lots of them. We've had a bit of a background task running to
remove them when possible/convenient and try to avoid adding new ones.
I'm not op
> FWIW, I do actually agree with not exposing plugin points to things
> that are not stable APIs and if they didn't already exist, I'd not
> approve adding them. I'd actually go further and say not even the
> virt driver API should be a plugin point, since we arbitrarily change
> it during developm
> When reviewing https://review.openstack.org/#/c/107954/ it occurred to
> me that maybe we should consider having some kind of generic object
> wrapper that could do notifications for objects. Any thoughts on this?
I think it might be good to do this in a repeatable, but perhaps not
totally autom
> Please respond with +1s or any concerns.
+1.4142135623
--Dan
signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> The second option would be to make a copy of the old ImageCacheManager
>> in the Baremetal directory, and have the Baremetal driver
>> use that. This seems to me to be the better option, since it means
>> that when the Baremetal driver is removed, the old ImageCacheManager
>> code goes with it,
> That's different behaviour, because you can no longer pass arg=None. The
> fix isn't to change the behaviour of the code.
So use a sentinel...
--Dan
signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@l
> What criteria would be used to determine which drivers stay in-tree
> vs. maintained as forks? E.g. libvirt driver in, everyone else out?
> Open-platform drivers (libvirt, xen) in, closed-platform drivers
> (vmware, hyperv) out? Drivers for platforms with large (possibly
> non-OpenStack) producti
> 4. What is the max amount of time for us to report test results? Dan
> didn't seem to think 48 hours would fly. :)
Honestly, I think that 12 hours during peak times is the upper limit of
what could be considered useful. If it's longer than that, many patches
could go into the tree without a vot
> We could either have a single repo:
>
> openstack/nova-extra-drivers
This would be my preference for sure, just from the standpoint of
additional release complexity otherwise. I know it might complicate how
the core team works, but presumably we could get away with just having
driver mainta
> I think that all drivers that are officially supported must be
> treated in the same way.
Well, we already have multiple classes of support due to the various
states of testing that the drivers have.
> If we are going to split out drivers into a separate but still
> official repository then we
> My only request here is that we can make sure that new driver
> features can land for other drivers without necessarilky having them
> implemented for libvirt/KVM first.
We've got lots of things supported by the XenAPI drivers that aren't
supported by libvirt, so I don't think this is a problem
> If the idea is to gate with nova-extra-drivers this could lead to a
> rather painful process to change the virt driver API. When all the
> drivers are in the same tree all of them can be updated at the same
> time as the infrastructure.
Right, and I think if we split those drivers out, then we
> From the user perspective, splitting off the projects seems to be
> focussing on the ease of commit compared to the final user
> experience.
I think what you describe is specifically the desire that originally
spawned the thread: making the merging of changes to the hyper-v driver
faster by ha
> 4) Periodically, code from the new project(s) must be merged into Nova.
> Only Nova core reviewers will have obviously +2a rights here.
> I propose to do it on scheduled days before every milestone, differentiated
> per driver to distribute the review effort (what about also having Nova core
>
> The last thing that OpenStack needs ANY more help with is velocity. I
> mean, let's be serious - we land WAY more patches in a day than is
> even close to sane.
Thanks for saying this -- it doesn't get said enough. I find it totally
amazing that we're merging 34 changes in a day (yesterday) whic
> +1 - I think we really want to have a strong preference for a stable
> api if we start separating parts out
So, as someone who is about to break the driver API all to hell over the
next six months (er, I mean, make some significant changes), I can tell
you that making it stable is the best way
> This system is running tempest against a VMWare deployment and posting
> the results publicly. This is really great progress. It will go a long
> way in helping reviewers be more confident in changes to this driver.
This is huge progress, congrats and thanks to the VMware team for making
this
> As I don't see how to "keep it in the review", I'll copy to openstack-dev.
Just keep making your comments in Gerrit. That way all the discussion
related to a specific patch is preserved with proper linkage in case we
ever need to go back to it.
> OK, I think I see what I need to do to do to not
> In the last meeting we discussed an idea that I think is worth trying at
> least for icehouse-1 to see if we like it or not. The idea is that
> *every* blueprint starts out at a Low priority, which means "best
> effort, but no promises". For a blueprint to get prioritized higher, it
> should ha
> Example 1:
> ==
>
> n-conductor log in tempest/devstack -
> http://logs.openstack.org/70/52870/3/check/check-tempest-devstack-vm-full/f46b756/logs/screen-n-cond.txt.gz
>
>
> Total log lines: 84076
> Total non DEBUG lines: 61
>
> Question: do we need more than 1 lev
> If we had DEBUG and DEBUG2 levels, where one of them would only be seen
> at the higher debug level, would that be useful?
I'm fine with not seeing those for devstack runs, yeah.
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
> 1) I agree with Russell that Nova should not try to manage VMs it didn't
> create.
Me too. A lot.
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L420
>
> https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/models.py#L43
This API and these models are what we are trying to avoid exposing to
the rest of nova. By wrapping these in our NovaObject-based s
> 1) Using objects as an abstraction for versioned data:
>This seems like a good direction overall, especially from the point-of-view
>of backwards compatibility of consumers of the object. However, after
> looking through some
>of the objects defined in nova/objects/, I am not sure if
> I'd like us to avoid meaningless reviewer churn here:
I'd like us to avoid trivial style guideline churn :)
> The case that made me raise this is this:
> folder_exists, file_exists, file_size_in_kb, disk_extents = \
> self._path_file_exists(ds_browser, folder_path, file_name)
>
> W
>> You're right, it's not really achievable without moving to a schemaless
>> persistence model. I'm fairly certain it was added to be humorous and
>> should not be considered an outcome of that session.
>
> But we can avoid most data migrations by adding any required
> conversion code into the o
> I assume this back support include also object and the N means major release
> like H, I, right?
Yes, I meant N=Icehouse, N-1=Havana, for example.
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-
> - Each third party plugin/driver shall designate a point of contact for
> the coordinated release cycle.
> - To be designated as compatible, a third-party plugin and/or driver
> code must implement external third party integration testing.
> - Policy is in effect immediately for new plugin/driver
Hi all,
As you know, Nova adopted a plan to require CI testing for all our
in-tree hypervisors by the Icehouse release. At the summit last week, we
determined the actual plan for deprecating non-compliant drivers. I put
together a page detailing the specific requirements we're putting in
place as
> You're not missing anything. But I think that's a bug, or at least an
> unexpected change in behaviour from how it used to work. If you follow
> instance_update() in nova.db.sqlalchemy.api just the presence of
> expected_task_state triggers the check. So we may need to find a way to
> pass tha
> Thanks for weighing in, I do hope to keep the conversation going.
Add my name to the list of people that won't be considering the trip as
a result.
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi
> Another issue that we might want to try and solve with this (and imho
> another reason to keep the framework for this in oslo) is that we might
> want to make these update aware, and allow for some graceful degradation
> or something similar when for example an updated service is stated with
> an
> To be fair, we test only the subset that is set via devstack on the
> stable side. That should be a common subset, but it is far from fully
> comprehensive. Nova has over 600 config variables, so additional tooling
> here would be goodness.
I'm surely not arguing against additional testing of co
Sorry for the delay in responding to this...
> * Moved the _obj_classes registry magic out of ObjectMetaClass and into
> its own method for easier use. Since this is a subclass based
> implementation,
> having a separate method feels more appropriate for a factory/registry
> patte
> Not having been at the summit (maybe the next one), could somebody
> give a really short explanation as to why it needs to be a separate
> service? It sounds like it should fit within the Nova area. It is,
> after all, just another hypervisor type, or so it seems.
But it's not just another hyper
> I think the plus of "avoiding decorating things" isn't really a huge
> win, and actually i think takes clarity away.
Hence the "(meh)" in my list :)
This wasn't really a sticking point when we were getting reviews on the
original base infrastructure, so I'm surprised people are so vehement
now.
> You can always got out to zuul directly and see patch status in
> progress
> - http://status.openstack.org/zuul/
Or, you can run dash.py, which will show you a personalized view of
zuul:
https://github.com/kk7ds/openstack-gerrit-dashboard
Running it with a refresh of thirty seconds or so will
> Just wondering what people thought about how necessary it is to keep
> XML support for the Nova v3 API, given that if we want to drop it
> doing so during the v2->v3 transition is pretty much the ideal time
> to do so.
Personally, I think we ought to drop it while we have the chance :)
--Dan
_
> Please respond with +1s or any concerns.
+1 from me.
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> We have two new compute drivers in the queue for Havana: docker [1]
> and z/vm [2]. I'd like to propose as a piece of criteria for
> inclusion that new drivers go into groups A or B.
I think this is a really good idea. As we continue to absorb new and
more complex drivers into the tree, the amo
> Please respond with +1s or any concerns.
My only concern is that now he and Mikal can collude on patches while
the rest of us are asleep. Actually, maybe that's okay.
+1 from me :)
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.o
> In the corner to my left, our current largest gate reset culprit
> appears to be neutron bug #1194026 - weighing in with 62 rechecks
> since June 24th (http://status.openstack.org/rechecks/)
So, with some of the highest rates of patch traffic we've seen over the
last couple of weeks before the H
> Currently cells can even get past devstack exercises, which are very
> minor sanity checks for the environment (nothing tricky).
I thought that the plan was to deprecate the devstack exercises and
just use tempest. Is that not the case? I'd bet that the devstack
exercises are just not even on a
> In the original context of using Conductor as a database proxy then
> the number of conductor instances is directly related to the number
> of compute hosts I need them to serve.
Just a point of note, as far as I know, the plan has always been to
establish conductor as a thing that sits between
> > I'd like to see if there are any opinions on if this should come
> > back as 'SOFT_DELETED' or if everyone is OK with mapping
> > soft-delete to 'DELETED' in the v3 API?
>
> I would like to see them merged. Having multiple kinds of deleted
> records is really confusing, and leads to bugs. T
> The affiliation of core team members should not come into a decision
> like this.
>
> It is assumed that all core team members are wearing their "upstream
> hat" and aren't there merely to represent their employers interests.
Mark beat me to it, but.. Yeah, what he said. Core members aren't
inv
> There's nothing I've seen so far that causes me alarm, but then
> again we're in the very early stages and haven't moved anything
> really complex.
The migrations (live, cold, and resize) are moving there now. These are
some of the more complex stateful operations I would expect conductor
to ma
> I had assumed that some of the task management state would exist
> in memory. Is it all going to exist in the database?
Well, our state is tracked in the database now, so.. yeah. There's a
desire, of course, to make the state transitions as
idempotent/restartable as possible, which may mean driv
> IIUC, Ceilometer is currently a downstream consumer of data from
> Nova, but no functionality in Nova is a consumer of data from
> Ceilometer. This is good split from a security separation point of
> view, since the security of Nova is self-contained in this
> architecture.
>
> If Nova schedular
> Nova-conductor is the gateway to the database for nova-compute
> processes. So permitting a single nova-conductor process would
> effectively serialize all database queries during instance creation,
> deletion, periodic instance refreshes, etc.
FWIW, I don't think anyone is suggesting a single
> I propose that we add Nikola Đipanov to the nova-core team [1].
>
> Please respond with +1/-1.
+1
--Dan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> Any solution where you need to modify sudoers every time the code
> changes is painful, because there is only one sudo configuration on a
> machine and it's owned by root.
Hmm? At least on ubuntu there is a default /etc/sudoers.d directory,
where we could land per-service files like nova-compute
> While it's true that there won't be an in-tree driver that supports
> the API for this release cycle, we have a commercial driver that
> supports it ( https://github.com/gridcentric/cobalt).
IMHO, out of tree virt drivers are completely out of scope here. We
change the virt driver API at will, a
Hi all,
As someone who has felt about as much pain as possible from the
dual-maintenance of the v2 and v3 API extensions, I felt compelled to
bring up one that I think we can drop. The baremetal extension was
ported to v3 API before (I think) the decision was made to make v3
experimental for Havan
> Because we landed a patch to tox upstream to use setup.py develop
> instead of sdist+install like our run_tests.sh scripts do - this means
> that with the new tox config changes, tox runs should be just as quick
> as run_tests.sh runs.
So. Freaking. Awesome.
--Dan
_
> Could we please do a FFE for this one has been waiting for a long
> time and we have done all that we were asked relatively quickly
> in H2 it was gated by API object refactor. Most of the current
> comments (pre last one by Dan) are due to
> https://bugs.launchpad.net/nova/+bug/1221396
> I think conductor-workers [1] should be an exception to the feature
> freeze for H3. The risk is very low since the default value of this
> configuration option doesn't change nova's behavior at all. Moreover,
> the patch [2] is small and uses common OpenStack code. The reward is
> high since it
> I propose a NACK on the FFE, and instead going with the above plan.
I agree, especially with Thierry's concerns. This is a lot of deep
change for little user-visible benefit (user-visible release notes
notwithstanding!)
--Dan
___
OpenStack-dev mailin
> This blueprint, implemented during the H3 timeframe
I remember this being a hot topic at the end of the previous cycle, and
the last summit. Thus, I was surprised to see it make such a late
entrance this cycle. Since it has seen little review thus far, I don't
think we should expect to squeeze i
> I have been trying to add a new table to the database.
Just FYI, new database interfaces should be added using objects, which
are automatically remotable. New conductor interfaces should not be
added to make database queries.
> #In class LocalAPI
> def xx(self, context, node_id, values):
>
> I propose we add Sylvain Bauza[1] to nova-core.
>
> Over the last few cycles he has consistently been doing great work,
> including some quality reviews, particularly around the Scheduler.
>
> Please respond with comments, +1s, or objections within one week.
+1 for tasty cheese.
--Dan
__
> Worth mentioning that OpenStack releases that come out at the same time
> as Ubuntu LTS releases (12.04 + Essex, 14.04 + Icehouse, 16.04 + Mitaka)
> are supported for 5 years by Canonical so are already kind of an LTS.
> Support in this context means patches, updates and commercial support
> (for
>> 1. Nova bootstrap will be called and it will perform db-migration. Since
>> current approach to nova code is add-only we shouldn't need to stop
>> services and old services should keep working on newer database. Also
>> for minor version upgrades there will be no action here unless there is
>> m
> So Dan, correct me if I'm wrong, but after this merges (we're working
> on L->M upgrade) we can simply go with version=auto on every node and
> just hit SIGHUP on conductors once we're done with all of compute
> nodes right?
You need to hit all the nodes with SIGHUP. The api, conductor and
compu
> Can we get another nova-core to take a final look at the above patch and
> approve it? That would get the Hyper-V CI tests passing for all open
> Nova patches.
Yep, I was following it all last week waiting for that CI pass report. I
would have been on this already, but unlike you, I know how to
> I like the idea of checking the md5 matches before each boot, as it
> mirrors the check we do after downloading from glance. Its possible
> thats very unlikely to spot anything that shouldn't already be worried
> about by something else. It may just be my love of symmetry that makes
> me like tha
> There was a conversation earlier this week in which, if I understood
> things correctly, one of the possible outcomes was that it might make
> sense for the new placement service (which will perform the function
> currently provided by the scheduler in Nova) to only get used over its
> REST API,
> I believe the only difference between your thoughts on this and my
> own are the implementation details of how those placement HTTP API
> calls would be made. I believe you want to see those calls done in
> the nova.objects.Inventory[List] object whereas I was hoping to have
> the resource tr
> According to the state of this review:
> https://review.openstack.org/#/c/317689/ the works aren't going to be
> done in this cycle.
This is a procedural -2 waiting for all the following patches to be
reviewed and passing 3rd party CI before we land them. We certainly
expect to get this work int
> I'm going to have to agree with Jay - and I believe with something Clint
> said earlier.
>
> Use metadata/config-drive to bootstrap you. Then use your actual tools
> (puppet/chef/ansible/salt/juju/cfengine) to interact with the node once
> it's enrolled in whatever you use to orchestrate cloud t
> For those deprecated options, does it mean it won’t be supported in
> Mitaka and future release at all? Or is there a grace period so that
> we, as a user, can gradually transition from Liberty to Mitaka?
The deprecation period is usually one cycle for something like this.
That means people ge
> Just curious - what is the motivation of removing the plug-ability
> entirely? Because of significant maintenance effort?
It's not a plugin interface and has never been stable. We've had a
long-running goal of removing all of these plug points where we don't
actually expect people to write stabl
> The question is whether we should do something like this:
>
> 1) As part of the normal execution of the service playbooks;
> 2) As part of the automated major upgrade (i.e. The step is not optional);
> 3) As part of the manual major upgrade (i.e. The step is optional);
> 4) Never.
I am not an o
Just as a heads up, we are removing the default flavors from nova in
this patch:
https://review.openstack.org/#/c/300127/
Since long ago, Nova's default flavors were baked in at the database
migration level. Now that we have moved them to another database
entirely, this means we have to migrate
> It still was more common then not I think? So making it less common
> is probably a step on the wrong direction.
The responses on the operators mailing list were 100% positive for removal.
As Dan said, calling these a standard is really not reasonable. They're
just defaults, copied from AWS yea
> I haven't researched this particular case in detail, so I could be
> misunderstanding its implications. But in general my impression,
> from the conversations that occur when these topics are raised, is
> that many prominent OpenStack developers do not care enough about
> release-to-release
> Ops != Cloud App developers. We've really made the latter's job hard,
> and are often increasing the difficulty on them. This pushes them
> away from OpenStack and eliminates a lot of potential users of
> OpenStack, meaning Ops have fewer users then they should. Lets not
> continue this trend. :/
> * nova-api should receive an acknowledgement from nova-compute. It is
> unclear to me why today it uses a non-reply mechanism - probably to
> free the worker as fast as it can.
Yes, wherever possible, we want the API to return immediately and let
the action complete later. Making a who
> In July? Oregon > San Antonio
In any month of any year, Oregon > San Antonio :P
+1 for Hillsboro again from me, but I'm just a tad biased :)
--Dan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscrib
>> I have wanted to make a change for a while that involves a TTL on
>> messages, along with a deadline record so that we can know when to retry
>> or revert things that were in flight. This requires a lot of machinery
>> to accomplish, and is probably interwoven with the task concept we've
>> had
> However I do want to point out that cells v2 is not just about dealing
> with scale in the database. The message queue is another consideration,
> and as far as I know there is not an analog to the "distributed
> database" option available for the persistence layer.
Yeah, it's actually *mostly*
> This would be a very interesting direction to explore. Focus on the pain
> points of the message queue and then look at addressing the beast that
> is the database layer separately. I am going to toss support in behind a
> lot of what has been said in this thread already. But I really wanted to
>
> I'm currently working on the spec for Project ID Validation in Nova
> using Keystone. The outcome of the Design Summit Session was that the
> Nova service user would use the Keystone policy to establish whether the
> requester had access to the project at all to verify the id. I was
> wondering i
>> Hmm... that's unfortunate, as we were trying to get some of our less
>> ephemeral items out of random etherpads and into the wiki (which has the
>> value of being google indexed).
Yeah, I'm kinda surprised anyone would consider a wiki-less world. I'm
definitely bummed at the thought of losing i
101 - 200 of 371 matches
Mail list logo
