On 06/02/2015 03:14 PM, Sylvain Bauza wrote: > Hi, > > Currently working on implementing the RequestSpec object BP [1], I had > some cool comments on my change here : > https://review.openstack.org/#/c/145528/12/nova/objects/request_spec.py,cm > > Since we didn't discussed on how to persist that RequestSpec object, I > think the comment is valuable. > > For the moment, the only agreed spec for persisting the object that we > have is [2] but there is also a corollar here that means that we would > have to persist more than the current fields > https://review.openstack.org/#/c/169901/3/specs/liberty/approved/add-buildrequest-obj.rst,cm > > > So, there are 2 possibilities : > #1, we only persist the RequestSpec for the sole Scheduler and in that > case, we can leave as it is - only a few fields from Instance are stored > #2, we consider that RequestSpec can be used for more than just the > Scheduler, and then we need to make sure that we will have all the > instance fields then. >
So these are 2 possibilities if we agree that we need to make progress on the spec as is defined and merged now. What I was complaining yesterday is that we don't seem to have done enough of high level investigation into this stuff before embarking on writing a set of specs that then due to their format obscure the problems we are actually trying to solve. Work around the scheduler touches on a lot of issues that have only recently been noticed. While I am all for the incremental approach, it seems silly to completely disregard the issues we already know about. We should have a high level overview of the problems we know we want to solve, and then come up with an incremental way of solving them, but not without keeping an eye on the big picture at all times. An ad-hoc list of individual issues that we know about and should be trying to solve (in no particular order) that all seem related to the data model design problem we are trying to take a stab at here: 1/ RequestSpec is an unversioned dict even though it's the central piece of a placement request for the scheduler 2/ There are scheduler_hints that are needed throughout the lifecycle of an instance but are never persisted so are lost after boot 3/ We have the Migration objects that are used both for resource tracking for instances being migrated, and as an indication of an instance being moved, but are not used in all the places we need this kind of book keeping (live migration, rebuild) 4/ Evacuate (an orchestrated rebuild) is especially problematic because it usually involves failure modes, which are difficult to identify and handle properly without a consistently used data model. 5/ Some of the recently added constraints that influence resource tracking (NUMA, CPU pinning) cannot simply be calculated from the flavor on the fly when tracking resources, but need to be persisted after a successful claim as they are dependent on the state of the host at that very moment (see [1]) 6/ Related to the previous one - there is data related to the instance in addition to the flavor that need to follow the '_old' and '_new' pattern (needs the values related to both source and destination host persisted during a migration/resize/live migration/) 7/ The issues cells v2 folks are hitting (mentioned above) where they don't want to have any Instances in the top level cell but still need to persist stuff. 8/ Issues with having no access to individual instance UUIDs in the scheduler, but a lot of data access for more complex filtering revolves around it being present. Most of the above have individual bugs that I can try to find and link here too. [1] https://bugs.launchpad.net/nova/+bug/1417667 The overall theme of all the above is (to paraphrase alaski from IRC) how to organize the big blob of data that is an instance in all of it's possible states, in such a way that it makes sense, nothing is missing, there is as little duplication as possible, and access patterns of different services that require different bits can work without massive overhead. > > I'm not strongly opiniated on that, I maybe consider that #2 is probably > the best option but there is a tie in my mind. Help me figuring out > what's the best option. > If we want to keep things moving forward on this particular BP - I'd go with adding the RequestSpec object and make sure the code that uses it is migrated. I believe that spike alone will leave us with much better idea about the problem. In addition - writing a high level spec/wiki that we can refer back to in individual BPs and see how they solve it would be massively helpful too. N. > -Sylvain > > [1] : > http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/request-spec-object.html > > [2] : > http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/persist-request-spec.html > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
