On 23 December 2015 at 12:01, Steven Hardy <sha...@redhat.com> wrote:
> On Tue, Dec 22, 2015 at 03:36:02PM +0000, Dougal Matthews wrote: > > Hi all, > > > > This topic came up in the 2015-12-15 meeting[1], and again briefly > today. > > After working with the code that came out of the deployment library > > spec[2] I > > had some concerns with how we are storing the templates. > > > > Simply put, when we are dealing with 100+ files from > > tripleo-heat-templates > > how can we ensure consistency in Swift without any atomicity or > > transactions. > > I think this is best explained with a couple of examples. > > > > Â - When we create a new deployment plan (upload all the templates to > > swift) > > Â Â how do we handle the case where there is an error? For example, > if we > > are > > Â Â uploading 10 files - what do we do if the 5th one fails for some > > reason? > > Â Â There is a patch to do a manual rollback[3], but I have concerns > > about > > Â Â doing this in Python. If Swift is completely inaccessible for a > short > > Â Â period the rollback wont work either. > > How does using a different data store help fix this error path? > > Regardless of the Swift/DB/Git choice, you have to detect something failed, > and mark the plan as in a failed state. > > > Â - When deploying to Heat, we need to download all the YAML files > from > > Swift. > > Â Â This can take a couple of seconds. What happens if somebody > starts to > > Â Â upload a new version of the plan in the middle? We could end up > > trying to > > Â Â deploy half old and half new files. We wouldn't have a consistent > > view of > > Â Â the database. > > If this happens, the API design is wrong - we should have a plan reference > a unique identifier, including a version (e.g through swift versioned > objects, git tags/branches or whatever), then on deployment heat should > download exactly one version of those artefacts (e.g via a swift tempurl or > a git URL, or whatever). > Swift gives us no way to get the current version. In a versioned container, the current version is referenced by the file path only, the previous versions are the file path with a timestamp. So there is no way to reference an object and not see updates unless we start tracking timestamps etc. in code. I hope I am understanding it correctly, this was one of the primary reasons that we discounted Swift as an option for Tuskar when I was looking into versioning around a year ago. I've had a look again and it seems the situation hasn't changed (but I will double check now). > > FYI heatclient already supports downloading template objects directly from > swift, and heat/heatclient already support downloading from http URLs, so > all we have to do AFAICS is generate a URL pointing to a consistent > version of the templates. > > > We had a few suggestions in the meeting: > > > > Â - Add a locking mechanism. I would be concerned about deadlocks or > > having to > > Â Â lock for the full duration of a deploy. > > I think we need to create a new version of a plan every time it's modified, > then on deployment the concern around locking is much reduced, or even > eliminated? > > > Â - Store the files in a tarball (so we only deal with one file). I > think > > we > > Â Â could still hit issues with multiple operations unless we > guarantee > > that > > Â Â only one instance of the API is ever running. > > > > I think these could potentially be made to work, but at this point > are we > > using the best tool for the job? > > > > For alternatives, I see a can think of a couple of options: > > > > - Use a database, like we did for Tuskar and most OpenStack API's do. > > Storing template files in a database with Tuskar was a mistake - we should > *not* do that again IMO - it was a very opaque interface and caused a lot > of confusion in use. > > Storing code (in this case yaml files) in some sort of versioned repository > is *such* a solved problem - it'll be total wheel-reinventing if we > implement template revision control inside a bespoke DB IMHO. > > I think swift is probably a workable solution (using versioning and > tempurl's), but we should really consider making the template store > pluggable, as having the API backed by an (operator visible) git repo is > likely to be a nicer and more familiar solution. > > The question around validation data I'm less clear on - we already store > discovery data in swift, which seems to work OK - how is the validation > data different, in that it warrants a bespoke DB data store? > > Steve > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
