> -----Original Message----- > From: Edison Su [mailto:edison...@citrix.com] > Sent: Monday, May 20, 2013 2:30 PM > To: dev@cloudstack.apache.org > Subject: RE: [MERGE]object_store branch into master > > > > > -----Original Message----- > > From: John Burwell [mailto:jburw...@basho.com] > > Sent: Monday, May 20, 2013 12:56 PM > > To: dev@cloudstack.apache.org > > Subject: Re: [MERGE]object_store branch into master > > > > All, > > > > Since this change is so large, it makes reviewing and commenting in > > detail extremely difficult. Would it be possible to push this patch > > through Review Board to ease comprehension and promote a conversation > about this patch? > > We can try to push it into Review Board.
The review board url is: https://reviews.apache.org/r/11277/, 25 pages... > > > > > Reading through the FS, I have the following questions regarding the > > operation of the NFS cache: > > > > What happens if/when the disk space of the NFS cache is exhausted? > > What are the sizing recommendations/guidelines for it? > > What strategy is used to age files out of the NFS cache? > As usual, admin can have multiple NFS secondary storages, admin can also > add multiple NFS cache storages. The NFS cache storage capacity plan will be > the same as NFS secondary storage. > If there multiple NFS cache storages, the current strategy will randomly > choose one of them. Currently, no clean up/aging out strategy implemented > yet But the situation can be improved: most of cached object can be deleted > after accessed once. Take template as example, if zone wide storage is used, > put template on cache storage has little value, as once the template is > downloaded into primary storage, suddenly all the hypervisor host can access > it. > I think the simple LRU algorithm to delete cached objects should be enough. > It can be added later, the cache storage has its own pom project, it's place > to > add more intelligence. > > > If two processes, process1 and process2, are both using a template, > > templateA, will both processes reference the same file in the NFS > > cache? If > It's possible, that one template can be downloaded into cache storage twice, > in case of concurrent accessed by two processes. The current > implementation is that, if two processes want to download the same > template from s3 into one primary storage at the same time, there is only > one template will be downloaded into cache storage. While, if two processes > want to download the same template into different primary storage, the > template will be cached twice. > > they are reading from the same file and process1 finishes before > > process2, will process1 attempt to delete process2? > > There is no way to delete while read, as each cached object has its own state > machine. If it's accessed by one process, the state will be changed to > "Copying", you can't delete an object when it's in "Copying" state. > > > If a file transfer from the NFS cache to the object store fails, what > > is the recovery/retry strategy? What durability guarantees will > > CloudStack supply when a snapshot, template, or ISO is in the cache, > > but can't be written to the object store? > > The error handling of cache storage shouldn't be different than without > cache storage. For example, directly backup snapshot from primary storage > to S3, without cache storage. If backup failed, then the whole process failed, > user needs to do it again through cloudstack API. So in cache storage case, if > push object from cache storage to s3 failed, then the whole backup process > failed. > > > What will be the migration strategy for the objects contained in S3 > > buckets/Swift containers from pre-4.2.0 instances? Currently, > > CloudStack tracks a mapping between these objects and templates/ISOs > > in the template_switt_ref and template_s3_ref table. > > We need to migrate DB from existing template_s3_ref to > template_store_ref, and put all the s3 information into image_store and > image_store_details tables. > > > > > Finally, does the S3 implementation use multi-part upload to transfer > > files to the object store? If not, the implementation will be limited > > to storing files no larger than 5GB in size. > Oh, this is something we don't know yet. We haven't try to upload a > template which is large than 5GB, so haven't met this issue. > Could you help to hack it up?:) > > > > > Thanks, > > -John > > > > On May 20, 2013, at 1:52 PM, Chip Childers <chip.child...@sungard.com> > > wrote: > > > > > On Fri, May 17, 2013 at 08:19:57AM -0400, David Nalley wrote: > > >> On Fri, May 17, 2013 at 4:11 AM, Edison Su <edison...@citrix.com> > wrote: > > >>> Hi all, > > >>> Min and I worked on object_store branch during the last one > > >>> and half > > month. We made a lot of refactor on the storage code, mostly related > > to secondary storage, but also on the general storage framework. The > > following goals are made: > > >>> > > >>> 1. An unified storage framework. Both secondary > > storages(nfs/s3/swift etc) and primary storages will share the same > > plugin model, the same interface. Add any other new storages into > > cloudstack will much easier and straightforward. > > >>> > > >>> 2. The storage interface between mgt server and resource is > unified, > > currently there are only 5 commands send out by mgt server: > > > copycommand/createobjectcommand/deletecommand/attachcommand/de > > ttachcommand, and each storage vendor can decode/encode all the > > entities(volume/snapshot/storage pool/ template etc) by its own. > > >>> > > >>> 3. NFS secondary storage is not explicitly depended on by other > > components. For example, when registering template into S3, template > > will be write into S3 directly, instead of storing into nfs secondary > > storage, then push to S3. If s3 is used as secondary storage, then nfs > > storage will be used as cache storage, but from other components point > > of view, cache storage is invisible. So, it's possible to make nfs > > storage as optional if s3 is used for certain hypervisors. > > >>> The detailed FS is at > > >>> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu > > >>> p+Object+Store+Plugin+Framework > > >>> The test we did: > > >>> > > >>> 1. We modified marvin to use new storage api > > >>> > > >>> 2. Test_volume and test_vm_life_cycle, test_template under > smoke > > test folder are executed against xenserver/kvm/vmware and devcloud, > > some of them are failed, it's partly due to bugs introduced by our > > code, partly master branch itself has issue(e.g. resizevolume doesn't > > work). We want to fix these issues after merging into master. > > >>> > > >>> The basic follow does work: create user vm, attach/detach volume, > > register template, create template from volume/snapshot, take > > snapshot, create volume from snapshot. > > >>> It's a huge change, around 60k LOC patch, to review the code, you > > >>> can > > try: git diff master..object_store, will show all the diff. > > >>> Comments/feedback are welcome. Thanks. > > >>> > > >>> > > >> > > >> > > >> Given the amount of change, can we get at least a BVT run against > > >> your branch done before merge? > > >> > > >> --David > > >> > > > > > > +1 to BVT please.
