Edison, Thanks, I will start going through it today. Based on other $dayjob responsibilities, it may take me a couple of days.
Thanks, -John On May 20, 2013, at 6:15 PM, Edison Su <edison...@citrix.com> wrote: > > >> -----Original Message----- >> From: Edison Su [mailto:edison...@citrix.com] >> Sent: Monday, May 20, 2013 2:30 PM >> To: dev@cloudstack.apache.org >> Subject: RE: [MERGE]object_store branch into master >> >> >> >>> -----Original Message----- >>> From: John Burwell [mailto:jburw...@basho.com] >>> Sent: Monday, May 20, 2013 12:56 PM >>> To: dev@cloudstack.apache.org >>> Subject: Re: [MERGE]object_store branch into master >>> >>> All, >>> >>> Since this change is so large, it makes reviewing and commenting in >>> detail extremely difficult. Would it be possible to push this patch >>> through Review Board to ease comprehension and promote a conversation >> about this patch? >> >> We can try to push it into Review Board. > > The review board url is: https://reviews.apache.org/r/11277/, 25 pages... > >> >>> >>> Reading through the FS, I have the following questions regarding the >>> operation of the NFS cache: >>> >>> What happens if/when the disk space of the NFS cache is exhausted? >>> What are the sizing recommendations/guidelines for it? >>> What strategy is used to age files out of the NFS cache? >> As usual, admin can have multiple NFS secondary storages, admin can also >> add multiple NFS cache storages. The NFS cache storage capacity plan will be >> the same as NFS secondary storage. >> If there multiple NFS cache storages, the current strategy will randomly >> choose one of them. Currently, no clean up/aging out strategy implemented >> yet But the situation can be improved: most of cached object can be deleted >> after accessed once. Take template as example, if zone wide storage is used, >> put template on cache storage has little value, as once the template is >> downloaded into primary storage, suddenly all the hypervisor host can access >> it. >> I think the simple LRU algorithm to delete cached objects should be enough. >> It can be added later, the cache storage has its own pom project, it's place >> to >> add more intelligence. >> >>> If two processes, process1 and process2, are both using a template, >>> templateA, will both processes reference the same file in the NFS >>> cache? If >> It's possible, that one template can be downloaded into cache storage twice, >> in case of concurrent accessed by two processes. The current >> implementation is that, if two processes want to download the same >> template from s3 into one primary storage at the same time, there is only >> one template will be downloaded into cache storage. While, if two processes >> want to download the same template into different primary storage, the >> template will be cached twice. >>> they are reading from the same file and process1 finishes before >>> process2, will process1 attempt to delete process2? >> >> There is no way to delete while read, as each cached object has its own state >> machine. If it's accessed by one process, the state will be changed to >> "Copying", you can't delete an object when it's in "Copying" state. >> >>> If a file transfer from the NFS cache to the object store fails, what >>> is the recovery/retry strategy? What durability guarantees will >>> CloudStack supply when a snapshot, template, or ISO is in the cache, >>> but can't be written to the object store? >> >> The error handling of cache storage shouldn't be different than without >> cache storage. For example, directly backup snapshot from primary storage >> to S3, without cache storage. If backup failed, then the whole process >> failed, >> user needs to do it again through cloudstack API. So in cache storage case, >> if >> push object from cache storage to s3 failed, then the whole backup process >> failed. >> >>> What will be the migration strategy for the objects contained in S3 >>> buckets/Swift containers from pre-4.2.0 instances? Currently, >>> CloudStack tracks a mapping between these objects and templates/ISOs >>> in the template_switt_ref and template_s3_ref table. >> >> We need to migrate DB from existing template_s3_ref to >> template_store_ref, and put all the s3 information into image_store and >> image_store_details tables. >> >>> >>> Finally, does the S3 implementation use multi-part upload to transfer >>> files to the object store? If not, the implementation will be limited >>> to storing files no larger than 5GB in size. >> Oh, this is something we don't know yet. We haven't try to upload a >> template which is large than 5GB, so haven't met this issue. >> Could you help to hack it up?:) >> >>> >>> Thanks, >>> -John >>> >>> On May 20, 2013, at 1:52 PM, Chip Childers <chip.child...@sungard.com> >>> wrote: >>> >>>> On Fri, May 17, 2013 at 08:19:57AM -0400, David Nalley wrote: >>>>> On Fri, May 17, 2013 at 4:11 AM, Edison Su <edison...@citrix.com> >> wrote: >>>>>> Hi all, >>>>>> Min and I worked on object_store branch during the last one >>>>>> and half >>> month. We made a lot of refactor on the storage code, mostly related >>> to secondary storage, but also on the general storage framework. The >>> following goals are made: >>>>>> >>>>>> 1. An unified storage framework. Both secondary >>> storages(nfs/s3/swift etc) and primary storages will share the same >>> plugin model, the same interface. Add any other new storages into >>> cloudstack will much easier and straightforward. >>>>>> >>>>>> 2. The storage interface between mgt server and resource is >> unified, >>> currently there are only 5 commands send out by mgt server: >>> >> copycommand/createobjectcommand/deletecommand/attachcommand/de >>> ttachcommand, and each storage vendor can decode/encode all the >>> entities(volume/snapshot/storage pool/ template etc) by its own. >>>>>> >>>>>> 3. NFS secondary storage is not explicitly depended on by other >>> components. For example, when registering template into S3, template >>> will be write into S3 directly, instead of storing into nfs secondary >>> storage, then push to S3. If s3 is used as secondary storage, then nfs >>> storage will be used as cache storage, but from other components point >>> of view, cache storage is invisible. So, it's possible to make nfs >>> storage as optional if s3 is used for certain hypervisors. >>>>>> The detailed FS is at >>>>>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu >>>>>> p+Object+Store+Plugin+Framework >>>>>> The test we did: >>>>>> >>>>>> 1. We modified marvin to use new storage api >>>>>> >>>>>> 2. Test_volume and test_vm_life_cycle, test_template under >> smoke >>> test folder are executed against xenserver/kvm/vmware and devcloud, >>> some of them are failed, it's partly due to bugs introduced by our >>> code, partly master branch itself has issue(e.g. resizevolume doesn't >>> work). We want to fix these issues after merging into master. >>>>>> >>>>>> The basic follow does work: create user vm, attach/detach volume, >>> register template, create template from volume/snapshot, take >>> snapshot, create volume from snapshot. >>>>>> It's a huge change, around 60k LOC patch, to review the code, you >>>>>> can >>> try: git diff master..object_store, will show all the diff. >>>>>> Comments/feedback are welcome. Thanks. >>>>>> >>>>>> >>>>> >>>>> >>>>> Given the amount of change, can we get at least a BVT run against >>>>> your branch done before merge? >>>>> >>>>> --David >>>>> >>>> >>>> +1 to BVT please. >
