Nitin, Please see my comments in-line below.
Thanks, -John On Jun 14, 2013, at 4:16 AM, Nitin Mehta <nitin.me...@citrix.com> wrote: > > > On 13/06/13 10:08 PM, "John Burwell" <jburw...@basho.com> wrote: > >> All, >> >> Edison Su, Min Chen, Animesh Chaturvedi, and myself met via >> teleconference on 11 June 2013 @ 1:30 PM EDT. The goal of the meeting >> was determine the path forward for merging the object_store branch by the >> 4.2 freeze date, 30 June 2013. The conversation focused on the following >> topics: >> >> * Staging area mechanism >> * Removing dependencies from the Storage to the Hypervisor layer >> * Dependencies of other patches on object_store >> * QA's desire to start testing the patch ASAP >> >> Min, Edison, and I agreed that the staging mechanism must age out files >> and use a reference count to ensure that file in-use are not prematurely >> purged. While we agree that some form of reservation system is required, >> Edison is concerned that it will be too conservative and create >> bottlenecks. > > Can you please elaborate on the fact that it is too conservative - we just > can't purge the files when they are still in use correct ? We can use a > combination of LRU + reference count, trying to purge the LRU files if > their reference count <= 0 as a start ? The issue is not around determining when to purge a file. The issue emerges around reservation sizes. Currently, if we take a snapshot of a 2 TB volume, we would have to reserve 2 TB in the staging area to ensure that we would have enough space for the maximum potential size of the snapshot. However, it is very unlikely that the snapshot will actually be this size. The concern becomes that large reservations would start starving out other processes. For 4.2, we didn't feel there was enough time to devise a "smarter" reservation mechanism. Therefore, in 4.3, there should be time to think the implications of various approaches through and devise a more efficient approach. > >> >> As we delved deeper into the subject of the storage to hypervisor >> dependencies and the reservation mechanism, we determined that NFS >> storage would still need to be the size of the secondary storage data >> set. Since the hypervisor layer has not been completely fitted to the >> new storage layer, NFS would be still required for a number of >> operations. Based on this realization, we decided to de-scope the >> staging mechanism, and leave the 4.2 object store functionality the same >> as 4.1. Therefore, NFS will remain the secondary storage of record, and >> object storage will serve as backup/multi-zone sync. > > I am not sure how we can comment its going to be the same as 4.1 - is it > from the end user perspective ? The internal semantics and their flow have > changed. This needs to be elaborated and properly documented. Also I am > not sure if the feature is supported on the upgrade path or is it ? Need > more documentation here. From an end user perspective, object stores will remain a backup of secondary storage. The user interface will likely be a bit nicer, but in terms of system architecture, the roles of object storage and NFS remain the same in 4.2 and 4.1. To my mind, when we support object stores as native secondary storage targets, it will be a new feature, and we should continue to support the backup model as well. Therefore, I don't see an upgrade path issue. > > >> In 4.3, we will fit the hypervisor layer for the new storage layer which >> will allow object stores to server as secondary storage. This work will >> include removing the storage to hypervisor dependencies. For 4.2, Edison >> and Min have implemented the critical foundation necessary to establish >> our next generation storage layer. There simply was not enough time in >> this development cycle to make these changes and fit the hypervisor layer. >> >> Due to the size of the patch, Animesh voiced QA's concerned regarding >> test scope and impact. As such, we want to get the merge completed as >> soon as possible to allow testing to begin. We discussed breaking up the >> patch, but we could not devise a reasonable set of chunks there were both >> isolated and significantly testable. Therefore, the patch can only be >> delivered in its current state. We also walked through potential >> dependencies between the storage framework changes and the solidfire >> branch. It was determined that these two merges could occur >> independently. >> >> Finally, Animesh is going to setup a meeting at Citrix's Santa Clara >> office on 26 June 2013 (the day after Collab ends) to discuss the path >> forward for 4.3 and work through a high-level design/approach to fitting >> the hypervisor layer to exploit the new storage capabilities. Details >> will be published to the dev mailing list. >> >> Thanks, >> -John >> >> On Jun 11, 2013, at 2:08 AM, Min Chen <min.c...@citrix.com> wrote: >> >>> It is 11th June. John is not free between 9:15am to 10am, that is why we >>> schedule it at 10:30am. >>> >>> Thanks >>> -min >>> >>> On 6/10/13 10:52 PM, "Nitin Mehta" <nitin.me...@citrix.com> wrote: >>> >>>> Hi Min, >>>> When you say tomorrow, what date is it 11th June or 12th ? Can the >>>> time be >>>> preponed by an hour please - its too late here ? >>>> >>>> Thanks, >>>> -Nitin >>>> >>>> On 11/06/13 11:06 AM, "Min Chen" <min.c...@citrix.com> wrote: >>>> >>>>> Hi there, >>>>> >>>>> To reach consensus on some remaining NFS cache issues on object_store >>>>> storage refactor work in a more effective manner, John, Edison and I >>>>> have >>>>> scheduled a GoToMeeting tomorrow to discuss them over the phone, any >>>>> interested parties are welcome to join and brainstorm. Here are >>>>> detailed >>>>> GTM information: >>>>> >>>>> Meeting Time: 10:30 AM 12:30 PM PST >>>>> >>>>> Meeting Details: >>>>> >>>>> 1. Please join my meeting. >>>>> https://www1.gotomeeting.com/join/188620897 >>>>> >>>>> 2. Use your microphone and speakers (VoIP) - a headset is >>>>> recommended. >>>>> Or, call in using your telephone. >>>>> >>>>> United States: +1 (626) 521-0017 >>>>> United States (toll-free): 1 877 309 2070 >>>>> >>>>> Access Code: 188-620-897 >>>>> Audio PIN: Shown after joining the meeting >>>>> >>>>> Meeting ID: 188-620-897 >>>>> >>>>> GoToMeeting® >>>>> Online Meetings Made Easy® >>>>> >>>>> Not at your computer? Click the link to join this meeting from your >>>>> iPhone®, iPad® or Android® device via the GoToMeeting app. >>>>> >>>>> Thanks >>>>> -min >>>> >>> >> >
