Thanks for pointing that out, Prachi.
On Mon, Jan 27, 2014 at 3:23 PM, Prachi Damle <prachi.da...@citrix.com>wrote: > There is already a ticket logged for this, > https://issues.apache.org/jira/browse/CLOUDSTACK-4942 > > Fix should be introducing a new table say storage_tag and follow the > design of host_tag and also upgrade steps for existing storage tags present > in the details. > > Prachi > > > -----Original Message----- > From: Marcus Sorensen [mailto:shadow...@gmail.com] > Sent: Monday, January 27, 2014 11:32 AM > To: dev@cloudstack.apache.org > Subject: Re: Tags on storagePool > > Right, for some reason I was thinking in the moment that all details would > have a value of 'true'. Since that's not the case, and this field is > clearly not a boolean field, we should just change them to storage_tag for > a value, as suggested. > > On Mon, Jan 27, 2014 at 12:28 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > > I agree...we should fix it in 4.4. > > > > Would you write up a JIRA ticket for it, Chris? > > > > Thanks! > > > > > > On Mon, Jan 27, 2014 at 12:26 PM, SuichII, Christopher < > > chris.su...@netapp.com> wrote: > > > >> I suspect the reason you don't have this problem is that you don't > >> have any storage pool details whose value is 'true'. That is the only > >> case where this problem arises. > >> > >> I think we do have a problem differentiating, though. If you list all > >> of the storage tags for a storage pool, then any storage pool details > >> whose value is 'true' will be assumed to be a storage tag. The only > >> way around this problem I see is to cross reference the results there > >> with all of the storage tags from disk offerings, and if you find a > >> storage pool detail that doesn't match a disk offering storage tag, > >> then assume it isn't a storage tag. > >> > >> Unfortunately it sounds like this is way out of scope for 4.3. I > >> think we need to start thinking about how we can fix this issue > >> moving forward now that there will be plugins out there impacted by > this. > >> > >> -Chris > >> -- > >> Chris Suich > >> chris.su...@netapp.com > >> NetApp Software Engineer > >> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat > >> > >> On Jan 27, 2014, at 2:21 PM, Marcus Sorensen <shadow...@gmail.com> > wrote: > >> > >> > I'm ok with whatever. It does seem like we have the ability to > >> > differentiate real storage tags from details, since we can > >> > list/edit those storage tags (I don't see all of my details showing > >> > up in the UI when I look at the tags for my primary storage), > >> > though I'm not immediately sure why or how at the moment. I think > >> > this affects Mike the most, since he's shipping a plugin to > >> > customers. I can just look at whatever changes are made and update my > pools manually, if need be. > >> > > >> > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski > >> > <mike.tutkow...@solidfire.com> wrote: > >> >> I agree...it certainly is not an ideal situation. > >> >> > >> >> Have you assessed the risk involved with changing storage tags > >> >> from > >> using > >> >> true as a value to using something like storage_tag as a value? > >> >> > >> >> If so, do you feel it is an acceptable amount of risk for 4.3 now > >> >> that > >> we > >> >> have already begun spinning up RC builds? > >> >> > >> >> Thanks > >> >> > >> >> > >> >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher < > >> >> chris.su...@netapp.com> wrote: > >> >> > >> >>> I think my only real concern with working around it in this > >> >>> manner is > >> that > >> >>> once a fix is applied, we now have a bunch of settings that do > >> >>> not > >> conform > >> >>> to the true/false pattern. In order to migrate them from using > >> >>> t/f, > >> yes/no > >> >>> or enabled/disabled back to using true/false, we would have to > >> >>> write > >> some > >> >>> kind of DB upgrade/migration in our plugin to update these > >> >>> values. In addition, I would have to change the ConfigKey from > >> >>> being a boolean to > >> a > >> >>> string and then back to a boolean once a solution is implemented. > >> >>> It is just some technical debt that would pile up for plugins > >> >>> with a pretty > >> nasty > >> >>> fix down the road. > >> >>> > >> >>> -Chris > >> >>> -- > >> >>> Chris Suich > >> >>> chris.su...@netapp.com > >> >>> NetApp Software Engineer > >> >>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat > >> >>> > >> >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski < > >> mike.tutkow...@solidfire.com> > >> >>> wrote: > >> >>> > >> >>>> It is a problem, but I think - at the moment - only NetApp, > >> >>>> SolidFire, > >> >>> and > >> >>>> Marcus have storage plug-ins. > >> >>>> > >> >>>> We could document this issue and the easy workaround of not > >> >>>> using true > >> >>> and > >> >>>> false as a value and try to address it in 4.4. > >> >>>> > >> >>>> > >> >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher < > >> >>>> chris.su...@netapp.com> wrote: > >> >>>> > >> >>>>> Any final thoughts on this? Letting this go into 4.3 could > >> potentially > >> >>>>> cause issues with anyone using the configuration system for > >> >>>>> plugins > >> in > >> >>> 4.3. > >> >>>>> > >> >>>>> -Chris > >> >>>>> -- > >> >>>>> Chris Suich > >> >>>>> chris.su...@netapp.com > >> >>>>> NetApp Software Engineer > >> >>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat > >> >>>>> > >> >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher < > >> >>> chris.su...@netapp.com> > >> >>>>> wrote: > >> >>>>> > >> >>>>>> This is the approach we'll have to take if we can't fix this > >> >>>>>> in > >> 4.3. It > >> >>>>> certainly works, but it isn't ideal. > >> >>>>>> > >> >>>>>> Does anyone else have any thoughts on the impact this might > >> >>>>>> have to > >> >>> 4.3? > >> >>>>>> > >> >>>>>> -Chris > >> >>>>>> -- > >> >>>>>> Chris Suich > >> >>>>>> chris.su...@netapp.com > >> >>>>>> NetApp Software Engineer > >> >>>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red > >> >>>>>> Hat > >> >>>>>> > >> >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski < > >> >>>>> mike.tutkow...@solidfire.com> wrote: > >> >>>>>> > >> >>>>>>> This isn't a great solution, but you could also change the > >> >>>>>>> value > >> for > >> >>>>> your > >> >>>>>>> plug-in from true or false to something like t or f. > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski < > >> >>>>>>> mike.tutkow...@solidfire.com> wrote: > >> >>>>>>> > >> >>>>>>>> Edison could confirm this perhaps, but I doubt any current > >> >>> installation > >> >>>>>>>> would have true for the value unless it was for a storage > >> >>>>>>>> tag (the > >> >>>>> plug-in > >> >>>>>>>> framework just came out in 4.2). > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski < > >> >>>>>>>> mike.tutkow...@solidfire.com> wrote: > >> >>>>>>>> > >> >>>>>>>>> I think your idea would be acceptable risk for 4.3. The > >> >>>>>>>>> upgrade > >> >>> logic > >> >>>>>>>>> would have to perform this true-to-storage_tag conversion, > >> >>>>>>>>> too, > >> >>>>> though. > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher < > >> >>>>>>>>> chris.su...@netapp.com> wrote: > >> >>>>>>>>> > >> >>>>>>>>>> I think the quickest, easiest change would be to keep > >> >>>>>>>>>> using the > >> tag > >> >>>>> name > >> >>>>>>>>>> as the key in the details table, but use a unique value > >> >>>>>>>>>> like > >> >>>>> 'storage_tag' > >> >>>>>>>>>> instead of 'true'. This wouldn't require any major logic > >> changes, > >> >>>>> just the > >> >>>>>>>>>> value used to check whether something is a storage tag. > >> >>>>>>>>>> > >> >>>>>>>>>> It is quite risky for 4.3. However my concern is that if > >> >>>>>>>>>> we let > >> >>> this > >> >>>>>>>>>> ship with 4.3, then any plugin that wants to use a storage > >> >>>>>>>>>> pool > >> >>>>> detail with > >> >>>>>>>>>> the value 'true' will make updating the storage tag system > >> >>>>>>>>>> MUCH > >> >>> more > >> >>>>>>>>>> difficult. As far as I can tell, there is no other way to > >> determine > >> >>>>> if > >> >>>>>>>>>> something is a storage tag than checking if the details > >> >>>>>>>>>> table > >> value > >> >>>>> is > >> >>>>>>>>>> 'true'. If there are other details with the value 'true', > >> >>>>>>>>>> then > >> we > >> >>>>> wouldn't > >> >>>>>>>>>> be able to differentiate between them for a DB upgrade > >> >>>>>>>>>> between > >> >>>>> versions. > >> >>>>>>>>>> > >> >>>>>>>>>> -Chris > >> >>>>>>>>>> -- > >> >>>>>>>>>> Chris Suich > >> >>>>>>>>>> chris.su...@netapp.com > >> >>>>>>>>>> NetApp Software Engineer > >> >>>>>>>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & > >> >>>>>>>>>> Red Hat > >> >>>>>>>>>> > >> >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski < > >> >>>>>>>>>> mike.tutkow...@solidfire.com> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>> I think at some point we need to use a key/value for > >> >>>>>>>>>>> storage > >> tags > >> >>>>> such > >> >>>>>>>>>> as > >> >>>>>>>>>>> the following: > >> >>>>>>>>>>> > >> >>>>>>>>>>> storageTags=value1,value2,value3 > >> >>>>>>>>>>> > >> >>>>>>>>>>> The problem with that is you have to parse the value cell > >> >>>>>>>>>>> each > >> >>> time > >> >>>>> you > >> >>>>>>>>>>> pull it out of the DB. > >> >>>>>>>>>>> > >> >>>>>>>>>>> That might be too risky of a change, though, for 4.3 at > >> >>>>>>>>>>> this > >> >>> point. > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII, Christopher < > >> >>>>>>>>>>> chris.su...@netapp.com> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> I have found an additional issue related to this. The > >> allocators > >> >>> do > >> >>>>>>>>>>>> properly ignore any storage pool details whose value is > >> >>>>>>>>>>>> true > >> that > >> >>>>> is > >> >>>>>>>>>> not > >> >>>>>>>>>>>> actually a storage pool. However, the list storage pools > >> >>>>>>>>>>>> API > >> does > >> >>>>>>>>>> NOT. When > >> >>>>>>>>>>>> creating the StoragePoolResponse, it is still assumed > >> >>>>>>>>>>>> that any > >> >>>>>>>>>> storage pool > >> >>>>>>>>>>>> detail with the value 'true' is a storage tag. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> For my plugin targeting 4.3, we create a storage pool > >> >>>>>>>>>>>> detail > >> >>> with a > >> >>>>>>>>>>>> true/false value, so this causes a problem with the > >> >>>>>>>>>>>> storage > >> pool > >> >>>>> UI. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Any thoughts on how to fix this? > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> -Chris > >> >>>>>>>>>>>> -- > >> >>>>>>>>>>>> Chris Suich > >> >>>>>>>>>>>> chris.su...@netapp.com > >> >>>>>>>>>>>> NetApp Software Engineer Data Center Platforms - Cloud > >> >>>>>>>>>>>> Solutions Citrix, Cisco & Red Hat > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena Prokharchyk < > >> >>>>>>>>>>>> alena.prokharc...@citrix.com> wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>>> Filed > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942 > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> -Alena. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> From: Prachi Damle <prachi.da...@citrix.com<mailto: > >> >>>>>>>>>>>> prachi.da...@citrix.com>> > >> >>>>>>>>>>>>> Date: Wednesday, October 23, 2013 2:04 PM > >> >>>>>>>>>>>>> To: Alena Prokharchyk <alena.prokharc...@citrix.com > <mailto: > >> >>>>>>>>>>>> alena.prokharc...@citrix.com>>, > >> >>>>>>>>>>>> "dev@cloudstack.apache.org > >> >>> <mailto: > >> >>>>>>>>>>>> dev@cloudstack.apache.org>" <dev@cloudstack.apache.org > >> <mailto: > >> >>>>>>>>>>>> dev@cloudstack.apache.org>> > >> >>>>>>>>>>>>> Subject: RE: Tags on storagePool > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Alena, > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I don't know why it was designed this way in first > >> >>>>>>>>>>>>> place - a > >> >>>>> design > >> >>>>>>>>>> like > >> >>>>>>>>>>>> host_tags where we have separate table to store tags is > >> >>>>>>>>>>>> much > >> >>> better > >> >>>>>>>>>> for > >> >>>>>>>>>>>> Allocators to work on. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> It is a bug, but will cause problem only if we land up > >> >>>>>>>>>>>>> with > >> >>>>> situation > >> >>>>>>>>>>>> explained below: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Given the existing design of storage tags, the > >> >>>>>>>>>>>>> Allocators > >> search > >> >>>>> the > >> >>>>>>>>>>>> details table using the name = > >> >>>>>>>>>>>> <tag-provided-in-disk-offering> > >> >>> and > >> >>>>>>>>>> value > >> >>>>>>>>>>>> =true > >> >>>>>>>>>>>>> Thus this will cause a problem in placement only if > >> >>>>>>>>>>>>> some > >> other > >> >>>>>>>>>> storage > >> >>>>>>>>>>>> pool detail happen to have the same 'name' as a > >> >>>>>>>>>>>> storage-tag > >> and > >> >>>>> also a > >> >>>>>>>>>>>> value = true. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> -Prachi > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> From: Alena Prokharchyk > >> >>>>>>>>>>>>> Sent: Wednesday, October 23, 2013 1:36 PM > >> >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org<mailto: > >> >>>>>>>>>>>> dev@cloudstack.apache.org> > >> >>>>>>>>>>>>> Subject: Tags on storagePool > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I came across a potential bug in the way allocators do > >> volumes > >> >>>>>>>>>> placement > >> >>>>>>>>>>>> on storage, based on storage tags. Prachi, can you > >> >>>>>>>>>>>> please > >> confirm > >> >>>>> if > >> >>>>>>>>>> the > >> >>>>>>>>>>>> bug is real. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> The tags are passed to the createStoragePool API in > >> >>>>>>>>>>>>> form of > >> >>>>>>>>>>>> tags='tag1,tag2,..': > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> ?command=createStoragePool&...&tags=alena > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> and stored in the storage_pool_details db table as: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> mysql> select *from storage_pool_details where > >> >>>>>>>>>>>>> mysql> pool_id=2; > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> | id | pool_id | name | value | > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> | 2 | 2 | alenatags | true | > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> 1 row in set (0.00 sec) > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Allocator code assumes that everything stored in > >> >>>>> storage_pool_details > >> >>>>>>>>>>>> table, having value=true - is a storage pool tag. And > >> >>>>>>>>>>>> this is > >> >>>>>>>>>> incorrect, as > >> >>>>>>>>>>>> the storage_pool_details table is used for storing diff > >> >>>>>>>>>>>> kinds > >> of > >> >>>>>>>>>> storage > >> >>>>>>>>>>>> pool details - not just tags - that can be later used by > >> various > >> >>>>>>>>>> cloudStack > >> >>>>>>>>>>>> managers. Like for example we save Storage level config > >> >>> parameters > >> >>>>> in > >> >>>>>>>>>> this > >> >>>>>>>>>>>> table ( > >> >>>>>>>>>>>> > >> >>>>>>>>>> > >> >>>>> > >> >>> > >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granula > >> r+Global+Configuration+Parameters > >> >>>>>>>>>>>> ). > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> The correct way to fix it would be - store tags as: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> | id | pool_id | name | value | > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> | 2 | 2 | tag | alena | > >> >>>>>>>>>>>>> +----+---------+-----------+-------+ > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> and fix StorageManager to retrive all the tags by the > "tag" > >> key. > >> >>>>> We > >> >>>>>>>>>> also > >> >>>>>>>>>>>> have to fix the DB upgrade, which can be tricky as we > >> >>>>>>>>>>>> will > >> have > >> >>> to > >> >>>>>>>>>> figure > >> >>>>>>>>>>>> out which detail is a tag. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> -Alena. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> -- > >> >>>>>>>>>>> *Mike Tutkowski* > >> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>>>>>>>>> e: mike.tutkow...@solidfire.com > >> >>>>>>>>>>> o: 303.746.7302 > >> >>>>>>>>>>> Advancing the way the world uses the > >> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >>>>>>>>>>> *(tm)* > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> -- > >> >>>>>>>>> *Mike Tutkowski* > >> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>>>>>>> e: mike.tutkow...@solidfire.com > >> >>>>>>>>> o: 303.746.7302 > >> >>>>>>>>> Advancing the way the world uses the cloud< > >> >>>>> http://solidfire.com/solution/overview/?video=play> > >> >>>>>>>>> *(tm)* > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> -- > >> >>>>>>>> *Mike Tutkowski* > >> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>>>>>> e: mike.tutkow...@solidfire.com > >> >>>>>>>> o: 303.746.7302 > >> >>>>>>>> Advancing the way the world uses the cloud< > >> >>>>> http://solidfire.com/solution/overview/?video=play> > >> >>>>>>>> *(tm)* > >> >>>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> -- > >> >>>>>>> *Mike Tutkowski* > >> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>>>>> e: mike.tutkow...@solidfire.com > >> >>>>>>> o: 303.746.7302 > >> >>>>>>> Advancing the way the world uses the > >> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >>>>>>> *(tm)* > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> *Mike Tutkowski* > >> >>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>> e: mike.tutkow...@solidfire.com > >> >>>> o: 303.746.7302 > >> >>>> Advancing the way the world uses the > >> >>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >>>> *(tm)* > >> >>> > >> >>> > >> >> > >> >> > >> >> -- > >> >> *Mike Tutkowski* > >> >> *Senior CloudStack Developer, SolidFire Inc.* > >> >> e: mike.tutkow...@solidfire.com > >> >> o: 303.746.7302 > >> >> Advancing the way the world uses the > >> >> cloud<http://solidfire.com/solution/overview/?video=play> > >> >> *(tm)* > >> > >> > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud<http://solidfire.com/solution/overview/?video=play> > > *(tm)* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *™*
