Preventing plugin developers from implementing their own installer is a pro, not a con, you've already listed one reason in cons against install scripts inside plugin tarball: if we centralize plugin installation and management logic in fuel, we can change it once for all plugins and don't have to worry about old plugins using an obsolete installer.
I think priorities here should be 1) easy of plugin development; and 2) ease of use. Pluggable architecture won't do us much good if we end up being the only ones being able to use it efficiently. Adding a little more complexity to fuelclient to allow moving a lot of fuel complexity from core to plugins is a good tradeoff. On Thu, Oct 23, 2014 at 8:32 AM, Evgeniy L <e...@mirantis.com> wrote: > Hi Mike, > > I would like to add a bit more details about current implementation and how > it can be done. > > Implement installation as a scripts inside of tar ball: > Cons: > * install script is really simple right now, but it will be much more > complicated > ** it requires to implement logic where we can ask user for login/password > ** use some config, where we will be able to get endpoints, like where is > keystone, nailgun > ** validate that it's possible to install plugin on the current version of > master > ** handle error cases (to make installation process more atomic) > * it will be impossible to deprecate the installation logic/method, because > it's on the plugin's side > and you cannot change a plugin which user downloaded some times ago, when > we get > plugin manager, we probably would like user to use plugin manager, instead > of some scripts > * plugin installation process is not so simple as it could be (untar, cd > plugin, ./install) > > Pros: > * plugin developer can change installation scripts (I'm not sure if it's a > pros) > > Add installation to fuel client: > Cons: > * requires changes in fuel client, which are not good for fuel client by > design (fuel client > should be able to work remotely from user's machine), current > implementation requires > local operations on files, it will be changed in the future releases, so > fuel-client will > be able to do it via api, also we can determine if it's not master node by > /etc/fuel/version.yaml > and show the user a message which says that in the current version it's > not possible > to install the plugin remotely > * plugin developer won't be able to change installation process (I'm not > sure if it's a cons) > > Pros: > * it's easier for user to install the plugin `fuel --install-plugin > plugin_name-1.0.1.fpb' > * all of the authentication logic already implemented in fuel client > * fuel client uses config with endpoints which is generated by puppet > * it will be easier to deprecate previous installation approach, we can just > install new > fuel client on the master which uses api > > Personally I like the second approach, and I think we should try to > implement it, > when we get time. > > Thanks, > > On Thu, Oct 23, 2014 at 3:02 PM, Mike Scherbakov <mscherba...@mirantis.com> > wrote: >>> >>> I feel like we should not require user to unpack the plugin before >>> installing it. Moreover, we may chose to distribute plugins in our own >>> format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd >>> rather stick with two actions: >>> >>> Assembly (externally): fpb --build <name> >>> >>> Installation (on master node): fuel --install-plugin <name> >>> >>> I like the idea of putting plugin installation functionality in fuel >>> client, which is installed >>> on master node. >>> But in the current version plugin installation requires files operations >>> on the master, >>> as result we can have problems if user's fuel-client is installed on >>> another env. >> >> >> I suggest to keep it simple for now as we have the issue mentioned by >> Evgeny: fuel client is supposed to work from other nodes, and we will need >> additional verification code in there. Also, to make it smooth, we will have >> to end up with a few more checks - like what if tarball is broken, what if >> we can't find install script in it, etc. >> I'd suggest to run it simple for 6.0, and then we will see how it's being >> used and what other limitations / issues we have around plugin installation >> and usage. We can consider to make this functionality as part of fuel client >> a bit later. >> >> Thanks, >> >> On Tue, Oct 21, 2014 at 6:57 PM, Vitaly Kramskikh >> <vkramsk...@mirantis.com> wrote: >>> >>> Hi, >>> >>> As for a separate section for plugins, I think we should not force it and >>> leave this decision to a plugin developer, so he can create just a single >>> checkbox or a section of the settings tab or a separate tab depending on >>> plugin functionality. Plugins should be able to modify arbitrary release >>> fields. For example, if Ceph was a plugin, it should be able to extend >>> wizard config to add new options to Storage pane. If vCenter was a plugin, >>> it should be able to set maximum amount of Compute nodes to 0. >>> >>> 2014-10-20 21:21 GMT+07:00 Evgeniy L <e...@mirantis.com>: >>>> >>>> Hi guys, >>>> >>>> Romans' questions: >>>> >>>> >> I feel like we should not require user to unpack the plugin before >>>> >> installing it. >>>> >> Moreover, we may chose to distribute plugins in our own format, which >>>> >> we >>>> >> may potentially change later. E.g. "lbaas-v2.0.fp". >>>> >>>> I like the idea of putting plugin installation functionality in fuel >>>> client, which is installed >>>> on master node. >>>> But in the current version plugin installation requires files operations >>>> on the master, >>>> as result we can have problems if user's fuel-client is installed on >>>> another env. >>>> What we can do is to try to determine where fuel-client is installed, if >>>> it's master >>>> node, we can perform installation, if it isn't master node, we can show >>>> user the >>>> message, that in the current version remote plugin installation is not >>>> supported. >>>> In the next versions if we implement plugin manager (which is separate >>>> service >>>> for plugins management) we will be able to do it remotely. >>>> >>>> >> How are we planning to distribute fuel plugin builder and its >>>> >> updates? >>>> >>>> Yes, as Mike mentioned our plan is to release it on PyPi which is python >>>> packages >>>> repository, so any developer will be able to run `pip install fpb` and >>>> get the tool. >>>> >>>> >> What happens if an error occurs during plugin installation? >>>> >>>> Plugins installation process is very simple, our plan is to have some >>>> kind of transaction, >>>> to make it atomic. >>>> >>>> 1. register plugin via API >>>> 2. copy the files >>>> >>>> In case of error on the 1st step, we can do nothing, in case of error on >>>> the 2nd step, >>>> remove files if there are any, and delete a plugin via rest api. And >>>> show user a message. >>>> >>>> >> What happens if an error occurs during plugin execution? >>>> >>>> In the first iteration we are going to interrupt deployment if there are >>>> any errors for plugin's >>>> tasks, also we are thinking how to improve it, for example we wanted to >>>> provide a special >>>> flag for each task, like fail_deploument_on_error, and only if it's >>>> true, we fail deployment in >>>> case of failed task. But it can be tricky to implement, it requires to >>>> change the current >>>> orchestrator/nailgun error handling logic. So, I'm not sure if we can >>>> implement this logic in >>>> the first release. >>>> >>>> Regarding to meaningful error messages, yes, we want to show the user, >>>> which plugin >>>> causes the error. >>>> >>>> >> Shall we consider a separate place in UI (tab) for plugins? >>>> >>>> +1 to Mike's answer >>>> >>>> >> When are we planning to focus on the 2 plugins which were identified >>>> >> as must-haves >>>> >> for 6.0? Cinder & LBaaS >>>> >>>> For Cinder we are going to implement plugin which configures GlusterFS >>>> as cinder backend, >>>> so, if user has installed GlusterFS cluster, we can configure our cinder >>>> to work with it, >>>> I want to mention that we don't install GlusterFS nodes, we just >>>> configure cinder to work >>>> with user's GlusterFS cluster. >>>> Stanislaw B. already did some scripts which configures cinder to work >>>> with GlusterFS, so >>>> we are on testing stage. >>>> >>>> Regarding to LBaaS, Stanislaw B. did multinode implementation, ha >>>> implementation is tricky >>>> and requires some additional work, we are working on it. >>>> >>>> Nathan's questions: >>>> >>>> Looks like Mike answered UI related questions. >>>> >>>> >> Do we offer any kind of validation for settings on plug-ins? Or >>>> >> some way to for the developer >>>> >> to ensure that setting that cannot be default or computed get >>>> >> requested for the plug-in? >>>> >>>> Yes, each field can have regexp which is used during the validation. >>>> >>>> Mike's questions: >>>> >>>> >> One minor thing from me, which I forgot to mention during the demo: >>>> >> verbosity of fpb run. I >>>> >> understand it might sound like a bikeshedding now, but I believe if >>>> >> we develop it right from >>>> >> the very beginning, then we can save some time later. So I would >>>> >> suggest normal, short INFO >>>> >> output, and verbose one with --debug. >>>> >>>> Agree. >>>> >>>> Thanks for your feedback, >>>> >>>> On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov >>>> <mscherba...@mirantis.com> wrote: >>>>> >>>>> Hi all, >>>>> I moved this conversation to openstack-dev to get a broader audience, >>>>> since we started to discuss technical details. >>>>> >>>>> Raw notes from demo session: >>>>> https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo. >>>>> >>>>> Let me start answering on a few questions below from Roman & Nathan. >>>>>> >>>>>> How are we planning to distribute fuel plugin builder and its updates? >>>>>> Ideally, it should be available externally (outside of master node). I >>>>>> don't >>>>>> want us to repeat the same mistake as we did with Fuel client, which >>>>>> doesn't >>>>>> seem to be usable as an external dependency. >>>>> >>>>> The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it >>>>> should be backward compatible with older Fuel release, i.e. when there is >>>>> Fuel 7.0 out, you should be still able to create plugin for Fuel 6.0. If >>>>> that it is going to be overcomplicated - I suggested to produce fpb for >>>>> every Fuel release, and name it like fpb60, fpb61, fpb70, etc. Then it >>>>> becomes easier to support and maintain plugin builders for certain >>>>> versions >>>>> of Fuel. >>>>> Speaking about Fuel Client - there is no mistake. It's been discussed >>>>> dozens of times, it's just lack of resources to make it on PyPI as well as >>>>> to fix a few other things. I hope it could be done as part of efforts from >>>>> [2]. >>>>> >>>>>> - Perhaps we have a separate settings tab just for Plug-Ins? For >>>>>> some complex plug-ins, they might require a dedicated tab. If we have >>>>>> too >>>>>> many tabs it could get messy. >>>>> >>>>> >>>>>> >>>>>> Shall we consider a separate place in UI (tab) for plugins? Settings >>>>>> tab seems to be overloaded. >>>>> >>>>> >>>>> This is certainly under planning and discussion for future releases. >>>>> See [1], for example. For 6.0, we agreed that we can just extend existing >>>>> Settings tab with plugins-related fields. >>>>> >>>>> One minor thing from me, which I forgot to mention during the demo: >>>>> verbosity of fpb run. I understand it might sound like a bikeshedding now, >>>>> but I believe if we develop it right from the very beginning, then we can >>>>> save some time later. So I would suggest normal, short INFO output, and >>>>> verbose one with --debug. >>>>> >>>>> Thanks for feedback folks!!! >>>>> >>>>> [1] >>>>> https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg37196.html >>>>> [2] >>>>> https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg37001.html >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Nathan Trueblood <ntruebl...@mirantis.com> >>>>> Date: Sat, Oct 18, 2014 at 3:24 AM >>>>> Subject: Re: plugins >>>>> >>>>> Agreed - I thought this initial PoC was great. >>>>> >>>>> A few initial thoughts about settings in the UI and plug-in in general: >>>>> >>>>> - Perhaps we have a separate settings tab just for Plug-Ins? For >>>>> some complex plug-ins, they might require a dedicated tab. If we have >>>>> too >>>>> many tabs it could get messy. >>>>> - It seems like we should consider how we handle the VMWare settings in >>>>> light of plug-ins as well. Since with VMWare we have a lot of setting to >>>>> configure and settings validation. >>>>> - Do we offer any kind of validation for settings on plug-ins? Or >>>>> some way to for the developer to ensure that setting that cannot be >>>>> default >>>>> or computed get requested for the plug-in? >>>>> >>>>> - We need to think carefully about both the plug-in developer >>>>> experience (how hard to test, get error messages, etc) and the experience >>>>> for the user who deploys the plug-in into an environment. >>>>> >>>>> >>>>> -Nathan >>>>> >>>>> On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov >>>>> <ralekseen...@mirantis.com> wrote: >>>>>> >>>>>> >>>>>> I watched both videos (creating a file with the text from UI && >>>>>> installing and starting a service). >>>>>> >>>>>> It looks pretty good!! Some initial feedback/questions: >>>>>> >>>>>> I like the fact that fuel plugin builder appends version to the name >>>>>> and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar to >>>>>> Java/Maven and is a good one. >>>>>> I feel like we should not require user to unpack the plugin before >>>>>> installing it. Moreover, we may chose to distribute plugins in our own >>>>>> format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd >>>>>> rather stick with two actions: >>>>>> >>>>>> Assembly (externally): fpb --build <name> >>>>>> Installation (on master node): fuel --install-plugin <name> >>>>>> >>>>>> How are we planning to distribute fuel plugin builder and its updates? >>>>>> Ideally, it should be available externally (outside of master node). I >>>>>> don't >>>>>> want us to repeat the same mistake as we did with Fuel client, which >>>>>> doesn't >>>>>> seem to be usable as an external dependency. >>>>>> How do we handle errors? >>>>>> >>>>>> What happens if an error occurs during plugin installation? >>>>>> What happens if an error occurs during plugin execution? Does it >>>>>> (should it?) fail the deployment? Will we show user an error message with >>>>>> the name of plugin that failed? >>>>>> >>>>>> Shall we consider a separate place in UI (tab) for plugins? Settings >>>>>> tab seems to be overloaded. >>>>>> When are we planning to focus on the 2 plugins which were identified >>>>>> as must-haves for 6.0? Cinder & LBaaS >>>>>> >>>>>> Once again, great job guys! >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov >>>>>> <mscherba...@mirantis.com> wrote: >>>>>>> >>>>>>> Thanks, Evgeny, excellent work! >>>>>>> Roman, I believe we are "green" with the feature. Watch yourself. >>>>>>> >>>>>>> Mike Scherbakov >>>>>>> #mihgen >>>>>>> >>>>>>> On Oct 17, 2014 8:25 PM, "Evgeniy L" <e...@mirantis.com> wrote: >>>>>>>> >>>>>>>> Hi guys, here are the videos from the demo >>>>>>>> >>>>>>>> Part 1 >>>>>>>> https://drive.google.com/file/d/0B7I3b5vI7ZYXUGY1QVYyX3NLTWc/view >>>>>>>> >>>>>>>> Part 2 >>>>>>>> https://drive.google.com/file/d/0B7I3b5vI7ZYXWkRmV05fT1VEQkk/view >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Mike Scherbakov >>>>> #mihgen >>>>> >>>>> >>>>> _______________________________________________ >>>>> OpenStack-dev mailing list >>>>> OpenStack-dev@lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> >>> >>> -- >>> Vitaly Kramskikh, >>> Software Engineer, >>> Mirantis, Inc. >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> >> -- >> Mike Scherbakov >> #mihgen >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Dmitry Borodaenko _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
