On Thu, Jan 16, 2014 at 2:03 PM, Alexei Kornienko < alexei.kornie...@gmail.com> wrote:
> Hello Joe, > > > continuous refactoring and syncing across 22+ repositories sounds like a > nightmare, one that I would like to avoid. > > You are right this is not easy. > > However I have several reasons to do that: > The hardest part is to bring basic stuff in sync across all projects > (That's what we are doing now). Later we'll work directly with oslo lib and > just sync changes from it. > My concern here is that is just extra churn for minimal value, assuming there are plans to completely overhaul this in the near future. Perhaps this blueprint is really two: 1) do short term work to reduplicate the common client functionality. This would cover the current patches and involve oslo-incubator, and is limited in scope and shouldn't require to many oslo sync iterations. 2) major overhaul of client libraries so they are all based off a common base library. This would cover namespace changes, and possible a push to move CLI into python-openstackclient > We could introduce a standalone library to avoid the need to sync oslo > code across all projects but it brings additional problems: > > 1) We would have to maintain rationale versioning and backwards > compatibility of this library. If we start library from scratch we'll have > to add/change lots of stuff before we'll reach some stability period. > > 2)Another option would be to follow waterfall process and create a solid > library interface before including it to all client projects. However such > approach this period can take unknown amount of time and can be easily > failed during integration stage cause requirements change or some other > reason. > > Please let me know what you think. > > Best Regards, > Alexei > > > On 01/16/2014 08:16 PM, Joe Gordon wrote: > > > > > On Thu, Jan 16, 2014 at 12:07 PM, Alexei Kornienko < > alexei.kornie...@gmail.com> wrote: > >> On 01/16/2014 06:15 PM, Jesse Noller wrote: >> >> >> On Jan 16, 2014, at 9:54 AM, Alexei Kornienko < >> alexei.kornie...@gmail.com> wrote: >> >> On 01/16/2014 05:25 PM, Jesse Noller wrote: >> >> >> On Jan 16, 2014, at 9:07 AM, Joe Gordon <joe.gord...@gmail.com> wrote: >> >> >> >> >> On Thu, Jan 16, 2014 at 9:45 AM, Jesse Noller <jesse.nol...@rackspace.com >> > wrote: >> >>> >>> On Jan 16, 2014, at 5:53 AM, Chmouel Boudjnah <chmo...@enovance.com> >>> wrote: >>> >>> >>> On Thu, Jan 16, 2014 at 12:38 PM, Chris Jones <c...@tenshu.net> wrote: >>> >>>> Once a common library is in place, is there any intention to (or >>>> resistance against) collapsing the clients into a single project or even a >>>> single command (a la busybox)? >>> >>> >>> >>> that's what openstackclient is here for >>> https://github.com/openstack/python-openstackclient >>> >>> >>> After speaking with people working on OSC and looking at the code >>> base in depth; I don’t think this addresses what Chris is implying: OSC >>> wraps the individual CLIs built by each project today, instead of the >>> inverse: a common backend that the individual CLIs can wrap - the latter is >>> an important distinction as currently, building a single binary install of >>> OSC for say, Windows is difficult given the dependency tree incurred by >>> each of the wrapped CLIs, difference in dependencies, structure, etc. >>> >>> Also, wrapping a series of inconsistent back end Client classes / >>> functions / methods means that the layer that presents a consistent user >>> interface (OSC) to the user is made more complex juggling >>> names/renames/commands/etc. >>> >>> In the inverted case of what we have today (single backend); as a >>> developer of user interfaces (CLIs, Applications, Web apps (horizon)) you >>> would be able to: >>> >>> from openstack.common.api import Auth >>> from openstack.common.api import Compute >>> from openstack.common.util import cli_tools >>> >>> my_cli = cli_tools.build(…) >>> >>> def my_command(cli): >>> compute = Compute(Auth(cli.tentant…, connect=True)) >>> compute.list_flavors() >>> >>> This would mean that *even if the individual clients needed or wanted >>> to keep their specific CLIs, they would be able to use a not “least common >>> denominator” back end (each service can have a rich >>> common.api.compute.py or api.compute/client.py and extend where needed. >>> However tools like horizon / openstackclient can choose not to leverage the >>> “power user/operator/admin” components and present a simplified user >>> interface. >>> >>> I’m working on a wiki page + blueprint to brainstorm how we could >>> accomplish this based off of what work is in flight today (see doug’s >>> linked blueprint) and sussing out a layout / API strawman for discussion. >>> >>> Some of the additions that came out of this email threads and others: >>> >>> 1. Common backend should provide / offer caching utilities >>> 2. Auth retries need to be handled by the auth object, and each >>> sub-project delegates to the auth object to manage that. >>> 3. Verified Mocks / Stubs / Fakes must be provided for proper unit >>> testing >>> >> >> I am happy to see this work being done, there is definitely a lot of >> work to be done on the clients. >> >> This blueprint sounds like its still being fleshed out, so I am >> wondering what the value is of the current patches >> https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z >> >> Those patches mainly sync cliutils and apiutils from oslo into the >> assorted clients. But if this blueprint is about the python API and not the >> CLI (as that would be the openstack-pythonclient), why sync in apiutils? >> >> Also does this need to go through oslo-incubator or can this start out >> as a library? Making this a library earlier on will reduce the number of >> patches needed to get 20+ repositories to use this. >> >> >> Alexei and others have at least started the first stage of a rollout - >> the blueprint(s) needs additional work, planning and discussion, but his >> work is a good first step (reduce the duplication of code) although I am >> worried that the libraries and APIs / namespaces will need to change if we >> continue these discussions which potentially means re-doing work. >> >> If we take a step back, a rollout process might be: >> >> 1: Solidify the libraries / layout / naming conventions (blueprint) >> 2: Solidify the APIs exposed to consumers (blueprint) >> 3: Pick up on the common-client-library-2 work which is primarily a >> migration of common code into oslo today, into the structure defined by 1 & >> 2 >> >> So, I sort of agree: moving / collapsing code now might be premature. I >> do strongly agree it should stand on its own as a library rather than an >> oslo incubator however. We should start with a single, clean namespace / >> library rather than depending on oslo directly. >> >> Knowing usual openstack workflow I'm afraid that #1,#2 with a waterfall >> approach may take years to be complete. >> And after they'll be approved it will become clear that this architecture >> is already outdated. >> We try to use iterative approach for clients refactoring. >> We started our work from removing code duplication because it already >> gives a direct positive effect on client projects. >> If you can show us better way of moving forward please help us by >> uploading patches on this topic. >> >> Talk is cheap. Show me the code. (c) Linus >> >> >> I don’t disagree and I’m pretty sure I and others have already thanked >> you for starting this but have expressed concerns about the API / layout - >> and many of the patches on that blueprint are still pending review due to >> those concerns. I’m not suggesting we need to spend the next six years >> designing it, I’m arguing for some basic design. From the current blueprint >> it is unclear where things will live, where it will live and how does it >> benefit everyone. >> >> So I’d like to work *with* you to expand the blueprints to address some >> basic design / namespacing / issues and not jump straight into refactoring >> 22+ command line tools without a clear idea of the the desired layout and >> design. >> >> The problem with defining design is that I don't know a convenient way >> of doing it cause UML, etherpad, blueprint's description are not sufficient >> to do this clearly. >> We try to use approach that I call "continuous refactoring" so as long as >> something can be improved we improve it. >> Our basic goal is to move towards SOLID principles and as long every new >> commit brings us one step closer to them I consider that overall design is >> improving. >> I've already posted here a draft idea of what we would like to see in the >> end: >> >> > continuous refactoring and syncing across 22+ repositories sounds like a > nightmare, one that I would like to avoid. > > > >> 1) Transport layer would handle all transport related stuff - HTTP, JSON >> encoding, auth, caching, etc. >> 2) Model layer (Resource classes, BaseManager, etc.) will handle data >> representation, validation >> 3) API layer will handle all project specific stuff - url mapping, etc. >> (This will be imported to use client in other applications) >> 4) Cli level will handle all stuff related to cli mapping - argparse, >> argcomplete, etc. >> >> >> >> >> >> >> jesse >> >> >> >> >> _______________________________________________ >> OpenStack-dev mailing >> listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> >> _______________________________________________ >> OpenStack-dev mailing >> listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > _______________________________________________ > OpenStack-dev mailing > listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
