On 01/18/2012 06:12 AM, Dan Wendlandt wrote: > Thanks Monty! Inline > > Dan > > On Mon, Jan 16, 2012 at 4:35 PM, Monty Taylor <mord...@inaugust.com > <mailto:mord...@inaugust.com>> wrote: > > Hey guys - > > Quick status update from me here: > > First of all, Jim's patch is win: > > https://review.openstack.org/#change,3072 > > > yup, this one is in. > > > > > I've got a python-quantumclient repo made and pushed to > https://github.com/emonty/python-quantumclient > > > Huh, this is a bit different than I was expecting.... I would have > expected that the 'quantum' cli utility would be part of the client > repo. It looks like your patch keeps this in the server code, and makes > the server code depend on the client code. I was under the impression > that with nova the 'nova' utility was actually packaged with the client > code, or is that not the case? I'm sure you have a reason for doing > this, but I'd be curious to know what it was :)
It is part of the client repo - it's just done via setup.py. (The old bin/quantum was simply a wrapper which called quantum.client.cli.main() ... setuptools console_scripts do that for us. > > I moved a good portion of quantum.common in to python-quantumclient > because both quantum and quantum.client need it, but since quantum will > depend on quantum.client, it means that it'll be available to both > things. I don't think this is a great solution long term, but it kinda > of physically works at the moment. Some of quantum.common can get moved > to openstack.common - the main thing that'll be odd long term will be > Exceptions ... but I'm going to suggest we try this for a moment, see > how much we hate it and then try to solve that. :) > > > Ok, that makes sense. > > > > I've also got python-quantumclient set up initially to use tox... so > just grabbing the repo and running "tox" should work and should do > virtualenv based tests in both python2.6 and python2.7. > > > Looks like both client and server now can run "tox". cool. > > > > I've got a patch up for quantum: > > https://review.openstack.org/3084 > > Which makes quantum a namespace package so that things can be installed > over top of things (which then allows the python-quantumclient repo > to work) > > After that, I've got this: > > https://github.com/emonty/quantum/tree/remove-client > > Which removes quantum.client and quantum.common as well as making > Which I can submit for review > > > Sure, submit for review if you think its ready. Probably better to get > detailed comments on gerrit than on the ML. I shall - we'll need python-quantumclient to exist first. I'll get python-quantumclient up so that we've got a good way to review the quantum patch. > > I think we should chat about how plugins all fit in to this picture, > > > In general, the client lib accesses only the API (which should be plugin > agnostic). There are definitely API extensions that are plugin specific > though. Is that what you are referring to, or something else? I context switched there - not related to client split and not relevant for now. :) > because I think we're on the verge of having some really kickass stuff > going on here. > > > indeed. thanks again for your help with this! > My pleasure! Glad I can be helpful! Monty > > Anyway - poke at stuff, play around with it - tell me if you think I'm > nuts or not. > > Monty > > On 01/12/2012 08:08 AM, Dan Wendlandt wrote: > > > > > > On Tue, Jan 10, 2012 at 5:32 PM, Monty Taylor > <mord...@inaugust.com <mailto:mord...@inaugust.com> > > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>> wrote: > > > > So, while looking at this, I discovered the file you guys have, > > version.py. I've been working on a standalone library, vcsversion > > (https://github.com/emonty/vcsversion) which does some similar > things > > and which encapsulates some of the logic that's current in the > > build_tarball.sh script. I'd love to (doesn't need to happen > to today) > > port any missing functionality that's in version.py into > vcsversion and > > get quantum et al using it. > > > > > > great. the more common code we can rip out of quantum proper and > put in > > a shared lib the better in my opinion. > > > > dan > > > > > > > > Great minds think alike, right? :) > > > > Monty > > > > On 01/10/2012 02:48 PM, Dan Wendlandt wrote: > > > Ok, based on the meeting today we are going ahead with this. > > > > > > Blueprint created and assigned to > > > monty: > > > https://blueprints.launchpad.net/quantum/+spec/separate-client-repo > > > > > > Monty, I assume that you will setup the infrastructure, then > at some > > > point need to hand off to a quantum dev (or not). Please > ping the > > list > > > when you get to that point. > > > > > > And as cdub mentioned at the end of the meeting, the sooner > we can do > > > this, the better in terms of distros that will need to > change their > > > packaging in time for essex. > > > > > > thanks for driving this. > > > > > > Dan > > > > > > > > > On Tue, Jan 3, 2012 at 1:00 PM, Monty Taylor > <mord...@inaugust.com <mailto:mord...@inaugust.com> > > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>> > > > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com> > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>>> wrote: > > > > > > ++ > > > > > > I think as a parallel thing, some of the things that are in > > > quantum.common _might_ want to live in openstack.common. > > > > > > On 01/03/2012 12:55 PM, Brad Hall wrote: > > > > I still vote for #2. I don't think there will be a ton of > > > duplication so we're probably ok in that regard. Also, > it sounds > > > like following suit will make life easier for the ci > folks. I > > think > > > #3 will be painful (both in initial work and maintenance). > > > > > > > > If we do #2 then we could put quantum-client in > > tools/pip-requires > > > and there wouldn't be much of any additional burden on the > > developer > > > unless they were also going to work on the client -- in > which case > > > pull a separate repo. > > > > > > > > Thanks, > > > > Brad > > > > > > > > On Jan 3, 2012, at 12:47 AM, Dan Wendlandt wrote: > > > > > > > >> Hi folks, > > > >> > > > >> tl;dr we need to decide on splitting client repo out > of main > > > repo, per monty's comments. Because this is what all other > > > openstack projects are doing, bias is to move in that > > direction, but > > > benefits are not clear. If we do split, my suggestion is to > > create > > > two repos (client/server), not three repos > (client/server/common) > > > >> > > > >> longer version: > > > >> > > > >> At the 11/29 netstack meeting, one of the topics > discussed was > > > how to handle monty's suggestion that we split out the > client into > > > another repo. > > > >> > > > >> From the 11/29 meeting, here's some background: > > > >> > > > >> 22:12:59 <mtaylor> danwent: > > > >> the problem is that we've got projects that need to > depend on > > > this stuff (horizon being a good example) > > > >> > > > >> 22:13:17 <mtaylor> > > > >> so we need to be able to reference the actual thing > that's > > > needed, wihtout pulling in the entire server > > > >> > > > >> 22:13:31 <danwent> mtaylor: > > > >> definitely understand that.. we want to be able to > package > > > things separately, definitely > > > >> > > > >> 22:13:40 <cdub> > > > >> sounds more like packagin then repo mgmt > > > >> > > > >> 22:13:51 <danwent> > > > >> I think the question is why packaging needs to be > tied to repo > > > so tightly. > > > >> > > > >> 22:13:58 <cdub> > > > >> e.g. simple to build multpile packages from a single > repo > > > >> > > > >> 22:13:59 <danwent> > > > >> i definitely understand that you have a lot of > experience > > here… :) > > > >> > > > >> 22:14:17 <danwent> cdub: > > > >> yes, in fact that is what we do already. > > > >> > > > >> 22:14:23 <cdub> > > > >> right > > > >> > > > >> 22:14:25 <mtaylor> > > > >> it's for a few reasons > > > >> > > > >> 22:14:42 <mtaylor> > > > >> pip requires git lines can't reference sub directories > > > >> > > > >> 22:15:01 <mtaylor> > > > >> so you wind up with venv installs that require > actual releases > > > to pypi to be consumable > > > >> > > > >> 22:15:36 <mtaylor> > > > >> additionally, once you start having submodules in tree, > > standard > > > tooling winds up being difficult to apply - knowing to > > interact with > > > a setup.py in trunk is straight forward > > > >> > > > >> 22:15:50 <mtaylor> > > > >> knowing that there are 4 setup.py files spread > throughout the > > > tree is less obviouos > > > >> > > > >> 22:16:05 <bhall> > > > >> as of yesterday all setup.py's are in the root :) > > > >> > > > >> 22:16:24 <danwent> > > > >> ok, so the trade-off here is if we want the pip > files of other > > > projects to automatically pull "trunk", we need separate > repos? > > > >> > > > >> 22:16:41 <mtaylor> > > > >> yes > > > >> > > > >> 22:17:21 <mtaylor> > > > >> also, it's the model we're using for the other > projects, > > so in > > > keeping with a consistent project-wide approach, I'd kind of > > want to > > > hear a really good reason that it can't work > > > >> > > > >> 22:17:28 <bhall> > > > >> how many repos are we talking about? client, common, > server, > > > plugins? > > > >> > > > >> 22:18:16 <danwent> > > > >> I'm also worried about trying to keep quantum simple > > enough that > > > its easy for new people to join and hack on the project > > > >> > > > >> 22:18:25 <mtaylor> > > > >> I don't think, as of yet, that we need separate ones for > > plugins > > > >> > > > >> 22:18:55 <mtaylor> > > > >> the most important one in my brain in client - as we > have a > > > python-*client project either in existence or coming in to > > existence > > > for every openstack project > > > >> > > > >> 22:19:08 <bhall> > > > >> could we go with two repos? client, > common/server/plugins > > > >> > > > >> 22:19:11 <mtaylor> > > > >> the common one is just because that's how you've > organized > > your > > > code - and i believe client depends on it, yeah/ > > > >> > > > >> 22:19:18 <mtaylor> > > > >> does client depend on common? > > > >> > > > >> 22:19:25 <danwent> > > > >> I believe that was the original goal > > > >> > > > >> 22:19:50 <danwent> > > > >> but perhaps we should revisit that… I'm not sure how > much is > > > currently shared. > > > >> > > > >> 22:19:58 <bhall> > > > >> I think it does currently > > > >> > > > >> 22:20:06 <bhall> > > > >> yeah, I don't think there is a lot of code that it > depends on > > > >> > > > >> 22:20:17 <danwent> mtaylor: > > > >> so the main goal here is pip install, which are mainly > > targetted > > > at developers right? > > > >> > > > >> 22:21:05 <danwent> > > > >> (I assume non-developers will get everything via > packages, > > which > > > should have dependencies described in terms of packages) > > > >> > > > >> 22:21:43 <mtaylor> danwent: > > > >> yes > > > >> > > > >> 22:22:12 <danwent> > > > >> ok, thanks for the explanations… I think I > understand this > > > better now, but still need to noodle on whether there > are ways we > > > can both be happy :) > > > >> > > > >> 22:22:18 <mtaylor> danwent: > > > >> there is no thought that people will be doing production > > > releases from pip :) > > > >> > > > >> 22:22:23 <mtaylor> danwent: awesome > > > >> > > > >> An here's some more follow-up from the 12/6 meeting: > > > >> > > > >> 22:18:55 <danwent> > > > >> first, as discussed last week with mtaylor, we need > to figure > > > out our strategy for whether we are going to split > quantum into > > > multiple repos. > > > >> > > > >> 22:19:49 <salv> > > > >> before deciding on a strategy let's decide whether > we want to > > > split the repos or not! :) > > > >> > > > >> 22:20:02 <mtaylor> danwent: > > > >> re that: we've got openstack.common going now - so > perhaps > > > quantum.common can live there? > > > >> > > > >> 22:20:03 <danwent> > > > >> we can have the detailed discussion on the ML, but the > > > high-level options seem to be (1) keep as is, all one > repo (2) > > split > > > into two repos, client and server, but potentially > duplicate the > > > little bits of shared code and (3) split into three > repos, client, > > > server, and common > > > >> > > > >> 22:20:09 <salv> > > > >> Splitting in which way exactly? client and server > components? > > > >> > > > >> 22:20:30 <danwent> mtaylor: > > > >> I was thinking something very similar myself, if we > decide > > to go > > > with #2 > > > >> > > > >> 22:20:57 <danwent> salv: > > > >> up for discussion > > > >> > > > >> 22:21:14 <danwent> #TODO: #danwent, start ML thread for > > splitting > > > repos, include #mtaylor as well > > > >> 22:21:17 <bhall> > > > >> my vote is for #2 (client and "the rest") > > > >> > > > >> 22:21:19 <mtaylor> #2 with some elements in > openstack.common > > > would be the thing that matches the other projects more > closely > > > >> 22:21:55 <danwent> if anyone else wants to comment > now, go > > ahead, > > > otherwise i'll forward this content to start a thead on > the ML > > > >> > > > >> It seems like we have three primary options (feel free to > > suggest > > > others). > > > >> > > > >> #1: do nothing, keep every in single repo > > > >> #2: split into two repos: client + server, but > potentially > > > duplicate some of the shared code currently in "common" > > > >> #3: split into three repos, client, server and common.. > > > >> > > > >> A couple thoughts: > > > >> > > > >> I'm still not totally clear on the value of splitting the > > repos. > > > Multiple repos are not required for anyone installing > > openstack via > > > packages, as we can easily generate separate client/server > > packages > > > when using a single repo (in fact, we already do).. My > > understanding > > > is that splitting the repos aims to help developers in > the case > > > where a project like horizon specifies a dependency on > quantum by > > > adding a direct reference to the quantum git repo in > pip-requires. > > > So one thing I'd like to better understand is the cost of > > > specifying a quantum git repo that includes client + > server, as > > > opposed to one that just includes the client. If its just a > > matter > > > of pulling down some more bytes, that doesn't seem that > > painful (the > > > quantum source is tiny anyway). Is there some greater > benefit to > > > splitting repos that I am missing? > > > >> > > > >> I think the cost of splitting repos is non-trivial, > as it makes > > > it harder for developers to get things running from > source tree, > > > (again, I'm assuming this is mainly a developer issue, > as users > > > should mainly install from packages). It may also mean > we need to > > > duplicate some common code, though the need for that may go > > down if > > > we can get both client + server code bases to have > common code in > > > openstack-common. > > > >> > > > >> Another important consideration is that it seems that all > > of the > > > other openstack projects are moving to a model with the > client > > in a > > > separate repo from the service itself. There would > definitely a > > > cost both in terms of developer confusion and pain to the > > > infrastructure team to Quantum being an oddball in that > > department. > > > This alone may be reason enough to follow suit, but I'd > love if > > > there was a clear cut reason other than "everyone else is > > doing it". > > > >> > > > >> If we do decide to split repos, my bias would be for > #2, which > > > avoid the even more significant complexity of having > three repos. > > > While some code in common may need to be duplicated, it may > > be the > > > case that such code is duplicated because it really > belongs in > > > something like openstack-common, which could be a > dependency of > > > both. A lof of the code in quantum.common seems to be > of that > > > variety (I suspect much of this code is actually only > used by the > > > server at this point). The two chunks of common code > used by the > > > main client library are: > > > >> > > > >> from quantum.common import exceptions > > > >> from quantum.common.serializer import Serializer > > > >> > > > >> Serializer seems to be pretty much all stock code that > > should be > > > in openstack-common. > > > >> > > > >> The quantum client only uses a few of the Exceptions > classes in > > > common/exeptions.py. A few of these may need to be > duplicated in > > > both the client/server code. Though if we make the changes > > > suggested by Aaron to use only standard HTTP error > codes, the > > > applicability of this code to the client decreases to > the point of > > > almost no overlap, I suspect. > > > >> > > > >> Please chime in with your thoughts. Right now I'm > not sure who > > > would work on this, so if you think you have some > cycles, please > > > speak up. > > > >> > > > >> Dan > > > >> > > > >> > > > >> -- > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > >> Dan Wendlandt > > > >> Nicira Networks: www.nicira.com > <http://www.nicira.com> <http://www.nicira.com> > > <http://www.nicira.com> > > > >> twitter: danwendlandt > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > >> > > > >> -- > > > >> Mailing list: https://launchpad.net/~netstack > > > >> Post to : netstack@lists.launchpad.net > <mailto:netstack@lists.launchpad.net> > > <mailto:netstack@lists.launchpad.net > <mailto:netstack@lists.launchpad.net>> > > > <mailto:netstack@lists.launchpad.net > <mailto:netstack@lists.launchpad.net> > > <mailto:netstack@lists.launchpad.net > <mailto:netstack@lists.launchpad.net>>> > > > >> Unsubscribe : https://launchpad.net/~netstack > > > >> More help : https://help.launchpad.net/ListHelp > > > > > > > > > > > > > > > > > > > > > > > -- > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > Dan Wendlandt > > > Nicira Networks: www.nicira.com <http://www.nicira.com> > <http://www.nicira.com> > > <http://www.nicira.com> > > > twitter: danwendlandt > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > > > > > > > > > -- > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Dan Wendlandt > > Nicira Networks: www.nicira.com <http://www.nicira.com> > <http://www.nicira.com> > > twitter: danwendlandt > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > > > -- > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Dan Wendlandt > Nicira Networks: www.nicira.com <http://www.nicira.com> > twitter: danwendlandt > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
