Thanks Monty! Inline Dan
On Mon, Jan 16, 2012 at 4:35 PM, Monty Taylor <mord...@inaugust.com> wrote: > Hey guys - > > Quick status update from me here: > > First of all, Jim's patch is win: > > https://review.openstack.org/#change,3072 yup, this one is in. > > > I've got a python-quantumclient repo made and pushed to > https://github.com/emonty/python-quantumclient Huh, this is a bit different than I was expecting.... I would have expected that the 'quantum' cli utility would be part of the client repo. It looks like your patch keeps this in the server code, and makes the server code depend on the client code. I was under the impression that with nova the 'nova' utility was actually packaged with the client code, or is that not the case? I'm sure you have a reason for doing this, but I'd be curious to know what it was :) > > > I moved a good portion of quantum.common in to python-quantumclient > because both quantum and quantum.client need it, but since quantum will > depend on quantum.client, it means that it'll be available to both > things. I don't think this is a great solution long term, but it kinda > of physically works at the moment. Some of quantum.common can get moved > to openstack.common - the main thing that'll be odd long term will be > Exceptions ... but I'm going to suggest we try this for a moment, see > how much we hate it and then try to solve that. :) > Ok, that makes sense. > > I've also got python-quantumclient set up initially to use tox... so > just grabbing the repo and running "tox" should work and should do > virtualenv based tests in both python2.6 and python2.7. > Looks like both client and server now can run "tox". cool. > > I've got a patch up for quantum: > > https://review.openstack.org/3084 > > Which makes quantum a namespace package so that things can be installed > over top of things (which then allows the python-quantumclient repo to > work) > > After that, I've got this: > > https://github.com/emonty/quantum/tree/remove-client > > Which removes quantum.client and quantum.common as well as making > Which I can submit for review > Sure, submit for review if you think its ready. Probably better to get detailed comments on gerrit than on the ML. > > I think we should chat about how plugins all fit in to this picture, > In general, the client lib accesses only the API (which should be plugin agnostic). There are definitely API extensions that are plugin specific though. Is that what you are referring to, or something else? > because I think we're on the verge of having some really kickass stuff > going on here. > indeed. thanks again for your help with this! > > Anyway - poke at stuff, play around with it - tell me if you think I'm > nuts or not. > > Monty > > On 01/12/2012 08:08 AM, Dan Wendlandt wrote: > > > > > > On Tue, Jan 10, 2012 at 5:32 PM, Monty Taylor <mord...@inaugust.com > > <mailto:mord...@inaugust.com>> wrote: > > > > So, while looking at this, I discovered the file you guys have, > > version.py. I've been working on a standalone library, vcsversion > > (https://github.com/emonty/vcsversion) which does some similar > things > > and which encapsulates some of the logic that's current in the > > build_tarball.sh script. I'd love to (doesn't need to happen to > today) > > port any missing functionality that's in version.py into vcsversion > and > > get quantum et al using it. > > > > > > great. the more common code we can rip out of quantum proper and put in > > a shared lib the better in my opinion. > > > > dan > > > > > > > > Great minds think alike, right? :) > > > > Monty > > > > On 01/10/2012 02:48 PM, Dan Wendlandt wrote: > > > Ok, based on the meeting today we are going ahead with this. > > > > > > Blueprint created and assigned to > > > monty: > > https://blueprints.launchpad.net/quantum/+spec/separate-client-repo > > > > > > Monty, I assume that you will setup the infrastructure, then at > some > > > point need to hand off to a quantum dev (or not). Please ping the > > list > > > when you get to that point. > > > > > > And as cdub mentioned at the end of the meeting, the sooner we can > do > > > this, the better in terms of distros that will need to change their > > > packaging in time for essex. > > > > > > thanks for driving this. > > > > > > Dan > > > > > > > > > On Tue, Jan 3, 2012 at 1:00 PM, Monty Taylor <mord...@inaugust.com > > <mailto:mord...@inaugust.com> > > > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>> > wrote: > > > > > > ++ > > > > > > I think as a parallel thing, some of the things that are in > > > quantum.common _might_ want to live in openstack.common. > > > > > > On 01/03/2012 12:55 PM, Brad Hall wrote: > > > > I still vote for #2. I don't think there will be a ton of > > > duplication so we're probably ok in that regard. Also, it > sounds > > > like following suit will make life easier for the ci folks. I > > think > > > #3 will be painful (both in initial work and maintenance). > > > > > > > > If we do #2 then we could put quantum-client in > > tools/pip-requires > > > and there wouldn't be much of any additional burden on the > > developer > > > unless they were also going to work on the client -- in which > case > > > pull a separate repo. > > > > > > > > Thanks, > > > > Brad > > > > > > > > On Jan 3, 2012, at 12:47 AM, Dan Wendlandt wrote: > > > > > > > >> Hi folks, > > > >> > > > >> tl;dr we need to decide on splitting client repo out of > main > > > repo, per monty's comments. Because this is what all other > > > openstack projects are doing, bias is to move in that > > direction, but > > > benefits are not clear. If we do split, my suggestion is to > > create > > > two repos (client/server), not three repos > (client/server/common) > > > >> > > > >> longer version: > > > >> > > > >> At the 11/29 netstack meeting, one of the topics discussed > was > > > how to handle monty's suggestion that we split out the client > into > > > another repo. > > > >> > > > >> From the 11/29 meeting, here's some background: > > > >> > > > >> 22:12:59 <mtaylor> danwent: > > > >> the problem is that we've got projects that need to depend > on > > > this stuff (horizon being a good example) > > > >> > > > >> 22:13:17 <mtaylor> > > > >> so we need to be able to reference the actual thing that's > > > needed, wihtout pulling in the entire server > > > >> > > > >> 22:13:31 <danwent> mtaylor: > > > >> definitely understand that.. we want to be able to package > > > things separately, definitely > > > >> > > > >> 22:13:40 <cdub> > > > >> sounds more like packagin then repo mgmt > > > >> > > > >> 22:13:51 <danwent> > > > >> I think the question is why packaging needs to be tied to > repo > > > so tightly. > > > >> > > > >> 22:13:58 <cdub> > > > >> e.g. simple to build multpile packages from a single repo > > > >> > > > >> 22:13:59 <danwent> > > > >> i definitely understand that you have a lot of experience > > here… :) > > > >> > > > >> 22:14:17 <danwent> cdub: > > > >> yes, in fact that is what we do already. > > > >> > > > >> 22:14:23 <cdub> > > > >> right > > > >> > > > >> 22:14:25 <mtaylor> > > > >> it's for a few reasons > > > >> > > > >> 22:14:42 <mtaylor> > > > >> pip requires git lines can't reference sub directories > > > >> > > > >> 22:15:01 <mtaylor> > > > >> so you wind up with venv installs that require actual > releases > > > to pypi to be consumable > > > >> > > > >> 22:15:36 <mtaylor> > > > >> additionally, once you start having submodules in tree, > > standard > > > tooling winds up being difficult to apply - knowing to > > interact with > > > a setup.py in trunk is straight forward > > > >> > > > >> 22:15:50 <mtaylor> > > > >> knowing that there are 4 setup.py files spread throughout > the > > > tree is less obviouos > > > >> > > > >> 22:16:05 <bhall> > > > >> as of yesterday all setup.py's are in the root :) > > > >> > > > >> 22:16:24 <danwent> > > > >> ok, so the trade-off here is if we want the pip files of > other > > > projects to automatically pull "trunk", we need separate repos? > > > >> > > > >> 22:16:41 <mtaylor> > > > >> yes > > > >> > > > >> 22:17:21 <mtaylor> > > > >> also, it's the model we're using for the other projects, > > so in > > > keeping with a consistent project-wide approach, I'd kind of > > want to > > > hear a really good reason that it can't work > > > >> > > > >> 22:17:28 <bhall> > > > >> how many repos are we talking about? client, common, > server, > > > plugins? > > > >> > > > >> 22:18:16 <danwent> > > > >> I'm also worried about trying to keep quantum simple > > enough that > > > its easy for new people to join and hack on the project > > > >> > > > >> 22:18:25 <mtaylor> > > > >> I don't think, as of yet, that we need separate ones for > > plugins > > > >> > > > >> 22:18:55 <mtaylor> > > > >> the most important one in my brain in client - as we have a > > > python-*client project either in existence or coming in to > > existence > > > for every openstack project > > > >> > > > >> 22:19:08 <bhall> > > > >> could we go with two repos? client, common/server/plugins > > > >> > > > >> 22:19:11 <mtaylor> > > > >> the common one is just because that's how you've organized > > your > > > code - and i believe client depends on it, yeah/ > > > >> > > > >> 22:19:18 <mtaylor> > > > >> does client depend on common? > > > >> > > > >> 22:19:25 <danwent> > > > >> I believe that was the original goal > > > >> > > > >> 22:19:50 <danwent> > > > >> but perhaps we should revisit that… I'm not sure how much > is > > > currently shared. > > > >> > > > >> 22:19:58 <bhall> > > > >> I think it does currently > > > >> > > > >> 22:20:06 <bhall> > > > >> yeah, I don't think there is a lot of code that it depends > on > > > >> > > > >> 22:20:17 <danwent> mtaylor: > > > >> so the main goal here is pip install, which are mainly > > targetted > > > at developers right? > > > >> > > > >> 22:21:05 <danwent> > > > >> (I assume non-developers will get everything via packages, > > which > > > should have dependencies described in terms of packages) > > > >> > > > >> 22:21:43 <mtaylor> danwent: > > > >> yes > > > >> > > > >> 22:22:12 <danwent> > > > >> ok, thanks for the explanations… I think I understand this > > > better now, but still need to noodle on whether there are ways > we > > > can both be happy :) > > > >> > > > >> 22:22:18 <mtaylor> danwent: > > > >> there is no thought that people will be doing production > > > releases from pip :) > > > >> > > > >> 22:22:23 <mtaylor> danwent: awesome > > > >> > > > >> An here's some more follow-up from the 12/6 meeting: > > > >> > > > >> 22:18:55 <danwent> > > > >> first, as discussed last week with mtaylor, we need to > figure > > > out our strategy for whether we are going to split quantum into > > > multiple repos. > > > >> > > > >> 22:19:49 <salv> > > > >> before deciding on a strategy let's decide whether we want > to > > > split the repos or not! :) > > > >> > > > >> 22:20:02 <mtaylor> danwent: > > > >> re that: we've got openstack.common going now - so perhaps > > > quantum.common can live there? > > > >> > > > >> 22:20:03 <danwent> > > > >> we can have the detailed discussion on the ML, but the > > > high-level options seem to be (1) keep as is, all one repo (2) > > split > > > into two repos, client and server, but potentially duplicate > the > > > little bits of shared code and (3) split into three repos, > client, > > > server, and common > > > >> > > > >> 22:20:09 <salv> > > > >> Splitting in which way exactly? client and server > components? > > > >> > > > >> 22:20:30 <danwent> mtaylor: > > > >> I was thinking something very similar myself, if we decide > > to go > > > with #2 > > > >> > > > >> 22:20:57 <danwent> salv: > > > >> up for discussion > > > >> > > > >> 22:21:14 <danwent> #TODO: #danwent, start ML thread for > > splitting > > > repos, include #mtaylor as well > > > >> 22:21:17 <bhall> > > > >> my vote is for #2 (client and "the rest") > > > >> > > > >> 22:21:19 <mtaylor> #2 with some elements in openstack.common > > > would be the thing that matches the other projects more closely > > > >> 22:21:55 <danwent> if anyone else wants to comment now, go > > ahead, > > > otherwise i'll forward this content to start a thead on the ML > > > >> > > > >> It seems like we have three primary options (feel free to > > suggest > > > others). > > > >> > > > >> #1: do nothing, keep every in single repo > > > >> #2: split into two repos: client + server, but potentially > > > duplicate some of the shared code currently in "common" > > > >> #3: split into three repos, client, server and common.. > > > >> > > > >> A couple thoughts: > > > >> > > > >> I'm still not totally clear on the value of splitting the > > repos. > > > Multiple repos are not required for anyone installing > > openstack via > > > packages, as we can easily generate separate client/server > > packages > > > when using a single repo (in fact, we already do). My > > understanding > > > is that splitting the repos aims to help developers in the case > > > where a project like horizon specifies a dependency on quantum > by > > > adding a direct reference to the quantum git repo in > pip-requires. > > > So one thing I'd like to better understand is the cost of > > > specifying a quantum git repo that includes client + server, as > > > opposed to one that just includes the client. If its just a > > matter > > > of pulling down some more bytes, that doesn't seem that > > painful (the > > > quantum source is tiny anyway). Is there some greater benefit > to > > > splitting repos that I am missing? > > > >> > > > >> I think the cost of splitting repos is non-trivial, as it > makes > > > it harder for developers to get things running from source > tree, > > > (again, I'm assuming this is mainly a developer issue, as > users > > > should mainly install from packages). It may also mean we > need to > > > duplicate some common code, though the need for that may go > > down if > > > we can get both client + server code bases to have common code > in > > > openstack-common. > > > >> > > > >> Another important consideration is that it seems that all > > of the > > > other openstack projects are moving to a model with the client > > in a > > > separate repo from the service itself. There would definitely > a > > > cost both in terms of developer confusion and pain to the > > > infrastructure team to Quantum being an oddball in that > > department. > > > This alone may be reason enough to follow suit, but I'd love > if > > > there was a clear cut reason other than "everyone else is > > doing it". > > > >> > > > >> If we do decide to split repos, my bias would be for #2, > which > > > avoid the even more significant complexity of having three > repos. > > > While some code in common may need to be duplicated, it may > > be the > > > case that such code is duplicated because it really belongs in > > > something like openstack-common, which could be a dependency of > > > both. A lof of the code in quantum.common seems to be of that > > > variety (I suspect much of this code is actually only used by > the > > > server at this point). The two chunks of common code used by > the > > > main client library are: > > > >> > > > >> from quantum.common import exceptions > > > >> from quantum.common.serializer import Serializer > > > >> > > > >> Serializer seems to be pretty much all stock code that > > should be > > > in openstack-common. > > > >> > > > >> The quantum client only uses a few of the Exceptions > classes in > > > common/exeptions.py. A few of these may need to be duplicated > in > > > both the client/server code. Though if we make the changes > > > suggested by Aaron to use only standard HTTP error codes, the > > > applicability of this code to the client decreases to the > point of > > > almost no overlap, I suspect. > > > >> > > > >> Please chime in with your thoughts. Right now I'm not sure > who > > > would work on this, so if you think you have some cycles, > please > > > speak up. > > > >> > > > >> Dan > > > >> > > > >> > > > >> -- > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > >> Dan Wendlandt > > > >> Nicira Networks: www.nicira.com <http://www.nicira.com> > > <http://www.nicira.com> > > > >> twitter: danwendlandt > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > >> > > > >> -- > > > >> Mailing list: https://launchpad.net/~netstack > > > >> Post to : netstack@lists.launchpad.net > > <mailto:netstack@lists.launchpad.net> > > > <mailto:netstack@lists.launchpad.net > > <mailto:netstack@lists.launchpad.net>> > > > >> Unsubscribe : https://launchpad.net/~netstack > > > >> More help : https://help.launchpad.net/ListHelp > > > > > > > > > > > > > > > > > > > > > > > -- > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > Dan Wendlandt > > > Nicira Networks: www.nicira.com <http://www.nicira.com> > > <http://www.nicira.com> > > > twitter: danwendlandt > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > > > > > > > > > -- > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Dan Wendlandt > > Nicira Networks: www.nicira.com <http://www.nicira.com> > > twitter: danwendlandt > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks: www.nicira.com twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
